Repository navigation
fix(fabric): check agent interfaces against their descriptor schema - #14038
Merged
Merged
Conversation
…26.9.34 and Hermes v0.22.5 Step 1 of #14011. Pin Fabric 78e79bfc, the reference implementation of the P0 interface adapters (NVIDIA/NeMo-Fabric#541, #540, #546), fetched through the upstream repository. It sits on Fabric main 624e6876 (0.7.0). This commit enables no interfaces. - Pins: FABRIC_REVISION and FABRIC_SHA256, nemo-fabric-core 0.7.0 in three crates and Cargo.lock, and pinned documentation links. - OpenClaw: the adapter supports only 2026.9.34, so OPENCLAW_IMAGE moves to that release's index digest. - Hermes: v0.22.5 (bf0203a7), the release Fabric tests, and its image digest. Its lock overrides browser-harness's Pillow 12.3.0 pin with 13.0.0, which re-resolving the export rejects. The image now installs the exported closure without re-resolving, and Hermes's own pm.pip_check accepts only the uv pip check mismatches those overrides cause. aiohttp, web_dist and tui_dist still install. - Wheels: nemo-fabric-runtime and the adapter contract now depend on nemo-fabric-evidence, which the image builds from the same source. - Catalog: regenerated. OpenClaw and Hermes gain settings_schema.properties.interfaces; Claude, Codex and Pi gain insecure_http_model_hosts, so NemoClaw now lists plain-HTTP endpoint hosts for them as it does for OpenCode; OpenClaw gains the model setting supports_usage_in_streaming, which onboarding asks for. - fabric_agent: Fabric's new provenance section becomes an input like every other section; the provider schema snapshot gains its block. - Tests: contract fixtures regenerated; onboarding tests omit OpenClaw's interfaces setting and use a startup timeout within the new maximum of 75 seconds. - HCL scenarios: the fixture image's digest covers the bundled catalog, and the OpenShift scenario's inference addresses derive from it, so the case documents and openshift-external-two-providers.tftest.hcl follow the new catalog. Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Live tests for #14011 found that OpenShell's sandbox refuses signals to a process group (EPERM), so stopping Hermes's interface children and Fabric's ACP agents, such as Deep Agents Code, failed. Fabric 6cad3504 fixes both: it merges the Hermes fix (7981efca) into the Deep Agents Code branch and fixes the ACP runner. Move every Fabric pin from 78e79bfc to 6cad3504: FABRIC_REVISION and FABRIC_SHA256, nemo-fabric-core in three crates and Cargo.lock, the bundled catalog, and pinned documentation links. No adapter descriptor changed, so the regenerated catalog differs only in fabric_revision and source_sha256. The catalog change moves the fixture image's digest again, so the HCL case documents and openshift-external-two-providers.tftest.hcl follow it. Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Part of #14011. Planning now compares harness.settings.interfaces with the selected adapter's descriptor in the image catalog, where Fabric's P0 adapters list each interface and its fields under settings_schema.properties.interfaces. An unsupported interface name or field fails the plan with a harness_interfaces check that names the setting, such as harness.settings.interfaces.terminal; an adapter that declares no interfaces supports none. Values, required fields and combinations, such as a missing port or port with port_range, stay with Fabric, which validates them when it plans the agent in the sandbox. A name that isn't safe to print is reported as harness.settings.interfaces. Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Part of #14011. After a successful apply, the result lists each enabled harness.settings.interfaces entry with its sandbox, loopback port, the token's path in the sandbox, and how to sign in. The token path follows Fabric's P0 convention, <artifact root>/.fabric/<adapter>/runtimes/<runtime-id>/interfaces/<name>.token, with /sandbox/artifacts from the agent's Fabric configuration and the runtime ID from the Fabric provider's state. Fabric's descriptors don't yet say which interfaces have a token or how to sign in (NVIDIA/NeMo-Fabric#565), so nemoclaw-fabric lists the P0 adapters': OpenClaw's dashboard takes its token in the #token= URL, Hermes's API server takes a Bearer token, and Hermes's dashboard has no sign-in. An interface NemoClaw doesn't know reports its port and points to the adapter's documentation, so a terminal interface can be added later. Apply text prints the gateway and workspace selection the interface guide describes, then for each interface the openshell forward service command, the openshell sandbox exec command that reads its token, and the sign-in step. JSON results carry the same facts as interfaces. No token is read or printed; a deployment without interfaces reads no extra state. Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Part of #14011. A live-docker test deploys OpenClaw's dashboard and Hermes's API server and dashboard on one managed Docker gateway and reaches them as a person would, with the forward and token commands apply reports, run through the pinned OpenShell CLI: - planning rejects interfaces.terminal for OpenClaw before creating anything; - apply reports each interface's port, token path and sign-in; - OpenClaw's Control UI loads, and its configuration endpoint accepts the retrieved token and refuses a missing or wrong one; - Hermes's API server accepts its token, refuses a missing or wrong one, and answers a chat completion; - Hermes's dashboard page loads, its API accepts the session token the page carries, and one browser chat message gets a reply; - every chat completion reached the model stand-in with the configured key; - after the Hermes dashboard process is killed, plan reports no change and the API server still answers; - no token appears in NemoClaw's output, state directory or YAML, and destroy succeeds. A model stand-in in the test process answers with a value unique to the run. The document grants the Hermes sandbox read-write /dev/pts through an explicit policy, because browser chat runs Hermes's terminal UI in a pseudo-terminal that the images' default policy denies. The runner now builds the Hermes image with OpenClaw's (and the shared live-images archive and per-commit pulls include it), downloads the OpenShell 0.1.6 CLI by checksum into .tools, and writes the test's document with a fresh UID, port, subnet and model port. Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Part of #14011. interfaces.md gets a section on enabling OpenClaw's dashboard and Hermes's API server and dashboard, forwarding them with openshell forward service, reading each token from its path under /sandbox/artifacts/.fabric, and signing in. It says that the agent's own tools can read the tokens and use the interfaces, that Hermes's dashboard has no sign-in and its browser chat runs without approvals, why forward service beats an exposed service URL, and what a failed Hermes interface process does to the agent. Browser chat needs read-write /dev/pts (/dev/ptmx alone isn't enough). As of this commit the docs grant it through an explicit policy and list it in limits.md (#13969); the later feat(images) commit makes the Hermes image's default policy allow it and replaces that guidance. agents.md gets a short Native Interfaces section linking there. limits.md drops the OpenClaw dashboard and Hermes interface rows and lists what remains: Deep Agents Code's terminal (#14011), the P0 conventions until handles report interfaces (NVIDIA/NeMo-Fabric#565), and token isolation, an interface check and exposed URLs (NVIDIA/NeMo-Fabric#566). The Fabric management design's upstream table drops the OpenClaw dashboard and lists those two Fabric issues. Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Part of #14011. The hosted Hermes fixture's v0 export declares Hermes's API server under harness.interfaces. Its v1 intent left it out, because Fabric's Hermes adapter couldn't serve it (#13321). It can now, so the v1 intent sets harness.settings.interfaces.api.port to the export's 8643, and the parity projection moves the export's interfaces into the adapter's settings instead of dropping them. NOTICE.md says so. Signed-off-by: Carlos Villela <cvillela@nvidia.com>
…nals Part of #14011. Hermes's browser chat runs its terminal UI in a pseudo-terminal, which the agent images' shared default policy doesn't allow, so the chat failed with "out of pty devices" unless the sandbox took an explicit policy. A sandbox's default policy is its image's: catalog.py records the image's runtime manifest, including its policy, in the io.nemoclaw.fabric.catalog label, and the Fabric provider reads it from there. Each harness stage already declares its adapters in /opt/nemoclaw/adapters.json; an entry can now also list read_write paths, which catalog.py appends to that image's runtime.policy.filesystem_policy.read_write. The Hermes stage declares /dev/pts and /dev/ptmx. Images that declare none get exactly the policy they had, so OpenClaw's and the others' are unchanged. Tests: - catalog contract: a declared path joins the image's policy once, an image without one keeps the manifest's policy, and a relative or non-list declaration fails the catalog; - live: the interface test's sandboxes now both take their image's default policy; the Hermes sandbox can open a pseudo-terminal and browser chat answers, and the OpenClaw sandbox can't open one. It failed on the Hermes check with the image built before this change. interfaces.md says the Hermes image's default allows browser chat and what an explicit policy needs instead; limits.md drops the browser chat row; the design and live-test docs describe the declaration and check. Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Fixes #14028. Planning accepted an interface without its required port, such as Hermes's api: {}, because the harness_interfaces check only compared interface and field names with the descriptor. Fabric rejected it at apply, inside the sandbox, after NemoClaw had started creating resources. The check now also validates each interface object against that interface's own JSON Schema in the image catalog's descriptor, settings_schema.properties.interfaces.properties.<name>, offline and with the settings schema's dialect and definitions. A missing required field, a wrong type or an out-of-range value fails the plan under harness_interfaces, naming the field, for example harness.settings.interfaces.api.port: is required by the image's adapter. The reason never repeats the rejected value. Since the rules come from each descriptor, OpenClaw's and Hermes's ports, Deep Agents Code's terminal and future adapters need no code here. The check stays limited to harness.settings.interfaces. NemoClaw checks interfaces early because its apply output and access instructions depend on them, and their schemas are self-contained. Other settings, and conditions that span settings, such as OpenClaw's dependentSchemas rule against port with port_range, are still Fabric's to validate when it plans the agent. Running Fabric's whole settings_schema here would repeat its planning (see the upstream boundaries in docs/design/scope.md). A descriptor schema that doesn't compile offline is left to Fabric too. Tests, failing before the change: a missing port, a string port, ports out of range and a wrong-typed field are rejected with the field's name, for the fixture adapter and the bundled OpenClaw and Hermes descriptors, and no reason echoes the value. Guards: valid settings and terminal: {} pass, and port_range with a dashboard port is left to Fabric. interfaces.md and agents.md describe the check. Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Collaborator
Author
Authoring session — opening
|
#14037's test step took 48.1 s against its 45 s budget on linux_amd64. ci_lifecycle went from 4.9 s to 9.3 s with the same 20 tests: since this branch's live-docker step installs the pinned OpenShell CLI, the three fixture tests that reach that step downloaded the 22 MB CLI from GitHub on every run. The fixture now puts a stand-in CLI where the pin installs it, .tools/openshell-<checksum>/openshell, as an earlier run would have, so the step finds it and skips the download. It also points the fixture's pinned URL at a non-HTTPS address, so any fixture that still tries to download fails at once instead of reaching the network; with only that change, the three tests failed with "artifact origin must use HTTPS". Locally, ci_lifecycle's 20 tests now take 1.2 s in total, and the test step 14.0 s wall. The budget is unchanged. Signed-off-by: Carlos Villela <cvillela@nvidia.com>
…ngs-schema Signed-off-by: Carlos Villela <cvillela@nvidia.com>
…schema # Conflicts: # crates/nemoclaw-fabric/src/capabilities.rs # crates/nemoclaw-sdk/tests/fabric_compatibility.rs # docs/agents.md # docs/interfaces.md
cv
enabled auto-merge (squash)
October 31, 2026 03:49
Contributor
|
v1 documentation preview: https://nvidia-preview-nemoclaw-v1-pr-14038.docs.buildwithfern.com/nemoclaw |
This was referenced Oct 31, 2026
cv
added a commit
that referenced
this pull request
Nov 1, 2026
## Summary Cuts the time of `cargo ci`'s unit-test step so it stays well under its 45 s budget on the `linux_amd64` runner, without raising any budget. ## Related Issue The test step failed its budget on several recent PRs (#14037, #14038, #14044) at 45.3 to 49.1 s, then passed on reruns at 33.9 to 44.0 s. ## Changes Twelve commits, each keeping results the same: - **Shared work in the product code:** the bundled catalog is read without copying; compiled JSON Schema validators, partial-document assessments, recent journey resolutions, regex patterns and the input schema are kept and reused instead of rebuilt; schemas are converted with `Schema::to_value()` instead of a serialize round trip. - **Two tests:** `release_sized_sboms_are_admissible` writes its 70 MB comment directly instead of through unoptimized `serde_json` (it still fails when the limit is lowered); `progress_arrives_before_exit_and_cancellation_still_works` runs `exec sleep` so an interrupt can't leave `sleep` running for the 5 s grace. ## Failure, Decision and Validation **Failure.** CI's `linux_amd64` runner has 4 vCPUs running 8 test threads, so the step is CPU-bound, and the suite sat at the 45 s budget: runner variance alone failed PRs. `AGENTS.md` says to make slow tests faster rather than raise budgets. **Decision.** Remove repeated work that tests exercise heavily, mostly in onboarding and journey resolution, rather than weaken tests. The riskiest change is the resolution cache in `JourneyState::resolve`; it relies on `Capabilities` never changing after construction, which its documentation states. It alone saves about 12% of journey CPU, so it can be dropped if reviewers prefer. **Validation.** Measured with the `ci` nextest profile pinned to 4 CPUs (`taskset -c 0-3`) against a clean `origin/v1`: 32.2 s to 22.2 s wall and 251 to 171 s summed. The onboarding TUI tests went from 57.1 to 17.4 s summed. Unrestricted on 20 cores: 13.6 to 11.4 s wall. `cargo ci` passes (1805 tests, 162 lifecycle tests); every commit was checked with clippy, `cargo fmt --check` and the full suite. Live suites weren't run, since nothing changes images or Kubernetes behavior. ## Type of Change - [x] Performance ## Verification - [x] `cargo ci` ## AI Disclosure - [x] AI agent used: Pi with Claude Opus 5.5 Signed-off-by: Carlos Villela <cvillela@nvidia.com> --------- Signed-off-by: Carlos Villela <cvillela@nvidia.com>
wscurran
added
area: cli
wscurran
added
integration: openclaw
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Failure
nemoclaw planaccepts an agent interface that its adapter's schema rejects, such as Hermes'sinterfaces.api: {}without the requiredport. The error only appears when Fabric validates the settings at apply, after NemoClaw has started creating resources (#14028).Decision
The
harness_interfacescheck from the previous PR now also validates each interface object against its JSON Schema in the image catalog's descriptor: missing required fields, wrong types, and out-of-range values fail planning, naming the field. Using the descriptor's own schema covers OpenClaw, Hermes, Deep Agents Code'sterminaland future adapters without hand-coded rules. Reasons name the field but don't echo the value.The check stays limited to
harness.settings.interfaces, which NemoClaw's access instructions depend on. Other settings, and rules that span settings, such as OpenClaw'sport_rangewith a dashboard port, stay with Fabric, per the upstream boundaries indocs/design/scope.md.Stacked on #14037. Fixes #14028.
Validation
port(also withenabled: true), a stringport, ports 0 and 65536, a non-booleanenabledand a non-object interface on a fixture adapter, and with the bundled descriptors Hermesapi: {}, a Hermesdashboardwithout a port, OpenClawdashboard: {}, OpenClaw port 65426, and a string Hermes port. Valid settings andterminal: {}still pass.cargo ci: 1,790 unit tests and 161 lifecycle tests, plus fmt, clippy, build, schema and bundle, passed.cargo ci live-docker: all 10 tests passed in 315.9 s.live-kind.