From 138d66ffef92bd73bbbd8bf9f1680d2a25a8c7da Mon Sep 17 00:00:00 2001 From: Imran Siddique <45405841+imran-siddique@users.noreply.github.com> Date: Wed, 7 Oct 2026 14:16:29 -0700 Subject: [PATCH 1/3] docs: plain-language pass on the docs site and README Every published prose page now opens with what it is for and who needs it, explains terms at first use, and moves deep detail into Technical detail blocks. Spec and RFC files get a plain lead-in only. ROADMAP conformance suite 0.5.1 -> 0.6.2 (conformance/pyproject.toml, release conformance-v0.6.2); quickstart notes PyPI agentrust-trace 0.11.0 no longer requires transparency. Co-Authored-By: Claude Opus 5.5 --- CONTRIBUTING.md | 6 +- LIMITATIONS.md | 28 ++-- MAINTAINERS.md | 2 + README.md | 10 +- ROADMAP.md | 14 +- SPONSORS.md | 4 +- docs/build-provenance-depth.md | 6 +- docs/conformance-method.md | 4 +- docs/crosswalks/acta-decision-receipts.md | 2 + docs/crosswalks/chap-review-decisions.md | 2 + docs/crosswalks/mitre-atlas.md | 2 + .../nkemba-institutional-reconstruction.md | 2 + docs/crosswalks/owasp-agentic-top-10.md | 2 + docs/glossary.md | 32 ++--- docs/integration/agt.md | 10 +- docs/integration/cmcp.md | 10 +- docs/integration/openshell.md | 37 +++-- docs/integration/pic-trace-bridge-v1.md | 7 + docs/integration/sandbox-runtime.md | 49 +++++-- docs/platforms/amd-sev-snp.md | 15 +- docs/platforms/bernstein.md | 46 +++--- docs/platforms/index.md | 16 ++- docs/platforms/intel-tdx.md | 8 +- docs/platforms/nvidia-h100.md | 8 +- docs/quickstart.md | 18 +-- docs/references-registry.md | 4 + docs/repository-consolidation.md | 2 + docs/rfcs/a2a-delegation-profile.md | 43 +++--- docs/rfcs/composable-zk-assurance.md | 2 + docs/rfcs/composite-component-appraisal.md | 2 + docs/rfcs/runtime-evidence-profile.md | 2 + docs/rfcs/tool-catalog-observed-digest.md | 2 + docs/rfcs/verifier-issued-trace-profile.md | 2 + docs/schema.md | 45 +++--- docs/trust-levels.md | 27 ++-- docs/tutorials/agt-adapter.md | 21 ++- docs/tutorials/anchoring-to-the-registry.md | 27 ++-- .../hardware-attestation-platforms.md | 16 ++- docs/tutorials/integrating-with-cmcp.md | 10 +- .../signing-your-first-trust-record.md | 19 ++- docs/tutorials/verifying-a-trust-record.md | 19 ++- docs/tutorials/verifying-the-audit-chain.md | 16 ++- docs/verification-outcome-statements.md | 6 +- docs/verification.md | 133 ++++++++++-------- docs/verifier-token-experimental.md | 2 + index.md | 26 ++-- spec/content-marking-v1.md | 2 + spec/mcp-profile-v0.3-draft.md | 2 + spec/registry-anchor-v1.md | 2 + spec/server-provenance-v1.md | 2 + spec/server-provenance-v2.md | 2 + spec/trace-v0.2.md | 2 + 52 files changed, 509 insertions(+), 269 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index bafad21a..ab301020 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,5 +1,7 @@ # Contributing to TRACE +This page explains how to propose a change to TRACE and what reviewers will ask for. Small fixes such as typos and broken links can go straight to a pull request; changes to the rules every implementation must follow need more discussion and a named organization behind them, as described below. + TRACE is an open specification. Contributions are welcome in four areas: the specification text, the JSON Schema, the examples, and the conformance test suite (in [trace-spec/conformance](https://github.com/agentrust-io/trace-spec/tree/main/conformance)). ## Running the reference-library checks @@ -31,6 +33,8 @@ The check you just fixed is not that something. ## Before a pull request is reviewed +In short: the automated tests must pass, and a bug fix must come with a test that would have caught the bug. + A pull request gets a full review once every `test` job of the `CI` workflow, which runs the checks above, succeeds on its head commit. Other checks on the pull request, such as the maintainer-approval gate, do not hold a review, @@ -86,7 +90,7 @@ This adds `Signed-off-by: Your Name `. PRs without DCO sign-off ### Spec changes (normative text) -Changes to `spec/trace-v0.2.md` that affect what implementations must do. Normative text is any statement using an RFC 2119 keyword in uppercase: what a conformant implementation MUST, SHOULD or MAY do. A normative change binds every implementation of TRACE, including implementations whose authors are not in the discussion. +These are changes to the rules every TRACE implementation has to follow, so they carry the most weight and the most process. Changes to `spec/trace-v0.2.md` that affect what implementations must do. Normative text is any statement using an RFC 2119 keyword in uppercase: what a conformant implementation MUST, SHOULD or MAY do. A normative change binds every implementation of TRACE, including implementations whose authors are not in the discussion. **Anyone may propose a Normative Change, however only Normative Contributions with an organizational sponsor willing to implement and maintain that element in the specification will be accepted.** diff --git a/LIMITATIONS.md b/LIMITATIONS.md index bef48af1..98b3c7dc 100644 --- a/LIMITATIONS.md +++ b/LIMITATIONS.md @@ -1,35 +1,37 @@ # Known Limitations -This document describes what TRACE does not do, and where layered defenses are needed. Honest scope boundaries prevent misplaced trust. +This page lists what TRACE does not do, so you know where you still need other safeguards. Read it before you rely on a TRACE record for anything that matters. A TRACE record (also called a Trust Record or claim) is a signed receipt describing one AI agent run; most of what follows is about the gap between "this receipt is genuine" and "what it describes really happened". ## What a TRACE claim does not prevent **Operator-forged software-only records** -A TRACE claim at Level 0 (software-only signing) is signed by a key held in software. A privileged operator with root access can produce a valid-looking Level 0 record for a run that never happened, or that violated policy. Level 0 is suitable for development and audit-trail tooling only, not for third-party verification. +Level 0 is the entry level, where a record is signed with an ordinary software key and no hardware evidence. Whoever runs the machine can read that key. A privileged operator with root access can produce a valid-looking Level 0 record for a run that never happened, or that violated policy. Level 0 is suitable for development and audit-trail tooling only, not for third-party verification. **Replay of a valid past record** -A TRACE claim proves a specific run happened; it does not prevent a verifier from being shown a valid record from an earlier run. Verifiers that rely on recency must bound `iat` in both directions (maximum age and allowed future clock skew), check `exp` when present, require nonce binding to a challenge, or anchor records to a public transparency log and check for freshness. +A genuine old record stays genuine, so someone can show you last week's record and pass it off as today's. A TRACE claim proves a specific run happened; it does not prevent a verifier from being shown a valid record from an earlier run. Verifiers that rely on recency must bound `iat` in both directions (maximum age and allowed future clock skew), check `exp` when present, require nonce binding to a challenge, or anchor records to a public transparency log and check for freshness. **Policy correctness** -The `policy.bundle_hash` field attests that a specific policy was in force at runtime. It does not attest that the policy achieves the intended security outcome. Policy review is a separate control. +A record can show which rules the agent ran under, but not whether those rules were any good. The `policy.bundle_hash` field attests that a specific policy was in force at runtime. It does not attest that the policy achieves the intended security outcome. Policy review is a separate control. **What happened inside the model** -The call transcript records tool invocations, arguments, and responses that are observable at the gateway boundary. It does not record the model's internal chain-of-thought, intermediate reasoning, or context window contents. Reasoning that influences behavior without producing a tool call is not captured. +TRACE sees what the agent did, not what the model thought. The call transcript records tool invocations, arguments, and responses that are observable at the gateway boundary. It does not record the model's internal chain-of-thought, intermediate reasoning, or context window contents. Reasoning that influences behavior without producing a tool call is not captured. **Cross-boundary data propagation** -The call graph summary uses temporal adjacency to approximate data flow between tool calls. It cannot definitively prove which specific data from one tool response influenced which subsequent call. The `provenance_disclaimer` field in every call graph summary is required for this reason. +TRACE can show that one tool call came after another, but not prove that data from the first actually fed the second. The call graph summary uses temporal adjacency to approximate data flow between tool calls. It cannot definitively prove which specific data from one tool response influenced which subsequent call. The `provenance_disclaimer` field in every call graph summary is required for this reason. **TEE side-channel attacks** -Hardware attestation proves the TRACE signing key and policy engine were measured in silicon before execution. It does not protect against side-channel attacks (cache timing, power analysis) targeting the TEE itself. TEE-level side-channel defense is the responsibility of the TEE platform vendor. +A TEE (trusted execution environment) is a sealed area of a processor that even the machine's operator cannot look into. Hardware attestation, the processor's signed report of what is running inside it, proves the TRACE signing key and policy engine were measured in silicon before execution. It does not protect against side-channel attacks (cache timing, power analysis) targeting the TEE itself. TEE-level side-channel defense is the responsibility of the TEE platform vendor. **Revocation of the signing key after issuance** -If the TRACE signing key is compromised after records are issued, existing records remain cryptographically valid. Key monitoring, rapid revocation, and transparency log integration are the required controls: TRACE provides the anchoring mechanism but cannot detect compromise itself. +If a signing key is stolen later, everything it already signed still checks out. If the TRACE signing key is compromised after records are issued, existing records remain cryptographically valid. Key monitoring, rapid revocation, and transparency log integration are the required controls: TRACE provides the anchoring mechanism but cannot detect compromise itself. **Pure offline verification cannot prove non-revocation** -Signature validity is permanent; trust is not. Nothing inside a record can retract the key that signed it, so a record signed by a since-revoked key verifies offline forever. Spec §3.2.1 accordingly requires verifiers to consult current revocation status at verification time, which is by definition an online step. `verify_record()` takes a `revocation` store, either a container of revoked identifiers or a callable performing a live CRL, status-endpoint, or SCITT lookup. It rejects a listed key, and fails closed when the store cannot answer. Without that store, verification is offline and its result means "this record was validly signed by this key", not "this key is still trusted". +Checking a record with no network connection tells you it was signed by a given key, but not whether that key has since been withdrawn. Signature validity is permanent; trust is not. Nothing inside a record can retract the key that signed it, so a record signed by a since-revoked key verifies offline forever. Spec §3.2.1 accordingly requires verifiers to consult current revocation status at verification time, which is by definition an online step. `verify_record()` takes a `revocation` store, either a container of revoked identifiers or a callable performing a live CRL, status-endpoint, or SCITT lookup. It rejects a listed key, and fails closed when the store cannot answer. Without that store, verification is offline and its result means "this record was validly signed by this key", not "this key is still trusted". ## Platform state is not appraised +An AMD processor's signed report says which software is running and, separately, how the machine itself is configured. The checks described here cover the first part only, so a report from a machine with weaker security settings passes just as cleanly. + ## 1. Scope diff --git a/spec/registry-anchor-v1.md b/spec/registry-anchor-v1.md index 3713143c..d030ef89 100644 --- a/spec/registry-anchor-v1.md +++ b/spec/registry-anchor-v1.md @@ -7,6 +7,8 @@ | Applies to | The `transparency` claim (TRACE v0.2 §3.1) and Level 2 conformance | | License | CC BY 4.0 | +**In plain terms.** Anchoring means publishing a fingerprint of a record in an append-only log, so anyone can later prove the record existed and was not quietly swapped. This document is for people building or checking that log; it is a normative companion to TRACE v0.2 and the basis for Level 2 conformance. + This document specifies how TRACE Trust Records are anchored into an append-only registry, and how a third party verifies, **without trusting the registry operator**, that a given record was included in an anchor. It is published here rather than only in the registry implementation for a reason that matters more than tidiness: an inclusion proof nobody outside can check is not transparency. A conforming verifier can be written from this document alone. The reference tooling named in §7 is one implementation, not the definition. diff --git a/spec/server-provenance-v1.md b/spec/server-provenance-v1.md index f04618be..5c7fade0 100644 --- a/spec/server-provenance-v1.md +++ b/spec/server-provenance-v1.md @@ -7,6 +7,8 @@ | Anchoring | [Registry Anchor Format v1](registry-anchor-v1.md) | | License | CC BY 4.0 | +**In plain terms.** MCP servers are the tool providers an AI agent calls. This format lets someone sign a statement about one server (what it is and which tools it offers) so others can check who said it. It is for people who publish or vet MCP servers, and it is a draft. + A signed statement **about** an MCP server: what it is, what tools it exposes, who is saying so, and how much that is worth. diff --git a/spec/server-provenance-v2.md b/spec/server-provenance-v2.md index 8c1a4c01..43f0f5bc 100644 --- a/spec/server-provenance-v2.md +++ b/spec/server-provenance-v2.md @@ -6,6 +6,8 @@ | Status | Draft proposal, companion to TRACE v0.2 | | License | CC BY 4.0 | +**In plain terms.** Version 2 of the MCP server provenance format adds four of a tool's self-declared behaviours (for example, whether it only reads data) to what gets signed, so a later change to them is detectable. It is for people who publish or check MCP server provenance records, and it is a draft proposal. + ## 1. Scope and inherited requirements diff --git a/spec/trace-v0.2.md b/spec/trace-v0.2.md index 25682c91..3f7d57ca 100644 --- a/spec/trace-v0.2.md +++ b/spec/trace-v0.2.md @@ -11,6 +11,8 @@ > **Note:** This is a pre-ratification draft. Fields, wire formats, and conformance requirements are subject to change before v1.0. Send feedback to: open an issue on this repository. +**In plain terms.** This is the rulebook for TRACE: it defines the Trust Record (a signed receipt describing one AI agent run), how records are anchored in a public log, and the exact checks a verifier runs. You need it if you are writing software that produces or checks records; to try TRACE first, start with the [quickstart](../docs/quickstart.md). It is a draft (v0.2, Developer Preview), so details can still change before v1.0. + ## Authority and conformance claims From c3505aa4377fcd69cda475702ffb3b65e4e2f85d Mon Sep 17 00:00:00 2001 From: Imran Siddique <45405841+imran-siddique@users.noreply.github.com> Date: Wed, 7 Oct 2026 14:17:25 -0700 Subject: [PATCH 2/3] docs(quickstart): match the hardware heading to the trust levels Hardware evidence is Level 1 and transparency is Level 2 (trust-levels.md). Co-Authored-By: Claude Opus 5.5 --- docs/quickstart.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/quickstart.md b/docs/quickstart.md index 526d8287..9e51e934 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -126,7 +126,7 @@ Expected: `PASS: saved record verified against the retained public key`. By defa The verification call above does not check hardware attestation or whether the record was published to a registry. Production verification also needs an issuer trust policy and any required revocation, nonce, measurement, and transparency checks. -## Add hardware attestation (Level 2) +## Add hardware evidence and transparency (Levels 1 and 2) To show that a record came from protected hardware, follow the [cMCP integration guide](integration/cmcp.md), [trust levels](trust-levels.md), and [platform documentation](platforms/index.md). Hardware evidence and transparency receipts require their own generation and verification steps. Installing a runtime or declaring a hardware platform does not automatically establish a conformance level. From 47eaf9be47ac7f7d8f922cce460043eb6bc73f73 Mon Sep 17 00:00:00 2001 From: Imran Siddique <45405841+imran-siddique@users.noreply.github.com> Date: Wed, 7 Oct 2026 14:22:15 -0700 Subject: [PATCH 3/3] docs(conformance): plain-language pass on the conformance site Ported from the archived trace-tests repository. Adds TR-APR to the per-level module lists (runner._LEVEL_MODULES), the Appraisal row in the README table, and removes the CI guide's claims that the suite reads CMCP_DEV_MODE or supports Python 3.10 (pyproject requires 3.11). Co-Authored-By: Claude Opus 5.5 --- conformance/LIMITATIONS.md | 67 ++++++++------- conformance/README.md | 19 +++-- conformance/docs/error-codes.md | 20 ++--- conformance/docs/levels.md | 85 +++++++++++-------- conformance/docs/modules.md | 4 +- conformance/docs/modules/tr-anc.md | 11 ++- conformance/docs/modules/tr-apr.md | 4 +- conformance/docs/modules/tr-env.md | 6 +- conformance/docs/modules/tr-pol.md | 6 +- conformance/docs/modules/tr-rte.md | 2 + conformance/docs/modules/tr-sca.md | 6 +- conformance/docs/modules/tr-sig.md | 4 +- conformance/docs/modules/tr-txn.md | 4 +- conformance/docs/quickstart.md | 12 ++- conformance/docs/tutorials/ci-integration.md | 16 ++-- .../tutorials/writing-conformance-tests.md | 10 +-- conformance/index.md | 44 +++++----- conformance/measurement/REPORT.md | 10 ++- 18 files changed, 194 insertions(+), 136 deletions(-) diff --git a/conformance/LIMITATIONS.md b/conformance/LIMITATIONS.md index 6ddb5a57..7f8bff4f 100644 --- a/conformance/LIMITATIONS.md +++ b/conformance/LIMITATIONS.md @@ -1,7 +1,9 @@ # Known Limitations What this suite does **not** establish. A conformance report is only useful if the reader knows -what it was never checking, so this is the companion to the report rather than a footnote to it. +what it was never checking, so read this page alongside any report. In short: a pass means the +record is well formed and complete for its level; it does not mean the agent behaved well, that +the hardware claims are true, or that anyone independent checked the result. ## What a pass means @@ -19,6 +21,8 @@ the disagreement is a bug worth reporting here. ## Where the checks stop **`TR-RTE` checks the shape of the attestation fields, not the attestation.** +Attestation is the processor's signed statement of what code it is running. The suite checks that +the record's fields describing it are well formed; it does not check the statement itself. It validates that `runtime.platform` is a recognised value, that `runtime.measurement` is a well formed digest, and that the RIM URI parses. It does not obtain a quote, and it does not verify one against AMD, Intel or a TPM manufacturer root. A record can satisfy `TR-RTE` at Level 1 carrying a @@ -26,6 +30,8 @@ syntactically perfect measurement that no hardware ever produced. Verifying the silicon vendor is the relying party's job, and `cmcp_verify` is where that happens. **`TR-ANC-002` proves inclusion relative to the receipt you hand it.** +In plain terms: it proves the record is in the log the receipt describes, not that the log is a +real public one. It replays the audit path against the Merkle root carried in that receipt. It does not fetch the `transparency` URI, and it does not establish that the root is one a public log actually published. A self-consistent receipt over a tree the submitter built themselves will pass. What @@ -73,31 +79,34 @@ The accounting extension covers only `TR-APR-001`, `TR-POL-003`, and `TR-SCA-002 Complete means that every attempted level has exactly one reconciled row for each of those three obligations. It does not mean every TRACE obligation was accounted for, every obligation was evaluated, or that the report independently proves which code ran or which fields it accessed. -An obligation absent from both the bounded registry and its rows is outside this completeness -claim and cannot be discovered by registry-to-row reconciliation alone. - -The pilot records `TR-SCA-002` at Level 0 as applicable but not attempted: the pinned schema -requires `build_provenance.digest`, while the scheduler first runs `TR-SCA` at Level 1. The row -carries that scheduler reason rather than silently presenting the state without an explanation. - -Each source locator includes a digest of the exact value resolved at its pinned trace-spec -revision. JSON sources use RFC 6901 pointers and RFC 8785-canonical value bytes. The operational -TR-POL-003 rule uses the unique exact text of verification item 5 in the pinned specification; -its schema fragments are listed separately as structural support. A reader who holds those -pinned bytes can resolve and compare the values. The suite does not fetch or authenticate -trace-spec at report time, and a matching value does not prove that a locator is sufficient or -that its checker is correct. - -Findings and accounting share one validated execution snapshot during supported report -construction. The emitted report remains editable and unsigned. Its registry hash identifies -registry content; it does not authenticate the rows, the report, or its producer. - -The optional policy resolver is trusted in-process Python code supplied by the caller. The -accounted path isolates nested public runs and freezes ordinary decision inputs, but it is not a -sandbox against a callback that rewrites interpreter globals, functions, classes, or source files. - -`producer_branch` is a checker-owned same-execution diagnostic. Some branches with the same -accounting meaning return indistinguishable findings, so the emitted report alone cannot -independently reconstruct every branch label. Applicability, evaluation state, prerequisite, -contribution, findings, and report tallies are revalidated at emission; the label is not -presented as independent proof of checker control flow. +Like the rest of the report, this section is unsigned and anyone can edit it. + +??? info "Technical detail: locators, snapshots, the resolver and branch labels" + An obligation absent from both the bounded registry and its rows is outside this completeness + claim and cannot be discovered by registry-to-row reconciliation alone. + + The pilot records `TR-SCA-002` at Level 0 as applicable but not attempted: the pinned schema + requires `build_provenance.digest`, while the scheduler first runs `TR-SCA` at Level 1. The row + carries that scheduler reason rather than silently presenting the state without an explanation. + + Each source locator includes a digest of the exact value resolved at its pinned trace-spec + revision. JSON sources use RFC 6901 pointers and RFC 8785-canonical value bytes. The operational + TR-POL-003 rule uses the unique exact text of verification item 5 in the pinned specification; + its schema fragments are listed separately as structural support. A reader who holds those + pinned bytes can resolve and compare the values. The suite does not fetch or authenticate + trace-spec at report time, and a matching value does not prove that a locator is sufficient or + that its checker is correct. + + Findings and accounting share one validated execution snapshot during supported report + construction. The emitted report remains editable and unsigned. Its registry hash identifies + registry content; it does not authenticate the rows, the report, or its producer. + + The optional policy resolver is trusted in-process Python code supplied by the caller. The + accounted path isolates nested public runs and freezes ordinary decision inputs, but it is not a + sandbox against a callback that rewrites interpreter globals, functions, classes, or source files. + + `producer_branch` is a checker-owned same-execution diagnostic. Some branches with the same + accounting meaning return indistinguishable findings, so the emitted report alone cannot + independently reconstruct every branch label. Applicability, evaluation state, prerequisite, + contribution, findings, and report tallies are revalidated at emission; the label is not + presented as independent proof of checker control flow. diff --git a/conformance/README.md b/conformance/README.md index 579e82c4..f2e2a303 100644 --- a/conformance/README.md +++ b/conformance/README.md @@ -29,9 +29,9 @@ Community updates and contributor highlights: [AgenTrust on LinkedIn](https://ww > Tracks [TRACE Spec v0.2](https://github.com/agentrust-io/trace-spec). -Check a TRACE record, inspect the findings, and produce a reproducible conformance report. The suite checks the record and supplied evidence; a passing report does not establish that an entire implementation meets every specification requirement. +TRACE is an open format for signed receipts that say what an AI agent ran and what it did ([the terms, in plain English](https://agentrust-io.com/#plain-terms)). This suite checks one of those receipts, a TRACE record, against the specification, tells you the highest conformance level it reaches, and writes a report anyone can reproduce. A pass covers that one record and the evidence supplied with it; it does not show that a whole product meets every requirement of the specification. -Eight modules cover envelope, signature, runtime, policy, appraisal, transcript, transparency, and provenance checks. Read the [limitations](LIMITATIONS.md) to interpret their results. +Eight modules, each a group of related checks, look at the envelope (the outer wrapper and its basic fields), signature, runtime (the hardware it says it ran on), policy (the rules the agent ran under), appraisal (a verifier's verdict on the hardware evidence), transcript (the log of tool calls), transparency (proof of entry in a public log) and provenance (how the software was built). Read the [limitations](LIMITATIONS.md) to see what each result does and does not tell you. ## Quick start @@ -43,8 +43,8 @@ trace-tests verify --record path/to/trust-record.jwt --level 1 \ ## A report you can hand to someone else -`verify` answers a question for the person running it. `report` produces an artifact for -somebody who was not there: an auditor, a counterparty, an acquirer. +`verify` prints results for the person running it. `report` writes files for somebody who +was not there: an auditor, a counterparty, an acquirer. ```bash trace-tests report --record trust-record.json --html report.html --json report.json --badge trace.svg @@ -66,7 +66,13 @@ conformance report that looks authoritative and cannot be checked is the same sh thing as a control plane writing its own log. `report.json` is stable under `schema: agentrust-io/trace-tests/report/1` for dashboards -and CI. Reports produced by the CLI include an additive, version-tagged +and CI. CLI reports also carry a small pilot section that accounts for three specific checks +one by one; it covers those three only, not all of TRACE. + +
+Technical detail: the obligation_accounting pilot + +Reports produced by the CLI include an additive, version-tagged `obligation_accounting` object for the bounded `TR-APR-001`, `TR-POL-003`, and `TR-SCA-002` pilot. During supported report construction, its rows are reconciled against the executable registry identified by `registry_id` and `registry_sha256`. @@ -86,6 +92,8 @@ and `obligation_accounting` is the sole new top-level member. Compatibility with consumers that require the exact historical key set is not established. The report remains an unsigned self-report; see [Known Limitations](LIMITATIONS.md). +
+ ## Test modules | Module | ID | Tests | @@ -94,6 +102,7 @@ remains an unsigned self-report; see [Known Limitations](LIMITATIONS.md). | Signature | `TR-SIG` | ES256/ES384/EdDSA, key binding, chain | | Runtime | `TR-RTE` | TEE platform, measurement format, RIM URI | | Policy | `TR-POL` | Bundle hash, enforcement mode, TEE binding | +| Appraisal | `TR-APR` | Appraisal status, verifier URI, policy reference, timestamp | | Transcript | `TR-TXN` | Tool-call transcript hash binding (Phase 2+) | | Transparency | `TR-ANC` | SCITT receipt URI, inclusion proof | | Provenance | `TR-SCA` | SLSA level, builder URI, digest format | diff --git a/conformance/docs/error-codes.md b/conformance/docs/error-codes.md index 99ff45b4..e8f87686 100644 --- a/conformance/docs/error-codes.md +++ b/conformance/docs/error-codes.md @@ -1,8 +1,8 @@ # Error Codes -All TRACE test failures emit a structured error code of the form `TR--`. +Look up a code from a failed or unverified check here: what it means and how to fix the record. Every finding the suite reports carries a code of the form `TR--`, where the middle part names the [module](modules.md) (the group of checks) and the number names the check. -## TR-ENV — Envelope +## TR-ENV: Envelope | Code | Description | How to fix | |------|-------------|------------| @@ -12,7 +12,7 @@ All TRACE test failures emit a structured error code of the form `TR------ @@ -24,7 +24,7 @@ The suite checks one record and the evidence supplied with it, reports the highe --- - Score a record, read the failures, and produce a report from the same run. + Install the suite, check a sample record, see what failed and why, and produce a report from the same run. [Getting Started](docs/quickstart.md) @@ -32,7 +32,7 @@ The suite checks one record and the evidence supplied with it, reports the highe --- - The report is not evidence, and it says so on its face. Each result's scope is set out module by module. + The report is not proof on its own, and it says so on its face. The limitations page sets out, check by check, what a result does and does not tell you. [Limitations](LIMITATIONS.md) @@ -40,7 +40,7 @@ The suite checks one record and the evidence supplied with it, reports the highe --- - Quote verification happens outside the suite. Check a real Intel TDX quote at [agentrust-io.com/verify](https://agentrust-io.com/verify/). + The suite does not check the processor's signed report itself; other tools do that. You can check a real Intel TDX report in your browser at [agentrust-io.com/verify](https://agentrust-io.com/verify/). [Runtime module](docs/modules/tr-rte.md) @@ -48,13 +48,13 @@ The suite checks one record and the evidence supplied with it, reports the highe --- - The suite scores TRACE records, the evidence step. The specification is at [trace.agentrust-io.com](https://trace.agentrust-io.com), and records can be anchored in the [TRACE Registry](https://agentrust-io.com/registry/). + AgenTrust covers four steps: the model, the agent, its actions, and the evidence. This suite scores TRACE records, the evidence step. The specification is at [trace.agentrust-io.com](https://trace.agentrust-io.com), and records can be entered in a public, append-only log, the [TRACE Registry](https://agentrust-io.com/registry/). [See the chain](https://agentrust-io.com/#chain) -The [eight modules](docs/modules.md) cover envelope, signature, runtime, policy, appraisal, transcript, transparency, and provenance checks. Read the [limitations](LIMITATIONS.md) to interpret what each result establishes. +The [eight modules](docs/modules.md) each look at one part of a record: the envelope (the outer wrapper and its basic fields), the signature, the runtime (the hardware it says it ran on), the policy (the rules the agent ran under), the appraisal (a verifier's verdict on the hardware evidence), the transcript (the log of tool calls), transparency (proof the record was entered in a public log) and provenance (how the software was built). Read the [limitations](LIMITATIONS.md) to see what each result does and does not tell you. ```bash pip install agentrust-trace-tests @@ -63,25 +63,29 @@ trace-tests verify --record path/to/trust-record.jwt --level 1 ## A report you can hand to someone else +`verify` prints results for you. `report` writes files you can give to someone who was not there, such as an auditor or a customer. + ```bash trace-tests report --record trust-record.json --html report.html --json report.json --badge trace.svg ``` -- `verify` answers a question for the person running it. `report` produces an artifact for somebody who was not there. -- `report` runs every level up to `--max-level`, because the useful answer is the highest level a record reaches, not whether it cleared the level someone happened to pick. -- The HTML report is self-contained: no scripts, no fonts, no external CSS, no badge service, nothing fetched when it is opened. +- `report` tries every level up to `--max-level`, so the answer is the highest level the record reaches, whichever level you happened to ask about. +- The HTML report is a single file that loads nothing else: no scripts, no fonts, no outside stylesheets, no badge service, nothing fetched when it is opened. + +To make a CI pipeline (the automated checks that run on every code change) fail below a level, add `--fail-under 1`. Without it the command always exits `0`, which is what you want when you only need the report. `report.json` keeps a stable layout, `schema: agentrust-io/trace-tests/report/1`, for dashboards and CI. -Use `--fail-under 1` to gate CI on a level. Without it the command always exits `0`, which is what you want when you are producing an artifact rather than enforcing a threshold. `report.json` is stable under `schema: agentrust-io/trace-tests/report/1` for dashboards and CI. +CLI reports also carry a small pilot section that accounts for three specific checks one by one. It covers those three only, not all of TRACE. -CLI reports add an independently versioned `obligation_accounting` member for a -bounded three-obligation pilot: `TR-APR-001`, `TR-POL-003`, and `TR-SCA-002`. -The rows and findings come from one execution snapshot, and the report refuses -an incomplete pilot matrix. This does not claim complete TRACE accounting. -The extension treats `report/1` as additively extensible; compatibility with -consumers requiring the exact historical top-level key set is not established. -See [Known limitations](LIMITATIONS.md) for the trust and replay boundary. +??? info "Technical detail: the obligation_accounting pilot" + CLI reports add an independently versioned `obligation_accounting` member for a + bounded three-obligation pilot: `TR-APR-001`, `TR-POL-003`, and `TR-SCA-002`. + The rows and findings come from one execution snapshot, and the report refuses + an incomplete pilot matrix. This does not claim complete TRACE accounting. + The extension treats `report/1` as additively extensible; compatibility with + consumers requiring the exact historical top-level key set is not established. + See [Known limitations](LIMITATIONS.md) for the trust and replay boundary. -A conformance report that looks authoritative and cannot be checked is the same shape of thing as a control plane writing its own log. So the report tells a reader who does not trust the sender to go and check the record instead, and gives them what they need to do it. +Anyone can edit a report, so a report that looks official but cannot be checked is no better than a system vouching for itself. The report therefore tells a reader who does not trust the sender to check the record directly, and gives them what they need to do it. ## Where to go next @@ -91,6 +95,8 @@ A conformance report that looks authoritative and cannot be checked is the same ## Test modules +The table uses the specification's own terms. Each module's page explains them. + | Module | ID | Tests | |---|---|---| | Envelope | `TR-ENV` | EAT structure, required fields, `iat` validity | diff --git a/conformance/measurement/REPORT.md b/conformance/measurement/REPORT.md index a322895e..88af9696 100644 --- a/conformance/measurement/REPORT.md +++ b/conformance/measurement/REPORT.md @@ -1,5 +1,7 @@ # Does `trace-tests` notice when its own conformance checks break? +In plain terms: this page asks whether the suite's own tests would notice if one of its checks quietly stopped working. The method is to break each check on purpose, one at a time, and count how many tests fail. The result below: every failure path is now caught by at least one test, and eight checks rely on exactly one. + **Measured** 2026-08-07 against `agentrust-io/trace-tests` @ `7fc189c` (release 0.4.1), with the normative schema from `agentrust-io/trace-spec` @ `dc7491c`. @@ -52,7 +54,7 @@ covered, and still count as verified. The sites are where the checks actually li `TR-TXN-001` carries the most weight. It is the Level 2 requirement that a tool transcript exist at all. If it regressed, implementations would continue to be stamped -Level 2 conformant without that property being checked — and the transcript is the claim +Level 2 conformant without that property being checked, and the transcript is the claim Level 2 is largely about. `TR-SIG-002` is the gate that rejects a key that is not Ed25519. With it inert, a record @@ -79,7 +81,7 @@ output is the conformance stamp rather than in the informative fixture set. `runtime.platform` enum. It holds nine values; the schema holds ten. `software-only` is missing. -The conformance module is correct — `tr_rte.py:_VALID_PLATFORMS` has all ten — so this is +The conformance module is correct (`tr_rte.py:_VALID_PLATFORMS` has all ten), so this is a defect in the self-test, not in what implementers run. Three-way comparison on a Level 0 record carrying `software-only`: @@ -90,7 +92,7 @@ tests/test_level0.py FAIL (contradicts both) ``` It is latent: no vector in `tests/vectors/` uses `software-only`, so nothing triggers it -today. It becomes active the moment one does — and it would then reject behaviour the +today. It becomes active the moment one does, and it would then reject behaviour the project deliberately added, in PR #16/#17, which updated the conformance module and added `tests/test_software_only_platform.py` while leaving this copy behind. @@ -140,7 +142,7 @@ is the property. The two are not related closely enough to substitute one for th - **An empty path is rejected before it can look valid.** `Path("")` is `.`, which is a directory, so the obvious existence check passes and the script measures whichever tree it happens to be standing in. The enum comparison had exactly this hole and reported - success having compared nothing — the failure it exists to find, in itself. It now + success having compared nothing: the failure it exists to find, in itself. It now refuses an empty argument, and refuses to report agreement when zero comparisons were made.