diff --git a/CHANGELOG.md b/CHANGELOG.md index 2ddb9985..06f3b995 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,8 @@ Format: [Semantic Versioning](https://semver.org/). Spec versions follow `MAJOR. ## [Unreleased] +- **Proposal: the embedded `signature` value must be canonically encoded (#247). Incompatible tightening of the draft's acceptance rules.** Section 3.2.2 already names the embedded `signature` field's encoding as base64url, no padding; it does not say whether a different spelling of the same bytes is the same record. A 64-byte signature (Ed25519 or ES256) spends 86 base64url characters on 512 bits, leaving 4 unused bits in the final character; RFC 4648 section 3.5 requires those bits to be zero. The added sentence requires the canonical spelling and requires a verifier to reject a record whose signature is not canonically encoded. The schema `pattern` is tightened so an 86-character value's final character must be `A`, `Q`, `g`, or `w`; `models.py` (through the shared pattern helper of #412), the packaged schema copy, and the v0.3 draft (regenerated from canonical) track it. **Newly rejected:** an 86-character `signature` ending in any of the other 60 base64url characters, which the previous pattern `^[A-Za-z0-9_-]+$` accepted; each valid signature had 15 such respellings. The reference library verified them through 0.10.0, because its decoder discarded the unused bits, and has refused them at decode time since 0.11.0 (#418); this change makes the schema, the models and the normative text give the same answer, so a consumer that validates a record without decoding its signature refuses it too. Values of any other length are unaffected; a 96-byte ES384 signature (128 characters) has no unused bits. **Existing records:** a record whose `signature` is non-canonical is rejected. The signature bytes are unchanged, so zeroing the final character's unused bits makes the record verify again without re-signing; the respelled record is nevertheless a different record for section 3.1.3, whose `parent_record_hash` covers the parent's `signature` member, so any child that names the old spelling must be re-issued. `examples/signature-encoding/` adds a canonical vector, reusing an already-published record and signature, and two non-canonical respellings of the same 64 bytes with different unused-bit patterns, plus a re-runnable `scan_published_signatures.py`. **Scan, limited to the revisions examined:** this repository's `examples/` at `63f4d15`, `agentrust-io/trace-tests` at `da4369b` and `agentrust-io/trace-registry` at `e26b85a` hold 232 signatures, all canonical; the 2 non-canonical values the scan also reports are this change's own deliberately non-canonical vectors. The scan says nothing about records outside those revisions. The `cnf.jwk.x` and trusted-key cases raise the same question for the specification text and are not part of this change; the library's decoder has treated them alike since #418. Requested by @imran-siddique on #247; proposed by @chernistry. + - Propose explicit verifier profile declarations and verifier-result field requirements for obligations 2 and 3 of #116, carried by Imran Siddique following the discussion with @lywinged. Obligations 1 and 4 remain deferred; the v0.1 cutover is unchanged. - **Breaking, proposed: `silent` mode allows a policy-denied action (#143, following #28).** Section 4.3 now says a deny MUST NOT block in `silent` mode and the audit chain MUST record every would-have-denied decision. v0.9.0 and v0.10.0 described `silent` as evaluated and enforced under the same `eat_profile`, so a producer that followed that text and blocks on deny no longer conforms, and a `silent` record does not say which meaning it was written under. Classified breaking under GOVERNANCE.md, so adoption needs the 30-day comment period and Project Lead sign-off. Section 4.3 also orders the modes: `advisory` is not weaker than `silent`. Enforce remains the default; schema constraints and runtime behavior are unchanged. Classification raised by @lywinged. diff --git a/docs/schema.md b/docs/schema.md index 3d817216..0453c9e7 100644 --- a/docs/schema.md +++ b/docs/schema.md @@ -32,7 +32,7 @@ can share one. A value that needs to be larger is carried as a string. The same | `appraisal` | object | **yes** | Verifier judgment | | `transparency` | string | no | Registry or SCITT anchor for the record. Optional below Level 2, where an unanchored record has no receipt to name. Use `null`, never `""` | | `cnf` | object | **yes** | Confirmation method: contains the `jwk` signing key | -| `signature` | string | **yes** | Base64url Ed25519 / ES256 / ES384 signature over the canonical record with only `signature` absent; `cnf` is included | +| `signature` | string | **yes** | Base64url Ed25519 / ES256 / ES384 signature over the canonical record with only `signature` absent; `cnf` is included. An 86-character value (a 64-byte Ed25519 or ES256 signature) MUST be canonically encoded per RFC 4648 section 3.5: it MUST end in `A`, `Q`, `g`, or `w` | diff --git a/examples/README.md b/examples/README.md index b9922e5a..91223820 100644 --- a/examples/README.md +++ b/examples/README.md @@ -49,6 +49,11 @@ checking any archived record, including these. verifies exactly as the pass does, unresolvable, and an issuer whose key the relying party does not hold. Every digest recomputes; see that directory's README and `docs/references-registry.md`. +- `signature-encoding/`: three signed Trust Records that separate a canonically-encoded + embedded signature from two non-canonical respellings of the same 64 bytes. Proposed, + not accepted normative text (#247); carries a `scan_published_signatures.py` tool + that reports how many published signatures are already canonical. See that + directory's README. The schema sets `additionalProperties: false`, so examples must not carry non-schema keys such as `_comment`. Keep descriptive notes in this file. diff --git a/examples/signature-encoding/01-canonical-signature.json b/examples/signature-encoding/01-canonical-signature.json new file mode 100644 index 00000000..ec57a022 --- /dev/null +++ b/examples/signature-encoding/01-canonical-signature.json @@ -0,0 +1,56 @@ +{ + "name": "canonical-signature", + "description": "An embedded signature in the canonical base64url spelling of its 64 bytes: the final character's four unused bits are zero, as RFC 4648 section 3.5 requires. Accepted today, and accepted under this proposal.", + "spec": "trace-v0.2 section 3.2.2, embedded signature bullet", + "profile": "trace.signature_encoding.proposal.v0", + "proposal": { + "issue": "agentrust-io/trace-spec#247", + "status": "under review, not accepted normative text" + }, + "encoding": "canonical", + "decoded_signature_hex": "c86f6b95f7f6fca94cc278604b3e6efb20224e5ed75593aee635d70f7a653e6e5859a502716ab756603fc6dc1c460dc90832914582d91c7f93a528c3318aca08", + "trusted_key": { + "kty": "OKP", + "crv": "Ed25519", + "x": "S9f_IqfsGG7jIzI7brZ7VVXCyF-_1jj_Guc1ynPKpuI" + }, + "record": { + "eat_profile": "tag:agentrust-io.com,2026:trace-v0.2", + "iat": 1785000000, + "subject": "spiffe://factory.example/agent/payments/prod", + "model": { + "provider": "example-provider", + "model_id": "example-model-1" + }, + "runtime": { + "platform": "software-only", + "measurement": "sha256:0000000000000000000000000000000000000000000000000000000000000000" + }, + "policy": { + "bundle_hash": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "enforcement_mode": "enforce" + }, + "data_class": "confidential", + "build_provenance": { + "slsa_level": 0, + "digest": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" + }, + "appraisal": { + "status": "affirming", + "verifier": "https://verifier.example/v1" + }, + "transparency": "https://rekor.example/api/v1/log/entries/0", + "cnf": { + "jwk": { + "kty": "OKP", + "crv": "Ed25519", + "x": "S9f_IqfsGG7jIzI7brZ7VVXCyF-_1jj_Guc1ynPKpuI" + } + }, + "signature": "yG9rlff2_KlMwnhgSz5u-yAiTl7XVZOu5jXXD3plPm5YWaUCcWq3VmA_xtwcRg3JCDKRRYLZHH-TpSjDMYrKCA" + }, + "expected": { + "outcome": "verified", + "failure": null + } +} diff --git a/examples/signature-encoding/02-non-canonical-respelling-low-bit.json b/examples/signature-encoding/02-non-canonical-respelling-low-bit.json new file mode 100644 index 00000000..4d2c46e7 --- /dev/null +++ b/examples/signature-encoding/02-non-canonical-respelling-low-bit.json @@ -0,0 +1,57 @@ +{ + "name": "non-canonical-signature-respelling-low-bit", + "description": "The same 64 signature bytes as 01-canonical-signature.json, respelled with the final character's unused bits set to binary 0001 instead of 0000: the smallest change that makes the spelling non-canonical. RFC 4648 section 3.5 calls this spelling non-canonical. Accepted today; this proposal requires a verifier to reject it.", + "spec": "trace-v0.2 section 3.2.2, embedded signature bullet", + "profile": "trace.signature_encoding.proposal.v0", + "proposal": { + "issue": "agentrust-io/trace-spec#247", + "status": "under review, not accepted normative text" + }, + "encoding": "non_canonical", + "canonical_counterpart": "01-canonical-signature.json", + "decoded_signature_hex": "c86f6b95f7f6fca94cc278604b3e6efb20224e5ed75593aee635d70f7a653e6e5859a502716ab756603fc6dc1c460dc90832914582d91c7f93a528c3318aca08", + "trusted_key": { + "kty": "OKP", + "crv": "Ed25519", + "x": "S9f_IqfsGG7jIzI7brZ7VVXCyF-_1jj_Guc1ynPKpuI" + }, + "record": { + "eat_profile": "tag:agentrust-io.com,2026:trace-v0.2", + "iat": 1785000000, + "subject": "spiffe://factory.example/agent/payments/prod", + "model": { + "provider": "example-provider", + "model_id": "example-model-1" + }, + "runtime": { + "platform": "software-only", + "measurement": "sha256:0000000000000000000000000000000000000000000000000000000000000000" + }, + "policy": { + "bundle_hash": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "enforcement_mode": "enforce" + }, + "data_class": "confidential", + "build_provenance": { + "slsa_level": 0, + "digest": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" + }, + "appraisal": { + "status": "affirming", + "verifier": "https://verifier.example/v1" + }, + "transparency": "https://rekor.example/api/v1/log/entries/0", + "cnf": { + "jwk": { + "kty": "OKP", + "crv": "Ed25519", + "x": "S9f_IqfsGG7jIzI7brZ7VVXCyF-_1jj_Guc1ynPKpuI" + } + }, + "signature": "yG9rlff2_KlMwnhgSz5u-yAiTl7XVZOu5jXXD3plPm5YWaUCcWq3VmA_xtwcRg3JCDKRRYLZHH-TpSjDMYrKCB" + }, + "expected": { + "outcome": "rejected", + "failure": "signature_not_canonical" + } +} diff --git a/examples/signature-encoding/03-non-canonical-respelling-all-bits.json b/examples/signature-encoding/03-non-canonical-respelling-all-bits.json new file mode 100644 index 00000000..fdfea198 --- /dev/null +++ b/examples/signature-encoding/03-non-canonical-respelling-all-bits.json @@ -0,0 +1,57 @@ +{ + "name": "non-canonical-signature-respelling-all-bits", + "description": "The same 64 signature bytes as 01-canonical-signature.json, respelled with the final character's unused bits set to binary 1111 instead of 0000: every unused bit set, rather than 02's single bit. Pairs with 02 so the boundary is not covered by one specific bad string: a verifier must check the unused bits are zero, not special-case the one respelling that most naturally comes to mind. Accepted today; this proposal requires a verifier to reject it.", + "spec": "trace-v0.2 section 3.2.2, embedded signature bullet", + "profile": "trace.signature_encoding.proposal.v0", + "proposal": { + "issue": "agentrust-io/trace-spec#247", + "status": "under review, not accepted normative text" + }, + "encoding": "non_canonical", + "canonical_counterpart": "01-canonical-signature.json", + "decoded_signature_hex": "c86f6b95f7f6fca94cc278604b3e6efb20224e5ed75593aee635d70f7a653e6e5859a502716ab756603fc6dc1c460dc90832914582d91c7f93a528c3318aca08", + "trusted_key": { + "kty": "OKP", + "crv": "Ed25519", + "x": "S9f_IqfsGG7jIzI7brZ7VVXCyF-_1jj_Guc1ynPKpuI" + }, + "record": { + "eat_profile": "tag:agentrust-io.com,2026:trace-v0.2", + "iat": 1785000000, + "subject": "spiffe://factory.example/agent/payments/prod", + "model": { + "provider": "example-provider", + "model_id": "example-model-1" + }, + "runtime": { + "platform": "software-only", + "measurement": "sha256:0000000000000000000000000000000000000000000000000000000000000000" + }, + "policy": { + "bundle_hash": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "enforcement_mode": "enforce" + }, + "data_class": "confidential", + "build_provenance": { + "slsa_level": 0, + "digest": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" + }, + "appraisal": { + "status": "affirming", + "verifier": "https://verifier.example/v1" + }, + "transparency": "https://rekor.example/api/v1/log/entries/0", + "cnf": { + "jwk": { + "kty": "OKP", + "crv": "Ed25519", + "x": "S9f_IqfsGG7jIzI7brZ7VVXCyF-_1jj_Guc1ynPKpuI" + } + }, + "signature": "yG9rlff2_KlMwnhgSz5u-yAiTl7XVZOu5jXXD3plPm5YWaUCcWq3VmA_xtwcRg3JCDKRRYLZHH-TpSjDMYrKCP" + }, + "expected": { + "outcome": "rejected", + "failure": "signature_not_canonical" + } +} diff --git a/examples/signature-encoding/README.md b/examples/signature-encoding/README.md new file mode 100644 index 00000000..5bb5ad76 --- /dev/null +++ b/examples/signature-encoding/README.md @@ -0,0 +1,158 @@ +# Signature-encoding vectors (proposed, not accepted) + +> These fixtures encode the obligation proposed in +> [agentrust-io/trace-spec#247](https://github.com/agentrust-io/trace-spec/issues/247). +> **No normative text for it has been accepted.** Do not read a passing vector here +> as a conformance requirement. + +Spec section 3.2.2 already names the embedded `signature` field's encoding as +base64url, no padding. What it leaves open is whether a *different spelling of +the same 64 bytes* is the same record. An Ed25519 or ES256 signature is 64 +bytes (512 bits); base64url without padding spends 86 characters (516 bits) on +them, so the final character carries 4 bits nothing signs. RFC 4648 section 3.5 +calls a spelling with non-zero unused bits non-canonical. A decoder that only +restores padding and decodes accepts it, because those bits are discarded, not +checked; a decoder that also checks them refuses the spelling. + +The proposal is not a choice of encoding. It is: does section 3.2.2 require the +canonical form of the encoding it already names. + +## The vectors + +| Fixture | Outcome | What it pins down | +|---|---|---| +| `01-canonical-signature.json` | verified | The signature is spelled canonically: the final character's unused bits are zero. | +| `02-non-canonical-respelling-low-bit.json` | rejected | The same 64 signature bytes, respelled with the final character's unused bits set to binary `0001`: the smallest change that makes the spelling non-canonical. | +| `03-non-canonical-respelling-all-bits.json` | rejected | The same 64 signature bytes again, respelled with the unused bits set to binary `1111`. | + +Two rejecting vectors, not one: `tests/test_adequacy_all_sets.py` flags a +boundary covered by a single rejecting vector, because an implementation could +special-case that one bad string and pass without implementing the rule of +checking the unused bits generally. `02` and `03` trip the same rule with two +different bad strings. + +All three fixtures carry the same `record`, `trusted_key` and +`decoded_signature_hex`; only `record.signature`'s final character differs +between them. `tests/test_signature_encoding.py` recomputes +`decoded_signature_hex` from the committed bytes rather than trusting the +declaration: it decodes every signature and asserts the raw bytes are equal +across all three, then re-derives which spellings are canonical from first +principles (decode, re-encode without padding, compare) rather than trusting +the `encoding` field either. + +`01` reuses the record, trusted key and signature already published at +`examples/verifier-compatibility/01-known-version-verified.json`, so this set +introduces no new signing key. `02` and `03` each change one character of that +same signature; the record each accompanies is otherwise byte-identical. + +## Format + +```jsonc +{ + "name": "...", + "description": "...", + "spec": "trace-v0.2 section 3.2.2, embedded signature bullet", + "profile": "trace.signature_encoding.proposal.v0", + "proposal": { + "issue": "agentrust-io/trace-spec#247", + "status": "under review, not accepted normative text" + }, + "encoding": "canonical" | "non_canonical", + "decoded_signature_hex": "...", // the 64 bytes all three fixtures share, as hex + "canonical_counterpart": "...", // 02 and 03 only: the file each respells + "trusted_key": { "kty": "OKP", "crv": "Ed25519", "x": "..." }, + "record": { "eat_profile": "...", "signature": "..." }, + "expected": { + "outcome": "verified" | "rejected", + "failure": null | "signature_not_canonical" + } +} +``` + +`expected.failure` is informative, the same convention +[`examples/verifier-compatibility/README.md`](../verifier-compatibility/README.md#what-failure-is-and-is-not) +states: it names which rule the vector expects to bite, not a wire format or an +exception message, and nothing in a portable adapter reads it. The +implementations measured so far do not share a failure class for it. Through +0.10.0 the reference library in this repository decoded a non-canonical +respelling leniently (the unused bits were discarded, not checked) and verified +it. Since 0.11.0 its decoder refuses the spelling (#418), and `verify_record` +reports that refusal before it reaches the schema step. The TypeScript verifier +under review in #376 refuses the same spelling and reports +`signature_malformed`. Tightening the schema pattern, as this proposal does, +adds the refusal to the two surfaces that never decode the value: schema +validation and the models. A consumer that only validates a record then gets +the same answer as one that verifies it. + +## Classification + +An incompatible tightening of the draft's acceptance rules: a record the +previous rules accepted can be rejected under this one. The maintainer set the +class on [#401](https://github.com/agentrust-io/trace-spec/pull/401). + +## Which spellings are newly rejected + +Only an 86-character `signature`, the length of a 64-byte Ed25519 or ES256 +signature. Its final character carries 2 signed bits and 4 unused ones, so each +signature has 16 spellings: one canonical, ending in `A`, `Q`, `g` or `w`, and 15 +that end in one of the other 60 base64url characters. The previous pattern, +`^[A-Za-z0-9_-]+$`, accepted all 16. The reference library verified all 16 +through 0.10.0, because its decoder discarded the unused bits, and has refused +the 15 at decode time since 0.11.0 (#418). The new pattern accepts the +canonical one only, so the schema and the models refuse the 15 as well. A +value of any other length is matched exactly as before; +a 96-byte ES384 signature spends 128 characters on 768 bits and has no unused +bits to set. + +## Handling existing records + +A record whose `signature` is non-canonical is rejected, with no grace period +in this proposal. Its signature bytes are not in question, so the holder can +zero the final character's unused bits and the record verifies again without +re-signing. The respelled record is still a different record under section +3.1.3: `delegation.parent_record_hash` is computed over the complete parent +record including its `signature` member, so a child that names the old +spelling no longer matches, and has to be re-issued against the new one. + +## Which existing records would stop passing + +`scan_published_signatures.py` walks every `*.json` file under a given root, +finds every 86-character `signature` value (the shape of a 64-byte embedded +signature), and reports how many are already canonically encoded. Run without +arguments it scans this repository's own `examples/`: + +```bash +python examples/signature-encoding/scan_published_signatures.py +``` + +Pass additional roots to reproduce a combined count over other checkouts, for +example the `agentrust-io/trace-tests` conformance corpus pinned at a specific +commit, or a read-only clone of `agentrust-io/trace-registry`: + +```bash +git clone https://github.com/agentrust-io/trace-tests /path/to/trace-tests +git -C /path/to/trace-tests checkout 3af2b53 +python examples/signature-encoding/scan_published_signatures.py \ + examples /path/to/trace-tests + +git clone https://github.com/agentrust-io/trace-registry /path/to/trace-registry +python examples/signature-encoding/scan_published_signatures.py /path/to/trace-registry +``` + +The numbers from one such run, and the exact command that produced them, belong +in the PR that carries this proposal, not in this file: they are a fact about +the revisions scanned on the day the scan ran, not about the vectors themselves, +and a count of canonical signatures in those revisions says nothing about +records minted elsewhere. Only +`signature` is scanned, at the top level of a Trust Record. `sig.value` in a +revocation statement or bundle carries the same base64url pattern. The +library's decoder has refused a non-canonical spelling of a bundle's +`sig.value` since #418; the schema pattern is untouched by this proposal, so +it is outside this scan's scope; see the PR for that as a candidate follow-up. + +## Boundary + +These vectors show whether a verifier's schema accepts or refuses a specific +respelling of a real, already-verifying signature. They make no claim about +the record's other fields, about revocation, or about freshness, each of which +is exercised elsewhere in `examples/`. diff --git a/examples/signature-encoding/scan_published_signatures.py b/examples/signature-encoding/scan_published_signatures.py new file mode 100644 index 00000000..0491992f --- /dev/null +++ b/examples/signature-encoding/scan_published_signatures.py @@ -0,0 +1,145 @@ +#!/usr/bin/env python3 +"""Count how many published embedded signatures are canonically encoded (#247). + +Spec section 3.2.2 already names the embedded `signature` field's encoding as +base64url, no padding. What was open is whether a non-canonical spelling of the +same 64 bytes is the same record: an Ed25519 or ES256 signature is 64 bytes, +base64url without padding spends 86 characters (516 bits) on 512 real bits, so +the final character carries 4 unused bits. RFC 4648 section 3.5 requires those +bits to be zero; a spelling that leaves them non-zero is non-canonical. + +This script answers the question the schema tightening needs answered before it +ships: which published records would stop passing. It walks every `*.json` file +under one or more root directories, finds every string value at the key +`signature` that is exactly 86 characters from the base64url alphabet, and +checks whether decoding and re-encoding it without padding reproduces the same +string. + +Only the `signature` key is scanned. `sig.value` in a revocation statement or +bundle (`schema/trace-revocation.json`, `schema/trace-revocation-bundle.json`) +carries the same base64url pattern and the same unused-bits question, but that +pattern is not touched by this change, so it is out of scope here. + +Usage: + python scan_published_signatures.py [ROOT ...] + +With no ROOT given, scans this repository's own `examples/` directory. Pass one +root per tree to reproduce a combined count over other checkouts, for example: + + python scan_published_signatures.py \\ + ../../examples \\ + /path/to/trace-tests-checkout-at-3af2b53 \\ + /path/to/trace-registry-checkout + +Exit code is 1 if any non-canonical signature is found, 0 otherwise, so the +script can also run as a regression gate. +""" +from __future__ import annotations + +import argparse +import base64 +import json +import re +import sys +from collections.abc import Iterator +from pathlib import Path +from typing import NamedTuple + +#: 86 base64url characters: the shape of a 64-byte (Ed25519 or ES256) signature +#: with no padding. See the module docstring for the bit-counting argument. +SIGNATURE_SHAPE = re.compile(r"^[A-Za-z0-9_-]{86}$") + + +class Finding(NamedTuple): + root: Path + file: Path + path: str + value: str + canonical: bool + + +def is_canonical(value: str) -> bool: + """True if *value* is the canonical base64url spelling of its decoded bytes. + + Decodes *value* (restoring the padding an unpadded encoding strips), then + re-encodes the result the same way `sign_record` does. A canonical spelling + round-trips to itself; a spelling with non-zero unused bits does not, + because encoding always emits zero there. + """ + raw = base64.urlsafe_b64decode(value + "=" * (-len(value) % 4)) + return base64.urlsafe_b64encode(raw).rstrip(b"=").decode() == value + + +def _walk(node: object, path: tuple[str, ...]) -> Iterator[tuple[str, str]]: + if isinstance(node, dict): + for key, value in node.items(): + yield from _walk(value, path + (str(key),)) + elif isinstance(node, list): + for index, value in enumerate(node): + yield from _walk(value, path + (str(index),)) + elif isinstance(node, str): + if path and path[-1] == "signature" and SIGNATURE_SHAPE.match(node): + yield ".".join(path), node + + +def scan(root: Path) -> Iterator[Finding]: + """Yield a `Finding` for every 86-character `signature` value under *root*.""" + for file in sorted(root.rglob("*.json")): + try: + data = json.loads(file.read_text(encoding="utf-8")) + except (json.JSONDecodeError, UnicodeDecodeError): + continue + for path, value in _walk(data, ()): + yield Finding(root, file, path, value, is_canonical(value)) + + +#: This repository's own examples/, used when no ROOT is given. +DEFAULT_ROOT = Path(__file__).resolve().parents[1] + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser( + description=__doc__.splitlines()[0] if __doc__ else None + ) + parser.add_argument( + "roots", + nargs="*", + metavar="ROOT", + help="directories to scan recursively for *.json " + f"(default: {DEFAULT_ROOT})", + ) + args = parser.parse_args(argv) + + roots = [Path(r).resolve() for r in args.roots] or [DEFAULT_ROOT] + + findings: list[Finding] = [] + for root in roots: + if not root.is_dir(): + print(f"skipping {root}: not a directory", file=sys.stderr) + continue + findings.extend(scan(root)) + + non_canonical = [f for f in findings if not f.canonical] + + for root in roots: + in_root = [f for f in findings if f.root == root] + canonical_in_root = sum(1 for f in in_root if f.canonical) + print(f"{root}: {len(in_root)} signature(s), {canonical_in_root} canonical") + + print( + f"total: {len(findings)} signature(s), " + f"{len(findings) - len(non_canonical)} canonical, " + f"{len(non_canonical)} non-canonical" + ) + + for finding in non_canonical: + print( + f" NON-CANONICAL: {finding.file} ({finding.path}): {finding.value}", + file=sys.stderr, + ) + + return 1 if non_canonical else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/schema/trace-claim-v0.3-draft.json b/schema/trace-claim-v0.3-draft.json index 5706137b..05277ee6 100644 --- a/schema/trace-claim-v0.3-draft.json +++ b/schema/trace-claim-v0.3-draft.json @@ -786,8 +786,8 @@ }, "signature": { "type": "string", - "description": "OPTIONAL embedded signature: base64url (no padding) signature by the cnf key over the canonical JSON form of the record with only this field absent. Every Trust Record MUST be signature-bound per spec section 3.2.2, but enveloped profiles (e.g. JWS, cMCP RuntimeClaim) carry the signature outside the record, so this field is not required by the schema.", - "pattern": "^[A-Za-z0-9_-]+$" + "description": "OPTIONAL embedded signature: base64url (no padding) signature by the cnf key over the canonical JSON form of the record with only this field absent. Every Trust Record MUST be signature-bound per spec section 3.2.2, but enveloped profiles (e.g. JWS, cMCP RuntimeClaim) carry the signature outside the record, so this field is not required by the schema. An 86-character value is the base64url encoding of a 64-byte signature (Ed25519 or ES256) and MUST be canonically encoded per RFC 4648 section 3.5: its final character's unused bits MUST be zero, so it MUST end in A, Q, g, or w.", + "pattern": "^(?:[A-Za-z0-9_-]{1,85}|[A-Za-z0-9_-]{85}[AQgw]|[A-Za-z0-9_-]{87,})$" } }, "additionalProperties": false, diff --git a/schema/trace-claim.json b/schema/trace-claim.json index b73f7f0b..dafefa27 100644 --- a/schema/trace-claim.json +++ b/schema/trace-claim.json @@ -715,8 +715,8 @@ }, "signature": { "type": "string", - "description": "OPTIONAL embedded signature: base64url (no padding) signature by the cnf key over the canonical JSON form of the record with only this field absent. Every Trust Record MUST be signature-bound per spec section 3.2.2, but enveloped profiles (e.g. JWS, cMCP RuntimeClaim) carry the signature outside the record, so this field is not required by the schema.", - "pattern": "^[A-Za-z0-9_-]+$" + "description": "OPTIONAL embedded signature: base64url (no padding) signature by the cnf key over the canonical JSON form of the record with only this field absent. Every Trust Record MUST be signature-bound per spec section 3.2.2, but enveloped profiles (e.g. JWS, cMCP RuntimeClaim) carry the signature outside the record, so this field is not required by the schema. An 86-character value is the base64url encoding of a 64-byte signature (Ed25519 or ES256) and MUST be canonically encoded per RFC 4648 section 3.5: its final character's unused bits MUST be zero, so it MUST end in A, Q, g, or w.", + "pattern": "^(?:[A-Za-z0-9_-]{1,85}|[A-Za-z0-9_-]{85}[AQgw]|[A-Za-z0-9_-]{87,})$" } }, "additionalProperties": false, diff --git a/spec/trace-v0.2.md b/spec/trace-v0.2.md index eff86041..1001a84c 100644 --- a/spec/trace-v0.2.md +++ b/spec/trace-v0.2.md @@ -434,7 +434,7 @@ Example result, in the appraisal of a verifier that re-ran the function and obta **Signature binding.** Every TRACE Trust Record MUST be cryptographically bound by a signature over its canonical JSON form, made by the key in `cnf`. Canonicalization is RFC 8785 (JCS) unless the profile declares a different canonicalization. The signature MAY be either: -- **Embedded:** carried in the record's top-level `signature` field (base64url, no padding), computed over the canonical form of the record with the `signature` field absent; or +- **Embedded:** carried in the record's top-level `signature` field (base64url, no padding), computed over the canonical form of the record with the `signature` field absent. The value MUST be the canonical base64url encoding (RFC 4648 section 3.5: unused trailing bits in the final character MUST be zero), and a verifier MUST reject a record whose `signature` is not canonically encoded. - **Enveloping:** carried by a signed wrapper structure, e.g. a JWS (RFC 7515) whose payload is the record, a COSE_Sign1 envelope, or cMCP's RuntimeClaim (signature over the canonical record, key in `trace.cnf.jwk`). **Canonical form (RFC 8785 JCS).** The canonical form of a TRACE record for signature purposes is produced by the following algorithm: diff --git a/src/agentrust_trace/_patterns.py b/src/agentrust_trace/_patterns.py index 2fbb53ef..52f9a071 100644 --- a/src/agentrust_trace/_patterns.py +++ b/src/agentrust_trace/_patterns.py @@ -11,7 +11,7 @@ (r"^P(\d+W|(\d+Y(\d+M)?(\d+D)?|\d+M(\d+D)?|\d+D)" r"(T(\d+H(\d+M)?(\d+S)?|\d+M(\d+S)?|\d+S))?" r"|T(\d+H(\d+M)?(\d+S)?|\d+M(\d+S)?|\d+S))$"), - '^[A-Za-z0-9_-]+$', + '^(?:[A-Za-z0-9_-]{1,85}|[A-Za-z0-9_-]{85}[AQgw]|[A-Za-z0-9_-]{87,})$', ) _PYTHON_PATTERNS = { pattern: re.compile( diff --git a/src/agentrust_trace/models.py b/src/agentrust_trace/models.py index 78440b06..7363ee84 100644 --- a/src/agentrust_trace/models.py +++ b/src/agentrust_trace/models.py @@ -37,6 +37,15 @@ _DURATION_TIME = r"(\d+H(\d+M)?(\d+S)?|\d+M(\d+S)?|\d+S)" _DURATION_DATE = r"(\d+Y(\d+M)?(\d+D)?|\d+M(\d+D)?|\d+D)" _DURATION_RE = rf"^P(\d+W|{_DURATION_DATE}(T{_DURATION_TIME})?|T{_DURATION_TIME})$" +# Base64url without padding, in its canonical spelling where the length leaves +# unused bits. An 86-character value is a 64-byte signature (Ed25519 or ES256): +# its final character carries 2 signed bits and 4 unused ones, which RFC 4648 +# section 3.5 requires to be zero, so that character is one of A, Q, g, w. Other +# lengths are matched as before. The same string as `schema/trace-claim.json` and +# `_patterns._ECMA_PATTERNS`, held there by +# `tests/test_the_schema_and_the_models_agree.py` and +# `tests/test_schema_regex_semantics.py`. +_SIGNATURE_RE = r"^(?:[A-Za-z0-9_-]{1,85}|[A-Za-z0-9_-]{85}[AQgw]|[A-Za-z0-9_-]{87,})$" # The JCS safe-integer range, RFC 8785 Appendix B note 1, raised to a MUST by spec # section 3.2.2. Mirrored here because these models are the other artifact a producer @@ -505,13 +514,17 @@ class TrustRecord(_TraceModel): """ cnf: ConfirmationKey signature: Annotated[ - str, Field(pattern=r"^[A-Za-z0-9_-]+$"), - AfterValidator(partial(_require_pattern, pattern=r"^[A-Za-z0-9_-]+$")), + str, Field(pattern=_SIGNATURE_RE), + AfterValidator(partial(_require_pattern, pattern=_SIGNATURE_RE)), ] | None = None """Optional embedded signature (base64url, no padding) by the cnf key over the canonical JSON form of the record with only this field absent. Every Trust Record must be signature-bound per spec section 3.2.2; enveloped profiles carry the signature - outside the record instead of in this field.""" + outside the record instead of in this field. + + An 86-character value is the base64url encoding of a 64-byte signature (Ed25519 or + ES256) and must be canonically encoded per RFC 4648 section 3.5: the final + character's unused bits must be zero, so it must end in A, Q, g, or w.""" @model_validator(mode="after") def _origin_cannot_claim_hardware(self) -> TrustRecord: diff --git a/src/agentrust_trace/schema/trace-v0.2.json b/src/agentrust_trace/schema/trace-v0.2.json index b73f7f0b..dafefa27 100644 --- a/src/agentrust_trace/schema/trace-v0.2.json +++ b/src/agentrust_trace/schema/trace-v0.2.json @@ -715,8 +715,8 @@ }, "signature": { "type": "string", - "description": "OPTIONAL embedded signature: base64url (no padding) signature by the cnf key over the canonical JSON form of the record with only this field absent. Every Trust Record MUST be signature-bound per spec section 3.2.2, but enveloped profiles (e.g. JWS, cMCP RuntimeClaim) carry the signature outside the record, so this field is not required by the schema.", - "pattern": "^[A-Za-z0-9_-]+$" + "description": "OPTIONAL embedded signature: base64url (no padding) signature by the cnf key over the canonical JSON form of the record with only this field absent. Every Trust Record MUST be signature-bound per spec section 3.2.2, but enveloped profiles (e.g. JWS, cMCP RuntimeClaim) carry the signature outside the record, so this field is not required by the schema. An 86-character value is the base64url encoding of a 64-byte signature (Ed25519 or ES256) and MUST be canonically encoded per RFC 4648 section 3.5: its final character's unused bits MUST be zero, so it MUST end in A, Q, g, or w.", + "pattern": "^(?:[A-Za-z0-9_-]{1,85}|[A-Za-z0-9_-]{85}[AQgw]|[A-Za-z0-9_-]{87,})$" } }, "additionalProperties": false, diff --git a/tests/test_adequacy_all_sets.py b/tests/test_adequacy_all_sets.py index e0596d30..daf5f698 100644 --- a/tests/test_adequacy_all_sets.py +++ b/tests/test_adequacy_all_sets.py @@ -138,6 +138,15 @@ def reproducibility_claim() -> list[Vector]: lambda e: e["outcome"], lambda e: list(e.get("codes") or [])) +def signature_encoding() -> list[Vector]: + """The signature-encoding set (spec section 3.2.2, proposal #247). One rule, one + code, and two rejecting vectors that trip it with different unused-bit patterns + (0001 and 1111), so the boundary is not covered by one specific bad string a + shortcut implementation could special-case.""" + return _load("signature-encoding", + lambda e: e["outcome"], lambda e: [e.get("failure")]) + + SETS = { "build-provenance-depth": (build_provenance_depth, _depth_boundary), "reproducibility-claim": (reproducibility_claim, None), @@ -145,6 +154,7 @@ def reproducibility_claim() -> list[Vector]: "canonicalization-boundary": (canonicalization_boundary, None), "delegation-link": (delegation_link, None), "verifier-compatibility": (verifier_compatibility, None), + "signature-encoding": (signature_encoding, None), } # Every set must be able to fail both unconditional implementations. A set that diff --git a/tests/test_schema_regex_semantics.py b/tests/test_schema_regex_semantics.py index e48add08..e5fb3b8c 100644 --- a/tests/test_schema_regex_semantics.py +++ b/tests/test_schema_regex_semantics.py @@ -122,6 +122,9 @@ def test_shipped_pattern_results_match_javascript(): "", "spiffe://example.org/agent", "did:web:example.org", "did:web:\U0001f600", "spiffe://exam\nple.org/agent", "sha256:" + "a" * 64, "sha384:" + "f" * 96, "abc_DEF-123", "P1D", "PT2H", "P\u0661D", "P\uff11D", "P1W", + # The signature pattern constrains the final character at 86 characters only: + # 85, a canonical 86, a non-canonical 86, and with the "x" suffix below, 86 and 87. + "A" * 85, "A" * 85 + "Q", "A" * 85 + "B", ] cases = [[pattern, value + tail + suffix] for pattern in sorted(set(_patterns(SCHEMA))) diff --git a/tests/test_signature_encoding.py b/tests/test_signature_encoding.py new file mode 100644 index 00000000..c3b8d44d --- /dev/null +++ b/tests/test_signature_encoding.py @@ -0,0 +1,252 @@ +"""Signature-encoding vectors (spec section 3.2.2, proposal #247). + +An Ed25519 or ES256 embedded signature is 64 bytes; base64url without padding +spends 86 characters (516 bits) on those 512 bits, so the final character +carries 4 bits nothing signs. RFC 4648 section 3.5 requires those unused bits +to be zero. `examples/signature-encoding/` carries three vectors that share the +same 64 decoded bytes and differ only in the final character's unused bits: +`01-canonical-signature.json` (zero, canonical), and two non-canonical +respellings, `02-non-canonical-respelling-low-bit.json` (binary 0001) and +`03-non-canonical-respelling-all-bits.json` (binary 1111). Two rejecting +respellings, not one: `tests/test_adequacy_all_sets.py` (criterion 2, "no +margin") flags a boundary a single vector covers, because an implementation +that special-cased that one bad string would pass it without implementing the +rule. + +Every fact this module checks is recomputed from the committed bytes rather +than trusted from the fixtures' own declarations, matching +`tests/test_canonicalization_boundary.py`: that the three signatures decode to +the same 64 bytes, and which of the three spellings is canonical. +""" + +from __future__ import annotations + +import base64 +import json +from pathlib import Path +from typing import Any + +import pytest +import rfc8785 +from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey +from jsonschema import ValidationError as SchemaValidationError +from pydantic import ValidationError as ModelValidationError + +from agentrust_trace import TrustRecord, validate_json, verify_record +from agentrust_trace.validate import iter_errors + +REPO_ROOT = Path(__file__).parent.parent +FIXTURE_DIR = REPO_ROOT / "examples" / "signature-encoding" +CANONICAL_PATH = FIXTURE_DIR / "01-canonical-signature.json" +NON_CANONICAL_PATHS = [ + FIXTURE_DIR / "02-non-canonical-respelling-low-bit.json", + FIXTURE_DIR / "03-non-canonical-respelling-all-bits.json", +] +ALL_PATHS = [CANONICAL_PATH, *NON_CANONICAL_PATHS] + + +def _load(path: Path) -> dict[str, Any]: + return json.loads(path.read_text(encoding="utf-8")) + + +def _b64u(value: str) -> bytes: + return base64.urlsafe_b64decode(value + "=" * (-len(value) % 4)) + + +def _is_canonical(value: str) -> bool: + """RFC 4648 section 3.5, checked directly: re-encoding always emits zero for + the unused bits, so a value that round-trips through decode/re-encode had + them at zero already.""" + return base64.urlsafe_b64encode(_b64u(value)).rstrip(b"=").decode() == value + + +def _verify(fixture: dict[str, Any]) -> Any: + """Exercise the public API with real schema, trust and age inputs, `iat` + pinned as "now" so the vector reproduces regardless of when the suite runs.""" + validate_json(fixture["record"]) + return verify_record( + fixture["record"], + fixture["trusted_key"], + allow_embedded_key=False, + max_age_seconds=60, + max_future_skew_seconds=0, + now=fixture["record"]["iat"], + ) + + +def test_vector_set_is_complete() -> None: + assert sorted(path.name for path in FIXTURE_DIR.glob("*.json")) == [ + "01-canonical-signature.json", + "02-non-canonical-respelling-low-bit.json", + "03-non-canonical-respelling-all-bits.json", + ] + + +@pytest.mark.parametrize("path", NON_CANONICAL_PATHS, ids=lambda p: p.stem) +def test_every_respelling_decodes_to_the_same_64_bytes_as_canonical(path: Path) -> None: + """A respelling of one signature, not a different signature.""" + canonical = _load(CANONICAL_PATH) + non_canonical = _load(path) + + canonical_sig = canonical["record"]["signature"] + non_canonical_sig = non_canonical["record"]["signature"] + canonical_raw = _b64u(canonical_sig) + non_canonical_raw = _b64u(non_canonical_sig) + + assert len(canonical_raw) == 64, "not a 64-byte signature; the vector premise is wrong" + assert canonical_raw == non_canonical_raw + assert canonical_raw.hex() == canonical["decoded_signature_hex"] + assert non_canonical_raw.hex() == non_canonical["decoded_signature_hex"] + + # Differ in exactly the final character, and nowhere else. + assert canonical_sig != non_canonical_sig + assert canonical_sig[:-1] == non_canonical_sig[:-1] + assert canonical_sig[-1] != non_canonical_sig[-1] + + # The two records are otherwise byte-identical: only `signature` differs. + canonical_record_no_sig = {k: v for k, v in canonical["record"].items() if k != "signature"} + non_canonical_record_no_sig = { + k: v for k, v in non_canonical["record"].items() if k != "signature" + } + assert canonical_record_no_sig == non_canonical_record_no_sig + + +def test_the_two_respellings_differ_from_each_other_too() -> None: + """Two distinct bad strings, not the same one committed twice under two names.""" + low_bit = _load(NON_CANONICAL_PATHS[0])["record"]["signature"] + all_bits = _load(NON_CANONICAL_PATHS[1])["record"]["signature"] + assert low_bit != all_bits + assert low_bit[:-1] == all_bits[:-1] + assert low_bit[-1] != all_bits[-1] + + +def test_declared_encoding_matches_recomputed_canonicality() -> None: + """`encoding` is recomputed here, never trusted from the fixture.""" + canonical = _load(CANONICAL_PATH) + assert canonical["encoding"] == "canonical" + assert _is_canonical(canonical["record"]["signature"]) is True + + for path in NON_CANONICAL_PATHS: + fixture = _load(path) + assert fixture["encoding"] == "non_canonical" + assert _is_canonical(fixture["record"]["signature"]) is False + + +def test_canonical_signature_is_schema_valid_and_verifies() -> None: + fixture = _load(CANONICAL_PATH) + assert fixture["expected"] == {"outcome": "verified", "failure": None} + + result = _verify(fixture) + assert result.profile == fixture["record"]["eat_profile"] + TrustRecord.model_validate(fixture["record"]) # the model agrees + + +@pytest.mark.parametrize("path", NON_CANONICAL_PATHS, ids=lambda p: p.stem) +def test_non_canonical_respelling_is_rejected_by_the_tightened_pattern(path: Path) -> None: + fixture = _load(path) + assert fixture["expected"] == { + "outcome": "rejected", + "failure": "signature_not_canonical", + } + + # The schema refuses it on its own, at `signature`, by `pattern`: the surface a + # consumer that validates without verifying relies on. + with pytest.raises(SchemaValidationError): + validate_json(fixture["record"]) + assert any( + list(error.path) == ["signature"] and error.validator == "pattern" + for error in iter_errors(fixture["record"]) + ) + + # Not `_verify()`: that helper calls `validate_json` directly, ahead of + # `verify_record`'s own try/except, so the raw `SchemaValidationError` above + # would reach here unwrapped instead of the `ValueError` `verify_record` + # documents for every rejection. Which step raises it is not pinned: since + # 0.11.0 the decoder refuses the spelling before the schema step runs, and + # before that the schema step was the first to see it. Both name the field. + with pytest.raises(ValueError, match="signature"): + verify_record( + fixture["record"], + fixture["trusted_key"], + allow_embedded_key=False, + max_age_seconds=60, + max_future_skew_seconds=0, + now=fixture["record"]["iat"], + ) + + with pytest.raises(ModelValidationError): + TrustRecord.model_validate(fixture["record"]) + + +@pytest.mark.parametrize("path", NON_CANONICAL_PATHS, ids=lambda p: p.stem) +def test_the_underlying_ed25519_signature_still_verifies_either_spelling(path: Path) -> None: + """The respelling is refused for its spelling, not because the bytes are wrong. + + Independent of the TRACE schema, the model, and this library's own + `_b64url_decode`: decoded with the standard library alone and checked with + `cryptography` alone, a non-canonical respelling verifies exactly like the + canonical spelling, because both decode to the same 64 bytes over the same + RFC 8785 preimage. This isolates the spelling, not a broken signature, as the + reason the record is refused. + """ + canonical = _load(CANONICAL_PATH) + non_canonical = _load(path) + trusted = canonical["trusted_key"] + assert trusted == non_canonical["trusted_key"] + assert trusted["kty"] == "OKP" and trusted["crv"] == "Ed25519" + + public_key = Ed25519PublicKey.from_public_bytes(_b64u(trusted["x"])) + preimage = rfc8785.dumps( + {k: v for k, v in canonical["record"].items() if k != "signature"} + ) + + public_key.verify(_b64u(canonical["record"]["signature"]), preimage) + public_key.verify(_b64u(non_canonical["record"]["signature"]), preimage) + + +_CANONICAL = _load(CANONICAL_PATH)["record"]["signature"] +_NON_CANONICAL = _load(NON_CANONICAL_PATHS[0])["record"]["signature"] + + +@pytest.mark.parametrize( + ("signature", "accepted"), + [ + (_CANONICAL, True), + (_NON_CANONICAL, False), + # One character either side of 86 is outside the rule: the pattern constrains + # the final character only at the length of a 64-byte signature. + (_NON_CANONICAL[:-1], True), + (_NON_CANONICAL + "A", True), + ("A" * 128, True), # the length of a 96-byte ES384 signature: no unused bits + ("", False), + # An end anchor that tolerates a final line terminator would let these through. + *[(_CANONICAL + tail, False) for tail in ("\n", "\r", "\u2028", "\u2029")], + (_CANONICAL[:-1] + "=", False), # padding is outside the alphabet + ], + ids=[ + "canonical-86", "non-canonical-86", "85", "87", "128", "empty", + "canonical+LF", "canonical+CR", "canonical+LS", "canonical+PS", "padding", + ], +) +def test_the_schema_and_the_model_agree_on_the_signature_boundary( + signature: str, accepted: bool +) -> None: + """One pattern string, two engines, one answer. + + The schema runs the pattern through the ECMA-262 adapter and the model runs it + through pydantic's engine plus the shared helper; this holds the two to the same + verdict on the values that sit on the pattern's edges. + """ + record = {**_load(CANONICAL_PATH)["record"], "signature": signature} + + schema_accepts = not any( + list(error.path) == ["signature"] for error in iter_errors(record) + ) + try: + TrustRecord.model_validate(record) + model_accepts = True + except ModelValidationError: + model_accepts = False + + assert schema_accepts is accepted + assert model_accepts is accepted