Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/conformance-docs.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
name: Conformance docs

on:
workflow_call:
push:
branches: [main]
paths:
Expand All @@ -16,7 +17,7 @@ permissions:
contents: read

concurrency:
group: conformance-docs-${{ github.ref }}
group: conformance-docs-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false

jobs:
Expand Down
46 changes: 41 additions & 5 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,18 +23,33 @@ on:
- "ROADMAP.md"
- "LIMITATIONS.md"
- "CNAME"
- "conformance/**"
- ".github/workflows/docs.yml"
- ".github/workflows/conformance-docs.yml"
pull_request:
paths:
- 'conformance/**'
- '.github/workflows/docs.yml'
- '.github/workflows/conformance-docs.yml'
- 'mkdocs.yml'
- 'docs/**'
workflow_dispatch:

permissions:
contents: write

concurrency:
group: docs-deploy
cancel-in-progress: false
group: ${{ github.event_name == 'pull_request' && format('docs-pr-{0}', github.ref) || 'docs-deploy' }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
conformance:
permissions:
contents: read
uses: ./.github/workflows/conformance-docs.yml
deploy:
name: Build and deploy docs
name: Build combined specification and conformance docs
needs: conformance
runs-on: ubuntu-latest

steps:
Expand Down Expand Up @@ -81,5 +96,26 @@ jobs:
run: |
sed 's|^docs_dir: \.$|docs_dir: .docs_build|' mkdocs.yml > .mkdocs_build.yml

- name: Build and deploy to GitHub Pages
run: mkdocs gh-deploy --force --clean --config-file .mkdocs_build.yml
- name: Build specification documentation
run: mkdocs build --clean --config-file .mkdocs_build.yml

- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: conformance-docs
path: site/conformance/

- name: Verify combined site
run: |
rm -f site/conformance/CNAME
test -s site/index.html
test -s site/conformance/index.html
test "$(cat site/CNAME)" = "trace.agentrust-io.com"

- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: trace-combined-docs
path: site/

- name: Deploy combined site to GitHub Pages
if: github.event_name != 'pull_request'
run: ghp-import -n -p -f site
30 changes: 22 additions & 8 deletions conformance/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,20 @@

## Unreleased

## v0.6.2 - 2026-10-02

### Added

- Portable anchor inclusion vectors contributed by dinakarjs: nine valid and
eighteen invalid cases, with a Python harness and independent JavaScript
cross-check (trace-spec #463; original trace-tests #137).

### Changed

- Maintain the conformance suite in trace-spec/conformance and publish through
its separate conformance publisher. Package name, import and CLI are unchanged.
- Serve conformance documentation under trace.agentrust-io.com/conformance/.

## v0.6.1 - 2026-09-26

### Changed
Expand Down Expand Up @@ -98,7 +112,7 @@ Each change here refuses input that 0.6.0 accepted, so a record that passed on
CI installs from hash-pinned locks (#101); actionlint and a test-environment
guard added (#100).

## v0.5.1 — 2026-08-22
## v0.5.1 - 2026-08-22

- Level 1 and Level 2 verification now requires a verifier-issued challenge via
`--expected-nonce` and checks it against the signed `runtime.nonce` using
Expand All @@ -107,29 +121,29 @@ Each change here refuses input that 0.6.0 accepted, so a record that passed on
CLI could report conformance for a fresh signed record containing an
attacker-chosen or replayed nonce.

## v0.5.0 — 2026-08-09
## v0.5.0 - 2026-08-09

### Added

- **`trace-tests report`: conformance results as an artifact somebody can forward.** `verify` answers a question for whoever ran it; a pass/fail in a terminal is useless to an auditor, a counterparty or an acquirer. The new command emits self-contained HTML, a machine-readable JSON document (`schema: agentrust-io/trace-tests/report/1`), and an SVG level badge.

It runs every level up to `--max-level` instead of one, because the answer a reader needs is the highest level the record reaches, not whether it cleared the level the person running the tool happened to choose. `--fail-under N` gates CI on a level; without it the command exits `0`, since producing an artifact and enforcing a threshold are different jobs.

**The report states that it is not evidence.** It is unsigned HTML describing one run of one suite version, and anyone can edit it, so every report carries the record's digest, the suite and authoring-library versions, and the command to reproduce the result — and tells a reader who does not trust the sender to go check the record instead. A conformance report that looks authoritative and cannot be checked is the same shape of thing as a control plane writing its own log, which is the problem this project exists to fix.
**The report states that it is not evidence.** It is unsigned HTML describing one run of one suite version, and anyone can edit it, so every report carries the record's digest, the suite and authoring-library versions, and the command to reproduce the result - and tells a reader who does not trust the sender to go check the record instead. A conformance report that looks authoritative and cannot be checked is the same shape of thing as a control plane writing its own log, which is the problem this project exists to fix.

Self-contained by construction: no scripts, no external CSS, no fonts, no badge service. A badge served from someone else's infrastructure would add a dependency to an artifact whose whole point is needing none. A test asserts the HTML fetches nothing.

The HTML and the JSON are rendered from one assembled structure so they cannot disagree about the verdict, and unverified findings count as failures from Level 1 up exactly as they do in `verify`.

## v0.4.1 — 2026-08-03
## v0.4.1 - 2026-08-03

### Fixed

- **`--version` reported the wrong version.** `__version__` was a second hardcoded literal alongside `pyproject.toml` and never moved, so it sat at `0.2.0` through both the 0.3.0 and 0.4.0 releases: `trace-tests --version` printed `0.2.0` from a 0.4.0 install while `importlib.metadata` correctly returned `0.4.0`. It is now read from installed distribution metadata, so there is one source of truth and the value cannot fall behind a release again.

This mattered more than a wrong string usually would. The v0.2 profile cutover shipped in 0.4.0, and a 0.2.x suite rejects every v0.2 record, so `--version` is exactly the command someone runs to work out whether their suite matches their producer. It was the one command that could not answer.

## v0.4.0 — 2026-07-28
## v0.4.0 - 2026-07-28

### Changed

Expand All @@ -139,19 +153,19 @@ Each change here refuses input that 0.6.0 accepted, so a record that passed on

- Registry, verifier, and documentation hosts moved from `agentrust.io` to `agentrust-io.com`.

## v0.3.0 — 2026-07-21
## v0.3.0 - 2026-07-21

- `azure-cvm-sev-snp` platform accepted (`runtime.platform`): Azure confidential VMs run SEV-SNP behind a Hyper-V paravisor (vTPM-rooted). Added to the bundled schema enum and the TR-RTE valid-platform set so Azure TRACE records pass conformance. Matches `agentrust-trace>=0.4`.

## v0.2.0 — 2026-06-19
## v0.2.0 - 2026-06-19

- DID subject support: `subject` now accepts `did:` URIs in addition to `spiffe://`.
- Embedded signature verification: plain TRACE records signed with `agentrust-trace sign_record()` are now cryptographically verified at all levels.
- SLSA Level 0: `build_provenance.slsa_level: 0` is now valid for software-only / development records.
- Software-only platform: `runtime.platform: "software-only"` accepted at Level 0.
- Private key leak detection: TR-SIG now fails records that embed a private key (`d` member) in `cnf.jwk`.

## v0.1.0 — 2026-05-01
## v0.1.0 - 2026-05-01

- Initial release with 7 test modules: TR-ENV, TR-SIG, TR-RTE, TR-POL, TR-TXN, TR-ANC, TR-SCA.
- Conformance levels 0, 1, 2.
6 changes: 3 additions & 3 deletions conformance/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ Community updates and contributor highlights: [AgenTrust on LinkedIn](https://ww
### Check a TRACE record and see which conformance level it reaches

<p align="center">
<a href="https://tests.agentrust-io.com">
<img src="https://img.shields.io/badge/%F0%9F%93%96_Full_Documentation-tests.agentrust--io.com-C17817?style=for-the-badge&logoColor=white" alt="Full Documentation" height="40">
<a href="https://trace.agentrust-io.com/conformance/">
<img src="https://img.shields.io/badge/%F0%9F%93%96_Full_Documentation-trace.agentrust--io.com%2Fconformance-C17817?style=for-the-badge&logoColor=white" alt="Full Documentation" height="40">
</a>
</p>

Expand Down Expand Up @@ -102,7 +102,7 @@ remains an unsigned self-report; see [Known Limitations](LIMITATIONS.md).

| | |
|---|---|
| 📖 Full documentation | [tests.agentrust-io.com](https://tests.agentrust-io.com) |
| 📖 Full documentation | [trace.agentrust-io.com/conformance/](https://trace.agentrust-io.com/conformance/) |
| 📄 TRACE Specification | [trace-spec](https://github.com/agentrust-io/trace-spec) |
| 🗂 Test schemas | [schemas/](schemas/) |
| 💬 Discussions | [GitHub Discussions](https://github.com/orgs/agentrust-io/discussions) |
Expand Down
4 changes: 2 additions & 2 deletions conformance/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ The suite checks one record and the evidence supplied with it, reports the highe
[What this proves, and what it does not](LIMITATIONS.md){ .md-button }

!!! tip "TL;DR"
[agentrust-trace-tests](https://pypi.org/project/agentrust-trace-tests/) 0.6.1 (Apache-2.0) runs eight modules against a record on your machine and writes a report carrying the record digest, the suite version and the command to reproduce it. A pass describes the record and says nothing about the agent, and TR-RTE checks the shape of attestation fields without verifying a quote against AMD or Intel roots.
[agentrust-trace-tests](https://pypi.org/project/agentrust-trace-tests/) 0.6.2 (Apache-2.0) runs eight modules against a record on your machine and writes a report carrying the record digest, the suite version and the command to reproduce it. A pass describes the record and says nothing about the agent, and TR-RTE checks the shape of attestation fields without verifying a quote against AMD or Intel roots.

<div class="grid cards" markdown>

Expand Down Expand Up @@ -104,4 +104,4 @@ A conformance report that looks authoritative and cannot be checked is the same

The suite tracks [TRACE Spec v0.2](https://trace.agentrust-io.com). See [Changelog](CHANGELOG.md) for what moved between suite versions.

**Status:** agentrust-trace-tests 0.6.1 · Apache-2.0 · tracks TRACE Spec v0.2 · Sponsored by OPAQUE, which funds the engineering, infrastructure and confidential-computing work behind these projects.
**Status:** agentrust-trace-tests 0.6.2 · Apache-2.0 · tracks TRACE Spec v0.2 · Sponsored by OPAQUE, which funds the engineering, infrastructure and confidential-computing work behind these projects.
2 changes: 1 addition & 1 deletion conformance/mkdocs.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
site_name: TRACE Tests
site_description: "Conformance test suite for TRACE: Trust Runtime Attestation and Compliance Evidence"
site_url: https://tests.agentrust-io.com
site_url: https://trace.agentrust-io.com/conformance/
repo_url: https://github.com/agentrust-io/trace-spec
repo_name: agentrust-io/trace-spec
edit_uri: edit/main/conformance/
Expand Down
4 changes: 2 additions & 2 deletions conformance/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "hatchling.build"

[project]
name = "agentrust-trace-tests"
version = "0.6.1"
version = "0.6.2"
description = "TRACE conformance test suite"
readme = "README.md"
license = { text = "Apache-2.0" }
Expand Down Expand Up @@ -36,7 +36,7 @@ dependencies = [
[project.urls]
Homepage = "https://github.com/agentrust-io/trace-spec"
Repository = "https://github.com/agentrust-io/trace-spec"
Documentation = "https://github.com/agentrust-io/trace-spec/tree/main/conformance#readme"
Documentation = "https://trace.agentrust-io.com/conformance/"
"Bug Tracker" = "https://github.com/agentrust-io/trace-spec/issues"

[project.scripts]
Expand Down
Loading