Skip to content

feat(registryctl): add registry-backed Notary tutorial#407

Merged
jeremi merged 1 commit into
release/1.0from
agent/redesign-registry-notary-tutorials
Jul 17, 2026
Merged

feat(registryctl): add registry-backed Notary tutorial#407
jeremi merged 1 commit into
release/1.0from
agent/redesign-registry-notary-tutorials

Conversation

@jeremi

@jeremi jeremi commented Jul 17, 2026

Copy link
Copy Markdown
Member

Summary

  • simplify the benefits workbook around one canonical date_of_birth, remove unused and derived person/application fields, and separate operational records from restricted identity and address projections
  • revise the spreadsheet tutorial to show representative responses, restricted source-field access, purpose and scope enforcement, and a hands-on disclosure-policy edit
  • redesign the Notary tutorial as the natural continuation: add Notary to the first project, match by name plus date of birth without a national ID, compare accepted, policy-rejected, and no-match results, then edit the acceptance rule so a pending record becomes accepted
  • add registryctl add notary with an editable authored project, generated local consultation path, credentials, fixtures, rollback protection, and lifecycle rebuilds
  • keep the tutorial evaluation-only and registry-backed: Relay owns source access, Notary receives minimized consultation output, and no credential profile or OID4VCI issuance capability is generated
  • validate fixed local runtime overrides before one atomic publication and mount generated config directories so restarts cannot retain stale files after bundle replacement
  • strengthen the executable tutorial gate so documented JSON responses and the complete Relay-to-Notary edit journey are checked against source-built binaries

Implementation gaps found and fixed

  • local overrides were initially applied after the compiler's atomic publication, which briefly exposed production defaults to the tutorial runtime; they now pass product validation before publication
  • mounting one generated Notary file retained an obsolete inode across atomic directory replacement; Notary now mounts the generated config directory, matching Relay
  • add-on sidecars could be overwritten or left behind after a failed setup; the command now refuses pre-existing destinations and rolls back the token, TLS files, and generated consultation state
  • owner-only generated configuration was unreadable to image-default users; only the consultation bootstrap, consultation Relay, and Notary now run as the generated project runtime identity, while PostgreSQL and JWKS retain their required image identities
  • the consultation Relay named cache volume was owned by the image UID and could not support an arbitrary project UID; it is now a private project-owned bind at state/relay-consultation/cache
  • root-owned projects retain the 65532 fallback safely: config ownership is assigned only inside the unpublished owner-only build tree, and the token is staged at 0600, assigned through its open descriptor, synced, and atomically renamed
  • the public tutorial no longer recommends an incomplete ad hoc source-CLI route; it requires matching 1.0+ CLI, immutable images, and image lock, and identifies source-image rebinding as internal CI only
  • generated persistence guidance now accurately distinguishes in-memory Notary evaluation state, the consultation Relay PostgreSQL volume, and the host cache that may contain source rows

Verification

  • exact head: 2ce08376330e47bfde65a58f27959bc5c1f2969d
  • exact parent: 0aef9cf329e5b31e530ca383da86b66e519b3d87
  • cargo fmt --all -- --check
  • cargo clippy --locked -p registryctl --all-targets -- -D warnings
  • cargo test --locked -p registryctl (239 tests)
  • npm run test:tutorial:registryctl (9/9)
  • npm run check:tutorial:dry-run
  • npm run check:tutorial:registryctl (full live Relay and registry-backed Notary edit journey)
  • docs content, markdown, style, and build checks
  • independent exact-head security review: no findings, approved
  • independent exact-head developer-experience review: no findings, approved

Remaining release gate

Keep this PR in draft until every applicable exact-head GitHub check is green. Matching published 1.0 CLI, images, and image lock still require release-candidate verification before this tutorial is advertised as the runnable release path.

jeremi commented Jul 17, 2026

Copy link
Copy Markdown
Member Author

Review disposition: keep this PR in draft and do not merge the current head.

Two blockers remain:

  1. The required registryctl tutorials from source check fails because the new consultation Relay/Notary services run as their image users while project authoring writes the mounted private config and workload-token paths as owner-only (0700/0600). The consultation Relay bootstrap exits with Permission denied reading /etc/registry-relay/relay.yaml. The focused repair is to run the new services with the generated project runtime UID, matching the existing Relay service, while retaining the private host-side file modes.
  2. Head cd7b0a8e is based on a72483fe and predates integrated feat(relay): publish runtime config schema #406 and feat(registryctl): add editor-ready project initialization #404. Rebase onto the then-current release/1.0 after the permissions fix and rerun the complete exact-head matrix.

The registry-backed trust review is otherwise clean: I found no source-free or self-attested issuance path. Relay owns registry access, Notary consumes minimized consultation output, and the documented result remains predicate-only.

Required before ready-for-review: focused permission fix, current-base rebase, green source tutorial and full remote matrix, then independent exact-head re-review.

jeremi commented Jul 17, 2026

Copy link
Copy Markdown
Member Author

Updated repair scope after independent security and DX review:

The previously proposed runtime-UID change is necessary but not sufficient.

Required focused repair:

  1. In the Notary add-on Compose fragment, run only registry-relay-consultation-bootstrap, registry-relay-consultation, and registry-notary as ${REGISTRY_STACK_RUNTIME_UID:-65532}:${REGISTRY_STACK_RUNTIME_GID:-65532}, matching the base Relay service.
  2. Keep PostgreSQL on its existing root-init/drop-privileges path and keep JWKS unchanged.
  3. Preserve generated private host permissions at 0700/0600.
  4. Replace the consultation Relay named cache volume with a private project bind directory at ./state/relay-consultation/cache, create it through the existing runtime-identity/state-dir mechanism, and include it in rollback handling. A fresh named volume is owned by image UID 65532 and becomes unwritable when the service runs as an arbitrary project UID.
  5. Rebase onto the current release/1.0, preserving the reviewed registry-backed-only issuance changes and the existing removal of the held hosted credential-tour link.

Acceptance gates:

  • Compose assertions for the exact three service identities, no PostgreSQL override, and bind-backed consultation cache.
  • Private directory/file mode assertions and rollback coverage.
  • Generated Notary config remains evaluation-only and registry-backed, with no credential/OID4VCI issuance capability.
  • Focused registryctl tests, formatting/lint/workspace checks, docs checks, and the full Ubuntu/Linux registryctl tutorial-from-source Docker journey.

Both independent reviews found no source-free/self-attested issuance regression and approved this bounded repair plan. The PR remains draft until the rebase, repair, exact-head checks, and independent re-review are complete.

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
@jeremi
jeremi force-pushed the agent/redesign-registry-notary-tutorials branch from cd7b0a8 to 2ce0837 Compare July 17, 2026 22:27
@jeremi
jeremi marked this pull request as ready for review July 17, 2026 22:57
@jeremi
jeremi merged commit 6f181f1 into release/1.0 Jul 17, 2026
19 checks passed
@jeremi
jeremi deleted the agent/redesign-registry-notary-tutorials branch July 17, 2026 22:57

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2ce0837633

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

"/v1/datasets/benefits_casework/entities/person_identity/records?id=per-2001",
403,
&[
bearer_header(secrets.value("IDENTITY_READER_RAW")),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Gate identity smoke checks on generated identity support

For projects initialized before this change, registryctl.yaml still has schema registryctl/v1 but secrets/local.env has no IDENTITY_READER_RAW and relay/config.yaml has no person_identity entity. registryctl smoke now unconditionally sends this identity-reader request, so upgrading the CLI makes otherwise valid older sample projects fail smoke instead of either skipping these probes or requiring a manifest/schema migration.

Useful? React with 👍 / 👎.

"scope": "registry:consult:registration-verification",
"iat": now,
"nbf": now.saturating_sub(5),
"exp": now.saturating_add(3600),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep the workload token valid for long-lived local stacks

When the local Notary stack is left running for more than an hour, this static Relay workload JWT expires. registryctl only rewrites secrets/notary-relay.jwt during start/restart preparation, while Notary reloads that token file for each Relay operation and Relay validates exp, so evaluations that need the consultation Relay start failing until the user restarts the stack. For the tutorial stack, mint for the expected local lifetime or add a refresh path that updates the running mount.

Useful? React with 👍 / 👎.

project.project.products.push("registry-notary".to_string());
project.runtime.notary_image = Some(image_lock.notary_image().to_string());
project.runtime.notary_base_url = Some(NOTARY_BASE_URL.to_string());
project.notary = Some(ProjectNotary {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include Notary in doctor reports after adding it

Once this writes a Notary section into registryctl.yaml, registryctl doctor --format json still only builds product doctor invocations for project.relay in product_doctor_invocations, so an add-on project can report OK while the generated Notary config or mounted Relay token is missing/invalid. Add a registry-notary doctor invocation for project.notary or surface that the Notary add-on is not covered.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant