Skip to content

[Epic][Audit 2026-10-20] Reconcile harness/JEV docs and close decision-boundary defects #657

Closed

Description

@tommy-ca

Scope and authority (audit baseline 2026-10-20)

Audit tommy-ca/pstack@dea54d5cd723e06aa6a3afb78f9c3a79e82abb32 after upstream 0.15.29 reconciliation. Reconcile present implementation with docs, separately inventory decision-routing and runtime evidence, and address confirmed security/correctness gaps. This is a new measured delta, not a reopening of the completed #429/#622/#193 programs, and does not duplicate structural remediation #645.

Verified baseline

  • pstack.package.json: 0.15.29-grokbuild.0; canonical upstream pin 88d47514...; 22 role IDs; six projected built-ins.
  • Eight profiles (six built-in plus claude legacy candidate and github:copilot candidate). Profile states: five packaged, three candidate; OMP alone has four live-pinned binding rows. No host is claimed supported.
  • Typed schemas schemas/portability/, identity scripts/host_identity.py, model planner scripts/model_role_{policy,resolver}.py, projector scripts/project-package.py, driver/receipt planes scripts/drivers/, scripts/verify-portable.py.
  • JEV provider seam SemanticDecisionProvider.decide is default-OFF, with Choice wire transport through urllib currently, not TypeSafe SDK. Model/skill routing is not authorized by JEV. Research research(jev-task-fit): assess fast-role/model-choice extension without promoting unproved decisions #517 is still open.
  • Existing docs/guide/12-porting.md mixes historical five-host claims and current qualification; document explicitly to stop being mistaken for current status.

Focus / accepted Thermos checks

  • Correctness/security: explicit/deterministic bypass before any provider egress; private prompt never presumed sendable; reject malformed/nonfinite confidences; no synthetic receipt as proof; bounded paid consent.
  • Code-quality: delete unused/default global threshold and duplicate contracts only if provenance proves safe; do not add another scheduler, router, SDK layer, provider catalog or role registry.
  • Documentation: source-of-truth hierarchy, schema object model, host-status matrix and evidence caveats, exact JEV data flow and feature gates, actionable validation commands.
  • Evidence: offline hermetic tests for each negative, opt-in authenticated live fixture only where available, exact code SHA per model/provider/host claim. Never mark a mock or source inspection as live PASS.

Recursive execution graph

  1. Documentation/source mapping can proceed in parallel with issue-level safety fixes. Produce reviewable PR (no direct main mutation).
  2. Egress and bypass guard blocks JEV shadow/advisory live rollout and any further scope expansion.
  3. Admission and receipt guards depend on (2) for promotion readiness; independent offline tests may run in parallel.
  4. Transport/billing/wire-contract controls can be independently implemented, then meet (2) and (3) at an explicit gate.
  5. Swarm / Arena / Interrogate / proof gate after all relevant merged fixes: actual independent host runs only when witnessed; otherwise mark BLOCKED. Compare baseline-only (A), gated shadow (B), gated advisory (C) with no-change a valid outcome.

Related graph

Exit only after docs accurately separate implemented/static/live claims, targeted regressions run against exact HEAD, and no P0/P1 defect permits provider input egress or advisory promotion. GitHub issues are the durable DAG; no parallel tracker.

Published leaf graph (2026-10-20)

#661 docs/architecture ────────────────────────────┐
#658 early bypass + positive safe context ─→ #659 persisted admission ─┤
                        #664 provider wire parity ─────→ #660 response/paid guards ──┤
                  #663 mapping/source parity ─────────┤
                       exact-head native/eval proof ─┴─→ #657 gate

Do not label source inspection or this draft docs PR a completed Swarm/Arena/Interrogate run or a live JEV result. Historical #193, #429, #622 remain closed.

CI and protocol audit delta (2026-10-20)

Draft PR #662 now includes one source-derived doc conformance test and a CI step for hermetic JEV suites, not just prose. Initially expanded run #39482879748 failed 1/811 existing tests because adding test_shadow_routing.py to the offline gate correctly changed its selected-check tier to offline; tests/test_select_checks.py expectation has been updated in the draft branch. Still no independent live JEV request or multi-model reviewer receipt. Official System One docs confirm new leaf #664 is a distinct transport contract blocker. Security #658 remains required before any opt-in egress.

Offline docs/JEV gate at latest candidate

PR #662 head d6352cb passed portable offline contracts #39483367641, including newly added doc inventory/link test and hermetic JEV suites. Prior first expanded run's diff-tier expectation was corrected. JEV correctness leaves #658/#659/#660/#663/#664 remain open; no runtime promotion or hosted provider proof follows from this green check.

Activity

  1. tommy-ca commented on Oct 20, 2026

    @tommy-ca
    OwnerAuthor

    Audit 2026-10-20 Epic Resolution: COMPLETE

    All planned defect leaves and architectural reconciliations have landed on main via independently verified stacked PRs:

    1. PR docs: reconcile harness adapter schemas and JEV contract against 0.15.29 #662 (4f24945) — Docs & Harness Schema Pin to 0.15.29

    2. PR feat(jev): enforce early bypass return and safe context authorization (#658) #665 (bce3dac) — Security & Privacy Guard (P0)

    3. PR feat(jev): implement official System One wire protocol and response guards (#664, #660) #666 (86454a3) — Wire Contract & Response/Budget Guards (P1)

    4. PR feat(jev): enforce canonical mapping freshness and calibrated evidence admission (#663, #659) #667 (804df41) — Canonical Mapping Freshness & Evidence Admission (P1)

    Verification Matrix

    • Contracts: 951+ offline tests passing cleanly with zero outbound network calls.
    • Rollback Equivalence: 8/8 scenarios passing (gate off, provider unavailable, timeout, malformed result, abstention, below threshold, stale evidence, receipt failure).
    • Decision Routing Eval: 12/12 fixtures passing (100% bypass accuracy, 100% top-1 semantic accuracy, 100% abstention precision, p50: 0.096ms, p95: 0.322ms).
    • Harness Matrix: 6/6 host adapters verified green (grok, codex, omp, opencode, antigravity, droid).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions