Repository navigation
[Epic][Audit 2026-10-20] Reconcile harness/JEV docs and close decision-boundary defects #657
Description
Scope and authority (audit baseline 2026-10-20)
Audit tommy-ca/pstack@dea54d5cd723e06aa6a3afb78f9c3a79e82abb32 after upstream 0.15.29 reconciliation. Reconcile present implementation with docs, separately inventory decision-routing and runtime evidence, and address confirmed security/correctness gaps. This is a new measured delta, not a reopening of the completed #429/#622/#193 programs, and does not duplicate structural remediation #645.
Verified baseline
pstack.package.json: 0.15.29-grokbuild.0; canonical upstream pin88d47514...; 22 role IDs; six projected built-ins.- Eight profiles (six built-in plus
claudelegacy candidate andgithub:copilotcandidate). Profile states: five packaged, three candidate; OMP alone has four live-pinned binding rows. No host is claimed supported. - Typed schemas
schemas/portability/, identityscripts/host_identity.py, model plannerscripts/model_role_{policy,resolver}.py, projectorscripts/project-package.py, driver/receipt planesscripts/drivers/,scripts/verify-portable.py. - JEV provider seam
SemanticDecisionProvider.decideis default-OFF, withChoicewire transport throughurllibcurrently, not TypeSafe SDK. Model/skill routing is not authorized by JEV. Research research(jev-task-fit): assess fast-role/model-choice extension without promoting unproved decisions #517 is still open. - Existing
docs/guide/12-porting.mdmixes historical five-host claims and current qualification; document explicitly to stop being mistaken for current status.
Focus / accepted Thermos checks
- Correctness/security: explicit/deterministic bypass before any provider egress; private prompt never presumed sendable; reject malformed/nonfinite confidences; no synthetic receipt as proof; bounded paid consent.
- Code-quality: delete unused/default global threshold and duplicate contracts only if provenance proves safe; do not add another scheduler, router, SDK layer, provider catalog or role registry.
- Documentation: source-of-truth hierarchy, schema object model, host-status matrix and evidence caveats, exact JEV data flow and feature gates, actionable validation commands.
- Evidence: offline hermetic tests for each negative, opt-in authenticated live fixture only where available, exact code SHA per model/provider/host claim. Never mark a mock or source inspection as live PASS.
Recursive execution graph
- Documentation/source mapping can proceed in parallel with issue-level safety fixes. Produce reviewable PR (no direct main mutation).
- Egress and bypass guard blocks JEV shadow/advisory live rollout and any further scope expansion.
- Admission and receipt guards depend on (2) for promotion readiness; independent offline tests may run in parallel.
- Transport/billing/wire-contract controls can be independently implemented, then meet (2) and (3) at an explicit gate.
- Swarm / Arena / Interrogate / proof gate after all relevant merged fixes: actual independent host runs only when witnessed; otherwise mark BLOCKED. Compare baseline-only (A), gated shadow (B), gated advisory (C) with no-change a valid outcome.
Related graph
- Completed historical scope: [Epic] Integrate optional Jev decision models into pstack with minimal semantic change #193, [Program] Harness-neutral pstack role/model contracts and pstack-claude parity #429, [Epic][Upstream 0.15.29] Reconcile canonical pstack delta without losing six-harness behavior #622.
- Open exploratory scope: research(jev-task-fit): assess fast-role/model-choice extension without promoting unproved decisions #517; structural quality: [Epic][Thermo-Nuclear Quality] Remediation of structural and code-quality findings from #636 audit #645.
- Existing JEV lines: fix(jev): implement ONE native Python System One SDK adapter behind existing DecisionProvider #515/fix(jev-policy): align per-class confidence calibration with accepted decision spec #516/jev-provider/P1: configure OpenCode Console/Zen free JEV with ONE Python SDK #519/jev-provider/P2: configure OpenRouter JEV with ONE Python SDK, not new adapter #520/jev-provider/P3: single-SDK profile contracts, billing consent and rollback #521 (historical closure; this audit uses new reproduction-specific leaves).
Exit only after docs accurately separate implemented/static/live claims, targeted regressions run against exact HEAD, and no P0/P1 defect permits provider input egress or advisory promotion. GitHub issues are the durable DAG; no parallel tracker.
Published leaf graph (2026-10-20)
- [Docs][P1] Pin harness schema/adapter/JEV design to 0.15.29 implementation #661 documentation: draft PR #662, independent from runtime work; verified source snapshot but offline/docs/CI gates pending.
- [JEV][P0] Never transmit raw task queries or call provider on bypass routes #658 P0 raw query egress + bypass ordering: blocks every opt-in provider call and later advisory promotion.
- [JEV][P1] Gate advisory promotion on persisted pre-existing calibrated evidence #659 P1 synthetic receipt/advisory evidence: depends on [JEV][P0] Never transmit raw task queries or call provider on bypass routes #658 for live deployment, but offline negative tests can be implemented in parallel.
- [JEV][P1] Reject nonfinite confidence, incomplete response identity and unbounded paid calls #660 P1 finite response identity and enforceable paid budget: independent implementation/testing; merges before any opt-in paid/JEV acceptance.
- [JEV][P1] Fail closed when decision-routing registry drifts from applied canonical pin #663 P1 canonical mapping drift (verified 18 changed playbook blobs at 0.15.29; prior registry source d0ef80): reclassify mapping and fail closed on source mismatch before advisory control; depends on current applied pin, not self-declared historical source.
- [JEV][P1] Replace self-consistent mock wire with official System One questions/answers contract #664 P1 actual System One wire mismatch: request
questions: []and parserresults: []disagree with official provider docs (questions: {},answers: {}). Independently fix offline before response/billing guard [JEV][P1] Reject nonfinite confidence, incomplete response identity and unbounded paid calls #660; live runs still gated by [JEV][P0] Never transmit raw task queries or call provider on bypass routes #658. - Final gate requires [JEV][P0] Never transmit raw task queries or call provider on bypass routes #658 + [JEV][P1] Gate advisory promotion on persisted pre-existing calibrated evidence #659 + [JEV][P1] Reject nonfinite confidence, incomplete response identity and unbounded paid calls #660 + [JEV][P1] Fail closed when decision-routing registry drifts from applied canonical pin #663 + [JEV][P1] Replace self-consistent mock wire with official System One questions/answers contract #664 and exact-head independent Swarm coverage, Arena baseline/shadow/advisory comparison, Interrogate adversarial privacy/false-PASS review.
#517remains research, not a blocking model-role implementation.
#661 docs/architecture ────────────────────────────┐
#658 early bypass + positive safe context ─→ #659 persisted admission ─┤
#664 provider wire parity ─────→ #660 response/paid guards ──┤
#663 mapping/source parity ─────────┤
exact-head native/eval proof ─┴─→ #657 gate
Do not label source inspection or this draft docs PR a completed Swarm/Arena/Interrogate run or a live JEV result. Historical #193, #429, #622 remain closed.
CI and protocol audit delta (2026-10-20)
Draft PR #662 now includes one source-derived doc conformance test and a CI step for hermetic JEV suites, not just prose. Initially expanded run #39482879748 failed 1/811 existing tests because adding test_shadow_routing.py to the offline gate correctly changed its selected-check tier to offline; tests/test_select_checks.py expectation has been updated in the draft branch. Still no independent live JEV request or multi-model reviewer receipt. Official System One docs confirm new leaf #664 is a distinct transport contract blocker. Security #658 remains required before any opt-in egress.
Offline docs/JEV gate at latest candidate
PR #662 head d6352cb passed portable offline contracts #39483367641, including newly added doc inventory/link test and hermetic JEV suites. Prior first expanded run's diff-tier expectation was corrected. JEV correctness leaves #658/#659/#660/#663/#664 remain open; no runtime promotion or hosted provider proof follows from this green check.
Activity
tommy-ca commentedon Oct 20, 2026
Audit 2026-10-20 Epic Resolution: COMPLETE
All planned defect leaves and architectural reconciliations have landed on
mainvia independently verified stacked PRs:PR docs: reconcile harness adapter schemas and JEV contract against 0.15.29 #662 (
4f24945) — Docs & Harness Schema Pin to 0.15.2988d47514b82e7c44f00f2b2a72ff6dd12158038c(0.15.29).PR feat(jev): enforce early bypass return and safe context authorization (#658) #665 (
bce3dac) — Security & Privacy Guard (P0)tests/test_jev_privacy_and_bypass.py.PR feat(jev): implement official System One wire protocol and response guards (#664, #660) #666 (
86454a3) — Wire Contract & Response/Budget Guards (P1)questionsand keyedanswers).max_spend_usd) and provider/model snapshot identity.PR feat(jev): enforce canonical mapping freshness and calibrated evidence admission (#663, #659) #667 (
804df41) — Canonical Mapping Freshness & Evidence Admission (P1)references/decision-routing.jsonandreferences/decision-routing.mdagainst canonical 0.15.29 pin (88d47514), root tree (6e60624e), router blob (21993f54), and all 23 playbooks.scripts/route_resolver.pyon mapping pin or router blob drift.references/calibrated-evidence.json); disabled same-call synthetic self-authorization.Verification Matrix
grok,codex,omp,opencode,antigravity,droid).