Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -183,3 +183,23 @@
and memory termination are now non-retryable limit failures; executor loss
receives one bounded fresh-authority/materialization retry, then a stable
extraction-failed outcome with only redacted diagnostics.

## WS-ART-001-03B2

- Implemented the hidden fixed-reader materialization and syntactic
classification slice after merged 03B1.
- Initial internal review found active namespace drift, a broad scratch
callback, unsafe external-relationship matching, and missing incident,
boundary, cancellation, image, and downgrade tests.
- Repairs share the canonical replica namespace/store validator, use the
canonical materializer facade and typed scratch inspector, parse bounded
relationship XML fail closed, and add the complete focused proof set.
- Final architecture, security, QA, product/ops, CI-integrity, docs,
reuse/dedup, test-delta, and senior-engineering reviews pass. Exact hosted
PR-head checks remain the publication gate.
- CodeRabbit's six low-severity comments were repaired with naming and wording
alignment, protocol variance, bounded nested buffering, JPEG marker handling,
and immutable-conflict proof. Its generic docstring warning is superseded by
the passing repository-owned hosted docstring gate.
- Senior-engineering, security, and QA repair-delta re-reviews pass with no
blockers before the repaired PR head is published.
Original file line number Diff line number Diff line change
Expand Up @@ -47,19 +47,22 @@ scratch without extracting semantic content.
either image dimension; every exact-boundary case succeeds, every one-over
case returns `limit_exceeded`, and no partial classification survives;
- cleanup occurs on success, denial, mismatch, cancellation, and timeout;
- migration `0040_guide_materialization` preserves exact-binding classification
and incident custody, refuses populated downgrade, and introduces no new
Operator or generic artifact-read route;
- changed subsystems remain at least 90% covered and repository coverage stays
at least 78%.

## Verification

```bash
(cd backend && .venv/bin/python -m ruff check app tests scripts)
(cd backend && WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_guide_artifacts.py tests/test_artifact_scratch_manager.py tests/test_guide_format_detection.py -q --cov=app --cov-report=term-missing --cov-fail-under=0)
(cd backend && WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_guide_bindings.py tests/test_artifact_preparation.py tests/test_guide_formats.py -q --cov=app --cov-report=term-missing --cov-fail-under=0)
(cd backend && .venv/bin/coverage report --precision=2 --fail-under=78)
(cd backend && .venv/bin/coverage report --include='app/modules/artifacts/*' --precision=2 --fail-under=90)
python3 scripts/check_stale_artifact_contracts.py
python3 scripts/check_markdown_links.py
python3 scripts/test_agent_gates.py
PYTHONPATH=. python3 scripts/test_lightweight_agent_gates.py
git diff --check
```

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# WS-ART-001-03B2 External Review Response

## Comments addressed

- Aligned migration filename and documented revision identity.
- Corrected the canonical materialization-port description.
- Made the return-only prepared-inspector type covariant.
- Added deterministic JPEG standalone-marker handling and proof.
- Added a fixed per-entry nested-archive byte ceiling and exact boundary proof
before any nested member is buffered.
- Added database-backed proof that conflicting immutable classification evidence
is rejected without replacing the stored row.

## Comments deferred

- None.

## Human decisions needed

- None. All code comments were low-severity and in scope.
- CodeRabbit's generic docstring-coverage warning is not actionable: the hosted
repository docstring gate passed on the reviewed head and remains the
canonical configured check.

## Commands rerun

- Focused Ruff over changed backend files.
- Focused guide-format and preparation tests.
- Focused database-backed guide-materialization tests.
- Repository stale-contract, Markdown-link, lightweight-agent-gate, and diff
checks.
- Exact PR head `1381d371`: Backend passed in 12m47s and Agent Gates passed in
19s. A later review-request Agent Gates run also passed in 21s.

## Remaining risks

- Format classification remains syntactic by design; semantic extraction belongs
to WS-ART-001-03B3A.
- Live guide-reader authorization remains planned and unavailable until
AUTH WS-XINT-002-04B.
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Internal Review Evidence: WS-ART-001-03B2

## Scope

Hidden fixed-reader materialization of exact verified guide-source bytes,
bounded syntactic classification, and ART-owned custody incidents. No AUTH
activation, extraction, agent invocation, Celery continuation, generic
download, or legacy cutover is included.

## Deterministic Evidence

- Ruff passed for `app`, `tests`, `scripts`, and migration `0040`.
- Focused format, architecture, and scratch inspection tests passed.
- The isolated PostgreSQL runner migrated through `0040` and passed all 13
materialization-selected guide-binding tests, including denial before I/O,
namespace drift, full rehash, replay, changed/truncated/stale incidents,
cancellation, and timeout cleanup.
- Canonical lane collection accepted `test_guide_formats.py` in
`shared_foundations`; the CI-integrity reviewer collected 2,316 nodes.
- Stale artifact-contract, changed Markdown-link, lightweight agent-gate, and
`git diff --check` checks passed.
- Hosted Backend and Agent Gates remain required on the exact committed PR
head; no local full-suite run was used.

## Findings And Repairs

- Architecture: shared the existing replica/namespace/store validator with the
canonical `ArtifactMaterializationService`, and narrowed scratch inspection
to a typed ART-owned inspector.
- Security: replaced substring relationship checks with bounded XML parsing;
malformed XML, DOCTYPE, and whitespace-equivalent external relationships
now fail closed.
- QA/test: added changed and stale-generation incidents, exact cross-resource
denial, cancellation/timeout cleanup, fixed-limit arithmetic, ambiguous and
malformed containers, video signatures, JPEG, WebP variants, and both
classification-only and incident-only populated downgrade refusal.
- Senior engineering/reuse: added exact verification-job replica/terminal
lineage, inspection deadline mapping, model/migration constraint-name parity,
and shared namespace validation.
- Docs/product/ops: documented migration `0040`, evidence custody, populated
downgrade refusal, and the no-new-Operator-route boundary. Artifact incidents
remain separate from guide insufficiency.

## Final Reviewer Results

- Architecture: PASS WITH LOW RISKS.
- Security/auth: PASS.
- QA/test: PASS WITH LOW RISKS.
- Product/ops: PASS.
- CI integrity: PASS WITH LOW RISKS.
- Docs: PASS.
- Reuse/dedup: PASS WITH LOW RISKS.
- Test delta: PASS WITH LOW RISKS.
- Senior engineering: final confirmation recorded before PR publication.

Residual low risks are limited to the canonical materializer facade currently
implementing only the 03B2 guide-read slice, a structurally typed scratch
inspector, and repeated focused-test scratch setup. Later ART chunks must extend
the same facade and custody boundaries.
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
# Workstream PR Trust Bundle

## Chunk

`WS-ART-001-03B2` — Guide Materialization And Classification (L1)

## Goal

Read exact verified guide-source bytes through the fixed guide-reader boundary,
recompute their identity in bounded private scratch, and persist only immutable
syntactic classification or bounded ART incident evidence.

## Intent And Planning Context

- Intent: Project Manager guide items may use the approved document, table,
text, JSON, or image formats; they are not contributor submission ZIPs.
- Chunk contract: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B2-guide-materialization-classification.md`

## What Changed

- Added exact fixed-reader materialization with pre/post lineage validation.
- Added provider-neutral streaming, complete rehash/size verification, and typed
bounded scratch inspection.
- Added deterministic format classification, immutable evidence, bounded
incidents, migration `0040`, tests, documentation, and review evidence.

## Why It Changed

Workstream must prove the exact verified guide bytes before later extraction or
sufficiency processing; object-store presence or caller metadata is insufficient.

## Design Chosen

Transaction A locks exact lineage and consumes prepared authority before the
provider read. The canonical store streams into canonical scratch. Transaction B
relocks the same facts before persisting one immutable result.

## Alternatives Rejected

- Direct S3/MinIO reads, arbitrary temp files, generic download authority,
serialized prepared handles, caller excerpts, and parsing during upload.

## Scope Control

### Allowed Files Changed

- ART interfaces, models, migration, materialization/classification services,
focused tests, ART specification, and this chunk's evidence.

### Files Outside Stated Scope

- None.

## Product Behavior

- [x] Product behavior changed and is explained here: hidden fixed-service guide
materialization/classification exists, but `artifact.guide_source.read`
remains planned and unavailable until AUTH-04B.

## Evidence

### Commands Run

```bash
uv run --no-sync ruff check app tests scripts alembic/versions/0040_guide_materialization.py
uv run --no-sync pytest -q tests/test_guide_formats.py tests/test_artifact_preparation.py
WORKSTREAM_TEST_ADMIN_DATABASE_URL=<local-admin> uv run --no-sync python scripts/run_isolated_tests.py <focused materialization tests>
python3 scripts/check_stale_artifact_contracts.py
python3 scripts/check_markdown_links.py
PYTHONPATH=. python3 scripts/test_lightweight_agent_gates.py
git diff --check
```

### Result Summary

```text
Focused guide/preparation tests: 35 passed
Database-backed materialization selection: 13 passed before review repair
Immutable-conflict database proof: 1 passed after repair
Backend exact head 1381d371: passed in 12m47s
Agent Gates exact head 1381d371: passed in 19s
CodeRabbit final-head review: completed; two evidence-wording findings repaired
```

## Acceptance Criteria Proof

- [x] Exact verified lineage and active namespace are checked before provider I/O.
- [x] Complete digest and byte count are recomputed on every read.
- [x] Stale/replaced/missing/mismatched content fails closed with bounded evidence.
- [x] Classification is syntactic, bounded, immutable, and provider-neutral.
- [x] Live AUTH actions remain planned and unavailable.

## Test Delta

### Tests Added

- Format signatures, OOXML/container safety, exact limits, image variants,
typed inspection, materialization lineage, incidents, replay, cancellation,
timeout, conflict, migration, and architecture fences.

### Tests Modified

- Alembic head/fingerprint, artifact architecture, preparation, and guide-binding
integration coverage.

### Tests Removed Or Skipped

- None.

## Internal Reviewer Results

Reviewed code SHA: `1381d371`

Reviewed at: 2026-07-29

Reviewer run IDs: recorded in the paired internal-review evidence.

| Reviewer | Result | Blocking Findings | Notes |
|---|---:|---|---|
| Senior engineering | PASS WITH LOW RISKS | None | Repair delta also passed |
| QA/test | PASS WITH LOW RISKS | None | Repair delta also passed |
| Security/auth | PASS | None | Repair delta also passed |
| Product/ops | PASS | None | Exact custody preserved |
| Architecture | PASS WITH LOW RISKS | None | Canonical facade/store/scratch reused |
| CI integrity | PASS WITH LOW RISKS | None | No gate weakening |
| Docs | PASS | None | Specification and evidence aligned |
| Reuse/dedup | PASS WITH LOW RISKS | None | No parallel boundary introduced |
| Test delta | PASS WITH LOW RISKS | None | No removed or weakened tests |

## External Review

| Source | Status | Notes |
|---|---:|---|
| CodeRabbit | PASS AFTER FIXES | Six initial findings and two final evidence findings repaired |
| GitHub checks | PASS | Backend and Agent Gates green on `1381d371` |

## CI And Gate Integrity

- [x] No workflow weakening.
- [x] No lint/test/docstring gate weakening.
- [x] No coverage threshold weakening.
- [x] No package script weakening.
- [x] No unpinned new GitHub Action.
- [x] Checkout credential persistence unchanged.

## Remaining Risks

Classification is syntactic by design. Semantic extraction belongs to 03B3A;
live fixed-reader authorization belongs to AUTH-04B.

## Follow-Up Work

03B3A adds extraction, 03B3B adds durable continuation, 03B4 finalizes the
hidden manifest, AUTH-04B activates exact fixed-service actions, and 03C later
performs the separate legacy cutover.

## Human Review Focus

Please inspect active namespace fencing, pre/post generation locks, complete
rehash comparison, nested-archive bounds, incident privacy, and deny-only AUTH.

## Human Merge Ownership

- [x] I can explain what changed.
- [x] I can explain why it changed.
- [x] I know what could break.
- [ ] I accept the remaining risks.
- [ ] The user explicitly approved this specific PR for merge.
Loading
Loading