From eaaf4824e6553c22ccaf19b2b4c4faedda60c4cf Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Thu, 30 Jul 2026 15:49:36 +0100 Subject: [PATCH 1/2] docs(artifacts): split complex format extractor work --- .../AUTH_HANDOFF.md | 2 +- .../CHUNK_MAP.md | 16 +++- .../DECISIONS.md | 20 +++-- .../DISCOVERY.md | 2 +- .../PLAN.md | 17 ++-- .../RISKS.md | 1 + .../STATUS.md | 6 ++ ...3B-guide-source-binding-materialization.md | 3 +- ...ART-001-03B3B-complex-format-extractors.md | 71 ---------------- ...T-001-03B3B1-parser-dependency-approval.md | 84 +++++++++++++++++++ .../chunks/WS-ART-001-03B3B2-pdf-extractor.md | 82 ++++++++++++++++++ ...RT-001-03B3B3A-ooxml-container-security.md | 70 ++++++++++++++++ .../WS-ART-001-03B3B3B-docx-extractor.md | 68 +++++++++++++++ .../WS-ART-001-03B3B3C-pptx-extractor.md | 68 +++++++++++++++ .../WS-ART-001-03B3B3D-xlsx-extractor.md | 68 +++++++++++++++ ...RT-001-03B3B4-image-metadata-extractors.md | 83 ++++++++++++++++++ ...001-03B4-guide-sufficiency-continuation.md | 7 +- 17 files changed, 574 insertions(+), 94 deletions(-) delete mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B-complex-format-extractors.md create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B1-parser-dependency-approval.md create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B2-pdf-extractor.md create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3A-ooxml-container-security.md create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3B-docx-extractor.md create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3C-pptx-extractor.md create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3D-xlsx-extractor.md create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B4-image-metadata-extractors.md diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md index 0608499f..74932f68 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md @@ -17,7 +17,7 @@ activation custody, and availability. exact resource/guard/surface manifest. 3. AUTH activates only that exact action through a separately reviewed AUTH contract after consuming ART evidence. -4. ART-03B1, 03B2, 03B3A, 03B3B, and 03B4 implement hidden authoritative binding, exact setup +4. ART-03B1, 03B2, 03B3A, 03B3B1-03B3B4, and 03B4 implement hidden authoritative binding, exact setup generation, verified materialization, bounded extraction, and Celery sufficiency continuation. `artifact.guide_source.binding.create` maps to fixed permission `artifact.binding.create`; `artifact.guide_source.read` diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md index d9f253e9..e464f57f 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md @@ -21,8 +21,14 @@ Each chunk is one PR. No later chunk starts automatically. | `WS-ART-001-03B1` | Add authoritative verified guide-source bindings and an exact setup-generation fence while fixed-service binding remains unavailable. | L1 | Proposed after 03A and AUTH-04A | | `WS-ART-001-03B2` | Add fixed-reader verified materialization, artifact incidents, and canonical format detection in bounded ART scratch. | L1 | Proposed after 03B1 | | `WS-ART-001-03B3A` | Add the isolated extraction framework, canonical records, and text/Markdown/JSON/CSV extractors. | L1 | Proposed after 03B2 | -| `WS-ART-001-03B3B` | Add explicitly approved PDF/OOXML/image-metadata extractors on the proven framework. | L1 | Proposed after 03B3A and dependency approval | -| `WS-ART-001-03B4` | Feed only complete same-generation canonical extracted material into the existing Celery sufficiency pipeline. | L1 | Proposed after 03B3B | +| `WS-ART-001-03B3B1` | Approve the exact pinned complex-parser dependency allowlist and deterministic CI gate without installing packages or changing runtime code. | L1 | Proposed after 03B3A | +| `WS-ART-001-03B3B2` | Install only the approved PDF dependency and add bounded PDF text extraction. | L1 | Proposed after 03B3B1 approval | +| `WS-ART-001-03B3B3A` | Add shared bounded OOXML container/security capabilities without document extraction. | L1 | Proposed after 03B3B1 approval | +| `WS-ART-001-03B3B3B` | Add bounded DOCX extraction on the approved OOXML capability. | L1 | Proposed after 03B3B3A | +| `WS-ART-001-03B3B3C` | Add bounded PPTX extraction on the approved OOXML capability. | L1 | Proposed after 03B3B3A | +| `WS-ART-001-03B3B3D` | Add bounded XLSX extraction on the approved OOXML capability. | L1 | Proposed after 03B3B3A | +| `WS-ART-001-03B3B4` | Install only the approved image dependency and add PNG/JPEG/WebP structural metadata extraction. | L1 | Proposed after 03B3B1 approval | +| `WS-ART-001-03B4` | Feed only complete same-generation canonical extracted material into the existing Celery sufficiency pipeline. | L1 | Proposed after 03B3B2, 03B3B3B-03B3B3D, and 03B3B4 | | `WS-ART-001-03C` | Remove legacy guide-source identity and add exact same-generation setup continuation. | L1 | Proposed after 03B1-03B4 and AUTH-04B | | `WS-ART-001-04A` | Accept one outer ZIP in bounded scratch, safely inspect its tree, normalize executable intent, produce canonical identities, and reject unchanged work before provider I/O. | L1 | Proposed after 03C and AUTH planned action registration | | `WS-ART-001-04B` | Run mandatory platform and locked Project Guide pre-submit checks against the same scratch-bound tree and executable semantics without durable storage. | L1 | Proposed after 04A | @@ -50,7 +56,11 @@ OBJECT-STORAGE-AMENDMENT -> 03B1 authoritative guide binding/setup generation -> 03B2 verified materialization/format detection -> 03B3A isolated extraction framework/text formats --> 03B3B approved complex document/image-metadata extractors +-> 03B3B1 pinned dependency approval and CI gate +-> 03B3B2 approved PDF extractor (independent after 03B3B1) +-> 03B3B3A shared OOXML security capability + -> 03B3B3B DOCX / 03B3B3C PPTX / 03B3B3D XLSX +-> 03B3B4 approved image metadata (independent after 03B3B1) -> 03B4 same-generation Celery sufficiency integration -> AUTH activation for exact 03B actions -> 03C guide-source clean cut/continuation diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DECISIONS.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DECISIONS.md index 37b57fb6..9b0be1c9 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DECISIONS.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DECISIONS.md @@ -400,7 +400,11 @@ under fixed input, output, container, time, memory, and document limits using only scratch-manager paths. Parser crash, malformed input, macros, external relationships, embedded executables, cancellation, timeout, and executor loss have bounded outcomes and cleanup. Production parser dependencies require -explicit human approval before implementation. +explicit human approval before implementation. Complex-format delivery is +split after 03B3A: 03B3B1 records the exact pinned dependency decision and +deterministic gate without installing packages; 03B3B2 delivers PDF, 03B3B3A +owns shared OOXML security, 03B3B3B-D separately deliver DOCX/PPTX/XLSX, and +03B3B4 delivers image metadata. The v0.1 extraction-policy limits are fixed, not caller-selectable: @@ -420,14 +424,14 @@ The v0.1 extraction-policy limits are fixed, not caller-selectable: | decompressed container bytes | 128 MiB | 03B2 container inspector | `limit_exceeded` | | container nesting depth | 8 | 03B2 container inspector | `limit_exceeded` | | compression ratio | 100:1 | 03B2 container inspector | `limit_exceeded` | -| PDF pages | 500 | 03B3B PDF adapter | `limit_exceeded` | -| PPTX slides | 300 | 03B3B PPTX adapter | `limit_exceeded` | -| XLSX sheets | 100 | 03B3B XLSX adapter | `limit_exceeded` | -| table rows per item | 100,000 | 03B3A CSV / 03B3B XLSX adapter | `limit_exceeded` | -| table cells per item | 1,000,000 | 03B3A CSV / 03B3B XLSX adapter | `limit_exceeded` | +| PDF pages | 500 | 03B3B2 PDF adapter | `limit_exceeded` | +| PPTX slides | 300 | 03B3B3C PPTX adapter | `limit_exceeded` | +| XLSX sheets | 100 | 03B3B3D XLSX adapter | `limit_exceeded` | +| table rows per item | 100,000 | 03B3A CSV / 03B3B3D XLSX adapter | `limit_exceeded` | +| table cells per item | 1,000,000 | 03B3A CSV / 03B3B3D XLSX adapter | `limit_exceeded` | | characters per cell | 32,768 | CSV/XLSX adapter | `limit_exceeded` | -| image pixels | 40 megapixels | 03B2 header inspector and 03B3B adapter | `limit_exceeded` | -| image width or height | 16,384 pixels | 03B2 header inspector and 03B3B adapter | `limit_exceeded` | +| image pixels | 40 megapixels | 03B2 header inspector and 03B3B4 adapter | `limit_exceeded` | +| image width or height | 16,384 pixels | 03B2 header inspector and 03B3B4 adapter | `limit_exceeded` | Exact-boundary, one-over-boundary, cleanup, cancellation, timeout, memory- termination, and executor-loss tests are mandatory. Executor loss leaves no diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DISCOVERY.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DISCOVERY.md index bbf6c187..3e4baacc 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DISCOVERY.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DISCOVERY.md @@ -154,7 +154,7 @@ bounded canonical extraction record and provenance in PostgreSQL; a future derived provider artifact requires a separately planned action and activation. No document-parsing production dependencies are installed. Any new production -dependency requires explicit human approval before 03B3B implementation. OOXML ZIP +dependency requires explicit human approval through 03B3B1 before format implementation. OOXML ZIP containers must pass one bounded classifier that distinguishes DOCX, PPTX, and XLSX by internal markers. Audio/video transcription and OCR are not required by current v0.1 setup and remain unsupported. PNG/JPEG/WebP classification and diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md index fc2eb1b7..e50c07c9 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md @@ -477,7 +477,7 @@ ART-03A upload remains a byte-custody request: authorize before intake, compute the server digest and size, admit once, store opaque bytes, and verify them. It does not parse content or wait for contributor submission checkers. -The expanded ART-03B work is delivered as five hidden PR-sized subchunks before +The expanded ART-03B work is delivered as eleven hidden PR-sized subchunks before AUTH-04B activates fixed-service binding/read: 1. `03B1` creates an authoritative one-item/one-content guide binding tied to @@ -490,9 +490,14 @@ AUTH-04B activates fixed-service binding/read: 3. `03B3A` installs the isolated no-network extraction framework, immutable content/usage provenance, and text, Markdown, JSON, and CSV extractors without new parser dependencies. -4. `03B3B` adds explicitly approved PDF, DOCX, PPTX, XLSX, and PNG/JPEG/WebP - metadata extractors on that proven framework. -5. `03B4` adds `setup_generation` to the identifier-only Celery payload, +4. `03B3B1` selects the exact pinned parser dependency allowlist and adds its + deterministic CI gate without changing packages or runtime code. +5. `03B3B2` installs only the approved PDF dependency and adds PDF extraction. +6. `03B3B3A` adds shared bounded OOXML container and security capabilities. +7. `03B3B3B`, `03B3B3C`, and `03B3B3D` separately add DOCX, PPTX, and XLSX. +8. `03B3B4` installs only the approved image dependency and adds PNG, JPEG, and + WebP structural metadata extraction without OCR. +9. `03B4` adds `setup_generation` to the identifier-only Celery payload, reloads current state, assembles all required canonical extracted sources, invokes sufficiency, and persists exact provenance. @@ -552,9 +557,9 @@ every terminal current-generation failure above. API, Operator, and Project Manager projections use only the exact redacted mapping and remediation table. None is a guide-insufficiency decision. -Any production parser dependency in 03B3B requires explicit human approval. Candidate +Any production parser dependency requires explicit human approval through 03B3B1. Candidate libraries must undergo current security, maintenance, license, transitive- -dependency, malformed-input, and cancellation review before `03B3B` may add +dependency, malformed-input, and cancellation review before a later format chunk may add them; no runtime plugin discovery or parser fallback is permitted. 1. Guide-source delivery is split into hidden byte ingest, AUTH activation, diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md index 963504c4..f309bcfd 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md @@ -55,6 +55,7 @@ | Caller metadata or excerpt reaches the sufficiency agent instead of verified bytes | Critical | Resolve exact ART bindings, fully rehash provider reads, extract under a versioned policy, and assemble agent input only from same-generation canonical extraction records. | | DOCX/PPTX/XLSX is confused with an arbitrary ZIP | Critical | Use one bounded container classifier with required internal-marker checks; ambiguous or ordinary ZIP is unsupported and never sent raw to an agent. | | Malicious document exploits a parser or extraction runtime | Critical | Typed extractor registry, isolated no-network subprocess, fixed CPU/time/memory/output/container limits, external-reference and macro rejection, cancellation, and scratch-owned cleanup. | +| One PR introduces several unrelated parser dependency graphs and becomes unreviewable | Critical | 03B3B1 approves the exact pinned allowlist without installation; PDF, OOXML, and image metadata then land in separate implementation PRs with format-specific lock, security, fixture, and coverage evidence. | | Artifact corruption is reported as guide insufficiency | Critical | Digest/size mismatch, missing/truncated objects, and provider failures create ART incident outcomes and stop setup before the sufficiency agent. | | Extraction output silently changes across deployments | High | Persist content-derived format/extractor/policy identity and digest separately from exact binding/run/generation usage provenance; never rewrite prior output. | | Read authority is reused to write a derived provider artifact | Critical | v0.1 stores only bounded extraction records in PostgreSQL; any derived provider write requires a separate action and AUTH activation. | diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md index 020f7e6f..34130da6 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md @@ -61,6 +61,12 @@ same-generation sufficiency continuation are separate PR-sized contracts. `WS-ART-001-03B1` is the first proposed implementation successor; this planning change contains no implementation. +After 03B3A merged, the original complex-format chunk was found too broad for +one dependency and parser-security review. It is replaced by 03B3B1 dependency +approval, 03B3B2 PDF, 03B3B3A shared OOXML security, separate 03B3B3B-D +DOCX/PPTX/XLSX adapters, and 03B3B4 image metadata. No dependency is +installed until the human owner approves 03B3B1's exact pinned allowlist. + AUTH `WS-XINT-002-04B` follows the complete hidden split-03B series and activates only fixed-service binding and guide read. ART-03C then removes the legacy identity/excerpt path and makes the verified pipeline authoritative. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B-guide-source-binding-materialization.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B-guide-source-binding-materialization.md index f371677c..e2143ff9 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B-guide-source-binding-materialization.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B-guide-source-binding-materialization.md @@ -15,7 +15,8 @@ implementation contracts. - `WS-ART-001-03B1`: authoritative binding and setup generation; - `WS-ART-001-03B2`: verified materialization, incidents, and classification; - `WS-ART-001-03B3A`: isolated framework and low-complexity extraction; -- `WS-ART-001-03B3B`: approved complex document/image-metadata extractors; +- `WS-ART-001-03B3B1` through `03B3B4`: dependency approval followed by + separate PDF, OOXML, and image-metadata extractors; - `WS-ART-001-03B4`: same-generation Celery and sufficiency integration. Each subchunk is one PR. AUTH `WS-XINT-002-04B` activates fixed binding/read diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B-complex-format-extractors.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B-complex-format-extractors.md deleted file mode 100644 index 79a7de61..00000000 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B-complex-format-extractors.md +++ /dev/null @@ -1,71 +0,0 @@ -# Chunk Contract: WS-ART-001-03B3B - Complex Format Extractors - -Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 03B3A; dependency approval required - -## Goal - -Add approved PDF, DOCX, PPTX, XLSX, and PNG/JPEG/WebP metadata adapters on the -already-proven isolated framework. - -## Entry Gate - -The human owner explicitly approves an exact pinned dependency allowlist after -review of license, maintenance, advisories, transitive graph, malformed-input -history, and cancellation behavior. The PR includes the package/lock diff, -machine-readable allowlist, dependency-tree evidence, license evidence, and -security-review record. CI fails if a parser dependency is outside that list. - -## Allowed Files - -- one typed adapter per approved format and explicit registry entries; -- approved package/lock/allowlist and deterministic dependency gate; -- bounded fixtures and format-confusion, macro/external relationship, embedded - content, page/slide/sheet/cell/image, parser crash, timeout, and coverage tests; -- related docs/evidence. - -## Not Allowed - -- framework redesign, runtime plugins/fallbacks, OCR, audio/video, ordinary ZIP - semantics, raw pixels/base64 to agents, provider writes/access, legacy cutover, - or AUTH availability edits. - -## Acceptance Criteria - -- exact 03B2 classification provenance is required before parser execution; -- all parsers stay inside the 03B3A isolation/limit/cancellation boundary; -- PDF rejects more than 500 pages; PPTX rejects more than 300 slides; XLSX - rejects more than 100 sheets, 100,000 rows, 1,000,000 cells, or 32,768 - characters in one cell; images reject more than 40 megapixels or 16,384 - pixels on either dimension; every breach records `limit_exceeded`; -- tests cover each exact boundary and one-over boundary, malformed containers, - parser crash, timeout, cancellation, cleanup, and executor loss without a - partial extraction or usage record; -- OOXML markers distinguish formats and reject macros, external relationships, - embedded executables, bombs, and ambiguity; -- PNG/JPEG/WebP output is metadata only and cannot satisfy required text; -- parser dependency allowlist and lock graph are deterministic and CI-enforced; -- changed subsystem coverage is at least 90%; repository coverage stays 78%. - -## Verification - -```bash -(cd backend && .venv/bin/python -m ruff check app tests scripts) -(cd backend && .venv/bin/python scripts/check_guide_extractor_dependencies.py) -(cd backend && WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_guide_format_detection.py tests/test_guide_extraction.py -q --cov=app --cov-report=term-missing --cov-fail-under=0) -(cd backend && .venv/bin/coverage report --precision=2 --fail-under=78) -(cd backend && .venv/bin/coverage report --include='app/modules/artifacts/*,app/core/config.py,app/interfaces/artifacts.py' --precision=2 --fail-under=90) -python3 scripts/check_stale_artifact_contracts.py -python3 scripts/check_markdown_links.py -python3 scripts/test_agent_gates.py -git diff --check -``` - -The exact PR head must pass hosted `Backend / test` and `Agent Gates / agent-gates`. -Every changed production module, including adapter registry, dependency gate, -configuration, and composition surfaces, must be included in a dedicated -retained 90% coverage report or an explicitly reviewed existing subsystem report. - -## Required Reviewers - -Senior engineering, architecture, QA/test, security/auth, product/ops, -reuse/dedup, CI integrity, test delta, and docs. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B1-parser-dependency-approval.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B1-parser-dependency-approval.md new file mode 100644 index 00000000..69648152 --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B1-parser-dependency-approval.md @@ -0,0 +1,84 @@ +# Chunk Contract: WS-ART-001-03B3B1 — Parser Dependency Approval + +## Parent initiative + +WS-ART-001 — Immutable Artifact Storage + +## Goal + +Produce the exact pinned PDF, OOXML, and image parser allowlist and deterministic +CI enforcement without installing packages or changing runtime behavior. + +## Why this chunk exists + +Untrusted document parsers are separate security and supply-chain boundaries. +Human approval must precede every package or lock change. +## Approved plan reference + +- PLAN: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md` +- CHUNK_MAP: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md` + +## Risk class + +L1 + +## SLA + +P2 + +## Allowed files + +```text +backend/config/guide_extractor_dependencies.json +backend/scripts/check_guide_extractor_dependencies.py +backend/tests/test_guide_extractor_dependencies.py +.github/workflows/backend.yml +.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/** +docs/spec_artifact_storage_service.md +``` + +## Not allowed + +Package/lock changes, application/runtime imports, parser adapters, fixtures +containing real customer data, AUTH/Celery/submission changes. + +## Acceptance criteria + +- Each candidate records exact package/version, license, maintenance, + advisories, transitive graph, native-code use, malformed-input history, + network behavior, and cancellation/timeout implications. +- The allowlist schema records package name, exact version, artifact hashes, + source/index, direct/transitive status, import names, native-wheel status, + license, advisory snapshot date, and exact format scope. +- The allowlist maps each dependency to only PDF, OOXML, or image metadata. +- CI fails closed on undeclared, unpinned, hash-drifted, or wrong-format parser + dependencies. +- Human approval of the exact list is recorded before any of 03B3B2, + 03B3B3A-03B3B3D, or 03B3B4 starts. + +## Verification commands + +```bash +(cd backend && python scripts/check_guide_extractor_dependencies.py) +(cd backend && uv run pytest -q tests/test_guide_extractor_dependencies.py) +python3 scripts/check_markdown_links.py +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/test_agent_gates.py +git diff --check +``` + +## Required reviewers + +Senior engineering, architecture, QA/test, security/auth, product/ops, +reuse/dedup, CI integrity, test delta, and docs. + +## Stop conditions + +Stop if scope expands, a dependency lacks approval/evidence, isolation must be +weakened, CI or coverage must be weakened, or a second runtime parser path is +required. + +## Human review focus + +Whether every proposed package is necessary, minimal, maintained, license-safe, +pure Python or explicitly native, and adequately isolated. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B2-pdf-extractor.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B2-pdf-extractor.md new file mode 100644 index 00000000..4bb62c32 --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B2-pdf-extractor.md @@ -0,0 +1,82 @@ +# Chunk Contract: WS-ART-001-03B3B2 — PDF Extractor + +## Parent initiative + +WS-ART-001 — Immutable Artifact Storage + +## Goal + +Install only the approved PDF dependency and add bounded PDF text extraction on +the existing 03B3A isolated framework. +## Approved plan reference + +- PLAN: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md` +- CHUNK_MAP: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md` + +## Risk class + +L1 + +## SLA + +P2 + +## Allowed files + +```text +backend/pyproject.toml +backend/uv.lock +backend/config/guide_extractor_dependencies.json +backend/app/modules/artifacts/guide_pdf.py +backend/app/modules/artifacts/guide_extraction.py +backend/app/modules/artifacts/guide_extraction_worker.py +backend/tests/test_guide_pdf.py +backend/tests/fixtures/guide_pdf/** +backend/tests/test_artifact_architecture.py +docs/spec_artifact_storage_service.md +.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/** +``` + +## Not allowed + +OOXML/image packages, framework redesign, OCR, JavaScript/actions, external +fetches, provider/AUTH/Celery/submission changes. + +## Acceptance criteria + +- Exact 03B2 PDF classification is required. +- Encrypted, malformed, attachments, embedded files, forms, XFA, launch/open + actions, active/external, and over-500-page PDFs fail with bounded outcomes. +- Parser imports and execution exist only in the isolated extraction child, + never API, materialization/provider, AUTH, Celery, or agent assembly paths. +- No network, filesystem escape, partial usage, or raw binary reaches agents. +- Exact 500/501 page, timeout, crash, cancellation, cleanup, and coverage proof. + +## Verification commands + +```bash +(cd backend && python scripts/check_guide_extractor_dependencies.py) +(cd backend && uv run ruff check app tests scripts) +(cd backend && uv run pytest -q tests/test_guide_pdf.py tests/test_guide_extraction.py) +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/check_markdown_links.py +git diff --check +``` + +Hosted Backend/Agent Gates must preserve 90% changed-subsystem and 78% repository coverage. + +## Required reviewers + +Senior engineering, architecture, QA/test, security/auth, product/ops, +reuse/dedup, CI integrity, test delta, and docs. + +## Stop conditions + +Stop if scope expands, a dependency lacks approval/evidence, isolation must be +weakened, CI or coverage must be weakened, or a second runtime parser path is +required. + +## Human review focus + +PDF parser graph, active-content handling, page-bound enforcement, deterministic +canonical text, and isolation. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3A-ooxml-container-security.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3A-ooxml-container-security.md new file mode 100644 index 00000000..d7d25d85 --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3A-ooxml-container-security.md @@ -0,0 +1,70 @@ +# Chunk Contract: WS-ART-001-03B3B3A — OOXML Container Security + +## Parent initiative + +WS-ART-001 — Immutable Artifact Storage + +## Goal + +Add shared bounded ZIP/XML marker and rejection capabilities without extracting DOCX, PPTX, or XLSX content. + +## Approved plan reference + +- PLAN: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md` +- CHUNK_MAP: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md` + +## Risk class + +L1 + +## SLA + +P2 + +## Allowed files + +```text +backend/app/modules/artifacts/guide_ooxml.py +backend/app/modules/artifacts/guide_extraction_worker.py +backend/tests/test_guide_ooxml.py +backend/tests/fixtures/guide_ooxml/** +backend/pyproject.toml +backend/uv.lock +docs/spec_artifact_storage_service.md +.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/** +``` + +## Not allowed + +No document adapter/registry activation, generic ZIP semantics, AUTH/Celery/submission changes. + +## Acceptance criteria + +- Require exact classification; reject ambiguity, macros, external relationships, embedded executables, traversal, bombs, unsafe XML and undeclared parser imports. Parser imports execute only in the isolated child. Prove limits, crash, timeout, cancellation, cleanup and dependency gate. + +## Verification commands + +```bash +(cd backend && uv run ruff check app tests) +(cd backend && python scripts/check_guide_extractor_dependencies.py) +(cd backend && uv run pytest -q tests/test_guide_ooxml.py) +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/check_markdown_links.py +git diff --check +``` + +## Required reviewers + +Senior engineering, architecture, QA/test, security/auth, product/ops, +reuse/dedup, CI integrity, test delta, and docs. + +## Human review focus + +Format-specific deterministic semantics, reuse of the shared secure container +boundary, exact limits, and absence of parser imports outside the isolated child. + +## Stop conditions + +Stop on unapproved dependencies, scope expansion, isolation/CI weakening, +architecture drift, or repeated repair failure. + diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3B-docx-extractor.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3B-docx-extractor.md new file mode 100644 index 00000000..f7237aef --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3B-docx-extractor.md @@ -0,0 +1,68 @@ +# Chunk Contract: WS-ART-001-03B3B3B — DOCX Extractor + +## Parent initiative + +WS-ART-001 — Immutable Artifact Storage + +## Goal + +Add deterministic bounded DOCX text/structure extraction using only the approved OOXML capability. + +## Approved plan reference + +- PLAN: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md` +- CHUNK_MAP: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md` + +## Risk class + +L1 + +## SLA + +P2 + +## Allowed files + +```text +backend/app/modules/artifacts/guide_docx.py +backend/app/modules/artifacts/guide_extraction.py +backend/app/modules/artifacts/guide_extraction_worker.py +backend/tests/test_guide_docx.py +backend/tests/fixtures/guide_docx/** +docs/spec_artifact_storage_service.md +.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/** +``` + +## Not allowed + +PPTX/XLSX behavior, package additions beyond approved lock, framework/AUTH/Celery/submission changes. + +## Acceptance criteria + +- Require exact DOCX classification; preserve deterministic paragraphs/tables with bounded output; reject unsafe OOXML; import parser only in isolated child; prove malformed, limits, crash, timeout, cancellation, cleanup and coverage. + +## Verification commands + +```bash +(cd backend && uv run ruff check app tests) +(cd backend && python scripts/check_guide_extractor_dependencies.py) +(cd backend && uv run pytest -q tests/test_guide_ooxml.py tests/test_guide_docx.py tests/test_guide_extraction.py) +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/check_markdown_links.py +git diff --check +``` + +## Required reviewers + +Senior engineering, architecture, QA/test, security/auth, product/ops, +reuse/dedup, CI integrity, test delta, and docs. + +## Human review focus + +Format-specific deterministic semantics, reuse of the shared secure container +boundary, exact limits, and absence of parser imports outside the isolated child. + +## Stop conditions + +Stop on unapproved dependencies, scope expansion, isolation/CI weakening, +architecture drift, or repeated repair failure. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3C-pptx-extractor.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3C-pptx-extractor.md new file mode 100644 index 00000000..bc30c57a --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3C-pptx-extractor.md @@ -0,0 +1,68 @@ +# Chunk Contract: WS-ART-001-03B3B3C — PPTX Extractor + +## Parent initiative + +WS-ART-001 — Immutable Artifact Storage + +## Goal + +Add deterministic bounded PPTX slide and notes extraction using only the approved OOXML capability. + +## Approved plan reference + +- PLAN: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md` +- CHUNK_MAP: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md` + +## Risk class + +L1 + +## SLA + +P2 + +## Allowed files + +```text +backend/app/modules/artifacts/guide_pptx.py +backend/app/modules/artifacts/guide_extraction.py +backend/app/modules/artifacts/guide_extraction_worker.py +backend/tests/test_guide_pptx.py +backend/tests/fixtures/guide_pptx/** +docs/spec_artifact_storage_service.md +.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/** +``` + +## Not allowed + +DOCX/XLSX behavior, unapproved packages, framework/AUTH/Celery/submission changes. + +## Acceptance criteria + +- Require exact PPTX classification; prove 300/301 slides, deterministic slide/notes order, unsafe OOXML rejection, child-only imports, malformed, crash, timeout, cancellation, cleanup and coverage. + +## Verification commands + +```bash +(cd backend && uv run ruff check app tests) +(cd backend && python scripts/check_guide_extractor_dependencies.py) +(cd backend && uv run pytest -q tests/test_guide_ooxml.py tests/test_guide_pptx.py tests/test_guide_extraction.py) +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/check_markdown_links.py +git diff --check +``` + +## Required reviewers + +Senior engineering, architecture, QA/test, security/auth, product/ops, +reuse/dedup, CI integrity, test delta, and docs. + +## Human review focus + +Format-specific deterministic semantics, reuse of the shared secure container +boundary, exact limits, and absence of parser imports outside the isolated child. + +## Stop conditions + +Stop on unapproved dependencies, scope expansion, isolation/CI weakening, +architecture drift, or repeated repair failure. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3D-xlsx-extractor.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3D-xlsx-extractor.md new file mode 100644 index 00000000..e40dbdd2 --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3D-xlsx-extractor.md @@ -0,0 +1,68 @@ +# Chunk Contract: WS-ART-001-03B3B3D — XLSX Extractor + +## Parent initiative + +WS-ART-001 — Immutable Artifact Storage + +## Goal + +Add deterministic bounded XLSX cell extraction using only the approved OOXML capability. + +## Approved plan reference + +- PLAN: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md` +- CHUNK_MAP: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md` + +## Risk class + +L1 + +## SLA + +P2 + +## Allowed files + +```text +backend/app/modules/artifacts/guide_xlsx.py +backend/app/modules/artifacts/guide_extraction.py +backend/app/modules/artifacts/guide_extraction_worker.py +backend/tests/test_guide_xlsx.py +backend/tests/fixtures/guide_xlsx/** +docs/spec_artifact_storage_service.md +.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/** +``` + +## Not allowed + +DOCX/PPTX behavior, formulas/macros/external data execution, unapproved packages, framework/AUTH/Celery/submission changes. + +## Acceptance criteria + +- Require exact XLSX classification; never evaluate formulas or fetch external data; prove 100/101 sheets, row/cell/character limits, deterministic values, child-only imports, unsafe OOXML, malformed, crash, timeout, cancellation, cleanup and coverage. + +## Verification commands + +```bash +(cd backend && uv run ruff check app tests) +(cd backend && python scripts/check_guide_extractor_dependencies.py) +(cd backend && uv run pytest -q tests/test_guide_ooxml.py tests/test_guide_xlsx.py tests/test_guide_extraction.py) +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/check_markdown_links.py +git diff --check +``` + +## Required reviewers + +Senior engineering, architecture, QA/test, security/auth, product/ops, +reuse/dedup, CI integrity, test delta, and docs. + +## Human review focus + +Format-specific deterministic semantics, reuse of the shared secure container +boundary, exact limits, and absence of parser imports outside the isolated child. + +## Stop conditions + +Stop on unapproved dependencies, scope expansion, isolation/CI weakening, +architecture drift, or repeated repair failure. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B4-image-metadata-extractors.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B4-image-metadata-extractors.md new file mode 100644 index 00000000..91041c94 --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B4-image-metadata-extractors.md @@ -0,0 +1,83 @@ +# Chunk Contract: WS-ART-001-03B3B4 — Image Metadata Extractors + +## Parent initiative + +WS-ART-001 — Immutable Artifact Storage + +## Goal + +Install only the approved image dependency and expose bounded PNG/JPEG/WebP +structural metadata without OCR or raw pixels. +## Approved plan reference + +- PLAN: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md` +- CHUNK_MAP: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md` + +## Risk class + +L1 + +## SLA + +P2 + +## Allowed files + +```text +backend/pyproject.toml +backend/uv.lock +backend/config/guide_extractor_dependencies.json +backend/app/modules/artifacts/guide_images.py +backend/app/modules/artifacts/guide_extraction.py +backend/app/modules/artifacts/guide_extraction_worker.py +backend/tests/test_guide_images.py +backend/tests/fixtures/guide_images/** +backend/tests/test_artifact_architecture.py +docs/spec_artifact_storage_service.md +.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/** +``` + +## Not allowed + +OCR, pixel/base64 agent input, EXIF secrets/location propagation, transforms, +PDF/OOXML packages, framework/AUTH/Celery/submission changes. + +## Acceptance criteria + +- Exact 03B2 PNG/JPEG/WebP classification is required. +- Output is a fixed metadata schema and cannot satisfy required textual guide + semantics. +- Parser imports and execution exist only in the isolated extraction child, + never API, materialization/provider, AUTH, Celery, or agent assembly paths. +- Dimensions above 16,384 or 40 megapixels fail before decode/allocation. +- Malformed/truncated/decompression-bomb inputs, metadata privacy, crash, + timeout, cancellation, cleanup, and coverage are proven. + +## Verification commands + +```bash +(cd backend && python scripts/check_guide_extractor_dependencies.py) +(cd backend && uv run ruff check app tests scripts) +(cd backend && uv run pytest -q tests/test_guide_images.py tests/test_guide_extraction.py) +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/check_markdown_links.py +git diff --check +``` + +Hosted Backend/Agent Gates must preserve 90% changed-subsystem and 78% repository coverage. + +## Required reviewers + +Senior engineering, architecture, QA/test, security/auth, product/ops, +reuse/dedup, CI integrity, test delta, and docs. + +## Stop conditions + +Stop if scope expands, a dependency lacks approval/evidence, isolation must be +weakened, CI or coverage must be weakened, or a second runtime parser path is +required. + +## Human review focus + +Metadata minimization, dimension checks before allocation, decompression-bomb +behavior, and absence of OCR/raw pixels. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B4-guide-sufficiency-continuation.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B4-guide-sufficiency-continuation.md index 34f8c524..a284e894 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B4-guide-sufficiency-continuation.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B4-guide-sufficiency-continuation.md @@ -1,6 +1,7 @@ -# Chunk Contract: WS-ART-001-03B4 - Guide Sufficiency Continuation +# Chunk Contract: WS-ART-001-03B4 — Guide Sufficiency Continuation -Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 03B3B +Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 03B3B2, +03B3B3B-03B3B3D, and 03B3B4 ## Goal @@ -15,7 +16,7 @@ same-generation canonical guide material and exact persisted provenance. - in-place evolution or replacement of existing `GuideSourceMaterial` and `GuideSourceItemMaterial`; no parallel sufficiency-material model; - sufficiency-report usage provenance; extraction models/migration remain owned - by 03B3A and complex adapter provenance remains owned by 03B3B; + by 03B3A and complex adapter provenance remains owned by 03B3B2-03B3B4; - focused stale-delivery, completeness, incident, unsupported, broker replay, agent-input, persistence, cancellation, and coverage tests; related docs. From 83e70de54020410b7ecf50f43a6dee0f412ace2c Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Thu, 30 Jul 2026 19:59:18 +0100 Subject: [PATCH 2/2] docs(artifacts): address complex extractor review --- .../AUTH_HANDOFF.md | 3 ++- .../CHUNK_MAP.md | 16 +++++++----- .../DECISIONS.md | 4 +-- .../STATUS.md | 4 +-- ...3B-guide-source-binding-materialization.md | 8 ++++-- ...T-001-03B3B1-parser-dependency-approval.md | 14 +++++++++- .../chunks/WS-ART-001-03B3B2-pdf-extractor.md | 6 ++++- ...RT-001-03B3B3A-ooxml-container-security.md | 26 ++++++++++++++++--- .../WS-ART-001-03B3B3B-docx-extractor.md | 20 ++++++++++++-- .../WS-ART-001-03B3B3C-pptx-extractor.md | 20 ++++++++++++-- .../WS-ART-001-03B3B3D-xlsx-extractor.md | 21 +++++++++++++-- ...RT-001-03B3B4-image-metadata-extractors.md | 16 +++++++++--- ...001-03B4-guide-sufficiency-continuation.md | 2 +- ...01-03B3B-SPLIT-external-review-response.md | 24 +++++++++++++++++ 14 files changed, 156 insertions(+), 28 deletions(-) create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-03B3B-SPLIT-external-review-response.md diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md index 74932f68..ea713eeb 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md @@ -17,7 +17,8 @@ activation custody, and availability. exact resource/guard/surface manifest. 3. AUTH activates only that exact action through a separately reviewed AUTH contract after consuming ART evidence. -4. ART-03B1, 03B2, 03B3A, 03B3B1-03B3B4, and 03B4 implement hidden authoritative binding, exact setup +4. ART-03B1, 03B2, 03B3A, 03B3B1, 03B3B2, 03B3B3A, 03B3B3B, + 03B3B3C, 03B3B3D, 03B3B4, and 03B4 implement hidden authoritative binding, exact setup generation, verified materialization, bounded extraction, and Celery sufficiency continuation. `artifact.guide_source.binding.create` maps to fixed permission `artifact.binding.create`; `artifact.guide_source.read` diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md index e464f57f..80091425 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md @@ -28,7 +28,7 @@ Each chunk is one PR. No later chunk starts automatically. | `WS-ART-001-03B3B3C` | Add bounded PPTX extraction on the approved OOXML capability. | L1 | Proposed after 03B3B3A | | `WS-ART-001-03B3B3D` | Add bounded XLSX extraction on the approved OOXML capability. | L1 | Proposed after 03B3B3A | | `WS-ART-001-03B3B4` | Install only the approved image dependency and add PNG/JPEG/WebP structural metadata extraction. | L1 | Proposed after 03B3B1 approval | -| `WS-ART-001-03B4` | Feed only complete same-generation canonical extracted material into the existing Celery sufficiency pipeline. | L1 | Proposed after 03B3B2, 03B3B3B-03B3B3D, and 03B3B4 | +| `WS-ART-001-03B4` | Feed only complete same-generation canonical extracted material into the existing Celery sufficiency pipeline. | L1 | Proposed after 03B3B2, 03B3B3B, 03B3B3C, 03B3B3D, and 03B3B4 | | `WS-ART-001-03C` | Remove legacy guide-source identity and add exact same-generation setup continuation. | L1 | Proposed after 03B1-03B4 and AUTH-04B | | `WS-ART-001-04A` | Accept one outer ZIP in bounded scratch, safely inspect its tree, normalize executable intent, produce canonical identities, and reject unchanged work before provider I/O. | L1 | Proposed after 03C and AUTH planned action registration | | `WS-ART-001-04B` | Run mandatory platform and locked Project Guide pre-submit checks against the same scratch-bound tree and executable semantics without durable storage. | L1 | Proposed after 04A | @@ -57,11 +57,15 @@ OBJECT-STORAGE-AMENDMENT -> 03B2 verified materialization/format detection -> 03B3A isolated extraction framework/text formats -> 03B3B1 pinned dependency approval and CI gate --> 03B3B2 approved PDF extractor (independent after 03B3B1) --> 03B3B3A shared OOXML security capability - -> 03B3B3B DOCX / 03B3B3C PPTX / 03B3B3D XLSX --> 03B3B4 approved image metadata (independent after 03B3B1) --> 03B4 same-generation Celery sufficiency integration +-> 03B3B1 branches to: + -> 03B3B2 approved PDF extractor + -> 03B3B3A shared OOXML security capability + -> 03B3B3B DOCX + -> 03B3B3C PPTX + -> 03B3B3D XLSX + -> 03B3B4 approved image metadata +-> 03B4 same-generation Celery sufficiency integration only after 03B3B2, + 03B3B3B, 03B3B3C, 03B3B3D, and 03B3B4 complete -> AUTH activation for exact 03B actions -> 03C guide-source clean cut/continuation -> AUTH planned registration of `artifact.submission_bundle.prepare` diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DECISIONS.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DECISIONS.md index 9b0be1c9..d378807b 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DECISIONS.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DECISIONS.md @@ -403,8 +403,8 @@ have bounded outcomes and cleanup. Production parser dependencies require explicit human approval before implementation. Complex-format delivery is split after 03B3A: 03B3B1 records the exact pinned dependency decision and deterministic gate without installing packages; 03B3B2 delivers PDF, 03B3B3A -owns shared OOXML security, 03B3B3B-D separately deliver DOCX/PPTX/XLSX, and -03B3B4 delivers image metadata. +owns shared OOXML security, 03B3B3B delivers DOCX, 03B3B3C delivers PPTX, +03B3B3D delivers XLSX, and 03B3B4 delivers image metadata. The v0.1 extraction-policy limits are fixed, not caller-selectable: diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md index 34130da6..44824ca3 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md @@ -63,8 +63,8 @@ implementation successor; this planning change contains no implementation. After 03B3A merged, the original complex-format chunk was found too broad for one dependency and parser-security review. It is replaced by 03B3B1 dependency -approval, 03B3B2 PDF, 03B3B3A shared OOXML security, separate 03B3B3B-D -DOCX/PPTX/XLSX adapters, and 03B3B4 image metadata. No dependency is +approval, 03B3B2 PDF, 03B3B3A shared OOXML security, separate 03B3B3B DOCX, +03B3B3C PPTX, and 03B3B3D XLSX adapters, and 03B3B4 image metadata. No dependency is installed until the human owner approves 03B3B1's exact pinned allowlist. AUTH `WS-XINT-002-04B` follows the complete hidden split-03B series and diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B-guide-source-binding-materialization.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B-guide-source-binding-materialization.md index e2143ff9..ffbdae2d 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B-guide-source-binding-materialization.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B-guide-source-binding-materialization.md @@ -15,8 +15,12 @@ implementation contracts. - `WS-ART-001-03B1`: authoritative binding and setup generation; - `WS-ART-001-03B2`: verified materialization, incidents, and classification; - `WS-ART-001-03B3A`: isolated framework and low-complexity extraction; -- `WS-ART-001-03B3B1` through `03B3B4`: dependency approval followed by - separate PDF, OOXML, and image-metadata extractors; +- `WS-ART-001-03B3B1`: pinned dependency approval and CI gate; +- `WS-ART-001-03B3B2`: PDF extraction; +- `WS-ART-001-03B3B3A`: shared OOXML container security; +- `WS-ART-001-03B3B3B`, `WS-ART-001-03B3B3C`, and + `WS-ART-001-03B3B3D`: separate DOCX, PPTX, and XLSX extraction; +- `WS-ART-001-03B3B4`: image metadata extraction; - `WS-ART-001-03B4`: same-generation Celery and sufficiency integration. Each subchunk is one PR. AUTH `WS-XINT-002-04B` activates fixed binding/read diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B1-parser-dependency-approval.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B1-parser-dependency-approval.md index 69648152..8078ff62 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B1-parser-dependency-approval.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B1-parser-dependency-approval.md @@ -51,10 +51,22 @@ containing real customer data, AUTH/Celery/submission changes. source/index, direct/transitive status, import names, native-wheel status, license, advisory snapshot date, and exact format scope. - The allowlist maps each dependency to only PDF, OOXML, or image metadata. +- Approval is anchored in the protected GitHub PR review and merge history, not + asserted by a contributor-controlled repository file. A repository + maintainer other than the dependency PR author must approve the exact PR head + after the allowlist's final change; stale-review dismissal must invalidate an + approval when those bytes change. The merged commit is the approved baseline. +- Any repository evidence record may mirror the GitHub review URL/ID, approver, + UTC timestamp, approved head commit, and lowercase SHA-256 of the allowlist, + but it is audit metadata and is never sufficient authority by itself. +- CI recomputes the current allowlist digest against the merged approved + baseline and fails closed on drift. The gate and its tests must reject + absent independent approval, self-authored/forged records, stale head + approval, and a digest changed after approval. - CI fails closed on undeclared, unpinned, hash-drifted, or wrong-format parser dependencies. - Human approval of the exact list is recorded before any of 03B3B2, - 03B3B3A-03B3B3D, or 03B3B4 starts. + 03B3B3A, 03B3B3B, 03B3B3C, 03B3B3D, or 03B3B4 starts. ## Verification commands diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B2-pdf-extractor.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B2-pdf-extractor.md index 4bb62c32..2f93eb99 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B2-pdf-extractor.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B2-pdf-extractor.md @@ -8,6 +8,9 @@ WS-ART-001 — Immutable Artifact Storage Install only the approved PDF dependency and add bounded PDF text extraction on the existing 03B3A isolated framework. + +`WS-ART-001-03B3B1` is a hard predecessor. Installation and imports fail +closed unless its merged protected GitHub approval baseline matches the exact pinned allowlist. ## Approved plan reference - PLAN: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md` @@ -57,7 +60,8 @@ fetches, provider/AUTH/Celery/submission changes. ```bash (cd backend && python scripts/check_guide_extractor_dependencies.py) (cd backend && uv run ruff check app tests scripts) -(cd backend && uv run pytest -q tests/test_guide_pdf.py tests/test_guide_extraction.py) +(cd backend && uv run pytest -q tests/test_guide_pdf.py tests/test_guide_extraction.py tests/test_artifact_architecture.py --cov=app.modules.artifacts --cov-report=term-missing --cov-fail-under=90) +(metadata_dir="$(mktemp -d)" && trap 'rm -rf "$metadata_dir"' EXIT && (cd backend && WORKSTREAM_TEST_ADMIN_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/postgres .venv/bin/python scripts/run_isolated_tests.py --metadata-json "$metadata_dir/result.json" --timeout-seconds 12600 -- .venv/bin/python -m pytest -q --ignore=tests/test_isolated_database_runner.py --cov=app --cov-report=term-missing --cov-fail-under=78)) python3 scripts/check_stale_artifact_contracts.py python3 scripts/check_markdown_links.py git diff --check diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3A-ooxml-container-security.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3A-ooxml-container-security.md index d7d25d85..bcc92819 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3A-ooxml-container-security.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3A-ooxml-container-security.md @@ -8,6 +8,10 @@ WS-ART-001 — Immutable Artifact Storage Add shared bounded ZIP/XML marker and rejection capabilities without extracting DOCX, PPTX, or XLSX content. +`WS-ART-001-03B3B1` is a hard predecessor. Package installation and parser +imports fail closed unless its merged protected GitHub approval baseline matches the exact pinned +allowlist. The DOCX, PPTX, and XLSX chunks inherit this same gate. + ## Approved plan reference - PLAN: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md` @@ -40,19 +44,36 @@ No document adapter/registry activation, generic ZIP semantics, AUTH/Celery/subm ## Acceptance criteria -- Require exact classification; reject ambiguity, macros, external relationships, embedded executables, traversal, bombs, unsafe XML and undeclared parser imports. Parser imports execute only in the isolated child. Prove limits, crash, timeout, cancellation, cleanup and dependency gate. +- Inspect central-directory metadata before reading entry bodies. Reject + duplicate/case-colliding normalized names, absolute/traversing names, + encrypted entries, nested archives, symlinks/special entries, macros, + external relationships, embedded objects/executables, DTD/entity use, and + any required marker conflict. Extra members are allowed only when their + normalized names fall within the fixed OPC package-part allowlist documented + in the implementation; unknown roots or executable-capable suffixes reject + before a format adapter runs. +- Inherit the fixed D42 limits: at most 2,000 entries, 128 MiB decompressed + bytes, nesting depth 8, 32 MiB input, 4 MiB output, 30 CPU seconds, 60 wall + seconds, 512 MiB address space, 32 descriptors, and no child processes/core + dumps. Each breach has the D42 bounded outcome before adapter dispatch. +- Require exact classification and reject ambiguity. Parser imports execute + only in the isolated child. Prove every rejection class and boundary plus + crash, timeout, cancellation, cleanup, and the dependency gate. ## Verification commands ```bash (cd backend && uv run ruff check app tests) (cd backend && python scripts/check_guide_extractor_dependencies.py) -(cd backend && uv run pytest -q tests/test_guide_ooxml.py) +(cd backend && uv run pytest -q tests/test_guide_ooxml.py --cov=app.modules.artifacts --cov-report=term-missing --cov-fail-under=90) +(metadata_dir="$(mktemp -d)" && trap 'rm -rf "$metadata_dir"' EXIT && (cd backend && WORKSTREAM_TEST_ADMIN_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/postgres .venv/bin/python scripts/run_isolated_tests.py --metadata-json "$metadata_dir/result.json" --timeout-seconds 12600 -- .venv/bin/python -m pytest -q --ignore=tests/test_isolated_database_runner.py --cov=app --cov-report=term-missing --cov-fail-under=78)) python3 scripts/check_stale_artifact_contracts.py python3 scripts/check_markdown_links.py git diff --check ``` +Hosted Backend/Agent Gates must preserve 90% changed-subsystem and 78% repository coverage. + ## Required reviewers Senior engineering, architecture, QA/test, security/auth, product/ops, @@ -67,4 +88,3 @@ boundary, exact limits, and absence of parser imports outside the isolated child Stop on unapproved dependencies, scope expansion, isolation/CI weakening, architecture drift, or repeated repair failure. - diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3B-docx-extractor.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3B-docx-extractor.md index f7237aef..8abc0461 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3B-docx-extractor.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3B-docx-extractor.md @@ -8,6 +8,10 @@ WS-ART-001 — Immutable Artifact Storage Add deterministic bounded DOCX text/structure extraction using only the approved OOXML capability. +`WS-ART-001-03B3B1` and `WS-ART-001-03B3B3A` are hard predecessors. Package +installation and imports fail closed unless the merged protected GitHub approval baseline matches +the exact pinned allowlist. + ## Approved plan reference - PLAN: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md` @@ -39,19 +43,31 @@ PPTX/XLSX behavior, package additions beyond approved lock, framework/AUTH/Celer ## Acceptance criteria -- Require exact DOCX classification; preserve deterministic paragraphs/tables with bounded output; reject unsafe OOXML; import parser only in isolated child; prove malformed, limits, crash, timeout, cancellation, cleanup and coverage. +- Require exact DOCX classification and the shared OOXML security boundary. +- Traverse body elements in document order. Emit paragraph text in run order; + emit each table in row then cell order, flattening nested tables at their + containing-cell position with fixed row/cell separators. Headers, footers, + comments, tracked-deletion text, and embedded objects are omitted and + recorded in omission facts. Streaming stops with `limit_exceeded` before + exceeding D42's 4 MiB canonical-output limit; no partial result is usable. +- Import the parser only in the isolated child and prove deterministic output, + malformed/unsafe input, exact output boundary, crash, timeout, cancellation, + cleanup, approval-gate, and coverage behavior. ## Verification commands ```bash (cd backend && uv run ruff check app tests) (cd backend && python scripts/check_guide_extractor_dependencies.py) -(cd backend && uv run pytest -q tests/test_guide_ooxml.py tests/test_guide_docx.py tests/test_guide_extraction.py) +(cd backend && uv run pytest -q tests/test_guide_ooxml.py tests/test_guide_docx.py tests/test_guide_extraction.py --cov=app.modules.artifacts --cov-report=term-missing --cov-fail-under=90) +(metadata_dir="$(mktemp -d)" && trap 'rm -rf "$metadata_dir"' EXIT && (cd backend && WORKSTREAM_TEST_ADMIN_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/postgres .venv/bin/python scripts/run_isolated_tests.py --metadata-json "$metadata_dir/result.json" --timeout-seconds 12600 -- .venv/bin/python -m pytest -q --ignore=tests/test_isolated_database_runner.py --cov=app --cov-report=term-missing --cov-fail-under=78)) python3 scripts/check_stale_artifact_contracts.py python3 scripts/check_markdown_links.py git diff --check ``` +Hosted Backend/Agent Gates must preserve 90% changed-subsystem and 78% repository coverage. + ## Required reviewers Senior engineering, architecture, QA/test, security/auth, product/ops, diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3C-pptx-extractor.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3C-pptx-extractor.md index bc30c57a..a1d10032 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3C-pptx-extractor.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3C-pptx-extractor.md @@ -8,6 +8,10 @@ WS-ART-001 — Immutable Artifact Storage Add deterministic bounded PPTX slide and notes extraction using only the approved OOXML capability. +`WS-ART-001-03B3B1` and `WS-ART-001-03B3B3A` are hard predecessors. Package +installation and imports fail closed unless the merged protected GitHub approval baseline matches +the exact pinned allowlist. + ## Approved plan reference - PLAN: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md` @@ -39,19 +43,31 @@ DOCX/XLSX behavior, unapproved packages, framework/AUTH/Celery/submission change ## Acceptance criteria -- Require exact PPTX classification; prove 300/301 slides, deterministic slide/notes order, unsafe OOXML rejection, child-only imports, malformed, crash, timeout, cancellation, cleanup and coverage. +- Require exact PPTX classification and the shared OOXML security boundary. +- Accept exactly 300 slides and reject 301 before extraction. Traverse slides + by presentation order, shapes by XML document order, and text paragraphs/runs + in document order; append speaker notes after their owning slide in notes XML + order. Omit masters, comments, hidden metadata, and embedded objects while + recording omission facts. Exceeding D42's output limit produces unusable + `limit_exceeded`, never partial agent input. +- Prove deterministic slide/notes output, unsafe/malformed input, child-only + imports, exact 300/301 and output boundaries, crash, timeout, cancellation, + cleanup, approval-gate, and coverage behavior. ## Verification commands ```bash (cd backend && uv run ruff check app tests) (cd backend && python scripts/check_guide_extractor_dependencies.py) -(cd backend && uv run pytest -q tests/test_guide_ooxml.py tests/test_guide_pptx.py tests/test_guide_extraction.py) +(cd backend && uv run pytest -q tests/test_guide_ooxml.py tests/test_guide_pptx.py tests/test_guide_extraction.py --cov=app.modules.artifacts --cov-report=term-missing --cov-fail-under=90) +(metadata_dir="$(mktemp -d)" && trap 'rm -rf "$metadata_dir"' EXIT && (cd backend && WORKSTREAM_TEST_ADMIN_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/postgres .venv/bin/python scripts/run_isolated_tests.py --metadata-json "$metadata_dir/result.json" --timeout-seconds 12600 -- .venv/bin/python -m pytest -q --ignore=tests/test_isolated_database_runner.py --cov=app --cov-report=term-missing --cov-fail-under=78)) python3 scripts/check_stale_artifact_contracts.py python3 scripts/check_markdown_links.py git diff --check ``` +Hosted Backend/Agent Gates must preserve 90% changed-subsystem and 78% repository coverage. + ## Required reviewers Senior engineering, architecture, QA/test, security/auth, product/ops, diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3D-xlsx-extractor.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3D-xlsx-extractor.md index e40dbdd2..94b9694b 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3D-xlsx-extractor.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B3D-xlsx-extractor.md @@ -8,6 +8,10 @@ WS-ART-001 — Immutable Artifact Storage Add deterministic bounded XLSX cell extraction using only the approved OOXML capability. +`WS-ART-001-03B3B1` and `WS-ART-001-03B3B3A` are hard predecessors. Package +installation and imports fail closed unless the merged protected GitHub approval baseline matches +the exact pinned allowlist. + ## Approved plan reference - PLAN: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md` @@ -39,19 +43,32 @@ DOCX/PPTX behavior, formulas/macros/external data execution, unapproved packages ## Acceptance criteria -- Require exact XLSX classification; never evaluate formulas or fetch external data; prove 100/101 sheets, row/cell/character limits, deterministic values, child-only imports, unsafe OOXML, malformed, crash, timeout, cancellation, cleanup and coverage. +- Require exact XLSX classification and the shared OOXML security boundary. +- Accept exactly 100 worksheets and reject 101 before extraction. Traverse + workbook sheet order, then ascending row and cell coordinates. Resolve shared + strings to text deterministically. Emit formula source text and a separately + labelled cached value when present, but never calculate it; emit merged-cell + content once at the anchor and record the covered range. Empty cells remain + omitted. Fixed row, cell, character, and D42 output limits fail the complete + result as `limit_exceeded`; no partial result is usable. +- Never evaluate formulas or fetch external data. Prove deterministic values, + exact 100/101 and other boundaries, child-only imports, unsafe/malformed + input, crash, timeout, cancellation, cleanup, approval-gate, and coverage. ## Verification commands ```bash (cd backend && uv run ruff check app tests) (cd backend && python scripts/check_guide_extractor_dependencies.py) -(cd backend && uv run pytest -q tests/test_guide_ooxml.py tests/test_guide_xlsx.py tests/test_guide_extraction.py) +(cd backend && uv run pytest -q tests/test_guide_ooxml.py tests/test_guide_xlsx.py tests/test_guide_extraction.py --cov=app.modules.artifacts --cov-report=term-missing --cov-fail-under=90) +(metadata_dir="$(mktemp -d)" && trap 'rm -rf "$metadata_dir"' EXIT && (cd backend && WORKSTREAM_TEST_ADMIN_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/postgres .venv/bin/python scripts/run_isolated_tests.py --metadata-json "$metadata_dir/result.json" --timeout-seconds 12600 -- .venv/bin/python -m pytest -q --ignore=tests/test_isolated_database_runner.py --cov=app --cov-report=term-missing --cov-fail-under=78)) python3 scripts/check_stale_artifact_contracts.py python3 scripts/check_markdown_links.py git diff --check ``` +Hosted Backend/Agent Gates must preserve 90% changed-subsystem and 78% repository coverage. + ## Required reviewers Senior engineering, architecture, QA/test, security/auth, product/ops, diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B4-image-metadata-extractors.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B4-image-metadata-extractors.md index 91041c94..066c64ac 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B4-image-metadata-extractors.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B3B4-image-metadata-extractors.md @@ -8,6 +8,9 @@ WS-ART-001 — Immutable Artifact Storage Install only the approved image dependency and expose bounded PNG/JPEG/WebP structural metadata without OCR or raw pixels. + +`WS-ART-001-03B3B1` is a hard predecessor. Installation and imports fail +closed unless its merged protected GitHub approval baseline matches the exact pinned allowlist. ## Approved plan reference - PLAN: `.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md` @@ -50,15 +53,22 @@ PDF/OOXML packages, framework/AUTH/Celery/submission changes. - Parser imports and execution exist only in the isolated extraction child, never API, materialization/provider, AUTH, Celery, or agent assembly paths. - Dimensions above 16,384 or 40 megapixels fail before decode/allocation. -- Malformed/truncated/decompression-bomb inputs, metadata privacy, crash, - timeout, cancellation, cleanup, and coverage are proven. +- Output is allowlisted to detected format, width, height, frame count, color + model, bit depth, and transparency. Discard EXIF, XMP, IPTC, ICC payloads, + PNG text chunks, comments, thumbnails, geolocation, and every other ancillary + metadata value; fixtures prove this for PNG, JPEG, and WebP. +- Inherit D42's 32 MiB input, 4 MiB output, CPU/wall-clock, address-space, + descriptor, child/core, termination, and scratch-cleanup rules. Prove these + resource boundaries plus malformed, truncated, decompression-bomb, 16,384 + dimension, 40-megapixel, crash, timeout, cancellation, and cleanup outcomes. ## Verification commands ```bash (cd backend && python scripts/check_guide_extractor_dependencies.py) (cd backend && uv run ruff check app tests scripts) -(cd backend && uv run pytest -q tests/test_guide_images.py tests/test_guide_extraction.py) +(cd backend && uv run pytest -q tests/test_guide_images.py tests/test_guide_extraction.py tests/test_artifact_architecture.py --cov=app.modules.artifacts --cov-report=term-missing --cov-fail-under=90) +(metadata_dir="$(mktemp -d)" && trap 'rm -rf "$metadata_dir"' EXIT && (cd backend && WORKSTREAM_TEST_ADMIN_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/postgres .venv/bin/python scripts/run_isolated_tests.py --metadata-json "$metadata_dir/result.json" --timeout-seconds 12600 -- .venv/bin/python -m pytest -q --ignore=tests/test_isolated_database_runner.py --cov=app --cov-report=term-missing --cov-fail-under=78)) python3 scripts/check_stale_artifact_contracts.py python3 scripts/check_markdown_links.py git diff --check diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B4-guide-sufficiency-continuation.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B4-guide-sufficiency-continuation.md index a284e894..3995b005 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B4-guide-sufficiency-continuation.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B4-guide-sufficiency-continuation.md @@ -1,7 +1,7 @@ # Chunk Contract: WS-ART-001-03B4 — Guide Sufficiency Continuation Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 03B3B2, -03B3B3B-03B3B3D, and 03B3B4 +03B3B3B, 03B3B3C, 03B3B3D, and 03B3B4 ## Goal diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-03B3B-SPLIT-external-review-response.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-03B3B-SPLIT-external-review-response.md new file mode 100644 index 00000000..72362cb2 --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-03B3B-SPLIT-external-review-response.md @@ -0,0 +1,24 @@ +# WS-ART-001-03B3B Split — External Review Response + +## CodeRabbit review on 2026-07-30 + +All 11 findings against PR #228 were verified and accepted. + +- Replaced range shorthand with every canonical chunk identifier and made the + dependency graph branch after 03B3B1 before joining at 03B4. +- Bound dependency approval to independent protected GitHub review/merge + history by a maintainer other than the PR author. Repository evidence is + audit-only; CI must reject self-authored/forged, stale-head, missing, or + digest-drifted approval. +- Made 03B3B1 a hard predecessor for PDF, OOXML, and image packages/imports. +- Added architecture/isolation where required and established 90% subsystem + plus 78% repository coverage commands to every parser contract. +- Defined the OOXML member/rejection policy and inherited exact D42 resource + limits before adapter dispatch. +- Made DOCX traversal, PPTX slide/notes ordering and 300/301 boundary, and XLSX + formula/shared-string/merged-cell semantics plus 100/101 boundary explicit. +- Restricted image output to a fixed structural allowlist, discarded all + ancillary metadata, and required inherited resource/termination tests. + +Validation after the repair is recorded in the PR and its exact-head hosted +checks. No runtime or package change is part of this planning PR.