Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -215,3 +215,29 @@
fixture-only resource probes, and expanded boundary/migration evidence.
- Architecture, security, senior engineering, product/ops, QA, docs, reuse,
CI-integrity, and test-delta repair reviews pass with no blockers.

## WS-ART-001-03B3B1

- Selected the minimal complex-format dependency set: `pypdf==6.14.2`,
`defusedxml==0.7.1`, and `Pillow==12.3.0`; no package or lock change occurs.
- Initial security, QA, senior-engineering, and product/ops review found that
version-only declarations did not bind installed wheel hashes, excluded
OOXML packages and optional groups could bypass the gate, format scopes were
global, and wheel identity/platform coherence was under-specified.
- Repairs require exact hash-bound PyPI wheel references, scan runtime,
optional, and dependency-group declarations, reject the prohibited OOXML
package set, validate wheel identity/tags/platform, and scope imports per
parser module.
- Focused proof passes with 34 tests and 92.94 percent checker coverage.
Architecture, security, QA, product/ops, senior engineering, CI integrity,
reuse/dedup, and test-delta reviews pass after repair. Docs confirmation and
exact hosted PR-head behavior remain publication evidence.
- The first approval-authorized hosted run reached the canonical lane inventory
and correctly failed because the new focused test module had not been assigned
to a lane. The repair adds it to `shared_foundations`; no lane, test, or
coverage policy is weakened.
- CodeRabbit's later review found three valid issues: ambiguous final-head
wording, unstable type failures, and `COMMENTED` reviews erasing approval.
All are repaired with regression proof. Its review-event checkout concern is
rejected because hosted review-event run `30600808957` passed exact-head
approval and executed the PR's lane inventory before its later failure.
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,12 @@ approval, 03B3B2 PDF, 03B3B3A shared OOXML security, separate 03B3B3B DOCX,
03B3B3C PPTX, and 03B3B3D XLSX adapters, and 03B3B4 image metadata. No dependency is
installed until the human owner approves 03B3B1's exact pinned allowlist.

03B3B1 implementation is active as a dependency-decision and CI-only chunk.
It proposes exact hashed wheels for `pypdf`, `defusedxml`, and `Pillow`, with
no package, lock, runtime import, or parser behavior change. Its approval gate
requires independent protected GitHub review of the exact final PR head before
merge; repository-authored evidence alone is not authority.

AUTH `WS-XINT-002-04B` follows the complete hidden split-03B series and
activates only fixed-service binding and guide read. ART-03C then removes the
legacy identity/excerpt path and makes the verified pipeline authoritative.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ P2
```text
backend/config/guide_extractor_dependencies.json
backend/scripts/check_guide_extractor_dependencies.py
backend/scripts/run_test_lanes.py
backend/tests/test_guide_extractor_dependencies.py
.github/workflows/backend.yml
.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/**
Expand Down Expand Up @@ -71,11 +72,11 @@ containing real customer data, AUTH/Celery/submission changes.
## Verification commands

```bash
(cd backend && python scripts/check_guide_extractor_dependencies.py)
(cd backend && uv run pytest -q tests/test_guide_extractor_dependencies.py)
(cd backend && python3 scripts/check_guide_extractor_dependencies.py)
(cd backend && PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python3 -m pytest -q -p pytest_cov.plugin tests/test_guide_extractor_dependencies.py --cov=scripts.check_guide_extractor_dependencies --cov-report=term-missing --cov-fail-under=90)
python3 scripts/check_markdown_links.py
python3 scripts/check_stale_artifact_contracts.py
python3 scripts/test_agent_gates.py
python3 -m unittest -v scripts.test_lightweight_agent_gates
git diff --check
```

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# WS-ART-001-03B3B1 External Review Response

## Comments addressed

- GitHub Backend run `30600808957` passed the exact-head approval gate, then
failed canonical collection with
`missing_lane_modules:tests/test_guide_extractor_dependencies.py`.
- The focused test module is now assigned to the existing
`shared_foundations` semantic lane. No new lane or coverage exception was
introduced.

## Comments deferred

- None.

## Human decisions needed

- A fresh independent approval of the exact final PR head is pending.
`abiorh-claw` approved `66ac70e6` before the CI repair, so that approval was
correctly dismissed when the head changed.

## CodeRabbit

- The initial check returned `pass` with `Review rate limited` and no comments.
- The repaired-head review posted four findings. Three valid findings are
addressed: exact-final-head wording, stable type-validation failures, and
preserving an active approval across a later `COMMENTED` review.
- The workflow-checkout finding is rejected with direct hosted evidence: review
event run `30600808957` passed exact-head approval against `66ac70e6`, executed
the PR's new code, and reached canonical lane collection. The checkout also
uses `fetch-depth: 0`, so both PR parents required by the diff were present.

## Commands rerun

- Focused dependency gate and tests.
- Stable malformed-value and approval-then-comment regression tests.
- Canonical semantic-lane collect-only validation.
- Ruff, markdown links, stale-contract checks, lightweight agent gates, and
diff checks.

## Remaining risks

- Hosted Backend and Agent Gates must pass on the repaired exact head after a
fresh independent approval.
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# WS-ART-001-03B3B1 PR Trust Bundle

## Chunk

`WS-ART-001-03B3B1` — Parser Dependency Approval.

## Goal

Approve and enforce the smallest exact PDF, OOXML-safety, and image-metadata
dependency set without installing packages or changing runtime behavior.

## Human-approved intent

The merged 03B3B split contract requires dependency approval before any PDF,
OOXML, or image parser implementation. The human owner explicitly started this
chunk after PR #228 merged.

## What changed

- Added a canonical allowlist for `pypdf==6.14.2`, `defusedxml==0.7.1`, and
`Pillow==12.3.0`, including exact approved wheel URLs and SHA-256 hashes.
- Added a deterministic checker for manifest shape, wheel identity/platform,
hash-bound future declarations, prohibited packages, format-scoped imports,
and independent exact-head GitHub approval.
- Added focused failure tests and wired the gate before backend installation.
- Registered the focused test module in the canonical `shared_foundations`
semantic lane after hosted inventory validation exposed the missing entry.
- Updated the ART specification, chunk command, status, and review evidence.

## Why it changed

Untrusted document parsers are a supply-chain boundary. A package name or
version alone cannot prove which bytes will later be installed, and a
repository-authored approval file cannot independently authorize itself.

## Design chosen

- PDF: pure-Python `pypdf`.
- OOXML safety: pure-Python `defusedxml` plus later ART-owned bounded container
and format adapters.
- Image metadata: native `Pillow`, restricted to two named Python 3.11/3.12
manylinux x86_64 wheels.
- Future runtime declarations must use the exact approved wheel URL with its
SHA-256 fragment. Optional/dependency-group parser declarations are denied.
- Allowlist changes require a current approving GitHub review from an
independent human owner/member/collaborator on the exact PR head.

## Alternatives rejected

- `python-docx`, `python-pptx`, `openpyxl`, `lxml`, and `XlsxWriter`: larger,
unnecessary graphs for the bounded v0.1 OOXML extraction design.
- Version-only pins: do not bind installed artifact bytes.
- Repository-local approval as authority: forgeable by the proposing PR.
- `pull_request_target`: an unnecessary privileged workflow boundary.

## Scope control

No `pyproject.toml`, lockfile, dependency install, application import, parser
adapter, AUTH, Celery, submission, or product lifecycle change is included.

## Product behavior

None. Project Manager guide-source handling remains distinct from contributor
one-ZIP submission handling. All parser behavior remains unimplemented.

## Acceptance criteria proof

- Canonical schema covers version, license, maintenance, advisories,
transitive graph, native-code, malformed-input, network, cancellation,
import, format, source, wheel URL, and hash facts.
- Three closed scopes are present exactly once.
- Plain pins, wrong hashes/URLs/wheels/tags/scopes, prohibited packages,
optional-group bypasses, cross-format imports, self/bot/stale/dismissed
approvals, and GitHub API failure all reject in focused tests.

## Tests/checks run

- Manifest checker: pass.
- Focused pytest: 34 pass; checker coverage 92.94 percent.
- Ruff lint and format: pass.
- Compile, markdown links, stale artifact/wording, lightweight agent gates,
and `git diff --check`: pass.

## Test delta

One new focused test module; no tests removed, skipped, or weakened.

## CI integrity

The gate runs before package installation. Existing backend tests, semantic
lanes, provider proof, and 78/90 percent coverage floors are unchanged. Token
permissions remain read-only: `contents: read` and `pull-requests: read`.

## Reviewer results

Architecture, security, QA, product/ops, senior engineering, CI integrity,
reuse/dedup, and test delta pass after valid findings were repaired. Docs
review is rerun on the staged evidence.

## External review

CodeRabbit's later substantive review produced three valid repaired findings
and one workflow-checkout finding rejected by direct hosted evidence. The first approval-authorized
Backend run found one missing semantic-lane inventory entry, now repaired.
Exact repaired-head hosted results remain required external evidence.

## Remaining risks

- Live GitHub review-event/check semantics require hosted proof.
- Review submitted/dismissed events rerun the full Backend job; this is an
accepted low operational cost for the v0.1 fail-closed required check.
- `defusedxml` is a stable, slow-release security utility; its isolation and
bounded OOXML use remain mandatory in later chunks.

## Follow-up work

After this chunk merges, `WS-ART-001-03B3B2` may install only the approved PDF
wheel and implement bounded PDF extraction. Later OOXML and image chunks remain
separate and require explicit starts.

## Human review focus

Confirm the three-package minimum, wheel URL/hash selection, native Pillow
platform bounds, prohibited package set, and independent exact-head approval
semantics.

## Human merge ownership

Only the human owner may approve and merge this PR. The PR must receive an
independent qualifying GitHub approval on its exact final head; this agent will
not merge it.
22 changes: 22 additions & 0 deletions .github/workflows/backend.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,17 @@ name: Backend

on:
pull_request:
pull_request_review:
types:
- submitted
- dismissed
push:
branches:
- main

permissions:
contents: read
pull-requests: read

env:
MINIO_IMAGE: quay.io/minio/minio:latest@sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e
Expand Down Expand Up @@ -55,6 +60,23 @@ jobs:
echo "job_start_epoch=${job_start_epoch}" >> "${GITHUB_OUTPUT}"
echo "tree_sha=${tree_sha}" >> "${GITHUB_OUTPUT}"

- name: Guide extractor dependency approval
working-directory: backend
env:
GITHUB_TOKEN: ${{ github.token }}
WORKSTREAM_PR_AUTHOR: ${{ github.event.pull_request.user.login }}
WORKSTREAM_PR_BASE_SHA: ${{ github.event.pull_request.base.sha }}
WORKSTREAM_PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
WORKSTREAM_PR_NUMBER: ${{ github.event.pull_request.number }}
shell: bash
run: |
set -euo pipefail
approval_args=()
if [[ "${GITHUB_EVENT_NAME}" != "push" ]]; then
approval_args+=(--require-pr-approval)
fi
python scripts/check_guide_extractor_dependencies.py "${approval_args[@]}"
Comment on lines +63 to +78

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Inspect workflow triggers, checkout inputs, and identity assertions for event coupling.
set -uo pipefail
fd -t f 'backend.yml' .github/workflows --exec rg -n -C4 'on:|pull_request|checkout|ref:|fetch-depth|GITHUB_SHA|tree_sha|require-pr-approval'

Repository: Flow-Research/workstream

Length of output: 3403


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "## Candidate files"
fd -t f 'check_guide_extractor_dependencies.py|guide_extractor|guide.*dependency|dependency' .

echo
echo "## workflows backend relevant lines"
sed -n '36,86p;250,290p;420,435p' .github/workflows/backend.yml

echo
echo "## scripts outline"
for f in $(fd -t f 'check_guide_extractor_dependencies.py' .); do
  echo "--- $f"
  wc -l "$f"
  sed -n '1,260p' "$f"
done

echo
echo "## Diff semantics references in repo"
rg -n "allowlist_changed|git diff|guide_dependency_git_diff_failed|pull request head|head_sha|base_sha|require-pr-approval|dependency" scripts .github/workflows backend -S

Repository: Flow-Research/workstream

Length of output: 50047


On pull_request_review, the dependency approval gate checks out the default branch while comparing against the pull-request head.

pull_request_review sets ${{ github.sha }} to the default-branch ref, but this job enables --require-pr-approval for ${{ github.event.pull_request.head.sha }} and the script runs git diff ${base_sha}...${head_sha} -- guide_extractor_dependencies.json. The approved head_sha can pass while the checked-out workflow actually validates default-branch artifacts.

For a fork PR, the head commit is missing from the clone, so git diff base...head -- guide_extractor_dependencies.json exits non-zero and the script fails with guide_dependency_git_diff_failed.

Use the pull-request head checkout for review events, or restrict this step to pull_request events and enforce approval by head separately.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/backend.yml around lines 63 - 78, Update the “Guide
extractor dependency approval” workflow step so pull_request_review events
validate the pull-request head checkout rather than the default-branch ref.
Ensure the referenced WORKSTREAM_PR_BASE_SHA and WORKSTREAM_PR_HEAD_SHA commits
are available, including for fork PRs, before invoking
check_guide_extractor_dependencies.py with --require-pr-approval; alternatively
restrict this step to pull_request events and preserve separate approval
enforcement for review events.


- name: Install backend and exact Ruff
working-directory: backend
run: |
Expand Down
Loading
Loading