Skip to content

feat(documents): add managed local OCR processing - #283

Merged
forhappy merged 2 commits into
mainfrom
codex/022-quality-document-ocr
Aug 23, 2026
Merged

feat(documents): add managed local OCR processing#283
forhappy merged 2 commits into
mainfrom
codex/022-quality-document-ocr

Conversation

@forhappy

@forhappy forhappy commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add bounded, versioned native processing for PDF, DOCX, PPTX, and XLSX with structural graph publication and gap-free semantic slicing
  • add optional local PP-OCRv6 OCR with deterministic preprocessing, geometry/provenance validation, partial-coverage diagnostics, and strict cache identities
  • add Compass-managed pinned model install/list/verify commands so users do not need Python, Tesseract, LibreOffice, Poppler, or another system OCR runtime
  • harden mutable source/cache reads with stream-level limits, reserve aggregate raster work before rendering, validate OCR origin/profile/completeness coherence, and check deadlines around every inference unit
  • serialize concurrent model installs with a bounded per-profile lock; reject unsafe URLs, control-bearing output, symlinked model state, corrupt markers, and non-canonical digests
  • expose document inspect and extract OCR flags, plus compatibility, security, performance, migration, qualification, and user documentation

Verification

  • cargo fmt --all -- --check
  • cargo test --workspace --lib --bins --locked
  • cargo test -p compass-files -p compass-ocr -p compass-media --locked (94 passed; installed-model acceptance remains opt-in)
  • cargo test -p compass-cli --test document_cli --locked (5 passed)
  • cargo clippy -p compass-files -p compass-ocr -p compass-media -p compass-core -p compass-cli --lib --bins --all-features --locked -- -D warnings
  • cargo clippy -p compass-ocr -p compass-media --all-targets --all-features --locked -- -D warnings
  • ./scripts/qualify_document_ocr_v1.sh --fixtures-only
  • scripts/check_product_boundary.sh

Tests used a clean worktree and a dedicated CARGO_TARGET_DIR on /Volumes/Workspace. The CLI link emitted the existing macOS __eh_frame size warning but completed successfully.

Qualification status

Plan 022 remains IN PROGRESS rather than DONE. The checked-in qualification manifest explicitly blocks a release quality claim until comparative Tesseract/ocrs measurements, x86_64 equivalence, multilingual/degraded and hostile-corpus measurements, and prerequisite plan review are complete. Extraction is local and usable now, OCR stays off by default, and enabling OCR never silently downloads a model.

@forhappy
forhappy force-pushed the codex/022-quality-document-ocr branch from 04e324e to cecac26 Compare August 23, 2026 20:59
@forhappy
forhappy force-pushed the codex/022-quality-document-ocr branch from 67e18f8 to f8fc697 Compare August 23, 2026 22:27
@forhappy
forhappy merged commit 52a026b into main Aug 23, 2026
6 of 14 checks passed
@forhappy

Copy link
Copy Markdown
Contributor Author

OCR completeness audit (2026-08-23): compared the original local OCR implementation commit and all OCR-related commits, tracked paths, and uncommitted paths in this repository against the current PR head. No OCR changes are missing. This PR includes the native PDF/DOCX/PPTX/XLSX processing pipeline, managed PP-OCRv6 runtime and models, CLI/core/media/semantic/history integration, bounded and secure processing hardening, tests, qualification scripts, compatibility/security/performance/migration documentation, and user guidance. The unrelated dirty React/framework changes in the original checkout were intentionally excluded.

@forhappy

Copy link
Copy Markdown
Contributor Author

Follow-up from the completeness/CI audit: pushed commit 74138ca. It adds the crate/version-scoped BSL-1.0 policy for OAR OCR geometry, records the newly published no-safe-upgrade ttf-parser advisory with a bounded rationale, bundles the exact Boost 1.0 license in releases, and repairs the post-rebase GraphViewModel/Rust 1.97 base-branch compile and lint failures. Local cargo-deny, full workspace Clippy/tests, product boundary, and document OCR fixture qualification all pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant