Skip to content

add reproducible pdf and ocr diagnostics - #32

Open
NisargOza wants to merge 2 commits into
MedARC-AI:mainfrom
NisargOza:pdf-ocr-evaluation
Open

NisargOza wants to merge 2 commits into
MedARC-AI:mainfrom
NisargOza:pdf-ocr-evaluation

Conversation

@NisargOza

@NisargOza NisargOza commented Oct 9, 2026 •

Copy link
Copy Markdown

Add a reproducible PDF/OCR diagnostic using two six-page WHO excerpts with source SHA-256 hashes, raw backend outputs and seven source-checked anchors per excerpt. The report records native PDFium, Tesseract and the AMFV Docling export results, including the detected date-column footnote that the original Docling markdown export omitted.

  • Add a standalone runner for native text, Tesseract and Firecrawl API v2 auto/forced-OCR modes, plus optional integration with the AMFV Docling exporter.
  • Declare a lightweight ocr dependency group for PDFium and Pillow. Tesseract remains a separately installed executable. The Docling integration requires adopt who guideline scraper with optional pdf extraction #31; until it is installed, the runner records an explicit skip.
  • Retain reviewed October 6 raw evidence and input hashes, preserve raw whitespace through scoped Git attributes, and ignore new local result directories by default.
  • Credit Ashu's original WHO fixtures from add who guideline scraper with pdf full text extraction #6 and retain WHO attribution/reuse terms. The fixture PDFs and attribution are identical to those in the WHO update, so either PR can supply them. This PR can merge independently for native text, Tesseract and Firecrawl scaffolding.
  • Document the method, table/status observations, limitations, and open OCR candidates. Anchor presence is not clinical accuracy, CER/WER, recall or proof of table alignment.

The full Firecrawl–FinePDFs comparison remains pending. No Firecrawl call was made because credentials were unavailable; FinePDFs was inspected but not executed. The stock/AMFV Docling measurements are not presented as FinePDFs results. Live Firecrawl integration remains untested.

Validation: uv run ruff check ., uv run ruff format --check ., uv run --group ocr pytest -q (49 passed), uv build --package amfv-datasets, source-hash/raw-output-count verification, and a fresh native/Tesseract runner smoke check on both fixtures. The smoke check recorded explicit skips for unavailable Docling integration and missing Firecrawl credentials. Public command tests cover raw-text/provenance output, missing credentials, and invalid raster resolution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant