Classify PDF fi ligatures as rendering differences in snippet audit #1863

Reproduction

The unchanged references_cache/PMID_38609451.md has a PDF ligature in speci fic and a line-break in attri- buting (lines 693-694). The primary XML has the contiguous sentence: attributing core metabolites to a specific co-culture partner is not possible.

scripts/evidence_snippet_audit.py reports MISMATCH (coverage 0.0) for that excerpt in the new batch110 discussion of Synechococcus_Pseudomonas_PhotoPHA_DNT_Coculture.yaml. The full-record linkml-reference-validator 0.1.7 check passes. Independent NFKD/case/punctuation/whitespace normalization confirms a contiguous literal match to both the fresh primary XML text and the unchanged cache.

Cause And Scope

alnum() lowercases/transliterates Greek and then strips non-ASCII characters without Unicode compatibility normalization, deleting the fi ligature rather than expanding it. This turns a faithful rendering difference into a fabrication-suspect MISMATCH. Related prior work: #257, #596; neither was found to specifically track this ligature case. Fresh all-state title/body search for ligature/NFKD/Unicode normalization matched only closed #3; local search included hidden and ignored files in scripts/tests/reports. PR comments were not exhaustively searched.

Acceptance

• Reproduce the ligature case with a focused regression and classify it as RENDERING, not MATCH.
• Preserve rejection of noncontiguous/assembled or substantively different text.
• Review Unicode-normalization scope explicitly; do not rewrite source caches or propagate PDF artifacts into curated excerpts to satisfy the audit.
• Compare both-root corpus classifications before and after the implementation.

Batch110 records the exact known false positive and independent proof; it does not change the shared auditor or claim the full-corpus snippet audit is clean.