Skip to content

Exclude appended selected excerpts from trusted .txt snippet-cache coverage #1362

Closed

Description

@realmarcin

Finding

scripts/evidence_snippet_audit.py:207-210 treats every .txt cache as independently retrieved source text and returns before the selected-excerpt exclusions used for Markdown caches. Real PubMed abstracts with appended curator-selected excerpts therefore certify those excerpts as source coverage.

Reproduced read-only using the production cache_text("PMID:42613016") function at the published batch25 tree (commit 5f7e0faecd6486addf38d462234f7cb6067d29a4; relevant code and cache unchanged by the website integration):

trusted: True
SELECTED PUBLIC PMC ARTICLE EXCERPTS marker in trusted text: True
selected-only phrase in trusted text: True

The selected-only phrase was The initial inoculation ratio was set to 1:1 in model simulations. It occurs after the explicit selected-excerpts marker in references_cache/PMID_42613016.txt, not in its independently retrieved PubMed abstract. The issue is provenance, not a finding that the biological statement is false. No upstream reference-validator end-to-end reproduction or exhaustive affected-snippet count is claimed here.

A narrow ignored-inclusive scan of references_cache/*.txt also finds PMID_41381587.txt with SELECTED FULL-TEXT EXCERPTS FROM PMC12816627. These two marker matches are not an exhaustive inventory of all possible curated-excerpt formats.

Acceptance

  • Preserve the independently retrieved PubMed abstract, but exclude appended selected-excerpt sections from the snippet auditor's trusted source text.
  • Cover both observed markers and genuine unmodified abstract/full-text caches with focused regression tests; retain the Markdown selected-excerpt guard from Do not promote selected-excerpt caches to independent snippet verification #1237.
  • Re-audit all records referencing affected caches and report newly exposed source-coverage gaps honestly. Do not repair them by copying curated snippets back into trusted cache content or by weakening matching.
  • Keep historical review outputs as provenance, and distinguish primary-source reading from independent local cache verification in new review outputs.

Relationship To Earlier Work

Closed #1237 addresses Markdown content_type: selected_excerpts; this is the earlier .txt return path, which that fix does not reach. Closed #592 concerns a related circular-validation defect in snippet-listing caches. Closed #1019-#1022 and #1028 added selected excerpts for DCPP/RAMC records; they did not establish independent cache provenance. Closed #1147 concerns a full-text retrieval sentinel, not this audit trust decision.

Deduplication checked all 696 current open/closed issue titles and bodies plus ignored-inclusive scoped local reports, history, research, tests, and scripts. Every issue comment was not searched. Identified during preparation for the next causal-graph batch; not part of PR #1361, and not fixed by its graph-only changes.

Activity

  1. realmarcin commented on Oct 6, 2026

    @realmarcin
    ContributorAuthor

    Addressed in #1363 at d35e526e30385056f3b99c9a93c85a0d791b309b with an exact-head self-adversarial COMMENT review: #1363 (review). 60 passed in 131.45s (0:02:11); 324 passed in 188.25s (0:03:08); 460 records and 920 cache files byte-preserved; 641 of 643 cited-reference views unchanged. Both the original false trust and the malformed-boundary candidate defect were reproduced before correction. Actual post-commit docs-current passed. Source-coverage follow-up #596 is not closed by the validator repair; this issue closes only on protected main merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions