Skip to content

63 claims across 5 records lost their evidence to #754 and need re-evidencing #765

Description

@realmarcin

Consequence of #754, recorded so it is not mistaken for curation that was never
done.

Removing five non-open-access article bodies left 78 evidence items
unverifiable
; they were deleted, and 63 claim blocks now carry no evidence at
all
:

record reference removed items dropped blocks left with none
Saccharomyces_Chlamydomonas_Fungal_Algal_Mutualism PMID:24994654 (Science) 17 15
Cheese_Rind_InSitu_InVitro_Model_Community PMID:25036636 (Cell) 21 17
hCom2_Complex_Gut_Microbiome PMID:36070752 (Cell Host & Microbe) 17 12
Propanotrophic_Chlorinated_Ethene_Cometabolism_Enrichment doi:10.1016/j.jenvman.2026.129957 13 10
Dehalococcoides_Pelobacter_Acetylene_TCE_Coculture PMID:28075122 10 9

The claims are still there and are not known to be wrong — they were curated from
text this repository may no longer hold. What is gone is the ability to show
that each came from its source.

What can be recovered, and what cannot

Each reference's abstract is cached and some snippets survived on it: 3 of 20,
2 of 17, 4 of 21, 16 of 29, 18 of 28. The abstracts are short — Science's is
950 characters — so most of what these records assert about methods and results
simply is not in them.

Three routes, in order of preference:

  1. Re-evidence from the abstract where it actually supports the claim. Cheap,
    and it is what the surviving snippets already do.
  2. Find an open-access source that reports the same finding — a preprint, a
    review, or a later OA paper by the same group. The claim then stands on text
    we can hold.
  3. Omit the claim or record uncertainty, per the evidence policy, where
    neither works. This is the correct outcome, not a failure.

Do not re-add the removed text to make the validator green; that is the thing
#754 exists to prevent, and tests/test_no_paywalled_text_in_repo.py now blocks
it.

Knock-on already recorded

tests/test_cultivation_source_studied_the_community.py can no longer score the
cultivation_setup of Cheese_Rind_InSitu_InVitro_Model_Community and
Dehalococcoides_Pelobacter_Acetylene_TCE_Coculture, because those blocks lost
every reference. Both are registered in _BLIND under "the cultivation_setup
cites no reference at all", with the reason spelled out — that check's usual
remedy is "cache the cited source's full text", which is precisely what is now
forbidden for these two.

Worth knowing for scoping

evidence_snippet_audit.py reports the corpus's pre-existing evidence backlog
separately (#596). This issue is a distinct, newly created gap with a known cause
and a known list, and should not be folded into that count.

Additional scope from the current-main refresh of PR #766 (2026-09-13)

PR #766 now also applies
the same publication-body policy to four local-file caches added on main after
the original removal. Their previously cached PubMed abstracts are retained.
The original issue text above is preserved as the original scope.

The refresh removes 12 additional evidence items from 10 claim blocks across
three further records
. Six of those blocks are left without any evidence;
the other four retain evidence for some claims, so 12 must not be added to the
original 63-block count. Scientific fields, taxa, interaction nodes and causal
edges are preserved. These additions require the same re-evidencing review.

Record References trimmed Evidence items removed Affected claim blocks Blocks left without evidence
Drosophila_FiveSpecies_Gnotobiotic_Gut_Microbiota PMID:24242251, PMID:25692519 6 6 4
Altered_Schaedler_Flora_Gnotobiotic_Mouse_Community PMID:26323627 5 3 1
Trichodesmium_Alteromonas_Marine_Consortium PMID:28440800 1 1 1

Affected Drosophila blocks are both bacterial taxonomy entries, both environmental
factors, growth medium, and the PMID:25692519 dataset. ASF loses review evidence
from oxygen niche partitioning, all evidence from the ASF356/ASF492 SCFA node,
and three growth-medium items; the independent PMID:15128534 niche evidence and
one husbandry excerpt remain. Trichodesmium loses its growth-medium evidence.

Public Europe PMC core metadata checked on 2026-09-13 reports
PMID:24242251,
PMID:26323627, and
PMID:28440800
as not open access.
PMID:25692519
has an open-access flag and identifies an NIH author manuscript, while the
removed cache was a publisher-version extraction with a rights assertion.
Publisher-deposited Crossref metadata
provides text/data-mining terms. A redistribution license for that cached
version was not verified; this records version/license uncertainty, not a legal
determination. No license exemption or test threshold was added.

The combined PR therefore removes 90 evidence items across eight records,
trims nine article caches to their retained abstracts, and deletes the
original publication PDF. Each of the additional 12 excerpts passed against
the former cache and failed against the retained abstract before removal.

Activity

  1. realmarcin commented on Oct 5, 2026

    @realmarcin
    ContributorAuthor

    Partial update from PR #1247, not a request to close this issue:

    • The existing ASF356/ASF492 fermentation graph node in Altered_Schaedler_Flora_Gnotobiotic_Mouse_Community now has a short primary-study excerpt from PMID:23828891. The claim is limited to the reported isolate-culture result; it does not establish measured in vivo cross-feeding or member-specific host mediation.
    • The ASF graph's measurement-workflow arrow and unsupported interaction types were removed. Spatial observations and general SCFA/Ffar2 host results remain, with unresolved oxygen and ASF-member mediation qualified explicitly.
    • No removed publication body was restored. The new Markdown cache is explicitly selected_excerpts, not a full-text cache. The independent snippet audit still reports one ASF mismatch because the retained independent cache is abstract-only; this gap is disclosed rather than converted into a clean full-text verification result.
    • ASF growth-medium evidence and the other records and claim blocks in 63 claims across 5 records lost their evidence to #754 and need re-evidencing #765 remain outside this graph-only batch. 63 claims across 5 records lost their evidence to #754 and need re-evidencing #765 and the separate snippet backlog 97 evidence snippets remain unverifiable, effectively all on closed-access sources #596 should remain open.

    Review decisions, source-access limits, and validation results are committed under reports/causal_graph_review/, including the batch4 decision ledger and self-adversarial review. This was not independent reviewer approval.

  2. realmarcin commented on Oct 5, 2026

    @realmarcin
    ContributorAuthor

    Partial cheese-rind graph remediation is in PR #1276. It adds only the retained abstract evidence for reproducible in-vitro succession, removes unsupported ecological typing, and qualifies the moisture/pH/succession summaries to their distinct assay memberships. It does not restore any removed publication body or claim that the current four-genus roster defines every experiment.

    The record is explicitly marked needs_research in the hash-bound review ledger. Moisture and deacidification nodes still lack adequate retained exact-system evidence; complete methods, figure images and membership reconciliation remain unfinished. This does not close #765 or certify unrelated media/aroma/cultivation claims.

  3. realmarcin commented on Oct 6, 2026

    @realmarcin
    ContributorAuthor

    Graph-only corrections published in #1393. This non-graph follow-up remains OPEN; raw metadata and caches were preserved, not certified by graph curation.

  4. realmarcin commented on Oct 6, 2026

    @realmarcin
    ContributorAuthor

    Graph-only repairs published in #1409. This follow-up remains OPEN: existing-structure correction does not certify non-graph metadata or finish new-structure research. #1408 retains needs_research and dry-run-only Edison status. No removed publisher bodies or metadata evidence restored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions