fix: keep check-bib off same-title records and read Hangul titles as titles

[moonweave]

Why

The E2E evaluation (#37) and holdout-v2 found two engine problems.

1. The search path attaches a reference to the wrong CrossRef record, so the reader is sent to another paper or a retraction is missed. Observed cases:

• Wakefield 1998 without a DOI matched a same-title 1998 Lancet letter (Sabra et al.). It got "first author differs" instead of PAPER_RETRACTED.
• A fabricated JCIM reference matched an SI component (type: component, .s003).
• Feynman Lectures (a book) PASSed on a 1964 Physics Today review that carries the book's title.

2. A Korean reference with a title was treated as title-less, because only Latin words were counted. A DOI swap was then explained as "The reference has no article title".

What

Fix 1: fix: keep search-path matches from landing on a same-title record

Candidates that cannot be the cited work are dropped:

• component and peer-review records.
• Records about another work. These now include replies, quoted comments, and retraction, correction, or expression-of-concern notices ("Retraction—<title>", "Authors' reply", 'Comment on "…"').
• A retracted paper's own "RETRACTED:" title is not dropped, and neither is an ordinary title such as "Response to selection …".

A reference that reads as a book matches only book records (book, monograph, edited-book, chapters, …), never journal reviews of the book. A reference reads as a book when:

• it is BibTeX @book/@inbook/@incollection or RIS BOOK/CHAP; or
• it is plain text that names a publisher or an edition and has no journal volume or page.

A partial title match needs an agreeing first author. A title that only mostly appears in plain text (≥ 80% of its words) now counts only when the first author also agrees. A title found whole still counts with a different first author, so chimera references keep their "first author differs" WARN.

Matching candidates are ranked instead of taken in search order:

1. First-author agreement.
2. Type fit: posted-content first when the reference says preprint/arXiv/ChemRxiv/…, otherwise posted-content, reference-entry, and dataset last.
3. Exact year.
4. Container agreement.
5. Search position.

Fix 2: fix: count Hangul and CJK words when deciding a reference has a title

• The title-less checks (_titleless_differences, _looks_titleless) count Hangul words and Han/kana runs, one per four characters, in addition to Latin words.
• A leading list of Hangul or Han author names ("최수아, 정민호") is left out of the count.
• When CrossRef stores a Korean title (original-title) for the DOI, a Korean reference's title is compared with it by syllable pairs, so particles such as "하이드로젤의" still match.
  • Below 50% overlap, the reason is the existing "This DOI belongs to …, which this reference does not mention".
  • Without a Korean title in CrossRef, the result stays "could not confirm", as before.

Tests and data

• Unit tests use real CrossRef search and work responses, trimmed, in tests/fixtures/crossref/wrong_record_cases.json. The tests make no network calls.
• Positive control: run against main's engine, 14 of the 15 new tests fail and 1 errors (_script_title_words does not exist there). The controls pass on both engines: a cited letter, an edited-book record, a cited book review, a Korean citation without a title, and a Korean title with spacing variants.
• benchmarks/references-regress-v3.jsonl (23 items) was frozen in 6f4386b before any run. It contains the three failure types and Hangul-title cases, plus close-call controls where the cited record really is a letter, a review, a book, or a preprint. It is a regression set, not a held-out score.
• benchmarks/results/2026-10-09-{6f4386b,fe8e810}-*.json: before and after runs of all four sets, each on a fresh cache, REF_VERIFY_MAILTO unset.

Before / after

regress-v3 (frozen before the runs; n per category in parentheses)

	Before (6f4386b)	After (fe8e810)
Real PASS (12)	10	12
Real PASS on the wrong record	1 (r3-ctrl-07, a preprint resolved to the journal version)	0
Retracted caught (1)	0 (Wakefield → Sabra letter, WARN)	1
Fabricated flagged (6)	6 (2 attached to SI components)	6 (none attached to a record)
Unindexed books, PASS or WARN on a journal review (4)	4 (2 PASS, 2 WARN)	0 (all UNVERIFIED)
Korean DOI swap reported as "no article title"	2 (r3-hangul-01, -06)	0 (both "This DOI belongs to …")

Regression check on the earlier sets. No set gained a fabricated PASS or a REJECT on an unindexed reference, lost a real PASS, or lost a PAPER_RETRACTED.

Set	Before	After	Changed items
holdout-v2 (99)	real 40/45, fab 29/29, retracted 8/8, unindexed REJECT 0/13	same	h2-notincr-05 Feynman: PASS on the review → UNVERIFIED; h2-real-17: WARN on an unrelated record → UNVERIFIED
holdout-v1 (86, in-sample)	real 40/40, fab 26/26, retracted 10/10, unindexed REJECT 0/10	same	2 unindexed items: WARN on a wrong record → UNVERIFIED
v1 (142, in-sample)	real 66/66, fab 43/43, retracted 16/16, unindexed REJECT 1/17	same	3 fabricated items: WARN on a wrong record → UNVERIFIED

• Cold-run time is unchanged: holdout-v2 101 s before, 99 s after.
• No code was tuned on holdout-v2; it was only run to check for regressions. One holdout-v2 item, Feynman (h2-notincr-05), was among the reported failure cases, so that item is no longer a clean held-out measurement.

Risks

• Book detection is a heuristic. A plain-text journal reference that names a publisher-like word and has no volume or page would match only book records. The result is UNVERIFIED (WARN) instead of PASS, never REJECT.
• Fewer resolved DOIs. A reference whose title only partly matches and whose first author cannot be read from the text now stays UNVERIFIED instead of "resolved to X, but the first author differs", so some correct partial matches lose their suggested DOI. Five items across the sets moved this way, all from a wrong record.
• Hangul comparison: a heavily reworded Korean title that still cites the right DOI could be reported as naming another paper (WARN, never REJECT). The leading-name rule assumes names of 2–4 syllables separated by commas.
• CJK counting (one word per four Han/kana characters) is unit-tested only; none of the sets has a Chinese or Japanese reference.
• About-other-work titles: a real paper whose title starts with a quoted "Comment on '…'" or with "Authors' reply" would be skipped.

Verification

• PYTHONPATH=src python3 -m unittest discover -s tests: OK.
• python3 -m py_compile src/ref_verify/*.py tests/*.py scripts/*.py: OK.
• Before/after benchmark runs as above.

🤖 Generated with Claude Code