Repository navigation
Send CJK-script references to the fine-tuned reference review - #212
Open
thesanogoeffect wants to merge 2 commits into
Open
thesanogoeffect wants to merge 2 commits into
thesanogoeffect wants to merge 2 commits into
Conversation
The review gate's coverage check reads an unspaced CJK run as one word, so a v5.1 parse that lumps authors, title and journal into one span passes every source check and skips the review. v4.5's weaker parses were flagged and rescued by it; v5.1's are not. Flag references containing Han, kana or Hangul as cjk_script, treat it as an input signal like long_reference, and keep the review's parse for such references when no source check gets worse.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #212 +/- ##
=======================================
Coverage 90.56% 90.57%
=======================================
Files 309 309
Lines 57337 57346 +9
Branches 10383 10386 +3
=======================================
+ Hits 51929 51940 +11
+ Misses 3449 3448 -1
+ Partials 1959 1958 -1 🚀 New features to boost your workflow:
|
thesanogoeffect
marked this pull request as ready for review
October 18, 2026 16:16
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Before: with the v5.1 parser, Chinese references often skip the fine-tuned review. On unspaced full-width references (GB/T 7714 style), v5.1 puts the authors, title and journal all into the title and leaves the authors empty. That parse passes every source check, because the checks only test that each field occurs in the reference string, not that it is assigned right. v4.5 failed worse on these references and usually lost the year too, so
missing_printed_yearsent them to review and the review rescued them. On the sealed blind set, zh reference matching F1 dropped from 0.94 to 0.71 (11 papers), while review calls fell from 1,154 to 743.After: any reference with Han, kana or Hangul gets a
cjk_scriptflag and goes to review whenREF_LLM_BASE_URLis set. For those references the review's parse is kept whenever no source check gets worse. Coverage can't tell CJK fields apart, so requiring a strict gain would reject a correct split.How:
cjk_scriptjoinslong_referenceas an input-level signal thatimproves()ignores when it compares flag sets. Latin-script references keep the old rule (strictly fewer flags or higher coverage). Supported title, author and year evidence still can't disappear, and the per-list review cap is unchanged.Checked on non-sealed references with the ONNX parser and the default routing settings. No LLM calls were made, so this measures routing only:
On synthetic full-width references, v5.1 gets the title right on 0 of 24 (v4.5 on 4). On spaced Chinese references, real mixed-script rows and the non-CJK controls, v5.1 is better than v4.5. There are no real pure-Chinese references outside the sealed set.
Possible follow-ups, not in this PR: a general
missing_authorsflag, or normalising CJK punctuation and spacing before parsing (24 of 24 titles right on spaced input), plus real GB/T 7714 training rows.