Repository navigation
docs: rewrite the README for first-time readers and AI assistants - #39
Merged
Merged
Conversation
Holdout-v1 was used to tune #30 and #33, so the chart's headline no longer measured unseen references. The chart now comes from the 99-item held-out v2 run on 1.3.1, and the retracted row says how many of its retractions CrossRef cannot mark, so the 8/8 rate and the 12-item bar no longer read as a contradiction. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first screen did not say what ref-verify is, who it is for, how to try it, or how to read a verdict; the reference-list check sat below the skill install, and reference material buried the story. The README now leads with the problem, a 30-second run on a labelled sample list with real 1.3.1 output, what each verdict asks of the reader (with how often UNVERIFIED was legitimate in the test sets), three ways to use it, the held-out scorecard, limits, privacy and network use, and the facts an agent needs. Commands, modes, error codes, the cache, scope, and examples move unchanged to docs/REFERENCE(.ko).md, and the doc tests now read them there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Within the first screen, the README did not answer what ref-verify is, who it is for, how to start, or how to read a result. The reference-list check, which is the feature most people want, sat below the skill install and usage. Reference material (modes, error codes, cache, scope) sat in the body and buried the story. This PR changes docs only; the engine is untouched.
What
README.md and README.ko.md
Both files have the same structure in the same order, so a reader can switch languages:
pipx install ref-verify(oruvx), thencheck-bibon the newexamples/references.txt. The README shows the real ref-verify 1.3.1 output. The file holds seven labelled references: three real, one real with the wrong year, one made up, one retracted, and one Korean thesis not in CrossRef. Together they produce all four verdicts.UNVERIFIEDbreakdown from the three test sets run on 1.3.1. Of 82UNVERIFIEDresults, 42 were made up and 36 were legitimate but not in CrossRef. Each set links to its results file.benchmarks/results/2026-10-09-8170e66-holdout-v2.json):crossref.py,openalex.py,semantic_scholar.py,pubmed.py,http.py, andcache.py.Other changes
docs/REFERENCE.mdanddocs/REFERENCE.ko.md: everything the README used to carry (skill install, request examples, check-bib details, CLI, modes, error codes, saved reports, cache, scope, what it catches, examples, development checks), moved verbatim except for heading renames.check-file'sROW_CHECK_ERROR.scripts/render_scorecard.pyfrom the held-out v2 results, with--devpointing at the v1 results from the same run. The renderer's retracted row now reads "n = 12 (4 unmarked)" and "8/8 marked ones caught", so the rate's denominator is visible.tests/test_skill_docs.py:check-bib, and--reportstay asserted in the README itself, and the Korean README is now held to the same.docs/REFERENCE.ko.md.benchmarks/README.mdnow says the README chart comes from held-out v2. CHANGELOG: an Unreleased "Changed" entry.Content removed on purpose
An automated diff of the old README sentences against the new README plus the docs found these drops. Everything else is present verbatim or under a renamed heading.
01c7a37(86 items, its numbers, its results-file links). It is superseded by held-out v2 on 1.3.1. The held-out v1 numbers now appear as in-sample results from 1.3.1, and the old results files stay inbenchmarks/results/.Checks
uvx --from ref-verify==1.3.1, a clean venvpip install ref-verify==1.3.1, andpipx install ref-verify==1.3.1(isolatedPIPX_HOME).blob/mainlink was checked: the target file exists in this branch and the anchor exists in GitHub's rendered HTML.docs/REFERENCE*.md,examples/references.txt, and its raw URL in the 30-second block return 404 until this merges.raw.githubusercontent.comURLs because the README is also the PyPI description; the test forbids relative.github/image paths. The scorecard SVGs at those URLs show the old chart until this merges.gh api markdownusingmode=markdownandcontext=Moonweave-Research/ref-verify.mode=gfmturns every source line break into a hard break, the way comments render, which a repository README does not do. The renders were wrapped in github-markdown-css and screenshotted in light and dark at 830 px and at a 390 px layout. They were reviewed by eye: code blocks scroll horizontally on phones, as on GitHub.PYTHONPATH=src python3 -m unittest discover -s tests: OK.python3 -m py_compile src/ref_verify/*.py tests/*.py scripts/*.py: OK.🤖 Generated with Claude Code