Repository navigation
Conversation
The Grades tab reads the existing admin export for a fact-decomposition dataset and shows every reviewer's decisions on the model's claims: - Overview: one row per task with the source text color-coded by claim, how often humans disagreed with the model's grade, with each other's grades, and flagged a review issue, plus the claims reviewers added. - Task view: one row per claim with each reviewer's grade and review choice, the extraction-issue notes, and a "By reviewer" dialog. - Analysis: the questions the review data is for (extraction, atomic claims, duplicates, missing claims, grade agreement), each answered with the numbers behind it and how far the reviewers agreed. The Export tab gains "Download everything as one file". No server-side changes.
Analysis: a 3x3 grid of the model's grade against the humans' grades, one bar per task on the three review-issue cards (click opens the task), and a split agreement bar under every Human agreement heading. Overview: a thin meter under each rate tile, the review-issues meter split by flag. Counting changes: a claim two reviewers both added counts once; a review "issue" means one of the three flag buttons, so a grade change alone is a grade disagreement, not a review issue; claims made with Split into atoms are not counted as claims the model missed. Wording: one name per measure across tiles, filter chips, both tables and the Analysis cards; "Multiple Facts" spelled as the review page's button; the humans-vs-humans metric explained behind an info icon wherever it appears. Removed the row tint, the search box and duplicated bullets. Review choices live in one list (reviewOptions.ts).
Real questions are whole clinical vignettes and real answers run to several paragraphs, so one Overview row filled the screen. The Question and Source text cells now show their first few lines, with a Show more / Show less link that appears only when the text is cut off.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a Grades tab to the admin page (web/ frontend only, no server changes). It reads the existing admin export for a fact-decomposition dataset and shows what reviewers did with the model's claims:
Also adds "Download everything as one file" to the Export tab.
Tests:
bun run check(types, lint, 35 unit tests),bun run build, and a new Playwright spec,tests/admin-grades.spec.ts, which imports one row, reviews it, and checks all three views read it back. The full e2e suite passes on this branch (101 tests).