Repository navigation
fix(server): Clear answers without source evidence - #454
Conversation
Validate extracted answers and explanations against verified source evidence at final merge. Preserve questions, source links and review flags without adding model calls, including checkpoint recovery and ZIP export. Co-authored-by: Codex <codex@openai.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a87f936027
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Preserve labeled answer-key entries and permitted table stem prefixes. Compare scalar answer text without numeric coercion so source spellings remain intact, and cover all three review findings with regressions. Co-authored-by: Codex <codex@openai.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8e21f30cc5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Accept terminal punctuation only for choice and boolean answers, require all corroborating source fields to agree, and restore stored headers for resumed tabular fragments. Cover all three additional review findings with regressions. Co-authored-by: Codex <codex@openai.com>
Integrate current source-consistency and OCR evidence handling before validating the combined import pipeline. Co-authored-by: Codex <codex@openai.com>
Retain source-conflict warnings before clearing unsupported published fields. Keep source-backed choice labels equivalent by identity regardless of order or case, and verify the combined import and grading-consistency path. Co-authored-by: Codex <codex@openai.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cea35e7943
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Require numbered entries before splitting an answer key, keep decimal answers inside their source record, and match unnumbered choice stems after verifying their printed option lines. Cover local labels, CRLF sources and ambiguous extra content. Co-authored-by: Codex <codex@openai.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aa318ceb96
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 905f7d088d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0b444e897c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 749f026ecb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Summary
Imports could retain model-inferred answers and explanations despite the extraction prompt prohibiting question solving. Validate non-null
answerPayloadandanalysisagainst verified source evidence during final merge, including resumed unit results. Clear unsupported fields independently while preserving questions, source associations, missing-field flags and human-review warnings.Cover text, Tencent OCR, native table rows and explicitly linked answer keys. Reject ambiguous numbering and evidence across failed pages. TXT/Markdown imports also check the original UTF-8 source when Tencent text differs. Add 205 provenance regressions and update synthetic fixtures so supplied answers actually appear in their source evidence; ZIP export retains nulls and review flags.
Preserve supplied answers from labeled or corroborating keys, local multiline fields, native tables, and numeric ordering/matching exercises. Normalize permitted type prefixes, terminal punctuation and table layout whitespace while preserving literal scalar spelling. Restore headers for resumed tabular fragments and resolve printed row identities before worksheet scoping.
Require stem prefixes and verified choice/item blocks for question association, including unnumbered exercises. Reject duplicate-key JSON, overly nested answer JSON, changed item identities and ambiguous extra content. Retain source-conflict warnings before unsupported fields are cleared.
AI assistance: Codex implemented the change, reviewed the complete diff and connected import/recovery/export paths, reproduced the original behavior with the baseline merge function, and ran local validation using provider/model substitutes.
Linked issue
N/A — focused bug fix; CONTRIBUTING.md permits a direct PR without a separate issue.
Affected areas
AI service document-import final merge, service tests and evaluation documentation. App/Web consume the existing null-answer and review fields; shared contracts and native application code are unchanged.
Validation
Choose relevant checks from CONTRIBUTING.md.
make verify AI_PYTHON=/Users/sheny/.local/share/uv/python/cpython-3.14-macos-aarch64-none/bin/python3.14749f026e: 3,301 tests, 94% total coverage; lock consistency, Ruff, Pyright, 30 evaluation fixtures, recovery probes, sdist/wheel build and package integrity. Reports:server/reports/checks/probes.xml,coverage.xml,probes.json.749f026e: 375 targeted tests, including 205 provenance cases. Added 39 cases for unnumbered structured answers, item-ID mappings, source-stem association, supported field labels, and duplicate/deep JSON. The existingtext-03.txtformat fixture also passes both actual text and OCR graph paths with model substitutes.git diff --check; main integration35206f13; verified that clearing unsupported published fields retains the upstream source-conflict warning and original source.749f026ehas been triggered; final results and fresh hosted review are pending. Fifteen reproduced and tested review findings are resolved. Local full-diff review and checks do not establish hosted CI or fresh-review acceptance.Risk and rollout
sourceText. No additional model calls, automatic retries, credential handling changes or provider charges.Checklist