Repository navigation
fix: fix the leftovers that the review of questions as steps found - #38
Merged
Merged
Conversation
- The viewer marks a step whose request failed, or whose case stopped, as failed, with the reason; before, it showed "The model is answering…" for good. - The viewer finds a pick by the facts that `picking` writes, with the new `pickKeys` that the replay check also uses; a choice question that is no pick is no longer charted as one. `FrameChoice.key` is gone. - WikiRace ends at a dead end without a request to give up, as an unfinished race with the reason "dead end". - rubric-judge ends the gathering in its state at the judge's limit, so a fork before the checks asks only the checks, and goes to the checks at once when only stop is left. Its outcome loses `steps`. - Fix the trace and result comments, the package description, the path probability in docs/MATH.md and the plan, the viewer README, and the README's list of examples; remove the test of the removed flag.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A review of #37, by a reviewer that did not write it, found bugs and leftovers of the old interface.
This PR fixes them.
The main bug: the viewer showed "The model is answering…" for good on a step whose request failed.
What Changed
Most fixes are small. Two change behavior in the examples.
Frame.failed).pickingwrites. The new core helperpickKeysdoes this, and the replay check uses it too. A choice question that is no pick, such as the README'steam, is no longer charted as a pick. The unusedFrameChoice.keyis removed.unfinishedwith the reasondead end. Before feat!: make every step a set of questions #37, such a race also ended without a request.steps, which had a second meaning next toresult.steps. The README says to fork atsteps - 1.src/cli/layout.ts, the package description, P(path) indocs/MATH.mdand the plan (choice answers only, because a bool answer names no label), the viewer README, the README's list of the three examples, and a test that named the removed--state-fromflag.Testing
All tests use the fake decider; no model was called.
pnpm checkat the root passes. Tests: root 71, viewer 39, chess 75, rubric-judge 49, WikiRace 20.pickKeys; the failed-step panel; a thrown case's step; the WikiRace dead end; the judge's fork after its limit; the judge with only stop left.Risks
The risk is low. These changes follow the design of #37, and the full check passes.
gathered: false. A fork of such a trace under a judge with a higher limit can still gather more. No real runs exist with feat!: make every step a set of questions #37 yet.