Repository navigation
feat(judge): return to the provider once a minute limit resets - #17
Merged
Merged
Conversation
A rate-limited Groq used to hand the rest of the run to the local model, even when its limit reset a minute later. A 429 whose Retry-After is at most 120 s (60 s when absent) now switches only until that time has passed; then one request probes the provider and, if it answers, the judge is back on it with a one-time notice. Nobody waits and the provider is never asked on every batch, so both goals of spec 11 hold. A daily limit, a 413, a key error and a server error stay sticky. Each completion now carries the client that answered it, so an opinion in flight across a switch or a return is named for the model that gave it. A dead local model is never announced as judging again, and the return notice says it failed rather than "stood in for 0 requests". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K46EEsxMvuAuM8wjvPNWLM
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Retry-Afteris at most 120 s (60 s when the header is absent) switches the judge to the local model only until that time has passed. The first request after that probes the provider. If it answers, the judge is back on it, and a one-time notice says so ("Groq answering again — local ollama qwen3.5:9b stood in for N requests"). Only one thread probes; the others keep using the local model until the probe has answered.Completionnow carries the client that answered it, so an opinion in flight across a switch or a return is named for the model that actually gave it (rule 6). Before, it took whatever the judge was routed to when the opinion was read.Retry-After: 0reads "~1s". The waiters of a re-switch are woken on the right event.docs/superpowers/specs/2026-10-05-judge-cooldown-design.md(amends evidence-search spec §11), OPEN-ITEMS §22 (with one follow-up left open on purpose), CHANGELOG, README.Why
The user asked whether the judge could go back to Groq once its minute limit resets, instead of staying on
qwen3.5:9bfor the rest of the run. Groq's free tier forgpt-oss-120bis 8K tokens per minute, about one full batch. They chose option A, a cooldown with return. Both goals of the original §11 decision hold: nobody waits, and the provider is asked again only once the time it named has passed.An independent review found three important bugs in the first version: a dead local model re-announced as judging, opinions misattributed in both directions, and a lost wake-up on
_ready. It also found four nits. A re-review confirmed the fixes and was clean.Checklist
ruff checkandruff format --checkare clean (andmypy --strict)pytestpasses (full suite; the threading tests ran 25 times without a flake)🤖 Generated with Claude Code
https://claude.ai/code/session_01K46EEsxMvuAuM8wjvPNWLM