Skip to content

feat(judge): return to the provider once a minute limit resets - #17

Merged
Yigtwxx merged 1 commit into
mainfrom
feat/judge-cooldown
Oct 5, 2026
Merged

Yigtwxx merged 1 commit into
mainfrom
feat/judge-cooldown

Conversation

@Yigtwxx

@Yigtwxx Yigtwxx commented Oct 5, 2026

Copy link
Copy Markdown
Owner

What this changes

  • A short rate limit is no longer permanent. A 429 whose Retry-After is at most 120 s (60 s when the header is absent) switches the judge to the local model only until that time has passed. The first request after that probes the provider. If it answers, the judge is back on it, and a one-time notice says so ("Groq answering again — local ollama qwen3.5:9b stood in for N requests"). Only one thread probes; the others keep using the local model until the probe has answered.
  • Everything else is sticky, as before: a long 429 (daily limit), 413, 401/403, 5xx, a timeout, or no answer.
  • Honest attribution. Each Completion now carries the client that answered it, so an opinion in flight across a switch or a return is named for the model that actually gave it (rule 6). Before, it took whatever the judge was routed to when the opinion was read.
  • A dead local model stays dead. It is never re-announced as "judging", it is not warmed again, and the return notice says it failed instead of "stood in for 0 requests".
  • Retry-After: 0 reads "~1s". The waiters of a re-switch are woken on the right event.
  • Docs: spec docs/superpowers/specs/2026-10-05-judge-cooldown-design.md (amends evidence-search spec §11), OPEN-ITEMS §22 (with one follow-up left open on purpose), CHANGELOG, README.

Why

The user asked whether the judge could go back to Groq once its minute limit resets, instead of staying on qwen3.5:9b for the rest of the run. Groq's free tier for gpt-oss-120b is 8K tokens per minute, about one full batch. They chose option A, a cooldown with return. Both goals of the original §11 decision hold: nobody waits, and the provider is asked again only once the time it named has passed.

An independent review found three important bugs in the first version: a dead local model re-announced as judging, opinions misattributed in both directions, and a lost wake-up on _ready. It also found four nits. A re-review confirmed the fixes and was clean.

Checklist

  • ruff check and ruff format --check are clean (and mypy --strict)
  • pytest passes (full suite; the threading tests ran 25 times without a flake)
  • Type annotations on every new function signature
  • No network access in unit tests, no real sleeps (injected clock)
  • If a new reported state was added, spec section 15 is updated too (no new verification state; new judge notices only)
  • This change does not make the tool sound more confident than its evidence

🤖 Generated with Claude Code

https://claude.ai/code/session_01K46EEsxMvuAuM8wjvPNWLM

A rate-limited Groq used to hand the rest of the run to the local model,
even when its limit reset a minute later. A 429 whose Retry-After is at
most 120 s (60 s when absent) now switches only until that time has
passed; then one request probes the provider and, if it answers, the
judge is back on it with a one-time notice. Nobody waits and the
provider is never asked on every batch, so both goals of spec 11 hold.
A daily limit, a 413, a key error and a server error stay sticky.

Each completion now carries the client that answered it, so an opinion
in flight across a switch or a return is named for the model that gave
it. A dead local model is never announced as judging again, and the
return notice says it failed rather than "stood in for 0 requests".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K46EEsxMvuAuM8wjvPNWLM
@vercel

vercel Bot commented Oct 5, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
proofpath Ignored Ignored Oct 5, 2026 12:54pm UTC

@Yigtwxx
Yigtwxx merged commit 09a4970 into main Oct 5, 2026
11 checks passed
@Yigtwxx
Yigtwxx deleted the feat/judge-cooldown branch October 5, 2026 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant