[shekharprateek]

Why

vend/swe-router/rubric.md states the six questions the router asks. Nothing in this repository said why those six. A reader who installs the skill can see the mechanism and has no way to evaluate the design.

This is the first of the rubric docs upstream has that we don't — 3 of 39 rubric-related paths came over in PR #33 (the rubric itself, swe_router_rubric.py and its tests); the 36 under docs/ and docs/metrics/ did not.

Not a verbatim port

Upstream wrote this page before the rubric shipped, so three passages describe as pending what PR #33 has since delivered. Each was verified against the shipped files before rewriting:

Upstream says	Reality on main
the shipped skill "still uses the older design, a consequence floor and a complexity tier", tag swe-router-floor-tier	The shipped skill is the rubric. That tag isn't in this repo's history.
route.py "knows four tier names rather than bucket names"	BUCKETS = ("easy", "moderate", "hard", "very_hard")
when_unsure "phase 1 implements that in the skill"	Live — --confidence low returns status unsure with no recommendation
"Which file holds what": an in-flight move between directories	Both files are vended now

Porting it as-is would have published a plan for a future that already happened.

The closing "What remains before it ships" section listed work PR #33 completed, so it's replaced by "What shipped" — plus the two caveats that didn't change by shipping: one repository, one run per model per task, and a saving measured against always using the costliest model.

Every number re-checked against the shipped policy

Before republishing figures I diffed them against vend/swe-router/routing-policy.json:

• bars 60 / 70 / 75, +5 when verification is 2
• buckets 0-1 / 2-3 / 4-5 / 6-8
• tie band 3 points
• "bucketing": "absolute"

That last one is also the page's own finding — relative bucketing and a per-model fitted line both failed to beat absolute buckets, because 21 tasks run once each can't separate them. Nice to be able to confirm the shipped config matches the conclusion.

Policy

Model names abstracted to class, per the convention the other analysis pages use, and the page states that where the figures appear. Zero real model names remain (checked). The three retained findings are the ones that bound how far to trust the router:

• a cheap judge scores the rubric about as well as a frontier one, at ~4% of the cost — the expensive model is for the work, not for describing it
• two refinements did no better than plain buckets
• difficulty costs models different amounts — which is why the bucket picks which column to read rather than raising the bar

Discoverability

Linked from the two places a reader would look: a documentation-map row in the root README beside the SKILL.md row, and the sentence in vend/swe-router/README.md that introduces the six questions without justifying them.

Checks

Repo-wide link sweep over 77 markdown files, resolving anchors as well as paths, reports only the pre-existing SUPPORT.md → ../../issues GitHub idiom. 635 tests pass, no new failures.

Still unported, for reference

-evaluation.md (162L, 12 model mentions) and -scores.md (138L, 5) are results pages needing a full abstraction pass; the 4 per-dataset score pages include two for the cli-cli-v1 dataset, which isn't here either; and the 25 replay JSONs under docs/metrics/swe-router-rubric-replays/ would re-introduce that directory wholesale.