Skip to content

docs(readme): fix the swe-router video embed and regenerate the Step 1 chart - #40

Merged
shekharprateek merged 1 commit into
mainfrom
docs/fix-readme-video-and-regen-chart
Oct 20, 2026
Merged

shekharprateek merged 1 commit into
mainfrom
docs/fix-readme-video-and-regen-chart

Conversation

@shekharprateek

Copy link
Copy Markdown
Contributor

Description

Two fixes to the front page, plus the chart rebuild they forced.

The video

The asset was never broken — it still 302s to a valid mp4. The markup around it was. GitHub rewrites any video embed into its own <details> player, so the wrapper the README added could only get in the way. Checked against the /markdown API rather than by eye:

Form fed to GitHub What came back
bare asset URL <details> player
<video src=...> <details> player
plain markdown link <details> player

So the chrome is GitHub's either way, and the wrapper was doing three things, all bad:

  • <p align="center"> never applied. A block element cannot nest in a paragraph, so GitHub closed the <p> before the player and left a stray empty <p></p> behind it. The video was never centred.
  • The fallback was discarded. The rendered <video> body came back empty — the "Your browser cannot play this video inline" text and the download link were dropped outright. Dead markup that read like a working fallback to anyone maintaining the file.
  • width="820" was dropped for GitHub's own max-height:640px.

Five lines become one: the bare asset URL on its own line, the form GitHub documents. Same player, no broken wrapper, no dead fallback. The centred italic caption below it is untouched and still centres. markdownlint will flag MD034/no-bare-urls on that line — that is the intended form, and the file already carries MD033 warnings by design.

The Step 1 chart

Regenerated docs/images/cost-quality-abstracted.png from docs/diagrams/cost-quality-abstracted.html at 1600×1104 with a 2× device scale factor, which reproduces the committed 3200×2208 geometry exactly. The diagram also gained a data point that had not been rendered yet, and the rebuild was the moment to settle the naming:

  • Renumbered the closed-source labels to run over the full measured set: gen 3→gen 5, gen 4→gen 6, gen 2→gen 4, mid-tier gen 3→gen 5. The chart was the last artifact still calling closed-frontier-gen5 "gen 3", so one model now carries one name across the chart, vend/swe-router/models.json, allowed-models.txt and both swe-router evaluation docs.
  • Corrected the counts to twenty-three measured and eleven omitted. That number had lagged the data twice (nineteen, then twenty-one, against 23 rows in models.json).
  • Repointed the footnote at vend/swe-router/models.json instead of stating the numbering is local to the chart, which stopped being true.

One more README correction

The caption under the chart told readers to regenerate this diagram "with the chart scripts in benchmarks/scripts/". It is hand-maintained HTML, and those scripts plot from run metrics. The caption now carries the actual render recipe and sends the plot scripts to their real use.

Type of Change

  • Bug fix
  • New feature / enhancement
  • Documentation update
  • Refactor (no functional change)

Checklist

  • I have read CONTRIBUTING.md
  • Shell scripts pass bash -n syntax check
  • Python files pass python3 -m py_compile
  • I have updated documentation if needed
  • I have not included credentials, keys, or sensitive data

Verification

  • Every relative link in README.md resolves.
  • "Seven files" in Step 2 checks out against SKILL_FILES in vend/swe-router/install.sh.
  • Chart values cross-checked against models.json: $11.95/82.83, $4.44/81.16, $4.67/76.97, $5.32/74.69, $4.18/66.32.
  • cd benchmarks && uv run python -m unittest discover -s tests → 614 tests, one failure (test_judge_common.test_lowercase_preferred_on_collision), which also fails on a stashed tree: a case-insensitive-filesystem issue unrelated to this change. pre-commit passes.

One claim left alone: the Models-45 from 11 providers badge. litellm-mantle.yaml carries roughly 39, but the badge may be counting the repo's whole reach, so it needs an owner's call rather than a guess.

…1 chart

The video asset was fine; the markup around it was not. GitHub rewrites any
video embed -- bare URL, <video> tag, even a plain link -- into its own
<details> player, so the wrapper the README added could only get in the way,
and did three things:

  - <p align="center"> never applied. A block element cannot nest in a
    paragraph, so GitHub closed the <p> before the player and left a stray
    empty <p></p> behind it.
  - the fallback text and the "Download the swe-router explainer" link were
    discarded outright -- the rendered <video> body came back empty. Dead
    markup that read like a working fallback.
  - width="820" was dropped for GitHub's own max-height.

Replace all five lines with the bare asset URL on its own line, the form
GitHub documents. Verified against the /markdown API rather than by eye.

Regenerate docs/images/cost-quality-abstracted.png from the diagram source at
1600x1104 with a 2x device scale factor, which reproduces the committed
3200x2208 geometry, and bring the chart in line with the masked data:

  - renumber the closed-source labels to the full measured set, so one model
    carries one name across the chart, models.json, allowed-models.txt and
    both swe-router evaluation docs. The chart was the last artifact calling
    closed-frontier-gen5 "gen 3".
  - correct the counts to twenty-three measured and eleven omitted; the
    subtitle had lagged the data twice already.
  - point the footnote at vend/swe-router/models.json instead of saying the
    numbering is local to the chart, which is no longer true.

Also correct the caption under the chart: it told readers to regenerate this
diagram with the scripts in benchmarks/scripts/, but it is hand-maintained
HTML. It now carries the render recipe and sends the plot scripts to their
real use.
@shekharprateek
shekharprateek merged commit 14d5e9d into main Oct 20, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant