Skip to content

docs: open-source model support, egress-allowlist correction, nav regrouping - #12

Closed
xuefei-wang wants to merge 3 commits into
mainfrom
worktree-ancient-mixing-starlight
Closed

xuefei-wang wants to merge 3 commits into
mainfrom
worktree-ancient-mixing-starlight

Conversation

@xuefei-wang

Copy link
Copy Markdown
Contributor

Docs-only. Three independent changes, one commit each.

1. Document open-source / self-hosted model support

Nothing in the docs said which providers KSI accepts, or what running an
open-weight model would involve. MODEL_PROVIDER takes only anthropic and
openai — there is no vllm, ollama, or openrouter provider.

Adds a page covering the high-level shape rather than an implementation spec,
since this is a future contribution:

  • How you'd wire it — put a proxy speaking the Anthropic Messages format in
    front of the model and reuse the existing SDK path, rather than writing a
    provider adapter. The Claude Agent SDK honors ANTHROPIC_BASE_URL /
    ANTHROPIC_AUTH_TOKEN, and the runner already forwards environment into it.
  • Why it half-works today — host-side phases (forum, distillation,
    reflection) can be redirected right now; the containerized agent can't,
    because the base-URL variables aren't forwarded into the container.
  • What it affects — egress allowlist, prompt caching, cost reporting,
    structured output, and the direct adapters that bypass the SDK.
  • What to expect — multi-turn tool-calling robustness is the real risk, not
    the plumbing.

Includes a warning that setting a base URL today doesn't error — it splits
traffic
between the local server and the hosted API, which is the confusing
failure mode. A brief version lands in the FAQ.

2. Correct the egress-allowlist recipe in architecture

docs/architecture.md documented
KSI_EGRESS_ALLOW=host bash scripts/run_ksi.sh ... as the way to allowlist a
custom provider endpoint. That variable is only read from the TypeScript
runner's process.env, which the Python CLI builds from the provider profile
plus a fixed system-key list — so a shell-exported value never reaches
deriveEgressAllowlist and the documented form silently no-ops.

Found while researching change 1. Notes the constraint instead of leaving a
recipe that appears to work. No code change here; flagging it as a real gap.

The existing unit test passes env straight into derive(), so it exercises
the function but not the plumbing that feeds it.

3. Regroup nav so pages sit under what they're about

Four placements didn't match content:

Page Was Now
Running larger runs Programmatic API Getting started
Your own tasks standalone tab Getting started
Contributing Getting started Extending KSI
Benchmarks after Reference before FAQ

"Running larger runs" documents CLI flags and explicitly frames ksi.run(...)
as a layer over the parser it describes — it opens by naming its own place in
the progression after the quickstart and your-own-tasks walkthroughs. Grouping
those three makes the new-user path one sequence and drops the tab count 10 → 9.

Landing-page cards track the change (they were deliberately unified with the nav
tabs in 7f99358).

Verification

  • mkdocs build --strict passes (validates nav and internal links).
  • Page URLs derive from file paths, not nav position — /experiments/,
    /benchmarks/, /CONTRIBUTING/ all render at unchanged paths, so no links
    break.
  • No orphaned pages: every file under docs/ is reachable from the nav.

Claims about SDK behavior were checked against the shipped packages
(@anthropic-ai/claude-agent-sdk@0.1.77, @openai/agents@0.8.5) and the
half-works/doesn't-work split was confirmed by running the profile loader and
runner-env builder directly.

🤖 Generated with Claude Code

xuefei-wang and others added 3 commits July 21, 2026 21:18
`KSI_EGRESS_ALLOW` and the `*_BASE_URL` variables are read from the
TypeScript runner's `process.env`, which the Python CLI builds from the
provider profile plus a fixed system-key list. Neither is on either list,
so a value exported in the shell never reaches `deriveEgressAllowlist` —
the documented `KSI_EGRESS_ALLOW=... bash scripts/run_ksi.sh` form
silently no-ops.

Note the constraint rather than leaving a recipe that appears to work.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
KSI accepts only `anthropic` and `openai` as `MODEL_PROVIDER`, and nothing
in the docs said so or explained what running an open-weight model would
involve.

Adds a page covering the high-level shape: put an Anthropic-Messages-format
proxy in front of the model and reuse the existing SDK path rather than
writing a provider adapter; host-side phases can be redirected today while
the containerized agent cannot, since the base-URL variables aren't
forwarded into the container. Notes what it affects (egress allowlist,
prompt caching, cost reporting, structured output) and flags multi-turn
tool-calling robustness as the real risk. Left as a future contribution
rather than an implementation spec.

Also warns that setting a base URL today splits traffic between the local
server and the hosted API instead of failing loudly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Four placements didn't match content:

- "Running larger runs" sat under "Programmatic API", but it documents CLI
  flags and explicitly frames `ksi.run(...)` as a layer over the parser it
  describes. It opens by naming its own place in the progression after the
  quickstart and your-own-tasks walkthroughs — so it belongs with them.
- "Your own tasks" was a standalone tab despite being step two of that same
  progression.
- "Contributing" sat under "Getting started", where a first-time user meets
  PR conventions before running the demo. It's contributor material, and
  extending.md already points at it.
- "Benchmarks" trailed after "Reference" even though it's a usage guide for
  running the reference benchmarks yourself.

Groups the three-step new-user path under "Getting started", moves
Contributing under "Extending KSI", and lifts Benchmarks out of the
post-reference slot. Landing-page cards track the change (they were
deliberately unified with the nav tabs in 7f99358).

Page URLs are derived from file paths, so no links change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@xuefei-wang
xuefei-wang deleted the worktree-ancient-mixing-starlight branch July 22, 2026 04:40
@xuefei-wang

Copy link
Copy Markdown
Contributor Author

Superseded by #15 — this PR was auto-closed when the head branch was renamed to docs/open-source-models-and-nav, and GitHub won't reopen a PR whose head ref no longer exists. Same three commits, rebased onto #11.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant