Skip to content

Fix new-user friction: doctor diagnostics, reasoning_effort validation, quickstart, docs - #11

Merged
xuefei-wang merged 9 commits into
mainfrom
fix/new-user-friction
Jul 22, 2026
Merged

xuefei-wang merged 9 commits into
mainfrom
fix/new-user-friction

Conversation

@xuefei-wang

Copy link
Copy Markdown
Contributor

Improvements to the first-time-user experience, grounded in friction hit while following the getting-started docs end to end.

Code fixes

  • Doctor surfaces why a profile is unusable — it reported a generic "no usable key" even when a profile had a key but was missing MODEL_PROVIDER/MODEL. It now shows the specific ProviderConfigError per profile (e.g. .env.openai: Missing MODEL_PROVIDER).
  • REASONING_EFFORT is validated at profile load — an invalid value (e.g. minimal) previously 400'd every agent/forum/distill call with the error buried in the runtime sidecar DB. It's now rejected up front against {none, low, medium, high, xhigh}.
  • quickstart.sh honors TASKS_PATH — it was a hardcoded assignment while PROFILE/EXPERIMENT_NAME were overridable, so TASKS_PATH=... bash scripts/quickstart.sh silently ran the bundled demo. Now ${TASKS_PATH:-...} and documented.
  • Downgraded a benign cross-task "0 rows" WARNING to INFO — it fired on clean runs (forum off / gen 1 / all-solved) and read as an error.

New tests cover the doctor reason surfacing and the reasoning-effort validation.

Docs

  • getting-started: note explaining why the demo doesn't show the full loop (single generation, forums off, all-solved leads to early stop); expanded step 3 into per-task and cross-task forums; collapsed the verbose sample-output/artifacts/DB-check block into a details block.
  • nav: enabled pymdownx.details; promoted Getting started to a single link and moved Contributing to its own top-level entry (was nested under Getting started).
  • index: render the tagline emphasis with <strong> (the raw-HTML block wasn't processing bold markdown).
  • faq: dropped "benchmark" from the KSI definition; removed the --improvement-strategy Q and A.
  • Removed the TB2 Verifier Trust Boundary (architecture) and Removed compatibility flags (experiments) sections.
  • Pinned the header title to the site name so it no longer swaps to the current section on scroll.

Verification

  • uv run pytest on the affected suites (providers, doctor, cli, distill): 232 passed
  • uv run ruff check on changed Python: clean
  • mkdocs build --strict: clean

🤖 Generated with Claude Code

https://claude.ai/code/session_01XvzJBrjK196FnT8cG3hVFf

xuefei-wang and others added 9 commits July 21, 2026 16:57
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KhxqgzVNZB1P82smfaM2Lo
The doctor reported a generic "no usable key" even when a profile had a key
but was missing MODEL_PROVIDER/MODEL; it now surfaces the specific
ProviderConfigError per profile. load_provider_profile now rejects an invalid
REASONING_EFFORT (e.g. 'minimal') for openai up front, instead of letting every
agent/forum/distill call 400 with the error buried in the runtime sidecar DB.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvzJBrjK196FnT8cG3hVFf
TASKS_PATH was a hardcoded assignment while PROFILE/EXPERIMENT_NAME were
overridable, so 'TASKS_PATH=... bash scripts/quickstart.sh' silently ran the
bundled demo. Make it ${TASKS_PATH:-...} and document the knob.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvzJBrjK196FnT8cG3hVFf
0 cross-task posts is expected when the cross-task forum is off, at gen 1, or
when none were produced — it fired as a scary WARNING on clean runs. Downgrade
to INFO and clarify the message.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvzJBrjK196FnT8cG3hVFf
…tput details

Add a note on why the demo doesn't show steps 3-5 (single generation, forums
off, all-solved -> early stop). Expand step 3 into per-task and cross-task
forums. Move the verbose sample-output/artifacts/DB-check block into a
collapsible details block.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvzJBrjK196FnT8cG3hVFf
Enable pymdownx.details (collapsible blocks); promote Getting started to a
single nav link and move Contributing to its own top-level entry. Render the
landing tagline emphasis with <strong> (raw HTML block didn't process **).
Drop 'benchmark' from the KSI definition and remove the --improvement-strategy
FAQ. Remove the TB2 Verifier Trust Boundary (architecture) and Removed
compatibility flags (experiments) sections. Pin the header title to the site
name so it doesn't swap to the section on scroll.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvzJBrjK196FnT8cG3hVFf
…n guides

- CONTRIBUTING: cd ksi -> cd KSI (clone creates KSI/, errors on case-sensitive FS)
- CONTRIBUTING/extending: brand product name as KSI in prose/headings
  (code/import/path references to `ksi` left unchanged)
- improvement_strategies: behaviour -> behavior for spelling consistency

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@xuefei-wang
xuefei-wang merged commit f52ced4 into main Jul 22, 2026
3 checks passed
@xuefei-wang
xuefei-wang deleted the fix/new-user-friction branch July 22, 2026 04:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant