Skip to content

Reject un-set-up repositories and route recovery to appmap-setup - #15

Closed
evlawler wants to merge 1 commit into
mainfrom
claude/inspiring-tesla-0tlnv5
Closed

evlawler wants to merge 1 commit into
mainfrom
claude/inspiring-tesla-0tlnv5

Conversation

@evlawler

@evlawler evlawler commented Sep 23, 2026 •

Copy link
Copy Markdown

Problem

Field testing (setting up gold traces on Stirling, Windows) showed that when a user skips appmap-setup, every downstream piece goes down a failure path that does not name the real cause:

  • The gold-traces / review skills hit a missing gold_traces/manifest.yaml or appmap.yml and improvise — including reaching for shell text-editing commands that do not exist in the environment (Windows cmd.exe, restricted agents).
  • MCP servers started with appmap query mcp --appmap-dir … fail with "query DB not found … Run appmap index first", but indexing cannot help because nothing was ever recorded.

Setup must run first, and every other skill should reject a not-set-up state and route the user to appmap-setup instead of working around it.

Changes

Engine (manage.mjs)

  • New doctor command — a setup preflight. Checks, in order: the manifest, the nearest-ancestor appmap.yml, the record commands block, the AppMap CLI and its version (≥ 3.201.0, required by sanitize), then the curated entries and their committed baselines. Prints one line per check; exits non-zero on a hard failure with an explicit "run the appmap-setup skill" instruction and "do not create or edit this configuration by hand". An empty entries list exits 0 with a pointer to the gold-traces Bootstrap (that state is setup-complete by design).
  • New init command — seeds gold_traces/ (baseline/appmaps/ included) and writes manifest.yaml itself, taking the record configuration as flags (--framework, --runner, --args, or --record-command). No cp, no heredocs, no text editing. Refuses to overwrite an existing manifest.
  • The existing hard errors for a missing manifest, a missing appmap.yml, and an unconfigured commands block now name appmap-setup as the fix.

Skills

  • appmap-gold-traces: new "First, verify the repository is set up" section (run doctor, stop on failure); Bootstrap steps 1–2 now use init and defer to appmap-setup; the engine-command reference documents doctor and init.
  • appmap-review: same preflight before any compare; the ad-hoc two-file mode is documented as the one exemption.
  • appmap-setup: declared the entry point ("every other AppMap skill … refuses to run without it"); the pre-check and Phase 5 use doctor/init, and a green doctor is the definition of "set up". Removed the stale claim that the YAML parser can't read [].
  • appmap-record: documents the "query DB not found" MCP error and its real fix (record first, via setup).
  • README.md: states the ordering up front.

Tests — 8 new cases in manage.test.mjs covering init (seeding, overwrite refusal, flag validation, no-flags guidance) and doctor (missing manifest, unconfigured commands, full pass with stub CLI, version gate). All 55 pass, plus the frameworks/review suites (47) — same invocation as CI.

Risk analysis

  • doctor/init are additive commands; they dispatch before the manifest loads, so no existing command's control flow changes. The only behavior changes to existing paths are error-message texts (three sites), which nothing parses.
  • init's generated manifest is new YAML-writing code; free-form values (runner, args, record template) are always JSON-double-quoted, which is valid YAML for every edge case (leading -, : , #). Covered by a round-trip test through the real YAML reader.
  • doctor's CLI probe spawns <appmap_cli> --version without a shell; a .cmd shim on Windows PATH would not be found and would report the CLI missing (a false negative that still routes to the right fix). The standard installs (~/.appmap/bin/appmap(.exe), release binary on PATH) are .exe/ELF and resolve fine.
  • Stricter posture risk: sessions that previously "muddled through" without setup will now stop and ask for /appmap-setup. That is the intended change; the exemption for appmap-review's ad-hoc file mode is preserved.
  • The skills are installed from tagged releases, so nothing changes for users until the next release is cut.

Companion docs PR: getappmap/applandinc.github.io#1584 — "If setup was skipped" section in the gold-traces reference.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PL1Hzq3gd7XuTnz7pSmiCh

A repository where appmap-setup never ran fails downstream in ways that
do not name the real cause: missing manifests, missing appmap.yml, MCP
"query DB not found" errors, and agents improvising config with shell
text-editing tools that are not available everywhere the skills run.

Make the setup state an enforced precondition:

- manage.mjs `doctor`: a setup preflight that checks the manifest,
  appmap.yml, the record commands, and the AppMap CLI (>= 3.201.0), one
  line per check, and exits non-zero naming appmap-setup as the fix.
- manage.mjs `init`: seeds gold_traces/ (baseline/appmaps included) and
  writes manifest.yaml from --framework/--runner/--args or
  --record-command, so bootstrap needs no cp, heredocs, or text editing.
  Refuses to overwrite an existing manifest.
- Engine errors for a missing manifest, missing appmap.yml, and an
  unconfigured commands block now route to the appmap-setup skill.
- appmap-gold-traces and appmap-review SKILL.md gain a "verify the
  repository is set up" preflight; appmap-setup Phase 5 uses `init` and
  ends with a green `doctor` as the definition of "set up";
  appmap-record documents the "query DB not found" recovery; README
  states the ordering.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PL1Hzq3gd7XuTnz7pSmiCh
@evlawler evlawler closed this Sep 23, 2026
@kgilpin

kgilpin commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Why did you close this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants