Skip to content

E2E test: clean install and doctor in the user image (no LLM) #5231

Closed

Description

@Trecek

Maintainer setup

None beyond #5230. This test makes no model calls: no MiniMax, Anthropic or OpenAI key is used.

Depends on: #5229, #5230.

What it proves

A user installing AutoSkillit from scratch gets a working, self-consistent installation: in a fresh user container, autoskillit install succeeds and autoskillit doctor then reports no ERROR results.

Expected to fail today

Observed 2026-10-03 in a fresh container from the prototype image (AutoSkillit 0.10.1248, Claude Code 2.1.280):

  • autoskillit install exited 0 and printed Plugin published: autoskillit@autoskillit-local (scope: user), yet autoskillit doctor then reported autoskillit not registered, Plugin cache directory missing, Missing hooks and Hook registry has changed since last install. Run 'autoskillit install'.
  • Two ERROR results: installation_record_invalid (RECORD path ../../../bin/autoskillit, from the image's split UV_TOOL_DIR/UV_TOOL_BIN_DIR layout — check) and secret_scanning_hook (no .pre-commit-config.yaml in the empty working directory).
  • autoskillit doctor exited 0 despite those errors: the command never derives an exit status from severity (command). The test must gate on the JSON output, not the exit code.

Each real defect this test exposes gets its own bug issue; the test lands in the catalog marked expected-failure with a link to that bug until it is fixed.

Scope

  • Catalog entry: no LLM, runs whenever the selector picks it (cheap enough to also run on every PR touching src/autoskillit/cli/, src/autoskillit/workspace/ or scripts/docker/).
  • Steps: fresh container (no home volume) → autoskillit install → autoskillit init in a scratch git repository with a pre-commit config → autoskillit doctor --output-json.
  • Assertion: zero error results; each allowed warning is listed explicitly in the test with the reason it is expected.

Acceptance criteria

  • The test runs in CI in the user image and gates on doctor's JSON.
  • Every ERROR it reports today is either fixed or linked to an open bug.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestin-progressIssue is being actively worked onrecipe:implementationRoute through implementation recipe

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions