Skip to content

feat(workflow): choose agent and model separately, ACP as the default runtime - #71

Open
FreshlyBrewedCode wants to merge 1 commit into
acp/adapterfrom
acp/authoring
Open

FreshlyBrewedCode wants to merge 1 commit into
acp/adapterfrom
acp/authoring

Conversation

@FreshlyBrewedCode

@FreshlyBrewedCode FreshlyBrewedCode commented Oct 2, 2026 •

Copy link
Copy Markdown
Owner

What

This PR implements ADR 0013 §2–§4 on the authoring and config surface, and makes the ACP adapter (#63) the default runtime.

  • agent?: "claude" | "opencode" next to model at every level:
    • AgentCallOptions and the workflow agent defaults;
    • schedule agent;
    • the run request: POST /api/runs takes agent: { agent?, model? } (validated, 400 otherwise), and factory start takes --agent and --model.
  • config.agent has default, models and hostSettings (default "ignore", per agent). It is validated at load: unknown agents, empty models and bad hostSettings values throw and name the field. Workflow and schedule agent choices are validated at load too. adapter is still the injection point for replay and fakes.
  • The resolution rule lives in src/runtime/agent-choice.ts. The levels, most specific first, are call > run (request or schedule) > workflow > config.
    • Agent: the most specific level that names one, else agent.default.
    • Model: the most specific level that names one, counting only the level that chose the agent and the levels more specific than it. Otherwise agent.models[agent].
    • A step whose model resolves to nothing fails before AgentStepStarted and before any agent process starts. The message names the fix: agent step "implement": no model for agent "claude": name one with \model` on the ctx.agent call, the run, the schedule or the workflow, or set `agent.models.claude` in factory.config.ts`.
  • Per-call dispatch:
    • AgentAdapterOptions gains agent, resolved like model.
    • The AgentRuntime service now carries the config rung (defaults: { default, models }) beside the adapter. run.ts resolves the choice from Effect context (ADR 0012 §4), and the boundaries stay plain async.
    • The default adapter, acpAgentsAdapter(hostSettings), keeps one acpAdapter per agent definition, built on first use, and dispatches on options.agent.
    • An injected adapter (replay, fakes) receives the agent and may ignore it. So there is still exactly one adapter seam, and config.agent.adapter keeps working for every agent.
  • hostSettings feeds acpAgent(kind, { hostSettings }).
  • Removed DEFAULT_MODEL (Add a config-level default agent model #40) and permissionMode (permissionMode is declared on the authoring surface and read by nobody #39). The commit message records why.
  • AgentStepStarted.agent is optional, so old logs have none. Run detail shows agent above model, and the event summary reads implement · claude · haiku.
  • factory init writes agent.default: "claude" and models: { claude: "sonnet", opencode: "opencode/big-pickle" }.
  • This repo's .factory/factory.config.ts gets the same models, but with default: "opencode" (see Deviations). implement-parent-issue drops its own hardcoded omniroute/… model. That model is a host-config provider, and hostSettings: "ignore" hides it. The workflow now takes an optional agent / model input.
  • Daemon start runs checkAgents(configuredAgents(config)) in the background and logs each problem. The configured agents are the default, every agent with a model, and every agent a workflow or schedule names. The check is skipped when an adapter is injected.

Fixes to the layer below (#69, branch acp/adapter)

The live runs turned up two defects in the adapter. Both are fixed on that branch, and the stack is rebased:

  1. fix(runtime): send ACP an absolute cwd for a relative workspace. ACP's session/new requires an absolute cwd, but the default workspace root is relative (.factory/workspaces). As a result, every daemon-allocated Claude step failed with "cwd must be an absolute path". The fake ACP agent now validates cwd the same way claude-agent-acp does. The new test fails without the fix and passes with it.
  2. fix(runtime): always pin the session's model over ACP. With host settings ignored, claude-agent-acp still reports the user's model setting (haiku on this host) as the current model, while the Agent SDK runs its own default. The adapter skipped set_config_option when the requested model equalled that reported value. So a step asking for haiku actually ran on a 1M-context model, costing 0.080 USD for 330 output tokens, and the model said it was "Claude Opus 5.5". After the fix, the same run reports a 200k window, costs 0.050 USD, and the model says "I'm Claude Haiku 4.5".

Testing

  • bun run check is green: 429 tests on this layer, and 378 on acp/adapter alone.
  • nix develop --command bun run test:e2e: 15 passed.
  • New tests:
    • agent-choice.test.ts: a 15-row table for the resolution rule. It covers carry-across, the chooser's own model, the run level, the config-less case, an unknown agent and a missing model. The same file checks the removals: DEFAULT_MODEL is absent, and @ts-expect-error guards permissionMode.
    • run.test.ts: the adapter receives the resolved agent and model, and AgentStepStarted records both. Also the run level, and an unresolvable model that fails with no AgentStepStarted and no adapter call.
    • agent-runtime.test.ts: acpAgentsAdapter launches the resolved agent per step over the fake ACP agent, on the right model, with per-agent hostSettings. Also the runtime defaults.
    • config.test.ts: defaults, carry-through, six rejection cases, workflow and schedule validation, and configuredAgents.
    • http.test.ts: the request agent picks that agent's configured model; the request model wins; there are three 400s.
    • daemon.agents.test.ts: the check runs with the configured agents, is skipped for an injected adapter, logs each problem, and survives a failing check.
    • cli-argv.test.ts: --agent / --model, and an unknown --agent is rejected.
    • init.test.ts: the generated config loads, and its agent block validates.
    • events.test.ts: AgentStepStarted.agent round-trips, and an old log without it still decodes.
    • run-events.test.ts: the SPA fold and the event summary.

Live validation

The setup is a scratch project: a local bare origin, a slug that is not on GitHub, and a workflow with no write-back. Its config is agent: { default: "claude", models: { claude: "haiku", opencode: "opencode/big-pickle" } }, and factory serve runs on port 3017.

run AgentStepStarted result
factory start hello-local --watch (default agent) agent: "claude", model: "haiku" completed in 7.8 s. acp.usage reports a 200k window and 0.050 USD. The model replied "I'm Claude Haiku 4.5". math.ts changed
factory start hello-local --agent opencode --watch agent: "opencode", model: "opencode/big-pickle" (from agent.models, not haiku) completed in 25.6 s. A 200k window, 0 USD. The model replied "I'm big-pickle". math.ts changed
factory start hello-local --model no-such-model --watch agent: "claude", model: "no-such-model" failed in 0.76 s, before the prompt: claude has no model "no-such-model" (12 offered, e.g. default, opus, …)
factory serve with PATH=/usr/bin:/bin — the daemon logs agents: opencode is configured but cannot run on this host: \opencode` is not on PATH`

Deviations

  • No built-in model fallback. Add a config-level default agent model #40's acceptance asked for "a built-in fallback for config-less paths". ADR 0013 §2 overrides that: factory always sends a model and keeps none of its own. Without a config, steps run on the built-in default agent (DEFAULT_AGENT = "opencode", kept for continuity) and must name their model themselves. Otherwise they fail with the message above.
  • An unresolvable model fails at the step's start, not at RunStarted. A per-call option may still supply the model, so the run can't know earlier. The failure still comes before the step event and before any process is spawned.
  • Default agent for this repo. factory init defaults to claude, but this repo's own config defaults to opencode. Its runs (implement-parent-issue, on the prod instance) stay where they ran before, and on a free model. The models entries match the init template.
  • Test fakes now need a model. fakeAgents(adapter) in replay/adapter.ts supplies FAKE_AGENT_MODELS, and the test call sites use it.

Notes for later layers

  • feat(events): context and cost per agent step, ACP-aware run detail #65:
    • AgentStepStarted.agent is optional, and AgentStepView.agent is string | undefined in the SPA.
    • The usage signal is still ignored in agent-step.ts.
    • The live Claude run produced acp.usage chunks with context and cost, ready to record.
  • refactor(runtime): remove the opencode adapter and in-sandbox serve #66:
    • The opencode adapter is no longer the default but still exists (opencode-adapter.ts, prepareWorkspace, sandbox-config.ts). acpAgentsAdapter.prepareWorkspace is a no-op.
    • The replay corpus still reads opencode chunks through extractOpencodeSignal.
    • agent-runtime.test.ts imports opencodeAdapter only to assert that it is not the default.
  • test(e2e): live validation of ACP agents, docs and ADR acceptance #67:
    • factory start --agent/--model and POST /api/runs agent exist for the live legs.
    • The two adapter fixes above, the absolute cwd and always pinning the model, belong in ADR 0013's amendments and the finding. The second one weakens finding 13 §1's "currentValue" reading for Claude.
    • Docs need the agent config block, the --agent/--model flags, and that a config-less run must name a model.

Closes #64
Closes #39
Closes #40
Part of #62

🤖 Generated with Claude Code

… runtime

ADR 0013 §2-§4 on the authoring and config surface.

- `agent?: "claude" | "opencode"` next to `model` on `AgentCallOptions`,
  workflow `agent` defaults, schedule `agent`, and the run request
  (`POST /api/runs` `agent: {agent?, model?}`, `factory start --agent
  --model`).
- `config.agent`: `default`, `models`, `hostSettings` (default "ignore"),
  validated at load; `adapter` stays the injection point for replay/fakes.
- Resolution (`runtime/agent-choice.ts`): agent from the most specific level
  that names one, else `agent.default`; model from the most specific level at
  or above the one that chose the agent, else `agent.models[agent]`. A step
  whose model resolves to nothing fails before it starts, naming the fix.
- The default runtime is the ACP runtime: one `acpAdapter` per agent
  definition, picked per step by the new `AgentAdapterOptions.agent`. The
  agent runtime service carries the config rung (default agent, models).
- `AgentStepStarted.agent` (optional for old logs); run detail shows it.
- `factory init` and this repo's config write `agent.default` and
  `agent.models`. The daemon checks the configured agents at start and logs
  what cannot run.

Removed `DEFAULT_MODEL` (#40): model ids belong to the agent and the host,
and an agent left on its own default runs on the host's settings, so factory
keeps no built-in model; projects set `agent.models`. Removed
`permissionMode` (#39): nothing read it, and every permission ask is now
answered by the ACP client (`allow_always`, then `allow_once`), so a per-call
mode would promise control the runtime overrides. A behavioural mode (e.g.
read-only plan) can come back as an explicit, validated `mode` passthrough.

Closes #64, closes #39, closes #40

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant