Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/wiki/_Sidebar.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,10 @@
- [[CI scheduling and cost|guides/ci/Scheduling-and-Cost]]
- [[Hosted providers|guides/ci/Hosted-Providers]]
- [[Cache and artifacts|guides/ci/Cache-and-Artifacts]]
- [[Walkthrough: a home lab|guides/ci/Walkthrough-Home-Lab]]
- [[Walkthrough: several projects on one fleet|guides/ci/Walkthrough-Multiple-Projects]]
- [[Walkthrough: a team|guides/ci/Walkthrough-Team]]
- [[Walkthrough: agents|guides/ci/Walkthrough-Agents]]
- [[CI reference|guides/ci/Reference]]
- [[CI agent reference|guides/ci/Agent-Reference]]
- [[Auto placement and pools|guides/ci/Auto-Placement-and-Pools]]
Expand Down
110 changes: 110 additions & 0 deletions .github/wiki/guides/ci/Walkthrough-Agents.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# Walkthrough: agents

This page walks a coding agent through CI. The agent starts a run, waits for it, and reads the evidence. It can do this through the CLI, through MCP tools or through HTTP routes. All three return the same JSON.

Commands that need a real run are shown as text, each with the test that runs it. The full list of MCP tools and HTTP routes, with each command's side effect and confirmation rule, is the generated machine-surface table at the top of the [CI plugin page](../../plugins/CI.md). The command and flag tables are in the generated [Reference](./Reference.md#commands).

## The CLI loop

Start the run in the background with `--detach --json`. The reply is one JSON document with the run id and the exact commands to wait for it, follow it, read its evidence, cancel it and read its logs.

<!-- proven-by: TestDetach -->
```text
nself ci --detach --json
```

Wait for the result, then read the evidence. `runs wait` returns when the run ends; `--timeout` bounds the wait.

<!-- proven-by: TestRunsWait -->
```text
nself ci runs wait <run-id> --json
```

<!-- proven-by: TestBareJSONIsOneEvidenceEnvelope -->
```text
nself ci evidence <run-id> --json
```

To watch progress instead of waiting, follow the run. Each line is one JSON event, and a reader that reconnects can resume from the last sequence number it saw.

<!-- proven-by: TestFollowReplayFromSeq -->
```text
nself ci runs follow <run-id> --json
```

The document shapes are in [Evidence and Machine Output](./Evidence-and-Machine-Output.md).

## The MCP loop

Start the MCP server with `nself mcp` inside the project directory. The agent needs only three tools for the loop:

1. `nself_ci` starts a run. Pass `{"args": ["--detach"]}`.
2. `nself_ci_runs_wait` waits for it. Pass `{"args": ["<run-id>"]}`.
3. `nself_ci_evidence` returns the evidence. Pass `{"args": ["<run-id>"]}`.

Every other tool is listed in the machine-surface table. Commands that only make sense in a terminal, that stream, or that ask questions have no tool.

## Confirmations

A bare run executes code, so it needs a confirmation. The first `nself_ci` call does not run anything. It returns a request with a `confirm` value. The agent shows the request to its user and, once approved, calls `nself_ci` again with the same arguments plus that `confirm` value. Only the second call starts the run. Destructive commands, such as revoking a node, work the same way. The confirmation column of the machine-surface table says which commands need it.

```text
first call: {"args": ["--detach"]}
reply: {"kind": "request", "confirm": "<64 hex characters>", ...}
second call: {"args": ["--detach"], "confirm": "<the same value>"}
```

## The HTTP routes

`nself admin serve` exposes the same commands as HTTP routes under `/v1/commands/`, for example `/v1/commands/ci/runs/wait`. Send a POST with a bearer token and a JSON body such as `{"args": ["<run-id>"]}`. The reply carries the same data as the CLI's `--json` output. Confirmations use the same `confirm` round trip.

`runs follow` has no MCP tool. Over HTTP, `/v1/commands/ci/runs/follow` answers with an NDJSON stream: one JSON event per line, flushed as the run moves.

## One server per project

An MCP or HTTP server works on the project directory it was started in. Start one server per project, each from its own checkout.

## No commit status from agents by default

A run started through MCP or HTTP is a machine-invoked run: the server marks it with `NSELF_INVOKED_BY`. A CLI agent should export that variable itself. A machine-invoked run posts no commit status, and prints one line saying so. Add `--post-status` when the agent's result should count as the commit's status.

<!-- proven-by: TestMachineDefaults -->
```text
nself ci --detach --json --post-status
```

## Capping the trust of agent runs

The policy key `trust.machine_invoked_cap` caps the source trust class of every machine-invoked run. The default is `owner`, which is no cap. For agents that run unattended, set it to `untrusted`. A machine-invoked caller can lower the cap but never raise it again.

<!-- proven-by: TestPolicyMachineInvokedCap -->
```text
nself ci policy set --scope user trust.machine_invoked_cap untrusted
```

For a CLI agent the cap is advisory, because the agent controls its own environment. See [machine-invoked runs](./Trust-Security-and-Privacy.md#machine-invoked-runs).

## Code an agent wrote

Run code that an agent wrote, and that nobody has read yet, as untrusted. `--trust untrusted` needs the sandboxed container level; it never falls back to a weaker level, never gets service containers and never gets internet access. If the host cannot provide the sandbox, nothing runs.

<!-- proven-by: TestResolveLevelAndNetwork -->
```text
nself ci --trust untrusted --detach --json -- go test ./...
```

The isolation levels are in [Trust, Security and Privacy](./Trust-Security-and-Privacy.md#the-sandboxed-container-level-gvisor).

## Proven by

| Step | Test |
|---|---|
| Detached run and the handle's commands | `TestDetach` |
| Detached run outlives its parent | `TestDetachSurvivesParent` |
| Wait for a run | `TestRunsWait` |
| Evidence as one JSON document | `TestBareJSONIsOneEvidenceEnvelope` |
| Follow replays from a sequence number | `TestFollowReplayFromSeq` |
| MCP tools, HTTP routes, same output on CLI, MCP and HTTP, confirmation round trip | `scripts/ci/ci-machine-surface.sh` |
| No status from machine-invoked runs unless `--post-status` | `TestMachineDefaults` |
| Trust cap key, lower but never raise | `TestPolicyMachineInvokedCap` |
| `--trust untrusted` needs the sandbox | `TestResolveLevelAndNetwork` |
151 changes: 151 additions & 0 deletions .github/wiki/guides/ci/Walkthrough-Home-Lab.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
# Walkthrough: a home lab

This page walks one real setup from start to finish: a Mac Studio on your desk runs the coordinator, two Mac Minis on the same LAN run jobs, and a small VPS you rent runs Linux jobs. Everything here is personal mode: you are the only person with access.

The page shows the order of steps and links the guide that explains each one. Commands that need real machines are shown as text, each with the test that runs it. The tables of commands, flags, codes and doctor checks live in the generated [Reference](./Reference.md#commands).

Names used below: `desk` is the Mac Studio, `mini1` and `mini2` are the Minis, `vps1` is the VPS at `203.0.113.20`. Replace them with your own.

## The desk Mac runs the coordinator

The coordinator keeps the run store, the node registry and the keys. In personal mode it starts with no extra flag. Its webhook and user API listeners stay on loopback by default; only the agent listener needs to be reachable from the Minis. The listener ports are described in [Nodes and Runners](./Nodes-and-Runners.md#team).

<!-- proven-by: TestServeModeFlagAndStart -->
```text
nself ci serve
```

A coordinator started without `--team` stays personal. Do not add `--team` here: the switch is permanent for that data directory, and a home lab used by one person does not need it.

## Adding the Minis

There are two ways to add a Mac. Use `nodes add --system` when you can reach the Mini over SSH. Use pairing when you cannot, or when you do not want the coordinator to hold an SSH key for it.

**Add over SSH.** `--system` installs the agent as a system service: a hidden agent user, one job user per slot, and a LaunchDaemon that runs the agent in pull mode. Get the host-key fingerprint from the Mini itself, not from the network.

<!-- proven-by: TestMacOSNode -->
```text
nself ci nodes add ci@mini1 --system --slots 2 --host-key <fingerprint> --agent-binary <path-to-nself-ci-agent> --project <owner/repo> --name mini1 --json
```

The bootstrap checks every account and folder it creates. Until jobs actually run through the job shim, the node is recorded as not UID-separated, and installing it grants no secret or signing rights. See [provisioning versus bootstrap](./Nodes-and-Runners.md#vps).

**Pair instead.** Run `nodes pair` on the desk Mac. It opens a short window and prints a `nself-ci-agent join` command, the coordinator fingerprint and a one-use code. Run the printed join command on the Mini, type the code there, and check that both fingerprints match before you approve.

<!-- proven-by: TestNodesPairEndToEnd -->
```text
nself ci nodes pair --label mini2 --agent-url https://desk.example:3864 --cert-ttl 30m
```

A paired Mini runs the same agent in pull mode: it only makes outbound connections to the coordinator. Never put the pairing code in a command line or a shared log.

## Labels: which jobs may run where

Give each node the labels your jobs ask for. The Minis have Xcode, so they get the `xcode` label.

<!-- proven-by: TestSeveralProjects -->
```text
nself ci nodes label mini1 xcode
nself ci nodes label mini2 xcode
```

A job that needs Xcode says so in `.nself/ci.yaml`. A runner is eligible only when it has every label the job asks for:

```yaml
jobs:
ios:
kind: test
run: [xcodebuild, test]
requires: {labels: [xcode]}
```

Check where each job would go before you push. `plan` shows the placement; `explain` lists every runner that was not chosen and why.

<!-- proven-by: TestPlanPlacementJSON -->
```text
nself ci plan --json
```

<!-- proven-by: TestScenario6ExplainListsEveryAlternative -->
```text
nself ci explain
```

The reasons (`labels.mismatch`, `tools.missing` and others) are listed in [Scheduling and Cost](./Scheduling-and-Cost.md#job-requirements-and-labels) and in the [reason table](./Reference.md#reason-codes).

## Reservations on the desk machine

If the desk Mac also takes jobs, keep part of it for yourself. Reservations are scheduler limits: CI work never uses the reserved CPU or memory. `--interactive protect` holds new work back while you are using the machine. `--on-battery` matters only for laptops; a desk machine can leave it at `allow`.

<!-- proven-by: TestNodesSetFlagsMayFollowOrPrecedeTheNodeAndKeepWhatIsNotGiven -->
```text
nself ci nodes set desk --reserve-cpu 2 --on-battery allow
nself ci nodes set desk --reserve-mem 2048
```

A reservation that would leave nothing for CI is refused. More controls are in [Nodes and Runners](./Nodes-and-Runners.md#lan).

## The personal VPS

A VPS is added the same way as a Mini, with its host key pinned. The SSH carrier forces strict host-key checking against the coordinator's own pin file, so a changed key stops the connection before any job runs. Connect as root or a user with passwordless sudo so `--system` can create the agent and job users.

<!-- proven-by: TestSystemInstallBootstrapsAndKeepsTheNodeUnseparated -->
```text
nself ci nodes add root@203.0.113.20 --system --slots 3 --host-key <fingerprint> --agent-binary <path-to-nself-ci-agent> --json
```

Never add a machine that serves your deployed apps. Deploy hosts are not eligible for CI; see [deploy hosts are ineligible](./Nodes-and-Runners.md#vps).

## Secrets on a node

A node gets no project secrets until you grant them. Grant the `project` class to the VPS if its jobs need project secrets. The `release` class needs a node dedicated to releases.

<!-- proven-by: TestGrantProjectCallsCliAndAudits -->
```text
nself ci nodes secrets grant vps1 --class project
```

Every grant and revoke is written to the audit log. The secret rules are in [Trust, Security and Privacy](./Trust-Security-and-Privacy.md#secrets).

## When a Mini sleeps

A sleeping or frozen Mini stops renewing its lease. It fences itself before the coordinator's lease expires, so it cannot finish the old job late. The coordinator marks the attempt lost and runs it again on another eligible node. If the Mini wakes and reports the old result, that result is refused. A job that failed on its own merits is not retried.

Mark machines that sleep on purpose as intermittent so the scheduler prefers always-on nodes:

<!-- proven-by: TestNodesSetJSONEnvelopeCarriesTheStoredSettings -->
```text
nself ci nodes set mini2 --reserve-mem 1024 --persistence intermittent --json
```

Lost leases and what to check are covered in [Troubleshooting](./Nodes-and-Runners.md#troubleshooting).

## Backing up the coordinator

Back up the desk Mac's data directory to a file. The archive does not contain the key-encryption key (KEK). Keep the KEK somewhere else; without it the archive cannot be restored, and a wrong KEK is refused before anything is written.

<!-- proven-by: TestBackupRestoreCommandsRoundTrip -->
```text
nself ci serve backup --data-dir <coordinator-data-dir> --out <archive> --json
nself ci serve restore --data-dir <new-data-dir> --in <archive> --kek <saved-kek-file> --json
```

Stop the coordinator before a restore. What restore keeps and resets is in [backup/restore and the KEK](./Nodes-and-Runners.md#team).

## Proven by

| Step | Test |
|---|---|
| Personal coordinator start, no `--team` | `TestServeModeFlagAndStart` |
| Add a Mac with `--system` (LaunchDaemon, users, layout) | `TestMacOSNode` |
| Pair a node and run the printed join | `TestNodesPairEndToEnd` |
| Labels and the placement they cause | `TestSeveralProjects` |
| Placement plan in JSON | `TestPlanPlacementJSON` |
| Every alternative explained | `TestScenario6ExplainListsEveryAlternative` |
| Reservations and battery policy | `TestNodesSetFlagsMayFollowOrPrecedeTheNodeAndKeepWhatIsNotGiven` |
| Add a Linux VPS with `--system` | `TestSystemInstallBootstrapsAndKeepsTheNodeUnseparated` |
| Grant project secrets to a node | `TestGrantProjectCallsCliAndAudits` |
| Release secrets need a dedicated node | `TestGrantReleaseNeedsDedicatedNode` |
| A frozen node fences and the job runs elsewhere | `TestFaultScenario9Freeze` |
| Intermittent node setting | `TestNodesSetJSONEnvelopeCarriesTheStoredSettings` |
| Backup and restore with the KEK | `TestBackupRestoreCommandsRoundTrip` |
95 changes: 95 additions & 0 deletions .github/wiki/guides/ci/Walkthrough-Multiple-Projects.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Walkthrough: several projects on one fleet

This page walks one fleet that serves two projects. Project A and project B share a Mac and a VPS. A second VPS belongs to project A only. Every project keeps its own queue share, cache and statistics.

Names used below: `mac1` has Xcode, `vps-shared` serves both projects, `vps-a` serves project A only. `~/src/project-a` and `~/src/project-b` are the two checkouts. Commands that need real machines are shown as text, each with the test that runs it. The command and flag tables are in the generated [Reference](./Reference.md#commands).

## One pool for several projects

A node runs jobs only for the projects it is authorized for. Authorize by checkout directory or by repository id; a directory is turned into the same repository id that evidence uses.

<!-- proven-by: TestSeveralProjects -->
```text
nself ci nodes projects add mac1 ~/src/project-a ~/src/project-b
nself ci nodes projects add vps-shared ~/src/project-a ~/src/project-b
nself ci nodes projects list vps-shared
```

To take a project off a node, remove it. `--all` removes every project from that node.

<!-- proven-by: TestSeveralProjects -->
```text
nself ci nodes projects remove vps-shared <repo-id>
```

## A VPS for one project

Authorize the dedicated VPS for project A only. Project B's jobs never land there, even when it is idle.

<!-- proven-by: TestSeveralProjects -->
```text
nself ci nodes projects add vps-a ~/src/project-a
nself ci nodes projects list vps-a
```

A personal node is never chosen for a private repository it is not authorized for. Adding the project makes it eligible.

## Pinning a project with labels

Labels describe a node; a project can require extra labels for all its jobs. Give `vps-a` a label, then require that label at project scope for project A. Project A's jobs then need it on top of their own `requires.labels`.

<!-- proven-by: TestSeveralProjects -->
```text
nself ci nodes label mac1 xcode
nself ci nodes label vps-a project=a
nself ci policy set --scope project sched.overrides.labels '["project=a"]'
```

Run the `policy set` line inside project A's checkout. After it, all of A's jobs go to `vps-a`. A job of A that also needs `xcode` waits: `mac1` has Xcode but not `project=a`, and `vps-a` has the reverse. `explain` names the override as the reason.

<!-- proven-by: TestScenario6ExplainListsEveryAlternative -->
```text
nself ci explain
```

How overrides combine with job requirements is in [Scheduling and Cost](./Scheduling-and-Cost.md#overrides).

## Fairness and the queue

When the fleet is busy, the scheduler shares capacity between projects and never admits more than the fleet can hold. Interactive runs start before queued background work. `runs queue` shows each waiting run with its position, its band and the expected wait.

<!-- proven-by: TestRunsQueueListsPositionBandAndWait -->
```text
nself ci runs queue
```

The fairness rules and presets are in [Scheduling and Cost](./Scheduling-and-Cost.md#how-a-runner-is-chosen).

## Cache and artifacts stay separate

Each cache entry lives in a namespace made of the project, the trust class of the source and whether the ref is protected. The same key in another project's namespace neither blocks nor reveals the entry. Artifacts are stored per run, and runs belong to one project. Nothing needs to be set up for this. See [namespaces](./Cache-and-Artifacts.md#namespaces-per-trust-and-per-pr).

## Statistics per project

`runs stats` summarises runs, queue time and failures. `--project` limits it to one project; `--since` sets the window. A caller of a team coordinator sees only the projects it has a role on.

<!-- proven-by: TestStatsReadWindowAndProject -->
```text
nself ci runs stats --since 168h --project <owner/repo> --json
```

The fields are described in [CI health](./Scheduling-and-Cost.md#ci-health).

## Proven by

| Step | Test |
|---|---|
| Authorize, list and remove projects on a node; per-project VPS; label override | `TestSeveralProjects` |
| A personal node needs authorization for a private repo | `TestPersonalNodePrivateRepoOptIn` |
| Every alternative explained | `TestScenario6ExplainListsEveryAlternative` |
| Fair share under saturation | `TestScenario10SaturationStaysWithinCapacityAndFair` |
| Interactive before background | `TestScenario10InteractiveStartsBeforeQueuedBackground` |
| Queue position, band and wait | `TestRunsQueueListsPositionBandAndWait` |
| Cache namespaces do not leak | `TestSameKeyStringInAnotherNamespaceNeitherBlocksNorRevealsTheTrustedEntry` |
| Stats window and project filter | `TestStatsReadWindowAndProject` |
| Stats scoped to the caller's projects | `TestStatsScopedCallerSeesOnlyAllowedProjects` |
Loading
Loading