feat: multi-instance session lease, write fencing, and fs/watch reliability

[7Sageer]

Related Issue

No linked issue — the problem is explained in the next section.

Problem

With kimi web sharing one home across servers (#1826), several kap-server instances can now run against the same KIMI_CODE_HOME — but session journals, config files, workspace registry, and minidb stores had no cross-process coordination: journal appends could interleave, config writes could clobber each other, a second writer could delete a live lock, and clients had no way to discover or reach the instance actually holding a session.

Single-instance operation also had correctness gaps of its own: a failed journal write could silently lose session events, the fs/watch layer went stale on .gitignore changes and issued per-server (not per-connection) sequence numbers, replay failures surfaced as silent gaps, and server shutdown ordering was nondeterministic.

What changed

Single-instance reliability

• Session event journal: failed writes keep their content and go sticky-explicit (JournalStorageError) instead of being silently dropped; per-batch fsync; open reads the last valid header; cold reads perform zero writes.
• Broadcaster: writeFailure gate (failed journal write → no in-memory tail, readSince throws), deterministic dispose drain, replay failure now triggers resync_required; deterministic, idempotent server close ordering in start.ts.
• fs/watch: per-connection monotonic sequence numbers, pinned truncated semantics (clients rebuild baseline via REST), .gitignore double-cache invalidation; the __global__ session is now fully volatile; the host fs watcher filters special files (sockets/fifos/devices) so one unwatcherdable entry cannot poison a whole directory watch.

Cross-process lock unification

• agent-core-v2 gains CrossProcessLockService: heartbeat-TTL (fd keepalive) and TTL modes, token-guarded release/update, rename-isolated stale takeover, creation window, PID-reuse detection via process start-time identity (conservative on macOS where the identity is unavailable).
• config.toml and the workspace registry now do locked read-modify-write with on-disk format preserved; mcp.json gains a watch.
• minidb's lockfile adopts the same protocol on top of upstream's cluster-era lock work (token-guarded release, quarantine-aside takeover, creation window), keeping its watch/settle exactly-one construction and renew().

Multi-instance session ownership (behind KIMI_CODE_EXPERIMENTAL_MULTI_SERVER)

• Per-session write leases with fencing: 40921 SESSION_HELD_BY_PEER with typed details (routable / creating / holder-unresponsive / held-by-local-instance) plus unregistered-writer detection; a write-authority hard gate rechecks the lease inside AppendLogStore, session metadata, and flush paths.
• kimi-web redirects to the holder instance and replays pending approvals/questions from the holder's snapshot; klient follows redirects transparently with anti-loop bounds, ported onto the new contract-driven facade (SessionRedirectChannel + KlientConnection).
• Cross-instance discovery: session.list_changed volatile hint via two-layer fs watch, and an ownership field on GET /sessions.
• Dual-instance e2e infrastructure (relocated into klient/test/e2e following feat(klient): contract-driven facade with http/ipc/memory transports #1768) covering double-open refusal with journal integrity, SIGSTOP/SIGCONT non-takeover, and kill -9 dead takeover with rename isolation.

Stale-write fencing and skill hot reload

• Session file ledger + agent file fencing: hard gate when the flag is on, advisory tool-result note when off; fs-watch dirty ticks feed the staleness signal.
• Skill root watcher hot reload: catalog reload on change, turn-boundary listing reminder injection, and a skill_catalog.changed volatile event pushed to web clients.

Verification

• The lease/commit protocol was model-checked with Quint before implementation: the no-commit-mutex design carries an inductive-invariant proof for dead takeover, and counterexample traces pinned live takeover and out-of-protocol force-unlock as the cases the implementation must hard-gate (it does).
• tsc typecheck across agent-core-v2, kap-server, klient, protocol, minidb, kimi-web, node-sdk; full package test suites for all of the above, including the relocated dual-instance e2e.

Known limitations and follow-ups

• session.list_changed relies on fs event delivery. On macOS (chokidar 4 without native fsevents), directory notifications can be coalesced and held under multi-instance fs load (a >45s stall was observed in an in-process dual-instance test; the e2e pins determinism with a test-side tree kicker). The hint is volatile and self-healing — clients re-pull on focus/reconnect, and the ownership join on GET /sessions is the authoritative path — so this is a freshness degradation, not a correctness hole. Follow-up: a slow readdir reconciliation loop in SessionListWatchService; Linux (inotify) behavior is unverified.
• On macOS, PID-reuse detection degrades to conservative "held" (the platform exposes no process start-time identity via the sysctl OID used on Linux).
• New telemetry events (session.journal_write_failed, file.stale_conflict) are defined but not yet wired to an appender.
• TUI does not yet consume skill_catalog.changed (node-sdk is in-process; there is no WS event surface to forward) — web clients get it.
• kap-server's full vitest suite is load-sensitive on heavily-loaded machines (session-boot and server.close() steps can hit their 5s/10s timeouts when the disk is saturated by a preceding heavy suite). Every affected file passes in isolation (68/68 in the exact failing files); the sensitivity comes from the new fsync-heavy durability paths and is worth a dedicated hardening pass.

Checklist

• I have read the CONTRIBUTING document.
• I have linked a related issue, or explained the problem above.
• I have added tests that prove my feature works.
• Ran gen-changesets skill, or this PR needs no changeset.
• Ran gen-docs skill, or this PR needs no doc update (the multi-instance behavior is behind an experimental flag; no user-facing docs to update yet).

[changeset-bot]

🦋 Changeset detected

Latest commit: d1ff815

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@moonshot-ai/kimi-code	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[pkg-pr-new]

pnpm dlx https://pkg.pr.new/@moonshot-ai/kimi-code@75473fc

npx https://pkg.pr.new/@moonshot-ai/kimi-code@75473fc

commit: 75473fc