rust mode: host mirror pull wedges with the cursor frozen mid-corpus — transforms stay healthy while the entire ctx_memory surface (reads AND writes) refuses

[iceteaSA]

Summary

On a clean rust-mode activation, authority completes, the module serves transforms beautifully, and ctx_memory goes entirely dead — writes refuse with MC-C01, reads refuse with MC-C02 — because the host mirror pull stops advancing partway through the corpus and never resumes.

The distinguishing feature is that nothing reports unhealthy. ck health magic-context is ok with consecutive_error_count: 0, transform passes run every ~13s at 28–33ms, and the plugin log has zero errors. Only a direct look at mirror_cursors shows it.

This is not #390 (workspace collision), #391 (drain rollback), #393 (rekey), or #379 (pointer loss) — all four of those fixes held perfectly on this run, verified below.

Observed

Activation succeeded on every axis I could check:

context.db  authority_managed            1 row, captured 13:05:24Z
            authority_capture_bounds     memories → 4850 (matches MAX(memories.id)), notes → 406
            authority_repair_pending     0 rows
store.db    mc_authority                 memories=MODULE   notes=MODULE
            mc_authority_route_bindings  /home/icetea/projects/magic-context → git:93eea8cd…
            mc_authority_pending_memory_references  0 rows

The module was genuinely serving, from the daemon journal:

mc-pass-timing session=ses_… total=28.7 handler_total=72.8 transform_execute=35.5
  projection_reused_messages=341 projection_projected_messages=2
  store_memories=0.0 seed_or_sync=4.7 retry_attempts=0

Every ~13s, for the whole window. ck health: transform lane advancing, in_flight=0, consecutive_error_count: 0.

But the mirror cursor never moved:

                          13:05:26Z        13:15:01Z        (and 13:16, 13:22…)
mirror_cursors.memories   3726             3726             3726
  updated_at              1789391126089    1789391126089    1789391126089
mirror_live_memory_rows   275              275              275

MAX(memories.id) is 4850 across 4057 rows, so the cursor stalled with roughly a third of the corpus unmirrored, and updated_at is byte-frozen — not slow, not retrying, simply not running again. Notes' cursor was untouched from a prior session entirely (1788121121540, weeks old).

Meanwhile every ctx_memory call refused, in both directions:

ctx_memory(action:"write",…)  → Memory writes are paused while the engine syncs. Retry in a moment. (MC-C01)
ctx_memory(action:"get",…)    → Memory access is temporarily unavailable. Retry in a moment. (MC-C02)

Sustained for 10+ minutes across repeated attempts. "Retry in a moment" is accurate phrasing for a transient drain and misleading for this, which never clears.

Why it is easy to misread

The failure presents as healthy. Transforms are the loud, visible subsystem and they are fine — arguably better than TS mode. The dead subsystem is quiet by nature: memory tools only refuse when something asks, and a long autonomous stretch may not ask for many minutes. I only caught it because I happened to write a memory right after flipping.

A count check also passes here — no rows are lost, nothing is corrupt. The state is stalled, not damaged.

The useful discriminator, if it helps triage: sample context.db mirror_cursors twice ~40s apart. Frozen updated_at with a non-empty mirror_live_memory_rows distinguishes a wedge from a slow drain, which is otherwise indistinguishable from the outside.

Mechanism — hypothesis, not source-confirmed

Flagging where I could not close the loop rather than asserting.

syncModuleDomain (hook.ts:878-886) calls drainMirrorPages({db, module, domain, limit: 1000}), and its only invocation for memories (hook.ts:1012) sits after a successful module memory mutation:

// Auto-search and local RPC/dashboard reads consume the mirror,
// so publish the module mutation to that read model before return.
await syncModuleMemories();

If that is the sole drain trigger for the memories domain, then an activation whose initial drain does not carry the cursor to the end has no scheduled path to resume: the drain needs a successful mutation, and mutations are being refused. Whether the refusal is upstream or downstream of that, I could not determine from the outside — authorityState() (ctx-memory/tools.ts:554-563) issues a live authorityStatus query to the module rather than reading mc_authority, so the module's runtime view can disagree with its own persisted table, which is exactly the split I observed (table says MODULE, behaviour says draining).

TRANSFORM_MEMORY_MIRROR_PAGE_BUDGET = 20 with limit: 1000 bounds a single drain at 20 pages. With 4057 memories that should comfortably cover the corpus in one pass, so a budget exhaustion alone does not obviously explain a stop at 3726 — unless the budget is consumed and the continuation depends on a trigger that never fires.

If the "drain resumes only on the next successful mutation" reading is right, the fix direction is a drain that is not solely mutation-driven — resume on transform pass, or on authority-status query, or any already-periodic event.

Recovery, and one thing worth documenting

doctor drain-authority recovered it completely and safely:

Authority drained back to TypeScript for git:93eea8cd….

context.db  authority_managed   0 rows
store.db    mc_authority        memories=TS  notes=TS
damage      rows gone 0 · supersede pointers lost 0 · orphaned embeddings 0 · MAX(id) unchanged

Worth stating explicitly for anyone hitting this: drain first, do not flip config and restart. With 275 rows in flight, a config revert is precisely the condition that strands superseded_by_memory_id permanently NULL (the #379 class — and repairNullClobberedMemoryRows does not repair supersede pointers). Draining first made the subsequent revert a clean no-op, with a full identity-bearing fingerprint identical to pre-flip.

Note that doctor drain-authority itself currently requires a workaround when the daemon publishes outside the default path — filed separately as #443. That one compounds this one: the escape hatch needs a symlink before it can rescue you from the wedge.

Residue after drain: mirror_live_memory_rows retained its 275 rows. That appears inert — context-authority.ts:1195 deletes the table wholesale before repopulating on the next activation — so I did not clean it, but flagging in case it should be cleared on drain.

Verified NOT recurring

Stated because these were the prior blockers and they all held, which is what made this new failure legible:

issue	check	result
#390	workspace row reused, no UNIQUE collision	held
#391	importance/source_type vs pre-flip snapshot	held
#393	id set + id>superseded_by pair set	0 rekeyed
#379	supersede pointers after drain	0 lost

Fingerprint before/after activation differed in exactly one field: mirror_identity 274 → 301, the module correctly registering backlinks.

Environment

Upstream dc952bf3 (v0.42.3) plus two unrelated fork commits (skill-memory, external-memory tee — neither touches authority, mirror, or crates/). crates/ and Cargo.lock byte-identical to upstream. ck-mc built from that exact tree against subconscious 856e26f7 (subc-core 0.17.39 / protocol 0.19.1 / client-rs 0.12.2), daemon ck 0.17.39, generations matched. Store schema 53, context.db v84 + fork lane. 4057 memories, 919 supersede pointers, 3802 embeddings. Single activation → 10min observation → drain. Pre-flip and post-revert snapshots retained.

[magic-alfonso]

Thanks for the precise capture, especially the two-sample cursor discriminator. Executed against the real module with a 2,505-row corpus, the picture is a bit different from the hypothesis, and I want to be exact about what is and is not established.

The transform pass already re-arms the mirror drain whenever its memory projection is unacknowledged, so the drain does not depend on a successful mutation; on the pre-fix code a page-budget stop resumes on the next transform (cursor 2000 → 2505 across passes). Your 1,726 → 3,726 move is the exact signature of one exhausted pass at the old 20 × 100 page budget, but a budget stop alone cannot explain a cursor frozen for ten minutes across healthy transforms.

What execution did reproduce is a real freeze the old code allowed: a memory written on the module side that advances the changefeed without changing the row/render projection left the host believing the mirror was caught up, so no pass pulled again. That is fixed on master (9959204): transform responses carry the module's memory feed head and the host keys its projection on it, so a hidden feed append re-arms the bounded ride-along; the transform page size is 1,000 (20,000 feed rows per pass under the same 20-page cap); a caught-up mirror performs no pull on stable passes, pinned byte-identical.

Two more corrections from the same run. authority.status reads the persisted mc_authority row, not a separate runtime cache, so the table and a same-key query cannot disagree; the MC-C02 read refusals came from the facade's session resolver, independent of mirror completeness. Under MODULE authority, reads (including the Dreamer's list) now route to the module instead of presenting a partial host mirror as complete. And the frozen state is no longer silent: ck health magic-context, session.status and /ctx-status report the feed head, host cursor, cursor age and pending rows, with MC-M01 when the cursor is behind and older than 40 seconds and MC-M02 for a durable host/module authority mismatch, replacing the "retry in a moment" copy for a state that does not clear on its own. doctor drain-authority also clears the project's mirror_live_memory_rows on the way back to TS.

Your exact 3,726 freeze cannot be uniquely assigned from the values captured, because the mirror frontier is MAX(mc_changefeed.feed_seq), not MAX(memories.id). If it recurs on 0.42.4 or later, the two numbers that settle it are that frontier and the transform provenance for the session (both now in session.status). The host half ships in v0.42.4; the module half lands with the next ck-mc build.