Skip to content

feat(merge): make merge asynchronous via runway#247

Open
behinddwalls wants to merge 1 commit into
preetam/mergeconflict-asyncfrom
preetam/merge-async
Open

feat(merge): make merge asynchronous via runway#247
behinddwalls wants to merge 1 commit into
preetam/mergeconflict-asyncfrom
preetam/merge-async

Conversation

@behinddwalls

@behinddwalls behinddwalls commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

Why?

The committing merge ran synchronously inside the orchestrator, blocking the partition lease on a slow, I/O-heavy operation and coupling SubmitQueue to the merger's latency. This moves the merge to an asynchronous round-trip with Runway — mirroring the merge-conflict check (#245), but for the committing merge.

What?

Adds batch → merge ⇢ (runway) ⇢ mergesignal → conclude/speculate:

  • merge (new) consumes a batch ready to land, builds the full MergeRequest from the batch's member requests (one MergeStep per request in Contains order, carrying each request's change and land strategy), and publishes it to Runway's merger queue keyed by the batch id as the client-owned correlation id.
  • mergesignal (new) consumes Runway's MergeResult off merger-signal, correlates by the echoed id, and transitions the batch to Succeeded (merged) or Failed (could not) — then fans out to conclude (so member requests pick up the outcome) and speculate (so dependents can re-plan). Purely result-driven; no poll loop.
  • DLQ reconcilers drive the batch to a terminal state on dead-letter.

Reuses the Runway merge contract from #260 — the same MergeRequest/MergeResult, carried on the committing merger/merger-signal topics rather than the dry-run check topics. Runway's service implementation is out of scope.

Test Plan

  • ./tool/bazel build //...
  • ./tool/bazel test //... --test_tag_filters=-integration,-e2e (57 tests pass)

Stack

  1. docs(rfc): add message queue contract RFC #259
  2. feat(api/runway): add external merge queue contract #260
  3. feat(orchestrator): make merge-conflict check asynchronous via runway #245
  4. @ feat(merge): make merge asynchronous via runway #247

@behinddwalls behinddwalls force-pushed the preetam/merge-async branch from 92d1ef3 to f89fcb9 Compare June 16, 2026 20:12
@behinddwalls behinddwalls force-pushed the preetam/mergeconflict-async branch 2 times, most recently from 2ee27cf to a970815 Compare June 17, 2026 06:00
@behinddwalls behinddwalls force-pushed the preetam/merge-async branch from f89fcb9 to 2b21765 Compare June 17, 2026 06:00
@behinddwalls behinddwalls force-pushed the preetam/mergeconflict-async branch from a970815 to bc59a27 Compare June 17, 2026 06:09
@behinddwalls behinddwalls force-pushed the preetam/merge-async branch from 2b21765 to cd3ef63 Compare June 17, 2026 06:09
@behinddwalls behinddwalls force-pushed the preetam/mergeconflict-async branch from bc59a27 to d9dc61f Compare June 17, 2026 06:20
@behinddwalls behinddwalls force-pushed the preetam/merge-async branch from cd3ef63 to 2d17620 Compare June 17, 2026 06:20
@behinddwalls behinddwalls force-pushed the preetam/mergeconflict-async branch from d9dc61f to 7ab64d3 Compare June 17, 2026 17:33
@behinddwalls behinddwalls force-pushed the preetam/merge-async branch from 2d17620 to 4a525e5 Compare June 17, 2026 17:33
@behinddwalls behinddwalls force-pushed the preetam/mergeconflict-async branch from 7ab64d3 to b6042a6 Compare June 17, 2026 18:00
@behinddwalls behinddwalls force-pushed the preetam/merge-async branch from 4a525e5 to 288e995 Compare June 17, 2026 18:00
@behinddwalls behinddwalls force-pushed the preetam/mergeconflict-async branch from b6042a6 to 2a8ea85 Compare June 17, 2026 20:20
@behinddwalls behinddwalls force-pushed the preetam/merge-async branch from 288e995 to 1c0a803 Compare June 17, 2026 20:20
@behinddwalls behinddwalls force-pushed the preetam/merge-async branch from 1c0a803 to fb39522 Compare June 17, 2026 21:30
@behinddwalls behinddwalls force-pushed the preetam/mergeconflict-async branch from 2a8ea85 to 6f9a08f Compare June 17, 2026 21:30
@behinddwalls behinddwalls force-pushed the preetam/mergeconflict-async branch from 6f9a08f to 5d3231c Compare June 17, 2026 21:50
@behinddwalls behinddwalls force-pushed the preetam/merge-async branch from fb39522 to 90a02ae Compare June 17, 2026 21:50
@behinddwalls behinddwalls force-pushed the preetam/mergeconflict-async branch from 5d3231c to eb1ba84 Compare June 17, 2026 21:55
@behinddwalls behinddwalls force-pushed the preetam/merge-async branch from 90a02ae to 39c599f Compare June 17, 2026 21:55
@behinddwalls behinddwalls force-pushed the preetam/mergeconflict-async branch from eb1ba84 to eadb853 Compare June 17, 2026 22:05
@behinddwalls behinddwalls force-pushed the preetam/merge-async branch from 39c599f to 9f666f7 Compare June 17, 2026 22:05
@behinddwalls behinddwalls force-pushed the preetam/mergeconflict-async branch from eadb853 to e77077c Compare June 17, 2026 22:20
@behinddwalls behinddwalls force-pushed the preetam/merge-async branch from 9f666f7 to 443f9cb Compare June 17, 2026 22:21
@behinddwalls behinddwalls force-pushed the preetam/mergeconflict-async branch from e77077c to e0d4543 Compare June 17, 2026 23:39
@behinddwalls behinddwalls force-pushed the preetam/merge-async branch from 443f9cb to 2bad628 Compare June 17, 2026 23:39
Rework the merge stage from a synchronous in-process pusher call into a
runway round-trip, mirroring the merge-conflict check. The merge
controller now builds a full runway MergeRequest from the batch's member
requests (one MergeStep per request, in Contains order) and publishes it
to the runway-owned merger queue, keyed by the batch id as the
correlation id. A new mergesignal controller consumes the MergeResult off
merger-signal, transitions the batch to Succeeded/Failed, and fans out to
conclude and speculate; a mergesignal DLQ reconciler fails the batch on
an unprocessable result.

The in-process pusher extension is retired from the orchestrator wiring
(left in-tree but unused, like mergechecker); removal is a follow-up.
workflow.md and extension-contract.md updated to reflect both the check
and the merge crossing into runway over the shared MergeRequest/
MergeResult contract.
@behinddwalls behinddwalls force-pushed the preetam/mergeconflict-async branch from e0d4543 to f8bc7f6 Compare June 17, 2026 23:42
@behinddwalls behinddwalls force-pushed the preetam/merge-async branch from 2bad628 to deee682 Compare June 17, 2026 23:42
@behinddwalls behinddwalls marked this pull request as ready for review June 18, 2026 00:09
@behinddwalls behinddwalls requested review from a team and sbalabanov as code owners June 18, 2026 00:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant