From 6bfe1d11e190f58e7a14195b7d58cd0f27fcd9f7 Mon Sep 17 00:00:00 2001 From: "kevin.new" Date: Tue, 16 Jun 2026 20:58:41 +0000 Subject: [PATCH 1/3] doc(runway): add workflow RFC for runway domain Runway is a consumer-only landing service that owns VCS operations (mergeability checking and landing) on behalf of SubmitQueue. This RFC describes the two independent queue flows (check and land), branch serialization via partition key, statelessness, and ownership model. Co-Authored-By: Claude Opus 4.6 (1M context) --- doc/rfc/index.md | 4 ++ doc/rfc/runway/workflow.md | 84 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 88 insertions(+) create mode 100644 doc/rfc/runway/workflow.md diff --git a/doc/rfc/index.md b/doc/rfc/index.md index 6fec9e6d..f72f2eec 100644 --- a/doc/rfc/index.md +++ b/doc/rfc/index.md @@ -16,3 +16,7 @@ Design documents and technical proposals, grouped by scope. Shared/cross-cutting ## Stovepipe - [Stovepipe Workflow](stovepipe/workflow.md) - Post-merge trunk-validation pipeline: ingest trunk push events (webhook + fallback poll), batch since last green, build to validate, record per-commit health, bisect to the offending commit, hand off to a remediation extension + +## Runway + +- [Runway Workflow](runway/workflow.md) - Landing service: check request mergeability, land scored batches, publish results back to SubmitQueue diff --git a/doc/rfc/runway/workflow.md b/doc/rfc/runway/workflow.md new file mode 100644 index 00000000..a3a86cf2 --- /dev/null +++ b/doc/rfc/runway/workflow.md @@ -0,0 +1,84 @@ +# Runway Workflow + +Runway is the landing service: it owns VCS operations — mergeability checking and landing (resolve, apply, push, finalize) — on behalf of SubmitQueue. Unlike SQ and Stovepipe, it is a consumer-only domain with no gateway and no proto API; work arrives via topic queues, results leave via topic queues. The orchestrator subscribes to two inbound topics (runway-check, runway-land) and publishes to two outbound topics (sq-check-result, sq-land-result). There are no internal hops between stages and no cycles — each controller consumes one inbound message and produces one outbound result. + +## Check and land + +The two queues operate at different granularities: + +- **check** is request-level. SubmitQueue publishes a check message to determine whether a request's changes can merge cleanly against the target branch. Runway performs a read-only trial merge and publishes per-change mergeability results back. + +- **land** is batch-level. SubmitQueue publishes a job message to land a batch. The job carries the resolved content for each request in the batch (runway has no access to SubmitQueue's request store, so the message must be self-contained). Runway pre-validates, lands, finalizes, and publishes a result with per-item outcomes back. + +These are independent input→output flows. Check can run without land ever running (SubmitQueue may skip external mergeability checks for some queues), and land does not depend on a prior check having passed through runway. + +## Branch serialization + +The partition key `repo/target` on both inbound topics serializes all VCS operations for a given branch. The message queue delivers messages with the same partition key to the same consumer in order — at most one check or land operation is in flight for any given branch at any time. + +This replaces the LandQueue extension from the synchronous design — no Redis, no MySQL queue table, no distributed lock. The tradeoff: serialization granularity is fixed at the partition key level. If pipelining becomes necessary (overlapping check N+1 with land N for the same branch), it can be implemented within the controllers without changing the queue topology. + +The outbound topics partition by SubmitQueue queue name (not repo/branch), matching SubmitQueue's existing fan-out model where state updates for the same queue must be serialized. + +## Workflow + +``` + ┌─────────────────────────────────────────────┐ + │ submitqueue orchestrator │ + │ │ + │ │ │ │ + └───────┼───────────────────────┼─────────────┘ + │ │ + Check (per request) Job (per batch) + │ │ + ▼ ▼ + [runway-check] [runway-land] + │ │ + check ctrl land ctrl + (read-only) (pre-validate + push) + │ │ + CheckResult Result + │ │ + ▼ ▼ + [sq-check-result] [sq-land-result] + │ │ + ▼ ▼ + ┌───────┼───────────────────────┼─────────────┐ + │ check-result ctrl land-result ctrl │ + │ (update request (update batch state, │ + │ mergeability) fan out to conclude) │ + │ │ + │ submitqueue orchestrator │ + └─────────────────────────────────────────────┘ +``` + +## Per-controller summary + +| Controller | In | Out | One-line role | +|---|---|---|---| +| **check** | Check | CheckResult → sq-check-result | Check mergeability of a request's changes against the target branch (read-only) | +| **land** | Job | Result → sq-land-result | Pre-validate, land, and finalize a batch's changes on the target branch | + +The check controller always publishes a result — even when all changes are mergeable — so SubmitQueue receives a definitive answer. On infrastructure error it nacks for retry. + +The land controller publishes a conflict result (and acks) when pre-validation detects a conflict; SubmitQueue handles rebatching. On infrastructure error it nacks for retry. On success it publishes per-item outcomes (commit SHAs, whether new commits were produced) so SubmitQueue can update its request state. Conflicts are expected outcomes, infrastructure errors are not — this matches the pattern established by SubmitQueue's merge controller. + +## Idempotency + +Runway has no persistent state — no request store, no job store, no database. Idempotency is achieved through the VCS contract: land detects already-pushed changes (commit SHAs reachable from HEAD) and treats them as already-landed; closing an already-closed PR is a no-op. Check is read-only and naturally idempotent. The message queue provides at-least-once delivery; idempotent consumers handle duplicates. + +## DLQ reconciliation + +Runway does not implement DLQ reconciliation controllers. Unlike SubmitQueue and Stovepipe — where a stuck message leaves a request or commit in a non-terminal state forever — runway is stateless. A check or land message that exhausts retries and lands in the DLQ simply means SubmitQueue never receives a result for that request or batch. SubmitQueue's own timeout and retry logic handles the missing response (re-publishing the check or re-batching the requests). Adding DLQ controllers to runway would duplicate that recovery logic without adding safety, since there is no persistent state to reconcile. + +If runway later gains persistent state (e.g., a job store for observability), DLQ controllers should be added following the same pattern as SubmitQueue and Stovepipe. + +## Ownership by service + +### Orchestrator + +The orchestrator is the only service. It subscribes to two inbound topics (runway-check, runway-land), performs VCS operations through a pluggable extension, and publishes results to two outbound topics (sq-check-result, sq-land-result). It owns no persistent data. + +### Shared: the messaging queue + +Runway communicates with SubmitQueue only through the messaging queue, consistent with the ownership model established by SubmitQueue and Stovepipe. The inbound topics are owned by runway; the outbound topics are owned by SubmitQueue. This keeps the dependency direction clean: runway depends on the shared entity package for message payloads, but not on SubmitQueue's internal state management. From 564ebad696c91c726b8d1c3e21d879d6fd9080a1 Mon Sep 17 00:00:00 2001 From: "kevin.new" Date: Tue, 16 Jun 2026 22:11:59 +0000 Subject: [PATCH 2/3] fix(rfc): clean up workflow diagram box-drawing characters MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove dangling pipe characters and empty lines inside diagram boxes that render as strikethroughs in GitHub rich diff. Change connector characters from cross (┼) to tee (┬) at box edges. Co-Authored-By: Claude Opus 4.6 (1M context) --- doc/rfc/runway/workflow.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/doc/rfc/runway/workflow.md b/doc/rfc/runway/workflow.md index a3a86cf2..cf072a97 100644 --- a/doc/rfc/runway/workflow.md +++ b/doc/rfc/runway/workflow.md @@ -25,9 +25,7 @@ The outbound topics partition by SubmitQueue queue name (not repo/branch), match ``` ┌─────────────────────────────────────────────┐ │ submitqueue orchestrator │ - │ │ - │ │ │ │ - └───────┼───────────────────────┼─────────────┘ + └───────┬───────────────────────┬─────────────┘ │ │ Check (per request) Job (per batch) │ │ @@ -43,11 +41,10 @@ The outbound topics partition by SubmitQueue queue name (not repo/branch), match [sq-check-result] [sq-land-result] │ │ ▼ ▼ - ┌───────┼───────────────────────┼─────────────┐ + ┌───────┬───────────────────────┬─────────────┐ │ check-result ctrl land-result ctrl │ │ (update request (update batch state, │ │ mergeability) fan out to conclude) │ - │ │ │ submitqueue orchestrator │ └─────────────────────────────────────────────┘ ``` From f186c3c08d80837ef000121dd487fa43703e1201 Mon Sep 17 00:00:00 2001 From: "kevin.new" Date: Tue, 16 Jun 2026 22:27:55 +0000 Subject: [PATCH 3/3] docs(rfc): revise runway workflow to match existing RFC style Remove historical context, implementation rationale, and defensive justifications. Match the factual tone and structure of the submitqueue and stovepipe workflow RFCs. Co-Authored-By: Claude Opus 4.6 (1M context) --- doc/rfc/runway/workflow.md | 38 +++++++++++++++----------------------- 1 file changed, 15 insertions(+), 23 deletions(-) diff --git a/doc/rfc/runway/workflow.md b/doc/rfc/runway/workflow.md index cf072a97..61b960ac 100644 --- a/doc/rfc/runway/workflow.md +++ b/doc/rfc/runway/workflow.md @@ -1,24 +1,22 @@ # Runway Workflow -Runway is the landing service: it owns VCS operations — mergeability checking and landing (resolve, apply, push, finalize) — on behalf of SubmitQueue. Unlike SQ and Stovepipe, it is a consumer-only domain with no gateway and no proto API; work arrives via topic queues, results leave via topic queues. The orchestrator subscribes to two inbound topics (runway-check, runway-land) and publishes to two outbound topics (sq-check-result, sq-land-result). There are no internal hops between stages and no cycles — each controller consumes one inbound message and produces one outbound result. +Runway is the landing service: it owns VCS operations — mergeability checking and landing — on behalf of SubmitQueue. The orchestrator subscribes to two inbound topics (`runway-check`, `runway-land`) and publishes results to two outbound topics (`sq-check-result`, `sq-land-result`). It is a consumer-only service with no gateway; work arrives via topic queues and results leave via topic queues. ## Check and land The two queues operate at different granularities: -- **check** is request-level. SubmitQueue publishes a check message to determine whether a request's changes can merge cleanly against the target branch. Runway performs a read-only trial merge and publishes per-change mergeability results back. +- **check** is request-level. A check message carries a request's changes and merge strategy. Runway performs a read-only trial merge and publishes per-change mergeability results back. -- **land** is batch-level. SubmitQueue publishes a job message to land a batch. The job carries the resolved content for each request in the batch (runway has no access to SubmitQueue's request store, so the message must be self-contained). Runway pre-validates, lands, finalizes, and publishes a result with per-item outcomes back. +- **land** is batch-level. A job message carries the resolved content for each request in the batch (runway has no access to SubmitQueue's request store, so the message is self-contained). Runway pre-validates, lands, and publishes a result with per-item outcomes back. -These are independent input→output flows. Check can run without land ever running (SubmitQueue may skip external mergeability checks for some queues), and land does not depend on a prior check having passed through runway. +These are independent input-output flows. Check can run without land ever running, and land does not depend on a prior check. ## Branch serialization -The partition key `repo/target` on both inbound topics serializes all VCS operations for a given branch. The message queue delivers messages with the same partition key to the same consumer in order — at most one check or land operation is in flight for any given branch at any time. +The partition key `repo/target` on both inbound topics serializes all VCS operations for a given branch. The message queue delivers messages with the same partition key to the same consumer in order, so at most one check or land operation is in flight for any given branch at any time. -This replaces the LandQueue extension from the synchronous design — no Redis, no MySQL queue table, no distributed lock. The tradeoff: serialization granularity is fixed at the partition key level. If pipelining becomes necessary (overlapping check N+1 with land N for the same branch), it can be implemented within the controllers without changing the queue topology. - -The outbound topics partition by SubmitQueue queue name (not repo/branch), matching SubmitQueue's existing fan-out model where state updates for the same queue must be serialized. +The outbound topics partition by SubmitQueue queue name, matching SubmitQueue's fan-out model where state updates for the same queue are serialized. ## Workflow @@ -42,9 +40,9 @@ The outbound topics partition by SubmitQueue queue name (not repo/branch), match │ │ ▼ ▼ ┌───────┬───────────────────────┬─────────────┐ - │ check-result ctrl land-result ctrl │ - │ (update request (update batch state, │ - │ mergeability) fan out to conclude) │ + │ check-result ctrl land-result ctrl │ + │ (update request (update batch state, │ + │ mergeability) fan out to conclude) │ │ submitqueue orchestrator │ └─────────────────────────────────────────────┘ ``` @@ -53,29 +51,23 @@ The outbound topics partition by SubmitQueue queue name (not repo/branch), match | Controller | In | Out | One-line role | |---|---|---|---| -| **check** | Check | CheckResult → sq-check-result | Check mergeability of a request's changes against the target branch (read-only) | -| **land** | Job | Result → sq-land-result | Pre-validate, land, and finalize a batch's changes on the target branch | +| **check** | Check | CheckResult -> sq-check-result | Check mergeability of a request's changes against the target branch (read-only) | +| **land** | Job | Result -> sq-land-result | Pre-validate, land, and finalize a batch's changes on the target branch | The check controller always publishes a result — even when all changes are mergeable — so SubmitQueue receives a definitive answer. On infrastructure error it nacks for retry. -The land controller publishes a conflict result (and acks) when pre-validation detects a conflict; SubmitQueue handles rebatching. On infrastructure error it nacks for retry. On success it publishes per-item outcomes (commit SHAs, whether new commits were produced) so SubmitQueue can update its request state. Conflicts are expected outcomes, infrastructure errors are not — this matches the pattern established by SubmitQueue's merge controller. +The land controller publishes a conflict result (and acks) when pre-validation detects a conflict; SubmitQueue handles rebatching. On infrastructure error it nacks for retry. On success it publishes per-item outcomes (commit SHAs, whether new commits were produced) so SubmitQueue can update its request state. ## Idempotency -Runway has no persistent state — no request store, no job store, no database. Idempotency is achieved through the VCS contract: land detects already-pushed changes (commit SHAs reachable from HEAD) and treats them as already-landed; closing an already-closed PR is a no-op. Check is read-only and naturally idempotent. The message queue provides at-least-once delivery; idempotent consumers handle duplicates. - -## DLQ reconciliation - -Runway does not implement DLQ reconciliation controllers. Unlike SubmitQueue and Stovepipe — where a stuck message leaves a request or commit in a non-terminal state forever — runway is stateless. A check or land message that exhausts retries and lands in the DLQ simply means SubmitQueue never receives a result for that request or batch. SubmitQueue's own timeout and retry logic handles the missing response (re-publishing the check or re-batching the requests). Adding DLQ controllers to runway would duplicate that recovery logic without adding safety, since there is no persistent state to reconcile. - -If runway later gains persistent state (e.g., a job store for observability), DLQ controllers should be added following the same pattern as SubmitQueue and Stovepipe. +Runway has no persistent state — no request store, no job store, no database. Idempotency is achieved through the VCS contract: land detects already-pushed changes (commit SHAs reachable from HEAD) and treats them as already-landed; closing an already-closed PR is a no-op. Check is read-only and naturally idempotent. ## Ownership by service ### Orchestrator -The orchestrator is the only service. It subscribes to two inbound topics (runway-check, runway-land), performs VCS operations through a pluggable extension, and publishes results to two outbound topics (sq-check-result, sq-land-result). It owns no persistent data. +The orchestrator is the only service. It subscribes to two inbound topics (`runway-check`, `runway-land`), performs VCS operations through a pluggable extension, and publishes results to two outbound topics (`sq-check-result`, `sq-land-result`). It owns no persistent data. ### Shared: the messaging queue -Runway communicates with SubmitQueue only through the messaging queue, consistent with the ownership model established by SubmitQueue and Stovepipe. The inbound topics are owned by runway; the outbound topics are owned by SubmitQueue. This keeps the dependency direction clean: runway depends on the shared entity package for message payloads, but not on SubmitQueue's internal state management. +Runway communicates with SubmitQueue only through the messaging queue. The inbound topics are owned by runway; the outbound topics are owned by SubmitQueue.