moraxu · Nov 1, 2026
@@ -5,6 +5,7 @@ Per-marathon YAML files consumed by ``test_disagg_cancel_stress.py``.
 | File | Model | KV cache | Transceiver |
 |------|-------|----------|-------------|
 | `marathon_python_v2_deepseek.yaml` | `DeepSeek-V3-Lite/bf16` (MLA) | V2 | Python |
+| `marathon_python_v2_musespark_open_attach.yaml` | `musespark_open` (served name; attaches to an externally launched 1P1D cluster) | V2 | Python |
 
 Only configurations listed in `_MARATHON_CONFIGS` in
 `test_disagg_cancel_stress.py` are actually parametrized; the other
@@ -43,6 +44,57 @@ When switching from `log_only` to `full_cancel_poison`, update
 the test-list timeout together. The top-level README has the exact
 checklist.
 
+## Attach mode: `stress_config.external_cluster`
+
+By default the harness launches the disaggregated cluster locally via
+`setup_disagg_cluster` (one node, `Popen` workers). Models whose
+per-instance GPU footprint exceeds one node (e.g. TP4 instances on
+4-GPU nodes, where 1P1D needs 8 GPUs) cannot be launched that way; for
+those, boot the cluster externally (one node per instance) and let the
+marathon **attach** to it:
+
+```yaml
+stress_config:
+  mode: log_only            # full_cancel_poison is rejected (see below)
+  external_cluster:
+    server_url: ${TRTLLM_TEST_EXTERNAL_CLUSTER_URL}
+    server_log: ${TRTLLM_TEST_EXTERNAL_CLUSTER_LOG_DIR}/orch/server.log   # optional
+    worker_logs:            # required, >=1 entry; feeds the log scanner
+      - role: ctx           # ctx | gen; per-role index = YAML order
+        path: ${TRTLLM_TEST_EXTERNAL_CLUSTER_LOG_DIR}/ctx/server.log
+      - role: gen
+        path: ${TRTLLM_TEST_EXTERNAL_CLUSTER_LOG_DIR}/gen1/server.log
+    worker_health_urls: []  # optional; each is GET-checked once at setup()
+    model_name: musespark_open # optional; falls back to top-level `model`
+```
+
+When the block is present, `harness.setup()` skips the local launcher
+entirely: it binds the orchestrator endpoint for the probe thread,
+verifies `server_url` + `worker_health_urls` respond with HTTP 200,
+and registers the log paths for the log-scanner thread. The attach
+test itself needs **no GPUs** — only HTTP reachability to the
+orchestrator and filesystem access to the worker logs (a shared
+filesystem when the cluster spans nodes).
+
+**Endpoint indirection is via environment variables**, not YAML
+templating: string fields may contain `${VAR}` references, expanded by
+the harness at `setup()` time. An unset variable is a hard error at
+setup, but parse/validation is env-independent, so checked-in attach
+YAMLs pass `test_all_marathon_yamls_parse_and_validate` (and pytest
+collection) with nothing exported; the parametrized marathon test
+**skips** when a referenced variable is unset. The conventional
+variables used by the shipped attach YAML are
+`TRTLLM_TEST_EXTERNAL_CLUSTER_URL` (orchestrator base URL) and
+`TRTLLM_TEST_EXTERNAL_CLUSTER_LOG_DIR` (serve directory containing the
+worker/orchestrator logs).
+
+Attach mode supports `mode: log_only` only. `full_cancel_poison` is
+rejected by `StressConfig.validate()` because fault injection signals
+locally-launched worker subprocesses by PID and respawns them through
+the shared launcher, neither of which exists for an external cluster.
+Teardown of the external cluster is likewise the launcher's job, not
+the harness's — `harness.stop()` only stops its own threads.
+
 ## Backend-knob axis: KV-cache manager × transceiver runtime
 
 The registered marathon config explicitly selects `NIXL` with the `PYTHON`

@@ -0,0 +1,62 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+# Muse Spark Open marathon in attach mode: V2 KV cache + Python transceiver, 1P1D.
+#
+# At TP4 per instance, 1P1D needs 8 GPUs. On 4-GPU nodes the two instances
+# cannot share one node (two weight-resident TP4 engines exceed device memory),
+# so the cluster is launched externally, one node per instance, and the
+# marathon attaches to it through stress_config.external_cluster instead of
+# launching workers itself. That is also why no context_servers or
+# generation_servers sizing appears here.
+#
+# The run environment provides the endpoints through ${VAR} references, which
+# the harness resolves at setup() time (see README.md in this directory):
+#   TRTLLM_TEST_EXTERNAL_CLUSTER_URL      e.g. http://nodename:8340
+#   TRTLLM_TEST_EXTERNAL_CLUSTER_LOG_DIR  serve directory containing
+#                                         ctx/server.log, gen1/server.log and
+#                                         orch/server.log
+#
+# model: is the name the external cluster serves (the request envelope), not a
+# checkpoint path: attach mode loads no weights.
+
+model: musespark_open
+backend: pytorch
+
+# ============================================================================
+# Stress harness configuration (consumed by harness.py::StressConfig).
+# Schema documented in ../README.md.
+# ============================================================================
+stress_config:
+  mode: log_only
+  duration_min: 10
+
+  kv_cache_manager: v2
+  transceiver: python
+
+  external_cluster:
+    server_url: ${TRTLLM_TEST_EXTERNAL_CLUSTER_URL}
+    server_log: ${TRTLLM_TEST_EXTERNAL_CLUSTER_LOG_DIR}/orch/server.log
+    worker_logs:
+      - role: ctx
+        path: ${TRTLLM_TEST_EXTERNAL_CLUSTER_LOG_DIR}/ctx/server.log
+      - role: gen
+        path: ${TRTLLM_TEST_EXTERNAL_CLUSTER_LOG_DIR}/gen1/server.log
+
+  log_only_probe:
+    interval_s: 30
+    prompt: "Write one sentence about reliable distributed inference."
+    max_tokens: 32
+    seed: 42
+    request_timeout_s: 300   # first probe pays one-time JIT/warmup cost
+
+  log_scan:
+    hard_zero_patterns:
+      - "Broken promise"
+      - "Segfault"
+      - "Segmentation fault"
+      - "SIGSEGV"
+      - "0xffffffffffffffff"
+      - "use-after-free"
+      - "heap-use-after-free"
+      - "AddressSanitizer:.*use-after-free"
+      - "double[- ]free"