Skip to content

refactor(test-utils): one tracing capture layer for span and event assertions - #2659

Merged
gold-silver-copper merged 4 commits into
0xPlaygrounds:mainfrom
gold-silver-copper:arch-smell-798902
Oct 1, 2026
Merged

gold-silver-copper merged 4 commits into
0xPlaygrounds:mainfrom
gold-silver-copper:arch-smell-798902

Conversation

@gold-silver-copper

@gold-silver-copper gold-silver-copper commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

refactor(test-utils): one tracing capture layer for span and event assertions

Description

Smell: duplicated test fixtures. The unit tests of rig-core and rig-agent, plus one cassette driver, asserted on telemetry through twelve hand-written tracing_subscriber::Layer + Visit pairs. Each one re-implemented "remember every span's name, target, parent and fields" or "remember every WARN event's message":

file private capture types removed
rig-core/src/driver/dyn_model/tests.rs Spans, Values
rig-core/src/telemetry/equivalence_tests.rs Spans, Values (a copy of the one above, plus parents)
rig-core/src/telemetry/tests.rs ModalityCapture*, CapturedFields/FieldCapture*, CapturedWarnings/WarningCapture*, CapturedSpan/SpanCaptureLayer/StringFieldVisitor, contains_string
rig-core/src/driver/tests.rs RecordFields, an inline Warnings layer, a shared Visit
rig-agent/src/agent/streaming/tests.rs CapturedSpan(s), SpanCaptureLayer, two visitors, CapturedField
rig-agent/src/agent/engine/tests.rs Captured/CaptureLayer/FieldVisitor, ResultValueLayer/ResultValueVisitor
rig-agent/src/agent/builder/tests.rs an inline Warnings layer
rig-cassette/tests/common/request_identity.rs Recorded, Visitor, Capture

Why it hurts. Every copy made its own choice about which record_* methods it overrides (one silently dropped every &str into Debug quoting, another ignored non-u64 fields, another kept only the newest span and attached every record call to it whatever its id), whether it sees creation-time values or only record calls, how it stringifies numbers, and how it resolves parents. A new telemetry test had to pick one of twelve slightly different captures, or write a thirteenth.

Change. rig_core::test_utils::TraceCapture (behind the existing test-utils feature) is one layer that records, per span, its id, name, target, parent, declared fields, creation-time values, every later record call in order and its follows_from edges, and, per event, its level, target and fields. Values are JSON (strings stay strings, integers are numbers). Every copy above now uses it. Each test keeps its assertions and the same view of the trace it asserted on: creation-time values vs later recordings (initial / recorded / record_count), all values of a field across spans (values_of), the last span (last_span), WARN messages (warnings), or WARN-and-above events (the builder test filters events() itself to keep its >= WARN scope).

Measured: git diff --shortstat origin/main...HEAD = 13 files changed, 582 insertions(+), 1366 deletions(-): net LOC -784. The shared layer is ~250 lines including docs and a doctest.

Decisions

  • The layer lives in rig_core::test_utils (not rig-test-support) because rig-core's and rig-agent's own unit tests need it, and they cannot depend on the facade-level support crate. That makes tracing-subscriber an optional rig-core dependency enabled only by test-utils; it was already a dev-dependency, so Cargo.lock gains nothing. Checked: cargo check -p rig-core --no-default-features --features test-utils --lib --target wasm32-unknown-unknown.
  • rig-cassette's tracing-subscriber dev-dependency is removed: request_identity.rs was its only user.
  • Assertions that compared stringified values now compare JSON values (Some("\"embeddings\"") became Some(json!("embeddings")), Some("4") became Some(json!(4))). These are the same recorded values in a typed form, not weaker checks.
  • telemetry::tests' old SpanCaptureLayer kept only the most recent span and attached every record call to it. The tests now read last_span(), whose recorded holds only that span's own recordings. Every such test asserts on recordings made to the span it reads, so they all still pass, and a stray recording on another span can no longer satisfy them by accident.

Rejected candidates in this lane

  • recorded_request / recorded_response / recorded_stream_chunks wrappers in 17 OpenRouter/OpenAI/Mistral/DeepSeek cassette matrix files: each is a 3-line wrapper over an existing crate::cassettes::recorded_* helper that pins the provider name. Inlining them is cosmetic (-150) and doesn't remove a real abstraction, and it would add churn next to chore: derive cassette suites from the tree and sort-check manifest lists #2437's cassette work.
  • scripted_unary in 11 ecs_faults / ecs_matrix_long_loop files: the bodies differ in client, wire, model and thinking dialect. A shared constructor would save ~5 lines per copy and add a generic signature, so roughly -40 net. Not worth it.
  • text_of / tool_request / drain helpers: they share a name but take different types and do different work. Not real duplication.

Changelog

  • (test-utils) Add rig_core::test_utils::TraceCapture, a tracing layer that records spans (values at creation and every later recording, parents, follows_from) and events for test assertions. The test-utils feature now enables tracing-subscriber.

Migration

None.

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Test refactor (plus one test-utils addition)

Testing

  • cargo nextest run --locked --profile local -p rig-core --lib: 1516 passed
  • cargo nextest run --locked --profile local -p rig-agent --lib: 661 passed
  • cargo test --locked -p rig-core --doc: includes the new trace_capture doctest
  • RIG_PROVIDER_TEST_MODE=replay cargo nextest run --locked --profile local -p rig-cassette --all-features --test <groq|xai|anthropic|openai> request_identity: all passed (cassettes replayed, none recorded or changed)
  • cargo clippy --locked -p rig-core -p rig-agent --all-features --tests -- -D warnings
  • cargo check --locked -p rig-core --no-default-features --features test-utils --lib --target wasm32-unknown-unknown

Checklist:

  • I have updated READMEs and Rust docs affected by this change
  • I have added tests that prove my fix is effective or that my feature works (existing tests ported; a doctest covers the new type)
  • I did not edit CHANGELOG.md or MIGRATING.md (they are generated at release)

Notes

No cassette, golden or recorded request changed.

@gold-silver-copper
gold-silver-copper marked this pull request as ready for review October 1, 2026 00:17
@gold-silver-copper
gold-silver-copper added this pull request to the merge queue Oct 1, 2026
Merged via the queue into 0xPlaygrounds:main with commit 47dea85 Oct 1, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant