Skip to content

fix(anthropic): materialize generator messages, system, and tools before extraction (#825) - #865

Closed
Pr1meGG wants to merge 5 commits into
open-telemetry:mainfrom
Pr1meGG:fix/genai-anthropic-iterator
Closed

Pr1meGG wants to merge 5 commits into
open-telemetry:mainfrom
Pr1meGG:fix/genai-anthropic-iterator

Conversation

@Pr1meGG

@Pr1meGG Pr1meGG commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Materialize generator and one-shot iterator inputs for messages, system, and tools in kwargs before telemetry extraction in Anthropic create and stream methods when content capture is enabled, leaving them unmaterialized when content capture is disabled.

This prevents generator exhaustion in Anthropic instrumentation, mirroring the fix in #864 for OpenAI (Issue #825).

Known gaps or concerns

None.

Copilot AI balanced review requested due to automatic review settings October 9, 2026 07:02
@Pr1meGG
Pr1meGG requested a review from a team as a code owner October 9, 2026 07:02
@opentelemetry-pr-dashboard

opentelemetry-pr-dashboard Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Pull request dashboard status

Closed · refreshed 2026-10-19 12:10 UTC

Status above doesn't look right?
  • Anything look wrong? Report it with what you expected; it helps us improve the dashboard.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

It does not fix linked OpenAI issue #825 and lacks async stream regression coverage.

2 open findings
What changed in this PR

Fixes one-shot iterator handling in Anthropic instrumentation, though linked issue #825 concerns OpenAI.

Changes:

  • Materializes messages, system, and tools iterators.
  • Adds sync and async regression tests.
  • Adds a changelog fragment.
File Description
patch.py Materializes request iterators before extraction and SDK calls.
test_sync_messages.py Tests sync create and stream behavior.
test_async_messages.py Tests async create behavior.
.changelog/​825.fixed Documents the Anthropic fix.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

SDK and the tool definitions read it. Call this before either of them does:
``create`` reads the request after this wrapper, but ``stream`` serializes
it while building the manager, before the invocation is created.
def _materialize_iterables(kwargs: dict[str, Any]) -> None:
kwargs: dict[str, Any],
) -> AsyncMessagesStreamManagerWrapper[Any]:
_materialize_tools(kwargs)
_materialize_iterables(kwargs)
@opentelemetry-pr-dashboard

This comment has been minimized.

@Pr1meGG

Pr1meGG commented Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

Responding to review findings

Both findings are addressed.

Finding 1: #825 is OpenAI-only — this PR does not fix it

Addressed. The PR was never intended to fix the OpenAI issue; it fixes the same class of bug for Anthropic. Evidence:

The PR title includes (#825) as the originating issue reference. I can remove it if preferred — happy to.

Finding 2: Missing async stream messages/system regression tests

Addressed. The tests already exist in this PR (added after the review).

Test Line Asserts request body Asserts telemetry
test_async_messages_stream_messages_generator_is_recorded 1172 seen["body"]["messages"] == [user_msg] input_messages span
test_async_messages_stream_system_generator_is_recorded 1217 seen["body"]["system"] == [system_block] system_instruction span
test_async_messages_stream_tools_generator_is_recorded 1127 seen["body"]["tools"] == [tool_def] tool_definitions span

All use MockTransport to capture the serialized request, then assert both the JSON body and the span attributes. All 8 async generator tests + all 5 sync generator tests pass:

$ pytest -k generator
6 passed, 170 deselected

These tests were added as part of PR #865 itself but appear to have been missed in review.

@MikeGoldsmith

Copy link
Copy Markdown
Member

Thanks for the work on this @Pr1meGG, and for the detailed replies on the review threads.

I think this one has been overtaken by #873, which landed the same fix for opentelemetry-instrumentation-genai-anthropic using the shared IteratorCapture helper in opentelemetry-util-genai. Looking at the current diff against main, the patch.py change isn't in it any more. The merge in f87e02d resolved that file to main's version, so what's left is the changelog fragment and the tests.

On those two, .changelog/865.fixed describes the same user-visible change as 873.fixed, so we'd end up listing it twice in the release notes. And tests/test_iterator_transparency.py came in with #873 and parametrises messages, system and tools across sync / async, streaming and non-streaming, and capture content on and off, so I think it already covers what the tests here assert.

Given that, I think it makes sense to close this. If you spot a case that the matrix in test_iterator_transparency.py misses, a small test-only PR for it would be welcome, and easier to review on its own.

Thanks again 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants