Repository navigation
Feat/ollama instrumentation - #1012
piyush182004 wants to merge 8 commits into
Conversation
Add opentelemetry-instrumentation-genai-ollama, which records chat and embeddings telemetry for the Ollama Python library. Instruments chat (including streaming) and embed on Client and AsyncClient. The module-level ollama.chat and ollama.embed are bound to a default client at import time, so they are re-bound to the instrumented client and restored on uninstrument.
A chat call without messages only loads or unloads the model, so it no longer records a chat span. server.port fell back to 11434 when the host had no explicit port, which is wrong for hosts like https://ollama.com. The port is now recorded only when the URL has one. A negative num_predict means no limit (-1) or fill the context (-2), so it is no longer recorded as gen_ai.request.max_tokens.
An embed call without input embeds nothing, so it no longer records an embeddings span. Messages with nothing to record are left out of the input messages, as in the OpenAI instrumentation. Add tests for clients created before instrument(), message objects, tool generators, stream errors, abandoned streams, and concurrent async calls, and document that tool call ids and images are not captured.
Pull request dashboard statusWaiting on the author · refreshed 2026-11-03 05:34 UTC Respond to 3 review items (e.g. link a commit, explain why not, ask a follow-up): Status above doesn't look right?
|
|
Do we have native instrumentation in this parent sdk? |
I just checked ollama has no native instrumentation ! |
| assert sorted( | ||
| s.attributes[GenAI.GEN_AI_USAGE_OUTPUT_TOKENS] for s in spans | ||
| ) == sorted(response.eval_count for response in responses) | ||
| assert len({s.context.trace_id for s in spans}) == 3 |
There was a problem hiding this comment.
The async paths have no error tests, and neither does embed (sync or async). There's also no test for a caller-side error raised in the loop body before the stream is drained. Something like:
from opentelemetry.semconv.attributes import error_attributes
from opentelemetry.trace import StatusCode
@pytest.mark.asyncio
async def test_async_chat_error_is_recorded_and_reraised(
instrument_no_content, span_exporter, vcr
):
with vcr.use_cassette("test_chat_error_is_recorded_and_reraised.yaml"):
with pytest.raises(ollama.ResponseError):
await ollama.AsyncClient(host=HOST).chat(
model="no-such-model:1b", messages=MESSAGES
)
(span,) = span_exporter.get_finished_spans()
assert span.status.status_code == StatusCode.ERROR
assert (
span.attributes[error_attributes.ERROR_TYPE]
== "ollama._types.ResponseError"
)
@pytest.mark.asyncio
async def test_async_chat_stream_error_is_recorded(
instrument_no_content, span_exporter, vcr
):
with vcr.use_cassette("test_chat_stream_error_is_recorded.yaml"):
stream = await ollama.AsyncClient(host=HOST).chat(
model="no-such-model:1b", messages=MESSAGES, stream=True
)
with pytest.raises(ollama.ResponseError):
[chunk async for chunk in stream]
(span,) = span_exporter.get_finished_spans()
assert span.status.status_code == StatusCode.ERROR
assert (
span.attributes[error_attributes.ERROR_TYPE]
== "ollama._types.ResponseError"
)
@pytest.mark.asyncio
async def test_async_chat_stream_caller_error_ends_span(
instrument_no_content, span_exporter, vcr
):
with vcr.use_cassette(
"test_async_chat_stream_records_the_drained_response.yaml"
):
stream = await ollama.AsyncClient(host=HOST).chat(
model=CHAT_MODEL, messages=MESSAGES, stream=True
)
with pytest.raises(ValueError):
async for _ in stream:
raise ValueError("boom")
await stream.aclose()
(span,) = span_exporter.get_finished_spans()
assert span.status.status_code == StatusCode.ERRORThere was a problem hiding this comment.
Added the async chat, async stream and embed error tests.
For the caller error test, the span ends as UNSET, not ERROR. The error happens inside my own loop, so the stream wrapper never sees it, and close() after that counts as a normal stop in util-genai. Other instrumentations work the same way.
For now the test checks that the span ends once after close().
Should it be ERROR? If yes, I think that change belongs in util-genai. I can open a separate PR for it if you want.
There was a problem hiding this comment.
UNSET is fine here - the wrapper can't see an error raised in the loop body. Could you add a variant using async with stream: that asserts ERROR, since that's the path where we can record it? No util change needed.
…dimensions Messages and tools are no longer copied into lists before the call, so Ollama gets what the caller passed. The dimension count now comes from the dimensions argument instead of the response.
Covers async and embed errors, streams closed after a caller error, duration and token usage metrics, the completion hook, streamed tool calls and reasoning, and the chat span not being current between stream reads. Drops the nested inference test.
| "Programming Language :: Python :: 3.15", | ||
| ] | ||
| dependencies = [ | ||
| "opentelemetry-api ~= 1.43", |
There was a problem hiding this comment.
All the other packages use ~= 1.45 for opentelemetry-api now. Could you match them so the shared OTel deps stay in sync?
| "opentelemetry-api ~= 1.43", | |
| "opentelemetry-api ~= 1.45", |
| messages_capture = IteratorCapture.from_property( | ||
| kwargs, "messages", enabled=capture_content | ||
| ) | ||
| tools_capture = IteratorCapture.from_property(kwargs, "tools") |
There was a problem hiding this comment.
nit: tool definitions count as content (util only emits them when capture is on), so this could be gated like messages - that also keeps the caller's tools unwrapped when capture is off. Same in _async_chat.
| tools_capture = IteratorCapture.from_property(kwargs, "tools") | |
| tools_capture = IteratorCapture.from_property( | |
| kwargs, "tools", enabled=capture_content | |
| ) |
| assert sorted( | ||
| s.attributes[GenAI.GEN_AI_USAGE_OUTPUT_TOKENS] for s in spans | ||
| ) == sorted(response.eval_count for response in responses) | ||
| assert len({s.context.trace_id for s in spans}) == 3 |
There was a problem hiding this comment.
UNSET is fine here - the wrapper can't see an error raised in the loop body. Could you add a variant using async with stream: that asserts ERROR, since that's the path where we can record it? No util change needed.
Description
Fixes #1011
Adds
opentelemetry-instrumentation-genai-ollamafor the Ollama Python library. It recordschatspans (including streaming, tool calls, and reasoning) andembeddingsspans forClientandAsyncClient, plus the module-levelollama.chatandollama.embed, which are bound to a default client at import time and are re-bound on instrument.Known gaps: Ollama returns no response id, so chat spans have no
gen_ai.response.id(declared in the conformance scenarios). Chat calls without messages and embed calls without input are not recorded, since no inference runs. Tool call ids and images are not captured.generateand the deprecatedembeddingsmethod are not instrumented yet.Type of change
How has this been tested?
Cassettes were recorded against a local Ollama server (
qwen2.5:0.5b,deepseek-r1:1.5b,nomic-embed-text).py311-test-instrumentation-genai-ollama-oldest(ollama 0.4.0)py311-test-instrumentation-genai-ollama-latestpy315-test-instrumentation-genai-ollama-latestpy314-test-instrumentation-genai-ollama-conformance(Weaver)typecheckandprecommitChecklist
See CONTRIBUTING.md
for the style guide, changelog guidance, and more.