Skip to content

feat: Add OpenAI Batch API Instrumentation - #2369

Open
Luca Forstner (lforst) wants to merge 8 commits into
mainfrom
lforst/dum-e/zurich-58ae796ad4
Open

feat: Add OpenAI Batch API Instrumentation#2369
Luca Forstner (lforst) wants to merge 8 commits into
mainfrom
lforst/dum-e/zurich-58ae796ad4

Conversation

@lforst

@lforst Luca Forstner (lforst) commented Aug 17, 2026

Copy link
Copy Markdown
Member

Adds resumable tracing for OpenAI Batch jobs. openaiFilesCreateTraced() starts a task span with pending LLM spans from a batch input upload; openaiBatchesRetrieveTraced() records OpenAI lifecycle timestamps; completeOpenAIBatchTrace() adds outputs and errors from caller-supplied batch files.

const inputFile = await openaiFilesCreateTraced(client.files)({
  file,
  purpose: "batch",
});

const batch = await client.batches.create({
  input_file_id: inputFile.id,
  endpoint: "/v1/chat/completions",
  completion_window: "24h",
});

Complete the trace from a poller or webhook handler. File requests start concurrently; supplied response bodies are streamed and consumed:

const batch = await openaiBatchesRetrieveTraced(client.batches)(batchId);

await completeOpenAIBatchTrace({
  inputFileId: batch.input_file_id,
  inputFileContent: client.files.content(batch.input_file_id),
  outputFileContent: batch.output_file_id
    ? client.files.content(batch.output_file_id)
    : undefined,
  errorFileContent: batch.error_file_id
    ? client.files.content(batch.error_file_id)
    : undefined,
});
openai.batch [task]
├── Chat Completion [llm]  input → output + tokens
├── Chat Completion [llm]  input → output + tokens
└── Chat Completion [llm]  input → error

@lforst

Copy link
Copy Markdown
Member Author

Abhijeet Prasad (@AbhiPrasad) refined the API a bit, see PR description.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant