Repository navigation
feat(agno): capture resolved workflow HITL step requirements as user input messages (#906) - #1009
VimalN2005 wants to merge 4 commits into
Conversation
…input messages (open-telemetry#906) In Agno workflows, human-in-the-loop (HITL) step reviews are represented by StepRequirement objects passed via step_requirements or run_response. Since workflow steps are orchestration checkpoints rather than LLM tool calls, resolved step requirements were previously omitted. Capture resolved StepRequirement objects on Workflow.continue_run and Workflow.acontinue_run spans as role: "user" input messages when content capture is enabled, representing human decisions and inputs during workflow orchestration. Fixes open-telemetry#906 Signed-off-by: VimalN2005 <vimalsahani2005@gmail.com>
Pull request dashboard statusWaiting on the author · refreshed 2026-11-01 22:38 UTC Respond to 1 review item (e.g. link a commit, explain why not, ask a follow-up):
Status above doesn't look right?
|
…ment (open-telemetry#906) Address reviewer feedback: - Retain only the last (active) requirement when step_requirements is passed explicitly or read from run_response, preventing duplicate history from earlier pause cycles. - Extract executor-level HITL resolution details (tool_call_id, tool_name, confirmed, confirmation_note, result, user_input, user_feedback) from executor_requirements. - Format code and tests with ruff to fix CI pre-commit failure. - Add unit test for executor-level HITL requirements. Signed-off-by: VimalN2005 <vimalsahani2005@gmail.com>
|
Thanks @DylanRussell! I reviewed semconv PR #188 — that proposal introduces individual step execution spans (
|
lmolkova
left a comment
There was a problem hiding this comment.
Thanks for picking this up! I'm not sure gen_ai.input.messages is the right place for this as is. The user message currently carries a JSON dump of Agno state (step_id, step_name, executor ids, tool_call_id, confirmation_message), and most of that isn't what the human said. confirmation_message is the developer-defined prompt shown to the human, and executor-level results are tool responses that belong on the inner agent's span.
Two options:
- narrow it to the human's actual answers (
confirmed,user_input,selected_choices,edited_output,rejection_feedback) with no framework metadata, or - park it until open-telemetry/semantic-conventions-genai#188 settles how workflow steps and HITL should look.
…open-telemetry#1009) Signed-off-by: VimalN2005 <vimalsahani2005@gmail.com>
…st (open-telemetry#1009) Signed-off-by: VimalN2005 <vimalsahani2005@gmail.com>
Problem
In #731,
Workflow.continue_runandWorkflow.acontinue_runwere instrumented to emitinvoke_workflowspans. However, resolved human-in-the-loop (HITL) step requirements (StepRequirement) were omitted fromgen_ai.input.messagesbecause workflow steps are orchestration checkpoints rather than LLM tool calls (role: "tool"), as described in #906.Solution
StepRequirementobjects duringWorkflow.continue_runandWorkflow.acontinue_runtogen_ai.input.messageswithrole: "user", representing human interaction and input that resumes the workflow.step_id&step_nameconfirmed,confirmation_message,rejection_feedbackuser_inputselected_choicesoutput_reviewedandedited_outputOTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=false), step requirements are omitted from spans.run_response.step_requirements, only the active (latest) step requirement is captured.Fixes #906
Type of Change