Skip to content

feat: add opt-in AgentPond tracing - #17

Open
marcusschiesser wants to merge 1 commit into
PentesterFlow:mainfrom
marcusschiesser:agent/add-agentpond-tracing
Open

feat: add opt-in AgentPond tracing#17
marcusschiesser wants to merge 1 commit into
PentesterFlow:mainfrom
marcusschiesser:agent/add-agentpond-tracing

Conversation

@marcusschiesser

Copy link
Copy Markdown

What changed

  • adds opt-in OpenInference LLM spans around PentesterFlow's real OpenAI-compatible chat and streaming path
  • exports spans through AgentPond's environment-driven Files SDK exporter
  • records structural request/response metadata only; prompts, responses, targets, and credentials are excluded
  • documents local setup and keeps .agentpond/ state out of git

AgentPond fits PentesterFlow's audit-oriented workflow because operators can keep trace storage local or select their own Files SDK provider. Nothing is exported unless AGENTPOND_ENABLED=true and a Files SDK environment is configured.

Validation

Commands run:

npm ci --dry-run
npm run typecheck
npm run lint
npm run test
npm run build

All passed: 62 test files / 657 tests, plus the production bundle.

End-to-end trace validation used the existing in-process OpenAI-compatible server fixture, which exercises the actual OpenAIClient HTTP and SSE paths:

AGENTPOND_ENABLED=true \
FILES_SDK_PROVIDER=fs \
FILES_SDK_ROOT="$PWD/.agentpond/envs/local/objects" \
AGENTPOND_PROJECT_ID=default-project \
npm run test -- src/llm/openai.test.ts

npx agentpond sync
npx agentpond traces list --limit 10 --json
npx agentpond traces get 794674826ab0764c2a8c0d6edfbbc8cb --json

The focused integration suite passed all 17 tests. AgentPond processed 15 objects / 30 events, and trace 794674826ab0764c2a8c0d6edfbbc8cb read back as openai-compat.chat with model qwen-coder, OpenInference span kind LLM, and privacy-safe request/output counts.

Limitations

This first integration covers OpenAI-compatible backends (including LM Studio, Kimi, OpenRouter, and DeepSeek routing through that client). The separate Ollama, Gemini, and Anthropic clients are unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant