Skip to content

feat(ui): show DefenseClaw enforcement spans in Monitor#249

Closed
vineethsai7 wants to merge 3 commits into
agentcontrol:feature/68338-add-defense-claw-evaluatorfrom
vineethsai7:codex/defenseclaw-monitor-spans
Closed

feat(ui): show DefenseClaw enforcement spans in Monitor#249
vineethsai7 wants to merge 3 commits into
agentcontrol:feature/68338-add-defense-claw-evaluatorfrom
vineethsai7:codex/defenseclaw-monitor-spans

Conversation

@vineethsai7

@vineethsai7 vineethsai7 commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Stack note: This PR is intentionally based on feature/68338-add-defense-claw-evaluator and is a UI/observability follow-up to #248. It contains only the Monitor-tab commits. Once #248 lands, this PR can be retargeted to main.

Problem

Agent Control's Monitor tab currently shows aggregate execution, trigger, and error statistics, but an operator cannot inspect the DefenseClaw enforcement event that caused a request to be blocked. That makes it difficult to verify the integration end to end or correlate a denial with its trace, span, request, rule, and blocked input.

Solution

Query recent observability events for the selected agent and render them below the aggregate charts. The latest event is expanded automatically while still respecting a user's explicit accordion selection.

For DefenseClaw enforcement events, the view shows:

  • trace, span, request, rule, action, and duration metadata;
  • the exact blocked prompt, raw request body, and enforcement reason when DefenseClaw sends content;
  • an UNREDACTED or METADATA ONLY badge so the privacy mode is explicit;
  • a distinct event-query error state instead of presenting API failures as an empty result.

The query recomputes its relative start time on every refetch and does not retain a prior agent's event data while a new agent query is loading.

What Changed

Path Change
ui/src/core/api/client.ts Exposes the observability events query through the UI API client.
ui/src/core/hooks/query-hooks/use-agent-events.ts Adds the agent-scoped rolling-window query hook.
ui/src/core/page-components/agent-detail/monitor/index.tsx Integrates recent executions and event-query states into Monitor.
ui/src/core/page-components/agent-detail/monitor/recent-executions.tsx Renders expandable enforcement events and exact blocked content.
ui/tests/agent-stats.spec.ts Covers unredacted, metadata-only, error, selection, and refetch behavior.
ui/tests/fixtures.ts Adds observability event fixtures and loads them in authenticated test setup.
evaluators/contrib/defenseclaw/README.md Documents how DefenseClaw events appear in Agent Control.

Security and Privacy

  • Event content is rendered as ordinary React text; no raw HTML rendering is used.
  • Metadata-only events never render absent content fields.
  • Removing previous-query retention prevents content from one agent flashing in another agent's view.
  • Whether content is unredacted is controlled by the DefenseClaw integration; the UI labels the received mode explicitly.

Review Feedback Addressed

  • Authenticated Playwright fixtures now register the observability event route.
  • Monitor imports follow the repository sort order.
  • Accordion selection is derived without synchronously mutating state from an effect.
  • Conditional rendering uses explicit null branches and cannot leak falsey numeric values into the DOM.

Test Plan

  • npm run lint — passed with no warnings or errors.
  • npm run typecheck — passed.
  • npm run prettify:check — passed.
  • npx playwright test tests/agent-stats.spec.ts tests/auth.spec.ts --reporter=line --workers=2 — 17 passed.
  • Live DefenseClaw integration — a blocked request appeared in Monitor with its exact prompt, raw request, reason, trace ID, span ID, request ID, and matching rule IDs.

@vineethsai7 vineethsai7 marked this pull request as ready for review July 9, 2026 17:35
@namrataghadi-galileo namrataghadi-galileo self-requested a review July 9, 2026 17:53
Comment thread ui/tests/fixtures.ts
Comment thread ui/src/core/page-components/agent-detail/monitor/index.tsx
Comment thread ui/src/core/page-components/agent-detail/monitor/recent-executions.tsx Outdated
@codecov

codecov Bot commented Jul 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@yashsheth46 yashsheth46 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Are we already logging some metadata from defenseclaw execution to be surfaced through agent-control monitoring?

  2. Will that metadata be similar to what we populate in control spans, so splunk can pick it up as well?

trace/span/request correlation, control and rule identity, action, and duration.
When the DefenseClaw integration is enabled, it includes exact blocked input,
raw request body, and enforcement reason by default. Monitor labels those spans
`UNREDACTED` and renders the content in the execution drill-down. DefenseClaw

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why "UNREDACTED"? Is redaction the only action we usually take? How about UNCHANGED or ALLOWED?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. UNREDACTED is not the control action; it describes only the disclosure state of the event payload. DefenseClaw sets metadata.content_unredacted=true when the prompt, raw request body, and reason arrive without redaction. The separate action badge remains DENY. This feed reports final blocks, so ALLOWED would be incorrect, and UNCHANGED could be read as an enforcement decision.

There are actually three useful content states: FULL CONTENT, REDACTED CONTENT, and METADATA ONLY (include_content=false). Those labels are clearer than UNREDACTED, and the README should describe this as a content/privacy label rather than an action.

Copy link
Copy Markdown
Collaborator Author

Thanks — answers to both questions:

  1. Yes, for final managed blocks. The complementary DefenseClaw implementation in cisco-ai-defense/defenseclaw#457 tails DefenseClaw's structured event stream and emits one SDK ControlExecutionEvent per Agent Control control whose managed rule IDs contributed to a final block. It deliberately does not report every DefenseClaw execution. The event carries the deterministic execution ID; trace/span/request correlation; Agent Control control ID/name; matching DefenseClaw rule IDs; pre/post stage; llm_call/tool_call; action=deny; confidence; timestamp and duration; evaluator; severity, direction, connector, policy/evaluation IDs when present; and, subject to DefenseClaw's redaction/content settings, the blocked input, raw body, and reason.

  2. The payload is compatible with Agent Control control spans and Splunk OTLP ingestion, but current delivery has an important boundary. ControlExecutionEvent is the core observability model. The Python SDK's OTEL adapter converts it to an agent_control.control_execution span, preserves the supplied trace ID and parent span ID, maps the standard fields to agent_control.*, and maps event metadata to agent_control.metadata.*. Splunk Observability can ingest that shape through OTLP. DefenseClaw also sends the source structured verdict through its normal OTel/Splunk log fan-out, with trace/request/evaluation/rule correlation and DefenseClaw's global redaction policy.

The current DefenseClaw integration selects the SDK's default server sink so these events populate Agent Control Monitor. The built-in otel sink is an alternative selection, not an automatic dual-write. Therefore, today Agent Control Monitor receives the ControlExecutionEvent, while a configured Splunk destination receives DefenseClaw's native structured verdict event. If the requirement is for the same agent_control.control_execution span to go to both Monitor and Splunk simultaneously, we should add/configure an explicit composite fan-out sink rather than claim this PR already provides that dual delivery.

@namrataghadi-galileo namrataghadi-galileo deleted the branch agentcontrol:feature/68338-add-defense-claw-evaluator July 13, 2026 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants