Skip to content
nenedesignPublic

About

Open agent quality layer: capture agent runs, render a plain-English HTML report.

Topics

Resources

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

17 Commits

Folders and files

Repository files navigation

AgentView: Observability & Traceability for Non-Technical Teams

See what your AI agent actually did, and why it failed, in one picture.

Agent traceability and observability for teams that need to show their work, not just ship it.

AgentView ships two complementary layers:

  1. SDK layer: wrap your agent code with four decorators, get a self-contained HTML report.
  2. MCP proxy layer: wrap an MCP server with zero agent code changes, get a local browser dashboard showing every tool call in real time.

Both layers write the same JSONL trace format. Both produce output a non-technical stakeholder can understand in about thirty seconds.


MCP proxy dashboard

The proxy layer sits between Claude Desktop and any MCP server. It intercepts every JSON-RPC call in both directions and shows the full session activity in a browser at localhost:7346. Every tool call, every empty result, every server error appears as a glass-card session timeline with a plain-English status pill: Completed, Returned no usable data, Failed, or In progress.

The wedge against Langfuse, Phoenix, and MLflow: those platforms require SDK instrumentation. This requires nothing. Install the proxy, restart Claude Desktop, open the dashboard. The agent stays unchanged.

Quickstart: proxy dashboard

pip install agentview

# Wrap an MCP server transparently
agentview proxy -- npx -y @modelcontextprotocol/server-filesystem /tmp

# Or inject the proxy into Claude Desktop's config automatically
agentview configure filesystem

# Start the local dashboard (prints a URL with session token to stderr)
agentview serve

The proxy writes a JSONL trace for every session. The dashboard reads those traces and groups activity by a 2-second inter-call window. The session token is cryptographically random and required on all API endpoints; the URL printed at startup is the only way to access the dashboard.

To restore Claude Desktop's config to its original state:

agentview restore filesystem

SDK report

agentview is a Python SDK that wraps your agent code, records what happened, and turns the recording into a single self-contained HTML page. The page shows every step your agent took as a card on a vertical timeline, in the shape of a delivery-tracking view. Green worked. Yellow returned nothing. Red failed or made an unsupported claim. A non-technical viewer can look at the report and understand what went wrong in about thirty seconds, without asking anyone.

The library ships with three demo runs. Two of them fail on purpose, in different ways, so the failure story is visible on first open.

Quickstart: SDK report

pip install agentview
agentview demo
# Open any of the three files printed under agentview_demo/reports/

To wrap your own agent, use four decorators:

from agentview import (
    observe_agent, observe_tool, observe_model, observe_validation,
    current_run, ValidationResult,
)

@observe_tool(name="search_catalog", label="Search product catalog")
def search_catalog(query: str) -> list[dict]:
    return db.search(query)

@observe_model(name="respond_to_user", label="Model wrote the reply")
def respond(query: str, results: list[dict]) -> str:
    return llm.complete(f"Summarize {results} for {query}")

@observe_validation(name="check_answer", label="Check answer against search results")
def check_answer(reply: str, results: list[dict]) -> ValidationResult:
    if not results and "found" in reply.lower():
        return ValidationResult(
            status="INVALID",
            category="unsupported_claim",
            message="The answer claimed matches but the search returned nothing.",
        )
    return ValidationResult(status="VALID", message="The answer matched the search results.")

@observe_agent(name="product_search", label="Product search agent")
def run_agent(query: str) -> str:
    current_run().set_metadata({"query": query})
    results = search_catalog(query)
    reply = respond(query, results)
    check_answer(reply, results)
    return reply

run_agent("wireless headphones")

Every run writes a JSONL trace file. Render it:

agentview report traces/run_abc123.jsonl -o report.html
open report.html

What the report looks like

A single HTML file, around 18 KB. At the top, a boarding-pass summary card with a big status glyph, a one-sentence plain-English outcome, and four facts (outcome, steps, duration, where it failed). Below the summary, a vertical timeline. Each step is a card with a past-tense action-verb title ("Searched the product catalog"), a one-sentence outcome, and the elapsed time. Failed steps get a red-bordered callout with a "Why this failed" block that quotes the validator in plain English. Validators appear as nested substeps under the step they judged, in a PR-review-thread style. At the bottom, a one-paragraph recap of the whole run. There is also a print stylesheet for anyone who prints things.

The file is offline, self-contained, and safe to open in any browser. It never loads anything from the internet.

Run agentview demo and open one of the three reports it writes. That is faster than any screenshot.


What you are looking at (for a non-technical reader)

The timeline shows every step the agent took, top to bottom, in the order it happened. The dot next to each step is color-coded.

  • Green worked as expected.
  • Yellow returned nothing or partial results.
  • Red failed, or made a claim that a validator marked as unsupported.
  • Grey was not judged here.

A red step opens a "Why this failed" block below the step card. The block quotes the validator in plain English and names what kind of failure it was (unsupported claim, mismatched reference, empty tool result). Validators that judged a step appear as substeps nested under that step. A collapsed "developer detail" block under each step holds the raw span attributes for anyone who wants them.


What it captures

  • Which tools ran, in what order, with what shape of input and output (types and lengths, not values, by default).
  • Which model calls happened, and how long each took.
  • Which validators fired, what they judged, and which step they judged.
  • The final status of the run, rolled up from every step.
  • MCP proxy: every JSON-RPC message in both directions, grouped by session and inter-call window.

What it does not capture (by default)

  • Prompt text and model responses. Off by default. Opt in per-run if you need it.
  • Secrets. Redacted before storage.
  • Anything network-loaded in the report. The HTML file is fully offline.
  • MCP traffic is written to a local trace file only. AgentView does not transmit anything externally.

These defaults are deliberate. A report you can hand to a stakeholder is worth more than a report you cannot share because it leaks data.


How it differs from Langfuse, Phoenix, MLflow, LangSmith

Those platforms are trace databases. They store spans well and show them in dashboards for developers. They do not produce a portable, layperson-legible HTML file, they treat validation as an afterthought, and they require SDK instrumentation in the agent code.

AgentView is a small library with an opinionated product decision: the primary artefact is something a non-technical stakeholder can understand. For the SDK layer, that is a single HTML page. For the proxy layer, that is a local browser dashboard showing activity as it happens.

agentview Trace platforms
Primary artefact Self-contained HTML report or local dashboard Server-hosted dashboard
Layperson-first UX Yes, timeline cards with plain-English sentences No, developer table view
MCP proxy (zero code changes) Yes No
SDK instrumentation Optional, four decorators Required
Portable One file, opens anywhere, offline Requires platform access
Validation as a first-class span kind Yes No, treated as attribute or eval score
Server required No (dashboard is local-only) Usually yes
Cost Zero Usually usage-based or seat-based

AgentView is not a replacement for those platforms. The JSONL trace format is stable and you can emit to OpenTelemetry, so you can use AgentView locally and export to a platform separately.


Security

The local dashboard binds to 127.0.0.1 only by default. Every session gets a cryptographically random token; the startup URL (printed to stderr) is the only way to access the API. No data leaves the machine. See docs/security.md for the full threat model.


Design principles

  1. Capture once, interpret many ways. The JSONL trace file is the source of truth. The HTML report is one interpretation. The browser dashboard is another. Evaluators, CI gates, and platform adapters can all read the same file.
  2. Layperson-first UX. Every design decision is judged against whether a non-technical viewer can understand the output without help. A screenshot has to be legible to someone who has never used the library.
  3. Claim-versus-evidence validation. A validator is a first-class span kind. It records what claim was made, what evidence was actually available, and what it judged.
  4. Privacy by default. Prompts and responses off, secrets scrubbed. Users opt in to content capture.
  5. Portable single file. The HTML report is self-contained. No network. No server. No dependencies at open time.

Roadmap

v0.2.0 (this release) ships the SDK report loop and the MCP proxy dashboard. Future work:

  • v0.3: Evaluate. Rule-based and LLM-as-judge evaluators that annotate spans with pass or fail verdicts.
  • v0.4: Enforce. CI quality gate as a reusable GitHub Action. Baseline comparison across runs. Platform adapters via OpenTelemetry.
  • v0.5: Scale. Multi-agent orchestration semantics, live-tail UI, token and cost attribution.

What is inside

SDK layer:

  • src/agentview/decorators.py: the public API. @observe_agent, @observe_tool, @observe_model, @observe_validation.
  • src/agentview/events.py: the JSONL event contract, as Pydantic v2 models. Any external tool can validate a trace file without importing the SDK.
  • src/agentview/report/: the report builder and renderer.
  • examples/product_search/: the runnable demo agent.

MCP proxy layer:

  • src/agentview/proxy/forwarder.py: transparent stdio proxy. Byte-for-byte forwarding, signal propagation, zero stdout contamination.
  • src/agentview/proxy/interceptor.py: stateful capture hook. Writes JSONL events for every JSON-RPC message.
  • src/agentview/proxy/config_safety.py: 10-step guided config flow for claude_desktop_config.json. SHA-256 conflict detection, atomic write, backup, restore.
  • src/agentview/dashboard/: local FastAPI dashboard with session reconstruction, activity grouping, and security middleware.
  • docs/security.md: threat model, CSP policy, browser extension note.

Tests:

  • 137 tests across proxy integrity (A1), JSONL capture (A3), config safety (A2), and local security (A4).

Requirements

  • Python 3.10 or newer.
  • Pydantic v2.
  • FastAPI + uvicorn (dashboard only; installed automatically).

Nothing else at runtime. No external server. No network calls from the dashboard.


License

Apache-2.0. See LICENSE.

About

Open agent quality layer: capture agent runs, render a plain-English HTML report.

Topics

Resources

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages