Skip to content

[Deep Agent 12/15] Specialised subagents + Langfuse prompt management (#48) - #61

Open
stavp-star wants to merge 2 commits into
deep-agent/40-guidancefrom
deep-agent/48-subagents-prompts
Open

stavp-star wants to merge 2 commits into
deep-agent/40-guidancefrom
deep-agent/48-subagents-prompts

Conversation

@stavp-star

Copy link
Copy Markdown
Collaborator

Closes #48 · Part of epic #49 · Stack position 12/15. Base: deep-agent/40-guidance (#60), auto-retargets on merge.

What

Subagents (deep_agent/subagents.py), delegated to with task:

Subagent Tools (allow-list) Returns
schema-scout search_metadata, get_database_tables, get_table_columns, get_table_profile, get_column_profile, get_table_relationships compact brief: tables, needed columns, join keys and cardinality, date-column types
value-resolver search_column_values, get_column_distinct_values, get_column_profile, search_business_terms, resolve_location_to_wkt, get_sample_rows one line per filter: exact value(s) / LIKE / WKT literal / date bounds
result-verifier execute_trino_query, get_sample_rows PASS/FAIL per semantic-validation check, with evidence

Each subagent gets its own prompt (prompts/<name>.md), the skills, a tool-call cap (DEEP_AGENT_SUBAGENT_MAX_TOOL_CALLS), the built-in tool filter, and DeepAgentContextMiddleware. That closes the #55 limitation: subagents now see the request's connection_id (tested) and today's date (tested). DEEP_AGENT_SUBAGENT_MODEL optionally runs subagents on a smaller or faster model. The general-purpose subagent stays available; the main prompt names the three specialists and says to skip delegation for simple questions.

Bug found and fixed: deepagents passes the parent's state into a subagent and returns the subagent's full state as an update. With the append reducer from #58, every task call would have duplicated the parent's executed_queries. They now merge with a de-duplicating reducer keyed by (query_id, executed_sql).

Prompt management (deep_agent/prompt_registry.py)

  • Langfuse text prompts deep-agent/system, deep-agent/schema-scout, deep-agent/value-resolver, deep-agent/result-verifier, fetched by label (DEEP_AGENT_PROMPT_LABEL, default production) with the SDK cache (DEEP_AGENT_PROMPT_CACHE_TTL_S, default 60 s).
  • Fallback: the packaged .md is used when Langfuse is unreachable, the prompt doesn't exist yet, or DEEP_AGENT_LANGFUSE_PROMPTS=false.
  • PromptRefreshMiddleware swaps the leading prompt part of the system message for the currently served version on every model call. Moving a label in Langfuse changes behaviour without a redeploy or agent rebuild (tested). The skills, harness, and memory sections that deepagents appends are preserved.
  • build_run_config puts the served prompt client in metadata["langfuse_prompt"], so Langfuse links generations to the prompt version (for langfuse_compare_prompt_versions), and records prompt_source / prompt_version.
  • scripts/sync_deep_agent_prompts.py --label staging [--dry-run] pushes the packaged prompts as new versions. Not run: it writes to your Langfuse project, so that's your call.

Acceptance criteria

  • schema-scout, value-resolver (and result-verifier) exist with explicit allow-lists. A test asserts the tools actually offered to the subagent's model.
  • Main agent peak context tokens −30% on multi-table questions with accuracy ≥ baseline. Pending a live benchmark (deep-agent-bench, [Deep Agent 10/15] Bound the agent loop and cut latency: limits, tool surface, connection reuse, caching (#42) #59).
  • Prompts fetched from Langfuse by label with local fallback; generations linked to the prompt version
  • [~] Label change takes effect without deploy: tested with a mocked Langfuse client. Staging demo pending: run the sync script first.
  • Summarisation / eviction thresholds tested on a 30-turn thread over the largest catalog. Pending a live catalog. deepagents' defaults (SummarizationMiddleware, large-tool-result eviction) are active and unchanged.

Testing

  • tests/deep_agent/test_subagents_prompts.py: 13 tests (full delegation loop, allow-lists, context and date propagation, reducer idempotency, prompt fallback/label/refresh, sync script)
  • Unit tests now always use the packaged prompts (autouse fixture) to stay offline
  • Full suite (excluding real-services e2e): 374 passed, 2 skipped

🤖 Generated with Claude Code

- subagents.py: schema-scout, value-resolver, result-verifier with
  explicit tool allow-lists, own prompts, skills, context/date middleware,
  tool-call cap and optional smaller model (DEEP_AGENT_SUBAGENT_MODEL)
- prompt_registry.py: deep-agent/<key> text prompts fetched from Langfuse
  by label (DEEP_AGENT_PROMPT_LABEL) with packaged .md fallback;
  PromptRefreshMiddleware serves label changes without a redeploy
- run config links generations to the served prompt version
  (langfuse_prompt metadata) and records prompt_source/prompt_version
- executed_queries reducer de-duplicates records (subagent state round
  trips re-sent the parent's history)
- scripts/sync_deep_agent_prompts.py pushes packaged prompts to Langfuse

Closes #48

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…'t refetched per model call

The Langfuse SDK caches fetched prompts but not misses; PromptRefreshMiddleware
calls load_prompt on every model call, so an absent prompt (404 -> fallback)
cost a network round trip on each call. Misses are now remembered for
DEEP_AGENT_PROMPT_CACHE_TTL_S.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 20, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 36ab8d28-a054-40e0-9f91-6454cdc869d3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@stavp-star

Copy link
Copy Markdown
Collaborator Author

Follow-up fix pushed to this branch (adcdce9): Langfuse prompt misses are now cached for DEEP_AGENT_PROMPT_CACHE_TTL_S. Found while testing #43. The SDK caches fetched prompts but not misses, and PromptRefreshMiddleware calls load_prompt on every model call, so until the prompts are pushed to Langfuse each model call paid a 404 round trip. 2 tests added. Merged forward into #62 and #63.

🤖 Generated with Claude Code

@benb-mafat benb-mafat left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i dont see any prompts in langfuse or actually fetched. also subagents may be wasteful and need to think if these are worth adding. maybe location sub agent is needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants