Skip to content

feat(prompt): Report limits plainly with the closest useful answer - #2417

Closed
sentry-junior[bot] wants to merge 1 commit into
mainfrom
feat/prompt-limit-reporting
Closed

sentry-junior[bot] wants to merge 1 commit into
mainfrom
feat/prompt-limit-reporting

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Adds one line to the conversation-mode <failure-reporting> block (block 6 in policies/system-prompt.md):

When a policy or a missing capability stops part of the request, say so plainly, then give the closest useful answer you can.

Why: getsentry/junior-prod#576 makes Sentry's SOUL.md voice only. Since #2371, Personality comes after all core rules, so behavior rules left in SOUL can win conflicts with core rules. I checked each behavior line in SOUL against the core blocks. All of them already have a core home except this one, so this PR moves it into core instead of dropping it. Every install gets it now, not only Sentry's.

SOUL line Core home
lead with the answer <output>: "Start with the answer or result"
explicit about confidence, separate facts from inference, never pretend certainty <evidence>: "Do not present an inferred fact as verified", plus the uncertain-answer rule
critical of source quality and recency <evidence>: verification source order, plus "empty, stale, ambiguous" retry
don't claim abilities you don't have <safety>: "Stay within … the runtime's available capabilities", plus <evidence> claims about access
execute, verify, summarize <task-execution> and <evidence>
shortest complete answer, one compact response <output> length target
policy or missing capability: say so, give the closest useful answer none, added here

Automation and child-agent runs keep their own failure paths (finishAutomationRun reason, reportBlocked), so the line is only in conversation mode.

Checks: prettier and @sentry/junior typecheck pass. This is a prompt change, so the behavioral and integration evals should run on it.

Refs getsentry/junior-prod#576

via David Cramer.

--

View Session [Sentry]

Move the last core-behavior line from the consumer SOUL.md into the
<failure-reporting> block, so every install gets it and the Personality
block can be voice only (getsentry/junior-prod#576).

Co-Authored-By: David Cramer <david@sentry.io>
@vercel

vercel Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
junior-docs Ready Ready Preview Oct 30, 2026 4:22am UTC

Request Review

@sentry-junior

sentry-junior Bot commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

Closing without merging. We dropped the line from SOUL.md in getsentry/junior-prod#578 instead of moving it to core. Core already covers the cases that matter: the evidence and failure-reporting rules (say what was checked and what blocked a stronger answer), the Slack posting-limitation rule, and task-execution (deliver the work and state the open question). No eval shows a gap without it, so per policies/system-prompt.md we shouldn't add a rule without evidence. If an eval later shows the bot stonewalling on a missing capability, we can revisit with that evidence.

@sentry-junior sentry-junior Bot closed this Oct 30, 2026
@sentry-evals

sentry-evals Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Sentry Evals

junior-behavioral

Run Status Passed Failed Errored Cost Tokens Duration
Head Error 76 5 1 $0.91 0 1,402.42s

Base commit 345da0f has no eval run to compare against.

junior-integration

Run Status Passed Failed Errored Cost Tokens Duration
Head Passed 102 0 0 $0.69 0 1,235.74s

Base commit 345da0f has no eval run to compare against.

This branch was successfully deployed

1 active deployment
Preview – junior-docs — 55ddca6e Deployed Oct 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant