Repository navigation
[Bug]: DeepSeek PR review fails after three reasoning-only responses #429
Description
Summary
The Codex PR Review workflow is routed to the DeepSeek review script for the current provider configuration. On PR #428, the configured deepseek-v4-flash model returned an empty final message.content on all three attempts, while returning substantial reasoning_content. The script failed before posting any GitHub review.
This is a review-model/output integration failure, not a code-review rejection or a failed product test. The deeper cause (provider behavior, thinking configuration, or output-budget exhaustion) is not yet proven by the retained logs.
Evidence
- Affected PR: fix(desktop): persist runs and recover question workflows #428
- Reviewed head:
abf810e9e7f7bcfd3e1e1d915c817e81e144101d - Failed workflow: https://github.com/OpenCoworkAI/open-codesign/actions/runs/35456727130
- Failed job / step: pr-review / Run DeepSeek for PR Review
- Time: 2026-09-19, approximately 17:00:52–17:02:36 UTC
- Configured model:
deepseek-v4-flash; effort:high - Response model field:
deepseek-flash
Sanitized failure shape:
DeepSeek output invalid on attempt 1/3: DeepSeek API returned no message content
DeepSeek output invalid on attempt 2/3: DeepSeek API returned no message content
Error: DeepSeek API returned no message content
Process completed with exit code 1
The response previews show message.content: "" with non-empty reasoning_content. The workflow did not run the Codex action fallback and did not submit any review. Request credentials/endpoints are intentionally omitted here.
Current implementation and suspected contributors
- Review entrypoint does not override the output budget.
- Shared request helper defaults to
max_tokens: 8192, requests JSON output, and only explicitly enables thinking fordeepseek-reasoner/deepseek-v4-pro*. Fordeepseek-v4-flashit omits the thinking setting; the observed response nevertheless contains reasoning. - Retry handling adds a JSON-format reminder but keeps the same budget/model/thinking policy.
- Empty-content diagnostics truncate the raw response before the useful trailing
finish_reason/ usage fields, so the logs do not establish whether the output budget was exhausted.
A large review input (27 changed files, 3938 additions in this case) plus reasoning consuming the output budget is a plausible hypothesis, not a confirmed root cause.
Expected behavior
A valid, non-empty JSON review body is posted, or the workflow clearly reports that automated review was unavailable. Model/provider failures must not be presented as code findings, and reasoning text must not be substituted for a completed review or approval.
Suggested follow-up / acceptance criteria
- Log safe structured metadata on empty output: requested/returned model, finish reason, usage, and content/reasoning lengths; do not dump credentials, prompts, or raw reasoning.
- Verify the provider's current thinking/output-budget semantics for
deepseek-v4-flashand configure them explicitly where supported. - Add a bounded, cost-aware recovery policy for reasoning-only or length-exhausted output instead of repeating the same ineffective request three times (appropriate output budget, smaller review chunks, or an explicitly configured fallback).
- Add mocked API regression tests for reasoning-only output,
finish_reason=length, valid final JSON, malformed JSON, and exhausted retries; assert no blank/fabricated review is posted. - Reproduce with a similarly sized PR and verify successful review submission or an actionable infrastructure-failure summary.
Merge disposition
Per the maintainer's explicit request, #428 was squash-merged after verifying that CI (lint/typecheck/tests/build), CodeQL, and Dependency Review all passed, with no submitted blocking review. The only failing check was this automated review execution. The merge used an exact head-SHA guard and did not use an admin override or change repository protections.
Merge commit: c25a5765c6ce7eec6c7e184643613bfcd3bfa403.
Activity
github-actions commentedon Sep 19, 2026
Thanks for the detailed report — the failure shape, the retained evidence, and the separation between "automated review unavailable" and "code-review rejection" are all clear and useful. This is a defect in the repo's own CI review tooling under
.github/scripts/, not in the shipped app, so it doesn't affect the design pipeline; still worth fixing so the review gate produces an honest result instead of an opaque red check.What I can confirm from the public checkout
.github/scripts/deepseek-pr-review.mjs:3— the PR review entrypoint importscallDeepSeekJsonWithRetriesfrom the shared helper, so the request shape, JSON-mode handling, and retry policy all live in one place..github/scripts/deepseek-issue-response.mjs:3and:19— the issue responder uses the same helper and the same "you are running through DeepSeek chat completions" framing. Any fix to the shared helper covers both workflows, which is the right place to land it..github/scripts/deepseek-common.mjs:205-206— the helper normalizes base URLs by stripping/v1/responses//responsessuffixes, i.e. it targets chat-completions-style endpoints. That is consistent with the request shape you describe.The specific line ranges you cite (
deepseek-pr-review.mjsL277–L284,deepseek-common.mjsL224–L298 and L341–L375) are not verifiable from the material available to me, so I'm treating the "no output-budget override" and "thinking enabled only fordeepseek-reasoner/deepseek-v4-pro*" claims as your reading of the current head rather than as confirmed by me. They're plausible and match the observed behavior, but the exact offsets may have moved.On the mechanism
The error string in your log (
DeepSeek API returned no message content) shows the script already has an explicit non-empty-content guard, and that guard is why nothing was posted. That part is working as intended — the problem is the recovery policy, not the validation.Two things in your evidence stand out as worth pinning down before calling a root cause:
deepseek-v4-flashyet the response still carries non-emptyreasoning_content. If reasoning tokens are counted againstmax_tokenson this endpoint, then a 27-file / 3938-addition diff plus reasoning can plausibly consume the entire budget and leavecontentempty. That's a hypothesis your logs can't currently confirm or refute — which is exactly why your first checklist item (logfinish_reason,usage, and content/reasoning lengths without dumping the prompt) is the right next step.finish_reason: "length"would settle it immediately.deepseek-v4-flashbut the response reportsdeepseek-flash. That may be a harmless alias/normalization on the provider side, or it may indicate the request is being routed to a different model family than the one whose thinking semantics you're assuming. Worth confirming what the provider echoes back versus what you send.Your acceptance criteria read correctly to me: bounded recovery (raise the budget or chunk the diff) instead of three identical retries, mocked regression tests for reasoning-only output /
finish_reason=length/ valid JSON / malformed JSON / exhausted retries, and an assertion that no blank or fabricated review is ever posted. The last one matters most — an automatic review that silently substitutes reasoning text for findings would be worse than this failure.Need more info (if not already retained)
finish_reason,usage(completion / reasoning token counts if exposed), and the returnedmodelstring.max_tokensfor this model.no message contentsignature, link it and I'll fold it in rather than track it twice.Open-CoDesign Bot
fix(ci): recover incomplete DeepSeek review output (#431)
Sun-sunshine06 commentedon Oct 13, 2026
Closing as completed after verifying live behavior with the repair from #431 in the trusted base.
Acceptance is for the verified current deepseek-flash/high configuration. The original incident used deepseek-v4-flash/high; its retained logs lack finish reason/usage, so output-budget exhaustion remains an unproven sole cause of that historical incident. The later length-exhaustion case and successful recovery are directly verified.
These results establish automation availability and actual review submission, not code approval: the posted reviews are COMMENTED. Broader context coverage, batching, and live validation of the shared issue-response caller are outside this closure claim. No new provider calls or blind reruns were needed for this verification.
github-actions commentedon Oct 13, 2026
Thanks for the detailed closure and for separating what the live runs actually verified from what remains unproven. That distinction is the useful part of this report — the original
deepseek-v4-flashincident still has no retainedfinish_reason/usage, so output-budget exhaustion is a plausible but unestablished cause for that run, while the laterfinish_reason=lengthcase plus successful recovery after the budget increase is directly evidenced. Recording both without collapsing them into a single root cause is the right call.I can't independently re-verify the linked workflow runs, job logs, or posted reviews from the public checkout, so I'm taking those as your verification rather than restating them as confirmed here. Two points I can check against the repository itself:
package.json—"test:scripts": "node --test .github/scripts/deepseek-common.test.mjs". Any regression suite for reasoning-only output,finish_reason=length, valid/malformed JSON, and exhausted retries belongs in that file so it runs on the normalpnpm testpath.Your framing of the outcome is consistent with how automated review should be treated: successful step execution and an actual submitted review establish that the automation is available, not that the code is approved — the reviews are
COMMENTED, and that is not a substitute for human or blocking review.The items you explicitly left outside the closure — broader context coverage, diff batching, and live validation of the shared issue-response caller — are worth keeping visible as follow-ups rather than folding into a completed bug. If you open dedicated issues for those, I can triage them on their own terms rather than reopening this one.
Open-CoDesign Bot