Skip to content

fix: preserve nested task failure details - #371

Merged
wangbill (YunchuWang) merged 2 commits into
mainfrom
yunchuwang-preserve-inner-failure
Sep 24, 2026
Merged

wangbill (YunchuWang) merged 2 commits into
mainfrom
yunchuwang-preserve-inner-failure

Conversation

@YunchuWang

@YunchuWang wangbill (YunchuWang) commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

Summary

What changed?

  • Preserve the received innerFailure chain in public task failure details, activity/sub-orchestration errors, retry handlers and predicates, client get/wait/query state, and history.

  • Add a read-only optional TaskFailureDetails.innerFailure, a matching FailureDetails getter and optional fourth constructor argument. Existing constructor arguments and the task message / entity errorMessage APIs are unchanged.

  • Use one internal protobuf-to-public converter. Retain original sub-orchestration failure protobufs in the in-memory backend and forward the chain through durable-functions/testing.

  • Preserve received details beneath the existing TaskFailedError wrapper when an uncaught or rethrown task failure is serialized. This narrow writer change is necessary for activity failures to reach terminal client state intact; it does not reconstruct JavaScript Error.cause objects.

  • Follow-up 6f0aeae: guard received-detail forwarding with an object-identity Set. A user-mutated cycle preserves the task wrapper and each unique failure, then ends with an explicit CircularFailureDetails marker; finite chains are not depth-truncated.

Why is this change needed?

Issues / work items

  • N/A; this is an independent failure-chain parity improvement based on main, not stacked on another open PR.

Project checklist

  • Release notes are not required for the next release
    • Otherwise: Notes added to CHANGELOG.md
  • Backport is not required
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • All required tests have been added/updated (unit tests, E2E tests)
  • Breaking change?
    • If yes:
      • Impact: Additive optional fields; existing retry decisions and error strings remain unchanged.
      • Migration guidance: None required.

AI-assisted code disclosure (required)

Was an AI tool used? (select one)

  • No
  • Yes, AI helped write parts of this PR (e.g., GitHub Copilot)
  • Yes, an AI agent generated most of this PR

If AI was used:

  • Tool(s): GitHub Copilot.
  • AI-assisted areas/files: Failure details and their conversions, directly affected testing adapters, focused regression tests, README and changelogs.
  • What you changed after AI output: Agent-led implementation and verification; human review remains pending.

AI verification (required if AI was used):

  • I understand the code and can explain it
  • I verified referenced APIs/types exist and are correct
  • I reviewed edge cases/failure paths (timeouts, retries, cancellation, exceptions)
  • I reviewed concurrency/async behavior
  • I checked for unintended breaking or behavior changes

Testing

Automated tests

  • Latest cycle follow-up (6f0aeae): 5 suites / 78 tests passed, core build, scoped ESLint, normal hooks and patch checks passed. Self-cycle, three-node cycle and caught/mutated/rethrown failure regressions first failed safely before the fix; a bounded test getter prevents unbounded allocation during RED. Distinct equal-valued objects, existing >10-level finite chains and explicit JavaScript cause precedence remain covered.
  • Result: Passed. Recorded runtime RED failures before the implementation, including in-memory uncaught activity propagation, actual loopback gRPC, and the Functions testing projection.
  • Initial implementation (fd684cf) core focused run: 17 suites, 382 tests. Covers failure details, both retry mechanisms/task kinds, executor, state, client waits/query, history, in-memory serialization, entity compatibility and the existing writer depth boundary.
  • Functions testing adapter: 1 suite, 11 tests.
  • npm run build:core and the affected Functions TypeScript build passed.
  • ESLint passed for all changed TypeScript files, as did git diff --check.
  • Prettier passed for new files; baseline-aware checks found no new formatting differences in changed files. Existing CRLF/baseline formatting was not mass-reformatted.

Manual validation (only if runtime/behavior changed)

  • Environment (OS, Node.js version, components): Windows, Node.js v24.14.0; in-memory backend and an actual loopback grpc-js server/worker/client fixture.
  • Steps + observed results:
    1. Automated in-memory tests exercise a three-level activity/sub-orchestration cause chain through retry inspection, catch/rethrow, history, and terminal client state; a separate uncaught activity test covers the ordinary propagation path.
    2. The local gRPC fixture serializes an actual worker activity failure and reads it through client get, both waits, query, and history.
    3. No manual/cloud validation was performed. Azure resources were not created; Azure/real-sidecar E2E and the full repository test suite were not run.
  • Evidence (optional): The focused tests are included in this PR; hosted CI is pending.

Notes for reviewers

  • Missing inner details remain undefined; empty strings and optional stack traces are preserved. Reading or forwarding received wire details does not add a depth cap. Forwarding detects repeated object identities and appends { errorType: "CircularFailureDetails", message: "A circular innerFailure reference was detected." } as a terminal marker, rather than hanging or silently dropping the cause. This guards mutable JavaScript details; it is not a new retry rule.
  • The ordinary Error.cause writer still has its existing ten-inner-level bound. Explicitly assigned JavaScript causes retain precedence.
  • Nested details do not automatically change retry decisions: existing root error-type fallback and top-level non-retriable checks are unchanged. User handlers may inspect the new chain.
  • The entity-specific failure type retains its existing errorMessage shape. No protocol/generated-file changes, new dependencies, configuration, resources, or unrelated parity features are included.

Expose innerFailure throughout task errors, retry inspection, client state,
history and in-memory testing. Keep received details below TaskFailedError
when an uncaught or rethrown failure is serialized.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e0af01a5-0dfa-4e71-a660-c4186e65d7e0
Copilot AI lite review requested due to automatic review settings September 23, 2026 20:06
Comment thread packages/durabletask-js/src/utils/pb-helper.util.ts Fixed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The Functions testing adapter must recursively project inner failures into documented plain-object fields.

Review effort: Lite
Findings: None

What changed in this PR

This PR preserves nested task failure details across public APIs, retries, history, clients, gRPC, and testing adapters.

Changes:

  • Adds recursive innerFailure support and protobuf conversion.
  • Preserves failure chains through serialization and propagation.
  • Adds focused tests, documentation, and changelog updates.
File Summary
README.md Documents nested failure inspection.
packages/​durabletask-js/​test/​retry-handler-task.spec.ts Tests nested retry inspection.
packages/​durabletask-js/​test/​failure-details.spec.ts Tests failure-chain propagation.
packages/​durabletask-js/​test/​failure-details-grpc.spec.ts Tests gRPC propagation.
packages/​durabletask-js/​src/​utils/​pb-helper.util.ts Preserves received failure chains.
packages/​durabletask-js/​src/​utils/​history-event-converter.ts Converts nested history failures.
packages/​durabletask-js/​src/​utils/​failure-details.util.ts Centralizes recursive conversion.
packages/​durabletask-js/​src/​testing/​test-client.ts Preserves in-memory client chains.
packages/​durabletask-js/​src/​testing/​in-memory-backend.ts Forwards sub-orchestration details.
packages/​durabletask-js/​src/​task/​retryable-task.ts Exposes nested details to retry policies.
packages/​durabletask-js/​src/​task/​retry/​retry-context.ts Documents nested retry failures.
packages/​durabletask-js/​src/​task/​retry-handler-task.ts Exposes nested details to retry handlers.
packages/​durabletask-js/​src/​task/​failure-details.ts Adds recursive public failure details.
packages/​durabletask-js/​src/​task/​exception/​task-failed-error.ts Preserves converted nested details.
packages/​durabletask-js/​src/​orchestration/​index.ts Preserves orchestration failure chains.
packages/​durabletask-js/​src/​client/​client.ts Preserves client failure chains.
packages/​azure-functions-durable/​test/​unit/​testing.spec.ts Tests nested Functions testing failures.
packages/​azure-functions-durable/​src/​testing/​index.ts Forwards nested testing failures.
packages/​azure-functions-durable/​CHANGELOG.md Documents the adapter change.
CHANGELOG.md Documents the core change.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Track received failure objects by identity and append a CircularFailureDetails
marker on a repeated reference. Preserve the task wrapper and unique prefix
without truncating finite chains or changing ordinary Error.cause handling.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e0af01a5-0dfa-4e71-a660-c4186e65d7e0
@YunchuWang
wangbill (YunchuWang) merged commit f09e987 into main Sep 24, 2026
31 checks passed
@YunchuWang
wangbill (YunchuWang) deleted the yunchuwang-preserve-inner-failure branch September 24, 2026 23:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants