Skip to content

feat: [1/6] add core payload contracts and bounded processing - #388

Open
wangbill (YunchuWang) wants to merge 1 commit into
mainfrom
yunchuwang-payload-phase-1-core
Open

wangbill (YunchuWang) wants to merge 1 commit into
mainfrom
yunchuwang-payload-phase-1-core

Conversation

@YunchuWang

@YunchuWang wangbill (YunchuWang) commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Summary

What changed?

  • Add provider-independent PayloadStore/options/errors and explicit protobuf field mapping, retaining structural ES2020-compatible cause options.
  • Enforce inclusive UTF-8 thresholds and size limits, cap per-message I/O at eight, drain started operations and report the lowest message-order failure.
  • Cover permanent activity/orchestrator upload failure conversion, all history/entity oneofs and 150,000-element entity batches.
  • Reserve core 0.4.1 and synchronize only its lock entry and the Functions exact pin. No transport, Azure Blob dependencies, cleanup APIs or future-phase exports are included.

Why is this change needed?

  • Phase 1/6 of the requested large-payload implementation split. This PR's base is upstream main.
  • Behavioral reference: durabletask-dotnet 91739687. The complete stack preserves the original implementation at c9b53fcb3ab8585c4b3b22f9c0500cc9dd122c5a, with the documented instance-kind routing correction.

Issues / work items

  • Resolves: N/A (user-requested phased feature review).
  • Related: original oversized PR #387. It is preserved while the coordinator verifies this replacement stack.

Project checklist

  • Release notes are not required for the next release
    • Otherwise: Notes added to CHANGELOG.md
  • Backport is not required
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • All required tests have been added/updated (unit tests, E2E tests)
  • Breaking change?
    • If yes:
      • Impact: No intentional API break in this phase; the payload feature remains opt-in.
      • Migration guidance: Do not release partial phases. Merge the stack in order and publish the completed core before dependent packages.

AI-assisted code disclosure (required)

Was an AI tool used? (select one)

  • No
  • Yes, AI helped write parts of this PR (e.g., GitHub Copilot)
  • Yes, an AI agent generated most of this PR

If AI was used:

  • Tool(s): GitHub Copilot.
  • AI-assisted areas/files: This phase's source, regression tests, and directly related metadata/documentation.
  • What you changed after AI output: Split the implementation along independently buildable boundaries; retained focused regression evidence and removed unrelated formatting-only changes. Human review is still required.

AI verification (required if AI was used):

  • I understand the code and can explain it
  • I verified referenced APIs/types exist and are correct
  • I reviewed edge cases/failure paths (timeouts, retries, cancellation, exceptions)
  • I reviewed concurrency/async behavior
  • I checked for unintended breaking or behavior changes

These human attestations are intentionally left unchecked.


Testing

Automated tests

  • Result: Passed locally for this phase, without later-phase files.
  • npm ci --offline --no-audit --no-fund — passed.
  • npm run build — passed all four existing workspaces on this snapshot.
  • npm run lint — passed.
  • npm run test:unit -w @microsoft/durabletask-js — passed 1,811 tests in 90 suites.
  • prettier --check on the changed TS/JSON files — passed.
  • git diff --check origin/main...HEAD — passed.
  • Hosted CI: pending/not inferred. Existing main-base filters mean some workflows will not run for stacked PRs; no workflow triggers were broadened.

Manual validation (only if runtime/behavior changed)

  • Environment (OS, Node.js version, components): Windows, Node.js v24.14.0; Jest and the owning SDK build/type-check.
  • Steps + observed results:
    1. Materialized this phase on its declared predecessor, with no future-phase imports.
    2. Ran the exact owning checks above and inspected the incremental patch.
    3. Kept the original feature branch/history and primary user checkout unchanged.
  • Evidence (optional): Full local logs are session artifacts, not repository changes.

Notes for reviewers

Merge order Phase PR Base branch
1 Core contracts #388 main
2 SDK integration #389 yunchuwang-payload-phase-1-core
3 Azure Blob provider #390 yunchuwang-payload-phase-2-sdk
4 Purge protocol/client #391 yunchuwang-payload-phase-3-blob
5 Purge worker #392 yunchuwang-payload-phase-4-client
6 Release/usage/consumers #393 yunchuwang-payload-phase-5-worker

Merge 1 → 2 → 3 → 4 → 5 → 6. Each PR targets its predecessor; retarget the next PR only after its prerequisite merges. Do not publish packages or describe a partial phase as the complete feature.

  • This phase is a tested foundation only. Public client/worker opt-in is phase 2; no Azure provider is imported.
  • Core 0.4.1 is reserved for the complete API set. No release tag or npm publication is performed.
  • Full live DTS/mixed-.NET execution has not been run locally (Docker daemon unavailable). The final phase records aggregate local Blob/Azurite, source/wire interoperability and packed-consumer evidence without claiming absent hosted checks.

Phase 1 of 6: add provider-independent payload contracts, explicit protobuf mapping, UTF-8 size limits, bounded I/O and permanent worker upload failure conversion. Reserve the 0.4.1 core API version without enabling transport or cleanup yet.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 81e20512-d3a2-4f8e-a574-f396140ec9c7
Copilot AI balanced review requested due to automatic review settings October 16, 2026 19:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Continue-as-new carryover payloads bypass externalization and configured size limits.

1 open finding
What changed in this PR

Adds the core foundation for optional external payload storage, ahead of transport and provider integration.

Changes:

  • Introduces storage contracts and explicit protobuf payload mapping.
  • Adds bounded processing, size checks, failure conversion, and regression tests.
  • Reserves core version 0.4.1 and synchronizes dependency metadata.
File Description
packages/​durabletask-js/​test/​payloads.spec.ts Tests mapping, limits, failures, and concurrency.
packages/​durabletask-js/​src/​payloads/​run-bounded.ts Bounds concurrency and drains started operations.
packages/​durabletask-js/​src/​payloads/​payload-store.ts Defines storage contracts, options, and errors.
packages/​durabletask-js/​src/​payloads/​payload-processor.ts Maps and processes protobuf payload fields.
packages/​durabletask-js/​src/​index.ts Exports payload contracts.
packages/​durabletask-js/​package.json Reserves core 0.4.1.
packages/​azure-functions-durable/​package.json Updates the exact core dependency.
package-lock.json Synchronizes version metadata.
CHANGELOG.md Documents the foundation and phased release.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

action.getSendentitymessage()?.getEntityoperationsignaled()?.getInput(),
action.getSendentitymessage()?.getEntityoperationcalled()?.getInput(),
);
for (const event of action.getRewindorchestration()?.getNewhistoryList() ?? []) {
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants