Skip to content

feat: add versioned task registration and dispatch - #368

Merged
wangbill (YunchuWang) merged 2 commits into
mainfrom
yunchuwang-versioned-task-dispatch
Sep 22, 2026
Merged

wangbill (YunchuWang) merged 2 commits into
mainfrom
yunchuwang-versioned-task-dispatch

Conversation

@YunchuWang

@YunchuWang wangbill (YunchuWang) commented Sep 21, 2026 •

Copy link
Copy Markdown
Member

Summary

What changed?

Latest follow-up: 27c167b preserves explicitly empty activity/child version fields on the protobuf wire and the empty child default. Only three production expressions changed; concrete automatic filters remain unchanged. Both the former failing case and the full real Azure DTS feature matrix now pass.

One worker can now host multiple implementations of the same orchestrator or activity name, selected by the recorded/request version rather than by name alone.

worker.addNamedOrchestrator("Order", orderV1, "v1");
worker.addNamedOrchestrator("Order", orderV2, "v2");
worker.addNamedActivity("Price", priceV1, "v1");
worker.addNamedActivity("Price", priceV2, "v2");
// Inferred names also work: addOrchestrator(fn, version?), addActivity(fn, version?).
await client.scheduleNewOrchestration("Order", input, { version: "v2" });

The optional final registration argument is also available on TestOrchestrationWorker and DurableTaskAzureManagedWorkerBuilder.

  • Registry, executors, worker: store distinct name/version registrations; replay dispatch uses ExecutionStarted.version; activity dispatch uses ActivityRequest.version. Exact matches win. An unversioned registration accepts other request versions only when that name has no versioned registrations, following .NET's fallback rule. Missing registrations retain the JS error types, include the requested version, and are non-retriable.
  • Context, options, test backend/client: unspecified child versions use the worker's versioning.defaultVersion, not the parent's version. Unspecified activities use the orchestration instance's version, not the worker default. Explicit versions override defaults, including "" to select unversioned. In-memory scheduling, history, activity work items, and child starts now preserve versions. Retries retain the selected version; continueAsNew behavior is unchanged.
  • Filters and Azure-managed builder: transfer every registration without collapsing same-name versions. Auto filters group by logical name, use concrete registered versions for versioned names, and retain wildcard behavior for unversioned-only names. Strict filters use the configured worker version, including empty. Worker acceptance runs before either exact or fallback dispatch for both orchestration and activity requests.
  • Functions integration, tests, documentation: exercise inherited embedded-worker dispatch/defaults through classic wrappers; document the host's unique-function-name boundary. Update the three package changelogs and READMEs with compatibility/migration guidance.

No new dependencies, protobuf changes, routing framework, or configuration flags.

Why is this change needed?

Previously, versions could travel on the wire and constrain worker selection, but registrations collided by name, activities ignored their request version, and children ignored the worker default. Worker filtering alone cannot select between same-name implementations inside one process. This change connects registration, dispatch, and scheduling end to end.

Immutable .NET reference

Behavior was checked against microsoft/durabletask-dotnet at 92474e9e35c66d64de36cabd0a17652376d37873:

Compatibility decisions

Versions are case-insensitive opaque strings, not semantic registration identities ("1" and "1.0" remain distinct). Missing/null/empty registration versions are unversioned; whitespace-only registration versions are rejected, while whitespace-only incoming wire versions resolve as unversioned. Nonempty versions are not trimmed.

Existing JS task names remain case-sensitive, unlike .NET. Native function names are unchanged. Existing unversioned call sites stay type-compatible and retain the reference fallback until the name opts into versioned registrations. Entities are unchanged.

Acceptance and local identity are independent. Omitted matchStrategy means None; Strict with an empty worker version no longer silently disables acceptance checks. Activity mismatches use Reject/abandon or an explicit non-retriable Fail response. The pinned .NET activity processor has an unconditional early return on the mismatch branch, even for Fail; this PR deliberately does not reproduce that dropped-response bug and instead honors the documented Fail contract and the existing JS orchestration completion pattern.

Issues / work items

  • Implements the requested same-process versioned registration and dispatch parity.
  • No linked issue; no unrelated PR scope included.

Project checklist

  • Release notes are not required for the next release
    • Otherwise: Notes added to CHANGELOG.md and the affected package changelogs
  • Backport is not required
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • All required tests have been added/updated (unit tests, E2E tests)
  • Breaking change?
    • If yes:
      • Impact: previously ignored activity versions and child defaults now affect scheduling/dispatch; activity acceptance is enforced; adding a versioned registration removes that name's catch-all fallback; registration misses are non-retriable.
      • Migration guidance: keep all implementations needed by in-flight versions; use explicit version: "" to retain unversioned calls; drain affected instances before changing replay-sensitive defaults or mixing worker versions. See the README versioning section.

AI-assisted code disclosure (required)

Was an AI tool used? (select one)

  • No
  • Yes, AI helped write parts of this PR (e.g., GitHub Copilot)
  • Yes, an AI agent generated most of this PR

If AI was used:

  • Tool(s): GitHub Copilot.
  • AI-assisted areas/files: implementation, tests, documentation, and PR preparation across the file groups above.
  • What you changed after AI output: inspected actual JS and immutable .NET source; corrected test fixture/type issues; narrowed child-retry verification to executor replay after discovering a pre-existing in-memory retry-ID limitation. No human review or approval is claimed.

AI verification (required if AI was used):

  • I understand the code and can explain it
  • I verified referenced APIs/types exist and are correct
  • I reviewed edge cases/failure paths (timeouts, retries, cancellation, exceptions)
  • I reviewed concurrency/async behavior
  • I checked for unintended breaking or behavior changes

Testing

Automated tests

  • Passed at 27c167b: 107 suites, 1,968 tests across core, Azure-managed unit tests, and Functions provider unit tests, including real loopback gRPC fixtures.
  • Command: npx --no-install jest --runInBand --silent --testPathPattern 'packages.(durabletask-js.test|durabletask-js-azuremanaged.test.unit|azure-functions-durable.test.unit).'
  • New regressions were first observed failing before implementation (name collisions, wrong-version execution, missing defaults, ignored activity acceptance, and retryable registration misses).
  • Covers same-name versions, unversioned fallback boundaries, null/empty/unknown versions, duplicate rejection, case/name identity, replay, auto filters, both mismatch strategies, builder transfer, classic wrappers, and activity/child retry version retention.
  • Passed: npm run build:core, npm run build:azuremanaged, npm run build -w durable-functions; scoped ESLint on all changed TypeScript; scoped Prettier checks; git diff --check. Normal Husky/lint-staged hooks also passed.

Manual validation (only if runtime/behavior changed)

  • Environment: Windows, Node.js 24.14.0, npm 11.9.0.
  • Runtime validation was automated: a real local grpc-js server sent versioned work items and captured worker completions/abandons; the in-memory backend exercised workflows without a sidecar; embedded Functions requests were serialized protobuf/base64.
  • Follow-up real Azure DTS validation on 2026-09-22: the initial empty-version failure at 25fa0c4 was fixed in 27c167b by preserving explicit empty protobuf wrappers and the unversioned child default. The original filtered reproduction and the no-default control now pass; all six named scenarios in the real production Azure matrix passed (one sequential Jest test). This covers same-name dispatch/defaults, actual filters/fallback, non-retriable unknown versions, Strict Fail, acknowledged Reject/recovery, and activity/child retries. Exact-head evidence, initial/intermediate failures, test isolation, and cleanup status. No live Functions host deployment is claimed.

Notes for reviewers

  • app.orchestration / app.activity still require unique Functions host names. Embedded DurableFunctionsWorker version dispatch is not a new host registration capability.
  • The in-memory worker models versioned dispatch and child defaults, not backend filtering or queue Reject semantics.
  • The existing in-memory backend rejects a retried child's reused instance ID. That unrelated instance-reuse limitation is unchanged. Child retry retention is verified through real executor replay for both policy/timer and handler paths; activity retries are additionally exercised end to end in-memory.
  • Built directly on fetched upstream main at ecddf16e5eda45b47cf03074919f922892790770.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e0af01a5-0dfa-4e71-a660-c4186e65d7e0
Copilot AI lite review requested due to automatic review settings September 21, 2026 19:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The child rewind path loses its recorded version, and activity mismatch messages use orchestration wording.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 Medium severity

Open (1)
What changed in this PR

Adds version-aware task registration and dispatch across workers, scheduling, replay, retries, filters, and integrations.

Changes:

  • Supports multiple same-name task implementations by version.
  • Preserves version inheritance and fallback behavior.
  • Adds tests, documentation, and changelog updates.
  • Two findings remain: child rewind version preservation and activity mismatch wording.
File Summary
README.md Documents versioning and migration behavior.
packages/​durabletask-js/​test/​work-item-filters.spec.ts Updates filter coverage.
packages/​durabletask-js/​test/​versioned-in-memory.spec.ts Tests in-memory versioning.
packages/​durabletask-js/​test/​versioned-dispatch.spec.ts Tests versioned dispatch.
packages/​durabletask-js/​test/​versioned-dispatch-grpc.spec.ts Tests gRPC dispatch and acceptance.
packages/​durabletask-js/​test/​orchestration-id-reuse-policy.spec.ts Updates versioned scheduling options.
packages/​durabletask-js/​src/​worker/​work-item-filters.ts Implements version-aware filters.
packages/​durabletask-js/​src/​worker/​versioning-options.ts Updates versioning options.
packages/​durabletask-js/​src/​worker/​task-hub-grpc-worker.ts Registers, dispatches, and validates versions.
packages/​durabletask-js/​src/​worker/​runtime-orchestration-context.ts Applies activity and child defaults.
packages/​durabletask-js/​src/​worker/​registry.ts Stores and resolves versioned registrations.
packages/​durabletask-js/​src/​worker/​orchestration-executor.ts Dispatches orchestrations by version.
packages/​durabletask-js/​src/​worker/​exception/​orchestrator-not-registered-error.ts Reports missing orchestrator versions.
packages/​durabletask-js/​src/​worker/​exception/​activity-not-registered-error.ts Reports missing activity versions.
packages/​durabletask-js/​src/​worker/​activity-executor.ts Dispatches activities by version.
packages/​durabletask-js/​src/​utils/​pb-helper.util.ts Marks registration failures non-retriable.
packages/​durabletask-js/​src/​testing/​test-worker.ts Adds versioned test-worker support.
packages/​durabletask-js/​src/​testing/​test-client.ts Preserves scheduled versions.
packages/​durabletask-js/​src/​testing/​in-memory-backend.ts Preserves versions in in-memory execution.
packages/​durabletask-js/​src/​task/​options/​task-options.ts Documents version defaults.
packages/​durabletask-js-azuremanaged/​test/​unit/​worker-builder.spec.ts Tests builder version transfer.
packages/​durabletask-js-azuremanaged/​src/​worker-builder.ts Transfers versioned registrations.
packages/​durabletask-js-azuremanaged/​README.md Documents managed-worker versioning.
packages/​durabletask-js-azuremanaged/​CHANGELOG.md Records managed-worker changes.
packages/​azure-functions-durable/​test/​unit/​worker.spec.ts Tests Functions integration behavior.
packages/​azure-functions-durable/​README.md Documents Functions boundaries.
packages/​azure-functions-durable/​CHANGELOG.md Records Functions changes.
CHANGELOG.md Records core versioning changes.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread packages/durabletask-js/src/testing/in-memory-backend.ts
Comment thread packages/durabletask-js/src/worker/task-hub-grpc-worker.ts
@YunchuWang

wangbill (YunchuWang) commented Sep 22, 2026 •

Copy link
Copy Markdown
Member Author

Resolved in 27c167b: the minimal wire-presence fix and unchanged-filter control now pass on real Azure DTS, along with all six named feature scenarios. Latest evidence and cleanup. The original failed-run evidence below is retained for history.

Real Azure DTS validation: FAILED — not ready for review

Tested unchanged head 25fa0c400c1a0f1196b7fc14a0fa3e71b108b522 on 2026-09-22 against the public production Azure DTS service (West US 2, Consumption), using a newly isolated task hub in the requested Private Test Sub WANGBILL subscription. Authentication was Azure CLI over TLS; the authenticated Hello/GetInstance preflight passed. No emulator or fabricated service work items were used.

Reproduced blocker

With workItemFilters: "auto", a child name registered as unversioned, 1.0.0, and 2.0.0 produces the actual wire filter:

{ "versions": ["", "1.0.0", "2.0.0"] }

A versioned parent explicitly scheduling that child with version: "" remains Running. In a fresh controlled reproduction the child persisted as Pending, with no history or work-item delivery during the 20-second observation window.

This was not merely an idle worker or a connectivity timeout:

Control Actual production-service observation
Same filtered worker, same child name, version 1.0.0 Delivered and Completed with the expected implementation/output.
Same filtered worker, same child name, version 2.0.0 Delivered and Completed with the expected implementation/output.
Same simple parent/child logic with work-item filters disabled The explicit-unversioned child was delivered with version ""; child and parent Completed correctly.

The failed filtered and successful unfiltered cases both used the published SDK's unchanged child serialization (empty version wrapper omitted). This establishes a production compatibility/routing boundary around the mixed-version filter, not its internal root cause. It does not prove that the wrapper omission alone is the cause, nor that a backend fix rather than SDK handling is necessarily required. No workaround was used to declare a pass.

What did work before the blocker

Both same-name parent versions were genuinely delivered/replayed in one worker. Each received correct persisted activity results for inherited parent version, explicit 2.0.0, and explicit-unversioned activity. Default child version 2.0.0 and explicit child version 1.0.0 were delivered and completed correctly. The combined workflow then stalled at the explicit-unversioned child, so these observations are not a full-scenario pass.

The six-scenario suite stopped at its first failing scenario; remaining fallback/unknown-version, Strict Fail/Reject recovery, and retry scenarios were not run. The fresh filtered reproduction independently confirmed the blocker after an unfiltered positive control.

Additional service constraint

The initial fixture used symbolic v1/v2 values. Azure rejected StartInstance with INVALID_ARGUMENT: versions must be integer Major[.Minor[.Patch]]. Fixtures were corrected to 1.0.0/2.0.0; that initial input rejection is retained separately and is not the dispatch failure above. The SDK's opaque registration strings and the service's accepted version syntax must not be conflated; symbolic examples in this PR are not directly runnable against this Azure endpoint.

Readiness and cleanup

The requested pass condition is not met; ready for review has not been applied. Production source/commit are unchanged. All test workers/clients stopped, and the test-only overlay and controlled reproduction scripts were archived outside the checkout, then removed from it. Most owned instances were purged/read back absent; two original stuck roots required deletion of the exclusive test hub. The task-owned temporary role, task hub (including both stuck roots), scheduler, and resource group have all been deleted. Azure group exists confirmed the resource group absent at 2026-09-22T21:21:38Z. No customer resources were used or deleted.

Local evidence: pr368-azure-final-summary.json, both Jest result/log pairs, persisted-history/actual-delivery JSON, and pr368-azure-fresh-auto-control.{cjs,json,log} in the validation session artifacts. No authentication tokens were recorded.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e0af01a5-0dfa-4e71-a660-c4186e65d7e0
@YunchuWang

wangbill (YunchuWang) commented Sep 22, 2026 •

Copy link
Copy Markdown
Member Author

Fixed and revalidated on real Azure DTS — PASS

The empty-version failure reported above is fixed in 27c167bcd1c764162eeaf9c0fefaea27290ec965 (tree f63844122225d4420d0767817a272c786ffbb769). The follow-up changes only three production expressions, plus focused regression tests/documentation:

  • Preserve an explicitly empty protobuf version wrapper for activity and child scheduling instead of dropping it with a truthiness check.
  • Resolve an unspecified child version with no worker default to the concrete empty/unversioned version.
  • Automatic filters, registration lookup and retry policies are unchanged. No wildcard workaround or backend modification was used.

Real production-service verification

Ran against a fresh, isolated Azure DTS Consumption task hub in Private Test Sub WANGBILL, West US 2, with Azure CLI authentication over TLS. Production code remained byte-identical to the committed head. The test-only overlay was archived and removed afterward.

The original controlled reproduction now passes: with the same concrete child filter ["", "1.0.0", "2.0.0"], the outgoing child action has a present version wrapper containing ""; Azure delivers the empty-version child and both parent and child persist Completed. A second control with neither an explicit child version nor a worker default also passes.

All six named scenarios passed in the real-Azure matrix (one sequential Jest test, 31.332 seconds total):

Scenario Verified outcome
Same-process versioned dispatch and defaults Same-name 1.0.0/2.0.0 orchestrators and activities run correctly; activity inherits the parent version, child inherits worker default, explicit numeric/empty overrides reach the expected implementation.
Automatic filters and legacy wildcard Actual subscription versions match registrations; unknown version stays held while positive delivery barriers complete; unversioned-only fallback works.
Unknown registration Unknown versions fail non-retriably rather than invoking the mixed name's unversioned implementation; exact-version positive control succeeds.
Strict Fail Incompatible orchestration/activity versions report failure without executing the rejected implementation.
Strict Reject and recovery Azure acknowledges activity abandonment; a compatible worker receives the same execution/task/version and completes it.
Version retention across retries Both policy and handler retries preserve selected activity and child versions, with two observed attempts for each and no wrong-version implementation calls.

Observed 70 real work-item deliveries, 15 activity responses and one acknowledged abandonment/recovery. All 21 created instance IDs plus one absence probe were read back absent after cleanup. Both focused controls also completed and were purged. No emulator, fabricated work-item responses or service fault injection was used.

One intermediate run passed the first two scenarios but exposed test isolation between an earlier filtered worker and a later unfiltered worker using the same names. Independent scenarios now use distinct names; their assertions were retained, including the unfiltered case's own positive exact-version control. That run and the original failures remain archived; this is not a skipped assertion or production workaround.

Additional validation and limits

  • 107 suites / 1,968 tests passed, with core, Azure-managed and Functions provider builds, scoped lint/format and normal commit hooks passing.
  • This verifies production DTS, not a live Functions host deployment. Version values used with DTS are numeric (1.0.0, 2.0.0), as required by the service; README now calls out that constraint.
  • All workers/clients stopped and the checkout is clean. All task-owned Azure resources have been deleted: temporary role, task hub, scheduler and resource group. Azure confirmed the resource group absent at 2026-09-22T21:51:19Z; no customer resources were touched.

Evidence includes exact-head metadata, actual delivery and persisted history JSON, Jest output, cleanup reads and archived reproduction scripts. The latest full-run overlay SHA-256 is DCD06FDB824EC00614EA4A786FAFFFE3DCE48B2558B7F9D7E09434DCA93F1522.

@YunchuWang
wangbill (YunchuWang) merged commit deb60cc into main Sep 22, 2026
31 checks passed
@YunchuWang
wangbill (YunchuWang) deleted the yunchuwang-versioned-task-dispatch branch September 22, 2026 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants