[Azure Functions] Add opt-in Unicode JSON serialization (#292)

* feat(functions): add opt-in Unicode JSON serialization

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 54e13dcd-4b0d-4c0b-ab5b-bbb958fd5792

* chore(functions): adopt released Unicode codec API

Require azure-functions 2.4.0b1 and exercise its released opt-in codec directly. Align documentation and samples with the available shared-library API.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 54e13dcd-4b0d-4c0b-ab5b-bbb958fd5792

---------

Co-authored-by: Bernd Verst <beverst@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 54e13dcd-4b0d-4c0b-ab5b-bbb958fd5792

Author: 3 people

.github/workflows/durabletask-azurefunctions.yml

@@ -57,7 +57,7 @@ jobs:
     needs: run-tests
     runs-on: ubuntu-latest
     # The end-to-end suite launches a real Azure Functions host, which requires
-    # ``azure-functions>=2.3.0b2``. That build is published to PyPI and pulled
+    # ``azure-functions>=2.4.0b1``. That build is published to PyPI and pulled
     # in as a declared dependency of ``azure-functions-durable``.
     # Run the suite in both payload-typing modes: the default (loose) codec and
     # strict typing (``AZURE_FUNCTIONS_DURABLE_STRICT_TYPING=1``), which requires

azure-functions-durable/CHANGELOG.md

@@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased
 
+ADDED
+
+- Added `DFApp.configure_serialization(ensure_ascii=False)` to opt in to compact
+Unicode JSON across durable clients, orchestration and entity workers, activity
+bindings, and registered blueprints. The default remains ASCII-escaped JSON.
+The opt-in uses the shared codec's `ensure_ascii` option available in
+`azure-functions` v2.4.0b1; unsupported SDKs fail explicitly at configuration time.
+
+CHANGED
+
+- Updated the minimum `azure-functions` dependency to v2.4.0b1.
+
 ## v2.0.0rc3
 
 ADDED

azure-functions-durable/MIGRATION_GUIDE.md

@@ -66,7 +66,7 @@ later.
 Replace the 1.x package constraint in `requirements.txt`:
 
 ```text
-azure-functions>=2.3.0b2
+azure-functions>=2.4.0b1
 azure-functions-durable>=2.0.0b1,<3
 ```
 

azure-functions-durable/README.md

@@ -32,6 +32,56 @@ Key capabilities include durable orchestrations and sub-orchestrations, durable
 timers, external events, durable entities, retries, versioning, durable HTTP
 calls (`context.call_http(...)`), recurring scheduled tasks, and history export.
 
+## Unicode JSON serialization
+
+By default, durable payloads preserve the shared `azure-functions` codec's
+ASCII-escaped JSON representation. With `azure-functions>=2.4.0b1`, opt in to
+literal Unicode JSON on the root `DFApp` at startup:
+
+```python
+import azure.durable_functions as df
+
+app = df.DFApp()
+app.configure_serialization(ensure_ascii=False)
+```
+
+The shared `df_dumps(..., ensure_ascii=...)` API was released in
+[`azure-functions` 2.4.0b1](https://github.com/Azure/azure-functions-python-library/releases/tag/2.4.0b1),
+which is the minimum required shared SDK version. Dependency installation
+selects a compatible release automatically. If an older SDK is manually
+installed despite that requirement, configuring the opt-in raises `RuntimeError`
+rather than silently ignoring it. Leaving the option unset, or explicitly
+choosing `ensure_ascii=True`, preserves the existing JSON encoding.
+
+There is one encoding policy per Python worker process. It applies to both
+synchronous and asynchronous durable clients, orchestration and entity workers,
+activity binding outputs, and all registered blueprints, including handlers
+created before configuration. Configure every scaled-out worker consistently
+before invocations. Repeating the same setting is allowed; a conflicting setting
+or changing the policy after the first durable JSON serialization raises
+`ValueError`.
+
+Literal Unicode reduces escaping overhead for non-Latin text. For example,
+a JSON object containing 100,000 Cyrillic characters occupies approximately
+200,012 UTF-8 bytes rather than 600,012 ASCII-escaped bytes. Actual reduction
+depends on the payload. The shared codec preserves surrogate-code-unit and
+ASCII DEL escapes, custom-object envelopes, and strict/loose typing semantics.
+This option does not normalize Unicode or change the decoder; existing escaped
+payloads remain readable.
+
+> [!WARNING]
+> Enabling or disabling this option changes serialized bytes for non-ASCII
+> text. Review applications deriving hashes, signatures, IDs, or replay
+> decisions from serialized JSON before rolling out or rolling back a change.
+> Do not change the setting inside orchestrator code or during invocations.
+
+The policy applies before large-payload externalization, so uploads use the
+selected JSON encoding and size thresholds use its serialized UTF-8 bytes.
+Reduced escaping does not eliminate aggregate request limits or replace
+externalization. Protocol envelopes, reference tokens, management responses, and
+JSON serialized explicitly by application code are not reformatted by this
+setting.
+
 ## Large payloads
 
 Configure a `durabletask.payload.PayloadStore` once at app startup to store large

azure-functions-durable/azure/durable_functions/decorators/durable_app.py

@@ -493,6 +493,23 @@ class DFApp(Blueprint, FunctionRegister):
     Exports the decorators required to declare and index DF Function-types.
     """
 
+    def configure_serialization(self, *, ensure_ascii: bool = True) -> None:
+        """Configure JSON encoding for this app and its registered blueprints.
+
+        The default preserves ASCII-escaped JSON. Set ``ensure_ascii=False``
+        to emit ordinary Unicode literally with ``azure-functions>=2.4.0b1``.
+        An older SDK raises RuntimeError when this option is requested.
+
+        Call at startup in every worker process, before durable payloads are
+        serialized. Clients, orchestrations, entities, and activity bindings
+        share one policy per process, including handlers created earlier.
+        Repeating the same setting is allowed. A conflicting setting, or
+        changing the policy after serialization starts, raises ValueError.
+        """
+        from ..internal.serialization import configure_serialization
+
+        configure_serialization(ensure_ascii=ensure_ascii)
+
     def configure_large_payloads(self, *, payload_store: PayloadStore) -> None:
         """Enable payload externalization for this app and its blueprints.
 

azure-functions-durable/azure/durable_functions/internal/converters.py

@@ -32,7 +32,6 @@
 from azure.functions._durable_functions import (
     EntityContext,
     OrchestrationContext,
-    df_dumps,
     df_loads,
 )
 
@@ -43,6 +42,7 @@
     ORCHESTRATION_TRIGGER,
 )
 from .payloads import ActivityPayload, get_transport_payload_store
+from .serialization import serialize_json
 
 _TriggerMetadata = Optional[Mapping[str, meta.Datum]]
 
@@ -162,7 +162,7 @@ def encode(cls, obj: Any, *,
         if isinstance(obj, ActivityPayload):
             return meta.Datum(type='json', value=obj.value)
         try:
-            result = df_dumps(obj)
+            result = serialize_json(obj)
         except TypeError as e:
             raise ValueError(
                 f'activity trigger output must be json serializable ({obj})') from e

azure-functions-durable/azure/durable_functions/internal/serialization.py

@@ -10,13 +10,72 @@
 """
 
 from __future__ import annotations
+import inspect
+from threading import Lock
 from azure.functions._durable_functions import df_dumps, df_loads
 
-from typing import Any, override
+from typing import Any, Callable, Protocol, TypeGuard, override
 
 from durabletask.serialization import JsonDataConverter
 
 
+class _ConfigurableJsonEncoder(Protocol):
+    def __call__(self, value: Any, *, ensure_ascii: bool = True) -> str:
+        ...
+
+
+_serialization_lock = Lock()
+_ensure_ascii: bool | None = None
+_unicode_encoder: _ConfigurableJsonEncoder | None = None
+
+
+def _supports_ensure_ascii(
+        encoder: Callable[[Any], str]) -> TypeGuard[_ConfigurableJsonEncoder]:
+    parameter = inspect.signature(encoder).parameters.get("ensure_ascii")
+    return parameter is not None and parameter.kind in (
+        inspect.Parameter.KEYWORD_ONLY,
+        inspect.Parameter.POSITIONAL_OR_KEYWORD,
+    )
+
+
+def configure_serialization(*, ensure_ascii: object = True) -> None:
+    """Select one JSON encoding policy before durable payload serialization."""
+    global _ensure_ascii, _unicode_encoder
+    if not isinstance(ensure_ascii, bool):
+        raise TypeError("ensure_ascii must be a bool")
+    with _serialization_lock:
+        if _ensure_ascii is not None and _ensure_ascii != ensure_ascii:
+            raise ValueError(
+                "JSON serialization is already configured with "
+                f"ensure_ascii={_ensure_ascii} in this process; configure "
+                "serialization once before serializing durable payloads")
+        if not ensure_ascii:
+            encoder = df_dumps
+            if not _supports_ensure_ascii(encoder):
+                raise RuntimeError(
+                    "ensure_ascii=False requires azure-functions>=2.4.0b1, whose "
+                    "df_dumps supports the ensure_ascii keyword argument. "
+                    "Install a compatible azure-functions SDK before opting in.")
+            _unicode_encoder = encoder
+        _ensure_ascii = ensure_ascii
+
+
+def serialize_json(value: Any) -> str:
+    """Apply the process-wide policy using the shared Functions JSON codec."""
+    global _ensure_ascii
+    with _serialization_lock:
+        if _ensure_ascii is None:
+            _ensure_ascii = True
+        ensure_ascii = _ensure_ascii
+        encoder = _unicode_encoder
+    if ensure_ascii:
+        # Older shared SDKs do not accept the optional keyword argument.
+        return df_dumps(value)
+    if encoder is None:
+        raise RuntimeError("Unicode JSON serialization has no configured encoder")
+    return encoder(value, ensure_ascii=False)
+
+
 class FunctionsDataConverter(JsonDataConverter):
     """:class:`DataConverter` that serializes via azure-functions' codec.
 
@@ -38,7 +97,7 @@ class FunctionsDataConverter(JsonDataConverter):
     def serialize(self, value: Any) -> str | None:
         if value is None:
             return None
-        return df_dumps(value)
+        return serialize_json(value)
 
     @override
     def deserialize(self, data: str | None, target_type: type | None = None) -> Any:

azure-functions-durable/pyproject.toml

@@ -32,7 +32,7 @@ readme = "README.md"
 dependencies = [
     "durabletask[opentelemetry]>=1.12.0",
     "azure-identity>=1.19.0",
-    "azure-functions>=2.3.0b2"
+    "azure-functions>=2.4.0b1"
 ]
 
 [project.urls]

azure-functions-durable/samples/durable-entities/requirements.txt

@@ -1,2 +1,2 @@
-azure-functions>=2.3.0b2
+azure-functions>=2.4.0b1
 azure-functions-durable>=2.0.0b1,<3

azure-functions-durable/samples/fan-out-fan-in/requirements.txt

@@ -1,2 +1,2 @@
-azure-functions>=2.3.0b2
+azure-functions>=2.4.0b1
 azure-functions-durable>=2.0.0b1,<3