[Azure Functions] Add opt-in Unicode JSON serialization

azure-functions-durable/CHANGELOG.md

@@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased
 
+ADDED
+
+- Added `DFApp.configure_serialization(ensure_ascii=False)` to opt in to compact
+Unicode JSON across durable clients, orchestration and entity workers, activity
+bindings, and registered blueprints. The default remains ASCII-escaped JSON.
+The opt-in requires an `azure-functions` SDK exposing the shared codec's
+`ensure_ascii` option; unsupported SDKs fail explicitly at configuration time.
+Coordinate the dependency minimum in a release after that upstream API ships.
+
 ## v2.0.0rc3
 
 ADDED

azure-functions-durable/README.md

@@ -32,6 +32,58 @@ Key capabilities include durable orchestrations and sub-orchestrations, durable
 timers, external events, durable entities, retries, versioning, durable HTTP
 calls (`context.call_http(...)`), recurring scheduled tasks, and history export.
 
+## Unicode JSON serialization
+
+By default, durable payloads preserve the shared `azure-functions` codec's
+ASCII-escaped JSON representation. With a compatible shared SDK, opt in to
+literal Unicode JSON on the root `DFApp` at startup:
+
+```python
+import azure.durable_functions as df
+
+app = df.DFApp()
+app.configure_serialization(ensure_ascii=False)
+```
+
+This requires the shared `df_dumps(..., ensure_ascii=...)` API tracked in
+[Azure/azure-functions-python-library#372](https://github.com/Azure/azure-functions-python-library/pull/372).
+An older SDK raises `RuntimeError` when the opt-in is configured, rather than
+silently ignoring it. Leaving the option unset, or explicitly choosing
+`ensure_ascii=True`, remains compatible with older shared SDKs.
+
+There is one encoding policy per Python worker process. It applies to both
+synchronous and asynchronous durable clients, orchestration and entity workers,
+activity binding outputs, and all registered blueprints, including handlers
+created before configuration. Configure every scaled-out worker consistently
+before invocations. Repeating the same setting is allowed; a conflicting setting
+or changing the policy after the first durable JSON serialization raises
+`ValueError`.
+
+Literal Unicode reduces escaping overhead for non-Latin text. For example,
+a JSON object containing 100,000 Cyrillic characters occupies approximately
+200,012 UTF-8 bytes rather than 600,012 ASCII-escaped bytes. Actual reduction
+depends on the payload. The shared codec preserves surrogate-code-unit and
+ASCII DEL escapes, custom-object envelopes, and strict/loose typing semantics.
+This option does not normalize Unicode or change the decoder; existing escaped
+payloads remain readable.
+
+> [!WARNING]
+> Enabling or disabling this option changes serialized bytes for non-ASCII
+> text. Review applications deriving hashes, signatures, IDs, or replay
+> decisions from serialized JSON before rolling out or rolling back a change.
+> Do not change the setting inside orchestrator code or during invocations.
+
+The policy applies before large-payload externalization, so uploads use the
+selected JSON encoding and size thresholds use its serialized UTF-8 bytes.
+Reduced escaping does not eliminate aggregate request limits or replace
+externalization. Protocol envelopes, reference tokens, management responses, and
+JSON serialized explicitly by application code are not reformatted by this
+setting.
+
+Release coordination must set the shared-SDK dependency minimum after the
+upstream API is released. This feature does not guess an unreleased SDK version
+or change package versions.
+
 ## Large payloads
 
 Configure a `durabletask.payload.PayloadStore` once at app startup to store large

azure-functions-durable/azure/durable_functions/decorators/durable_app.py

@@ -493,6 +493,23 @@ class DFApp(Blueprint, FunctionRegister):
     Exports the decorators required to declare and index DF Function-types.
     """
 
+    def configure_serialization(self, *, ensure_ascii: bool = True) -> None:
+        """Configure JSON encoding for this app and its registered blueprints.
+
+        The default preserves ASCII-escaped JSON. Set ``ensure_ascii=False``
+        to emit ordinary Unicode literally with a compatible ``azure-functions``
+        SDK. An older SDK raises RuntimeError when this option is requested.
+
+        Call at startup in every worker process, before durable payloads are
+        serialized. Clients, orchestrations, entities, and activity bindings
+        share one policy per process, including handlers created earlier.
+        Repeating the same setting is allowed. A conflicting setting, or
+        changing the policy after serialization starts, raises ValueError.
+        """
+        from ..internal.serialization import configure_serialization
+
+        configure_serialization(ensure_ascii=ensure_ascii)
+
     def configure_large_payloads(self, *, payload_store: PayloadStore) -> None:
         """Enable payload externalization for this app and its blueprints.
 

azure-functions-durable/azure/durable_functions/internal/converters.py

@@ -32,7 +32,6 @@
 from azure.functions._durable_functions import (
     EntityContext,
     OrchestrationContext,
-    df_dumps,
     df_loads,
 )
 
@@ -43,6 +42,7 @@
     ORCHESTRATION_TRIGGER,
 )
 from .payloads import ActivityPayload, get_transport_payload_store
+from .serialization import serialize_json
 
 _TriggerMetadata = Optional[Mapping[str, meta.Datum]]
 
@@ -162,7 +162,7 @@ def encode(cls, obj: Any, *,
         if isinstance(obj, ActivityPayload):
             return meta.Datum(type='json', value=obj.value)
         try:
-            result = df_dumps(obj)
+            result = serialize_json(obj)
         except TypeError as e:
             raise ValueError(
                 f'activity trigger output must be json serializable ({obj})') from e

azure-functions-durable/azure/durable_functions/internal/serialization.py

@@ -10,13 +10,72 @@
 """
 
 from __future__ import annotations
+import inspect
+from threading import Lock
 from azure.functions._durable_functions import df_dumps, df_loads
 
-from typing import Any, override
+from typing import Any, Callable, Protocol, TypeGuard, override
 
 from durabletask.serialization import JsonDataConverter
 
 
+class _ConfigurableJsonEncoder(Protocol):
+    def __call__(self, value: Any, *, ensure_ascii: bool = True) -> str:
+        ...
+
+
+_serialization_lock = Lock()
+_ensure_ascii: bool | None = None
+_unicode_encoder: _ConfigurableJsonEncoder | None = None
+
+
+def _supports_ensure_ascii(
+        encoder: Callable[[Any], str]) -> TypeGuard[_ConfigurableJsonEncoder]:
+    parameter = inspect.signature(encoder).parameters.get("ensure_ascii")
+    return parameter is not None and parameter.kind in (
+        inspect.Parameter.KEYWORD_ONLY,
+        inspect.Parameter.POSITIONAL_OR_KEYWORD,
+    )
+
+
+def configure_serialization(*, ensure_ascii: object = True) -> None:
+    """Select one JSON encoding policy before durable payload serialization."""
+    global _ensure_ascii, _unicode_encoder
+    if not isinstance(ensure_ascii, bool):
+        raise TypeError("ensure_ascii must be a bool")
+    with _serialization_lock:
+        if _ensure_ascii is not None and _ensure_ascii != ensure_ascii:
+            raise ValueError(
+                "JSON serialization is already configured with "
+                f"ensure_ascii={_ensure_ascii} in this process; configure "
+                "serialization once before serializing durable payloads")
+        if not ensure_ascii:
+            encoder = df_dumps
+            if not _supports_ensure_ascii(encoder):
+                raise RuntimeError(
+                    "ensure_ascii=False requires an azure-functions SDK whose "
+                    "df_dumps supports the ensure_ascii keyword argument. "
+                    "Install a compatible azure-functions SDK before opting in.")
+            _unicode_encoder = encoder
+        _ensure_ascii = ensure_ascii
+
+
+def serialize_json(value: Any) -> str:
+    """Apply the process-wide policy using the shared Functions JSON codec."""
+    global _ensure_ascii
+    with _serialization_lock:
+        if _ensure_ascii is None:
+            _ensure_ascii = True
+        ensure_ascii = _ensure_ascii
+        encoder = _unicode_encoder
+    if ensure_ascii:
+        # Older shared SDKs do not accept the optional keyword argument.
+        return df_dumps(value)
+    if encoder is None:
+        raise RuntimeError("Unicode JSON serialization has no configured encoder")
+    return encoder(value, ensure_ascii=False)
+
+
 class FunctionsDataConverter(JsonDataConverter):
     """:class:`DataConverter` that serializes via azure-functions' codec.
 
@@ -38,7 +97,7 @@ class FunctionsDataConverter(JsonDataConverter):
     def serialize(self, value: Any) -> str | None:
         if value is None:
             return None
-        return df_dumps(value)
+        return serialize_json(value)
 
     @override
     def deserialize(self, data: str | None, target_type: type | None = None) -> Any: