feat(otel): client-side redaction of PII and/or secrets

README.md

@@ -46,6 +46,7 @@ Mistral AI API: Our Chat Completion and Embeddings APIs specification. Create yo
   * [Resource Management](#resource-management)
   * [Debugging](#debugging)
   * [IDE Support](#ide-support)
+  * [Telemetry \& Observability](#telemetry--observability)
 * [Development](#development)
   * [Contributions](#contributions)
 
@@ -836,7 +837,7 @@ print(res.choices[0].message.content)
 operations. These operations will expose the stream as [Generator][generator] that
 can be consumed using a simple `for` loop. The loop will
 terminate when the server no longer has any events to send and closes the
-underlying connection.  
+underlying connection.
 
 The stream is also a [Context Manager][context-manager] and can be used with the `with` statement and will close the
 underlying connection when the context is exited.
@@ -1277,9 +1278,146 @@ Generally, the SDK will work well with most IDEs out of the box. However, when u
 
 <!-- Placeholder for Future Speakeasy SDK Sections -->
 
+
+## Telemetry & Observability
+
+The SDK can emit [OpenTelemetry](https://opentelemetry.io/) traces for the API calls it makes (chat, agents, embeddings, OCR, …), following the
+[GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/).
+Spans capture the operation, model, token usage, and — unless redacted — the input/output messages and tool calls. Telemetry is **opt-in** and lives in the `mistralai.extra.observability` module.
+
+### Installation
+
+Install the `telemetry` extra:
+
+```bash
+pip install "mistralai[telemetry]"
+# or: uv add "mistralai[telemetry]"
+```
+
+### Enabling telemetry
+
+Either set an environment variable before creating the client:
+
+```bash
+export MISTRAL_SDK_TELEMETRY=dedicated   # dedicated | global | false
+```
+
+or configure it in code:
+
+```python
+import os
+from mistralai.client import Mistral
+from mistralai.extra.observability import configure_telemetry
+
+with Mistral(api_key=os.environ["MISTRAL_API_KEY"]) as client:
+    # Dedicated mode (default): the SDK creates and owns an OTLP exporter that
+    # ships spans to the Mistral telemetry endpoint. Spans are redacted before
+    # export.
+    configure_telemetry(client)
+
+    client.chat.complete(
+        model="mistral-small-latest",
+        messages=[{"role": "user", "content": "Hello!"}],
+    )
+```
+
+### Provider modes
+
+`configure_telemetry(client, provider=...)` selects where spans go and who owns the export pipeline:
+
+| `provider` | Who owns the exporter | Where spans go | Redaction |
+| ---------- | --------------------- | -------------- | --------- |
+| `"dedicated"` (default) | The SDK | Mistral telemetry endpoint | Applied automatically |
+| `"global"` | Your application | Your global OpenTelemetry provider | **Not** applied — you need to wrap your own exporter |
+| a `TracerProvider` | Your application | The provider you pass | **Not** applied — you need to wrap your own exporter |
+
+In `global`/custom modes your application owns the pipeline, so the `redaction` argument is ignored (a warning is logged). Wrap your own exporter with `RedactingSpanExporter` to redact spans there:
+
+```python
+from opentelemetry import trace
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import BatchSpanProcessor
+from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
+from mistralai.extra.observability import RedactingSpanExporter, configure_telemetry
+
+provider = TracerProvider()
+provider.add_span_processor(
+    BatchSpanProcessor(RedactingSpanExporter(OTLPSpanExporter()))
+)
+trace.set_tracer_provider(provider)
+
+# SDK spans now flow through your global provider (already redacted above).
+configure_telemetry(client, provider="global")
+```
+
+### Redaction
+
+In dedicated mode, redaction is on by default. Control it with the `redaction` argument, which also accepts any of the reusable policies from `mistralai.extra.observability`:
+
+```python
+from mistralai.extra.observability import AttributeRedactionPolicy
+
+configure_telemetry(client)                                        # default policy (regex)
+configure_telemetry(client, redaction=AttributeRedactionPolicy())  # very conservative key-oriented policy
+configure_telemetry(client, redaction=False)                       # disabled - no redaction
+configure_telemetry(                                               # custom callback to control how attributes are redacted
+    client,
+    redaction=lambda key, value: None if "email" in key else value,
+)
+```
+
+| Policy | Strategy | Trade-off |
+| ------ | -------- | --------- |
+| `RegexRedactionPolicy` (default, `redaction=True`) | Content-oriented: keeps keys and structure, redacts matched substrings (secret tokens plus PII — emails, card-like sequences, IPv4). | Redacts most sensitive data while preserving observability value; may miss free-form PII or secrets not in the pattern set. |
+| `AttributeRedactionPolicy` | Key-oriented: redacts whole values for sensitive keys (explicit set, fragment match, or non-primitive value), then scans kept values for secret token patterns. | Very conservative, but erases most prompt/response content. |
+| `CallbackRedactionPolicy` (`redaction=<callable>`) | Your `(key, value) -> value \| None` masker per attribute; return `None` to drop the attribute. | Full control; you own the logic. |
+
+The built-in defaults are exported as constants, so you can extend them instead of replacing them wholesale:
+
+```python
+import re
+
+from mistralai.extra.observability import (
+    DEFAULT_PII_SECRET_PATTERNS,
+    DEFAULT_SENSITIVE_ATTRIBUTE_KEYS,
+    AttributeRedactionPolicy,
+    RegexRedactionPolicy,
+)
+
+# Content-oriented: add a custom secret pattern to the default set.
+configure_telemetry(
+    client,
+    redaction=RegexRedactionPolicy(
+        patterns=(*DEFAULT_PII_SECRET_PATTERNS, re.compile(r"\bacme-[a-z0-9]{16}\b")),
+    ),
+)
+
+# Key-oriented: mask an extra application attribute on top of the defaults.
+configure_telemetry(
+    client,
+    redaction=AttributeRedactionPolicy(
+        sensitive_keys=DEFAULT_SENSITIVE_ATTRIBUTE_KEYS | {"app.customer.email"},
+    ),
+)
+```
+
+*Note: the `RedactingSpanExporter` primitive is reusable by any OpenTelemetry application, independent of the Mistral client.*
+
+### Environment variables
+
+| Variable | Description | Default |
+| -------- | ----------- | ------- |
+| `MISTRAL_SDK_TELEMETRY` | Auto-enable telemetry: `dedicated`, `global`, or `false`. | unset (disabled) |
+| `MISTRAL_OTLP_TRACES_ENDPOINT` | Override the OTLP traces endpoint used in dedicated mode. | `https://api.mistral.ai/telemetry/v1/traces` |
+| `MISTRAL_SDK_DEBUG_TRACING` | Set to `true` for verbose tracing logs. | `false` |
+| `MISTRAL_API_KEY` | Used as the bearer token for the dedicated-mode exporter. | — |
+
+Runnable examples live in [`examples/mistral/observability`](/examples/mistral/observability).
+
+
 # Development
 
 ## Contributions
 
-While we value open-source contributions to this SDK, this library is generated programmatically. Any manual changes added to internal files will be overwritten on the next generation. 
-We look forward to hearing your feedback. Feel free to open a PR or an issue with a proof of concept and we'll do our best to include it in a future release. 
+While we value open-source contributions to this SDK, this library is generated programmatically. Any manual changes added to internal files will be overwritten on the next generation.
+We look forward to hearing your feedback. Feel free to open a PR or an issue with a proof of concept and we'll do our best to include it in a future release.

examples/mistral/observability/dedicated_telemetry.py

@@ -0,0 +1,31 @@
+#!/usr/bin/env python
+"""Dedicated telemetry mode.
+
+The SDK creates and owns an OTLP exporter that ships spans to the Mistral
+telemetry endpoint. Spans are redacted before export.
+
+Requires the telemetry extra: pip install "mistralai[telemetry]"
+"""
+
+import os
+
+from mistralai.client import Mistral
+from mistralai.extra.observability import configure_telemetry
+
+
+def main() -> None:
+    api_key = os.environ["MISTRAL_API_KEY"]
+
+    with Mistral(api_key=api_key) as client:
+        # Dedicated mode is the default; redaction is on by default.
+        configure_telemetry(client)
+
+        response = client.chat.complete(
+            model="mistral-small-latest",
+            messages=[{"role": "user", "content": "What is the best French cheese?"}],
+        )
+        print(response.choices[0].message.content)
+
+
+if __name__ == "__main__":
+    main()

examples/mistral/observability/global_provider_with_redaction.py

@@ -0,0 +1,44 @@
+#!/usr/bin/env python
+"""Global provider mode with application-owned redaction.
+
+In global (or custom TracerProvider) mode your application owns the OTEL export
+pipeline, so `configure_telemetry`'s redaction argument is ignored. To redact
+spans you wrap your own exporter with RedactingSpanExporter.
+
+Requires the telemetry extra: pip install "mistralai[telemetry]"
+"""
+
+import os
+
+from opentelemetry import trace
+from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import BatchSpanProcessor
+
+from mistralai.client import Mistral
+from mistralai.extra.observability import RedactingSpanExporter, configure_telemetry
+
+
+def main() -> None:
+    api_key = os.environ["MISTRAL_API_KEY"]
+
+    # Build your own provider and wrap the exporter with redaction.
+    provider = TracerProvider()
+    provider.add_span_processor(
+        BatchSpanProcessor(RedactingSpanExporter(OTLPSpanExporter()))
+    )
+    trace.set_tracer_provider(provider)
+
+    with Mistral(api_key=api_key) as client:
+        # SDK spans flow through the global provider configured above.
+        configure_telemetry(client, provider="global")
+
+        response = client.chat.complete(
+            model="mistral-small-latest",
+            messages=[{"role": "user", "content": "Say hello."}],
+        )
+        print(response.choices[0].message.content)
+
+
+if __name__ == "__main__":
+    main()

examples/mistral/observability/redaction_policies.py

@@ -0,0 +1,142 @@
+#!/usr/bin/env python
+"""Choosing and extending a redaction policy in dedicated telemetry mode.
+
+The `redaction` argument of `configure_telemetry` accepts:
+  - True (default): the regex (content-oriented) policy
+  - False: redaction disabled
+  - a RedactionPolicy instance (e.g. AttributeRedactionPolicy)
+  - a (key, value) -> value | None callback
+
+Built-in policies expose their defaults as public constants so you can
+extend them instead of replacing them wholesale:
+  - AttributeRedactionPolicy: DEFAULT_SENSITIVE_ATTRIBUTE_KEYS,
+    DEFAULT_SENSITIVE_ATTRIBUTE_FRAGMENTS, DEFAULT_SAFE_ATTRIBUTE_KEYS,
+    DEFAULT_TOKEN_PATTERNS
+  - RegexRedactionPolicy: DEFAULT_PII_SECRET_PATTERNS
+
+CallbackRedactionPolicy is different in kind: instead of extending constants you
+supply a `(key, value) -> value | None` function and decide per attribute. It is
+the only policy that can return None to drop an attribute entirely; the built-in
+policies replace values in place and never remove keys.
+
+`demonstrate()` applies the policies directly to a sample attribute mapping and
+prints the before/after; it needs no API key and no network. `main()` wires an
+extended policy into `configure_telemetry` and issues a live request (requires
+the telemetry extra: pip install "mistralai[telemetry]").
+"""
+
+import os
+import re
+
+from opentelemetry.util.types import AttributeValue
+
+from mistralai.client import Mistral
+from mistralai.extra.observability import (
+    DEFAULT_PII_SECRET_PATTERNS,
+    DEFAULT_SENSITIVE_ATTRIBUTE_KEYS,
+    AttributeRedactionPolicy,
+    CallbackRedactionPolicy,
+    RegexRedactionPolicy,
+    configure_telemetry,
+)
+
+# A custom attribute key your application sets and wants masked, on top of the
+# built-in sensitive keys.
+CUSTOMER_EMAIL_KEY = "app.customer.email"
+
+# A custom secret shape (e.g. an internal token) the default patterns don't know.
+ACME_TOKEN_PATTERN = re.compile(r"\bacme-[a-z0-9]{16}\b")
+
+
+def build_attribute_policy() -> AttributeRedactionPolicy:
+    """Key-oriented policy: defaults plus one extra sensitive key."""
+    return AttributeRedactionPolicy(
+        sensitive_keys=DEFAULT_SENSITIVE_ATTRIBUTE_KEYS | {CUSTOMER_EMAIL_KEY},
+    )
+
+
+def build_regex_policy() -> RegexRedactionPolicy:
+    """Content-oriented policy: default patterns plus one extra secret shape."""
+    return RegexRedactionPolicy(
+        patterns=(*DEFAULT_PII_SECRET_PATTERNS, ACME_TOKEN_PATTERN),
+    )
+
+
+def custom_mask(key: str, value: AttributeValue) -> AttributeValue | None:
+    """Callback: full control over each attribute.
+
+    Return the value to keep it, a transformed value to mask it, or None to drop
+    the attribute entirely (something the built-in policies cannot do).
+    """
+    # Drop your application's private namespace outright.
+    if key.startswith("app."):
+        return None
+    # Leave model/usage metadata untouched.
+    if key.startswith(("gen_ai.request", "gen_ai.usage")):
+        return value
+    # Mask anything else with your own marker.
+    return "***"
+
+
+def build_callback_policy() -> CallbackRedactionPolicy:
+    """Callback-based policy wrapping the custom_mask function above."""
+    return CallbackRedactionPolicy(custom_mask)
+
+
+def demonstrate() -> None:
+    """Show, without any network call, what each extended policy redacts."""
+    attributes = {
+        # Safe key, survives both policies.
+        "gen_ai.request.model": "mistral-small-latest",
+        # Built-in sensitive key: whole value dropped by the attribute policy.
+        "gen_ai.input.messages": "user: my card is 4111 1111 1111 1111",
+        # Custom key we added to the attribute policy's sensitive set.
+        CUSTOMER_EMAIL_KEY: "jane@example.com",
+        # Free-form value carrying secrets: caught by the regex policy's patterns.
+        "http.request.body": "Authorization: Bearer sk-abcdefghijklmnopqrst; "
+        "internal=acme-0123456789abcdef",
+    }
+
+    for label, policy in (
+        ("AttributeRedactionPolicy (extended keys)", build_attribute_policy()),
+        ("RegexRedactionPolicy (extended patterns)", build_regex_policy()),
+        ("CallbackRedactionPolicy (custom function)", build_callback_policy()),
+    ):
+        print(f"\n{label}")
+        redacted = policy.redact_attributes(attributes)
+        for key in attributes:
+            # A callback may drop a key entirely (returns None), so it can be
+            # absent from the redacted mapping.
+            after = redacted[key] if key in redacted else "<dropped>"
+            print(f"  {key}")
+            print(f"    before: {attributes[key]}")
+            print(f"    after : {after}")
+
+
+def main() -> None:
+    demonstrate()
+
+    api_key = os.environ.get("MISTRAL_API_KEY")
+    if not api_key:
+        print("\nSet MISTRAL_API_KEY to run the live configure_telemetry example.")
+        return
+
+    with Mistral(api_key=api_key) as client:
+        # Wire the extended attribute policy into dedicated telemetry mode.
+        configure_telemetry(client, redaction=build_attribute_policy())
+
+        # Alternatives:
+        # configure_telemetry(client, redaction=build_regex_policy())
+        # configure_telemetry(client, redaction=custom_mask)          # bare callback
+        # configure_telemetry(client, redaction=build_callback_policy())
+        # configure_telemetry(client, redaction=False)               # disable entirely
+
+        response = client.chat.complete(
+            model="mistral-small-latest",
+            messages=[{"role": "user", "content": "Say hello."}],
+        )
+        print(response.choices[0].message.content)
+
+
+if __name__ == "__main__":
+    main()