Rename evaluate to decide and Noul to predicate

docs/ai-python/content/docs/basics/model-operations.mdx

@@ -1,8 +1,8 @@
 ---
 title: Non-LLM models
-description: Generate media and embeddings, transcribe audio, rerank documents, and evaluate state.
+description: Generate media and embeddings, transcribe audio, rerank documents, and make decisions.
 type: guide
-summary: Use dedicated models for images, video, speech, embeddings, transcription, reranking, and evaluation.
+summary: Use dedicated models for images, video, speech, embeddings, transcription, reranking, and decisions.
 ---
 
 Use `ai.ops` for model calls that do not produce a language-model message.
@@ -253,9 +253,9 @@ for ranked in result.value:
 Each result contains the document's original index and relevance score. Results
 are ordered from the highest score to the lowest.
 
-## Evaluate state
+## Make decisions
 
-Use `ai.ops.experimental.evaluate` to ask choice, score, and Noul questions about one shared state. Define matching Pydantic models for the questions and answers:
+Use `ai.ops.experimental.decide` to ask choice, score, and predicate questions about one shared state. Define matching Pydantic models for the questions and answers:
 
 ```python
 import pydantic
@@ -264,16 +264,16 @@ import pydantic
 class Questions(pydantic.BaseModel):
     department: ai.ops.experimental.ChoiceQuestion
     severity: ai.ops.experimental.ScoreQuestion
-    requests_refund: ai.ops.experimental.NoulQuestion
+    requests_refund: ai.ops.experimental.PredicateQuestion
 
 
 class Answers(pydantic.BaseModel):
     department: ai.ops.experimental.ChoiceAnswer
     severity: ai.ops.experimental.ScoreAnswer
-    requests_refund: ai.ops.experimental.NoulAnswer
+    requests_refund: ai.ops.experimental.PredicateAnswer
 
 
-result = await ai.ops.experimental.evaluate(
+result = await ai.ops.experimental.decide(
     ai.get_model("typesafe-ai/jev"),
     {"message": "Please refund the duplicate charge."},
     Questions(
@@ -285,7 +285,7 @@ result = await ai.ops.experimental.evaluate(
             instructions="How severe is this?",
             criteria=["Cosmetic", "Has workaround", "Blocking"],
         ),
-        requests_refund=ai.ops.experimental.NoulQuestion(
+        requests_refund=ai.ops.experimental.PredicateQuestion(
             instructions="Is the customer requesting a refund?",
         ),
     ),
@@ -299,11 +299,11 @@ When question IDs are dynamic, pass a mapping and omit `output_type`. The item
 value is then a validated answer mapping:
 
 ```python
-result = await ai.ops.experimental.evaluate(
+result = await ai.ops.experimental.decide(
     ai.get_model("typesafe-ai/jev"),
     state,
     {
-        question_id: ai.ops.experimental.NoulQuestion(instructions=instructions)
+        question_id: ai.ops.experimental.PredicateQuestion(instructions=instructions)
         for question_id, instructions in dynamic_questions.items()
     },
 )

docs/ai-python/content/docs/reference/ai.providers/ai-gateway/index.mdx

@@ -549,7 +549,7 @@ The Python provider currently supports:
 - Structured output.
 - Function and provider-executed tools.
 - Model listing and connection checks.
-- Image, video, and speech generation, embeddings, transcription, reranking, and evaluation through `ai.ops`.
+- Image, video, and speech generation, embeddings, transcription, reranking, and decisions through `ai.ops`.
 
 It does not currently expose realtime sessions, credit queries, spend reports,
 or generation lookup helpers.
@@ -586,7 +586,7 @@ Methods and properties:
 - `embed(...)`: Embeds text values.
 - `transcribe(...)`: Transcribes audio.
 - `rerank(...)`: Reranks documents against a query.
-- `evaluate(...)`: Evaluates typed questions against shared state.
+- `decide(...)`: Decides typed questions about shared state.
 - `aclose()`: Closes provider-owned resources.
 
 ### GatewayParams

docs/ai-python/content/docs/reference/ai.providers/index.mdx

@@ -31,7 +31,7 @@ provider-specific namespaces.
 - `embed`: Embed text values.
 - `transcribe`: Transcribe audio.
 - `rerank`: Rerank documents against a query.
-- `evaluate`: Evaluate typed questions against shared state.
+- `decide`: Decide typed questions about shared state.
 
 Application code normally uses `ai.stream`, `ai.experimental_generate`, or
 [`ai.ops`](/docs/reference/ops) instead of calling provider methods directly.

docs/ai-python/content/docs/reference/ops.mdx

@@ -2,7 +2,7 @@
 title: "ai.ops"
 description: Reference for dedicated model operations.
 type: reference
-summary: Reference for image, video, speech, embedding, transcription, reranking, and evaluation APIs.
+summary: Reference for image, video, speech, embedding, transcription, reranking, and decision APIs.
 ---
 
 `ai.ops` contains model operations that return standalone values instead of
@@ -239,13 +239,13 @@ provider.
 - `index`: Position of the document in the original input list.
 - `score`: Relevance score for the query.
 
-## experimental.evaluate
+## experimental.decide
 
-Evaluation questions, answers, `EvaluationInput`, and `EvaluationParams` live
+Decision questions, answers, `DecisionInput`, and `DecideParams` live
 under `ai.ops.experimental`.
 
 ```python
-await ai.ops.experimental.evaluate(
+await ai.ops.experimental.decide(
     model,
     state,
     questions,
@@ -257,25 +257,25 @@ await ai.ops.experimental.evaluate(
 
 Arguments:
 
-- `model`: Evaluation model.
+- `model`: Decision model.
 - `state`: Shared JSON-compatible string, object, or array.
 - `questions`: A nonempty Pydantic model or mapping whose fields contain
-  `ChoiceQuestion`, `ScoreQuestion`, or `NoulQuestion` values.
+  `ChoiceQuestion`, `ScoreQuestion`, or `PredicateQuestion` values.
 - `output_type`: Required with a Pydantic question model. It must be a Pydantic
   model class with the same fields and corresponding answer types. Omit it with
   mapped questions.
-- `params`: Optional `EvaluationParams`.
+- `params`: Optional `DecideParams`.
 
 Pydantic questions return `Item[Answers]`, whose value is an instance of
 `output_type`. Mapped questions return
-`Item[dict[str, ChoiceAnswer | ScoreAnswer | NoulAnswer]]`:
+`Item[dict[str, ChoiceAnswer | ScoreAnswer | PredicateAnswer]]`:
 
 ```python
-result = await ai.ops.experimental.evaluate(
+result = await ai.ops.experimental.decide(
     model,
     state,
     {
-        question_id: ai.ops.experimental.NoulQuestion(instructions=instructions)
+        question_id: ai.ops.experimental.PredicateQuestion(instructions=instructions)
         for question_id, instructions in dynamic_questions.items()
     },
 )
@@ -295,17 +295,21 @@ Question and answer types:
   index `0`. `ScoreAnswer` contains a fractional `score` and an optional complete
   `probabilities` distribution keyed by string indices. With a distribution,
   `score` is its probability-weighted mean.
-- `NoulQuestion`: Estimates whether its instructions are true. Its optional
-  `criteria` mapping can describe `"true"` and `"false"`. `NoulAnswer.noul`
+- `PredicateQuestion`: Estimates whether its instructions are true. Its optional
+  `criteria` mapping can describe `"true"` and `"false"`. `PredicateAnswer.probability`
   is the estimated probability of true, not confidence in either outcome.
 
 `ChoiceAnswer.confidence` and `ScoreAnswer.confidence` expose a value from 0 to 1
 when the provider supplies it, or `None` otherwise. For TypeSafe through AI Gateway,
 these values come from `provider_metadata["typesafe"]["confidence"]`, which is also
-preserved on the result. Noul answers have no separate confidence field.
+preserved on the result. Predicate answers have no separate confidence field.
 
 Instructions accept strings, JSON objects, or JSON arrays. Criteria descriptions
 accept the same values or `None`. Nested keys are model input and have no
 SDK-defined meaning.
 
-`EvaluationParams` contains `provider_options`, keyed by provider name.
+`DecideParams` contains `provider_options`, keyed by provider name.
+
+The old `evaluate`, `EvaluationInput`, `EvaluationParams`, and `Noul*` names,
+`type="noul"`, and `PredicateAnswer.noul` still work but emit a
+`DeprecationWarning`.

docs/ai-python/content/docs/reference/telemetry.mdx

@@ -95,7 +95,7 @@ Built-in data types:
 | `GenerateVideoSpanData` | `generate_video` | One video generation operation. |
 | `RerankSpanData` | `rerank` | One reranking operation. |
 | `TranscribeSpanData` | `transcribe` | One transcription operation. |
-| `EvaluateSpanData` | `evaluate` | One typed evaluation operation. |
+| `DecideSpanData` | `decide` | One typed decision operation. |
 | `ToolExecutionSpanData` | `tool_execution` | One tool execution, from dispatch to result. |
 | `HookSpanData` | `hook` | One hook suspension, from deferred until resolved or cancelled. |
 | `CustomSpanData` | `custom` | A user span made with `span("name")`; its attributes live in `attrs`. |
@@ -398,7 +398,7 @@ The adapter names spans and sets attributes per the `gen_ai` conventions:
 | `AiGenerateSpanData` | `chat {model}` | `chat` |
 | `EmbedSpanData` | `embeddings {model}` | `embeddings` |
 | `GenerateAudioSpanData`, `GenerateImageSpanData`, `GenerateVideoSpanData` | `generate_content {model}` | `generate_content` |
-| `TranscribeSpanData`, `RerankSpanData`, `EvaluateSpanData` | `{kind} {model}` | The span data kind |
+| `TranscribeSpanData`, `RerankSpanData`, `DecideSpanData` | `{kind} {model}` | The span data kind |
 | `ToolExecutionSpanData` | `execute_tool {tool_name}` | `execute_tool` |
 | Other | The span's own name | — |
 

examples/models/gateway/evaluation.py → examples/models/gateway/decisions.py

@@ -1,11 +1,11 @@
-"""Evaluate plain text and structured data with Jev through AI Gateway.
+"""Ask decision questions about text and data with Jev through AI Gateway.
 
 Set ``AI_GATEWAY_API_KEY`` and run:
 
-    uv run python examples/models/gateway/evaluation.py
+    uv run python examples/models/gateway/decisions.py
 
 The two calls below are independent examples. The first asks one question about
-plain text. The second asks choice, score, and Noul questions about shared
+plain text. The second asks choice, score, and predicate questions about shared
 structured state in one request.
 """
 
@@ -17,23 +17,23 @@
 
 
 class RefundQuestions(pydantic.BaseModel):
-    requests_refund: ai.ops.experimental.NoulQuestion
+    requests_refund: ai.ops.experimental.PredicateQuestion
 
 
 class RefundAnswers(pydantic.BaseModel):
-    requests_refund: ai.ops.experimental.NoulAnswer
+    requests_refund: ai.ops.experimental.PredicateAnswer
 
 
 class TicketQuestions(pydantic.BaseModel):
     queue: ai.ops.experimental.ChoiceQuestion
     urgency: ai.ops.experimental.ScoreQuestion
-    refund_warranted: ai.ops.experimental.NoulQuestion
+    refund_warranted: ai.ops.experimental.PredicateQuestion
 
 
 class TicketAnswers(pydantic.BaseModel):
     queue: ai.ops.experimental.ChoiceAnswer
     urgency: ai.ops.experimental.ScoreAnswer
-    refund_warranted: ai.ops.experimental.NoulAnswer
+    refund_warranted: ai.ops.experimental.PredicateAnswer
 
 
 async def main() -> None:
@@ -42,13 +42,13 @@ async def main() -> None:
         print("Set AI_GATEWAY_API_KEY to run this example.")
         return
 
-    # Ask a single Noul question about plain text. Noul answers are
+    # Ask a single predicate question about plain text. Predicate answers are
     # probabilities rather than only true or false.
-    refund_result = await ai.ops.experimental.evaluate(
+    refund_result = await ai.ops.experimental.decide(
         model,
         "Please refund the duplicate charge on my account.",
         RefundQuestions(
-            requests_refund=ai.ops.experimental.NoulQuestion(
+            requests_refund=ai.ops.experimental.PredicateQuestion(
                 instructions="Is the customer asking for a refund?",
             )
         ),
@@ -57,12 +57,12 @@ async def main() -> None:
 
     print(
         "refund requested:",
-        f"{refund_result.value.requests_refund.noul:.0%}",
+        f"{refund_result.value.requests_refund.probability:.0%}",
     )
 
     # Ask several question types about the same structured state. This is useful
     # when related decisions should use exactly the same source information.
-    ticket: ai.ops.experimental.EvaluationInput = {
+    ticket: ai.ops.experimental.DecisionInput = {
         "message": (
             "I was charged $240 twice for the same renewal. The service works, "
             "but please refund the duplicate charge."
@@ -88,7 +88,7 @@ async def main() -> None:
         "service_status": "operational",
     }
 
-    result = await ai.ops.experimental.evaluate(
+    result = await ai.ops.experimental.decide(
         model,
         ticket,
         TicketQuestions(
@@ -110,7 +110,7 @@ async def main() -> None:
                     "Critical: security incident, outage, or ongoing loss",
                 ],
             ),
-            refund_warranted=ai.ops.experimental.NoulQuestion(
+            refund_warranted=ai.ops.experimental.PredicateQuestion(
                 instructions="Does the evidence warrant a refund?",
                 criteria={
                     "true": "A duplicate settlement or billing error is shown",
@@ -120,7 +120,7 @@ async def main() -> None:
         ),
         output_type=TicketAnswers,
         # Provider options are optional and apply only to this request.
-        params=ai.ops.experimental.EvaluationParams(
+        params=ai.ops.experimental.DecideParams(
             provider_options={
                 "gateway": {
                     "zeroDataRetention": True,
@@ -141,7 +141,7 @@ async def main() -> None:
 
     print(
         "\nrefund warranted:",
-        f"{answers.refund_warranted.noul:.0%}",
+        f"{answers.refund_warranted.probability:.0%}",
     )
 
     # Every operation also exposes framework and provider metadata.

examples/models/gateway/jev_python_ast.py

@@ -775,7 +775,7 @@ async def generate(
             status(f"Filling {'.'.join(map(str, path))} ({kind})")
             started = time.perf_counter()
             try:
-                result = await ai.ops.experimental.evaluate(
+                result = await ai.ops.experimental.decide(
                     model, state.model_dump(), questions, output_type=Answers
                 )
                 break

skills/ai/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: ai
-description: AI SDK for Python (the `ai` package). Use for Python model calls (LLMs, image, video, speech, embedding, transcription, reranking, or evaluation), agents, model interaction tests, tool calling, subagents, approvals, durable execution, telemetry, AI SDK UI backends, and custom providers.
+description: AI SDK for Python (the `ai` package). Use for Python model calls (LLMs, image, video, speech, embedding, transcription, reranking, or decisions), agents, model interaction tests, tool calling, subagents, approvals, durable execution, telemetry, AI SDK UI backends, and custom providers.
 metadata:
   sdk-version: "0.5.0"
 ---
@@ -71,7 +71,7 @@ read the listed local notes before writing code.
 | Provider clients, options, discovery | `basics/providers.md` | — |
 | Structured output, complex streams | `basics/streaming.md` | — |
 | Buffered language-model calls | `basics/streaming.md` | — |
-| Images, video, speech, embeddings, transcription, reranking, evaluation | `basics/model-operations.md` | — |
+| Images, video, speech, embeddings, transcription, reranking, decisions | `basics/model-operations.md` | — |
 | Events and serialization | `basics/messages-and-events.md` | — |
 | Advanced tools, streaming, aggregation | `basics/tools.md` | [streaming-tools.md](references/streaming-tools.md) |
 | Advanced agent behavior | `basics/agents.md` | — |

src/ai/experimental_telemetry/__init__.py

@@ -5,6 +5,9 @@
 See :mod:`.span` for the full story.
 """
 
+import warnings
+from typing import TYPE_CHECKING, Any
+
 from .span import (
     FIRST_TOKEN,
     HOOK_CANCELLED,
@@ -17,9 +20,9 @@
     AiGenerateSpanData,
     AiStreamSpanData,
     CustomSpanData,
+    DecideSpanData,
     DictSink,
     EmbedSpanData,
-    EvaluateSpanData,
     GenerateAudioSpanData,
     GenerateImageSpanData,
     GenerateVideoSpanData,
@@ -60,9 +63,9 @@
     "AiGenerateSpanData",
     "AiStreamSpanData",
     "CustomSpanData",
+    "DecideSpanData",
     "DictSink",
     "EmbedSpanData",
-    "EvaluateSpanData",
     "GenerateAudioSpanData",
     "GenerateImageSpanData",
     "GenerateVideoSpanData",
@@ -90,3 +93,19 @@
     "use_span",
     "use_time",
 ]
+
+if TYPE_CHECKING:
+    EvaluateSpanData = DecideSpanData
+else:
+
+    def __getattr__(name: str) -> Any:
+        # Deprecated names, kept as aliases of their replacements.
+        if name == "EvaluateSpanData":
+            warnings.warn(
+                "ai.experimental_telemetry.EvaluateSpanData is deprecated, "
+                "use ai.experimental_telemetry.DecideSpanData",
+                DeprecationWarning,
+                stacklevel=2,
+            )
+            return DecideSpanData
+        raise AttributeError(f"module {__name__!r} has no attribute {name!r}")