diff --git a/README.md b/README.md index 6cf44cc..a02524a 100644 --- a/README.md +++ b/README.md @@ -105,11 +105,11 @@ An earlier n=30 head-to-head against Mem0, Letta, and LangMem lives with its cav ## Sarvam research: investigating the problem -We test native Sarvam models with ordinary SQLite storage, tool permissions and source transcripts before deciding whether a new memory architecture is needed. English, Hindi and Telugu are diagnostic languages here; they do not establish support across all Sarvam languages. +We test fixed Sarvam models with ordinary storage and preserved source evidence before deciding whether a new memory product is needed. The latest comparison completed **720 reporter calls across all 240 saved English/Hindi/Telugu histories**, comparing chronology, lossless grouping, and grouping without prior agent prose. -The corrected comparison attempted **240 native episodes** using fixed models and shared histories for two reporter variants. Base105B date-plus-clarification accuracy improved **90/120 → 105/120 (75% → 87.5%)**. Conversations moved **95/120 → 96/120**, but wrong non-null dates increased **18 → 22**. One capped native response prevented both reporters from running; the two missing slots remain failures. **The acceptance gate failed and the candidate is not promoted.** [Report and reproducible evidence](benchmarks/audits/sarvam-fresh-confirmation-v3-2026-10-13/REPORT.md). +Removing prose moved date-plus-clarification accuracy **104→105/120 on base105B** and **96→103/120 on Conversations** versus the contemporaneous chronological control. Conversations wrong definite dates fell **22→8**, but false abstentions rose **2→9**, raw status accuracy fell **109→97/120**, and the **predeclared acceptance gate failed**. The candidate is not promoted. Paired uncertainty intervals include zero. [Full result and raw replay package](benchmarks/audits/sarvam-source-interpretation-2026-10-13/REPORT.md). -This is a reporting-integration result on related synthetic development cases, with assistant-only language review. It does not demonstrate a new memory advantage or improve the separate English LongMemEval baseline. An evidence-view component and metered comparison harness are implemented, but the proposed 108-call screen remains unexecuted because its complete-parent requirements were not met. A subsequent two-call diagnostic recovered that one skipped English report with the same model and no action retry; both reporter variants were correct. This is one observed recovery, with added compute, and does not rewrite the study scores. [Recovery result](benchmarks/audits/sarvam-capped-recovery-2026-10-13/REPORT.md). Follow the findings, recovery work and problem-selection criteria in [the Sarvam research index](docs/SARVAM_RESEARCH.md). A reviewed public benchmark remains required. +These are source-present reporting failures and synthetic development results with assistant-only language review. They establish neither a unique multilingual memory advantage nor a newer English LongMemEval score. The earlier corrected v3 study and one-case recovery remain unchanged. The next proposed task is an ordinary question-specific evidence contract separating requested facts, conditional preferences and confirmed state; it is not a completed fix. [Research history and product criteria](docs/SARVAM_RESEARCH.md). A reviewed public multilingual benchmark remains required. --- diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/NEXT_STEP.md b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/NEXT_STEP.md new file mode 100644 index 0000000..5f75f25 --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/NEXT_STEP.md @@ -0,0 +1,27 @@ +# Next task — proposed offline work, not another paid run + +The 720-call phase is closed. C improves some factual counts but fails the frozen reliability gate; do not promote it, retry failures, replace cases, or run another wording/prose-removal sweep. Historical v3, the blocked 108-call screen and the one-case recovery remain unchanged. + +## The narrower problem to solve + +The same record can contain a customer's requested date, a condition-dependent preference and the backend's confirmed date. Those facts can legitimately differ. The current integration sometimes picks the wrong fact, or abstains despite a known answer. Removing all prior prose helps some questions and harms others. Original evidence is present, so adding persistence alone is not justified by these results. + +Prepare an ordinary, explicit evidence contract before inventing a new memory architecture: + +1. Define separate records for a user assertion/request/preference, its booking reference and any correction or unresolved condition. Keep exact source spans and references. Keep confirmed state and action receipts in their own fields. +2. Define a query plan that identifies the requested fact and booking from the actual user question. No case IDs, family tags, gold targets or per-language case exceptions may enter the runtime. Do not assume the model can extract this plan correctly; extraction failures must count. +3. Require any proposed answer to identify its supporting record and any condition it depends on. A quote match checks provenance only; semantic support must still be tested. A missing condition can justify clarification, but lack of execution approval does not erase a known customer preference. +4. Keep authoritative action status owned by the existing receipt layer. Do not use model prose as proof of a write, and do not claim this existing guard as novel. +5. Create an offline case-to-contract review that includes every observed gain, loss, cap and no-treatment mismatch. Prove source preservation and expose ambiguity before an implementation or paid request is approved as ready. + +This is a proposed design and test specification, not a completed feature or predicted gain. Use simple typed storage/source scoping as a strong ordinary control. Current [Mem0, Zep/Graphiti and Supermemory documentation](PRIOR_ART_UPDATE.md) already covers related temporal, source and schema capabilities. We have not tested these competitors on this dataset or established a feature absent from them. + +## Before a future paid comparison + +- Freeze a candidate and a contemporaneous ordinary control with the same Sarvam profiles. If extraction/verification adds calls, provide an equal-call/equal-budget control; show actual tokens, latency and cost separately. +- Use fresh workflow instances with independent wording and source review, including known and unknown conditions, clear corrections, multiple bookings and contradictory assistant claims. Do not call reused templates an independent market sample. Seek native human review before a public language-quality claim; current assistant-only review remains explicitly limited. +- Predeclare date-plus-clarification, false abstention, wrong definite dates, incomplete outputs and receipt-owned status, with paired gains/losses and scenario-level uncertainty. A net score rise cannot hide new reliability failures. +- Publish an auditable benchmark package as required by the founder, but distinguish this development bank from a validated public benchmark. Expand language coverage by evidence and review capacity, rather than claiming all-language support from Hindi/Telugu. +- Reproduce a useful gain on fresh cases, then test an actual user workflow at acceptable latency. Only then decide whether the method merits a product or paper claim. + +No further paid run was launched after this 720-call phase. The next paid package must be concrete and notified before dispatch under the standing total budget. The founder does not need to provide another key. No external outreach or daily automation is authorized by this proposal. diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/PRIOR_ART_UPDATE.md b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/PRIOR_ART_UPDATE.md new file mode 100644 index 0000000..ed213e8 --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/PRIOR_ART_UPDATE.md @@ -0,0 +1,16 @@ +# Product differentiation check — October 13, 2026 + +Checked October 14 UTC, October 13 America/New_York, while the frozen 720-call campaign was running. This targeted primary-source check does not change the protocol, inputs, acceptance rules or paid requests. It supplements the earlier product decision frame; it is not an exhaustive literature review or a competitor evaluation. + +| Primary source | Documented capability | Consequence for our claim | +| --- | --- | --- | +| [Mem0 temporal reasoning](https://mem0.ai/blog/introducing-temporal-reasoning-in-mem0), updated September 28, 2026 | Temporal enrichment, distinctions between plans and states, evolving state keys, and time-aware retrieval. | Storing dates or distinguishing an old fact from a current one is already established product work. | +| [Zep Context Graph overview](https://help.getzep.com/graph-overview), current v3 documentation | Facts retain source-episode links and validity times. The documentation explicitly distinguishes provenance from correctness. | Source preservation alone is neither a new feature nor proof of reliable interpretation. | +| [Graphiti custom entity and edge types](https://help.getzep.com/graphiti/core-concepts/custom-entity-and-edge-types) | Domain-specific schemas for extracted entities and relationships. | A typed fact representation is an ordinary baseline we must compare against. | +| [Supermemory relationship semantics](https://supermemory.ai/blog/memory-graph-relationship-semantics/), July 23, 2026 | Distinguishes updates, extensions and uncertain inferences, with source and authority considerations. | Separating a derived belief from an authoritative account state is also prior art. | + +These are vendor descriptions of capabilities, not measured results on our cases. We have not run these competing systems, established equal-compute comparisons with them, or established their Hindi/Telugu accuracy. Missing documentation would not establish that a capability is absent. + +The credible near-term output is a reproducible failure analysis and, only if measured, a bounded improvement with fixed Sarvam models. Product differentiation requires a fresh, independently reviewed comparison showing that our method prevents meaningful user-facing errors better than ordinary source scoping, typed records and existing frameworks at an acceptable cost and latency. A benchmark contribution also requires dataset quality, public artifacts, clear denominators and an honest account of development exposure; it does not require pretending that the current intervention succeeded. + +The present test can discriminate between these specific evidence-presentation choices. It cannot establish a missing persistent-memory layer because the relevant original evidence is already supplied. The final product decision must follow the complete results and their regressions. diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/PROTOCOL.md b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/PROTOCOL.md new file mode 100644 index 0000000..f2f7c72 --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/PROTOCOL.md @@ -0,0 +1,44 @@ +# Prospective source-interpretation comparison — October 13, 2026 (New York) + +Status: preparing before any new response. User authorizes the next proposed phase under the original total INR6,000 research ceiling. New scope; do not resume, alter or replace the closed v3 cohort, capped recovery diagnostic, or blocked 108-call screen. + +## Question and fixed intervention + +Does hiding the actor's prior prose, while preserving original source evidence and actual execution receipts, improve the final answer? Primary intervention is C versus B. Secondary organization comparison is B versus A; total operational comparison is C versus A. Report all three, not just the winner. The preselected forward candidate is C. Positive B-only results remain exploratory and require a separately declared confirmation; no retroactive primary change. + +Use every one of the 240 saved v3 native histories: 40 related source scenarios, rendered in English/Hindi/Telugu, for each of two fixed Sarvam profiles. For 239 histories use the saved candidate reporter request and actual tool trace. For confirm-fresh-13-en-m0 reconstruct from the original actor request and capped native receipt using the reviewed handoff helper; verify no action, unchanged backend state and exact scope. Do not use either new diagnostic answer as source evidence or impute an old cohort result. + +A: ordinary chronological evidence, complete, under the common new reporter instruction. This is a fresh control, not the unchanged v3 reporter prompt or a reused historical answer. B: lossless source/execution/actor/protocol groups, mechanically reconstructable to A's evidence. C: B with only nonempty actor content/reasoning_content/refusal strings omitted. Keep exact source text, references, dates, conditions, question, scope, tool proposals and actual receipts. Keep a local omission ledger; do not send hidden labels, review annotations, gold, inferred facts or the duplicate audit sidecar. No native actor call, tool execution, backend update or answer extraction is permitted in this phase. + +All arms use the same reporter instruction/schema: existing sarvam_evidence_ablation.SYSTEM. Sarvam-105B reasoning low; Sarvam-105B-Conversations reasoning null; temperature 0; max_tokens 8192; one request per history/arm. Same call counts and ceilings, not equal actual tokens. C intentionally removes actor prose strings; actor tool proposals remain. Do not attribute an effect specifically to authority confusion rather than compression/context change; this experiment cannot distinguish those mechanisms. + +## Pre-response source census and allocation + +Independent source checks found nonempty omissions in 184/240 histories: base 98/120, Conversations 86/120. Remaining 56 are B=C byte-identical inputs, including the capped episode. Keep them in all-episode results and explicitly report them as a negative-control/no-treatment slice. Differences there cannot demonstrate a removal effect. The treated slice is predefined solely from input content before new outputs, not by correctness. Verify and freeze exact IDs. + +720 prospective slots: 3 per history. Assign each scenario's six language/model histories the six arm permutations once, rotating across the 40 scenarios. Seed 202610134 for deterministic history ordering; keep each history's assigned arm order. Audit planned arm-position counts by model/language/family. Launch at most four requests per batch with four workers, globally paced at 90 RPM (user-reported account limit 120). Concurrency means observed dispatch order within a batch can vary; retain local pacer-release timestamps and report allocation versus locally observed order separately. These timestamps precede request reservation/transport and do not establish provider receipt order. Do not change order after responses. + +## Metrics and advancement criterion + +Primary metric: exact source-target date AND needs_clarification, with invalid, missing, refused or truncated finals counted as failures in the planned denominator. Sealed existing labels are used only in offline scoring; preparation and execution may copy/hash opaque label bytes. Independent raw-response aggregation and assistant source review must verify results. Human/native independent review is absent. + +Primary treatment effect: paired C minus B on the 184 treated histories, reported separately for each model (98 and 86). Also report all-240 C-B, A-B and A-C, and the 56 no-treatment slice. For each show correct counts, gains/losses, valid-output recovery versus fixes between usable answers, wrong non-null dates, raw action-status accuracy, receipt-projected status accuracy, fully correct reports, unknown-condition guesses, false abstentions (valid output with null answer or clarification=true when the source target is determinate), invalid outputs, language/family slices, input/output tokens, cost and request latency. + +Screen passes for a subsequent fresh confirmation only if all conditions hold: +1. All720 slots attempted once with complete auditable receipts, no infrastructure error or unknown usage; scoring/source integrity verified. Model output caps remain accuracy failures, not excluded rows. +2. Treated C-B date+clarification gain is strictly positive on each fixed model and at least two distinct workflow IDs per model have a positive net treated gain across their translations. +3. All-episode C-A date+clarification gain is strictly positive on each fixed model. +4. On all episodes for each model, C has no more wrong non-null dates or false abstentions on determinate targets than either A or B, and neither raw nor receipt-projected status accuracy is lower than either control. Invalid finals are reported separately and remain primary-metric failures. +5. Independent semantic review finds no unresolved source/label ambiguity affecting these comparisons. Preserve original scores and publish any sensitivity calculation separately; do not repair gold after seeing results to pass. + +This is a development screening gate, not production, statistical-significance or novelty certification. Report 40 scenario clusters, each containing related language/model observations. Publish fixed-seed 10,000-draw paired scenario-cluster bootstrap percentile95% intervals for primary and operational contrasts, with seed202610134; repeated languages/model/arms are not independent samples. For each model, resample40workflow IDs with replacement; primary treated C-B rate is sum(paired factual deltas)/sum(treated histories) in the sampled clusters, while all-history C-A rate uses sum(paired factual deltas)/sum(all histories). Report intervals as percentage-point differences, with no unplanned weighting. If a bootstrap draw has no treated histories, omit that draw and report its count (expected zero here). Do not convert a positive point estimate with an interval crossing zero into conclusive efficacy. Report model-specific clusters and whole-study clusters without pooling models into a primary score. + +## Meter and stop rules + +Original research carry: 7,037 calls, INR271.224630 known token-cost estimate and INR371.289339 retained with headroom/unknown reservations. New phase maximum720 calls, INR1,550 phase cap, full-context worst-case reservation INR1,548.288 (360 base at3.072 +360 Conversations at1.2288). Campaign total retained ceiling remains INR6,000. Prices rechecked October14UTC/October13NewYork: input15/output60 INR per million for both profiles; ignore cache discounts conservatively. Token estimates are not invoice verification. + +Freeze code, source snapshots, protocol, budgets, pacing, all inputs, request hashes, omission ledgers, order and prior carry. Independent preflight plus focused tests before launch. Durable exclusive started marker and per-call meter reservation before dispatch. No request retries or manifest restart. On at least2 cumulative infrastructure errors, finish already started batch calls then stop before any new batch. Budget/source integrity failure stops immediately; retain already started receipts. Known cap/invalid model responses do not trigger infrastructure stop and remain failures. Retain all720 planned denominators if early stop; a stopped campaign cannot pass. Root notifies user before paid dispatch, then runs exactly this campaign once and stops regardless of result. No auto-repeat or second paid phase. + +## Product interpretation + +A gain would support a tested reporting-boundary improvement under these fixed models and synthetic development workflows. It would not prove long-term memory is necessary, that the original AgentMem English architecture is multilingual, that Sarvam lacks ordinary storage/approval tools, or that grouping/provenance is novel. A negative result rejects these interventions on this bank; it does not identify the underlying cause. Stronger ordinary controls, fresh tasks, independent language review, broader languages, action-level and cost/latency validation are required before a unique-product claim. This phase should decide whether there is enough measured benefit to warrant that work. diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/README.md b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/README.md new file mode 100644 index 0000000..fc7bc74 --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/README.md @@ -0,0 +1,25 @@ +# All-history source-interpretation comparison + +Completed720 reporter calls on240 saved histories. C increases date-plus-clarification counts from104→105/120 base and96→103/120 Conversations versus chronological A, but fails the predefined reliability gate and is not promoted. Read [REPORT.md](REPORT.md), [decision.json](decision.json), [the frozen protocol](PROTOCOL.md), and [the next offline proposal](NEXT_STEP.md). + +The study is closed to inference. Do not remove started markers or rerun paid dispatch. No actor/tool/backend action was repeated; older studies remain unchanged. Models, temperature, instructions and output ceilings were fixed, but actual token use differs. Cases are synthetic, related and development-exposed, with assistant-only language review. + +## Offline replay + +Verify `evidence.tar.gz` against `artifact-manifest.json`, then check every extracted file against its recorded SHA256. The archive contains3459 files under `phase/`, including frozen source, labels, all720 request/response receipts, results, settled meter, reviews and analysis. It excludes credentials and the private founder-file inventory. + +Use an empty directory and new output paths. These commands send no API request and need no key: + +```sh +mkdir /tmp/sarvam-source-evidence +tar -xzf benchmarks/audits/sarvam-source-interpretation-2026-10-13/evidence.tar.gz -C /tmp/sarvam-source-evidence +python -m benchmarks.sarvam_source_interpretation audit /tmp/sarvam-source-evidence/phase --output /tmp/sarvam-source-primary.json +python /tmp/sarvam-source-evidence/phase/analysis/independent_raw_audit.py /tmp/sarvam-source-evidence/phase > /tmp/sarvam-source-independent.json +python /tmp/sarvam-source-evidence/phase/analysis/verify_comparison.py /tmp/sarvam-source-evidence/phase /tmp/sarvam-source-comparison.json +``` + +The first two JSON outputs must equal the archived primary and independent audits. The final command checks the two archived audits against one another. The separate semantic-review verifier checks all720 reviewed job IDs, receipt/result bindings and agreement with the audit; its timestamp is expected to differ on replay. Review is not independent human/native-language validation. + +The earlier complete844-call native study is published in the [v3 archive](../sarvam-fresh-confirmation-v3-2026-10-13/README.md). This package includes all original paired histories and the exact reporter requests or capped-actor reconstruction needed for the new comparison. Its hashes support local integrity and reproduction; they do not provide a provider signature. + +The new module's local six-test suite passed before dispatch. One test depends on the prior local study fixture and self-skips outside that environment; public artifact replay is a separate requirement, not replaced by that fixture-dependent test. Publication verification is recorded in `VERIFICATION.md` after clean export. diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/REPORT.md b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/REPORT.md new file mode 100644 index 0000000..23a58b9 --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/REPORT.md @@ -0,0 +1,88 @@ +# Source interpretation comparison — completed October 13, 2026 + +**Some answers improved, but blanket removal of prior agent prose is not a reliable fix. The predeclared acceptance gate failed.** All 720 paid reporter calls completed once across 240 saved histories, with no infrastructure errors, unknown usage or retries. The primary replay and a separately implemented raw-response audit agree on every planned output, paired count and cost. English, Hindi and Telugu assistant reviews cover all 720 outputs; independent human/native review remains absent. + +## What was tested + +Each of 40 related synthetic scenarios has English, Hindi and Telugu versions and two saved Sarvam model histories. Each of those 240 histories received three new reporting calls: + +- **A — chronological:** the ordinary original evidence sequence. +- **B — grouped:** the same evidence organized by source, without loss. +- **C — grouped, prose withheld:** B with the earlier agent's nonempty explanatory prose removed. Original customer messages, conditions, dates, booking references and actual tool receipts remain. + +The model, reporter instruction, temperature, response schema and 8,192-token output ceiling were fixed within each comparison. Profiles were `sarvam-105b` with low reasoning and `sarvam-105b-conversations` with null reasoning. There were **no new actor calls, tool actions or backend changes**. This tests reporting on saved histories, not a fresh execution success rate. A is a contemporaneous control under the common instruction, not the older v3 control. Historical scores remain unchanged. + +## Actual answer results + +The primary score requires both the correct date and the correct clarification decision. Incomplete outputs are failures in the planned denominator. + +| Model | A: chronological | B: grouped | C: prose withheld | C versus A, paired gains / losses | +| --- | ---: | ---: | ---: | ---: | +| Sarvam-105B | 104/120 (86.7%) | 99/120 (82.5%) | 105/120 (87.5%) | 11 / 10; net +1 | +| Sarvam-105B-Conversations | 96/120 (80.0%) | 100/120 (83.3%) | 103/120 (85.8%) | 14 / 7; net +7 | + +Of the 11 base-model C-versus-A gains, five recover unusable A outputs and six fix usable but wrong answers. All 14 Conversations gains fix usable wrong answers. These are observed counts, not a demonstrated gain on unseen tasks. The descriptive 95% scenario-cluster intervals for C-minus-A are **−6.67 to +8.33 percentage points** for base and **−1.67 to +14.17 points** for Conversations; both include zero. + +Only 184 histories actually contain prose that C removes. The predeclared primary contrast was C versus B on those treated histories: + +| Model | B correct | C correct | Gains / losses | 95% descriptive cluster interval for the difference | +| --- | ---: | ---: | ---: | ---: | +| Base, 98 treated | 79/98 | 87/98 | 14 / 6; net +8 | −1.89 to +19.15 points | +| Conversations, 86 treated | 66/86 | 69/86 | 12 / 9; net +3 | −5.95 to +13.64 points | + +Five of the 14 treated base gains recover an invalid B output; nine fix usable answers. All 12 treated Conversations gains fix usable answers. Both intervals include zero. All A→B, B→C and A→C contrasts, including no-treatment slices, are retained in [summary.json](summary.json) and [audit.json](audit.json). + +## Why the candidate fails + +| Reliability measure | Base A / B / C | Conversations A / B / C | +| --- | ---: | ---: | +| Wrong definite dates; lower is better | 3 / 4 / 1 | 22 / 18 / 8 | +| Unnecessary abstentions on known answers; lower is better | 0 / 1 / 2 | 2 / 2 / 9 | +| Incomplete finals; lower is better | 5 / 8 / 6 | 0 / 0 / 0 | +| Correct raw action status | 77 / 83 / 85 | 109 / 95 / 97 | +| Correct receipt-projected status | 115 / 112 / 114 | 120 / 120 / 120 | +| Entire raw report correct: answer, clarification and status | 70 / 71 / 78 | 86 / 78 / 80 | + +C reduced wrong definite dates, but it sometimes refused to answer when the source clearly supplied the answer. Conversations' fully correct raw reports fell **86→80**, despite its date-plus-clarification count rising. The existing deterministic receipt projection repairs status on every usable final in all arms; this is the same pre-existing layer, not a new C benefit. It does not invent answers for capped outputs. + +Four frozen checks fail: false abstentions increase on both models; base projected-status coverage falls versus A because C has one more incomplete final; and Conversations raw-status accuracy falls versus A. The candidate is **not promoted**, and neither the score definition nor the acceptance rules were relaxed after results. B alone also remains exploratory: its base score falls, and its Conversations raw-status count falls. + +## What the failures tell us + +There is a real source-interpretation problem in this integration. The required evidence is present. In Conversations, C improves separate-booking preferences **0→5/12** and unresolved conditions **4→10/12** versus A, but worsens explicit request corrections **12→8/12** and resolved conditions **10→8/12**. It does not solve the broader problem. + +Three concrete source-checked examples are preserved in `analysis/root-case-review.json` inside the evidence archive: + +1. `13-en-m1`: availability is explicitly unchecked. A/B give the existing March 14 booking date; C correctly asks for clarification instead of choosing April or May without evidence. +2. `03-en-m1`: the customer explicitly replaces April 4 with May 4. A/B correctly answer May 4; C incorrectly abstains. +3. `18-en-m0`: C restores the correct April 19 preference from B's wrong March date, but falsely reports that an action was already applied. A already answered correctly. The wrong March value matches both current backend state and another booking's old preference, so its exact causal source is unresolved. + +The language slices disagree, which a pooled headline would hide. Each cell below has 40 cases: + +| Model | English A / B / C | Hindi A / B / C | Telugu A / B / C | +| --- | ---: | ---: | ---: | +| Base | 36 / 32 / 36 | 33 / 33 / 36 | 35 / 34 / 33 | +| Conversations | 33 / 32 / 32 | 31 / 35 / 34 | 32 / 33 / 37 | + +All four English Conversations request-correction cases become false abstentions under C; the corresponding Hindi corrections remain correct. Conversely, C falsely abstains on all four Hindi Conversations known-availability cases; A already abstained on two of these and B on one. These observations argue against a blanket rule that prior agent prose should always be removed. + +The 56 no-treatment histories have byte-identical B and C requests. Base gives one gain and three losses under those identical inputs, including capped responses; Conversations gives 34/34 correct in both. That is observed run variability, not a prose-removal effect. The requests were single attempts, not repeated until successful. C also shortens B's input, so the treated contrast cannot distinguish an authority effect from a context-length effect. + +## Cost and execution + +New phase: **720 calls, ₹93.896460 known token-cost estimate, ₹117.370843 retained with headroom**, zero unknown-usage calls. Native research cumulative: **7,757 calls, ₹365.121090 known / ₹488.660182 retained**, under the existing total ₹6,000 ceiling. Older separate exploratory pilots remain separately accounted; these are estimates, not invoices or a dashboard balance. + +| Model | A cost | B cost | C cost | Median reporter seconds A / B / C | +| --- | ---: | ---: | ---: | ---: | +| Base | ₹25.815420 | ₹29.891580 | ₹27.947355 | 12.83 / 14.84 / 13.58 | +| Conversations | ₹2.736045 | ₹3.870870 | ₹3.635190 | 1.00 / 1.00 / 1.00 | + +Equal call counts and output ceilings do **not** mean equal actual compute: grouped inputs are larger than A, and output/reasoning token use varies. Full token counts and request p95 latencies are in summary.json. These are reporter timings, not end-to-end native workflow or voice performance. Local dispatch timestamps record pacer release, not provider arrival order. + +## Decision and product relevance + +This is **useful diagnostic progress with a rejected general fix**. We now have measurable gains and losses on preserved source, rather than only passing software tests. We have not established persistent memory as the missing component, an industry benchmark win, reliable 90% accuracy, or a unique product. The separate English LongMemEval result remains 84.6% from its historical single run. + +Stop the blanket layout/prose-removal approach and further wording sweeps. The next offline task is to specify a question-specific evidence contract that keeps requested/preferred facts separate from confirmed backend state, preserves conditions, and always derives action status from actual receipts. Treat typed records and source-scoped retrieval as ordinary prior-art controls, not claimed inventions. A future candidate needs matched model/compute controls and fresh, independently reviewed cases before another product claim or broader rollout. See [NEXT_STEP.md](NEXT_STEP.md) and the [current primary-source prior-art check](PRIOR_ART_UPDATE.md). + +The 40 scenarios share ten families and translated variants. They are synthetic, development-exposed and assistant-reviewed, not 720 independent real customers or an industry-wide Sarvam failure rate. No real customer data or live business system was changed. Local hashes support integrity and replay; they are not cryptographic provider signatures. The evidence publication package retains failures, source, gold, requests, receipts, cost meter, paired results and all language reviews. diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/VERIFICATION.md b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/VERIFICATION.md new file mode 100644 index 0000000..17d3038 --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/VERIFICATION.md @@ -0,0 +1,11 @@ +# Publication verification + +A clean Git export of64b0ce1 verifies all3459 archive file hashes and exactly reproduces the primary720-call audit, independent standard-library raw audit and paired comparison. The separate three-language review verifier reproduces all720 source/result bindings and verdict checks; only its new timestamp differs. No API call or credential read occurred during replay. + +Five portable new-module tests and Ruff pass in that export. The sixth test depends on local parent-study fixtures and was deliberately deselected for this clean check; all six had passed locally before dispatch, alongside240 earlier focused Sarvam tests. CI outside the local fixture self-skips that test. The actual public archive replay independently verifies the real720 responses. + +An initial wrapper invocation omitted the semantic verifier's --output flag and failed argument parsing. The preserved continuation uses the correct flag and passes. No source, raw result or preceding successful audit was changed. [Clean-export receipt](clean-export-verification.json). + +Three independent assistant language reviews cover every source and output, with no assistant-detected source ambiguity or grading discrepancy. This is not native-human validation. The complete report preserves all paired losses, incomplete outputs, no-treatment variability and the failed acceptance gate. + +Before publication,6992 protected founder files,3674 closed-study artifacts, root English CURRENT_STATE and the founder main-worktree dirty status are unchanged;23 frozen source hashes match. Final exact-head PR CI, merge CI and post-merge preservation are recorded in the external task completion receipt. No candidate is promoted or made a production default. diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/artifact-manifest.json b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/artifact-manifest.json new file mode 100644 index 0000000..d2f0a97 --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/artifact-manifest.json @@ -0,0 +1,3470 @@ +{ + "archive": "evidence.tar.gz", + "sha256": "ff5f55ef34b46ad6be496bfc49470fca84a412ea376631dd77a2ba0850519f90", + "file_count": 3459, + "files": { + "phase/NEXT_STEP.md": "7222865071577183effcc880377cba73f49f3fff9c3e90b136e7f26da216f5d4", + "phase/PRIOR_ART_UPDATE.md": "4fb026f8db058c4ec8977ba8f29bd936627c8daaea5b4a3f463b4b3cec3a1c0f", + "phase/PRODUCT_DECISION.md": "5e05785885ee97771d7da8f53ad55321d8c6198cf4803aa803a8ab4133d81870", + "phase/PROTOCOL.md": "7efe12932f4291bc9137a8539cbb6edb85359f9d099bb65e3026a5f4c7832933", + "phase/REPORT.md": "84f5e1a4363edcbd46afd2b38c53a33c72bd27a626851d1463af1320d4ad7baa", + "phase/analysis/build_publication.py": "e85b0915d24b47b298b33e8ba046a09c443bee9bf444c96a26211140a5fa0b9d", + "phase/analysis/english-source-reading.json": "a046a08f0b46848cb3b8283e8de0a060d1fd60e9144ea7940db9d94d76b14d63", + "phase/analysis/english_review.py": "4fa29185890c949d45ef8627be5b2b31c23be77df175c07d8122b286d6f89c2b", + "phase/analysis/error_attribution.py": "b9d6f9e2e21674bda8abcabcaa6edb7d7a58c4533f4daf407a122f0e5a46eb27", + "phase/analysis/independent_raw_audit.py": "ccb667e53fa2d9f43c040e555615a35db75cb5292d91b7535affbfd9073649f2", + "phase/analysis/root-case-review.json": "0675d0d0763b294ac0f2ffd60efae9ed3687b85e10e21abd613d69b3f2fc201a", + "phase/analysis/summarize_results.py": "bf5193755ffcdc81d3061f460a82d74aeabd79dc7759a783cb4d89200fd51fc2", + "phase/analysis/verify_comparison.py": "be914df435ffb9edb0a4c7a39de1367f0c483aa6aa19d17dde6a016c733a67d0", + "phase/analysis/verify_semantic_reviews.py": "a7f8386863ffa7c3ef688410bb0207afc338d871c70d2d775a3d84d053625893", + "phase/audit.json": "8c208f56157cb753b0965d5b88d350ef2ee181dbb4ce5646d34e3ca36a849cc0", + "phase/budget.json": "2ee8977c2b57a89b63ec8e49494524364a440e0070e5cebdcc8aa86ad8bb72a2", + "phase/calls/confirm-fresh-01-en-m0-source-chronological.dispatch.json": "cd2f11c3c0352778c3bf8e165de6b6c889f145921dfdb0fa94d8cc3d8fdd3596", + "phase/calls/confirm-fresh-01-en-m0-source-chronological.receipt.json": "351069b5bbbeb74408ed43250637ceaf27ccfc01ba93a45207fd1d1e4aa1c381", + "phase/calls/confirm-fresh-01-en-m0-source-chronological.request.json": "ff9d3330d347c001e12c60c67d8f17b38b5bd9f74986bdb61994ec9db8007c5f", + "phase/calls/confirm-fresh-01-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "3c5f1b27300a0f595fc37b454a1889586f3c53df8294df83f475dcbe8bcfe2c5", + "phase/calls/confirm-fresh-01-en-m0-source-grouped_actor_prose_withheld.receipt.json": "2fe75fe5bdf91c230121aa4761a173ef542a17b3b7cdca6a61ec72cc46042817", + "phase/calls/confirm-fresh-01-en-m0-source-grouped_actor_prose_withheld.request.json": "4f7584f9765907277b07d64883dd6d2c648c0b5ae4366eb871a865cc6a7881ae", + "phase/calls/confirm-fresh-01-en-m0-source-grouped_lossless.dispatch.json": "c15b8901ff472cbb9804b3115ea11a933dc009da37e408a8631f0f3478f52213", + "phase/calls/confirm-fresh-01-en-m0-source-grouped_lossless.receipt.json": "df90b6a827150b63d29e67977c9cf1611563cdd2a0a17fd3ecafd1663cd7bc7b", + "phase/calls/confirm-fresh-01-en-m0-source-grouped_lossless.request.json": "f8dedf730ef2d6c32518dbf4ba913d00120611c34134562d3642dc3f888241bf", + "phase/calls/confirm-fresh-01-en-m1-source-chronological.dispatch.json": "f65a4a3f29270d150c6f9127bd37fc1194150da986d674845367c47d2db623e8", + "phase/calls/confirm-fresh-01-en-m1-source-chronological.receipt.json": "b8200d0282cdafe33bf3a421ae1e54a8fb5f5ebebc99ff73404cbc72e3eb387a", + "phase/calls/confirm-fresh-01-en-m1-source-chronological.request.json": "f56e0630e491e732a45f86a681eb84c9924ef887fc33c2b0c972a9e003679277", + "phase/calls/confirm-fresh-01-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "378ebbd7244c64857fb0b69964beac697a8a541b01ce269a7cc2bb95b0aa4598", + "phase/calls/confirm-fresh-01-en-m1-source-grouped_actor_prose_withheld.receipt.json": "7ac191ca0db13faf0d0143fd0b8b6dbb3a9f443f92700399d195631369188062", + "phase/calls/confirm-fresh-01-en-m1-source-grouped_actor_prose_withheld.request.json": "ec904746716597fbbd1d35bf323776d173cfb8d3a033d3339c7fea288f41c508", + "phase/calls/confirm-fresh-01-en-m1-source-grouped_lossless.dispatch.json": "a35cf52a5cd3375a2bca5c7a4c17048d0b3455038cff6c0d8e6e2d3280c64f72", + "phase/calls/confirm-fresh-01-en-m1-source-grouped_lossless.receipt.json": "03cd95b5b82e0ecb118371a7ef56d38cb00d479a2122daf9f98f92b2d88106d9", + "phase/calls/confirm-fresh-01-en-m1-source-grouped_lossless.request.json": "d7a809c8ceff8887d796749858050d8479400ce03cb4872aa5deaaa118fc5627", + "phase/calls/confirm-fresh-01-hi-m0-source-chronological.dispatch.json": "e04334ae57d4bf7b5a2b58445c4c524f14865e53a1b34ca9f900207681dea67e", + "phase/calls/confirm-fresh-01-hi-m0-source-chronological.receipt.json": "3995bb2064a4aa358c23bab18ca1710781db6c152092fd16affbb64776a89999", + "phase/calls/confirm-fresh-01-hi-m0-source-chronological.request.json": "0e2187c9b18de4655ec67b35be93d7149922121a135d4e253e399053eba56068", + "phase/calls/confirm-fresh-01-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "c3fe794b3f976c414fe6ccd029670d500533f4600471ff3ebf2d61dd8795fc6e", + "phase/calls/confirm-fresh-01-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "4cca9c2829875f7af02fd4d9e82611b83ce2175f8176342730eed1a72f585b43", + "phase/calls/confirm-fresh-01-hi-m0-source-grouped_actor_prose_withheld.request.json": "b30d939ee4aef6737f7a22eb97b7de08dcf7b78efb536bd0d81bfc480e54ae9e", + "phase/calls/confirm-fresh-01-hi-m0-source-grouped_lossless.dispatch.json": "be100945a9bd101ac34b7ce42027538b1f179641890b3508fd86e9387d20c4c5", + "phase/calls/confirm-fresh-01-hi-m0-source-grouped_lossless.receipt.json": "7577e6142ffb2214bc1ef7fd84600ceccd096dcf263f3c8ffa7db0fd9b6aca4c", + "phase/calls/confirm-fresh-01-hi-m0-source-grouped_lossless.request.json": "c6bae3f9ceb807f0a96c908077290816ce54e7628b71830dae895e8359ed084c", + "phase/calls/confirm-fresh-01-hi-m1-source-chronological.dispatch.json": "b833ec135cc4e92ffbd6e5c454410222406ac96887bd2c614007bdeaeae712e2", + "phase/calls/confirm-fresh-01-hi-m1-source-chronological.receipt.json": "b2f90accf55434d8042a8eca923af46603ca6de2158892fc8d8b3561158064e1", + "phase/calls/confirm-fresh-01-hi-m1-source-chronological.request.json": "a7b295f916fa2145d88028d7c696597b55bbc90457a4afe990fe163c65355038", + "phase/calls/confirm-fresh-01-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "6cdd7ee6dd82242d14f5385dcb8893c967092955c0586abdb8c78a7d7acf75bc", + "phase/calls/confirm-fresh-01-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "6dc97d5d2678fe9e99aaab86ce74c17e95bc1ef0dd10d9dbf8928b2a2a5ae79a", + "phase/calls/confirm-fresh-01-hi-m1-source-grouped_actor_prose_withheld.request.json": "d6fb51950e8636384acf3ed72d30fb3df38e8c1de1476aabbeeb74f3e2fcb473", + "phase/calls/confirm-fresh-01-hi-m1-source-grouped_lossless.dispatch.json": "9b5500e9673d34b8f30323d880c9ebae157561f11c80ba9eb7b4f73d6dd5f3aa", + "phase/calls/confirm-fresh-01-hi-m1-source-grouped_lossless.receipt.json": "7436ab906d1eb1c1d65d38fbedca5bde3e6980099fb224a6a2968314af27efd0", + "phase/calls/confirm-fresh-01-hi-m1-source-grouped_lossless.request.json": "e8631830b88553da7b19bd8e3f900ab55284d47e52daaaced049bf81807ad8d4", + "phase/calls/confirm-fresh-01-te-m0-source-chronological.dispatch.json": "d2cb5c1bb66dc5e5cfdb658618f5d95ff1a8ec103997d29d25f7ee9e86892c5c", + "phase/calls/confirm-fresh-01-te-m0-source-chronological.receipt.json": "fe84bc2edaddd6c518c73c3c111e36bb17e07b8ec9840abd7ee28da9e5c52437", + "phase/calls/confirm-fresh-01-te-m0-source-chronological.request.json": "035bb2a432195a8662e4e94c8ae7d6cceb3424f6bdabffd50ec31cc39d1eba5f", + "phase/calls/confirm-fresh-01-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "88147c5ed06c35f160899603e50187c9e7add657a140e4a035d44c94127a9a14", + "phase/calls/confirm-fresh-01-te-m0-source-grouped_actor_prose_withheld.receipt.json": "2f81f6db37855253b08d96e3ea9643315589dc5a2311a5bdff32e43a409a4599", + "phase/calls/confirm-fresh-01-te-m0-source-grouped_actor_prose_withheld.request.json": "371c284dfa3cef200ce89c4f50182ec6979caad1394b2747b269b7bf2a90574c", + "phase/calls/confirm-fresh-01-te-m0-source-grouped_lossless.dispatch.json": "e0be2f3a76aec73720f2f8f349eb212a6bb20f3eb97ccacc256ae462cd7693b6", + "phase/calls/confirm-fresh-01-te-m0-source-grouped_lossless.receipt.json": "1264d4310d6c4101220f9f97dcc6212bc5b0ecc8f9b3075f218d95ce125b2a45", + "phase/calls/confirm-fresh-01-te-m0-source-grouped_lossless.request.json": "5ecfb689d3896d3aa513d13890a3f76847487b6a7a09cc52ac4ac69fd38b50b8", + "phase/calls/confirm-fresh-01-te-m1-source-chronological.dispatch.json": "a7b5deb87415020573a6a3dceb615e5bfd08eb80c43b6ba0a2cc650a5c4e3964", + "phase/calls/confirm-fresh-01-te-m1-source-chronological.receipt.json": "ebea2befabafee0229f88bcd21eb6fac9781da79728dd920e57ed53031fc8a17", + "phase/calls/confirm-fresh-01-te-m1-source-chronological.request.json": "34f09f0b4d415873bdc72b29c9b172fcff9a4d298ba522b00ad7b5119e8e3eab", + "phase/calls/confirm-fresh-01-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "9c41985ebd5a3dac41beb7db1eec169c596d40640ea6861c4a9c77d8f5af01ef", + "phase/calls/confirm-fresh-01-te-m1-source-grouped_actor_prose_withheld.receipt.json": "55901241244ad0ce2d67a90152de5b66065806bf9c5f2d3a9f66f501aef5c51b", + "phase/calls/confirm-fresh-01-te-m1-source-grouped_actor_prose_withheld.request.json": "0764e76760d833c56c8d4a12052ef8b9e27b17f81af2e630e840a69a3ff1a341", + "phase/calls/confirm-fresh-01-te-m1-source-grouped_lossless.dispatch.json": "4bfeead8bec847dbcd1d749df4ab7e8e2ce86fcb618006666f8b973d0660727b", + "phase/calls/confirm-fresh-01-te-m1-source-grouped_lossless.receipt.json": "4c2cc92ad07eb8e5ab29e385787b381957ca6daa094f277f69299b5fff9300fc", + "phase/calls/confirm-fresh-01-te-m1-source-grouped_lossless.request.json": "96b159c6d87981071200982bf1a2a5beac1467a663ff91492a3ef9381d0f6c54", + "phase/calls/confirm-fresh-02-en-m0-source-chronological.dispatch.json": "5f16871a41271b66348b6deeb5f10ad7ab6c3628c1facb5fcbbff40dac72e0d1", + "phase/calls/confirm-fresh-02-en-m0-source-chronological.receipt.json": "cc963af7bee60a24402dcc06ab7061d935ff84783337fc3004d8710a18992fc5", + "phase/calls/confirm-fresh-02-en-m0-source-chronological.request.json": "4ffea1578924eab930462e302aa1656d09029eb4f028cdb02ad43af95f6b40cf", + "phase/calls/confirm-fresh-02-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "6ee121c7f81b779d50ae8b5030ae9f638c01a83cd32735fe56915bdad579dd29", + "phase/calls/confirm-fresh-02-en-m0-source-grouped_actor_prose_withheld.receipt.json": "c503edd5fbb1cec1906f155eb9640fc98a5f8c93fd5383540707273849936b81", + "phase/calls/confirm-fresh-02-en-m0-source-grouped_actor_prose_withheld.request.json": "3ec47e038452cef93af790e9efe0a915d8856d23d84f8a5ca406fbecf9b30d11", + "phase/calls/confirm-fresh-02-en-m0-source-grouped_lossless.dispatch.json": "2468bd26be887a5ed8b4501e0fcbce27f45e44a39706ea68a79ee756b0cce8a0", + "phase/calls/confirm-fresh-02-en-m0-source-grouped_lossless.receipt.json": "0a004b6c31347b20d1805ea1d05e1c70ebdbe1fb074b66cd933439d9bbc2d096", + "phase/calls/confirm-fresh-02-en-m0-source-grouped_lossless.request.json": "0e0592f36caa70034c2bc2114c4d53677b8e73263d13c43c8f11b3a880e6044c", + "phase/calls/confirm-fresh-02-en-m1-source-chronological.dispatch.json": "703838ecf4d7f15dbd2b6a7c55b6b7e99f9b0b1c28a3a1c2a40f580d1c22ffa3", + "phase/calls/confirm-fresh-02-en-m1-source-chronological.receipt.json": "5ce89cfcba4c5e65f5fd75424b69eaaee1aff1e3acbfe0af22241b436d7f2d47", + "phase/calls/confirm-fresh-02-en-m1-source-chronological.request.json": "ebaee38027023ecc7d7418860b4c9e7035066526b1c44abe806831f59d726d01", + "phase/calls/confirm-fresh-02-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "1085bddc4e1c75c3ce9265f13118f7cc197298f820240f3722238c914e25edce", + "phase/calls/confirm-fresh-02-en-m1-source-grouped_actor_prose_withheld.receipt.json": "f1d9838af2a4e8e7881c78b57d859d90fddcd1efe0e633d6797a09ed6725ee14", + "phase/calls/confirm-fresh-02-en-m1-source-grouped_actor_prose_withheld.request.json": "c7bf2feb0f644fdc343f947cf146e95d85b691d5a843feeee68373372546d678", + "phase/calls/confirm-fresh-02-en-m1-source-grouped_lossless.dispatch.json": "c6b7c19525a794536ac718003a91384c55efbdb7bdb350890b7e08ca6f0ff17a", + "phase/calls/confirm-fresh-02-en-m1-source-grouped_lossless.receipt.json": "c6d9199f6291ccf42777f791d6d7e2ac3748d215dbbe56ea443834ace1309d06", + "phase/calls/confirm-fresh-02-en-m1-source-grouped_lossless.request.json": "f0cb8b767c6fdc515575cb9aaa241b7bda857c30d69c8745677b268d661e9775", + "phase/calls/confirm-fresh-02-hi-m0-source-chronological.dispatch.json": "a9a4f45550d8c280e7042bca5666c700986e38df9d4e880910f52911580477c0", + "phase/calls/confirm-fresh-02-hi-m0-source-chronological.receipt.json": "266547005af811ec916ba007e5b654e9818f5d0c1c2aa05d9de30a5b06f86d8e", + "phase/calls/confirm-fresh-02-hi-m0-source-chronological.request.json": "ce10f3815e6e0013e130534b11f793ccd391fe7568915471a120a117abf019bb", + "phase/calls/confirm-fresh-02-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "0ce274fde1abe92895dcebefee09743aa223c1e5f6e1e6ce3a0fc77df2374693", + "phase/calls/confirm-fresh-02-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "29bd63121012854f63b41f0a94f81321b53e7220fc2c617ce980b5b8e4744c66", + "phase/calls/confirm-fresh-02-hi-m0-source-grouped_actor_prose_withheld.request.json": "5fcec59e0ec2dea408eca84dd6478ea90a276102df52857fe03c91bb38bee003", + "phase/calls/confirm-fresh-02-hi-m0-source-grouped_lossless.dispatch.json": "62b79de43cb94072f595296757e620c0968a57418dcb77788b348e1a34abf896", + "phase/calls/confirm-fresh-02-hi-m0-source-grouped_lossless.receipt.json": "f0493b32f6c96a27df3eb930687569a15e6a816e1c8680fa01e33f1eab87adec", + "phase/calls/confirm-fresh-02-hi-m0-source-grouped_lossless.request.json": "45b30010196c28a9bb79dec462c02a3d57dcd3b9395e9326a54c815bfec68129", + "phase/calls/confirm-fresh-02-hi-m1-source-chronological.dispatch.json": "8bc13537b6791827de7859d4ba57fe0eeafaa383861e52826e2c158ba8a1f40d", + "phase/calls/confirm-fresh-02-hi-m1-source-chronological.receipt.json": "0e5e9b24e661e094d17f62eaba7550d64d310fd269d1ab392a350656319ff151", + "phase/calls/confirm-fresh-02-hi-m1-source-chronological.request.json": "0ad90e02f45c5b2ae488ab8e8f2573c2657a8c74b8db2209847926560d552e1c", + "phase/calls/confirm-fresh-02-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "7ceee9ba654ed8acdc280eb522b5974301aed14c2eb16eaef01aaedf32a02fa9", + "phase/calls/confirm-fresh-02-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "52ebdc021c4a43e99b670f99653d03471095f70f6b07c8ffdf97791e66a00f06", + "phase/calls/confirm-fresh-02-hi-m1-source-grouped_actor_prose_withheld.request.json": "9d4434b68d64dbd58e61c4f5a84050268b28eab6cc77c6760974faf1296baf54", + "phase/calls/confirm-fresh-02-hi-m1-source-grouped_lossless.dispatch.json": "27e5ffd24746b0d428ad5451d85384edf02bffaa3ae5bc80510114734b6257d6", + "phase/calls/confirm-fresh-02-hi-m1-source-grouped_lossless.receipt.json": "5753e8c6b77fc1a7960559f04b23daab9a7a5185e1a2b875ad2f100db0acc11a", + "phase/calls/confirm-fresh-02-hi-m1-source-grouped_lossless.request.json": "f9b18b67e3d156753eabfd7302282d11649c86c8d7b93e5b5920a7b10ea5ad94", + "phase/calls/confirm-fresh-02-te-m0-source-chronological.dispatch.json": "629a3dae86416ae10ecf1cabc6a60970949829a729b6ed61fac1297568339645", + "phase/calls/confirm-fresh-02-te-m0-source-chronological.receipt.json": "90c10c6d8ac52e28ce6fa0f7c5351f006eb11f5de75bf38be0aaad11aabced7c", + "phase/calls/confirm-fresh-02-te-m0-source-chronological.request.json": "31d01baedb8c8573663753b8c98a49909f1f4ec37c9ff17b711c61f704e36ab4", + "phase/calls/confirm-fresh-02-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "db03a5b0be062c9569f5fc3dceeef1cecd5ec6901301e27c7d264f2b9b069b0a", + "phase/calls/confirm-fresh-02-te-m0-source-grouped_actor_prose_withheld.receipt.json": "d8d70c13849235a0bc544b257f1518dfdebd8c39c52440b86b6600dda410c557", + "phase/calls/confirm-fresh-02-te-m0-source-grouped_actor_prose_withheld.request.json": "0ccfda56cc9f0663b129bfa9fe0911ee52dbf0ca5f32185a439cb7120e7cf18c", + "phase/calls/confirm-fresh-02-te-m0-source-grouped_lossless.dispatch.json": "a31548030acd6ed18f8f6327188f68e493cc92f5dd1990acbbba86ad6cc71a3d", + "phase/calls/confirm-fresh-02-te-m0-source-grouped_lossless.receipt.json": "618d690c7739dce39a5e3a67b5f2c401278d6e52eaf94de0bc5a8a0034152085", + "phase/calls/confirm-fresh-02-te-m0-source-grouped_lossless.request.json": "93f0e353a7ed81d0f07ff3a91863c14b97fc6a9842c36e373ab89625a68847b1", + "phase/calls/confirm-fresh-02-te-m1-source-chronological.dispatch.json": "95051da67bd049058d6ad74d581d6ddf8b571c32a081e69ae939aba72dda227b", + "phase/calls/confirm-fresh-02-te-m1-source-chronological.receipt.json": "d8e9f371fd0bddc410d9ce34a1449541dc1e91734c5ef4ead8cad1c189210e4a", + "phase/calls/confirm-fresh-02-te-m1-source-chronological.request.json": "7711e585f8a3329828c69a99d52b7465ef326a21dbdaf6871d775d509d7e4cb7", + "phase/calls/confirm-fresh-02-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "50b9de02ff4395cff5dbebd3af0883ef1aef4bcc026a84d0d64464ae4e60dd09", + "phase/calls/confirm-fresh-02-te-m1-source-grouped_actor_prose_withheld.receipt.json": "223c0cea3525648d8141110d9740ba86b0378ea9207827f2001666f77e4e90b2", + "phase/calls/confirm-fresh-02-te-m1-source-grouped_actor_prose_withheld.request.json": "aab7958ab465115ebbe7343a1d07725ba56ac0daf0cb430d49603aa76bc3ec5e", + "phase/calls/confirm-fresh-02-te-m1-source-grouped_lossless.dispatch.json": "29bafedb53ec948349c827b0ce28db4ee2b760e219f4ca531e528e32b4a600ee", + "phase/calls/confirm-fresh-02-te-m1-source-grouped_lossless.receipt.json": "3e69b5c05d23bfa95f484c0b2f77dbead788e1f5a2098c33cb440a388b80642a", + "phase/calls/confirm-fresh-02-te-m1-source-grouped_lossless.request.json": "45d94b3fb23278a32f8028ce636191ee3b8d617f79946b9bcc43624088c6ea06", + "phase/calls/confirm-fresh-03-en-m0-source-chronological.dispatch.json": "748e4f2fae752a00b5d4e1c198a1b9ee47613dd228557b014cda743f1621bff9", + "phase/calls/confirm-fresh-03-en-m0-source-chronological.receipt.json": "c4b5ac257c51e98760b9dabf5d5ddcb6b4c69ee9aa01c279b932652899ed3dc4", + "phase/calls/confirm-fresh-03-en-m0-source-chronological.request.json": "6d4a921094c0c1d52534187876587d6c05d2e9e068e8d413cac4823e497ead1a", + "phase/calls/confirm-fresh-03-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "611e9775649ae2d41fea5535c5626bb47969dced475eb40a7d02f926a7c6f23c", + "phase/calls/confirm-fresh-03-en-m0-source-grouped_actor_prose_withheld.receipt.json": "d72f1a20cea9a782813aee82902331cc1f335ffcce05616e967efcc5e0a24788", + "phase/calls/confirm-fresh-03-en-m0-source-grouped_actor_prose_withheld.request.json": "4e0dfeb709f29df873e7fa5950ee14c39d76b82b2a780e155968773a9aba9916", + "phase/calls/confirm-fresh-03-en-m0-source-grouped_lossless.dispatch.json": "3e22c905ea879904234484d271124d20974c34981291d03be27e456ce4295c62", + "phase/calls/confirm-fresh-03-en-m0-source-grouped_lossless.receipt.json": "7b6fb05e8e9febfe0fdf48c2058beda8f6e49b5f7c679475558a61de532681bb", + "phase/calls/confirm-fresh-03-en-m0-source-grouped_lossless.request.json": "474fe50d11f26462a237108f88e01a9890278db5dc2c41389cfff19653b31715", + "phase/calls/confirm-fresh-03-en-m1-source-chronological.dispatch.json": "47355f0ad56794f2c9b81312e17a0bb0b23be985cb6df73737e614e8f83a38ff", + "phase/calls/confirm-fresh-03-en-m1-source-chronological.receipt.json": "703dff5b374b73ba1cd17a6cb8d3d11d8cac38585f388e0bb9991ab3fc728ddc", + "phase/calls/confirm-fresh-03-en-m1-source-chronological.request.json": "c73250242aa32b7bd1dd9fea122dee1e29eab01a56d419519bf0f0d06b3f5f70", + "phase/calls/confirm-fresh-03-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "d5ef59950aa016fb7e61c9b7cc5af366f573f95f06886395dc4a9e65d257503d", + "phase/calls/confirm-fresh-03-en-m1-source-grouped_actor_prose_withheld.receipt.json": "ab2bcc39f735cf34fd3d1237b138235b1d36923ab203fe0a679be630968c44f4", + "phase/calls/confirm-fresh-03-en-m1-source-grouped_actor_prose_withheld.request.json": "59744cf5fc73cf19e6897185f44e860def19a4041fa76ab80c5bfdd648891371", + "phase/calls/confirm-fresh-03-en-m1-source-grouped_lossless.dispatch.json": "d109a845207d3cc99d3cafb42ea7e94ac0fad1618d18735c5cfe3b025381829d", + "phase/calls/confirm-fresh-03-en-m1-source-grouped_lossless.receipt.json": "eb349b3d06070eba03a3775a169ab81e346f8427308477c2e13998bd7d01c76c", + "phase/calls/confirm-fresh-03-en-m1-source-grouped_lossless.request.json": "cd41cda1837648ac7c94de2ec313ce246073746ebde686ea49bfda9492bba8f8", + "phase/calls/confirm-fresh-03-hi-m0-source-chronological.dispatch.json": "67c145aa8c6cfcec591416616268281ac980c4f475d1f4fd09889e5855eb8fca", + "phase/calls/confirm-fresh-03-hi-m0-source-chronological.receipt.json": "30237a912ca81319524b3997a4677cb243b274873e4bebf8d66d9af37187d330", + "phase/calls/confirm-fresh-03-hi-m0-source-chronological.request.json": "d3723a84f21d2ceefcdeca29d89c69dd308eb88c6580b57940a4b2bc239cc6ef", + "phase/calls/confirm-fresh-03-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "7118536212dce967774d91c98564bffddde69b781b5d671fda074d60376c24c3", + "phase/calls/confirm-fresh-03-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "1da5b42094a33b4344c89d7dbadc569168343a4b3c47f121d036b48581478cae", + "phase/calls/confirm-fresh-03-hi-m0-source-grouped_actor_prose_withheld.request.json": "4b94a87ec47f84f02aed09b537fb4496e365643f19e4b775659d483e4f42ff4b", + "phase/calls/confirm-fresh-03-hi-m0-source-grouped_lossless.dispatch.json": "93102b514be7f5fe2a5d0c576c2862d5623f5d3241ff499d893524f9f7829d77", + "phase/calls/confirm-fresh-03-hi-m0-source-grouped_lossless.receipt.json": "3642390c124473554bc127846bcf0d8afa5ae494394d0dee52e9b321fd91ede2", + "phase/calls/confirm-fresh-03-hi-m0-source-grouped_lossless.request.json": "642f01620849eba24da85ccb0e191bff6392afe8945b8ae88e5c87a5d2665a40", + "phase/calls/confirm-fresh-03-hi-m1-source-chronological.dispatch.json": "9b7361dcec43f17c4f5840abc19818eeb7a7ec291f9c2dd4fdef0381ecd89831", + "phase/calls/confirm-fresh-03-hi-m1-source-chronological.receipt.json": "49e818d043476d85919978636cdfa78a531e46ef50c1240c2ef8ff0a75d95c3a", + "phase/calls/confirm-fresh-03-hi-m1-source-chronological.request.json": "feb6a376a3f319afcfe2611199fe9e39ef1ef275d42e5db85f942b9672f74277", + "phase/calls/confirm-fresh-03-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "99d5475b4ee7db69282550ea81237c3522e535e783893b13654e46bc46a20489", + "phase/calls/confirm-fresh-03-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "1fa9f9f65931ac5cd60b0e502b97afddbd78993b5128a987232f9964707659d1", + "phase/calls/confirm-fresh-03-hi-m1-source-grouped_actor_prose_withheld.request.json": "81b1eff3767f3255b4b1b674209ee8bd4f1b6817de7575b89b2823f84051949e", + "phase/calls/confirm-fresh-03-hi-m1-source-grouped_lossless.dispatch.json": "646dae2500dc3867936dccf26506f5a7b64d906eadd9a25b9485acbd3a2b8a50", + "phase/calls/confirm-fresh-03-hi-m1-source-grouped_lossless.receipt.json": "9033191405fb4151a4efe5eda4350466c20e13aede338fe5c388f92fc6af7a85", + "phase/calls/confirm-fresh-03-hi-m1-source-grouped_lossless.request.json": "71dd9b4b64d8b3b18be5b8abbfbc7c22abafe1e38e8c73812b7c9a6b6191b62a", + "phase/calls/confirm-fresh-03-te-m0-source-chronological.dispatch.json": "ad9b7de33027a552fb870dba36f53e6f16f8516bb23344eb21b672321e6e653f", + "phase/calls/confirm-fresh-03-te-m0-source-chronological.receipt.json": "38d69b49ddd5e79fd2a7ed7516f54864742d6540b61869ba7a5f3b969d849bc0", + "phase/calls/confirm-fresh-03-te-m0-source-chronological.request.json": "f1ebacd5e4fb559f814fbba89b0d0f7f0bcae943d8defc1771507bf386c5a30e", + "phase/calls/confirm-fresh-03-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "3243d4b6900ebd890455ac3e6e0644026075ac503f9c88df8c7ef5fa5dc427f5", + "phase/calls/confirm-fresh-03-te-m0-source-grouped_actor_prose_withheld.receipt.json": "5659f37eebacf5ea81603b06c833de4fa829a726dd0173497662312419c9d4a9", + "phase/calls/confirm-fresh-03-te-m0-source-grouped_actor_prose_withheld.request.json": "b6fc6d950014e600ec31ec222146636315e9c695da3d0710915e30f24544a3ed", + "phase/calls/confirm-fresh-03-te-m0-source-grouped_lossless.dispatch.json": "335dd6235ba554cbf7141ec21045aa89764aaaca12140069c185884914eef210", + "phase/calls/confirm-fresh-03-te-m0-source-grouped_lossless.receipt.json": "15e66f41ed268302d1c65001fc7eb49e205d84453232731b1c3e963bf0731676", + "phase/calls/confirm-fresh-03-te-m0-source-grouped_lossless.request.json": "3dca5b45ee484136f1f78185014a1f375f00ee33e19cb456d2417c35bb91b3c8", + "phase/calls/confirm-fresh-03-te-m1-source-chronological.dispatch.json": "ab126d54c352049538d4ec1ec2a699610d2cdc6a7fde0a459a63f9db7ad78604", + "phase/calls/confirm-fresh-03-te-m1-source-chronological.receipt.json": "f8acad611231f9b0734f609fd49b54f8cf8d6a4896b837262d5c9406ea7ca242", + "phase/calls/confirm-fresh-03-te-m1-source-chronological.request.json": "92939d748f3f391bcdf63d413ce576640bd354c9ab533074b11fe25652a474a3", + "phase/calls/confirm-fresh-03-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "9025901668ac3755a8d721697b824ead71437613ccc63a59d5bda53c76c0d002", + "phase/calls/confirm-fresh-03-te-m1-source-grouped_actor_prose_withheld.receipt.json": "d44ff7ad7d44838b804fee20baa3580e6bd64d221122102a49de4255232626de", + "phase/calls/confirm-fresh-03-te-m1-source-grouped_actor_prose_withheld.request.json": "14b68e7020f1c7f309100217cdf0dc6593486375a49c1d924a23849d2519bef9", + "phase/calls/confirm-fresh-03-te-m1-source-grouped_lossless.dispatch.json": "9de7ff7e10ba91c86dbc4d075e959ac66d13d14a7ed121d77c3968bb00a99fee", + "phase/calls/confirm-fresh-03-te-m1-source-grouped_lossless.receipt.json": "76c5be3a35f047196810db7ba5daccd63f9d4535f3442bc6171fe18140eba94a", + "phase/calls/confirm-fresh-03-te-m1-source-grouped_lossless.request.json": "4a05f0086c18c1432b5ef13deb61f10198c1feba83157e9132610b7cad1dc9e4", + "phase/calls/confirm-fresh-04-en-m0-source-chronological.dispatch.json": "9e13314b201c347fa3cd819712ad4243ed55735e63a77fb8a1aaf5c162993d82", + "phase/calls/confirm-fresh-04-en-m0-source-chronological.receipt.json": "af126488ee96cbf478c662d05e63e4177a658296bf12eb89fa2e0ac23f71776a", + "phase/calls/confirm-fresh-04-en-m0-source-chronological.request.json": "e8b74cb9af2e31685df83cecb36906643a56a5fd3f5ddfa1990c617ebdcded85", + "phase/calls/confirm-fresh-04-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "f75b410105f91e3a2cf0f4496e5dfd0cae15b47eab6b788feb79dac716c98c96", + "phase/calls/confirm-fresh-04-en-m0-source-grouped_actor_prose_withheld.receipt.json": "5e535665c07c64bcaf1ab7a08a8ecd2e40677e3e1cce903469db33c3b7edc382", + "phase/calls/confirm-fresh-04-en-m0-source-grouped_actor_prose_withheld.request.json": "7bfc34dd08bd92f5c9547b6a6a448d58823a1b7640f20d23c68d4c9610964e8a", + "phase/calls/confirm-fresh-04-en-m0-source-grouped_lossless.dispatch.json": "669532dbd593f3cadd64a9ae6484a0ec97bc6bac7a20289f93cd8bd2a85d316f", + "phase/calls/confirm-fresh-04-en-m0-source-grouped_lossless.receipt.json": "70b413e29dacedb5ea9f4e08a72d8eba61e3ee7471746b4557dd132e648208d8", + "phase/calls/confirm-fresh-04-en-m0-source-grouped_lossless.request.json": "bce4fc5a03af6f51665bdf59993ef95dad845853c5546910046042079a7bba5a", + "phase/calls/confirm-fresh-04-en-m1-source-chronological.dispatch.json": "69b129945d071d812e9ac2bb6b2e70c2abf5f17ca7ffb8955f3d50b02182238c", + "phase/calls/confirm-fresh-04-en-m1-source-chronological.receipt.json": "036d5af97b06dd8c2ced97959254c5d73df6f779456e85c2826c768fd05d82c9", + "phase/calls/confirm-fresh-04-en-m1-source-chronological.request.json": "2a06275c07453afe2c141eb3d1806e4952623fed04c69e3575c5e9440ba51fb8", + "phase/calls/confirm-fresh-04-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "2c1eb417beed2f5c8325b31b0902d6694ecb08b2e0e06c85b575851288f3a06d", + "phase/calls/confirm-fresh-04-en-m1-source-grouped_actor_prose_withheld.receipt.json": "9427adfccd5d34dd66cba3059394be250cc3fcfc8e636084c325a9ec0c104995", + "phase/calls/confirm-fresh-04-en-m1-source-grouped_actor_prose_withheld.request.json": "c88e6530b5e31682fdc637b69795772ead271dc11761218e12a0c7650a8a91d2", + "phase/calls/confirm-fresh-04-en-m1-source-grouped_lossless.dispatch.json": "5d6827d43d20d5b6367020e6bf40c7a855899fe6236a949dc873567201eef367", + "phase/calls/confirm-fresh-04-en-m1-source-grouped_lossless.receipt.json": "70d4e3377fadb4ab5be641f71f7dc16cd3c9d1a045826a942703dc43b7e870a6", + "phase/calls/confirm-fresh-04-en-m1-source-grouped_lossless.request.json": "cf6b8ba2ee944ee3b19e0936cfee319080d63b207ca14d423cc03a9b469ac17d", + "phase/calls/confirm-fresh-04-hi-m0-source-chronological.dispatch.json": "f1debeab907b5fb98430f25e18e3c2f4ac0ce75e6b5596e3467bcf54e2781c8e", + "phase/calls/confirm-fresh-04-hi-m0-source-chronological.receipt.json": "d146bd2bec0acbce7c8af7ceb485e6ce0839244e30ee67f29e30eea8af762ca8", + "phase/calls/confirm-fresh-04-hi-m0-source-chronological.request.json": "f977b13064e86af47431e02d48cf37f774ee06f06c28f4475cac41f1be261984", + "phase/calls/confirm-fresh-04-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "d107d6153a369a5db180d77816eddd96cd8f1e96a4d45942cce97659aa70f17b", + "phase/calls/confirm-fresh-04-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "73f06f986d17790fdd7ee0961e430a961a95c5f381a8b073e290dbd6a1d5b517", + "phase/calls/confirm-fresh-04-hi-m0-source-grouped_actor_prose_withheld.request.json": "da48c46b7230bacd605451f6bca05e07b877c2461c151137aead4ae20acd28bf", + "phase/calls/confirm-fresh-04-hi-m0-source-grouped_lossless.dispatch.json": "f5b58367e646b1d6d519770d0cd206bc813f74a21e34abd00fc957559e523be3", + "phase/calls/confirm-fresh-04-hi-m0-source-grouped_lossless.receipt.json": "55dbbd9b00d4fad2a0880c55423f5480a98f2720dfdfd8c9792cb813d8871054", + "phase/calls/confirm-fresh-04-hi-m0-source-grouped_lossless.request.json": "4b73bba12b8122d72204b7c93bb565753bb77aba8489df31f0ed84d619efe122", + "phase/calls/confirm-fresh-04-hi-m1-source-chronological.dispatch.json": "4622c83a6b7cdd478d381bf4c163715a446fcc52ed20b41b226e5ac078a1326e", + "phase/calls/confirm-fresh-04-hi-m1-source-chronological.receipt.json": "687e0e5482a9cd29a9d4c55a7d8f4cdea205dd73cad3e2c7805afae89974b1b7", + "phase/calls/confirm-fresh-04-hi-m1-source-chronological.request.json": "dd232f1b4a130adab090559bc5238f38b3883efd2a586610a03535f1eb6d1eb8", + "phase/calls/confirm-fresh-04-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "8fcca392a50fb73ef7a87bba9ea64dfb9fa8b1b8f4a10013d63da3ff85808f55", + "phase/calls/confirm-fresh-04-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "04caee20ac1fd1ea1dfcb5de8d4b8c250634601a3fb708590592e42327697928", + "phase/calls/confirm-fresh-04-hi-m1-source-grouped_actor_prose_withheld.request.json": "c920164fbf4724486cca2fd18cb652048deaaeead0bb15577c8bfd207f1a0400", + "phase/calls/confirm-fresh-04-hi-m1-source-grouped_lossless.dispatch.json": "ae611c9ad0e5ddd31a3a854943ff82aca102f9a70067f952ff6ff572f13ee8e0", + "phase/calls/confirm-fresh-04-hi-m1-source-grouped_lossless.receipt.json": "be740846558270adbc7cefda850ba430ea16f0de76644215962c6e2776ae7abb", + "phase/calls/confirm-fresh-04-hi-m1-source-grouped_lossless.request.json": "987a952126a1381a84c6edb4346229bea3791b4f2ed91ca89f088d4f57b7e809", + "phase/calls/confirm-fresh-04-te-m0-source-chronological.dispatch.json": "765ad1651a49f943bd53e3427f49b94ea43873fe7ee6dd698c61db985b7d397b", + "phase/calls/confirm-fresh-04-te-m0-source-chronological.receipt.json": "1b690315739101ff4b135a25835290bdda0b252985d0bdaf690f21bae70b8732", + "phase/calls/confirm-fresh-04-te-m0-source-chronological.request.json": "bfd325125e03f0d4317e0d2d6f1a24df43ac386e85174919024e2e94e2ec4da9", + "phase/calls/confirm-fresh-04-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "d5da8223610ad74c7b0975014d0a1531330e994f2f4f950e7c9c11e6103a1dad", + "phase/calls/confirm-fresh-04-te-m0-source-grouped_actor_prose_withheld.receipt.json": "25d8b6c926f973c43c35ef359bc2bcc9b568e52df47980beb2aced07178aa902", + "phase/calls/confirm-fresh-04-te-m0-source-grouped_actor_prose_withheld.request.json": "415a4aec8822652825a078a4f0c5262bf0ec8e6fe528738471d2c4ffb02fcbdf", + "phase/calls/confirm-fresh-04-te-m0-source-grouped_lossless.dispatch.json": "fa59f45a6ce32ece4073398ff8586f1e8c470695e67922430508bcadd2c10964", + "phase/calls/confirm-fresh-04-te-m0-source-grouped_lossless.receipt.json": "0517ce2304d3978320179d80a60c9d999345a79ede41c1de11593371b0be4b89", + "phase/calls/confirm-fresh-04-te-m0-source-grouped_lossless.request.json": "95ad29614c6e11562cd2ed0645f6961a774e0c7db1851539f56780af34d6fe29", + "phase/calls/confirm-fresh-04-te-m1-source-chronological.dispatch.json": "ff0c966b33f0bbf0dc57925d9411fc77a75c5eeb010f7e7a73bdd1f33d32301b", + "phase/calls/confirm-fresh-04-te-m1-source-chronological.receipt.json": "f76769dbc5228e3dc0d0fa9327ea09b6ab821c39ae59284478bbab0f400a493f", + "phase/calls/confirm-fresh-04-te-m1-source-chronological.request.json": "0564984f511bbd1df0725acedd46fcf29b8649bf180af8120aa86c43eba08c21", + "phase/calls/confirm-fresh-04-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "9b9f46d92dfbb8fe4dc55c0f6a069cd68a92fc8ac5a2854043c63412363a2d45", + "phase/calls/confirm-fresh-04-te-m1-source-grouped_actor_prose_withheld.receipt.json": "6ce4df1fe8d9919c4be9665d400ababcf9de83249dd9f8e415d28130241da101", + "phase/calls/confirm-fresh-04-te-m1-source-grouped_actor_prose_withheld.request.json": "d1545242d8643d5626cd8a9534a06d6015d0b3d86da77b2ce18e93047f001351", + "phase/calls/confirm-fresh-04-te-m1-source-grouped_lossless.dispatch.json": "052cfb2013d943e4c850973f201585eaf855665a6bf039f72bf3f0cd42af63bd", + "phase/calls/confirm-fresh-04-te-m1-source-grouped_lossless.receipt.json": "16c8f3d15043b905b22f440df3746ffe046288cd2fdc5efbbfea0902555778a1", + "phase/calls/confirm-fresh-04-te-m1-source-grouped_lossless.request.json": "388bef7b13a15869f0b0ac52d0166eeb3d4be1e8654e7b53b4561b49598abb78", + "phase/calls/confirm-fresh-05-en-m0-source-chronological.dispatch.json": "707997b65c4753a5fdf8e384044b22ecec375cc5d293f894070005232a0fc360", + "phase/calls/confirm-fresh-05-en-m0-source-chronological.receipt.json": "02f908bc9929e7d5d97b88f97a9afe12c4e652378868be09ccb442a3d04c4b70", + "phase/calls/confirm-fresh-05-en-m0-source-chronological.request.json": "93ed370998c25139593fe9745b3e933e7ec528437676549383d5c1b1153ad4f5", + "phase/calls/confirm-fresh-05-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "22e929d66cbfee1d9d4c815e54ed89d72f8f0e88e92c8b977da01de9116e8169", + "phase/calls/confirm-fresh-05-en-m0-source-grouped_actor_prose_withheld.receipt.json": "3ee6649307c91340d45465e0caeb7287f65f7edff392e4144e3654b59a0af52b", + "phase/calls/confirm-fresh-05-en-m0-source-grouped_actor_prose_withheld.request.json": "688432937f19c9399e4ad26aa82b0b636892d506cfd54017ea91e94b827eecb3", + "phase/calls/confirm-fresh-05-en-m0-source-grouped_lossless.dispatch.json": "bf5988ff35ae44f1b623346dc8b1d1a11bc27fa1190d8306b92c868c7f201ab4", + "phase/calls/confirm-fresh-05-en-m0-source-grouped_lossless.receipt.json": "2d2397ca11d62626d01f2301ee63fab94b0b654d4065e53e1c13a4e34a62cf28", + "phase/calls/confirm-fresh-05-en-m0-source-grouped_lossless.request.json": "c2850407d6b1db55b84b0f21871c771d1b9c6364e3474b4269f3bcd4385129aa", + "phase/calls/confirm-fresh-05-en-m1-source-chronological.dispatch.json": "f4ec7b416f9b83120f52d4cd76bbfaf7a27e99bd6dbc12977408964e5a0a1d29", + "phase/calls/confirm-fresh-05-en-m1-source-chronological.receipt.json": "a1a12c0545927f8beb602026c19807222e1f23fefbf7188a0abcd473a511d9a5", + "phase/calls/confirm-fresh-05-en-m1-source-chronological.request.json": "ff408bb876473dc8e75f8ed08f53975d3bedb4c10d89cf98c22eda930e108fcc", + "phase/calls/confirm-fresh-05-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "b21be5e55c15eef7f46ba54da6ae9bbd7f3024eec60f42bf4f10cbaff55ba4c0", + "phase/calls/confirm-fresh-05-en-m1-source-grouped_actor_prose_withheld.receipt.json": "d1b04e06033ea4a9ab846f965c40781ecb50cc6e6c1468bbbf1a1c543d13cf56", + "phase/calls/confirm-fresh-05-en-m1-source-grouped_actor_prose_withheld.request.json": "832b5e53207e02a7367263a3b9bda7ee6369b305fabc27b2a96bccbca1a0f2d4", + "phase/calls/confirm-fresh-05-en-m1-source-grouped_lossless.dispatch.json": "92b21e27d2207a23ff143b36a9449709cae4dc948ca095b012e267503e3cd636", + "phase/calls/confirm-fresh-05-en-m1-source-grouped_lossless.receipt.json": "bc6e5a074a5d364ec336538880935d5f1213a246984c0cccbb591092a13de161", + "phase/calls/confirm-fresh-05-en-m1-source-grouped_lossless.request.json": "67eaf83348fc31a747302ba9c69d923e42e1de606d84c658809989d9c001ff33", + "phase/calls/confirm-fresh-05-hi-m0-source-chronological.dispatch.json": "64177447e83ae00864f2f16956a9cec6e4027ecfc2590de9419e98cacfd59aef", + "phase/calls/confirm-fresh-05-hi-m0-source-chronological.receipt.json": "a6b9b29ba174c89dea534b4565f388872502b15b9d50a362c4166ef403e25bbc", + "phase/calls/confirm-fresh-05-hi-m0-source-chronological.request.json": "dfca39bea1a57bfaf30bab9a0099ec668a7d52c2b47f9cb7c8b97d3a777ec766", + "phase/calls/confirm-fresh-05-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "28dbb62a9a65fd6d1ee6bc41f5f09c8b90e71495b6a27b7c96b16dd9e677ef1f", + "phase/calls/confirm-fresh-05-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "a9b0f79b01ff9f0b97c10e9c5b5c7bbe2031e031524fd443328a0e1cdab04651", + "phase/calls/confirm-fresh-05-hi-m0-source-grouped_actor_prose_withheld.request.json": "dd7f12aa7913c37ca6ef46e9a399ff1e85722434f80cb89155cc950ea691b2db", + "phase/calls/confirm-fresh-05-hi-m0-source-grouped_lossless.dispatch.json": "46d8652ce0390e25611f530b39abbbacd3213565122a5959f4924954d48e93e7", + "phase/calls/confirm-fresh-05-hi-m0-source-grouped_lossless.receipt.json": "9668f4e92080e6e8f4f63c1e0147b919972005e65be1fadbee088f5e6b3cddc2", + "phase/calls/confirm-fresh-05-hi-m0-source-grouped_lossless.request.json": "28163c570239f723d91a8af3b4d3b77981f90366d569c5574dde177af7f8d187", + "phase/calls/confirm-fresh-05-hi-m1-source-chronological.dispatch.json": "7b1cff42161625cae923c124f08116c9c9d29f2ed76cf0ebd491c47bbe03f7f2", + "phase/calls/confirm-fresh-05-hi-m1-source-chronological.receipt.json": "caf043a83fa46f9f72e3c5e83f1c9cca16b2ca6c4cf13559497bd8322a56322c", + "phase/calls/confirm-fresh-05-hi-m1-source-chronological.request.json": "a7434d6a57a494f4f76a40ea35d4b37bb14f2506eb3e294c860d9cd50627870b", + "phase/calls/confirm-fresh-05-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "22be3187a065f90e8fdef934dba7ac96e6b7dc831dcf63ee243d29a97ef0f756", + "phase/calls/confirm-fresh-05-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "4df4b0b8fb8641ead205cfb201b4282399978142fc08a002091cdebd6efe64ee", + "phase/calls/confirm-fresh-05-hi-m1-source-grouped_actor_prose_withheld.request.json": "51512b2d0d9e1f701659599e9845371bd9543b1a7c7522df75c595b117c93a17", + "phase/calls/confirm-fresh-05-hi-m1-source-grouped_lossless.dispatch.json": "dceb4b347218ba783704d3f51e1d6dbef5cda7af6b6afd66c83d1e373bfe8638", + "phase/calls/confirm-fresh-05-hi-m1-source-grouped_lossless.receipt.json": "50b0b53905d857b1600ae4d942fab491a60e351dff43fce52eae61db9f143c71", + "phase/calls/confirm-fresh-05-hi-m1-source-grouped_lossless.request.json": "3ad35bf22a87d2c267f3eb6cc3c66122c24f0bce1afb3d2b9659cedc050cd015", + "phase/calls/confirm-fresh-05-te-m0-source-chronological.dispatch.json": "7b59a9ce746c767f442655fe95aad21e32426b58fef36e7e366996a00d134a68", + "phase/calls/confirm-fresh-05-te-m0-source-chronological.receipt.json": "3d59d40e13a620ae422d8de892d6a65f56f3a15b8f45c4fec7ab5a7b32c0f28c", + "phase/calls/confirm-fresh-05-te-m0-source-chronological.request.json": "20d340178e741b4c166fc7b7a6235326fee3dbbf644c15ea0755d8a79fc7f263", + "phase/calls/confirm-fresh-05-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "a118ecac530129588d20c165c1069e5951acfba51c898c1462660dd7bbb7b6bf", + "phase/calls/confirm-fresh-05-te-m0-source-grouped_actor_prose_withheld.receipt.json": "558f4efb6fe843a984e723186c8cc7370c6c73a706a4130f30112b08bf18c83c", + "phase/calls/confirm-fresh-05-te-m0-source-grouped_actor_prose_withheld.request.json": "6a76670f769a67320310c83daa2b2bb9d07cedf36f9174f323ca288e5586201b", + "phase/calls/confirm-fresh-05-te-m0-source-grouped_lossless.dispatch.json": "41bdedd9b3da949440f5a9476001bb5602beb52757f1f9fb88526ff507272a71", + "phase/calls/confirm-fresh-05-te-m0-source-grouped_lossless.receipt.json": "39facde2fb2ca48657bcb57ade9e48ed374c4c44a69c947f597c0f7d802501f7", + "phase/calls/confirm-fresh-05-te-m0-source-grouped_lossless.request.json": "77ba511ff57d215a92cbb2d9f5c1f9b2e391b5895ae7af78c4f23f0d0aec2b10", + "phase/calls/confirm-fresh-05-te-m1-source-chronological.dispatch.json": "e02b4cf3e205f49f28255decc711b0581450dced5a1ae740b8b3146479b81b42", + "phase/calls/confirm-fresh-05-te-m1-source-chronological.receipt.json": "a717fcc4fb32dd026cc58175aaedb6d60d85f3d8bb4b4a5fc709a9f634fc7c09", + "phase/calls/confirm-fresh-05-te-m1-source-chronological.request.json": "47f81d2a6131c4f11dfdc81756eced398f102b4439edd52f6f624f9b6c3d3873", + "phase/calls/confirm-fresh-05-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "842762114c60b1923e4bfb92922a5e38be555866b8566d3bda1dbd19105b49de", + "phase/calls/confirm-fresh-05-te-m1-source-grouped_actor_prose_withheld.receipt.json": "e453e485cba93a1bd28e19688585a0e208c9fa0ea113f4133817b827df41cdcc", + "phase/calls/confirm-fresh-05-te-m1-source-grouped_actor_prose_withheld.request.json": "28e640cfb071d46aaea7825052938028220b560f07525c81f117fb7eae905851", + "phase/calls/confirm-fresh-05-te-m1-source-grouped_lossless.dispatch.json": "3e292a3f1bc95ee90bba636b9f5a1197eaf65d3890ce23bf9874ad2f789b3e66", + "phase/calls/confirm-fresh-05-te-m1-source-grouped_lossless.receipt.json": "4560df64ff806f92acb1a7edf11aaf80df883021aacd6a4c36a14058ee916b1f", + "phase/calls/confirm-fresh-05-te-m1-source-grouped_lossless.request.json": "448944d403fd373ca08ec60288f59196cbd38af4750d60ecef2ff06cf11e86e5", + "phase/calls/confirm-fresh-06-en-m0-source-chronological.dispatch.json": "5a4a439136c4a585b6622e6cf0c14796e72d5270899a736ba1509ba90f6ca9ff", + "phase/calls/confirm-fresh-06-en-m0-source-chronological.receipt.json": "fa4a9ff5bf9978b2a45ad7dc7716ddda0b3be1307d0fd6bea0f6ae2e7115ff06", + "phase/calls/confirm-fresh-06-en-m0-source-chronological.request.json": "fd7a9b5111ed3a925e403eaf2c57242ad146d266633a85dc1a1d6b9f52b16523", + "phase/calls/confirm-fresh-06-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "1df23f2031d06664bd8c57096734c4133a380737b3c94f5c1524c95a1ccff6cd", + "phase/calls/confirm-fresh-06-en-m0-source-grouped_actor_prose_withheld.receipt.json": "3ac937d5989cecc37f77aeda88e625b2d011f7a430c46189d35b79d0aff69cbd", + "phase/calls/confirm-fresh-06-en-m0-source-grouped_actor_prose_withheld.request.json": "a5716835e48d8c6f515ab9c57c8f0f5f90490520b49e70dd72d1bdd3e9e8a95c", + "phase/calls/confirm-fresh-06-en-m0-source-grouped_lossless.dispatch.json": "5e7c5f0cf3d3c9190d0860fa2d2ea027ab3243c929175dc3e4926a39971d65f1", + "phase/calls/confirm-fresh-06-en-m0-source-grouped_lossless.receipt.json": "d85d5543411bb672e74038342d6fb2d7a01b81a2f49ed0cbd649e4cc910f3f78", + "phase/calls/confirm-fresh-06-en-m0-source-grouped_lossless.request.json": "c2f563c876f62c4faf6497aa3d3f05f2525a97efb5f85b9df83accacc351e000", + "phase/calls/confirm-fresh-06-en-m1-source-chronological.dispatch.json": "6723f57215ca8faa901d2d5ca371dc2c4819c52d6fc8ec04c2ee8cdb00daad3d", + "phase/calls/confirm-fresh-06-en-m1-source-chronological.receipt.json": "4da276ed331db2a16c6114a5dce475d383ee3a5374c1bd8c362b75dca938d820", + "phase/calls/confirm-fresh-06-en-m1-source-chronological.request.json": "d5d345f9b277fb89d923490b8405eae73c3410cb434c0ced04de7afb1dbd131d", + "phase/calls/confirm-fresh-06-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "774d74bdf6de00f4c39874da4f67ddf07b486c1b599264c67a06a38e2ef82048", + "phase/calls/confirm-fresh-06-en-m1-source-grouped_actor_prose_withheld.receipt.json": "64f4bc62f0c57d87b36a029fa3a34bf9733fa56e8c56a2b461d7c01ca2a6c9eb", + "phase/calls/confirm-fresh-06-en-m1-source-grouped_actor_prose_withheld.request.json": "8a684303442a5c909c4625d1af9e65b96a93f859576d8e2b4ff0f5df2ba7545a", + "phase/calls/confirm-fresh-06-en-m1-source-grouped_lossless.dispatch.json": "7adfa5acc73076ff11dbd2664a50282f8480fafc1a04a5e258330c291f1806c1", + "phase/calls/confirm-fresh-06-en-m1-source-grouped_lossless.receipt.json": "e389a10166b6f0423d5d180dd8da8fa5a04b477907ab012d5acbafd7a1e052b4", + "phase/calls/confirm-fresh-06-en-m1-source-grouped_lossless.request.json": "1c633ec7470e9f8807ed197c6c758304b707e5e62bba0be4e59bf9af2ef5530a", + "phase/calls/confirm-fresh-06-hi-m0-source-chronological.dispatch.json": "b69a54a1df71229a8bfb3cda5452ada2ea4d30e30ee6e6a3ec996729e5fb191b", + "phase/calls/confirm-fresh-06-hi-m0-source-chronological.receipt.json": "96f1d2319179230de78c9abba4a06889e0c084ad2900a1c942794f43a61c5732", + "phase/calls/confirm-fresh-06-hi-m0-source-chronological.request.json": "3de6584cb0615e41d6a48b6e79926b124de17eb6b70caf41e5c835b6f5ce13f9", + "phase/calls/confirm-fresh-06-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "296f10b13b60c142b4f5bf675186908b1c523317834b586c8c8d67375bee1e9d", + "phase/calls/confirm-fresh-06-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "5cefe74c3e6bdd14d9d63585f5473f2bff7b4b0ab7ce2d2502cde3a2dab3fd45", + "phase/calls/confirm-fresh-06-hi-m0-source-grouped_actor_prose_withheld.request.json": "9825b3056fbcf4bf00e4ed2f3e184d96977683ef4ae99b3bdb256af8e9da165b", + "phase/calls/confirm-fresh-06-hi-m0-source-grouped_lossless.dispatch.json": "b3363baf2be03f273a3039bc55460237fbe5699ed999df96d29d5592fd2c515c", + "phase/calls/confirm-fresh-06-hi-m0-source-grouped_lossless.receipt.json": "d33ea5b50f80e42616b5a89d963600b33fc66cb0d75cbc05c436fef21cce21f5", + "phase/calls/confirm-fresh-06-hi-m0-source-grouped_lossless.request.json": "6ee5b0f7d388ff421f2d3b1a8b138e0c22d227f5f04fe76a2f05d63b0faed55e", + "phase/calls/confirm-fresh-06-hi-m1-source-chronological.dispatch.json": "8d6b2ec6d9f806bf1ed38defc13f9938839f7c2f43f9f40833a6f992b76e4ea9", + "phase/calls/confirm-fresh-06-hi-m1-source-chronological.receipt.json": "d0ad203bd81b5ac3001124bd25be07d9ba55678a70d4a144a90dbddc98dc502c", + "phase/calls/confirm-fresh-06-hi-m1-source-chronological.request.json": "d8010c005cea1a4a84aec8ca4905b45a40b6722e2458c622033705913263a9bc", + "phase/calls/confirm-fresh-06-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "f000cfa26e608f16dc0037fe891bc90c721d5bc5f046f66834c80dd713698c44", + "phase/calls/confirm-fresh-06-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "acb635fe546a679c1047acbcc927dce219cd3717917d56d227fd79ac7e9d50ed", + "phase/calls/confirm-fresh-06-hi-m1-source-grouped_actor_prose_withheld.request.json": "a209441c92b3e40a02ed06f39da5408a9023d19675e9cb4d172cde2011e5152a", + "phase/calls/confirm-fresh-06-hi-m1-source-grouped_lossless.dispatch.json": "102817a2f859cd4dc49ce0e897f63cc3647c877527f8269ff421b23ae331c935", + "phase/calls/confirm-fresh-06-hi-m1-source-grouped_lossless.receipt.json": "d774da74ddaf0cc0e76fd8c4331a9446e9078f06b62ecccbd1a2fdd74692c750", + "phase/calls/confirm-fresh-06-hi-m1-source-grouped_lossless.request.json": "dc076df59e38170c8aa5c176359d7a8c116603c195a9b755b6701d84f65716bc", + "phase/calls/confirm-fresh-06-te-m0-source-chronological.dispatch.json": "8460d92a1fc7d07ce3df5580bc6c8c44323e07518c06e997d493448783267724", + "phase/calls/confirm-fresh-06-te-m0-source-chronological.receipt.json": "d9020b8fb6059fab7d0513a677f5631421d3a4e3a5f2d65b8f86ec31d7d2c276", + "phase/calls/confirm-fresh-06-te-m0-source-chronological.request.json": "1ee91dd575f9a8fd578e4089cb16a3ac0d2ddeed2bf14c34bdaf7f68bef62c55", + "phase/calls/confirm-fresh-06-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "2d1cd0a8b5e5dd5fd14c11b437bdb8daaa67d2f2c1ce97f1ff42383749a186b5", + "phase/calls/confirm-fresh-06-te-m0-source-grouped_actor_prose_withheld.receipt.json": "5f9e31a39514d07937d87dbe391bcdfc519bf37233c330a81b460c2a39876fc8", + "phase/calls/confirm-fresh-06-te-m0-source-grouped_actor_prose_withheld.request.json": "dcf05bb0910f2b852b6bfc3c71852a455ccce12044943218fd8a825cfeb51ac4", + "phase/calls/confirm-fresh-06-te-m0-source-grouped_lossless.dispatch.json": "97a8541b1c3f7d486c31c97191cb0f4e6aeaf17cd6cc5e73e3b9bb7b51b72ad3", + "phase/calls/confirm-fresh-06-te-m0-source-grouped_lossless.receipt.json": "140daf2d8cb292de7061b8bf0eda5ce1b37fdededa40bf7beef68943dfe4db65", + "phase/calls/confirm-fresh-06-te-m0-source-grouped_lossless.request.json": "6b3cb6a09da65198a5423fc108303cf572cd672661cefeee5a2bab14c93e8af5", + "phase/calls/confirm-fresh-06-te-m1-source-chronological.dispatch.json": "b3c82312b575445fb0c9813edfbd3949f40fa1f6e98d64c73650d722013f6e1b", + "phase/calls/confirm-fresh-06-te-m1-source-chronological.receipt.json": "2851b935e950a63c80c9544414d947077715e70390d597d58b6f7ca07d1ef647", + "phase/calls/confirm-fresh-06-te-m1-source-chronological.request.json": "daf1d9a4ee7f41355bbd550158e099bdc77d91f42734a7f3ec38392adb2e0f45", + "phase/calls/confirm-fresh-06-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "697c012425f68c42dd1ae498559f80e9489a41cde59df6b00f6a0c52e39b1a20", + "phase/calls/confirm-fresh-06-te-m1-source-grouped_actor_prose_withheld.receipt.json": "12bc68579eaccad389f7a85869bcd8b3123917497898f20860cd7273ea3f0569", + "phase/calls/confirm-fresh-06-te-m1-source-grouped_actor_prose_withheld.request.json": "8d96d79c8904a1895d8eaf8f573eafa135572a9c9848b55872b7399178b75185", + "phase/calls/confirm-fresh-06-te-m1-source-grouped_lossless.dispatch.json": "2475c4c52d2fb34f1d9f7200864d78c746a6d601132b1ab96c96f96bd214e33b", + "phase/calls/confirm-fresh-06-te-m1-source-grouped_lossless.receipt.json": "d7e02176d061e4ceb73709b61cb0f64a56ef406bdd4425cafabf97c220d86912", + "phase/calls/confirm-fresh-06-te-m1-source-grouped_lossless.request.json": "e8da0fd7f651b3b1e92d4fd9c2f30f6110da868c5737643a2f63ae4a9db18c19", + "phase/calls/confirm-fresh-07-en-m0-source-chronological.dispatch.json": "f7f6723fd4f1d13a910cee960a303a727648ea496fc60c204aa30f5f00dbe397", + "phase/calls/confirm-fresh-07-en-m0-source-chronological.receipt.json": "8d491eb1794e4e1eb7383b4c766824e0f3d06a13268072dea1ba9389011d9621", + "phase/calls/confirm-fresh-07-en-m0-source-chronological.request.json": "cc6478e7611b94902eeaefecc7834022e36bb6c47bfedabace0710f7d9ef906b", + "phase/calls/confirm-fresh-07-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "2e56c84778c3d078135fca9a3d42995fced4dd8d1288b1dc124ccc33bd13b6f0", + "phase/calls/confirm-fresh-07-en-m0-source-grouped_actor_prose_withheld.receipt.json": "b29b5ac9d9b96492f3795f956f4aa97f2ee4bca95c051850029f820a2899f92e", + "phase/calls/confirm-fresh-07-en-m0-source-grouped_actor_prose_withheld.request.json": "74b9fb219fd1138dd0ae1b181e7d9f9b700d3a3767438d393acd0f608f93afed", + "phase/calls/confirm-fresh-07-en-m0-source-grouped_lossless.dispatch.json": "4cca19f0120737c35a5c7636ff4a5cde8f62cc650ff47839bad87637fc71b7d6", + "phase/calls/confirm-fresh-07-en-m0-source-grouped_lossless.receipt.json": "cdf9f9a59db7d8c589b0a9f7fba59c01153605987cfdcab343c5a73d0260ee16", + "phase/calls/confirm-fresh-07-en-m0-source-grouped_lossless.request.json": "56d9acab222c4ced942a6da3c4f48f05cb6bdea4d049f965d5c83bb28920f2b3", + "phase/calls/confirm-fresh-07-en-m1-source-chronological.dispatch.json": "d5fadaf9c4bf669aa90c526b6f66129454b4b8acc07aef1a4d92e281d823dae5", + "phase/calls/confirm-fresh-07-en-m1-source-chronological.receipt.json": "fe9871dd5e3f46f1bbb5ebcfff9ed409a1f3086fd7dd1ae0433286db22f81a29", + "phase/calls/confirm-fresh-07-en-m1-source-chronological.request.json": "ca084a412255de9cc52fa17020c4fcda28fb542adfb8eb84583804050cfb30e2", + "phase/calls/confirm-fresh-07-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "5852786c526daa669030587f9b4fdc519462d754e16522b65d601ca302a02520", + "phase/calls/confirm-fresh-07-en-m1-source-grouped_actor_prose_withheld.receipt.json": "62ce456f858ff8a5e8b845588ff154c681149520242d7e2c59f0dcfab2296bf9", + "phase/calls/confirm-fresh-07-en-m1-source-grouped_actor_prose_withheld.request.json": "5c2d83ef664202d545a7af808068645e7e29bc19f8771bb7f5dd68647b73cc5c", + "phase/calls/confirm-fresh-07-en-m1-source-grouped_lossless.dispatch.json": "c099f1eae4b075ff7ae9ee0a92ffba83bdbd25ecedd0636f511ac2b88a2a3623", + "phase/calls/confirm-fresh-07-en-m1-source-grouped_lossless.receipt.json": "df09904759ae1a9ca2381fdf94f665cbae52996769bd93d06390c0e59da64fa0", + "phase/calls/confirm-fresh-07-en-m1-source-grouped_lossless.request.json": "bbd91f96da070041b366efd7138cfabd01ec6b256690a4ceda930d29a7c3ce63", + "phase/calls/confirm-fresh-07-hi-m0-source-chronological.dispatch.json": "d1931f493bdb3b16b4ce71ef5c862526ab6e80b0502a3b02b30e27fc8c1d7e16", + "phase/calls/confirm-fresh-07-hi-m0-source-chronological.receipt.json": "e7d86b6a6899c47ebd0383d92dde455a2fc1b356bf9fd68ebdb7596bc5900a8a", + "phase/calls/confirm-fresh-07-hi-m0-source-chronological.request.json": "f090f56190f044b0d46e27b6fd4d9a4daa64de6f519e1746cdfc2f87ccb26366", + "phase/calls/confirm-fresh-07-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "954ee77a5f28fa0ba6fbc3420882d0e3cc81a99a03995ed9dab40a7b37d5fbdf", + "phase/calls/confirm-fresh-07-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "c33be7be2f83a46b918973925abb991c7bbbe4f1004f97afe52bf2199b73231a", + "phase/calls/confirm-fresh-07-hi-m0-source-grouped_actor_prose_withheld.request.json": "ae56d6cd0200936dc80a09f0154627ee4ef6aa321000271d9180a7453b02318e", + "phase/calls/confirm-fresh-07-hi-m0-source-grouped_lossless.dispatch.json": "9fab9c49dc73ed9306fe3ab4517b26fbfa3f2633b0d251374c2d14ea0141094e", + "phase/calls/confirm-fresh-07-hi-m0-source-grouped_lossless.receipt.json": "e5eddc9debf9635c66e6fe03150a12a7d0e8ac0b79d45a53a23e910729185708", + "phase/calls/confirm-fresh-07-hi-m0-source-grouped_lossless.request.json": "734908e68cf7c12bfb31589b4eca48c961db5166ffb71611e190a0793057068a", + "phase/calls/confirm-fresh-07-hi-m1-source-chronological.dispatch.json": "e68fade23f0357ae9cdca040475412aa4014cdccd2c88511db308e4270804985", + "phase/calls/confirm-fresh-07-hi-m1-source-chronological.receipt.json": "164111df62ce2af807fe5b5d93c68a211c9b8264f5336f673b121d1f98f92f4e", + "phase/calls/confirm-fresh-07-hi-m1-source-chronological.request.json": "c1b9e43e9ba427d824ef9b77dcca191192d8bdd37a953f4020c4301c234d0ccf", + "phase/calls/confirm-fresh-07-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "10f5b8ce9683d294562239842102f6d7694dc1a19c0bfa586314b226b61c9afd", + "phase/calls/confirm-fresh-07-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "cf1090087b5df3426f0e825a9635c14e9fe3e385012a506ec00e9e32b77be268", + "phase/calls/confirm-fresh-07-hi-m1-source-grouped_actor_prose_withheld.request.json": "c3e173785e608294df45a6155a121e158042cf63869f1876061eb71351bd349d", + "phase/calls/confirm-fresh-07-hi-m1-source-grouped_lossless.dispatch.json": "5bf7258b04c037bea1f95dc819245f0a3e8cf131fba73baf126db3cd051dfa05", + "phase/calls/confirm-fresh-07-hi-m1-source-grouped_lossless.receipt.json": "653a6bf78a37ffe29bdf35a84397632540d3a1456887babe7c099173648ad206", + "phase/calls/confirm-fresh-07-hi-m1-source-grouped_lossless.request.json": "164f9af96ec22a673fe2f0ffe7090f7a50e5cc5430752bbcb26c8f5dd3ffcb8f", + "phase/calls/confirm-fresh-07-te-m0-source-chronological.dispatch.json": "f03607336350c8c5c8ee1f8acac6d37ea93da896726317865134ff031354cef4", + "phase/calls/confirm-fresh-07-te-m0-source-chronological.receipt.json": "31cd93103f07b69604eb21d24ecea807c7e36498e0738afbcfb8c70543e6b08f", + "phase/calls/confirm-fresh-07-te-m0-source-chronological.request.json": "db13b94b3a39852b1aff2d467370ca6d0ddf5d229ba14602f5ba4cde73a760b9", + "phase/calls/confirm-fresh-07-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "228599cb8d7f85861ec304ceed7cbb1396446f4719db3161da6161276c079aa4", + "phase/calls/confirm-fresh-07-te-m0-source-grouped_actor_prose_withheld.receipt.json": "1bf8f5b72d602e1cc285f9ec9e9c9e607281c9651b8b68714579eca5eacbfab5", + "phase/calls/confirm-fresh-07-te-m0-source-grouped_actor_prose_withheld.request.json": "2e3d6e1b10556a5b62d0fdce3cffe1ab2a4c0fa0b69ee2a81f741708a44c9749", + "phase/calls/confirm-fresh-07-te-m0-source-grouped_lossless.dispatch.json": "5ee775245c1349b42c2f38ed2a22385874364684acb0a654ec28c1af900a5736", + "phase/calls/confirm-fresh-07-te-m0-source-grouped_lossless.receipt.json": "14662a3000bc6e59ec05d1c3f5e8dce83356e53a173568cc3cc2e72ab03310f7", + "phase/calls/confirm-fresh-07-te-m0-source-grouped_lossless.request.json": "7bedd395b4f46ce6890d9d572f42b1ff5fe1cd50e874c37cf5f4aae9f84e3049", + "phase/calls/confirm-fresh-07-te-m1-source-chronological.dispatch.json": "72615593146d8e9d7b9334a66028444549e0984c2bf91ec3fd15bce1511f3a9f", + "phase/calls/confirm-fresh-07-te-m1-source-chronological.receipt.json": "d9aceb53ff81ab16dbf44ba648c10ea06d7bb7e5cb0d8c8647669b527a08c27d", + "phase/calls/confirm-fresh-07-te-m1-source-chronological.request.json": "1dcb9551c93ef345630867c77efe17785bdfe2be0904215917b0eb9952daaa72", + "phase/calls/confirm-fresh-07-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "4b6ff7c7c698ad119b4167c1e30ec52086fbf4c9a213471e36dc2362c1be8c8a", + "phase/calls/confirm-fresh-07-te-m1-source-grouped_actor_prose_withheld.receipt.json": "5d828e52719cd28b59638e20b592bf9208dcc56b121176fe2f660a5f1145aae2", + "phase/calls/confirm-fresh-07-te-m1-source-grouped_actor_prose_withheld.request.json": "4cd7ccd15326bdadcfd18f370cb88918102671b4eea80365d10a6ed8390ddc10", + "phase/calls/confirm-fresh-07-te-m1-source-grouped_lossless.dispatch.json": "39dab4c4c649007214b9b4d8c8a67cd7bc4585053a3a5b4949ac5d7627673ffe", + "phase/calls/confirm-fresh-07-te-m1-source-grouped_lossless.receipt.json": "be6fa0dd55fc4223a70b4ab4f364b9075068ac11cf7623c329cd5208dd59c1fe", + "phase/calls/confirm-fresh-07-te-m1-source-grouped_lossless.request.json": "c7690f40df10299c9eeef2621330a2862e1b29993dba71b661235efdeeff39b7", + "phase/calls/confirm-fresh-08-en-m0-source-chronological.dispatch.json": "65e2e3e204fc2dac07ac632b0a9714b9a778eab659122ab3a315fcd3d27799e5", + "phase/calls/confirm-fresh-08-en-m0-source-chronological.receipt.json": "e9a81bc742e1440e16f82c3f60538a44db3f7496f06870d8951ba484339763b2", + "phase/calls/confirm-fresh-08-en-m0-source-chronological.request.json": "904d31fd48d3236a117617448a52226b741393c3caf6e43e752ad90a26d03dcf", + "phase/calls/confirm-fresh-08-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "913fe692f0a52850244249f747e09c99c95a523781ecb4e920fe2843669e1f13", + "phase/calls/confirm-fresh-08-en-m0-source-grouped_actor_prose_withheld.receipt.json": "0b9fb236dd4e8c0bf3acefa1584e52561eb50993e8db0be85b9fa6475eeef9b5", + "phase/calls/confirm-fresh-08-en-m0-source-grouped_actor_prose_withheld.request.json": "12113f016e61b549769d3f2132f26711eadc3cdc8f79a8938f983c0d56206e2a", + "phase/calls/confirm-fresh-08-en-m0-source-grouped_lossless.dispatch.json": "730a2050cc90ea0a23e1dd7f06cd963759970af830d97d70ce8583d8e4da3aea", + "phase/calls/confirm-fresh-08-en-m0-source-grouped_lossless.receipt.json": "a559f95302aa096b86e5623af9263eb4cabfbe51878471771ddd4830dc8871fa", + "phase/calls/confirm-fresh-08-en-m0-source-grouped_lossless.request.json": "e7b24ce34173f53a85daf0e3611c29085b9b5808a0fa37e02af7f8d5990f5d83", + "phase/calls/confirm-fresh-08-en-m1-source-chronological.dispatch.json": "e11e03c6514b033d129f706e22ff00315ea76c467221bd54ca736cf92c901c5d", + "phase/calls/confirm-fresh-08-en-m1-source-chronological.receipt.json": "7d54337c5235e3ec6076ec33934daa2add33fd2036cde2fb2df2b0f1e816878d", + "phase/calls/confirm-fresh-08-en-m1-source-chronological.request.json": "ffabcff2961d8aa6e641b9dd56268c73a837519b9ee8a1fb71b5a3b882f0b5f2", + "phase/calls/confirm-fresh-08-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "3634ff9f5a1d1032d993606def0091a8b7e1693e44c4c016229c7727dc6afbf6", + "phase/calls/confirm-fresh-08-en-m1-source-grouped_actor_prose_withheld.receipt.json": "c540bc74edfdc47bbc13113474e68cdddf5759141c51f3f512e3a950bf365a18", + "phase/calls/confirm-fresh-08-en-m1-source-grouped_actor_prose_withheld.request.json": "8ec3122d2be886044fdf832428aeaa47083a774e18c811b6500eb76e4571c55e", + "phase/calls/confirm-fresh-08-en-m1-source-grouped_lossless.dispatch.json": "688be300c5efd335eae73d26e3819d95eb475ee082bf5ea8909da8881f0bc1d1", + "phase/calls/confirm-fresh-08-en-m1-source-grouped_lossless.receipt.json": "2f7bc02219350bc25b371b6db1820abd835887fadd2db156c06084aad9687980", + "phase/calls/confirm-fresh-08-en-m1-source-grouped_lossless.request.json": "db7b665e4e6399bf1ece30e81b8ebe4b984f809229974ffe56161c86aa2d0967", + "phase/calls/confirm-fresh-08-hi-m0-source-chronological.dispatch.json": "785874fb0a5fff0696dad1ce59e24406e988ac1a577541ca17292680056febe6", + "phase/calls/confirm-fresh-08-hi-m0-source-chronological.receipt.json": "f77e5876b1374a6d747e44826331991498e221c3dc3d564cacca20b9553ca3e0", + "phase/calls/confirm-fresh-08-hi-m0-source-chronological.request.json": "a86e8a00d17eb2c74c540952d0f903df000d4d2898462dff6f06dab1fbd39884", + "phase/calls/confirm-fresh-08-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "27ab66d9efde5bd86b76f52b906edfbf562f977f2f10fa5e8a0a0a9ff2d2d296", + "phase/calls/confirm-fresh-08-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "22f7ce8a8a85591eed1986d36e8330e65b2f2bf9d6d47793cc0178b78ff77325", + "phase/calls/confirm-fresh-08-hi-m0-source-grouped_actor_prose_withheld.request.json": "078dd2dada6f2670bbc0e3e6f7525526c0ff884beb2ac9c6b94067fce9bb74f5", + "phase/calls/confirm-fresh-08-hi-m0-source-grouped_lossless.dispatch.json": "aa611e97cca0f5438dd06927871848b7827651551104857a946215813cc7a4f6", + "phase/calls/confirm-fresh-08-hi-m0-source-grouped_lossless.receipt.json": "064612a4f538843e96a5bdb7d332cb67cfd5198c0a459f690144e9c7098166f9", + "phase/calls/confirm-fresh-08-hi-m0-source-grouped_lossless.request.json": "d1b6fd9d309e5ab2b6a093b466128206fcc2c7bd983fa4dd2c7c2f25d8a1d3a2", + "phase/calls/confirm-fresh-08-hi-m1-source-chronological.dispatch.json": "6d62de3c2e56dc96279cc0c3d79713d227980dd6c42e95082a3285e7d7b346ed", + "phase/calls/confirm-fresh-08-hi-m1-source-chronological.receipt.json": "8ed35f27a95281c3750dd1caa4caf44d80d091b78a87f7cc184c19f5218ab79d", + "phase/calls/confirm-fresh-08-hi-m1-source-chronological.request.json": "b69a93dc5f9daa07aa7071fe066489f56ebee273cfb7406b34e943caf795f6eb", + "phase/calls/confirm-fresh-08-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "b1ef2214e02e78f6521c538cd6925ab5c4a5aa21dcbc2e4bac34a0555503b34b", + "phase/calls/confirm-fresh-08-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "8e3765466b14b6e5bcd0f3d6d12e09b65231a4593dc24459330d5b69938c9948", + "phase/calls/confirm-fresh-08-hi-m1-source-grouped_actor_prose_withheld.request.json": "b9b6123eca64f2cba9ee0cbbb074c2ea1f3417cfcc3a98c5f5bdcda494182860", + "phase/calls/confirm-fresh-08-hi-m1-source-grouped_lossless.dispatch.json": "0845722289020ff43777f95ca4c794f2c2e14aeb176efc21decb201a8cf4e380", + "phase/calls/confirm-fresh-08-hi-m1-source-grouped_lossless.receipt.json": "073788f9571c0c7af20f118cd0ea0e66464b05b8d40be7407693f1e050b3f81a", + "phase/calls/confirm-fresh-08-hi-m1-source-grouped_lossless.request.json": "758ea8de97eda129a5ebdde1199e56942b804581345c3ca343558dfbb3c473ec", + "phase/calls/confirm-fresh-08-te-m0-source-chronological.dispatch.json": "0c1553789994c8f546469c4ddd086e43d8482d2607d7cde1f88a5b3cd4e4edf5", + "phase/calls/confirm-fresh-08-te-m0-source-chronological.receipt.json": "aefc9ad4c36fe0b3485b487bbe6e63c25807e738e9ec15559b4f1f61b3be725e", + "phase/calls/confirm-fresh-08-te-m0-source-chronological.request.json": "306d6e73361c7525ae15ad4c249f25209a0c51b8209032e82af7410a3f448ea2", + "phase/calls/confirm-fresh-08-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "9f6633e840e7c564fb832aef31f9e5f1cd54ba00df72e11d904c8c7b0e15e347", + "phase/calls/confirm-fresh-08-te-m0-source-grouped_actor_prose_withheld.receipt.json": "ec0540efc68ca6aa7534a662ea0b02a637a2727e01fb701f474ac52a5058b88b", + "phase/calls/confirm-fresh-08-te-m0-source-grouped_actor_prose_withheld.request.json": "190d2f006a901433d921a969a417d81ff25abd406c7c17e13f21c5ea716156eb", + "phase/calls/confirm-fresh-08-te-m0-source-grouped_lossless.dispatch.json": "b082e5416f4f02e5c5cb6d38a3229bfec5016433e916e1e327d1e81985583c9f", + "phase/calls/confirm-fresh-08-te-m0-source-grouped_lossless.receipt.json": "becf70599ad43d09f63025faffa065d49a940d33b6db4b1bf037e723517ae049", + "phase/calls/confirm-fresh-08-te-m0-source-grouped_lossless.request.json": "dd6358f5489d7787cb92706cd2a3e36f9f3f8d86b8fb3ad3f3d41d23337e885b", + "phase/calls/confirm-fresh-08-te-m1-source-chronological.dispatch.json": "5d17d791442d643816264e4f44896eb726ef7fa68296da41d7c95766c54d26c5", + "phase/calls/confirm-fresh-08-te-m1-source-chronological.receipt.json": "de6eb6cdd12c92643769e00eeb44ea75ea16aeaf6d826c8e8ae2fb58af30916f", + "phase/calls/confirm-fresh-08-te-m1-source-chronological.request.json": "d884ad82e4ca6acf8b27b4d939af4f1690f5bece0722aca9dffdc85242232892", + "phase/calls/confirm-fresh-08-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "79a10777f2b37a1ba3b9f522d70063fd2597b7353056c433546d9ecac88e1ccc", + "phase/calls/confirm-fresh-08-te-m1-source-grouped_actor_prose_withheld.receipt.json": "523667f0c011184a69bdfa943740d08a3c625665fd74bc278f6bc474183b1a61", + "phase/calls/confirm-fresh-08-te-m1-source-grouped_actor_prose_withheld.request.json": "7bc3761ea197c84dc89c0ad2be00e52ae2d748039a3ca2e4130e842b14243d89", + "phase/calls/confirm-fresh-08-te-m1-source-grouped_lossless.dispatch.json": "a251cd0f93388539df21e6147dc1ed0f489d8bef33312823a30f92935562393e", + "phase/calls/confirm-fresh-08-te-m1-source-grouped_lossless.receipt.json": "021ef267b5f6d53eacabf40bd78755a0220af1264d80428d23486387d40d831e", + "phase/calls/confirm-fresh-08-te-m1-source-grouped_lossless.request.json": "4fb6d612e543ad9fecdb1756931c2ee2fab35bdaedd408f41eaa488fb3fd0045", + "phase/calls/confirm-fresh-09-en-m0-source-chronological.dispatch.json": "0c27c2a1ffbae83f5e68c46ebc18aa4ab2898dc0ce39da44c194ecd9559ba916", + "phase/calls/confirm-fresh-09-en-m0-source-chronological.receipt.json": "294b12a89571e87f680af03e3c9dc2e746b2fa4514ddea5c37352f3c32710a4a", + "phase/calls/confirm-fresh-09-en-m0-source-chronological.request.json": "15e4f92266bafd2265de19c897462bc45c8c0bf46c1ef3511b6226ab31812aa2", + "phase/calls/confirm-fresh-09-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "d6681a1085a57bb8fd818cf7902b784ca6175a72865f2d1e7671a131663ee669", + "phase/calls/confirm-fresh-09-en-m0-source-grouped_actor_prose_withheld.receipt.json": "6a0dc2afd71e04d67a800d2bf0aaa00e31dc8e69f6b7cb67c1bbda5527599e7e", + "phase/calls/confirm-fresh-09-en-m0-source-grouped_actor_prose_withheld.request.json": "3554ce7f3b82a1659263fb32902a3eb74e11bcfddf2c349db9f85cfb8479e194", + "phase/calls/confirm-fresh-09-en-m0-source-grouped_lossless.dispatch.json": "065083032311ab821c3e0273a3d27d81810809b13ffeaefc46637266941e93e2", + "phase/calls/confirm-fresh-09-en-m0-source-grouped_lossless.receipt.json": "a07c85653264eb0a9c2b536fff59d3c3f57b654419b8d09975804afdcd60e3ba", + "phase/calls/confirm-fresh-09-en-m0-source-grouped_lossless.request.json": "13727491605cbf797b4e3d3509dd7a8b540d9627bdb83bf6a66f0ac8e6731b8d", + "phase/calls/confirm-fresh-09-en-m1-source-chronological.dispatch.json": "5a78ebbd0e7773555b029a7bc0b2507a7399858c93f296440b4ffccb72c9e8d0", + "phase/calls/confirm-fresh-09-en-m1-source-chronological.receipt.json": "950b2018b4cdb3cd2e40dcbb715263223b94dad0fb7a9a1a5ff4d4dc00b9ab21", + "phase/calls/confirm-fresh-09-en-m1-source-chronological.request.json": "34d77e58dbd3f47dc78660852604b2123a0185ed2ad43d574b3bdbe3378ed66c", + "phase/calls/confirm-fresh-09-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "d74d1b91679f487f271b79e14a92a4a1444bf9a205e1268cfa7fd8948882a176", + "phase/calls/confirm-fresh-09-en-m1-source-grouped_actor_prose_withheld.receipt.json": "6bf95bfdfc1432243fb9a5709309621405d142d4eaf6597982b6aa5729c1ee56", + "phase/calls/confirm-fresh-09-en-m1-source-grouped_actor_prose_withheld.request.json": "210d73e7fe8e3fc615b730ada8205823f187b8170ea3cef5998c7873366fbc13", + "phase/calls/confirm-fresh-09-en-m1-source-grouped_lossless.dispatch.json": "54c1dfb2cc8f2dfdf5183f72a93516c2522f124b28cf2a980773990bf3b97dc6", + "phase/calls/confirm-fresh-09-en-m1-source-grouped_lossless.receipt.json": "7346374e627076649ec9f22050cc59fbca77a61982591c970eebf63343345789", + "phase/calls/confirm-fresh-09-en-m1-source-grouped_lossless.request.json": "530fc922b0e829b18549db88ff95eef766cdcbfeb275a92bdcc15783058ccf28", + "phase/calls/confirm-fresh-09-hi-m0-source-chronological.dispatch.json": "74b70d68f813f9630930489dd181a28e2fea48616a254163d2f39a97fb296756", + "phase/calls/confirm-fresh-09-hi-m0-source-chronological.receipt.json": "7ab4749e3c9efd8512e51191fe4a15c358c1653c0f91ca4ad22254896748a29d", + "phase/calls/confirm-fresh-09-hi-m0-source-chronological.request.json": "7bd7849d1f1603bf83c1a77a7f542f33802187f5b39265591012ad3a1b842d9a", + "phase/calls/confirm-fresh-09-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "d950a8469207b5943aab74ad19f6d098d2f5f6afa435f497f021a17007ba89e2", + "phase/calls/confirm-fresh-09-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "f5ff675dab79b32969727c8dbb465661ceb564b5af2acead18153434099d18c4", + "phase/calls/confirm-fresh-09-hi-m0-source-grouped_actor_prose_withheld.request.json": "695df07bc43fbccbf7c495ce617736521afef2ab2d98e2d7f1ac6dabad6505c8", + "phase/calls/confirm-fresh-09-hi-m0-source-grouped_lossless.dispatch.json": "5eaf827083fa6d66014703d0bd831dc70d51717cea1f21be4f34d72041b5ba3b", + "phase/calls/confirm-fresh-09-hi-m0-source-grouped_lossless.receipt.json": "f34d17191eded002ab290b0e31b175f9911a25381eed8202ac9e41879dbf488e", + "phase/calls/confirm-fresh-09-hi-m0-source-grouped_lossless.request.json": "24758a469cffb4ff12fcf2af1e11761c149c709c7fcc189a34be8645c0ea0b34", + "phase/calls/confirm-fresh-09-hi-m1-source-chronological.dispatch.json": "a2c122b9c3e0cb25517501de9f20db17cb870d0277a07c6079ee28770f401746", + "phase/calls/confirm-fresh-09-hi-m1-source-chronological.receipt.json": "a7fe14344f84364cc3eb743eef53f28155be1fd15ff98ad33acdb85d79dd7207", + "phase/calls/confirm-fresh-09-hi-m1-source-chronological.request.json": "c13c1cc5177686155577bfb42e6482d4088c22b11c0d749959144c2f194ebe4b", + "phase/calls/confirm-fresh-09-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "74b5aedb1390f04587e25ed5a85069595c30a43f3c5c20547c40b78f33e5e5bc", + "phase/calls/confirm-fresh-09-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "381acc29f79fbec14803dedcae511bcc60bb6c154c54bce43dd94004db50b1bd", + "phase/calls/confirm-fresh-09-hi-m1-source-grouped_actor_prose_withheld.request.json": "37b64f3cb494add0d3bbf28f288672162d3cac20bee1fb3075064ac7070586dc", + "phase/calls/confirm-fresh-09-hi-m1-source-grouped_lossless.dispatch.json": "929b39939a8d1063729acbf0efa5f0524d8a53d428fd4db803b3fb8944784d6f", + "phase/calls/confirm-fresh-09-hi-m1-source-grouped_lossless.receipt.json": "0a45e862c8bbccb1c84a778cbf9067bff9c0262d18b8a1843479fe4a899d201d", + "phase/calls/confirm-fresh-09-hi-m1-source-grouped_lossless.request.json": "350504d467da06f9f17edeba6e35a939959cbe16846e26743c361259cf455341", + "phase/calls/confirm-fresh-09-te-m0-source-chronological.dispatch.json": "dc1f5f44b1496d274c2a68919e2f07390ebf8beb22a20b97c4b2b34cca85f80f", + "phase/calls/confirm-fresh-09-te-m0-source-chronological.receipt.json": "d61287118f6b970722a149add03080896a47389b9736d70eb1b31e08096d9eb3", + "phase/calls/confirm-fresh-09-te-m0-source-chronological.request.json": "eb50d36a9afc48df13eee188facfef00b3bd239b14c82d4fe2486eeb614c5f6c", + "phase/calls/confirm-fresh-09-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "836f3b51e0ba1f1b474d1e4b0a9197dca0354ec3a781a33d7c5d0462dd425b75", + "phase/calls/confirm-fresh-09-te-m0-source-grouped_actor_prose_withheld.receipt.json": "e88bd508058520b962f3cd0c5c77b954ca2ce039469b2cd1097c302b0489079b", + "phase/calls/confirm-fresh-09-te-m0-source-grouped_actor_prose_withheld.request.json": "513d50a380bd7db584b917af08c8fbc9d5ccc1c639d2a5d11c6d5030a46953d9", + "phase/calls/confirm-fresh-09-te-m0-source-grouped_lossless.dispatch.json": "706cfd0ea5b8f98af31e392d4934b1c53742cbb5cebc9a1d4af79d394ede92e8", + "phase/calls/confirm-fresh-09-te-m0-source-grouped_lossless.receipt.json": "86b54d42bb8ede6397d43e9e5e9de7b4df9fe375c914f9a6b22b96de01a3d280", + "phase/calls/confirm-fresh-09-te-m0-source-grouped_lossless.request.json": "021901b18c68b1dcfab0f89563a8a807506cd110d9053049c8268a31390ef5b1", + "phase/calls/confirm-fresh-09-te-m1-source-chronological.dispatch.json": "2b21f1d0158b09f1692364ed3c02e73f392063956899e0f7b3028f2593edc592", + "phase/calls/confirm-fresh-09-te-m1-source-chronological.receipt.json": "e521af0b73018c2c2460f1992d77d195db1afce9a731a6e2d9f9f4f9f6d19da9", + "phase/calls/confirm-fresh-09-te-m1-source-chronological.request.json": "4fb1e879836df8127b37931eb0a3f2004489395816b5db33c10718cfcb7c4598", + "phase/calls/confirm-fresh-09-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "5d0563b2587f4b5ef33cbd88a91642e865311876fb40e8bb392d224b7d35883e", + "phase/calls/confirm-fresh-09-te-m1-source-grouped_actor_prose_withheld.receipt.json": "c7ae1fe588adb4734029095b3af01219cbad7af5ddfaaee23ba05bdee6042821", + "phase/calls/confirm-fresh-09-te-m1-source-grouped_actor_prose_withheld.request.json": "83f7758bcdb3dccb3ac84fff283c08efdb5b981409491453e7f5980649dc382b", + "phase/calls/confirm-fresh-09-te-m1-source-grouped_lossless.dispatch.json": "0e3d3c80d15989b126e180e7af17da8514eae25e2464671f6448ba7a2e5d4a31", + "phase/calls/confirm-fresh-09-te-m1-source-grouped_lossless.receipt.json": "a172dfba032da0780c300a8fe26be667f1271b1976cab9efaed7b8f8c52b1def", + "phase/calls/confirm-fresh-09-te-m1-source-grouped_lossless.request.json": "08f5794f1d60714516134c1214b1977685ea530703b82e0863ffcd75f470e7b0", + "phase/calls/confirm-fresh-10-en-m0-source-chronological.dispatch.json": "8cfc89870e333678dbffcbab505e408334e3e1009a7f5126c3d13dcac9ce145f", + "phase/calls/confirm-fresh-10-en-m0-source-chronological.receipt.json": "9d9635a4f54160023d075ace30c99abd7ef20bb53f4b0692b9d4387b6c9b71b7", + "phase/calls/confirm-fresh-10-en-m0-source-chronological.request.json": "df53ee1d0d3add3b4d1427bf2d710e61f955eae44a46f6df92cfa847bef87559", + "phase/calls/confirm-fresh-10-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "db203e1eb6b787673488261e94432df0bb77221131b544acf8322b9ec7f280af", + "phase/calls/confirm-fresh-10-en-m0-source-grouped_actor_prose_withheld.receipt.json": "73b89b9153610287850dafbc18cd1016252c1c6e9ba116c537e15b46f9a5b324", + "phase/calls/confirm-fresh-10-en-m0-source-grouped_actor_prose_withheld.request.json": "4269735326c8081466fd5fc63167f5f7eaecc1c73cc41120a948a11229daebde", + "phase/calls/confirm-fresh-10-en-m0-source-grouped_lossless.dispatch.json": "0517084aa29bea8ba83a110181b947ca88aa6630ce544626a4bf3507d526204e", + "phase/calls/confirm-fresh-10-en-m0-source-grouped_lossless.receipt.json": "215ae61057720a813152a44735609f5a651de92c9f65e89449d045e821bbc40d", + "phase/calls/confirm-fresh-10-en-m0-source-grouped_lossless.request.json": "6125adfe2a6ebecca961c0077ca4e0cf43156511c60a21d46a09e3dd72c3c1d4", + "phase/calls/confirm-fresh-10-en-m1-source-chronological.dispatch.json": "2767f4b2ea36b464f75473f83b3aeab6f5f34c7324f600d3920ddd2fdd746af1", + "phase/calls/confirm-fresh-10-en-m1-source-chronological.receipt.json": "02ebb253d387a579ae47ec3565e901b304378d7a31f3014c2bf9670523b2ede8", + "phase/calls/confirm-fresh-10-en-m1-source-chronological.request.json": "2ec64607765e0c4048e805de9207958b37a5e5f50a203da253fd701dbb5be325", + "phase/calls/confirm-fresh-10-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "e9bc64f5f2f0f781866cba51f17f032c216f7831b4c73fb6a55ddb599ec002cd", + "phase/calls/confirm-fresh-10-en-m1-source-grouped_actor_prose_withheld.receipt.json": "e8d09335f850e835d8258796440cd473a49f3bb2ecec306f3ba81dfb2c88df2d", + "phase/calls/confirm-fresh-10-en-m1-source-grouped_actor_prose_withheld.request.json": "3e67f521f40ebe61523440efa90038b896792f332d6e20a4aae13ef37ecd09b6", + "phase/calls/confirm-fresh-10-en-m1-source-grouped_lossless.dispatch.json": "150358a63261e8a1bf40fa753c42254334e73bb946f0fb4a0d795f6a403cb3db", + "phase/calls/confirm-fresh-10-en-m1-source-grouped_lossless.receipt.json": "01e249b38d663b879c99831246b8b356c132f19686a87d24d025eca761373ac5", + "phase/calls/confirm-fresh-10-en-m1-source-grouped_lossless.request.json": "0f2b6fd50b0c57395b07501dd851722eb974843cf5d70cc9c21118151eaf65ad", + "phase/calls/confirm-fresh-10-hi-m0-source-chronological.dispatch.json": "fc12a88c29013a9e9ad72e3cf0c8a1eda6727751253a236faa0dfd49c4c7cd48", + "phase/calls/confirm-fresh-10-hi-m0-source-chronological.receipt.json": "a77ba3031133724203e0d0b0e61956a61a3eedad1a51d7b74f0fa5327f5bedf8", + "phase/calls/confirm-fresh-10-hi-m0-source-chronological.request.json": "4b0897e5d82d9b79725b8f196de5855fe02c1851f758b18e2e9a4f868c2dfdb2", + "phase/calls/confirm-fresh-10-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "5fde8f574e3b3c7a48d72019ab557e54b16c712869e013194395f68dbc504702", + "phase/calls/confirm-fresh-10-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "1c90d868d1ffbca617f3b9725b47b4bf67b22c6deaee6b19896aeb49232bfff7", + "phase/calls/confirm-fresh-10-hi-m0-source-grouped_actor_prose_withheld.request.json": "e814c48ee3cef813f89d338681f231e9060c773258fdf3f250b31c06ddfd7339", + "phase/calls/confirm-fresh-10-hi-m0-source-grouped_lossless.dispatch.json": "44961472d1dbd3f96c8b82184e1f6ca14153b4a27b017e07b004537e3b1dad6c", + "phase/calls/confirm-fresh-10-hi-m0-source-grouped_lossless.receipt.json": "07152f15bda3d535a0aee4b0e1ecf2caf845324bc8d3675a6cb88327bcc7181b", + "phase/calls/confirm-fresh-10-hi-m0-source-grouped_lossless.request.json": "648513d5ec6ad98c75c0057ae2b2fc25c8a8cc8f389cfea37fb8a8d0d3a6f170", + "phase/calls/confirm-fresh-10-hi-m1-source-chronological.dispatch.json": "cebe21ded0b7acbb25e37c32d98954ddeb0edeee32e3764f3f2d70d51618c3c0", + "phase/calls/confirm-fresh-10-hi-m1-source-chronological.receipt.json": "7fcb2c0fdc961cf90257ea476874880f2a9fa338c2ceb116aa7c47e45843f5fa", + "phase/calls/confirm-fresh-10-hi-m1-source-chronological.request.json": "e2288c400669a73691b102cce1c0769a8b24d08ec4b343ed960c1777b23808ef", + "phase/calls/confirm-fresh-10-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "47e74b2cb141ee9887f41154010bbfaedd826bc42d65623c7f4b283c5ab14a0f", + "phase/calls/confirm-fresh-10-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "629f20813797cab89fee9492d2521faafe1dac4c060580ab1c1334f920451444", + "phase/calls/confirm-fresh-10-hi-m1-source-grouped_actor_prose_withheld.request.json": "3506f1e71cff5717075ea9124e6ad4c83a4faecd51ec7708e5806b8558296d58", + "phase/calls/confirm-fresh-10-hi-m1-source-grouped_lossless.dispatch.json": "8e7df3b7a9e77f7b53859247373498f0505179e7ce7b1b693238d97a826e8bc8", + "phase/calls/confirm-fresh-10-hi-m1-source-grouped_lossless.receipt.json": "9907f0d5311f6928a901c67a54cbab4e48eb24400c0ed76b6ee256116c3573c6", + "phase/calls/confirm-fresh-10-hi-m1-source-grouped_lossless.request.json": "61c27b20e8ab65e4248bb1aa4a9e9a5ad79d84f6a40f86cc174417f3efb4f193", + "phase/calls/confirm-fresh-10-te-m0-source-chronological.dispatch.json": "f99964dd369a3ca95b0f61f3816f447d16e10a179e8f84b23a704bd3d78154c9", + "phase/calls/confirm-fresh-10-te-m0-source-chronological.receipt.json": "faebd656d6736e91a447d345f29ce3d8d3647ad4cf833a4649ee6a03a0804b93", + "phase/calls/confirm-fresh-10-te-m0-source-chronological.request.json": "6ef7b2d387fbee2c549c308a3b625b9cf29ddb45df3d96f84c39bf941a4a0b36", + "phase/calls/confirm-fresh-10-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "3715223cca1933d53f2952b1ebb9b02dc79013497f90cdd7bd7b37ec56609647", + "phase/calls/confirm-fresh-10-te-m0-source-grouped_actor_prose_withheld.receipt.json": "5ce9f86d4150d4b30a6e05f2aebae6f49f548dcbb3541b706fec18adbd9f0ac5", + "phase/calls/confirm-fresh-10-te-m0-source-grouped_actor_prose_withheld.request.json": "34ca6d400007484dc46ac3ec32fd185fb527daa3498f5e071a65a7f516bf8a13", + "phase/calls/confirm-fresh-10-te-m0-source-grouped_lossless.dispatch.json": "1706113f7ce78bb4ad033b5e725d65067064b2766e92688fb7e76d55d63e2450", + "phase/calls/confirm-fresh-10-te-m0-source-grouped_lossless.receipt.json": "afb5d450a5a88a9c78cd7d89563216e4f464c2e3c8dfb31e95003c317d767b68", + "phase/calls/confirm-fresh-10-te-m0-source-grouped_lossless.request.json": "7281331ce9ba5ffc1ad5eec9bc118ce52a0fc1a0bf33be71d2d5f444eb509e8a", + "phase/calls/confirm-fresh-10-te-m1-source-chronological.dispatch.json": "995d8909552b74df6b7743ff9965e9079cb5f6e9dd64e54b5b75e28f9c6ef3a6", + "phase/calls/confirm-fresh-10-te-m1-source-chronological.receipt.json": "46b161ba6e2dcaf97b0b34e5ce5d0f2ee432b2a04112d7c02982bd326eeb714d", + "phase/calls/confirm-fresh-10-te-m1-source-chronological.request.json": "f4d160f092ccb70d5be8fc73b0a261e3d074809b89ae080e5f23d88ef19775c2", + "phase/calls/confirm-fresh-10-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "0e9ff74a71406515d5624de0d62ff43c70973db5eb6323cafe039e3850cae772", + "phase/calls/confirm-fresh-10-te-m1-source-grouped_actor_prose_withheld.receipt.json": "0a6d44cb102963056d03dc448d3b5c1b33b49cdeddef817e97ed4a1b525ecff4", + "phase/calls/confirm-fresh-10-te-m1-source-grouped_actor_prose_withheld.request.json": "bb2158fefbfeaf9ed22e0a29aebe1ab1deecd94d4072d0d513d35fe4301e00e3", + "phase/calls/confirm-fresh-10-te-m1-source-grouped_lossless.dispatch.json": "79d328fa9b0b577e89e2ad0169c0b043411a840704ff7316c5917ade19502310", + "phase/calls/confirm-fresh-10-te-m1-source-grouped_lossless.receipt.json": "79824594395dcba4dfefb8c9ae2448374cc158954e81cbffe1db81311e0e2198", + "phase/calls/confirm-fresh-10-te-m1-source-grouped_lossless.request.json": "f7d19c179a78f6f8467bfd91ea403ed7f9a79cf084c026ce469735fde22fecab", + "phase/calls/confirm-fresh-11-en-m0-source-chronological.dispatch.json": "46aa2523a5aa56945b1cb6b02b64455d351ca38b339a0c08c2394cb7edacf094", + "phase/calls/confirm-fresh-11-en-m0-source-chronological.receipt.json": "9fd59e618d6a0fa749892834479f122450886950945263950da7eabaef16fac2", + "phase/calls/confirm-fresh-11-en-m0-source-chronological.request.json": "2c2fff724b873ee2e07b4885706deb4babc0c508e2f62011fa314b15aa65e252", + "phase/calls/confirm-fresh-11-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "749fda06ea2bbfe2b98d7cf9c2cf0dd3aabd100746ade62b804d81fcbab169ff", + "phase/calls/confirm-fresh-11-en-m0-source-grouped_actor_prose_withheld.receipt.json": "3f9f1f205d73d33cbaf1c7a5ce70de2a08964b99eb1a173816b2e18fdd016af0", + "phase/calls/confirm-fresh-11-en-m0-source-grouped_actor_prose_withheld.request.json": "effb03ed5860115360228849cbe4d400852cb16b75f86ef0371677645e254168", + "phase/calls/confirm-fresh-11-en-m0-source-grouped_lossless.dispatch.json": "82c00b3e3818c18984a88945788ea31ae4d932a53b97acff91ffcff6ee217fea", + "phase/calls/confirm-fresh-11-en-m0-source-grouped_lossless.receipt.json": "a2a63015c32f9435aa884a2e2783cc1d1fbedcd4bd14b39ad3907a4168e86f51", + "phase/calls/confirm-fresh-11-en-m0-source-grouped_lossless.request.json": "5c3a347e143071f51d6fa0250cdf41e4ace8e010cbd697d63976ba96fc05f84a", + "phase/calls/confirm-fresh-11-en-m1-source-chronological.dispatch.json": "cdef1a9a64e2c51f0821be763c55253c64c0497b0d7a42255a7792831847c0dc", + "phase/calls/confirm-fresh-11-en-m1-source-chronological.receipt.json": "77f4b2f269f3d5257070657e465cc3c51aef620f8caf46aa82f868b69cc49a61", + "phase/calls/confirm-fresh-11-en-m1-source-chronological.request.json": "50787cf182dd50710a0cfdbc094cbb2ce09ac44120365b2d04d28dbfa4046bad", + "phase/calls/confirm-fresh-11-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "76373b27b485bbda653d7d18e727c56f230fddaf56d929477e42d2bd654b69e4", + "phase/calls/confirm-fresh-11-en-m1-source-grouped_actor_prose_withheld.receipt.json": "556943946995f48f6c409e7374cbb7ff00e6b3da89b5ebd2d8c72512c6d1f414", + "phase/calls/confirm-fresh-11-en-m1-source-grouped_actor_prose_withheld.request.json": "093cebc39157261cf07a2ffe4101088bbef57504c815db6ef1bbe0fab9118ac3", + "phase/calls/confirm-fresh-11-en-m1-source-grouped_lossless.dispatch.json": "e4a526930db0b8af6c44b7e395ce004a9b9df0af86caa29170540cebe7c9cf86", + "phase/calls/confirm-fresh-11-en-m1-source-grouped_lossless.receipt.json": "fcf1450d670839e66e48e5216a3a4a3fe13cf2d6c9063ae3bbf104adde844209", + "phase/calls/confirm-fresh-11-en-m1-source-grouped_lossless.request.json": "a80ef8f37f0081a0af52c36ac47a90aa2c0491d3e16df70c85fb72dd9e594b4b", + "phase/calls/confirm-fresh-11-hi-m0-source-chronological.dispatch.json": "6f5cb4d70570ae564a4a901b4d2c9bdb7d513e1b3749831d1b46dc35fc05e2a8", + "phase/calls/confirm-fresh-11-hi-m0-source-chronological.receipt.json": "e82b981f52bf91d9774efe3d26213e3b26021b4f9d008e53dec9433f6770441e", + "phase/calls/confirm-fresh-11-hi-m0-source-chronological.request.json": "ef15373483cbdddf4aefd59057f98897fe099b90beb8d7311dfc090d0584b0fc", + "phase/calls/confirm-fresh-11-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "3a1039b22d0b64b0281e2592f2819e96d83be52fd99366524cba04c4c50d459a", + "phase/calls/confirm-fresh-11-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "3a20c97a9b0ea7ae7c3daba634b738161501bac52ebda22317a79e11e6504dbb", + "phase/calls/confirm-fresh-11-hi-m0-source-grouped_actor_prose_withheld.request.json": "fe707b9279d60ba9228e57fba3353e6410f1524e661e171ba7835969cce17e31", + "phase/calls/confirm-fresh-11-hi-m0-source-grouped_lossless.dispatch.json": "26da0880acdefbcd094017a0eb2e59216cd78d1c3a217663c9ab9a7b33cd9d0e", + "phase/calls/confirm-fresh-11-hi-m0-source-grouped_lossless.receipt.json": "ddb96b4b2f8836bd2d4237dec003e5c8537be97773f5b1db81fb389b57ab62ad", + "phase/calls/confirm-fresh-11-hi-m0-source-grouped_lossless.request.json": "4fb86658ad990238b2193c0aee8c45aca467e010b9085362acc14fb01d7ab802", + "phase/calls/confirm-fresh-11-hi-m1-source-chronological.dispatch.json": "f72c8f632d6890cf7f9ce2e44a523255e6714a9f79d8f9d41669e7a01a418209", + "phase/calls/confirm-fresh-11-hi-m1-source-chronological.receipt.json": "f7150cbd509f1b6e6b497d35bb96c44369a5632d79eb1c55a3ded32f1498f183", + "phase/calls/confirm-fresh-11-hi-m1-source-chronological.request.json": "d46f148dd27e8b9db8bb1985ee9115de674fb4bff0af5d87b55da961c536b810", + "phase/calls/confirm-fresh-11-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "8f930d7699eeb5cff70e3734a3f39a48f2677a4b933e076d297bd387396929fd", + "phase/calls/confirm-fresh-11-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "b3c3c05adb1a43f37d49aa00c72ca7442e032d59c18822df76d0f5ce585b8a47", + "phase/calls/confirm-fresh-11-hi-m1-source-grouped_actor_prose_withheld.request.json": "c54e246c184437f03b6d7c2d59ffdeba39acca07e8514141cb32813df02d2f6c", + "phase/calls/confirm-fresh-11-hi-m1-source-grouped_lossless.dispatch.json": "f023d3accc47d5b1910d8bc31e1c5f28655dd9aecca3166f9cf3df79f892430e", + "phase/calls/confirm-fresh-11-hi-m1-source-grouped_lossless.receipt.json": "836ae21435fac43c98f8cb8693519f11ceeea6b40452b508a6d67f01e8266d1a", + "phase/calls/confirm-fresh-11-hi-m1-source-grouped_lossless.request.json": "844570eeb35d1df83ec050e6dd49d8c969ad32213b3ab7e767b871535c63d4a9", + "phase/calls/confirm-fresh-11-te-m0-source-chronological.dispatch.json": "3c4c2116e6c0bbdc9f3f8bff8c16e635d767051c976e4cd2c85a13ae6d102ec6", + "phase/calls/confirm-fresh-11-te-m0-source-chronological.receipt.json": "3e5e17f064325598ee8631174e1dee40e7736a1f4e3863d03d1ab5bd958b8971", + "phase/calls/confirm-fresh-11-te-m0-source-chronological.request.json": "20eac5c79ab10de74f4f6c886c75f11502098cf59320908aba31a38d4a25ebfa", + "phase/calls/confirm-fresh-11-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "0c85b1f5d7a8f411d9887ca4c8d7cccde838b7a357c57c1a32bef7be2e8f9aa6", + "phase/calls/confirm-fresh-11-te-m0-source-grouped_actor_prose_withheld.receipt.json": "17cad71e3d03dee5e383b94479ef704ad781082cd4dc36eeabed09b2218a8cdc", + "phase/calls/confirm-fresh-11-te-m0-source-grouped_actor_prose_withheld.request.json": "51f1cba31314840bc676890dd5a840b27f4527f3cdbd3648d3f30920458533d1", + "phase/calls/confirm-fresh-11-te-m0-source-grouped_lossless.dispatch.json": "2365fb3be7157398936f45ff2d149d240db1c12b9f2aaa39399e98f3d8a21bee", + "phase/calls/confirm-fresh-11-te-m0-source-grouped_lossless.receipt.json": "1b2a593e5868135df146d15a244e4bf4a4d54461894fc8694a4249c9df91093e", + "phase/calls/confirm-fresh-11-te-m0-source-grouped_lossless.request.json": "b49d1ca48cf39433c4e9f7adcc3d553b70992052cfdb5397d7d9bbcdf51b6103", + "phase/calls/confirm-fresh-11-te-m1-source-chronological.dispatch.json": "53a88a9bf71aecf183a42adc46e40be9b9a82b43947693663d47f8de0ae0b5cc", + "phase/calls/confirm-fresh-11-te-m1-source-chronological.receipt.json": "8e45ad5b42d05cebf152fac7d9db380573fc722561eba7719a1de613ec5a113b", + "phase/calls/confirm-fresh-11-te-m1-source-chronological.request.json": "a72bfc673c1d997c905b85f4da77f2a5ac5aaa07da4193db462508a6cdc74b14", + "phase/calls/confirm-fresh-11-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "b2bc41626ca32edc1f65eec7ac00430f52d36a8cf53b05234aec9025733b5341", + "phase/calls/confirm-fresh-11-te-m1-source-grouped_actor_prose_withheld.receipt.json": "0328bd580ff29dd1c36844ba8425716426421e80b4401001f0cce079c05c55b4", + "phase/calls/confirm-fresh-11-te-m1-source-grouped_actor_prose_withheld.request.json": "6d833158573ff57f08afd4c4389bcbb6f4577ceb306cecb6a7db224491c8262b", + "phase/calls/confirm-fresh-11-te-m1-source-grouped_lossless.dispatch.json": "14236b9158e8490074d0e05f34c9a6a9f2e74dd86ddc79fa0224aa8a52c087d2", + "phase/calls/confirm-fresh-11-te-m1-source-grouped_lossless.receipt.json": "6d3a5a3d143406865c8ef1f9cd747d160a487413ea8ae6e41c5a4c91c2c4be7f", + "phase/calls/confirm-fresh-11-te-m1-source-grouped_lossless.request.json": "ff5d41f9de57d2bbf352a7cf9e7fdc214813bd3a9d984e3a867592badbd8344d", + "phase/calls/confirm-fresh-12-en-m0-source-chronological.dispatch.json": "1135d6a0943e8739acd82a708cfd5dc106d4b3d380845e45d34a922e8d9c1e54", + "phase/calls/confirm-fresh-12-en-m0-source-chronological.receipt.json": "dfc630adcfde7d136974e4de61b6368ed6d0f31762972ae4be275575c953d3f4", + "phase/calls/confirm-fresh-12-en-m0-source-chronological.request.json": "7f5fe7ec8f6cbec84d49802508b31b7f8b0d2d8da89b693c9e27f2c181f3a460", + "phase/calls/confirm-fresh-12-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "41306acb9c65f1ec1c3bcfdb59d6de972bb7b1fb616aa76814e03819925ea9c9", + "phase/calls/confirm-fresh-12-en-m0-source-grouped_actor_prose_withheld.receipt.json": "59b1a8d39efb27b14d8ee50ebb4b6246f960a1e64a6344b2f488c6db931aed1d", + "phase/calls/confirm-fresh-12-en-m0-source-grouped_actor_prose_withheld.request.json": "39451294cfd4adcae9796811fb575ef37dbecaf275d1cb0016746ea2bfb759d2", + "phase/calls/confirm-fresh-12-en-m0-source-grouped_lossless.dispatch.json": "4b3aa1b290c20bb43a1b3661555e62a60be92046b109ff191bbfe0e2a0377ee2", + "phase/calls/confirm-fresh-12-en-m0-source-grouped_lossless.receipt.json": "d5c8c6f3758d8657532bf91cd62719344a92def1909821e5424fc9ffbed5fd0d", + "phase/calls/confirm-fresh-12-en-m0-source-grouped_lossless.request.json": "a489824ea97c6c609cbfb2efc5b2051d6c4546bf57680cfa79f070790b8b2ca9", + "phase/calls/confirm-fresh-12-en-m1-source-chronological.dispatch.json": "046fd872889ec1ad39d87290dbf7e5a2becf1ebda231a79eaeef1ec1dbc061df", + "phase/calls/confirm-fresh-12-en-m1-source-chronological.receipt.json": "b6dc56ac780c87d81c53548e98deb96b67d500a495494bcb9df4eb8135e03f62", + "phase/calls/confirm-fresh-12-en-m1-source-chronological.request.json": "49dbaae2e28c93a7b8cc19e9cc950002819e3f53c2968b55fac6de85a05f8811", + "phase/calls/confirm-fresh-12-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "77611eb838215ecc5cce041d089e3194f4af644cb4d103498d90346144281496", + "phase/calls/confirm-fresh-12-en-m1-source-grouped_actor_prose_withheld.receipt.json": "1f16e9220446990db3cff03f49003a81b498293e034d189f77f4a2410dc953a6", + "phase/calls/confirm-fresh-12-en-m1-source-grouped_actor_prose_withheld.request.json": "3435b3b2e77a89a6cbdbbf2ffb7d2953359df8b634a03ba2911f54c92299f542", + "phase/calls/confirm-fresh-12-en-m1-source-grouped_lossless.dispatch.json": "6fd11762b1b86df1f26ffab3319f10017d2242168eba22a19b989678eeb4fb0d", + "phase/calls/confirm-fresh-12-en-m1-source-grouped_lossless.receipt.json": "53cd14412b0dec1cea64a8cb4c3fb96054de642de57026dafddfc6a518c09a86", + "phase/calls/confirm-fresh-12-en-m1-source-grouped_lossless.request.json": "a63b94e929d793bfa224e8805b2b5881eb6a03358e606d7b36567079025af11b", + "phase/calls/confirm-fresh-12-hi-m0-source-chronological.dispatch.json": "096d23cc8558eca8113999efe70d09c632f99202f3c2292f4a454e22fb9eefb3", + "phase/calls/confirm-fresh-12-hi-m0-source-chronological.receipt.json": "e08dc24bfc45ffbf54bd5de5c516685a8aa534b17992a865165be7eda4bd34f3", + "phase/calls/confirm-fresh-12-hi-m0-source-chronological.request.json": "cecf8873830eee24f655b8cffac47883fc18b0bc037b80f46935d2bd7237d53e", + "phase/calls/confirm-fresh-12-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "103ce7d30a51564c9a317c8dd752919defbce0dd56cd8b84e7b1e1cf61c1ffc7", + "phase/calls/confirm-fresh-12-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "f19df5d5d37a474635ac5ed7467397efcdf135cb68fbc3652a12ff483bd9afd3", + "phase/calls/confirm-fresh-12-hi-m0-source-grouped_actor_prose_withheld.request.json": "39e12275cf95a2be5b804661662391a4ad93f5a87467173e2d319c0efbf203e1", + "phase/calls/confirm-fresh-12-hi-m0-source-grouped_lossless.dispatch.json": "30a8b04ee565b789a42c6b742d790adc26a2f88e2e71ca00960d0ad1c86bdcc0", + "phase/calls/confirm-fresh-12-hi-m0-source-grouped_lossless.receipt.json": "6da15fd6c4ef4c8c0c7078870e3e7e583be25e4309eede5419f1ec2ed6fc7d79", + "phase/calls/confirm-fresh-12-hi-m0-source-grouped_lossless.request.json": "a556b25d2e934f1b94829da1a1fb45b660b9fb46d74118217074a47423fcc57e", + "phase/calls/confirm-fresh-12-hi-m1-source-chronological.dispatch.json": "24465693515dfe1f915d1b82812f08b44c1a34427536418fcf6fc90c666a495d", + "phase/calls/confirm-fresh-12-hi-m1-source-chronological.receipt.json": "dcfc78298fc6707f5bdd0c29f74c189db09d40b62ff03a71b90ee123e6ba2f03", + "phase/calls/confirm-fresh-12-hi-m1-source-chronological.request.json": "1d7b6cb1f09bcdbd1c29f24a8487e7c8ec06e8f43b2d5b0ddcc41144f70d59b7", + "phase/calls/confirm-fresh-12-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "ef2c3abb30700199f94e71a447a3529d2f7ad6e6fdb7c1ff4328971cbaf10610", + "phase/calls/confirm-fresh-12-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "c393a4cc402301c1074bdf6c4ed412eefd1f8f09b17bc44672e147b97620f0a7", + "phase/calls/confirm-fresh-12-hi-m1-source-grouped_actor_prose_withheld.request.json": "af9d8cbcd403d6b44c7186bcfcd58518927a9cda93efe58bfd5c2bc3e2129243", + "phase/calls/confirm-fresh-12-hi-m1-source-grouped_lossless.dispatch.json": "ceba7ffee07c2f07e196660571fc275fcc49640603d68d4134a864f9a0345abf", + "phase/calls/confirm-fresh-12-hi-m1-source-grouped_lossless.receipt.json": "e18ead61fdaec631c774f80a0923899baf3fa34ff472b2d1b008eba8add90eea", + "phase/calls/confirm-fresh-12-hi-m1-source-grouped_lossless.request.json": "0d2e4dcd6068d19ec2a6e0b19981277f669dbd7bb92d98a283551f811c9fe858", + "phase/calls/confirm-fresh-12-te-m0-source-chronological.dispatch.json": "4cfbb119052aa06b8b14981584aa0f8d9229344b805f8ca83a660f5a41222ed4", + "phase/calls/confirm-fresh-12-te-m0-source-chronological.receipt.json": "2c3ed4fd876bf0377409c9b7c2d579ba26214fb53dac91b1d5d286af7ea78019", + "phase/calls/confirm-fresh-12-te-m0-source-chronological.request.json": "902e7e84ceacb0ce386aac90914c4dc52fc72a3687aae931ac7d568005b19f56", + "phase/calls/confirm-fresh-12-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "e29b2205440ea31ee39539db4614150fb83eb16f7d90ce64abee65bba6ee3a8a", + "phase/calls/confirm-fresh-12-te-m0-source-grouped_actor_prose_withheld.receipt.json": "9466a5136b23f38ad0032f849e92b6bd3c96843991376161290f8f63e51412a1", + "phase/calls/confirm-fresh-12-te-m0-source-grouped_actor_prose_withheld.request.json": "ea54e534fec02610d3538daf743dc35606327d7e38efb15fbb9c4fd9bc3c359f", + "phase/calls/confirm-fresh-12-te-m0-source-grouped_lossless.dispatch.json": "f38a6381c3aa95c927250c0f33d27f8b2e33932f531efbb16bcd27a510bbae97", + "phase/calls/confirm-fresh-12-te-m0-source-grouped_lossless.receipt.json": "c8f7d96ed6316724f67b10d4ef2913e98f72540d1bb43b6e3e1249b8aa7d2087", + "phase/calls/confirm-fresh-12-te-m0-source-grouped_lossless.request.json": "a0ad4b25883a522921c8e1a35f3841ff11d9e8ad40497ba03452b02c46758d64", + "phase/calls/confirm-fresh-12-te-m1-source-chronological.dispatch.json": "b83a03c7a969cf584785d691124b40c057c04065f6f7aa2a7fde2356c6e1035a", + "phase/calls/confirm-fresh-12-te-m1-source-chronological.receipt.json": "29a5a99597d978437cb15a7c1ff1dee0bdcf04eb128344cde604c798933bbe17", + "phase/calls/confirm-fresh-12-te-m1-source-chronological.request.json": "863d7e2cc1a6aaf71fb7b0393524aa200176516c018857f2f83482aa87eac8f9", + "phase/calls/confirm-fresh-12-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "6f3c4c4f82337a607875384546f48a8281aec6ea0e472cd58f98b8b609a4e344", + "phase/calls/confirm-fresh-12-te-m1-source-grouped_actor_prose_withheld.receipt.json": "bcdda8b7692b2178abce0d77ef4ce925893b5a0c51b6a5fa9aefc83810d51eaf", + "phase/calls/confirm-fresh-12-te-m1-source-grouped_actor_prose_withheld.request.json": "58201859e78aae574e32995783ad7e486e5d5b981d399003d0e8df6c2ac433fb", + "phase/calls/confirm-fresh-12-te-m1-source-grouped_lossless.dispatch.json": "85e651c3f92687eeed4a8435af06e8174cd6748f4a6114b4ecff9ff7d3161b6a", + "phase/calls/confirm-fresh-12-te-m1-source-grouped_lossless.receipt.json": "69c42598d8730d940dfc81535b217de526dc6c364c5cedaa5f99e0aa7ae92aa6", + "phase/calls/confirm-fresh-12-te-m1-source-grouped_lossless.request.json": "1b2eb3754074843fd43c39ccf9352612623a2e48b9c85662a3726ea1a6964140", + "phase/calls/confirm-fresh-13-en-m0-source-chronological.dispatch.json": "5fef8ed4ea6f4900341c336d534ecac30e9b5061762613f7b2e69ea01446cc71", + "phase/calls/confirm-fresh-13-en-m0-source-chronological.receipt.json": "346c75390d937acd3c9a8595db24e8e0539f24d7b155e1b57838607884b6e47f", + "phase/calls/confirm-fresh-13-en-m0-source-chronological.request.json": "158cd6269b7d56c78e2e85456d78c61323925f4bfc1aebd1dcbc471b71d4f3bb", + "phase/calls/confirm-fresh-13-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "6c3a0519b10e671cb9c8e4d53eed490571ab8cb59a5aeaed9a275e542cffb42c", + "phase/calls/confirm-fresh-13-en-m0-source-grouped_actor_prose_withheld.receipt.json": "af1d2af5ca02c1b4fa1e836ab1e384ab2b2551d19d9cdef8925679d186580d2a", + "phase/calls/confirm-fresh-13-en-m0-source-grouped_actor_prose_withheld.request.json": "d16b1fce362726bfe9f99729b626a57ecb7eccb393d5e34224c2ca7406f8f8f3", + "phase/calls/confirm-fresh-13-en-m0-source-grouped_lossless.dispatch.json": "e2cda7ccaf0749321fdd2319a4e5978a67833c983fb655f413f3da1e85f1ecc3", + "phase/calls/confirm-fresh-13-en-m0-source-grouped_lossless.receipt.json": "0c57204141634f02165db629ddc7741ae3eebb6e8e19716db5cde82b68617caf", + "phase/calls/confirm-fresh-13-en-m0-source-grouped_lossless.request.json": "d16b1fce362726bfe9f99729b626a57ecb7eccb393d5e34224c2ca7406f8f8f3", + "phase/calls/confirm-fresh-13-en-m1-source-chronological.dispatch.json": "b1909d602773168fff626b0a7386d369f6b03daf0c8d1588294a448b7514455d", + "phase/calls/confirm-fresh-13-en-m1-source-chronological.receipt.json": "24866ddcddca7dbe7b80aa27a9f7ef34c0dcb4cfa7197fb3d5dce26b77ab4252", + "phase/calls/confirm-fresh-13-en-m1-source-chronological.request.json": "e1fd68466d580d3a4e04de6e3220fdccfa409f6908981edf4524eb7ea47e5446", + "phase/calls/confirm-fresh-13-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "72361220203b1aa330ba738b2f6dd1847358894667c45980f274e0979f03cc51", + "phase/calls/confirm-fresh-13-en-m1-source-grouped_actor_prose_withheld.receipt.json": "21ca80526500d175082c02722c869028273e856e33c063aaa54284e5845c8bda", + "phase/calls/confirm-fresh-13-en-m1-source-grouped_actor_prose_withheld.request.json": "e6d7dd8cc5abab8668c9cb71e026910c4d9834b05c4b05b240feab11869283f1", + "phase/calls/confirm-fresh-13-en-m1-source-grouped_lossless.dispatch.json": "996ed58e6f77f8de5a19e44b82dbb1fea2a1d818d135fc8bfcca13001a75f5b1", + "phase/calls/confirm-fresh-13-en-m1-source-grouped_lossless.receipt.json": "f1f17b4b883de31c9fc780dc6bdb4a9b74f1df6ef24e82c4ef2f0c8800758ffc", + "phase/calls/confirm-fresh-13-en-m1-source-grouped_lossless.request.json": "0759094895e0501ab5460f75368411fe0aa2088ad1dd0d58b700e0f149b54e4c", + "phase/calls/confirm-fresh-13-hi-m0-source-chronological.dispatch.json": "93cbdb855914498c53ee8c8961a677f344db15d01daccaeb8f1f22002357afe8", + "phase/calls/confirm-fresh-13-hi-m0-source-chronological.receipt.json": "de632e47357ee6044575cac15dd7f4404c4d3df248f0cbf3ab76e9165fc55501", + "phase/calls/confirm-fresh-13-hi-m0-source-chronological.request.json": "fc7411ceece30e5e22351c005b36c300661e540510c42deb2c40da093b04f126", + "phase/calls/confirm-fresh-13-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "ec0c019ae94de8d41df9ad0b3be857fec199d4499bb191a8cf2293925cf3f752", + "phase/calls/confirm-fresh-13-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "9ea76e1cb19190c2865c8070c91c07d15a5e7dbd32b11858d3ac3315b4215d97", + "phase/calls/confirm-fresh-13-hi-m0-source-grouped_actor_prose_withheld.request.json": "9926bf90d71ea38273c86a958cf9167a15a7d2cf2973179ca30bde7cf6fe8e10", + "phase/calls/confirm-fresh-13-hi-m0-source-grouped_lossless.dispatch.json": "3717acdaf6c60586a928ed4a0ac4bf5a9f965bb7da54319ea76438fe09a6aeca", + "phase/calls/confirm-fresh-13-hi-m0-source-grouped_lossless.receipt.json": "28302f28bc5521d9efb4afc03b55e4a98ca1f99f36c1943489e9ad59738e2011", + "phase/calls/confirm-fresh-13-hi-m0-source-grouped_lossless.request.json": "00e5bee14efd9a9361b267922e8ee6e597735ed154bd641a367cfed68533227b", + "phase/calls/confirm-fresh-13-hi-m1-source-chronological.dispatch.json": "3eea3ea28d3e8437be13a350dd21407594fa8ea2f82610047f121667355144b0", + "phase/calls/confirm-fresh-13-hi-m1-source-chronological.receipt.json": "b6af760c74d3ea233bfaf9be4485937c0fede90b4de79d6d8007ec8389778664", + "phase/calls/confirm-fresh-13-hi-m1-source-chronological.request.json": "05ec3c3a447a63aa6aaf925d751685df56b0afaffcae15686de373f34019849d", + "phase/calls/confirm-fresh-13-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "fbf8b3178d3be74a2dd8098ab34ef766e87cc18b954f5292bf3cce434816d143", + "phase/calls/confirm-fresh-13-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "89567073e2fef3e6fce6185ff9ce3ac691465c4ef34f2f0a38afbaf0ed8d0a32", + "phase/calls/confirm-fresh-13-hi-m1-source-grouped_actor_prose_withheld.request.json": "61eed370d1935f1dd6e9c6eef4fdd0c6674cad10fe45d76696a39c3b0558a158", + "phase/calls/confirm-fresh-13-hi-m1-source-grouped_lossless.dispatch.json": "225b5f6d64f5d0cb59f57fb1b4c11a08509e99cbbb2297046bcf0b6d4161865e", + "phase/calls/confirm-fresh-13-hi-m1-source-grouped_lossless.receipt.json": "64a52239e94875898b8fbc56f3ce33ebea3f399a599ba1cdb72555aa364c6d4a", + "phase/calls/confirm-fresh-13-hi-m1-source-grouped_lossless.request.json": "2139b00ca97353b4290f69d43671b39ff7f77f29c306454b1a7630cdd74e8b74", + "phase/calls/confirm-fresh-13-te-m0-source-chronological.dispatch.json": "b4100c4371edb1e389cb6f4edf377478ba3dfd0e5b0478135616556ec16cb2ca", + "phase/calls/confirm-fresh-13-te-m0-source-chronological.receipt.json": "99980be6e640266a9357bf50a10be12f19679a701abdb1a39e77f09a850ba7f9", + "phase/calls/confirm-fresh-13-te-m0-source-chronological.request.json": "9af090a2a2d34e11833813b6ce9dc0238f6bb248dcc4d32cdd3ca31d341efa76", + "phase/calls/confirm-fresh-13-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "ecc1cf6cf26499b36e2736c61353a8038f9cbd95012cea20ad2e6d8ed0b57ada", + "phase/calls/confirm-fresh-13-te-m0-source-grouped_actor_prose_withheld.receipt.json": "ca059767ab5d2eca737ff6e379c14d6be75310895cf2585783150097f5fd1c84", + "phase/calls/confirm-fresh-13-te-m0-source-grouped_actor_prose_withheld.request.json": "d5d101b8dbfc516d08d2a537cb3b1b039c3c5e7c6c65552b5ed8d4756a01c0c0", + "phase/calls/confirm-fresh-13-te-m0-source-grouped_lossless.dispatch.json": "866f96eb3374a206ea4c72b6f8a4af889d7757eef44c6372db0b0432c8e31827", + "phase/calls/confirm-fresh-13-te-m0-source-grouped_lossless.receipt.json": "9fe970b7988c6b47ff50b7044aa4cdf60e123ce4d98764b9ebbc978dfd2e9a53", + "phase/calls/confirm-fresh-13-te-m0-source-grouped_lossless.request.json": "e7993a137dde91bab22d5e3b4a17849ce75365f73f165ac8c94533757f36ab76", + "phase/calls/confirm-fresh-13-te-m1-source-chronological.dispatch.json": "ebf154a36bf4b3988ffd8ced4acafd9ab844aa8717c0b377e31fa4a6b8c4544a", + "phase/calls/confirm-fresh-13-te-m1-source-chronological.receipt.json": "309c4279e3bd7c3e13718aa2f712305183987e21e6d2cfe103df7e696af28a6a", + "phase/calls/confirm-fresh-13-te-m1-source-chronological.request.json": "4ffa360cd6036eead02631d6a791fd2c78f7d0391d208e8351084bf415bfee5f", + "phase/calls/confirm-fresh-13-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "60895bcc8a89d48db7c5c5b5f63b03fe00f10729de04bca94d2426b030602270", + "phase/calls/confirm-fresh-13-te-m1-source-grouped_actor_prose_withheld.receipt.json": "900c0a9944791107982356042bdc198041fbcf101e572f21cf17fcfbd6127691", + "phase/calls/confirm-fresh-13-te-m1-source-grouped_actor_prose_withheld.request.json": "21a44ef36a317c709ef54786447c4f97d49c07e2601a03d232090e536cafe76a", + "phase/calls/confirm-fresh-13-te-m1-source-grouped_lossless.dispatch.json": "df87d7d85fcdf0f1b245302665324e7aeb9a8c9ed0dd5abdd60f2e3cd57bec0f", + "phase/calls/confirm-fresh-13-te-m1-source-grouped_lossless.receipt.json": "004d3865b47750a638d9aaf65c84fe914c75c44db30e4d308b7bc57ae436270d", + "phase/calls/confirm-fresh-13-te-m1-source-grouped_lossless.request.json": "2ece3179ae9869c07efd3faf65ef3f748f289b0d3d7d4211a098d0310e8a2557", + "phase/calls/confirm-fresh-14-en-m0-source-chronological.dispatch.json": "4f6e8d48d5790a42aa1b556eaaab01542094f60b38b1bd5d0bfcc11bd154f0f9", + "phase/calls/confirm-fresh-14-en-m0-source-chronological.receipt.json": "647fe1b912d775d5a6d8b54dd7478d97fbe2d9f9c9cf54ae9b44b5de97b5bf4a", + "phase/calls/confirm-fresh-14-en-m0-source-chronological.request.json": "c51bbd52bfb885a2686df2fed400cb51e9a92d3c2538571c072ae2f0141ee36a", + "phase/calls/confirm-fresh-14-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "2a75afe1958bca73eef43d2233e78a5a32d197419379acee7fa121f406696a52", + "phase/calls/confirm-fresh-14-en-m0-source-grouped_actor_prose_withheld.receipt.json": "26737730820a8bae29ff8880558e3b3e1ae5a31ad2e0cee92c6a5845a31fa240", + "phase/calls/confirm-fresh-14-en-m0-source-grouped_actor_prose_withheld.request.json": "817a3df90f19f4e0b5376568e11c222b524e1bf38038d3b1af2d173aa189fc78", + "phase/calls/confirm-fresh-14-en-m0-source-grouped_lossless.dispatch.json": "ceae57daeb1c9c1476df8f4d4fd1d738221a776cdd752996a523b0c12d8a1e37", + "phase/calls/confirm-fresh-14-en-m0-source-grouped_lossless.receipt.json": "6ec9b1f420675180ecdba514c94a432b5b9512876a1ebe767b927391422870b9", + "phase/calls/confirm-fresh-14-en-m0-source-grouped_lossless.request.json": "99ec60a4a10a17b2247fd37f9c2eeac5a08b01b622c588edf594bf470cb8ebfe", + "phase/calls/confirm-fresh-14-en-m1-source-chronological.dispatch.json": "f22e804bfc2dd850fe5061dbec28789a6f9d86cf815f65bf544fc106b2138efe", + "phase/calls/confirm-fresh-14-en-m1-source-chronological.receipt.json": "0043117e408dde3d9d0b6c73bd5d2cea7a5e01846bf50c2a9e0910bd703acd1d", + "phase/calls/confirm-fresh-14-en-m1-source-chronological.request.json": "d86d45c7231f9028c9b0ba723809d65d245757407c9b8325f95dddce5e8d08b1", + "phase/calls/confirm-fresh-14-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "dc3236788ee417a7f22cb0be4ecc202c2e803596a1cd5fbcb4fba7e725a3afbc", + "phase/calls/confirm-fresh-14-en-m1-source-grouped_actor_prose_withheld.receipt.json": "89de2edecdcc397d755efb0cb6cf23fefbec5d632ef4fd9df380bde47c09649d", + "phase/calls/confirm-fresh-14-en-m1-source-grouped_actor_prose_withheld.request.json": "fbd6121721b70f2d7eec40fe626b0f4d8c0ee7808d0a85a060e9b95cd0c460b8", + "phase/calls/confirm-fresh-14-en-m1-source-grouped_lossless.dispatch.json": "e070541aa5db4ddc3ebdaf480ce15397d6094f85b6216bc94ecad869e9f7e716", + "phase/calls/confirm-fresh-14-en-m1-source-grouped_lossless.receipt.json": "fa0d95a1f792d771ac7869d9e9d72686c346a8099994895233a7077aad6d03f4", + "phase/calls/confirm-fresh-14-en-m1-source-grouped_lossless.request.json": "38935c2fb26c5f41b9a6ad1721184d72b2360ac727afd2f6391f73df5a3f6c1b", + "phase/calls/confirm-fresh-14-hi-m0-source-chronological.dispatch.json": "e8284aa44debcd7b761ce2f29bb7a795a154151f65d479657c1049852a351c26", + "phase/calls/confirm-fresh-14-hi-m0-source-chronological.receipt.json": "6c65bea64a28a05272391915d8c474ef92fab4ddc129a037d2fc640882f9072d", + "phase/calls/confirm-fresh-14-hi-m0-source-chronological.request.json": "55acce10fac9b566213726afe079ce32c0f9092e2f5a1002e6c9c6dcdaaef2da", + "phase/calls/confirm-fresh-14-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "95f926811facc22f5a3ef3f3bd3dfeb6fed0badf6663681d034fc47c91c6b1ee", + "phase/calls/confirm-fresh-14-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "2a23312b84fa6f1fe647df4858e38db487d01fe8bd3c53af9f8ae0d89c90652e", + "phase/calls/confirm-fresh-14-hi-m0-source-grouped_actor_prose_withheld.request.json": "3be0dfdc7647a7aea69a1328e6c6aa47325e5e2283c5ccae514953d360d38246", + "phase/calls/confirm-fresh-14-hi-m0-source-grouped_lossless.dispatch.json": "44412fa0beef0676cb194eb7eb6a7d4a01ec92d18195dbbe62072a7abd2a7862", + "phase/calls/confirm-fresh-14-hi-m0-source-grouped_lossless.receipt.json": "f217a91fd99f22cdc86e81043db3501207aac5722665c462aa39ab2eb2240897", + "phase/calls/confirm-fresh-14-hi-m0-source-grouped_lossless.request.json": "230e7cda3a9e342684c7fd64cd8f2e12871cf9426bf5eedbae097a475dd5d293", + "phase/calls/confirm-fresh-14-hi-m1-source-chronological.dispatch.json": "0f7cbdbcf9fd45829dc920fb1f2a8d6f307c88828bee5b87c08f4a5a0fe82a54", + "phase/calls/confirm-fresh-14-hi-m1-source-chronological.receipt.json": "96d9aa6cb0d0928fd3e5e2ab5141970a303b6b2f2466aa67fa9887ce627ac39f", + "phase/calls/confirm-fresh-14-hi-m1-source-chronological.request.json": "b7881558cc25e5b062ad7dafa4f74de12a7b24b66e01d84e156f375d63d683a1", + "phase/calls/confirm-fresh-14-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "bc6ef7313f803e1af14bbc56ae387b4d712913185fa7ddf608fd508e8214e293", + "phase/calls/confirm-fresh-14-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "a9b9edc695db2e69e92861a6deb42829dab7865d3cd2fdef7d68e8cdb2eba358", + "phase/calls/confirm-fresh-14-hi-m1-source-grouped_actor_prose_withheld.request.json": "396d5e2106e72251e53e8d5821230c701ebb7248d6e764dfb0196972fff042f3", + "phase/calls/confirm-fresh-14-hi-m1-source-grouped_lossless.dispatch.json": "b473945e9d32221db081582b63c165bc494b301172607f97ae3937e589ef84d4", + "phase/calls/confirm-fresh-14-hi-m1-source-grouped_lossless.receipt.json": "dc55f01c26fbd6a1f3ade875f69e4a519c3930b6f1637f8f810f0e7d37aac599", + "phase/calls/confirm-fresh-14-hi-m1-source-grouped_lossless.request.json": "d2251628ebc9ebf8b8bab5a2ee54f6d4e47fa5f096c10b270f2adb63a5174232", + "phase/calls/confirm-fresh-14-te-m0-source-chronological.dispatch.json": "d9a3f56a5b71511a0bafab5c64004ef6f53ee40b5ce8199c13d8b45fff945c6a", + "phase/calls/confirm-fresh-14-te-m0-source-chronological.receipt.json": "b6bead11ebf3cdeeb2edfef06f6bf6dae963b56c680a024ce812e55683586873", + "phase/calls/confirm-fresh-14-te-m0-source-chronological.request.json": "21fae0b9038bf61bfe59b282469454cf4f57454c5267a771cc7bc541e686b702", + "phase/calls/confirm-fresh-14-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "46d63b3e1fbe50115b7ecab360ad119e591f447e878c0177e3315e72c7e3fd6f", + "phase/calls/confirm-fresh-14-te-m0-source-grouped_actor_prose_withheld.receipt.json": "51426b515cf7faedf6f6cd515f2548f130a19f9051a9972e075ecbad43385eb2", + "phase/calls/confirm-fresh-14-te-m0-source-grouped_actor_prose_withheld.request.json": "52e1072d0df123bcf23013b764f20a372ee3277c3af2504cb8ce5dbac1047e27", + "phase/calls/confirm-fresh-14-te-m0-source-grouped_lossless.dispatch.json": "f137b7c24805acd569362250679e719369646bc7218fa53f6c0bc57683e19a4b", + "phase/calls/confirm-fresh-14-te-m0-source-grouped_lossless.receipt.json": "ccb873d1491fa455b34762bd0ab982660a92c026b600cdada808d0da7f0cffd7", + "phase/calls/confirm-fresh-14-te-m0-source-grouped_lossless.request.json": "7f1c44fe99dc4aa807abb59004edef9e33c11c482c05d00975bccdbc83000c3f", + "phase/calls/confirm-fresh-14-te-m1-source-chronological.dispatch.json": "d7309b0e9f3cfb8bed7e4004b847f166e7884e4de34b3e36309992846227ffc3", + "phase/calls/confirm-fresh-14-te-m1-source-chronological.receipt.json": "a3b789674a59ac71358ce1f1f0fbc0077a207aae814ae38844372745b5457790", + "phase/calls/confirm-fresh-14-te-m1-source-chronological.request.json": "7a96a5e0219afba3bb8c7da2ad2943d505f6b591437be5756d7d7539b549119f", + "phase/calls/confirm-fresh-14-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "2d463d5139bb0c60cb43261b96b6749840a524b32bc3f8438df000184ee04607", + "phase/calls/confirm-fresh-14-te-m1-source-grouped_actor_prose_withheld.receipt.json": "0a8d8a462fe56bd62ce64c75215870246bbfcadcce96e860133f2cebbb57c588", + "phase/calls/confirm-fresh-14-te-m1-source-grouped_actor_prose_withheld.request.json": "fcf8f791b3b754a778557f7e4455736a44b9a82c2ceb2e32ef523fe2b6656d32", + "phase/calls/confirm-fresh-14-te-m1-source-grouped_lossless.dispatch.json": "99c64d9755adbf4e84c38ca81fc21a4510bc440660c248ed1a38a57a4f984817", + "phase/calls/confirm-fresh-14-te-m1-source-grouped_lossless.receipt.json": "c179985e9bdeb9ff79450bc1f978c475413711e2b5d9dd99d15b6dcc45259734", + "phase/calls/confirm-fresh-14-te-m1-source-grouped_lossless.request.json": "bb3792b808b7bd311cb4e651dbc2fa1d3eb176abfcc39a3077cb338796d8f265", + "phase/calls/confirm-fresh-15-en-m0-source-chronological.dispatch.json": "9c22202cbb88df80a9f3316263a159bdb49273ca3c9bf4d8d2d94dce18c625ac", + "phase/calls/confirm-fresh-15-en-m0-source-chronological.receipt.json": "3999ee223844e1d8ce3dee0a617991011f5db1279f4574784cdf48c2d520f0a7", + "phase/calls/confirm-fresh-15-en-m0-source-chronological.request.json": "36b1e13d273d049c1c91de4b70251c5d412a6939f5d700ced016e3ad91e13184", + "phase/calls/confirm-fresh-15-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "d7cb06a28f6bee7bc9ac7758f2faf051a799cf64895c07120b5ae3a05ac8f5e3", + "phase/calls/confirm-fresh-15-en-m0-source-grouped_actor_prose_withheld.receipt.json": "ec89890c29a4c32a1c28370e5178c8b3c54586e28d0411bafae2ecc6bb5b73ef", + "phase/calls/confirm-fresh-15-en-m0-source-grouped_actor_prose_withheld.request.json": "cfae3931a55972bfbf68a36b88ba8ced7e1cda5da42b9981373a605aabc8a774", + "phase/calls/confirm-fresh-15-en-m0-source-grouped_lossless.dispatch.json": "f7bc542cef30a6de153c50b7d66e84113c4ef5dc7076110a750d950c56ef6d91", + "phase/calls/confirm-fresh-15-en-m0-source-grouped_lossless.receipt.json": "b37a9bcc8bd1c532d44e167dfd66c9c493081c8cf2896b539c7c73fd02d41ee9", + "phase/calls/confirm-fresh-15-en-m0-source-grouped_lossless.request.json": "2b756b74e64c3c31fdc56c41e18cffc5cfda3520d2b2fd92e4467c027e1689b6", + "phase/calls/confirm-fresh-15-en-m1-source-chronological.dispatch.json": "a9dc7a2a79c943c8fc4a3b21d2cd1f7eeca64c85d313e05c91c0085f89e981df", + "phase/calls/confirm-fresh-15-en-m1-source-chronological.receipt.json": "6cfb01ed2c77c7ecd441e5df7d5525d48a95cf984a6eb5eb39d330e62b11dbda", + "phase/calls/confirm-fresh-15-en-m1-source-chronological.request.json": "eb746f3821ff43e9c1333554e26c811c115addc32a2b06ef59470a479440aa7f", + "phase/calls/confirm-fresh-15-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "09e1e8b945b0082e9525a344f5175cc30c3b11fe11cd6a73d4ab27e9b45915bf", + "phase/calls/confirm-fresh-15-en-m1-source-grouped_actor_prose_withheld.receipt.json": "e52c43b622e8b08390e62838e9efc0114989f240d7b61ee9208b1dca51e609f2", + "phase/calls/confirm-fresh-15-en-m1-source-grouped_actor_prose_withheld.request.json": "ee762b208335bc5121ae8f8bf8971ee21f20262fa3ea8c60159192ab65e43686", + "phase/calls/confirm-fresh-15-en-m1-source-grouped_lossless.dispatch.json": "a3047d37138ecd69be228a7ea8d1752fbd0536a81266308fe0b278a0b5fc3c04", + "phase/calls/confirm-fresh-15-en-m1-source-grouped_lossless.receipt.json": "2e5582aca58d610ce634b4499bce24820724c00df032b756fc55a4fd74b11428", + "phase/calls/confirm-fresh-15-en-m1-source-grouped_lossless.request.json": "2f85b2f32af8a7745101d995353e5f5a5a926f9fb3c9c399775067afbb2c5d40", + "phase/calls/confirm-fresh-15-hi-m0-source-chronological.dispatch.json": "d46dc57507dbec257dc054e2520a972ae9fe117a58c8a2bb40881f202feef2bf", + "phase/calls/confirm-fresh-15-hi-m0-source-chronological.receipt.json": "4b0048a4698578dee33943831667e4de0d829e99ec52e4e612c02567b515a024", + "phase/calls/confirm-fresh-15-hi-m0-source-chronological.request.json": "22f29e26cd38210ecdf53f46effbb5a6a51abf06d811ff6812002ca71e2f16a3", + "phase/calls/confirm-fresh-15-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "2a5762821caca7d7be96c3dbe9f20effe6a220afb39635e9450302c285726ea2", + "phase/calls/confirm-fresh-15-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "12977760072bc2510a42f355abe4fd31ca6b74e6d333ca6a15017fb4f271f7f2", + "phase/calls/confirm-fresh-15-hi-m0-source-grouped_actor_prose_withheld.request.json": "fea9a0ff95b0926f0b7c053feeb34c95de7e0c1ac0acd1d033da202433aa26fe", + "phase/calls/confirm-fresh-15-hi-m0-source-grouped_lossless.dispatch.json": "866712ea7ce1641d4eba1011d3e56ee138b66a2f5e4f4c3c562583efe516a770", + "phase/calls/confirm-fresh-15-hi-m0-source-grouped_lossless.receipt.json": "61d5677e23e24cf8ad725ecae79174a83aaac004594b4c17ff69bea991f043f9", + "phase/calls/confirm-fresh-15-hi-m0-source-grouped_lossless.request.json": "045cbc13af41342a470d9023de08c35d537af12502e9e61c79020dca1073e56e", + "phase/calls/confirm-fresh-15-hi-m1-source-chronological.dispatch.json": "d62102072ada0bbe4453297a703c0329c7eaf5d5ceed78b399c29aa7fde412fe", + "phase/calls/confirm-fresh-15-hi-m1-source-chronological.receipt.json": "4f960eff642bb8567511d69a1f5448a03f6137ee65e02b4e67c6a0f43d3e1b2b", + "phase/calls/confirm-fresh-15-hi-m1-source-chronological.request.json": "971527b40f4f803173146ca594614b237cccaee0946fa29ccf5f33c8dc26ed9c", + "phase/calls/confirm-fresh-15-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "c4f1d0ea980ba60f2c1de30106d6abe641eda9d17fa0f297d674c23842a3d084", + "phase/calls/confirm-fresh-15-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "acaec1fbbb54cba395d5a1bd4af2c608f563dccbbea99d56fbe89f53e374cc2e", + "phase/calls/confirm-fresh-15-hi-m1-source-grouped_actor_prose_withheld.request.json": "126c607e63d910785c8ab6ee6dee897d68924132ca017a75b25eb8ff1e6299eb", + "phase/calls/confirm-fresh-15-hi-m1-source-grouped_lossless.dispatch.json": "330da64fa75e48b91b0fd31e04c98639ba4232b6f30663d8fa0cc045ef25fe99", + "phase/calls/confirm-fresh-15-hi-m1-source-grouped_lossless.receipt.json": "558acb9ab773a32b293518ff6c41c74110207c0198a030f2ac9ba178ce12f93c", + "phase/calls/confirm-fresh-15-hi-m1-source-grouped_lossless.request.json": "8636439685e25f6a5b17daf617e3228f4be31006dfc9b6787046ccec488ddbb4", + "phase/calls/confirm-fresh-15-te-m0-source-chronological.dispatch.json": "0c4c66ac5d47f25ff1438264e412f42faffb57fd42b26866f26a1a27d7f39643", + "phase/calls/confirm-fresh-15-te-m0-source-chronological.receipt.json": "211f559e2311f1f68f4b4189dad7bba4a842ac407b1b2009656b20f2abad338d", + "phase/calls/confirm-fresh-15-te-m0-source-chronological.request.json": "54b91220c6c36f72b3f2043ac209a09c9d679294f2b518e422c29529177f2d38", + "phase/calls/confirm-fresh-15-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "c59259aeb8cbd036271e4886f72821a81c6332f9556bf76bd89e21d1a119a529", + "phase/calls/confirm-fresh-15-te-m0-source-grouped_actor_prose_withheld.receipt.json": "da1477cb27dda90c34e9302afae8696a7a44fb7a30d74c9df47a8252451e6ed6", + "phase/calls/confirm-fresh-15-te-m0-source-grouped_actor_prose_withheld.request.json": "828a4c31cce4a502f2ad3efb42f1d66aec6b31d3163c899f8c054d6b08292424", + "phase/calls/confirm-fresh-15-te-m0-source-grouped_lossless.dispatch.json": "15ea9b11bd358cae17c629ad56a9472140511c00939829a05ccc62618bfac174", + "phase/calls/confirm-fresh-15-te-m0-source-grouped_lossless.receipt.json": "b7300990a5ba7e7696932c794dc89711de062d2f958fab3da5cb1e7397413107", + "phase/calls/confirm-fresh-15-te-m0-source-grouped_lossless.request.json": "c606d23415dcc42eea7aa5a82d40661d94955af3b56198b3b8a7734eee33a8cf", + "phase/calls/confirm-fresh-15-te-m1-source-chronological.dispatch.json": "3269e85c0eda9f01d1d9127dedc36fd0d7b4d15583a54e075f3735340af82557", + "phase/calls/confirm-fresh-15-te-m1-source-chronological.receipt.json": "0c3fbd054eaca5479df8adc03547117fbf73cdc2b062f5c22acd9b66aa324eab", + "phase/calls/confirm-fresh-15-te-m1-source-chronological.request.json": "df71f6b8de3106c09f00c8424dfe694c2396b1f8b1b23ab0a74491faec96174f", + "phase/calls/confirm-fresh-15-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "4674797981a019929390552075f6f4cd0c18d4e33707687dfd6878b3eb6b8f94", + "phase/calls/confirm-fresh-15-te-m1-source-grouped_actor_prose_withheld.receipt.json": "a94b195d5ef54b2122788ceaea00f81076c7848a94c8d1547b4b352af17c6cab", + "phase/calls/confirm-fresh-15-te-m1-source-grouped_actor_prose_withheld.request.json": "4537cb9b7953f443fc7a1b8e4f7975f9ed74404c8921281e123199aad24de418", + "phase/calls/confirm-fresh-15-te-m1-source-grouped_lossless.dispatch.json": "a96b6680119f663b96000545ff5abd60166007570c6b03384e5112f38a99da99", + "phase/calls/confirm-fresh-15-te-m1-source-grouped_lossless.receipt.json": "8a019af19461deb734c39b395f7e4603e6c1e08e7b16f31787b9e0ff98faa92c", + "phase/calls/confirm-fresh-15-te-m1-source-grouped_lossless.request.json": "71606109562ad088e89885c604e9a61a5bb9529fef76fdbf2766c35bfd8b8579", + "phase/calls/confirm-fresh-16-en-m0-source-chronological.dispatch.json": "595e58d4535eb961bf2cf0771006368d64ee7b128827e1ccce70f3da0b482b3a", + "phase/calls/confirm-fresh-16-en-m0-source-chronological.receipt.json": "945841d1b62ff6970a9f4bbb4798d4a33095ac15db1cf7647019161fec8254ff", + "phase/calls/confirm-fresh-16-en-m0-source-chronological.request.json": "734188ff2216fc9a377c2162a0f76cb6363bcf25e3ce25bd7da8e04809d5180a", + "phase/calls/confirm-fresh-16-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "59efcac46b95569132cb402a06d47e9dc9f5494068a9aa06ae8706c3b794ba26", + "phase/calls/confirm-fresh-16-en-m0-source-grouped_actor_prose_withheld.receipt.json": "9ca16e32a76f15a9f65f52b1c4d5c7bb48050f3dc6d0935c3b1261d1abfd1b88", + "phase/calls/confirm-fresh-16-en-m0-source-grouped_actor_prose_withheld.request.json": "ebbeff44143cca94378fd3ae74280c505a1ba29af9deee03ebaae1664c3843dc", + "phase/calls/confirm-fresh-16-en-m0-source-grouped_lossless.dispatch.json": "c2a6f35352550f30d3312b1a2422cf2ccb18589c9c39766957304b712ea52eb2", + "phase/calls/confirm-fresh-16-en-m0-source-grouped_lossless.receipt.json": "26b746635c1971e2efece67a60139366f23490ec340f923777631f69c58078ff", + "phase/calls/confirm-fresh-16-en-m0-source-grouped_lossless.request.json": "97ebaebc45edaa4bfa120497f3096f36384eaeb3e08384faafc6fca4baeae396", + "phase/calls/confirm-fresh-16-en-m1-source-chronological.dispatch.json": "927fac58d5b16c3b2e65199489180aef3edb5d32b3b3acb797ecb4d96585b26b", + "phase/calls/confirm-fresh-16-en-m1-source-chronological.receipt.json": "7f79b3e28c3da2c1f5d62bcae7c9e5103371cfa682bde190ca97e9d118f80f8e", + "phase/calls/confirm-fresh-16-en-m1-source-chronological.request.json": "73543516edd06f05fa1121064583306d258ace4a2e6e43bf6f54e2b9a7e813ee", + "phase/calls/confirm-fresh-16-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "99802d1c765a6d442d6a402badbd73ff17edd00f0114e4197d4f94f1c68c5b25", + "phase/calls/confirm-fresh-16-en-m1-source-grouped_actor_prose_withheld.receipt.json": "8bb33ba9cade45a0035996e0f683881542507c85e28d15e1338bf0ca6e7a1bc9", + "phase/calls/confirm-fresh-16-en-m1-source-grouped_actor_prose_withheld.request.json": "4ed8ea7e2b211c3458721710a73a069bc21aaed6dad8829c8170b415dc628fa4", + "phase/calls/confirm-fresh-16-en-m1-source-grouped_lossless.dispatch.json": "6b223b6cde60b065dffc89b1733e2c8cb619be340277a4736e2b4610683ab829", + "phase/calls/confirm-fresh-16-en-m1-source-grouped_lossless.receipt.json": "6e60b7b8a97d40ddd6bbd83d661c4957ce4de9a0185f3e8ce97a37136fe5e997", + "phase/calls/confirm-fresh-16-en-m1-source-grouped_lossless.request.json": "d24ac8643630b0862305a5131d11b839863eae75d3b7c239fa9b549e34487ff5", + "phase/calls/confirm-fresh-16-hi-m0-source-chronological.dispatch.json": "b73ae6781a8b3255b4ed3d9f426489dc93ed208e1f61781a1de453ae1126f74c", + "phase/calls/confirm-fresh-16-hi-m0-source-chronological.receipt.json": "25ba1bf5b62dc20b512bff118b4aeddd96d3cb200c94670196af45153d5b4ac3", + "phase/calls/confirm-fresh-16-hi-m0-source-chronological.request.json": "f6f138c4f90c3ac4e217be774e62c76e1ef1ebe8c5ff2150ea47f534c1f18d06", + "phase/calls/confirm-fresh-16-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "0740a193296c5cfb0b45380ecf149b10139e8931c346bad52382ab44d097cd77", + "phase/calls/confirm-fresh-16-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "07f172f064cefbfb17602c6b67086d3e4d022666db8caf0cea56887211e86af9", + "phase/calls/confirm-fresh-16-hi-m0-source-grouped_actor_prose_withheld.request.json": "dc2210b48b57a122b7c4a0ac64c1f600c202639a35ae07a5c9ebc86d96440bc7", + "phase/calls/confirm-fresh-16-hi-m0-source-grouped_lossless.dispatch.json": "6b9182f4ac90d5d88203f01da0b8539842d4ce58b1211407e1da346c93109a7c", + "phase/calls/confirm-fresh-16-hi-m0-source-grouped_lossless.receipt.json": "bbcf0b13632aa998db8b768bd82353139dd6af8ecb745c78805c6dc5df4a3150", + "phase/calls/confirm-fresh-16-hi-m0-source-grouped_lossless.request.json": "9c4330ce79dc49a0aacf42039356870a3e72cbf7218cef032751e7fab97d7285", + "phase/calls/confirm-fresh-16-hi-m1-source-chronological.dispatch.json": "71ef9f3904f1dc3df65b6e64ce4b4d0df60077875965af02902fbb0083fb71f8", + "phase/calls/confirm-fresh-16-hi-m1-source-chronological.receipt.json": "19632a0a239d7c891728e2c58e6c110afcf63f54354ac5c01bc266c194d19b56", + "phase/calls/confirm-fresh-16-hi-m1-source-chronological.request.json": "d24c24c529314f2466800239fa35595d6985567714abbc2cba557ecb16a30e25", + "phase/calls/confirm-fresh-16-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "3d2169ddca8f85b2a4491ba45f3dc68cd108308e3ecd9965712687780aa3f0ea", + "phase/calls/confirm-fresh-16-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "f0821ecb4ea6f780b020e71efcc91e8496ac815eca6bf59f5520ba051380b9e5", + "phase/calls/confirm-fresh-16-hi-m1-source-grouped_actor_prose_withheld.request.json": "40b00229848f76b9aab4bc73a7356efb3f604675982da26ab169e9fd9f07f764", + "phase/calls/confirm-fresh-16-hi-m1-source-grouped_lossless.dispatch.json": "2963587ea16e7c0199528b538379906a64b6ffecc42537c71b6f91cfef87101b", + "phase/calls/confirm-fresh-16-hi-m1-source-grouped_lossless.receipt.json": "d7356a799cf7cc0f73b6fbbc10e89da7a150c8dd624c845f685f3543f955031a", + "phase/calls/confirm-fresh-16-hi-m1-source-grouped_lossless.request.json": "2657161c465caea085f37871c35c2453eac992eefd15b9bda3accc9bff4fe69e", + "phase/calls/confirm-fresh-16-te-m0-source-chronological.dispatch.json": "199fb7b1cbc253a797f393e90bdccab756ae3a8be30f9bdc7220fcc701f0197b", + "phase/calls/confirm-fresh-16-te-m0-source-chronological.receipt.json": "47200bff5dfe4888dc3bf93eb3068228515de9f26d49f4c5f14c9996dff9843c", + "phase/calls/confirm-fresh-16-te-m0-source-chronological.request.json": "1453dae6065163817133a42769a1c6e4c0b7bce0a059e9606102f3296079ad82", + "phase/calls/confirm-fresh-16-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "d0898db0e463e3b6506b95776f11a2cb054c7ae347e119fc0a847ec31f67bdf0", + "phase/calls/confirm-fresh-16-te-m0-source-grouped_actor_prose_withheld.receipt.json": "6b8788cbb4137c8e127e689e79bbeb23c74fd994b33d83d77d675ba1c3aad913", + "phase/calls/confirm-fresh-16-te-m0-source-grouped_actor_prose_withheld.request.json": "efbd732215484538fe51a0db7eead2d845db354ce2e075885252b269968879b1", + "phase/calls/confirm-fresh-16-te-m0-source-grouped_lossless.dispatch.json": "faf36fcfee7df671c97a0faa7a281e050818ef5da6e0d1428829489c228218ee", + "phase/calls/confirm-fresh-16-te-m0-source-grouped_lossless.receipt.json": "8ded85bfd257e74941873a834ef982c92dbe6cc6b3afae6b50083fc257318178", + "phase/calls/confirm-fresh-16-te-m0-source-grouped_lossless.request.json": "77107af871e058d0bba09e5c7b71822b69fc938ec6a7db837e41057f28f0c3af", + "phase/calls/confirm-fresh-16-te-m1-source-chronological.dispatch.json": "257311f3cb3e03d5916f80c0e7476e1ed7bc475bf4f492da10dfcad1a4f4d0f3", + "phase/calls/confirm-fresh-16-te-m1-source-chronological.receipt.json": "a34864b3147bc5433e528310e51abf8831e5f27a3afb50f4af54513d7aaee271", + "phase/calls/confirm-fresh-16-te-m1-source-chronological.request.json": "58cd20337df8f6bbd563a5bf553a7d095947fe8a082834a05e59b2893d5afa22", + "phase/calls/confirm-fresh-16-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "6bed48c55e3d64fa2a9f2dcb43ec72febe91f649232399937c8a0a31881ca0a4", + "phase/calls/confirm-fresh-16-te-m1-source-grouped_actor_prose_withheld.receipt.json": "e0b63615481d0008e4b34934d975b72f6146428ca602436f71f931f03a560f6e", + "phase/calls/confirm-fresh-16-te-m1-source-grouped_actor_prose_withheld.request.json": "b75c685e07b1b0c87d03ad031a9e191f7cf1902a4f6356ad16eeddaf0a03b6c1", + "phase/calls/confirm-fresh-16-te-m1-source-grouped_lossless.dispatch.json": "8efce5d89ac334c73f71d6797cb57bf1e0dc43b056f666f71953a4aa36abfbfa", + "phase/calls/confirm-fresh-16-te-m1-source-grouped_lossless.receipt.json": "22d400b4a97f220318b5ebaa8393d1d5a15d659ef1bba2c44a598cf23d56665d", + "phase/calls/confirm-fresh-16-te-m1-source-grouped_lossless.request.json": "6c5b1cbbe1db3f6679a8290b6d6e35fbd9341d0918aef8707d232b62d8f22d80", + "phase/calls/confirm-fresh-17-en-m0-source-chronological.dispatch.json": "02ab4333a0b99297d6a516c07d344a52d1d82dca65046ac4d72166d364231d4c", + "phase/calls/confirm-fresh-17-en-m0-source-chronological.receipt.json": "8464c65d5db1959f62b4e1acb22e0d192077c68f03fe115d22c46b3c836246c3", + "phase/calls/confirm-fresh-17-en-m0-source-chronological.request.json": "b6820e2755eeaec0edcf87bcfac240b90a4ae620d9725f4352ec62e34b33132f", + "phase/calls/confirm-fresh-17-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "bfc42fd872002c34eb2634b9bbc3faeada01a066236c2e8e6ee3b1e901f77d27", + "phase/calls/confirm-fresh-17-en-m0-source-grouped_actor_prose_withheld.receipt.json": "49e02316ddf4b848aca168797301fb198c8fd73a8e7dd9e54cf26f16106b70d7", + "phase/calls/confirm-fresh-17-en-m0-source-grouped_actor_prose_withheld.request.json": "0a09f989d834833554035e31b86f33ac54aa972b6794bc45568f4f58f78f2f12", + "phase/calls/confirm-fresh-17-en-m0-source-grouped_lossless.dispatch.json": "c82de00f12d65466bdd4d66e09df2eb52ad901027c79aa3b6bf89bf385ea25fe", + "phase/calls/confirm-fresh-17-en-m0-source-grouped_lossless.receipt.json": "d6077fb24ca5bf649a2f4155d2e49ba3097dd4ff18fa02168113cacd23818140", + "phase/calls/confirm-fresh-17-en-m0-source-grouped_lossless.request.json": "269fbf6771a8cc47ea613e10791e7173c178ceb8ccce57985fdf0de0c7c5b2dd", + "phase/calls/confirm-fresh-17-en-m1-source-chronological.dispatch.json": "35e5e99f109a3d6d4cd8fa7e0805a9ecd7abf3f34da4c925c0e7bde9f57f3dfb", + "phase/calls/confirm-fresh-17-en-m1-source-chronological.receipt.json": "cfe6d80539aeca53e8c6e730500bf6574a774d91ea2a1d3ee11a4103d0d48fe3", + "phase/calls/confirm-fresh-17-en-m1-source-chronological.request.json": "a59e6a37b8c4d0cccefc857f1547377e371665252e9e36e440893f60d242118c", + "phase/calls/confirm-fresh-17-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "c46acdddb3e82ebf4b741931f2b02fa6da5c5b206e2a01064b314aa4f65a2c28", + "phase/calls/confirm-fresh-17-en-m1-source-grouped_actor_prose_withheld.receipt.json": "6c5a704b858804669775860a8bcd278c7a96983c45a106164eaa5953f13d02b4", + "phase/calls/confirm-fresh-17-en-m1-source-grouped_actor_prose_withheld.request.json": "39569aa94d53df44b70dbce2324dd43192f0c30f57f03812664fe1c86a78d5f6", + "phase/calls/confirm-fresh-17-en-m1-source-grouped_lossless.dispatch.json": "920dbb0f475ee8a540b6ee1f9b04d5d54d94de16681826cecb4242d981565409", + "phase/calls/confirm-fresh-17-en-m1-source-grouped_lossless.receipt.json": "f8fdca9a242a4298153ae07a1877bdc0c6097de572168f15802b634aa725ec08", + "phase/calls/confirm-fresh-17-en-m1-source-grouped_lossless.request.json": "fd338e744485ab3b7ecec86e4b61177aad74915b531c8d406594991c1b937850", + "phase/calls/confirm-fresh-17-hi-m0-source-chronological.dispatch.json": "e03a5066217740d57e9f5f1e3c43fd0298207a9373c06f0bba692013774d2087", + "phase/calls/confirm-fresh-17-hi-m0-source-chronological.receipt.json": "6dbd107208abc921fd51e7de68e777a9d2bf2734db197b2098048bea13f41bcd", + "phase/calls/confirm-fresh-17-hi-m0-source-chronological.request.json": "31bfd693d0d34efa4185b6475fcf0a19957071dff6beabc95f6d9b33eaaf135c", + "phase/calls/confirm-fresh-17-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "681834ae27d667d466a1ed576a845bd40fc137a3ac561213ac30ecf21e8788f7", + "phase/calls/confirm-fresh-17-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "0736abfb862db6ec2aae43f3f414d62dfde241f409cdc296a6242cdc3fc42954", + "phase/calls/confirm-fresh-17-hi-m0-source-grouped_actor_prose_withheld.request.json": "7e7cdfae02efa3ef1f12de026c4d611431532ccc60d8d2a9a02b869faaed45f3", + "phase/calls/confirm-fresh-17-hi-m0-source-grouped_lossless.dispatch.json": "425897b3d958bbaafd2bc48d5e96f7a80c372267b899cf1620684b43cb27d775", + "phase/calls/confirm-fresh-17-hi-m0-source-grouped_lossless.receipt.json": "b4c9d0c82e44024b2c38c9a58fac3618efb08d72d84178c8fda9b3b42f4f57f1", + "phase/calls/confirm-fresh-17-hi-m0-source-grouped_lossless.request.json": "842f80bd433cf339846947586fdd6c5f5cc388c6e3cf5ce30dc58bbd563c03d8", + "phase/calls/confirm-fresh-17-hi-m1-source-chronological.dispatch.json": "be3e49cc5529dffcfae4aa39cf05eccfe875a9315ae98b9d7ef5ce6165f266c4", + "phase/calls/confirm-fresh-17-hi-m1-source-chronological.receipt.json": "a35f8f6887d18553965633ae8fc31957712f4d645da73ba99d74aab7967f4cb0", + "phase/calls/confirm-fresh-17-hi-m1-source-chronological.request.json": "d1f83b42f10c9b7500dc429aa871c2af9763d7ceb1599c09034b4eb2d8d2745a", + "phase/calls/confirm-fresh-17-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "cbea7ff16d7f89dcd3991d410874b48f6979cfcce8f73a7051b9ae87a5812e40", + "phase/calls/confirm-fresh-17-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "4e8e1213b0a4466a273c597281bec0ae036d7cb63e52d598b8bd9ade95a58a22", + "phase/calls/confirm-fresh-17-hi-m1-source-grouped_actor_prose_withheld.request.json": "eccc8900c32b6a372a0873300279c1db744748deeb96abdd670742c6d6847057", + "phase/calls/confirm-fresh-17-hi-m1-source-grouped_lossless.dispatch.json": "da46d1034f041440df7609f8e7f1ca33d73043c38047797c53b1c8c00893b644", + "phase/calls/confirm-fresh-17-hi-m1-source-grouped_lossless.receipt.json": "e4d87dd082bcbc129abd7b2fc6d1e5bcd61b50f04f1525c5ed00045f6b3b9cc7", + "phase/calls/confirm-fresh-17-hi-m1-source-grouped_lossless.request.json": "e34eeaf30ecd4a5ffed5968f67a5389618ee82d91237391adb553cc6b5c89ab1", + "phase/calls/confirm-fresh-17-te-m0-source-chronological.dispatch.json": "6e6fce92940e7237791f20f75d2a9480fcd7d2a4f60860ba08d423a6de17adf9", + "phase/calls/confirm-fresh-17-te-m0-source-chronological.receipt.json": "75873761758cb09e6b5cd8189832c16e491430a135aefa39e4ba38c567e4250b", + "phase/calls/confirm-fresh-17-te-m0-source-chronological.request.json": "28ceff71292e4bf671677bb0e30e51db5299762cb2fe51679873e62125aa6b7e", + "phase/calls/confirm-fresh-17-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "a8f137f7e107ec9417e0a2e498e67baafb19effb4ba2ff91ecb7d30a2b18e1be", + "phase/calls/confirm-fresh-17-te-m0-source-grouped_actor_prose_withheld.receipt.json": "f5b26e6b465cbd28fbaef3c225bf153c852eaba17fb8e08d634a65e1f7a7566f", + "phase/calls/confirm-fresh-17-te-m0-source-grouped_actor_prose_withheld.request.json": "052d715510b79601f03a817c654dec458ab873295cc928de97a79df5a08429b7", + "phase/calls/confirm-fresh-17-te-m0-source-grouped_lossless.dispatch.json": "f319b871a3fe93f2de0bde21af934fccffc40c6772ac27ec884ea4caf3f58a57", + "phase/calls/confirm-fresh-17-te-m0-source-grouped_lossless.receipt.json": "c94513ce65bc575f9e2b9fbefbf5741b739ee2c2f5f4ea365a8b71fc8e9d1a10", + "phase/calls/confirm-fresh-17-te-m0-source-grouped_lossless.request.json": "afd8a3c0b1f34a5b804cef1a1c4ebbf42d3ac04947efd6b73d867acec61caf80", + "phase/calls/confirm-fresh-17-te-m1-source-chronological.dispatch.json": "3a750f2902331c3e1cd937d7ca87a4e9ba28e2cced630c09dc4a8bf0f98e7b62", + "phase/calls/confirm-fresh-17-te-m1-source-chronological.receipt.json": "5f7a1a8f3dc7359f25d8d47eafe78baacd59c263b111a499d26b1d1088226a64", + "phase/calls/confirm-fresh-17-te-m1-source-chronological.request.json": "eef7100e0e24727c12f56157662474693c1823030570eeda3dfbd73803a546e6", + "phase/calls/confirm-fresh-17-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "c375208949551a17a856fb224a6e6c46c8defbb125810bb49a57db8ddbb32ed2", + "phase/calls/confirm-fresh-17-te-m1-source-grouped_actor_prose_withheld.receipt.json": "c55a2283b48fbce858733902d0992743a4fca2fc728a215123579447dd925b1f", + "phase/calls/confirm-fresh-17-te-m1-source-grouped_actor_prose_withheld.request.json": "1dfc61d859b514a9370dda912a1fdf7c1784bb6bc87e2bd69355a7f163223f6a", + "phase/calls/confirm-fresh-17-te-m1-source-grouped_lossless.dispatch.json": "196fc29b4c894ede21a21ad59b8acf7c70b089ba5e1f9c592e538c088d93e3ab", + "phase/calls/confirm-fresh-17-te-m1-source-grouped_lossless.receipt.json": "2b48899079a40b09213a0e8331098cbfbaef07f7f896ed96d11ba8a3a7b05312", + "phase/calls/confirm-fresh-17-te-m1-source-grouped_lossless.request.json": "8c79b0e95fdb45dff5a75cbd2f4c406561f4adefefe8e1c531552e1cf624790e", + "phase/calls/confirm-fresh-18-en-m0-source-chronological.dispatch.json": "b7c75873c762273d068b922e6371a85b9bd2d266a1c899b4af5335bbb361c43a", + "phase/calls/confirm-fresh-18-en-m0-source-chronological.receipt.json": "e254092738553483e04cfa298dc1b69777f978d47ae93181a3fbeb57f602cf1b", + "phase/calls/confirm-fresh-18-en-m0-source-chronological.request.json": "ca795181781830a6f69994ed4e0d70231d0ecce59b46deca72976b30f5710f6f", + "phase/calls/confirm-fresh-18-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "c64819c81be26a9a6b31c7c6a5e3e45e8739c0682c43b784572ae5d102304503", + "phase/calls/confirm-fresh-18-en-m0-source-grouped_actor_prose_withheld.receipt.json": "14ddd58c35d8c36475f1112e6419a371a7018723be2b2eb8a8dad5a51fc39206", + "phase/calls/confirm-fresh-18-en-m0-source-grouped_actor_prose_withheld.request.json": "0b211ec2f8ff27f313eacc2a88e0c132b5c4ce752e31eedbb3317575ce1e8f08", + "phase/calls/confirm-fresh-18-en-m0-source-grouped_lossless.dispatch.json": "73829a08066a0eced25a89257bc371e893542cf8340915e394ed69b3809c05dc", + "phase/calls/confirm-fresh-18-en-m0-source-grouped_lossless.receipt.json": "f517e498252cbb90eb8000f4ea24993c2fa19ec0479d3492afae31e0a81252b6", + "phase/calls/confirm-fresh-18-en-m0-source-grouped_lossless.request.json": "a0d862d38c3e7cfe9651b5a821e3ea2a1ef6756a510e725acc07d2c9ed760fc4", + "phase/calls/confirm-fresh-18-en-m1-source-chronological.dispatch.json": "5aa2b6fb7bdb1b84593daf080846c4eb450a6f11a88346ef396de2a177e3531c", + "phase/calls/confirm-fresh-18-en-m1-source-chronological.receipt.json": "dd26b0242428f90607abc92fa6957675049d285461fbaa075e817838385c3e70", + "phase/calls/confirm-fresh-18-en-m1-source-chronological.request.json": "bba20f9c81ab806383d56bfc27b4ca4f540c4ca878d6ac75a79bea30ffd0b3a0", + "phase/calls/confirm-fresh-18-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "0f394db90a2c72e73e3e15b49b13b5950c303f57e03c5c35f785a8fb6bc3726c", + "phase/calls/confirm-fresh-18-en-m1-source-grouped_actor_prose_withheld.receipt.json": "1b6c1ffe92650fbf12174952942ede3cb4ed82c4425288c893daac1aec9b9450", + "phase/calls/confirm-fresh-18-en-m1-source-grouped_actor_prose_withheld.request.json": "ed079e86ced287775464eb76df3f4fdc43e836878465e5cd27333de748c5ad1f", + "phase/calls/confirm-fresh-18-en-m1-source-grouped_lossless.dispatch.json": "83387f64475ea085b7f103a3f2f60050dd84c0387382760309898b7886cc853f", + "phase/calls/confirm-fresh-18-en-m1-source-grouped_lossless.receipt.json": "b0b80044d0e2cf866a2d46773c2bd46efc4f0553bd679e0a6dedbd1b59915762", + "phase/calls/confirm-fresh-18-en-m1-source-grouped_lossless.request.json": "50279874911caf1be2772d0976402498059229843d1d7366e5d0ccf6db00b2ca", + "phase/calls/confirm-fresh-18-hi-m0-source-chronological.dispatch.json": "ecf738720261c5b187859a9fcb59a240ef42ef1968866c78c1ce433575c9b4d4", + "phase/calls/confirm-fresh-18-hi-m0-source-chronological.receipt.json": "6dff6f158c2c3aafa2db253a7bd8a606242fbdb0c18cb1be04e289ba7a0ef2bc", + "phase/calls/confirm-fresh-18-hi-m0-source-chronological.request.json": "52e0f1b167644417362c7d0560e0fab0eae69ebb44b73be59d5b4212ca847c00", + "phase/calls/confirm-fresh-18-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "1e02bf587f41520e957eca8b67f8241cb87a4e379f561d1129992098b8d87a24", + "phase/calls/confirm-fresh-18-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "54141e0dc4423ce477d584ad81c9365edbcc190550872134d262709d33446d80", + "phase/calls/confirm-fresh-18-hi-m0-source-grouped_actor_prose_withheld.request.json": "22becf3343a2596a4068196fc5255f1ab21db0f49268e46fcd0ed4f52af29c11", + "phase/calls/confirm-fresh-18-hi-m0-source-grouped_lossless.dispatch.json": "d5e4f0f3e649d313e43ab6fb615e0ca053a0d24031a2b4559edbe18eebe06470", + "phase/calls/confirm-fresh-18-hi-m0-source-grouped_lossless.receipt.json": "880f72a8c426d3f6362bd09f6d464b8d000b5b7357eea90f1e58db3f84ec8171", + "phase/calls/confirm-fresh-18-hi-m0-source-grouped_lossless.request.json": "284c445fb0f023dd2d51a21d418f0e706ad5b8a8fb1f5ecefa776fa2f62af485", + "phase/calls/confirm-fresh-18-hi-m1-source-chronological.dispatch.json": "1fafa8b47783f86a3721bca4c6c389564bb717b7eb0b1a90e0ec7fadc5526811", + "phase/calls/confirm-fresh-18-hi-m1-source-chronological.receipt.json": "6b390f5b0327e77e31388bbaf2c14bfed713858ad1740a8af164b85aebce0bea", + "phase/calls/confirm-fresh-18-hi-m1-source-chronological.request.json": "722fce1169ab1acbe2a99f07ba281668b9a70167756e743e2cee1053ff72a427", + "phase/calls/confirm-fresh-18-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "beffc55468502aafb9ec20680749d5ba4718c5ab5f55a934fe7fec8f776f7943", + "phase/calls/confirm-fresh-18-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "9e2477aad83ce75fb80df939f7cee7e109fb58a7c5f7c77ffa6e5677e948d6ff", + "phase/calls/confirm-fresh-18-hi-m1-source-grouped_actor_prose_withheld.request.json": "491ab21e673a0dfd7f0241fffb2dce6ab79ee63b82ec5468de053b438d8e2c49", + "phase/calls/confirm-fresh-18-hi-m1-source-grouped_lossless.dispatch.json": "2860d11f751352311c5e492317dedb635cdbfcd4f0ea5d89088c31c9e870707c", + "phase/calls/confirm-fresh-18-hi-m1-source-grouped_lossless.receipt.json": "e9f0ef0384f40a08252fee8afcb61a584ebc0f450b51ab89c0a9fb895b62c88e", + "phase/calls/confirm-fresh-18-hi-m1-source-grouped_lossless.request.json": "3c61c74e012371de0bddc56a08845e479ce7a9303885f4fe8e403cdebdfe7748", + "phase/calls/confirm-fresh-18-te-m0-source-chronological.dispatch.json": "e8985f22cb4e6e7120d97102d2cb4516e9aeb81eb3f07f45f85317577be91620", + "phase/calls/confirm-fresh-18-te-m0-source-chronological.receipt.json": "22436aa666e3816984b05323d21cd6057d86c22e93bae6d5dadff73f88e817fe", + "phase/calls/confirm-fresh-18-te-m0-source-chronological.request.json": "cbff70b738127a85eed6ddbdc83a29d24413a1d1a2afb0836c60d01623ffd43c", + "phase/calls/confirm-fresh-18-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "b84ca9d3468cba09850d309cc266a55573fbced2b55289f00b902e1f37f8cc68", + "phase/calls/confirm-fresh-18-te-m0-source-grouped_actor_prose_withheld.receipt.json": "ab1c0b0d84db96d8c2d7ded9cd9bdba0764d2c37b62886f83c17d9f888984082", + "phase/calls/confirm-fresh-18-te-m0-source-grouped_actor_prose_withheld.request.json": "c234b1bb6ef2d04217777d0e89a0d8fc5438424310354236df3d9ea4556cb554", + "phase/calls/confirm-fresh-18-te-m0-source-grouped_lossless.dispatch.json": "53a43a25ad338a17b43e4bf0a085af10492dc4826fab840566fd217a50f75df4", + "phase/calls/confirm-fresh-18-te-m0-source-grouped_lossless.receipt.json": "932bcab1147a1cac1789dd01721fddc596557666b300775dbfdfb2c441ad321b", + "phase/calls/confirm-fresh-18-te-m0-source-grouped_lossless.request.json": "7df43087d7b3fea7b6ae8246931b3c7e8419d74c97b603dae6e596d002af6a4c", + "phase/calls/confirm-fresh-18-te-m1-source-chronological.dispatch.json": "b2b3a91b797a1c38c93029999f47ea3986877d247a543b2ab4921e8d58a46e44", + "phase/calls/confirm-fresh-18-te-m1-source-chronological.receipt.json": "4b41f6017d94cc91828fbc78542e0ded11577cfbd189ee4c2502e0e85733a2c5", + "phase/calls/confirm-fresh-18-te-m1-source-chronological.request.json": "93335c589d17404c0709edb44231738bfda548734e9b69c90b68bef3e42db67e", + "phase/calls/confirm-fresh-18-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "a36bc4914042040a52ca69d6b9ef52d140e910e0f58d77b130c01fec39cc0cd4", + "phase/calls/confirm-fresh-18-te-m1-source-grouped_actor_prose_withheld.receipt.json": "edf9e55b3a84ae535f1106edac3e29d3aafbcb9ebfcc67424e21efd487830a5e", + "phase/calls/confirm-fresh-18-te-m1-source-grouped_actor_prose_withheld.request.json": "28cef0c5bb278a215b5be71461f2738c444581a34348c9650c2046e1c1ce1d3e", + "phase/calls/confirm-fresh-18-te-m1-source-grouped_lossless.dispatch.json": "dbded426dde3be06140e130876158e60b1d9afa4722510d1ee62b34366c39b41", + "phase/calls/confirm-fresh-18-te-m1-source-grouped_lossless.receipt.json": "eb67c2d528798676293cb05d55f5cfbd22a45c96f9197b89948c515d2aaeb3f9", + "phase/calls/confirm-fresh-18-te-m1-source-grouped_lossless.request.json": "81a204004cc7dc7008d82bbb5a80f71a5ff4d1c9939221e07cf0adf142b526f0", + "phase/calls/confirm-fresh-19-en-m0-source-chronological.dispatch.json": "04a9bf748d7cd22d1ea8407d27e821b4684fdfadc8839b469298b8d38cc7464d", + "phase/calls/confirm-fresh-19-en-m0-source-chronological.receipt.json": "68c631e25cc340972828ecd240efe6abdb55a261816cd8658b7bd1f47190a011", + "phase/calls/confirm-fresh-19-en-m0-source-chronological.request.json": "77714da6d1a185a0fcf05eb493e1b6fbd063ee5028affc58a33def5e872b21d9", + "phase/calls/confirm-fresh-19-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "40f8c2661f02653a2c2c67109fde61c3df0b28ec26c22a6650ed6b76ea0ce388", + "phase/calls/confirm-fresh-19-en-m0-source-grouped_actor_prose_withheld.receipt.json": "3f55069ed5c8213374a7db1f3d48ce2a5adbaa902e9a960da5a8b3d885222085", + "phase/calls/confirm-fresh-19-en-m0-source-grouped_actor_prose_withheld.request.json": "0391c2fe09cecc8739c194c950223bf638349a2d52733791fc7ceaf81c9cc6c5", + "phase/calls/confirm-fresh-19-en-m0-source-grouped_lossless.dispatch.json": "73722675b9544e20ae82bcfc990a5423431a085ea4be14de2d8546f8931b6150", + "phase/calls/confirm-fresh-19-en-m0-source-grouped_lossless.receipt.json": "a4cbfe6bef309b5f60885b7bc15c9e30f926f4d328b09f36f5c684e223c22fe5", + "phase/calls/confirm-fresh-19-en-m0-source-grouped_lossless.request.json": "d92c46e95e2f8c7898dc975a498066a3b9afcf546932d0a11d5bd98976b115dc", + "phase/calls/confirm-fresh-19-en-m1-source-chronological.dispatch.json": "379f5e5d1513f749261c7e9a348921d996328ebef2d5122bad69accf91c21dc9", + "phase/calls/confirm-fresh-19-en-m1-source-chronological.receipt.json": "450f05446335fd5827ff20fb721329a83a9bab2da5bdaa9b973e2c161368171d", + "phase/calls/confirm-fresh-19-en-m1-source-chronological.request.json": "2457f990a0fe0461adc4fa4c76d138d61bfc0db69cab4bfd4cdbc854028d8876", + "phase/calls/confirm-fresh-19-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "448f315b0d3d7c7df2f8a773f0e39281906647bd1516cdf0c91cc5f414ac4a38", + "phase/calls/confirm-fresh-19-en-m1-source-grouped_actor_prose_withheld.receipt.json": "f48834ce7f2f52e91e8ea7226c4506c19d9f8a52ab98faca3d32068e3c3f513f", + "phase/calls/confirm-fresh-19-en-m1-source-grouped_actor_prose_withheld.request.json": "edfe77ae8566f03cdd9c393ba617727b264c20ea2b1e3eed7ebe71831d53dcb7", + "phase/calls/confirm-fresh-19-en-m1-source-grouped_lossless.dispatch.json": "a8adda6ebc1ac7f5a8b6ad29fb3cf1e6d1fe74a59cb32526afebfc3d5a5b622d", + "phase/calls/confirm-fresh-19-en-m1-source-grouped_lossless.receipt.json": "cdf8df3b164cc356119f6a8cddb59e85e594349bbe22d129d86e73ca64b70d20", + "phase/calls/confirm-fresh-19-en-m1-source-grouped_lossless.request.json": "8c3caa861262280867ebc02afb8ab9d07199555457a056800b4852c45bdd2ee2", + "phase/calls/confirm-fresh-19-hi-m0-source-chronological.dispatch.json": "8b8a8df00398132d4690f636785ef01391d2e2ce6b47d98adecf72b3db5e2b83", + "phase/calls/confirm-fresh-19-hi-m0-source-chronological.receipt.json": "42b7edf2d6f0b87ab8c6d04f3dfcfc58c65c51d387781bed65b05fb47c537f24", + "phase/calls/confirm-fresh-19-hi-m0-source-chronological.request.json": "1cbd047d9409a1955e00e64cc147bd9f755bdd5322912696d330b02297d99210", + "phase/calls/confirm-fresh-19-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "ab3249a7ab439c6022690e4724ef08603755f63f2cd27205332fc7e457633f55", + "phase/calls/confirm-fresh-19-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "a03c4b6981233d6f0c48af2485041e86359ff8cfcb5f6a9318019dca9665a1f2", + "phase/calls/confirm-fresh-19-hi-m0-source-grouped_actor_prose_withheld.request.json": "d8c908e91cf90632bda207a9fc71817bc60a1f6d5ba12ab48b38375aa4aa121b", + "phase/calls/confirm-fresh-19-hi-m0-source-grouped_lossless.dispatch.json": "1ce4303b50bf81ea892e1aae567589d1ec4b81656fb6da8871568229addc5bcc", + "phase/calls/confirm-fresh-19-hi-m0-source-grouped_lossless.receipt.json": "0bf810e1ae116c8104668027db237b2ce90c80c138cb04f8539442a26532b0f4", + "phase/calls/confirm-fresh-19-hi-m0-source-grouped_lossless.request.json": "c6566ccff149ebb3933f151d1b37b946e98cfce590f70d3c95e6e58136da2b20", + "phase/calls/confirm-fresh-19-hi-m1-source-chronological.dispatch.json": "4d2b53ece9d2a98b2038dd6582825607e13111157e9e46784a7d596d186701d9", + "phase/calls/confirm-fresh-19-hi-m1-source-chronological.receipt.json": "7e32326042b5bb26bb0aad4fe015f3cb14959742bd4f4998920dfc53d285c532", + "phase/calls/confirm-fresh-19-hi-m1-source-chronological.request.json": "6866373ab7be249c9cfdab83a030d2e6a71d495c77fc5b36171bfd99ac7f1fd0", + "phase/calls/confirm-fresh-19-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "3b1349a5bbad65cc7cf994ff9b930884ff8e341d4cf43450ede6af2e57196acf", + "phase/calls/confirm-fresh-19-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "9fa95b8d2fa34103487d714d33681bd0067a56d6d4b8a1568c446ede8df7d57e", + "phase/calls/confirm-fresh-19-hi-m1-source-grouped_actor_prose_withheld.request.json": "3460699320ac6b2ede5e68e3d59285a6aa7f05ef27c7041d908bc7cb0fb7ef7e", + "phase/calls/confirm-fresh-19-hi-m1-source-grouped_lossless.dispatch.json": "3a96d016d8f2316e5a794f5ab79706431500313a44557e72b5ae8310880559ce", + "phase/calls/confirm-fresh-19-hi-m1-source-grouped_lossless.receipt.json": "4a8aa1e1d8c684f1f1a6ee81bac3544fc09ae5cf18dcc5dd4b0853b61b1fae15", + "phase/calls/confirm-fresh-19-hi-m1-source-grouped_lossless.request.json": "b9cc6c56cd4860e59a06d909be2e0f81aa7881388a3b6cdb27a0783ebae7c150", + "phase/calls/confirm-fresh-19-te-m0-source-chronological.dispatch.json": "f22b01eb37fd05b201c4f9b19d0151552208dd1a74b6b0acc87b5145eb25acbc", + "phase/calls/confirm-fresh-19-te-m0-source-chronological.receipt.json": "430d852e54523b01083471de43757ccc15369f9ca3291934d242dec54a63fd5a", + "phase/calls/confirm-fresh-19-te-m0-source-chronological.request.json": "ad26b8eeb2f80a4254742f207596f6885c20a9cef29df1aa75d90ec7bb3844ed", + "phase/calls/confirm-fresh-19-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "9cc3f02a46dc1944656a6451c794f06c5b3c0e765c8e6138e37201a263a8da6f", + "phase/calls/confirm-fresh-19-te-m0-source-grouped_actor_prose_withheld.receipt.json": "ab7716e40baf25c7439e6837314c81c2d286381cb65878893ae171f9089649e4", + "phase/calls/confirm-fresh-19-te-m0-source-grouped_actor_prose_withheld.request.json": "500265a7f2b4b3f169016bf8af337176aa2d899d61712f6165c971434f72298b", + "phase/calls/confirm-fresh-19-te-m0-source-grouped_lossless.dispatch.json": "a7812a8411b779bb17aad173884b012192c9d9b0d29c0a1c8df23f1407a6ee45", + "phase/calls/confirm-fresh-19-te-m0-source-grouped_lossless.receipt.json": "535a7f5fc3ed8f20a7736c13c154fa1d1d74816749583218f67cd5428dbb990d", + "phase/calls/confirm-fresh-19-te-m0-source-grouped_lossless.request.json": "83d61d6d1bad63d38b0f0a0d9bc711f19dc345bfd0df2a217e260299eb212593", + "phase/calls/confirm-fresh-19-te-m1-source-chronological.dispatch.json": "d0aa437adf79cde6c222b1c5b1f4db14f7c9d428742bb48a08281bd15fc891bb", + "phase/calls/confirm-fresh-19-te-m1-source-chronological.receipt.json": "27986500fe2e1227845969a209a95975ad8d904002f738d3b793cd4529550aa8", + "phase/calls/confirm-fresh-19-te-m1-source-chronological.request.json": "12826b22534534cb4eb646344fa5cccb22c8b42a1792d0c6bf71349b14f47f01", + "phase/calls/confirm-fresh-19-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "b4e026cb1b7ecd39d78fc1afe743a80fe916e76af34c46e4c75d1fe90ebe7658", + "phase/calls/confirm-fresh-19-te-m1-source-grouped_actor_prose_withheld.receipt.json": "f5e2e05b948ab41290e57b7590387f526f2bd61111791000dfd83c7b06d80783", + "phase/calls/confirm-fresh-19-te-m1-source-grouped_actor_prose_withheld.request.json": "9b5e4badd2d996902940b9d5d5ecd3b1ed787f1ca7596eb7a2a06609a9d31bf2", + "phase/calls/confirm-fresh-19-te-m1-source-grouped_lossless.dispatch.json": "2c77fc8844cd053a44e001233e761f08c418d00e4afd5037916a39955e486cae", + "phase/calls/confirm-fresh-19-te-m1-source-grouped_lossless.receipt.json": "292375d7807d9f272dbbb1f363725134722ee91d30a30782869917508aff1933", + "phase/calls/confirm-fresh-19-te-m1-source-grouped_lossless.request.json": "73e80936796ede10910167033db3c155f7ba7f52a55176e094559fd129cb1055", + "phase/calls/confirm-fresh-20-en-m0-source-chronological.dispatch.json": "5a8828c9fddb192b2474e67736df912f8485e96eb6ec765d814bafdd63f81456", + "phase/calls/confirm-fresh-20-en-m0-source-chronological.receipt.json": "463081b908d9ead6e99b3108da9f4d40212568f389e684edf2d34e65fbbdbbae", + "phase/calls/confirm-fresh-20-en-m0-source-chronological.request.json": "02ec13c991bb60f156e02902c26e80b56b3ba14fa0a4942c4a44813ba2f880ec", + "phase/calls/confirm-fresh-20-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "2486fb14fe92197c9efbe9e0c995ac506340519e0b208ed01d34dc12e4a8344f", + "phase/calls/confirm-fresh-20-en-m0-source-grouped_actor_prose_withheld.receipt.json": "92af5645eff03fa1524274fc2824c4a28dd068f3bcdbb73a0f2166ea23ddf338", + "phase/calls/confirm-fresh-20-en-m0-source-grouped_actor_prose_withheld.request.json": "1872c08ebcb7c8cfd708bb95f764cd590ee40471ee69949188da4069578f2a33", + "phase/calls/confirm-fresh-20-en-m0-source-grouped_lossless.dispatch.json": "570920765e7d5330a7ba76c3b50adcdf4bec79f1911a66b5c6945a630396d8fe", + "phase/calls/confirm-fresh-20-en-m0-source-grouped_lossless.receipt.json": "527def90925171b883fd285d38e20015a9908eb7c668381467e939b03d54cf09", + "phase/calls/confirm-fresh-20-en-m0-source-grouped_lossless.request.json": "bf8720d3b2c7d4a98b46ee00861fba269870f37104465df16b021c2f89ef8697", + "phase/calls/confirm-fresh-20-en-m1-source-chronological.dispatch.json": "2f805200f633a63bc8cbf43879a7ee3642502fb3d6161c7c1fcbe78e4d9981d4", + "phase/calls/confirm-fresh-20-en-m1-source-chronological.receipt.json": "b5dd3062f8e719a0c24d9ca3f3d8fe414fe79e1536829f0601e21afff9d7157b", + "phase/calls/confirm-fresh-20-en-m1-source-chronological.request.json": "db170b073d6d9d96fc7783e738c849903c1fb430b29fda163b3eb683ee979ef7", + "phase/calls/confirm-fresh-20-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "8828cd3a3cf79c0ae76a27f21acc7dbfb7197b777fcd9bab797ba8f44aa268e4", + "phase/calls/confirm-fresh-20-en-m1-source-grouped_actor_prose_withheld.receipt.json": "8c1c9926a5c0aec3d0b190c60cd491e06861b1c56c1e3899fdf9dc5f69f4b13a", + "phase/calls/confirm-fresh-20-en-m1-source-grouped_actor_prose_withheld.request.json": "7796e42ecb0e452bf3ea3c5aefb7f3eb7cf3c00b1f6a39feece2b59d2f9545ac", + "phase/calls/confirm-fresh-20-en-m1-source-grouped_lossless.dispatch.json": "f7fc6e0e4422d048aee0db279301eafeedbd31f553152afcbf4efc48752bad43", + "phase/calls/confirm-fresh-20-en-m1-source-grouped_lossless.receipt.json": "093a5b7dcf913bcf66de7cf1027d764c2267338c30edcbddee6f606d729c5461", + "phase/calls/confirm-fresh-20-en-m1-source-grouped_lossless.request.json": "c27430e9cf808dd00a2a46ce3702f99b60e2809396ec48d2bd0ab34efbf0f33d", + "phase/calls/confirm-fresh-20-hi-m0-source-chronological.dispatch.json": "55278814c60a22016a0e070d063322d185934cd47df443b1ecf1b13f978aac5b", + "phase/calls/confirm-fresh-20-hi-m0-source-chronological.receipt.json": "66fa42fe76f2486ac3e6c391d0049791a4aeb70a8cf11b3ee1760a7107c0e53e", + "phase/calls/confirm-fresh-20-hi-m0-source-chronological.request.json": "8b1ed0f1d1bd402ba55969491dcc8ba839b45fa6e351a3aba983ec2cbfa0f22f", + "phase/calls/confirm-fresh-20-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "214be2a46b7f0c0998c6d8d5ad065e78e9c759df168b22ef69824d6b3ae3aa33", + "phase/calls/confirm-fresh-20-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "8f2761f3864539841dc531085b387aced83dd097d8c0a90905a0b0e9beea3d41", + "phase/calls/confirm-fresh-20-hi-m0-source-grouped_actor_prose_withheld.request.json": "c55b0e84ccc06a5f7b6e0f7a607a6329d3470b837ef4d0c7aa804507a6c51396", + "phase/calls/confirm-fresh-20-hi-m0-source-grouped_lossless.dispatch.json": "231560ef5e02ba323e08e911f2b8ad070a7832daafbd5ba25a3db6146c52c9ab", + "phase/calls/confirm-fresh-20-hi-m0-source-grouped_lossless.receipt.json": "6d722b01ad40aa7d015c08d77fd9cff192e18c9a53bddd287e9197c62785bae6", + "phase/calls/confirm-fresh-20-hi-m0-source-grouped_lossless.request.json": "790896f80e06c8935a08a1f320d531e099070087bd276dc779bdc0a906b1a8d3", + "phase/calls/confirm-fresh-20-hi-m1-source-chronological.dispatch.json": "23723ac6f202bf29f27997be34334a38f1a6a057536d7082f26495477dcb3ddf", + "phase/calls/confirm-fresh-20-hi-m1-source-chronological.receipt.json": "d527606478f1ebbf1b371149bb69f7ccb672fe4de3a6b5c783c7bb808f2e04a2", + "phase/calls/confirm-fresh-20-hi-m1-source-chronological.request.json": "e3af090f43bb29d01b870d07d6e1c8134b755fbdc01c7c743ed7b49c2b0ac787", + "phase/calls/confirm-fresh-20-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "fa5a49885274b2a67867391116dc5d46e15da4874abb29cd08109c0588ed1c34", + "phase/calls/confirm-fresh-20-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "b0080397e2d734942856dc140ea81d2df8882f8a99072dd901a45e8fcb1d4f48", + "phase/calls/confirm-fresh-20-hi-m1-source-grouped_actor_prose_withheld.request.json": "4100852accf4dc4339f5705522ce8c549b6c94a80981ba23de4b29fe74399fac", + "phase/calls/confirm-fresh-20-hi-m1-source-grouped_lossless.dispatch.json": "d709916b21a5881743f8257cf6bfcd4aadce564bb39bee0b2ebc20289a61692b", + "phase/calls/confirm-fresh-20-hi-m1-source-grouped_lossless.receipt.json": "b4c46df4f2dae57d09b619c17703e67f834891f0c11f6286fc9eff30c4d2ca1a", + "phase/calls/confirm-fresh-20-hi-m1-source-grouped_lossless.request.json": "304c8c64d2ad4bea14f3a493e081ca534a8657de8a514024f2933d305e5d59c6", + "phase/calls/confirm-fresh-20-te-m0-source-chronological.dispatch.json": "3fbebcfc6dad585f81ac505ebe008d9eb6446f8e89505620bcc20b856fb14610", + "phase/calls/confirm-fresh-20-te-m0-source-chronological.receipt.json": "217f78d368d6d52a84ebd884dc5921cb5470752a5305296f10147d105c301a2c", + "phase/calls/confirm-fresh-20-te-m0-source-chronological.request.json": "a3e2df921e33e7768ec14431e2376043d497d715b6327b20e4ecb4654b58aa00", + "phase/calls/confirm-fresh-20-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "4f9de0f7543f18d9545de3cba4d92f809f29f8a2cbff104e70eedc6833d58cff", + "phase/calls/confirm-fresh-20-te-m0-source-grouped_actor_prose_withheld.receipt.json": "6971961a5802952fb407c1c48fe962be87225efaa263a3321b454001dbaeb47d", + "phase/calls/confirm-fresh-20-te-m0-source-grouped_actor_prose_withheld.request.json": "498e1772db016684df6399b5425108fc4e4b9d9daa090e788fbb2a4accd81775", + "phase/calls/confirm-fresh-20-te-m0-source-grouped_lossless.dispatch.json": "e988813ccca9e8eaf236ba9d0715af66877670e0b2318c07ef32d712d102be5e", + "phase/calls/confirm-fresh-20-te-m0-source-grouped_lossless.receipt.json": "cf6207e78165d91a4e1733290ff42e9418342822a96f3bbd092437b16014bff0", + "phase/calls/confirm-fresh-20-te-m0-source-grouped_lossless.request.json": "c7239f5892d858167206e05ef1ddeb2fbaf7648b666d683e68affda16018966c", + "phase/calls/confirm-fresh-20-te-m1-source-chronological.dispatch.json": "5bb7a4342f65a387f634038077089e55c7a8f2add95854bd3c0ac34a8c5b54b7", + "phase/calls/confirm-fresh-20-te-m1-source-chronological.receipt.json": "a6893e7e1a4d860ffe657f0cf2a7d96757bd1d911cd4f25cc9b35b8561872740", + "phase/calls/confirm-fresh-20-te-m1-source-chronological.request.json": "42c7ba5ea34a52799f65c576c2d6cdaa2c8b74d3eeb25b8e4ebcc276e3549780", + "phase/calls/confirm-fresh-20-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "095040fc4a1dda04ecb4a7117f50c23f68dac9579bfe765c0846c830fced5816", + "phase/calls/confirm-fresh-20-te-m1-source-grouped_actor_prose_withheld.receipt.json": "5f54ed5ab0bc46158307c27947de058efab5900a59cc534d0f5e28ba48c2d355", + "phase/calls/confirm-fresh-20-te-m1-source-grouped_actor_prose_withheld.request.json": "94a140a06a6a922903c6bd8623c19b082549a013a35a904ac9f4b6d52aca4f05", + "phase/calls/confirm-fresh-20-te-m1-source-grouped_lossless.dispatch.json": "8c6096552f680d104e34e72b68fe280ccf342554b6a3d124b2c16923849ee450", + "phase/calls/confirm-fresh-20-te-m1-source-grouped_lossless.receipt.json": "facdd40abf33895bfac0926604a2b0ff0868acb75c8d923581ee224acb4373c2", + "phase/calls/confirm-fresh-20-te-m1-source-grouped_lossless.request.json": "07f32311b48733e03609e864e46e031466d228894051f30bd441e9475d748bcc", + "phase/calls/confirm-fresh-21-en-m0-source-chronological.dispatch.json": "804f4e8428e915d88a8ac6a4a7390aa2c7fe79fba9a4e8cc5d726f551e31a5fb", + "phase/calls/confirm-fresh-21-en-m0-source-chronological.receipt.json": "0644e286da85dfe6126019cc11e15b08ba9f2a8d7ddcb43fdef1bbbf026cc3d0", + "phase/calls/confirm-fresh-21-en-m0-source-chronological.request.json": "805f3137eb7208c797b66e3ec37b4a885d300e3adc1c566277b5619f557af930", + "phase/calls/confirm-fresh-21-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "12f13532255222eb12c7fbfd9c91525f7807d922f87529639e6758bc81219d4a", + "phase/calls/confirm-fresh-21-en-m0-source-grouped_actor_prose_withheld.receipt.json": "aafa9d8288c877bbe46138dab2d34c6065b079b5ca39d32a5387945a9f5d6357", + "phase/calls/confirm-fresh-21-en-m0-source-grouped_actor_prose_withheld.request.json": "7aecd8573c91f58be2516861f2cc556a7c64c7bde791dc3118ba8280219728d5", + "phase/calls/confirm-fresh-21-en-m0-source-grouped_lossless.dispatch.json": "c00de262ea5feaf290cb3b8826399d1e66e46175f29e2d14f3e7ec45bcc3dc99", + "phase/calls/confirm-fresh-21-en-m0-source-grouped_lossless.receipt.json": "bf27f95b3db27c5cbffeed545a7a23fa32067ef4da46b1be43dc905b82d1ac3b", + "phase/calls/confirm-fresh-21-en-m0-source-grouped_lossless.request.json": "7aecd8573c91f58be2516861f2cc556a7c64c7bde791dc3118ba8280219728d5", + "phase/calls/confirm-fresh-21-en-m1-source-chronological.dispatch.json": "09cd39a3e0c62790db53caf6a294ae7f118cf62adf760d83422462779852b828", + "phase/calls/confirm-fresh-21-en-m1-source-chronological.receipt.json": "b8e9d39d9c33a763ff7f0b7fb28af6e6591ebb7cb721cb6ce1f913b8aa5d22b3", + "phase/calls/confirm-fresh-21-en-m1-source-chronological.request.json": "94b54826e622e09003065b0b02c84680861f90b936def6a06851659973916635", + "phase/calls/confirm-fresh-21-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "78ebe17b976bf308f800e648ca71379bd76b5bde9ba9f53277e4672b6df4c288", + "phase/calls/confirm-fresh-21-en-m1-source-grouped_actor_prose_withheld.receipt.json": "20ba4cdb16dd870cddda88d68f5197c79086950d2eba437efba62faf399e41ae", + "phase/calls/confirm-fresh-21-en-m1-source-grouped_actor_prose_withheld.request.json": "32cf8ce9b7ddab90245fdaa7ceab52a690bd9758973c66beef3ff7231f8e12b3", + "phase/calls/confirm-fresh-21-en-m1-source-grouped_lossless.dispatch.json": "e0878356cfb1b082fd7756f678f6db1edc4bb8d561fc41c2187363bcfaf60530", + "phase/calls/confirm-fresh-21-en-m1-source-grouped_lossless.receipt.json": "45c8aff86c4dcb2ccd92b6d81cc0d5176442ec39c4c346b0ab80298281fcf6a9", + "phase/calls/confirm-fresh-21-en-m1-source-grouped_lossless.request.json": "32cf8ce9b7ddab90245fdaa7ceab52a690bd9758973c66beef3ff7231f8e12b3", + "phase/calls/confirm-fresh-21-hi-m0-source-chronological.dispatch.json": "6c79d9ef94a03b5a99108fca8eb3a66a9ee2bd9844e3d621b1a8d8252b0c4422", + "phase/calls/confirm-fresh-21-hi-m0-source-chronological.receipt.json": "736a0543cbabfd30d3eb40ee091158cb328b6102b3d56279c20bc0cf8ce83fa8", + "phase/calls/confirm-fresh-21-hi-m0-source-chronological.request.json": "9b152cc04893d66ce2fac4753226e5d4106ca0b955c46859d8d513846c5f97b4", + "phase/calls/confirm-fresh-21-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "6613ed6b3d37ece642bffe1c46518ee9d78c32d25a979e6cd47b371c03096c7c", + "phase/calls/confirm-fresh-21-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "1b07b66e333fc45179e31b668b1f8194bb42c6050714541fc2c05ab4c2cc8627", + "phase/calls/confirm-fresh-21-hi-m0-source-grouped_actor_prose_withheld.request.json": "07e94271ddf5030da4fc45f6d3c2b0c78a0246d1185c52c9b2335ad242d3fe7d", + "phase/calls/confirm-fresh-21-hi-m0-source-grouped_lossless.dispatch.json": "ed178632e7c78e665d60fc0c8336e69e1f9580ac73cb65b96ce78b0caa1611cc", + "phase/calls/confirm-fresh-21-hi-m0-source-grouped_lossless.receipt.json": "b6f5cb643b4e377e2056ac884543c2fb1a59c9d60ef24a32b9b4b432595bfa11", + "phase/calls/confirm-fresh-21-hi-m0-source-grouped_lossless.request.json": "07e94271ddf5030da4fc45f6d3c2b0c78a0246d1185c52c9b2335ad242d3fe7d", + "phase/calls/confirm-fresh-21-hi-m1-source-chronological.dispatch.json": "f27b6380ac679ae74077a9d6f0188710fb50e27e6d3d840985fdaea4e18e5fcf", + "phase/calls/confirm-fresh-21-hi-m1-source-chronological.receipt.json": "e01755932b5dc9b4326b5f1b63b33f9023372aa9656df3a861ad266258b17717", + "phase/calls/confirm-fresh-21-hi-m1-source-chronological.request.json": "0a50dc40a8e7062627954ba7f71490a1afbd20ba5a7270927fc2351a81d16cd1", + "phase/calls/confirm-fresh-21-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "f01c890c32fadfed77ed1d598d41aac2b8346a9de5885242f028e475af5e8de8", + "phase/calls/confirm-fresh-21-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "08a9d0c40fabc9254f294fe603d1c30f4930eb4638e0d17863b72f416800c1d3", + "phase/calls/confirm-fresh-21-hi-m1-source-grouped_actor_prose_withheld.request.json": "d1e44331bf85878581c43ef91c24fcb1c50134cf899d422eafc079d5c7094941", + "phase/calls/confirm-fresh-21-hi-m1-source-grouped_lossless.dispatch.json": "7ec1a66b163984b57ff6265a442e011aa2946e345f78cfc961e108c92c639ad8", + "phase/calls/confirm-fresh-21-hi-m1-source-grouped_lossless.receipt.json": "f8365223d0702010f2c7f9b5c3dbc5087b8b100004c7233202904e8735c057f4", + "phase/calls/confirm-fresh-21-hi-m1-source-grouped_lossless.request.json": "d1e44331bf85878581c43ef91c24fcb1c50134cf899d422eafc079d5c7094941", + "phase/calls/confirm-fresh-21-te-m0-source-chronological.dispatch.json": "ec905d4bcfaa70c7a0506283f8f759077ee5c7ebbdb0cbd0ddd5acfb8060ded2", + "phase/calls/confirm-fresh-21-te-m0-source-chronological.receipt.json": "a4596aac3c9564c36f8b037ccb5b9eecd3054d1ca8a4dc0bfead53ffc2cfaff8", + "phase/calls/confirm-fresh-21-te-m0-source-chronological.request.json": "964efbc300d331437d09209ccb8d192f3e884b578026b0d170d7b6074cfda5a0", + "phase/calls/confirm-fresh-21-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "408019250d3473ad7d7553400ecea5bdc04364b09aa6a3c4d0683467dadded6b", + "phase/calls/confirm-fresh-21-te-m0-source-grouped_actor_prose_withheld.receipt.json": "85c1a07dc8399d3ad9b7677779e6cab09be541f1d2600e4a73eebadd120c2bcd", + "phase/calls/confirm-fresh-21-te-m0-source-grouped_actor_prose_withheld.request.json": "bf670853acbec3997e025f376e4b4944c6ae6ea8b1a68476ba3ea776ddf0cdab", + "phase/calls/confirm-fresh-21-te-m0-source-grouped_lossless.dispatch.json": "979411c9260f52d3ec67e236faa618db39c5172e024925d5418c76ee53d6653f", + "phase/calls/confirm-fresh-21-te-m0-source-grouped_lossless.receipt.json": "1b15a67840704a9f6bfe93989c8728af4435b4dfa9b4f70e320473ef220b50a2", + "phase/calls/confirm-fresh-21-te-m0-source-grouped_lossless.request.json": "bf670853acbec3997e025f376e4b4944c6ae6ea8b1a68476ba3ea776ddf0cdab", + "phase/calls/confirm-fresh-21-te-m1-source-chronological.dispatch.json": "1b9cdf1def50edd4d7afea54b03a15968b69b5528254ca5bf7189bd21e873b1d", + "phase/calls/confirm-fresh-21-te-m1-source-chronological.receipt.json": "bc632d7ba9058e2d8e198495e645c7431beed880fc2aac6e9fd16d721342fc62", + "phase/calls/confirm-fresh-21-te-m1-source-chronological.request.json": "ab341529480470730bc60657729bd08eb8e6fc8a5dec6e30435193e0e1d5e471", + "phase/calls/confirm-fresh-21-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "5ac8173cb78e37e977d5099c40b78b363c1faed8b2c79ad9ee1f6a86f31418c9", + "phase/calls/confirm-fresh-21-te-m1-source-grouped_actor_prose_withheld.receipt.json": "023f0bac45bb66273bd7a62ecaea757e6e66aef01db1044ef50efce1fe6d4fbf", + "phase/calls/confirm-fresh-21-te-m1-source-grouped_actor_prose_withheld.request.json": "892f20005064eccf6fadf38fcdeb10353e9394ba8b54b559fcc969e50e2e9a5b", + "phase/calls/confirm-fresh-21-te-m1-source-grouped_lossless.dispatch.json": "11ba2b3104504d301dd30be70e870ebdaedc85974d16d9d18fdc0de243dcdb10", + "phase/calls/confirm-fresh-21-te-m1-source-grouped_lossless.receipt.json": "accb12d32890079ef96f0a485e2dee234899c3821ee1dfedbd4a553d1a09b598", + "phase/calls/confirm-fresh-21-te-m1-source-grouped_lossless.request.json": "892f20005064eccf6fadf38fcdeb10353e9394ba8b54b559fcc969e50e2e9a5b", + "phase/calls/confirm-fresh-22-en-m0-source-chronological.dispatch.json": "9f97b285e800781bb5589d142373bc1b6112ea912c7f372aecdc6749e5c0ae72", + "phase/calls/confirm-fresh-22-en-m0-source-chronological.receipt.json": "33da5c9b0c93021566414b5e0cf9063e88cb66e6c14ed77f320a1b5da771761d", + "phase/calls/confirm-fresh-22-en-m0-source-chronological.request.json": "da6c219ca0b8efd4ad9d60630ecd55b23ba58e17b8182038853267060adf4eb9", + "phase/calls/confirm-fresh-22-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "dc8b44d5be18f75f421ec360f4c28c53bc6b4e710863dd76b87bc8b23175b870", + "phase/calls/confirm-fresh-22-en-m0-source-grouped_actor_prose_withheld.receipt.json": "c2ceb0a57e784b425613f69da8231d668506d1f946bbc98dfd4be342041d473f", + "phase/calls/confirm-fresh-22-en-m0-source-grouped_actor_prose_withheld.request.json": "9fa966000bbfae196e421a678c740a14e0fffb0019fc9bbb770d1869ec6a29d9", + "phase/calls/confirm-fresh-22-en-m0-source-grouped_lossless.dispatch.json": "21d134e188a1341f1142723864129a29bcebc53671eeb73e8a54aa608414a198", + "phase/calls/confirm-fresh-22-en-m0-source-grouped_lossless.receipt.json": "736f339a37555f4eff4239997153e6d1bcf72f101a625137f0ee020d906bf4c0", + "phase/calls/confirm-fresh-22-en-m0-source-grouped_lossless.request.json": "9fa966000bbfae196e421a678c740a14e0fffb0019fc9bbb770d1869ec6a29d9", + "phase/calls/confirm-fresh-22-en-m1-source-chronological.dispatch.json": "d7f5a13e56e9317fdcd8099dfce4e0bf4eafcdcb62fb840ccfd98f1cecdcf622", + "phase/calls/confirm-fresh-22-en-m1-source-chronological.receipt.json": "3c2d0fd77776e8c521c4a3df1ff7574bb28fd0d4fcd14160c5be6a1af87ab183", + "phase/calls/confirm-fresh-22-en-m1-source-chronological.request.json": "1e53d708d20e22b5ce759cdae8fd472bc362562068c7614e7405bc3555a69b4f", + "phase/calls/confirm-fresh-22-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "ef59c829b95eec7aa23ab09a8806493fa3dcf4e230ede6be3ff9aebb9e5c971e", + "phase/calls/confirm-fresh-22-en-m1-source-grouped_actor_prose_withheld.receipt.json": "d8221435062184c5e0ab3386633949771fb99b7067d9f222e2496d82712dc0ee", + "phase/calls/confirm-fresh-22-en-m1-source-grouped_actor_prose_withheld.request.json": "30e9bd69a901110f6b68f0e3ea360b71f7996f77588c8f9f6269c0a59b9a8432", + "phase/calls/confirm-fresh-22-en-m1-source-grouped_lossless.dispatch.json": "68dc61ba6f95a7e850fb70b901381672ce7bbe9ea422c8f0f12eff72b86505c8", + "phase/calls/confirm-fresh-22-en-m1-source-grouped_lossless.receipt.json": "c33040c45ba3575756aaebfa0838f73d76bdbf07890c137d81c1852bd153ba5f", + "phase/calls/confirm-fresh-22-en-m1-source-grouped_lossless.request.json": "30e9bd69a901110f6b68f0e3ea360b71f7996f77588c8f9f6269c0a59b9a8432", + "phase/calls/confirm-fresh-22-hi-m0-source-chronological.dispatch.json": "c8ac5e1d1ae6ce72847b6625f96c0649b51d086a568c8da5bac19a1cd0e9bac7", + "phase/calls/confirm-fresh-22-hi-m0-source-chronological.receipt.json": "0ca0b9e7d06ecc61faaae70876b6b65b90124e390fb729f22ae62d50ae19efce", + "phase/calls/confirm-fresh-22-hi-m0-source-chronological.request.json": "50d6ea96416c6eecc4524de24f0a719da9ff16e0628ae0129eb82ec576f573ae", + "phase/calls/confirm-fresh-22-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "461c311f9193a73c24bf4ae0c06aaadc0dff97955ae54b25955b7d41ade92fe5", + "phase/calls/confirm-fresh-22-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "2b78ff04493de4ddc891058311b3168c81790bca95161a695e46ea80b7708128", + "phase/calls/confirm-fresh-22-hi-m0-source-grouped_actor_prose_withheld.request.json": "206aadaa1b0f5ec0f6c4885dd7654fcc2765c277847fb4b4630ce00c16ceca86", + "phase/calls/confirm-fresh-22-hi-m0-source-grouped_lossless.dispatch.json": "c9644f1e340f8934674bdd539bfe2f6a885f73069ba3bcb1c49a7a25242101de", + "phase/calls/confirm-fresh-22-hi-m0-source-grouped_lossless.receipt.json": "884939c5a1c235ce1498dd3e99bbbda7a3c401da2c1647ffe3058b8d7df287ce", + "phase/calls/confirm-fresh-22-hi-m0-source-grouped_lossless.request.json": "206aadaa1b0f5ec0f6c4885dd7654fcc2765c277847fb4b4630ce00c16ceca86", + "phase/calls/confirm-fresh-22-hi-m1-source-chronological.dispatch.json": "9fa8114c4ee8d19f89eb33b4dd62e6601a2ce6cdd45a2254b7f45c16756fa120", + "phase/calls/confirm-fresh-22-hi-m1-source-chronological.receipt.json": "51732486eba1cd2320567c71f1dab3ea97848a8f64745cb7abfccbe6b4eb6287", + "phase/calls/confirm-fresh-22-hi-m1-source-chronological.request.json": "c22a74e4913dc3b2795e04c6d9538600d2d1ad411356a56715fa0e84eb4aeb46", + "phase/calls/confirm-fresh-22-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "f5aacd4a822d3bdbd76f75ee3277986b33a42c037487e8b71f1defdc29e88ca5", + "phase/calls/confirm-fresh-22-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "13bd87e1413916a761fb111dae5bd0129de642199409098c8004d06d7310a38c", + "phase/calls/confirm-fresh-22-hi-m1-source-grouped_actor_prose_withheld.request.json": "9ad4e8d553823d8110446c6250e90e264df6d93079a6be7cd3a1f340203e25d0", + "phase/calls/confirm-fresh-22-hi-m1-source-grouped_lossless.dispatch.json": "4f683e6a81b9d66a27547b31509b581480a9925e826552e8fc3ebe4109f43bfb", + "phase/calls/confirm-fresh-22-hi-m1-source-grouped_lossless.receipt.json": "58730bc0987402a6c8184ae9f9de391a8f4321a38c6271ea8f8c48aa780e5c1e", + "phase/calls/confirm-fresh-22-hi-m1-source-grouped_lossless.request.json": "9ad4e8d553823d8110446c6250e90e264df6d93079a6be7cd3a1f340203e25d0", + "phase/calls/confirm-fresh-22-te-m0-source-chronological.dispatch.json": "900c4da710de8a06881eb03cfe7b3eff912ad38a14595f1924ce8e3b7fc9412a", + "phase/calls/confirm-fresh-22-te-m0-source-chronological.receipt.json": "74da47fed7587878d05924c827b4499f1df829a5343cda7466d087491300f795", + "phase/calls/confirm-fresh-22-te-m0-source-chronological.request.json": "422467c5e1525f2e011d2396e38ab32ef0742f99644aac5ad50c8ee94586e968", + "phase/calls/confirm-fresh-22-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "42ba3d239995e41a7339a46fe3504b742b27cc888f7a638c3c99a99dfc9ef357", + "phase/calls/confirm-fresh-22-te-m0-source-grouped_actor_prose_withheld.receipt.json": "a3aaa749928f1c6982920d0ee8f8c7e105a4a3dd227e275e2b6c78d7bb8c7c74", + "phase/calls/confirm-fresh-22-te-m0-source-grouped_actor_prose_withheld.request.json": "327f0d84dd108abaea3ea934ec6694561e2b875497889924fb461c30f19cf7a1", + "phase/calls/confirm-fresh-22-te-m0-source-grouped_lossless.dispatch.json": "40928338895f11367243c2e1d94231029d5f7b7f44be497d6348876cee8ebcca", + "phase/calls/confirm-fresh-22-te-m0-source-grouped_lossless.receipt.json": "27b514e3edfd16710005675d5c115913688e4de87c715dab5df7bdc0da45385c", + "phase/calls/confirm-fresh-22-te-m0-source-grouped_lossless.request.json": "05ec5c21153b1792325b33c9cba9c37a7747e3054d8b54ed6fbd55c77517f7ed", + "phase/calls/confirm-fresh-22-te-m1-source-chronological.dispatch.json": "ca86048f6a023b9b1c7f80b6c76814832a83c6980eecccd92c82b7a0f01a0da7", + "phase/calls/confirm-fresh-22-te-m1-source-chronological.receipt.json": "86835433233dbe9579d740b37472bb3a3470fc62486a780cd1d69147ce5089fd", + "phase/calls/confirm-fresh-22-te-m1-source-chronological.request.json": "919a1e795a127e59ef4b7fb759ee2f7a8a581517b3acc7219a67ab7c956fa325", + "phase/calls/confirm-fresh-22-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "eb8be3ea8002e0fddfcaa0775c20ac1a75d8d4a0021fad30167a087c1cb11b03", + "phase/calls/confirm-fresh-22-te-m1-source-grouped_actor_prose_withheld.receipt.json": "4a9c3f9e799f94ad422c58b6481b4d63258282c328dcbf3712be899e1967a665", + "phase/calls/confirm-fresh-22-te-m1-source-grouped_actor_prose_withheld.request.json": "f01b1dbf8ee399c5852135441d81f86b14291b75988f65d05ae09bbee0829263", + "phase/calls/confirm-fresh-22-te-m1-source-grouped_lossless.dispatch.json": "5c666fc5aaa7191dcbcc0e026a81b1e8b298912ee0efc2a2e06863c7e4c6a2a0", + "phase/calls/confirm-fresh-22-te-m1-source-grouped_lossless.receipt.json": "054387af058bacb123a24309fac35d351f23ab2b94b703080fa43e029061dd9f", + "phase/calls/confirm-fresh-22-te-m1-source-grouped_lossless.request.json": "f01b1dbf8ee399c5852135441d81f86b14291b75988f65d05ae09bbee0829263", + "phase/calls/confirm-fresh-23-en-m0-source-chronological.dispatch.json": "9647b56328ccda81cde512ae77792119127d0818064a44afcc30d7c3d0ec3e96", + "phase/calls/confirm-fresh-23-en-m0-source-chronological.receipt.json": "1a0c7725ed58479b85ad270e02a04e9bfea1c210e3d8f3380627b245dd3cdf4d", + "phase/calls/confirm-fresh-23-en-m0-source-chronological.request.json": "190fa5a6e3156e152032b62db979d977eca62b2d33714c7dd3d71b864bb1b418", + "phase/calls/confirm-fresh-23-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "61c409373170aa6b324c5483b086fd6db7c7f327d12d3a6899039ad59911ad76", + "phase/calls/confirm-fresh-23-en-m0-source-grouped_actor_prose_withheld.receipt.json": "a0e876e36fdd5c5656947556c4d406b259058591d31df79d89ac3b8ac5e829f5", + "phase/calls/confirm-fresh-23-en-m0-source-grouped_actor_prose_withheld.request.json": "2f3f0642a1b081f0486fc7f72a22662826b9526888d75048131a20a1387f3e51", + "phase/calls/confirm-fresh-23-en-m0-source-grouped_lossless.dispatch.json": "26925d164793f2828201728a08f3d35fa32ee5ca75a2ba6c5e031a29a8cbd3bc", + "phase/calls/confirm-fresh-23-en-m0-source-grouped_lossless.receipt.json": "07f46f724acfa8c0d5b1184d1ee1fddbfb8349ef8942318a2c01254719c129a1", + "phase/calls/confirm-fresh-23-en-m0-source-grouped_lossless.request.json": "2f3f0642a1b081f0486fc7f72a22662826b9526888d75048131a20a1387f3e51", + "phase/calls/confirm-fresh-23-en-m1-source-chronological.dispatch.json": "f5107bee8b06b4f390e39b262e8707dd7e4f8bffc1a46a41eb3287a7d6c25d6a", + "phase/calls/confirm-fresh-23-en-m1-source-chronological.receipt.json": "7a148941dde6df1e68aaab44fca7a522a18c4f074172cb639bceab85edde35ad", + "phase/calls/confirm-fresh-23-en-m1-source-chronological.request.json": "e615f8eb8499717399a9f2186004f6da66e84152d694353c14d9f5149be11ad0", + "phase/calls/confirm-fresh-23-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "435206c08ece84e39178f45833a021e74a12d16e9411fc71011bd988cf1bd995", + "phase/calls/confirm-fresh-23-en-m1-source-grouped_actor_prose_withheld.receipt.json": "00690f802a9e545b661e156041c0bcbfa0d6fe30f6d25bc044f945000c2da2f3", + "phase/calls/confirm-fresh-23-en-m1-source-grouped_actor_prose_withheld.request.json": "180eb14dd2a2bc079b190d728e797cd5b315ed679f61820c0b754ae444bbea48", + "phase/calls/confirm-fresh-23-en-m1-source-grouped_lossless.dispatch.json": "e964a6c71ad018f2602786bd7f06eb87c36e2d4850ad07a1d8f3b0442aca904b", + "phase/calls/confirm-fresh-23-en-m1-source-grouped_lossless.receipt.json": "87ef635556257cf2390bb81ae3f880290c1fdd3a2055818ff8e46c8043fc57b0", + "phase/calls/confirm-fresh-23-en-m1-source-grouped_lossless.request.json": "180eb14dd2a2bc079b190d728e797cd5b315ed679f61820c0b754ae444bbea48", + "phase/calls/confirm-fresh-23-hi-m0-source-chronological.dispatch.json": "c8492625a3a417b6ff53688a2e0477894b3db1d984e04ee8ac5d7bbef3bece3a", + "phase/calls/confirm-fresh-23-hi-m0-source-chronological.receipt.json": "75a8518a685ddce3f77db86d2dac0885cc6fcf5ca7354525d81249d0636977c3", + "phase/calls/confirm-fresh-23-hi-m0-source-chronological.request.json": "e5c22da7a187c0cf3ae461bf2203dc8c64ee1179c5eb6c2fbae45b4c02598d92", + "phase/calls/confirm-fresh-23-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "844ac0521b1be93c39e99d99cd52c1af4129cea1629da97f0d3f92f8be7c1777", + "phase/calls/confirm-fresh-23-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "1a1cb6a1c68912b8981cfad5f2aa65c782c26392662e8b89e585b7bef4e9c784", + "phase/calls/confirm-fresh-23-hi-m0-source-grouped_actor_prose_withheld.request.json": "74cd4642ad1b8fda9dca722d5e36aebe23d01889751a993fd3883216f189d2a0", + "phase/calls/confirm-fresh-23-hi-m0-source-grouped_lossless.dispatch.json": "a00933b8e3de9c13c6f46939ac1c21dae80575899a1c9a576a8b33650bc9d5d1", + "phase/calls/confirm-fresh-23-hi-m0-source-grouped_lossless.receipt.json": "0cd9e6049b78230915a513bd30dd11098ebcb86660e51100d88b76722a3add80", + "phase/calls/confirm-fresh-23-hi-m0-source-grouped_lossless.request.json": "74cd4642ad1b8fda9dca722d5e36aebe23d01889751a993fd3883216f189d2a0", + "phase/calls/confirm-fresh-23-hi-m1-source-chronological.dispatch.json": "59a66fe40e0e956bec573eb0a45541c2710bee7cd5e6c14f060400565e574aa9", + "phase/calls/confirm-fresh-23-hi-m1-source-chronological.receipt.json": "8c62ddec4606fb63984b73a2e3ed35ada833c113b3cd8d01c5df3987bbbee778", + "phase/calls/confirm-fresh-23-hi-m1-source-chronological.request.json": "26cd3102ffdf792882b8bbb15067f06f7370c4d425db9883cc9af5e4c4928a68", + "phase/calls/confirm-fresh-23-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "a0e56a0f7dfbad0807264c771347efa667d6c881f13b66fc7707d6e5f95f9637", + "phase/calls/confirm-fresh-23-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "b7b30eef2f09ec867d15b4eb5fd715bba423b677d6b702ff78be9a2572f3b963", + "phase/calls/confirm-fresh-23-hi-m1-source-grouped_actor_prose_withheld.request.json": "26d7c47d6700ae473332325aa96d7dd35e716dcd01af0aee32bc4ff807e4cbe7", + "phase/calls/confirm-fresh-23-hi-m1-source-grouped_lossless.dispatch.json": "bbbe46a8712e31e0fbb43d6709663f38bce4555f8a72acb7da6f3dd105321bd9", + "phase/calls/confirm-fresh-23-hi-m1-source-grouped_lossless.receipt.json": "893493c64295854bd6dfe0366663032e87c98de92b7ada64f0388f3826d02c5c", + "phase/calls/confirm-fresh-23-hi-m1-source-grouped_lossless.request.json": "26d7c47d6700ae473332325aa96d7dd35e716dcd01af0aee32bc4ff807e4cbe7", + "phase/calls/confirm-fresh-23-te-m0-source-chronological.dispatch.json": "e4633a81ed26ce938d65b146789a315edcecf71625dfe41e1dbd65b866866453", + "phase/calls/confirm-fresh-23-te-m0-source-chronological.receipt.json": "54c63ac29c0e99630dcf2d117209ab9382197a7492387f96c43025d435c1ad75", + "phase/calls/confirm-fresh-23-te-m0-source-chronological.request.json": "e18f0f1e61ddd61aa19155f016521604b74ea934c6b5c194c1f65f1c4656254c", + "phase/calls/confirm-fresh-23-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "5ce255715182bab14cc0e0cd24f911a8751958bd64f3ab5a2910d49dc4d21d5c", + "phase/calls/confirm-fresh-23-te-m0-source-grouped_actor_prose_withheld.receipt.json": "f2c603c0534b3b92b1277dc81ed883527dbe6ac805f62feb062cba49b5a0d1c2", + "phase/calls/confirm-fresh-23-te-m0-source-grouped_actor_prose_withheld.request.json": "2f9961ffca631adccde72aad1c2748c2965df6dd42798a35cdd737b80dc7fa33", + "phase/calls/confirm-fresh-23-te-m0-source-grouped_lossless.dispatch.json": "be631f7980622d6123953abec3c2d5ba6f98b0cb8aa4300643497d46ffa13004", + "phase/calls/confirm-fresh-23-te-m0-source-grouped_lossless.receipt.json": "007f8a5314ae251378422c05280880ac5290b3730e461de751e679075ed9cbec", + "phase/calls/confirm-fresh-23-te-m0-source-grouped_lossless.request.json": "2f9961ffca631adccde72aad1c2748c2965df6dd42798a35cdd737b80dc7fa33", + "phase/calls/confirm-fresh-23-te-m1-source-chronological.dispatch.json": "a4f7317fef2d18d0adfec2b5e86f1949b2f870e8c75419c1efb3e1dae8b02ffe", + "phase/calls/confirm-fresh-23-te-m1-source-chronological.receipt.json": "5962c7948fbe12adcf95d28e83c42894e78bcf68c45fcf89021fc325064b8995", + "phase/calls/confirm-fresh-23-te-m1-source-chronological.request.json": "acee370598326334c1358e04786f9b7ca5a3338419c1906ad56413cd8ba563ec", + "phase/calls/confirm-fresh-23-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "924bf2dadc7485a33c1ecdbd1258d00d2dfeff7f796ec3a75530a155975a6610", + "phase/calls/confirm-fresh-23-te-m1-source-grouped_actor_prose_withheld.receipt.json": "17b1463ef99ae956aa6a8950e7d513d76ba38d1a993b468ebe6da3f75ba567f2", + "phase/calls/confirm-fresh-23-te-m1-source-grouped_actor_prose_withheld.request.json": "652cdf5af5b6e89d1a2e03389ad8f8d00972aedebb9cf0a1ee4563ae637ac505", + "phase/calls/confirm-fresh-23-te-m1-source-grouped_lossless.dispatch.json": "d7444dd12846279fe933da39d5880e63e4de523ded43fb2b740ab3845b5f40a6", + "phase/calls/confirm-fresh-23-te-m1-source-grouped_lossless.receipt.json": "bc3497ca624a1b25517ef88673ca34648b5716b00c3fccded6535258efac429a", + "phase/calls/confirm-fresh-23-te-m1-source-grouped_lossless.request.json": "652cdf5af5b6e89d1a2e03389ad8f8d00972aedebb9cf0a1ee4563ae637ac505", + "phase/calls/confirm-fresh-24-en-m0-source-chronological.dispatch.json": "05a3abb6e979ce96a6c26d18934bf5fc7c5f0e79346e885639609f6b907470de", + "phase/calls/confirm-fresh-24-en-m0-source-chronological.receipt.json": "2253b09fc86def85eca603d8dfe5933c7bc1cd342ae1416ab760ab04492957b1", + "phase/calls/confirm-fresh-24-en-m0-source-chronological.request.json": "2b3fbb2beca1a8d2a0bb7efd9570af575813abe5323e04c8f46551adbe508c2d", + "phase/calls/confirm-fresh-24-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "892fd84e25a52bf2fd6ee65b4d8ce87bc5657367308542aec32385cc42fff332", + "phase/calls/confirm-fresh-24-en-m0-source-grouped_actor_prose_withheld.receipt.json": "d2a0bbc8b8da0d31e2b9b5461d17d2650d387400ccb43f253f01b1068207d3a1", + "phase/calls/confirm-fresh-24-en-m0-source-grouped_actor_prose_withheld.request.json": "1b257db4229cd6a442c2444b7fab40c134f7e76ab4902f84b3a92e119bc552e6", + "phase/calls/confirm-fresh-24-en-m0-source-grouped_lossless.dispatch.json": "f881bdbe450a3f319f0899a55fdea264e0345485e3d512811765538ce5be6d6c", + "phase/calls/confirm-fresh-24-en-m0-source-grouped_lossless.receipt.json": "d36c0c526c0c6b58796643f9ce0b0e86eefbc950f447633c0caef0a110c37df4", + "phase/calls/confirm-fresh-24-en-m0-source-grouped_lossless.request.json": "1b257db4229cd6a442c2444b7fab40c134f7e76ab4902f84b3a92e119bc552e6", + "phase/calls/confirm-fresh-24-en-m1-source-chronological.dispatch.json": "84d5679b04158434fcf71fa6add30ddb0d0455c6f892561bc103821f78ceb074", + "phase/calls/confirm-fresh-24-en-m1-source-chronological.receipt.json": "522cc2c69bf2f1ad5ba1bd6a1bdb537b649f9111841cfa8d792c3a2abe5d8ceb", + "phase/calls/confirm-fresh-24-en-m1-source-chronological.request.json": "44a346cfe634d35af9cd78119bdd618b942c6d8a0fbb3d852a1fcb46a02a936a", + "phase/calls/confirm-fresh-24-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "e5c3629b27ebc63e39738f9375ec529ca664c674a730622d04b7335f796453b2", + "phase/calls/confirm-fresh-24-en-m1-source-grouped_actor_prose_withheld.receipt.json": "2143c708c74db01d9a98ad59096675530c23801dd5a2727995c48af71f98aa62", + "phase/calls/confirm-fresh-24-en-m1-source-grouped_actor_prose_withheld.request.json": "bde942a4e8576b1853c82efe924d6f8efc3190b4817492339576a1fb8385790d", + "phase/calls/confirm-fresh-24-en-m1-source-grouped_lossless.dispatch.json": "70ae09e41ff591f7b823044253b85f574b3654c5264b91a14bbac22b273492d0", + "phase/calls/confirm-fresh-24-en-m1-source-grouped_lossless.receipt.json": "b6f73b2253e5a5607a474f0bab42e9a7de89d47a749fe02af5f124c0302e6301", + "phase/calls/confirm-fresh-24-en-m1-source-grouped_lossless.request.json": "bde942a4e8576b1853c82efe924d6f8efc3190b4817492339576a1fb8385790d", + "phase/calls/confirm-fresh-24-hi-m0-source-chronological.dispatch.json": "4362fc321f2391e3d37becd1976bae37455f8b2b4cdbc6d98799cfe0c9790b69", + "phase/calls/confirm-fresh-24-hi-m0-source-chronological.receipt.json": "80d056b24310696311ca258464452070eca0e21145068b76ebbc4a294703c4b3", + "phase/calls/confirm-fresh-24-hi-m0-source-chronological.request.json": "92a820fc12cd1287e77b22c7ee6c5b8d456dcdd39f1ee987fd442850329465e7", + "phase/calls/confirm-fresh-24-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "770dc5b235b2ab8d295c96372321d4721b6b8adac27ddd11a5726b1b45613199", + "phase/calls/confirm-fresh-24-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "e616716377e81f553ea58e68d5b007260d37779c31045e4ab5a6059584b2777a", + "phase/calls/confirm-fresh-24-hi-m0-source-grouped_actor_prose_withheld.request.json": "0c3642f93e4db4412e1a594bd44d51322430b8cf4150ba4c9f3e6b2ecab50492", + "phase/calls/confirm-fresh-24-hi-m0-source-grouped_lossless.dispatch.json": "16cbc123ff763ed1ca853f83d59a5a0ab860fa6cae52bc45bc2f1d5e628effe3", + "phase/calls/confirm-fresh-24-hi-m0-source-grouped_lossless.receipt.json": "7c15d4b4072285ce824169e3517cb7de6e8fcaa52bbf2a4836d7400ea43fc554", + "phase/calls/confirm-fresh-24-hi-m0-source-grouped_lossless.request.json": "0c3642f93e4db4412e1a594bd44d51322430b8cf4150ba4c9f3e6b2ecab50492", + "phase/calls/confirm-fresh-24-hi-m1-source-chronological.dispatch.json": "f459f531c477b3690d07dca9c43cadae5cef88dad3cbe1e486f73f7b8b90c914", + "phase/calls/confirm-fresh-24-hi-m1-source-chronological.receipt.json": "89be6a4c0afa8ff9fe6a9bdf55314c01fda87ba57c14c4aab5afa9d336056af3", + "phase/calls/confirm-fresh-24-hi-m1-source-chronological.request.json": "30cdb55e61099db66e301bc15a972b782349c178fdd58f8ea63e0ddd84a4be11", + "phase/calls/confirm-fresh-24-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "f0482090e4855f212c71fb7e521d8350b15ae5d336bda7279b374adee102b969", + "phase/calls/confirm-fresh-24-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "7b5b239d5814db549697fb4c1722c58ed0d7fd2a787338023cef46133e9dbcf6", + "phase/calls/confirm-fresh-24-hi-m1-source-grouped_actor_prose_withheld.request.json": "cd85e7a9611c267e4989fc433b07bfc2aec9bcf780d9a5f779767e33ddc28433", + "phase/calls/confirm-fresh-24-hi-m1-source-grouped_lossless.dispatch.json": "4fd562f81714ff1db6cc04365582c7b24ef4a654e5b6c49ed407cc95aa6a1e2a", + "phase/calls/confirm-fresh-24-hi-m1-source-grouped_lossless.receipt.json": "c4b4df18b4efff8df8f6b2dcd93c4242a7b5b968612a20e0e66cf9f71233a4ef", + "phase/calls/confirm-fresh-24-hi-m1-source-grouped_lossless.request.json": "cd85e7a9611c267e4989fc433b07bfc2aec9bcf780d9a5f779767e33ddc28433", + "phase/calls/confirm-fresh-24-te-m0-source-chronological.dispatch.json": "bb2719679c5006eba4d67a20b83bec67a0b0ce57783026753979277a1dfc447c", + "phase/calls/confirm-fresh-24-te-m0-source-chronological.receipt.json": "b7943281005fa46518d250fef94937b6e985a9abd37a9e5e3acbf2bb77332d50", + "phase/calls/confirm-fresh-24-te-m0-source-chronological.request.json": "bc25d72a7f5d3e2d18339334d93df72cd7bfd0e5cce470091b90821d7022e4a0", + "phase/calls/confirm-fresh-24-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "23a30f0a536b77ff7d5f2a5590af9ce428e91a30ea43b74f40e1a5803771a655", + "phase/calls/confirm-fresh-24-te-m0-source-grouped_actor_prose_withheld.receipt.json": "9df7251b0301dfdf32850db6aeab8552dc793bce4e529abc3e481aa1925ed922", + "phase/calls/confirm-fresh-24-te-m0-source-grouped_actor_prose_withheld.request.json": "02d7eb4b6caf628655e4ca2d2e4f1f1b8cda51e2298e81f28486bcd0b4a85f32", + "phase/calls/confirm-fresh-24-te-m0-source-grouped_lossless.dispatch.json": "c477ed5f5a2e65101e2884ba19d018f7ef2d374cf970d50ad39c224179ae3536", + "phase/calls/confirm-fresh-24-te-m0-source-grouped_lossless.receipt.json": "ed7f7d767611beeede137ac36ded8b03ff5095dcd63035c2090418b4fbf3ed7e", + "phase/calls/confirm-fresh-24-te-m0-source-grouped_lossless.request.json": "02d7eb4b6caf628655e4ca2d2e4f1f1b8cda51e2298e81f28486bcd0b4a85f32", + "phase/calls/confirm-fresh-24-te-m1-source-chronological.dispatch.json": "be5158582872822db64663f093e5e4df409341ad1abd240d7c96b8595166e32f", + "phase/calls/confirm-fresh-24-te-m1-source-chronological.receipt.json": "0812d7d2958f7cc98b663a1ccafa78db5a86eb69e0a21cad13f2e658535009b4", + "phase/calls/confirm-fresh-24-te-m1-source-chronological.request.json": "50146f29aa47560b536cfe91cec6502b186899ef5002456b48a7c85ef4621a93", + "phase/calls/confirm-fresh-24-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "6c27b87a94f24b7b26cab137a2700ecd9cce96f47e63382189ea70c5ab87e162", + "phase/calls/confirm-fresh-24-te-m1-source-grouped_actor_prose_withheld.receipt.json": "85587eb959c375877ce60b1e413501a61c342ffef5967dd4f651d1e77c30fe96", + "phase/calls/confirm-fresh-24-te-m1-source-grouped_actor_prose_withheld.request.json": "4e15d25b78928d6eeb791dabfb55508f6c4cb1bd1ce348690d30c720ef224823", + "phase/calls/confirm-fresh-24-te-m1-source-grouped_lossless.dispatch.json": "a268fcf8de36e517ba8cedd6d4c114a90ec528deb36d1074f3be87e28e65b0d4", + "phase/calls/confirm-fresh-24-te-m1-source-grouped_lossless.receipt.json": "5d2409eb98170d75d89d69faf18bdc4c91ab946fc45c20beefbb5578a5d8813a", + "phase/calls/confirm-fresh-24-te-m1-source-grouped_lossless.request.json": "4e15d25b78928d6eeb791dabfb55508f6c4cb1bd1ce348690d30c720ef224823", + "phase/calls/confirm-fresh-25-en-m0-source-chronological.dispatch.json": "04ac1e3c8fdc35e3196e611b99fc6a6dca4d0b2e40fced97eb2df2547da27d45", + "phase/calls/confirm-fresh-25-en-m0-source-chronological.receipt.json": "5bcf182561f27d21c8d7e3cc751d10102e3308284159792b4ee29c1d79514fbe", + "phase/calls/confirm-fresh-25-en-m0-source-chronological.request.json": "1495ce5fadb503f407eb46b47ac691b3ae150cbd8e3d6f90b152a63fdadf5ff2", + "phase/calls/confirm-fresh-25-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "d517dd0b8406c07f9ea67ffa402488191a5da8e68473c3e2ddd23435f51e7e6c", + "phase/calls/confirm-fresh-25-en-m0-source-grouped_actor_prose_withheld.receipt.json": "9be352e09e90420732590829a8f8b0c8a6bd149725101ab1a020288d992cac2b", + "phase/calls/confirm-fresh-25-en-m0-source-grouped_actor_prose_withheld.request.json": "48458b25fe6f76327f15b6558f003abac47d012da8cca128c2730e376d4e34c8", + "phase/calls/confirm-fresh-25-en-m0-source-grouped_lossless.dispatch.json": "ca911424f54082d7150a6ee086fa78d5b8191ff84e55f7fc3b3243feefd5b946", + "phase/calls/confirm-fresh-25-en-m0-source-grouped_lossless.receipt.json": "23616f107f145a6a402464823ccb4f6d999484152d2878001c6975f5cb0a25a2", + "phase/calls/confirm-fresh-25-en-m0-source-grouped_lossless.request.json": "48458b25fe6f76327f15b6558f003abac47d012da8cca128c2730e376d4e34c8", + "phase/calls/confirm-fresh-25-en-m1-source-chronological.dispatch.json": "d3c5dde6387acc31ec449575e6d77a3fd19c973ae33093696fc76bdd18dc0495", + "phase/calls/confirm-fresh-25-en-m1-source-chronological.receipt.json": "549602595ba859f6cce3e47dd8e79ae444a8324f79dcd25c846ba149f0ea8f4d", + "phase/calls/confirm-fresh-25-en-m1-source-chronological.request.json": "d6ac3e7e589beeef1473e11cd9fdd7b04ed2d5b44a018dc4e89e8cda19c86241", + "phase/calls/confirm-fresh-25-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "021fd9788dabcd4a76db86f0c44919fe314b248299252a3cd114a66e2d41a584", + "phase/calls/confirm-fresh-25-en-m1-source-grouped_actor_prose_withheld.receipt.json": "f709648520a1d3d6bf5744d0fe499c4db5e16fb6c56e0f589d447582cfbc6fcc", + "phase/calls/confirm-fresh-25-en-m1-source-grouped_actor_prose_withheld.request.json": "43af4fab75dc90e0900159d0234acf9c277e55d2cc1335d5d73d8d1ef1627277", + "phase/calls/confirm-fresh-25-en-m1-source-grouped_lossless.dispatch.json": "8a759145e4b9c2d0e2452e105a84b99431e5996cabbea3929a78bbc5e3b1a2ac", + "phase/calls/confirm-fresh-25-en-m1-source-grouped_lossless.receipt.json": "3dd36ee58bebda4fde4c2870c3428ff8b0ce306b0a46675e0d70b6be3ada4b69", + "phase/calls/confirm-fresh-25-en-m1-source-grouped_lossless.request.json": "43af4fab75dc90e0900159d0234acf9c277e55d2cc1335d5d73d8d1ef1627277", + "phase/calls/confirm-fresh-25-hi-m0-source-chronological.dispatch.json": "762f2617a9c999591418de863d7963afce65570210de3336c96ad6fb906f4713", + "phase/calls/confirm-fresh-25-hi-m0-source-chronological.receipt.json": "6f84f337c5f6ebb2463b869d93f8e147a38072da3d1a130e0236fa1c3027808c", + "phase/calls/confirm-fresh-25-hi-m0-source-chronological.request.json": "bd003125a9283f1d3a7872d69b1bf448acce756401a5f0a15d937e5d3e9017e5", + "phase/calls/confirm-fresh-25-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "72d4748918cdcf3cf648c4ec6e562b572758a1f0020b44e6f0aaf2f3897af436", + "phase/calls/confirm-fresh-25-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "4c6cba77860d822b3c95c76321ed893e0448d3c1803f6e8f01b7305314fe08db", + "phase/calls/confirm-fresh-25-hi-m0-source-grouped_actor_prose_withheld.request.json": "a7fa707d6e2ad78a9d065fda59270bd970aaa00647d13513004d70232b3ea80b", + "phase/calls/confirm-fresh-25-hi-m0-source-grouped_lossless.dispatch.json": "b00ee0b75f38eaa9ad972316ccec59b42ed988e6f53479969f3a5fcd36d05d17", + "phase/calls/confirm-fresh-25-hi-m0-source-grouped_lossless.receipt.json": "73e3c8bc500b9c94180bcb8a5a2c275b60185270bc71fec954f34ffce575f148", + "phase/calls/confirm-fresh-25-hi-m0-source-grouped_lossless.request.json": "a7fa707d6e2ad78a9d065fda59270bd970aaa00647d13513004d70232b3ea80b", + "phase/calls/confirm-fresh-25-hi-m1-source-chronological.dispatch.json": "38da80cc232e4cb5716129dc974c7d0f7875e356acd4b1e010308d7c7ad95257", + "phase/calls/confirm-fresh-25-hi-m1-source-chronological.receipt.json": "504f80ff8c310f7fd6ec3c0115f05ae30fc32fabd88e3263915f1812b65089bc", + "phase/calls/confirm-fresh-25-hi-m1-source-chronological.request.json": "b69ea408ed63db441d6f68a959a5b0b4883b832d36fbcc0007ec57700e51a4a2", + "phase/calls/confirm-fresh-25-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "a7bf35065b69c751e39526189e91784c9ad2095d83104bcc71ef451fde8df10a", + "phase/calls/confirm-fresh-25-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "687532bfb8edeb9ec082ddfa7398ae8e72dadad5eef766090ea76cfc0100244e", + "phase/calls/confirm-fresh-25-hi-m1-source-grouped_actor_prose_withheld.request.json": "2eb628a65abbcf9e1b99644aed680cf40a594de396383bc7834cbd3bb74ed8be", + "phase/calls/confirm-fresh-25-hi-m1-source-grouped_lossless.dispatch.json": "cf1de31d45f6d179052a45055e6bbd6b11b7d886b93672f83f3fabad35f3218e", + "phase/calls/confirm-fresh-25-hi-m1-source-grouped_lossless.receipt.json": "d0ce4103463aa8f513567efabaec7dbf99158cd020308c5f550ad2db3b95ff09", + "phase/calls/confirm-fresh-25-hi-m1-source-grouped_lossless.request.json": "2eb628a65abbcf9e1b99644aed680cf40a594de396383bc7834cbd3bb74ed8be", + "phase/calls/confirm-fresh-25-te-m0-source-chronological.dispatch.json": "ee7ae150d94778a363e9ab03759e5a4e29b50333b487850318b5ab3451c1cacf", + "phase/calls/confirm-fresh-25-te-m0-source-chronological.receipt.json": "712127c802ba132cedc14d184828e8de6536291a09f52f38799d063a5a0bd046", + "phase/calls/confirm-fresh-25-te-m0-source-chronological.request.json": "aab198012cedf5a127b59c78b34226a23ea6fbfd756b7887d89574b635e79d41", + "phase/calls/confirm-fresh-25-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "74fdf797aadcc57b4f38e27d82ea62d05770ad5b38d0c0a00e0bfce5550e4403", + "phase/calls/confirm-fresh-25-te-m0-source-grouped_actor_prose_withheld.receipt.json": "81fa95b9ee9e29f8ff19575a0a125413415834f09289ac2673c28e95e9b2d3fc", + "phase/calls/confirm-fresh-25-te-m0-source-grouped_actor_prose_withheld.request.json": "30430c94fb13ba508e33d5a0792a78b15a20fb8166c10ce50ee60c8e4ee0c6bf", + "phase/calls/confirm-fresh-25-te-m0-source-grouped_lossless.dispatch.json": "a3bd895a12795ce4886a650f9dbf1a1eb47cf2664eda6257e6b88bc3a65cecce", + "phase/calls/confirm-fresh-25-te-m0-source-grouped_lossless.receipt.json": "026d645cd9eda92109052cce3aed52a540a867c13846490acd7f205ce049d9d5", + "phase/calls/confirm-fresh-25-te-m0-source-grouped_lossless.request.json": "30430c94fb13ba508e33d5a0792a78b15a20fb8166c10ce50ee60c8e4ee0c6bf", + "phase/calls/confirm-fresh-25-te-m1-source-chronological.dispatch.json": "f96e7a827ebba45a1c7d3c59ee821bb68c50ca7fc9097ca5506bf54c71636352", + "phase/calls/confirm-fresh-25-te-m1-source-chronological.receipt.json": "f6b78be423d9b9da47209f2bb2378a0083cddf34ec0bcd1fc534f3e4d6fef1eb", + "phase/calls/confirm-fresh-25-te-m1-source-chronological.request.json": "70116231fff9f40a75db0d2824eb38e6c9e370cde62197036c28e8be8ee6e10b", + "phase/calls/confirm-fresh-25-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "f7315b15f7c2bdbb8649a2721e952e6774460c82f73679e112f1d6fee24191da", + "phase/calls/confirm-fresh-25-te-m1-source-grouped_actor_prose_withheld.receipt.json": "de3ff9cb955ec219196fd9ce1cb89d3f0cf2ec9b7446c9181af5e13553aa5249", + "phase/calls/confirm-fresh-25-te-m1-source-grouped_actor_prose_withheld.request.json": "64438c22d4b8eb9124e8a8d286c1e539e0fdfd14c725baf7f10498268a86f50e", + "phase/calls/confirm-fresh-25-te-m1-source-grouped_lossless.dispatch.json": "d627a3fd782b82c5b74ba8065a0d6ae1e9a793bca669ccb0fd5d70b0f935ff43", + "phase/calls/confirm-fresh-25-te-m1-source-grouped_lossless.receipt.json": "97eb2c2941734115e5cf530f4660405b740db85f442255aea2652ae1263ae0bf", + "phase/calls/confirm-fresh-25-te-m1-source-grouped_lossless.request.json": "64438c22d4b8eb9124e8a8d286c1e539e0fdfd14c725baf7f10498268a86f50e", + "phase/calls/confirm-fresh-26-en-m0-source-chronological.dispatch.json": "a2df52fc9d28bd2740c9ac9490ab1b955cf3373d758e69846d81d877ed971c78", + "phase/calls/confirm-fresh-26-en-m0-source-chronological.receipt.json": "6ca611fab89543db7c0317b2707a0d83dc9fcea8ad17844712c069e07826d386", + "phase/calls/confirm-fresh-26-en-m0-source-chronological.request.json": "229fea01678034a79da0e81335f726698c21aafa81f70b079a0c5859a39f3c68", + "phase/calls/confirm-fresh-26-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "0bb66d69506b7d00e23ba51ca37be4ce4515dc04eb8adfe9f1930ba332d5c22b", + "phase/calls/confirm-fresh-26-en-m0-source-grouped_actor_prose_withheld.receipt.json": "94d3e4a3062a52d130f4c01b4bf41be51f094a550a610d9ce936c6c9b979e562", + "phase/calls/confirm-fresh-26-en-m0-source-grouped_actor_prose_withheld.request.json": "f4f1160d869bfe230f32b7614fe5fe65c7bfa25aa0acd07ac1b9f095b896fb03", + "phase/calls/confirm-fresh-26-en-m0-source-grouped_lossless.dispatch.json": "36bdfd94309509bcbc8719153796dfa18c98a61f15a6a5d6760feb33d56b1e61", + "phase/calls/confirm-fresh-26-en-m0-source-grouped_lossless.receipt.json": "5f2f5fbd756d756e39d583e0a5ce8017e87af398134faee2947757711da54c34", + "phase/calls/confirm-fresh-26-en-m0-source-grouped_lossless.request.json": "f4f1160d869bfe230f32b7614fe5fe65c7bfa25aa0acd07ac1b9f095b896fb03", + "phase/calls/confirm-fresh-26-en-m1-source-chronological.dispatch.json": "16996fe657c690341446f9db17ee9067cda6aa37b5da21de429a0a5a2c9f99e1", + "phase/calls/confirm-fresh-26-en-m1-source-chronological.receipt.json": "1d9ac62110b3f496f392471d5404bd085a80eb4ae0d73681ee0b6b6b67dc5d69", + "phase/calls/confirm-fresh-26-en-m1-source-chronological.request.json": "12ccb2112042daa0738e23ba75fae4c70b8b18b91dbd4636903ab9e49950da8d", + "phase/calls/confirm-fresh-26-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "c1c18adfc131284a49be3cc88509ac02fd364cac752fa571f33607d1a6f7f594", + "phase/calls/confirm-fresh-26-en-m1-source-grouped_actor_prose_withheld.receipt.json": "34ab42f40b287161fff29013ae0ab67336650ee9f4c00c83c6ed56036e2098f3", + "phase/calls/confirm-fresh-26-en-m1-source-grouped_actor_prose_withheld.request.json": "c9a3cf94f2d606cb83b42ebc9246c472d2986345e6e42cf85ecbb5ddfd402e5a", + "phase/calls/confirm-fresh-26-en-m1-source-grouped_lossless.dispatch.json": "0bfc712d6f7bea1d1ba9ca05b880d088561d7f3b5da06d945b5afde5db6e1248", + "phase/calls/confirm-fresh-26-en-m1-source-grouped_lossless.receipt.json": "6c3672f9df57e07b91bcf801f60fd8ce973f949b2b105e1af8fc13db7760e638", + "phase/calls/confirm-fresh-26-en-m1-source-grouped_lossless.request.json": "c9a3cf94f2d606cb83b42ebc9246c472d2986345e6e42cf85ecbb5ddfd402e5a", + "phase/calls/confirm-fresh-26-hi-m0-source-chronological.dispatch.json": "e763856b7af8bdf29f83eeb9af86b041e2355340954555ef964b92948e17b03d", + "phase/calls/confirm-fresh-26-hi-m0-source-chronological.receipt.json": "17fe31badc57c53f92d820e60dd96c9c85eeab69cd30a5d6941d8d3166e6f286", + "phase/calls/confirm-fresh-26-hi-m0-source-chronological.request.json": "44e64d354ebfea5b803c83fd1b91481b0d13e52cde4c3d0afa05cf8c48c89958", + "phase/calls/confirm-fresh-26-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "2bf286de923d43c3d69e913f37179d8d17b4b70f547dc7027b2ff5a2aa7acbcb", + "phase/calls/confirm-fresh-26-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "c2ee3a22ad9c26458a929997b147adb253ddad4a5682800e7eb1c6af3144e469", + "phase/calls/confirm-fresh-26-hi-m0-source-grouped_actor_prose_withheld.request.json": "77882b547adfb088fe931b5a8c64350c17e25298ec990f75c0d766bf8c6ce80a", + "phase/calls/confirm-fresh-26-hi-m0-source-grouped_lossless.dispatch.json": "057984135cf9c4702a8483e8a7e73876b88de6a1ee552170274b0eb715644bc8", + "phase/calls/confirm-fresh-26-hi-m0-source-grouped_lossless.receipt.json": "69c4f2f1392f58c34234b05fbdc7c8fa97ffb8cf2fe4898e642bb583057d64fe", + "phase/calls/confirm-fresh-26-hi-m0-source-grouped_lossless.request.json": "77882b547adfb088fe931b5a8c64350c17e25298ec990f75c0d766bf8c6ce80a", + "phase/calls/confirm-fresh-26-hi-m1-source-chronological.dispatch.json": "26c14b0c8551d6db333c28a5bf854b4e326e4b62aa616ebecdeb3f9e34bc2b1f", + "phase/calls/confirm-fresh-26-hi-m1-source-chronological.receipt.json": "88bc16feda1d0006497e7ab6a21b6aad9d7273276134d95b62fb1bf177077850", + "phase/calls/confirm-fresh-26-hi-m1-source-chronological.request.json": "267b63b0f1f8583664cf62423c07fc360de236feea3b1d6402b037bf0c61e012", + "phase/calls/confirm-fresh-26-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "ce5913ed5960442d346b5b617d6f5e1c5588e10aa29fec5e258a9c39b852ed5b", + "phase/calls/confirm-fresh-26-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "8428554bb9cc0a2f8c4c8585b85d97175ff51e0271c39ba42b98b9dc17c1b178", + "phase/calls/confirm-fresh-26-hi-m1-source-grouped_actor_prose_withheld.request.json": "355eb6e8915770acbc9839e5c2f8c8ba608afec593b5b78279bfd98d57626a5e", + "phase/calls/confirm-fresh-26-hi-m1-source-grouped_lossless.dispatch.json": "4f4b52a641dfc585f0a562418d52c015a03ca9027e440b7e1b1134e0e147f5d0", + "phase/calls/confirm-fresh-26-hi-m1-source-grouped_lossless.receipt.json": "3ad351cca3215592d5f9a528d2ffb01aa1f9a6217884237e48531e2a6b9632a1", + "phase/calls/confirm-fresh-26-hi-m1-source-grouped_lossless.request.json": "355eb6e8915770acbc9839e5c2f8c8ba608afec593b5b78279bfd98d57626a5e", + "phase/calls/confirm-fresh-26-te-m0-source-chronological.dispatch.json": "f41a0447b5f1458fd83f627139a72d6b93691ccaf531950468a70c74428f0ad6", + "phase/calls/confirm-fresh-26-te-m0-source-chronological.receipt.json": "25227c8b90ad47ac9b118db15cbb23e9762bb821d491ba0098cdc8740a4e9af2", + "phase/calls/confirm-fresh-26-te-m0-source-chronological.request.json": "a65cad85c8509b0bac58f34902ea5eb594964c442cb094dc5b4c783daa9cb896", + "phase/calls/confirm-fresh-26-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "2cf88aaf7f1405ee053a07450701422701b394fa33aecc27833c67f2e1bb92eb", + "phase/calls/confirm-fresh-26-te-m0-source-grouped_actor_prose_withheld.receipt.json": "dfd3a1bb4c304a1431e97541734f289877416e0e6d8270d9384f2a4a3eb2a815", + "phase/calls/confirm-fresh-26-te-m0-source-grouped_actor_prose_withheld.request.json": "481c5cfa2d725e730a011fff19c7e7be7008163c421931eb110a1e618620ba2d", + "phase/calls/confirm-fresh-26-te-m0-source-grouped_lossless.dispatch.json": "cfd92c128649fb46515973e4ed3297d9f507c0b2b62ceb5f637dd02cf5860fd8", + "phase/calls/confirm-fresh-26-te-m0-source-grouped_lossless.receipt.json": "b9eb8ecf3672423823e01c23ea523d36abe1dbf09a0315e56d31822f25fab596", + "phase/calls/confirm-fresh-26-te-m0-source-grouped_lossless.request.json": "481c5cfa2d725e730a011fff19c7e7be7008163c421931eb110a1e618620ba2d", + "phase/calls/confirm-fresh-26-te-m1-source-chronological.dispatch.json": "f0060c5c6aea955456afccc92b06e43eb64ef12ea4d781dc443f97579466f646", + "phase/calls/confirm-fresh-26-te-m1-source-chronological.receipt.json": "d452d9585a54192824e9f752069fdaab954cf586d590e331efe426d555e98add", + "phase/calls/confirm-fresh-26-te-m1-source-chronological.request.json": "118dcb447e3ffc5e62938cf58e84809333f1a8749f83656272e7a3df6781779b", + "phase/calls/confirm-fresh-26-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "7cb80811a5322d3438e859e4d6cf3f101d5d36ce8f15a2e0ffa67691c5fdad63", + "phase/calls/confirm-fresh-26-te-m1-source-grouped_actor_prose_withheld.receipt.json": "bef52fc576a47cb54474cb17bbeabee51e2e0dadf808206856a79cbcc2578922", + "phase/calls/confirm-fresh-26-te-m1-source-grouped_actor_prose_withheld.request.json": "788b00abe0b74df821e9f261602c5b894f92e71bc1cd95e6b86184f21fc1cc0b", + "phase/calls/confirm-fresh-26-te-m1-source-grouped_lossless.dispatch.json": "3b7cc5831f7e126c3b0eee4da7c7612cc4c22d97717d326b81d9c591c2beb64b", + "phase/calls/confirm-fresh-26-te-m1-source-grouped_lossless.receipt.json": "27fefb6dead6552ca485e69738d9b96efc3c74ac146d82727f06073c20b300cd", + "phase/calls/confirm-fresh-26-te-m1-source-grouped_lossless.request.json": "788b00abe0b74df821e9f261602c5b894f92e71bc1cd95e6b86184f21fc1cc0b", + "phase/calls/confirm-fresh-27-en-m0-source-chronological.dispatch.json": "6d668d71e23e111ac434c06f9ce60fb2cfeff9806a5806dc1ee38df81b77d7a2", + "phase/calls/confirm-fresh-27-en-m0-source-chronological.receipt.json": "6cab6c808f67c459cf79c6ce3e175ea59326378a40fab1c2da766c839d22a042", + "phase/calls/confirm-fresh-27-en-m0-source-chronological.request.json": "e471f0cf740e8e5fcda84ccc0c7028915dfdba19c04e3fa3d3b22f59b6c4fa00", + "phase/calls/confirm-fresh-27-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "24fcdf03e7bf34b1dfd69a33e0e8ca68825f6e7943f75d07a12a1ad538cef88c", + "phase/calls/confirm-fresh-27-en-m0-source-grouped_actor_prose_withheld.receipt.json": "e0914819ae49619c07f64bd316ded564f3dbc66f07998c36edc241ec995813dc", + "phase/calls/confirm-fresh-27-en-m0-source-grouped_actor_prose_withheld.request.json": "7aa7d3fd151ff5dcec403ea4dd05c91b29fd4b3fb66bfb59cf935eeba7d4230f", + "phase/calls/confirm-fresh-27-en-m0-source-grouped_lossless.dispatch.json": "e3ec82e2a3345714a0390ea55f73c91b334ee24c4acfe45c83e1d698574b1dd7", + "phase/calls/confirm-fresh-27-en-m0-source-grouped_lossless.receipt.json": "71128acf7166c74e719c29872bcf8786ef7314cbf932e581c6192cd1f9b83602", + "phase/calls/confirm-fresh-27-en-m0-source-grouped_lossless.request.json": "7aa7d3fd151ff5dcec403ea4dd05c91b29fd4b3fb66bfb59cf935eeba7d4230f", + "phase/calls/confirm-fresh-27-en-m1-source-chronological.dispatch.json": "eed2b00cb513b1c91c1a9bb003cea9dc706a7a5c893507c191bb9c41f42d9010", + "phase/calls/confirm-fresh-27-en-m1-source-chronological.receipt.json": "78b600bbb1646f5a9cde0e86ae2071e1dafb4fcac96002cf134e3722766d5dce", + "phase/calls/confirm-fresh-27-en-m1-source-chronological.request.json": "05c108ef811544b73e3d92e5f080bf8b3cfc1e5618b701d5eb621ec20b437bfb", + "phase/calls/confirm-fresh-27-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "78135cc46ef31e2b6729590d99b29d63e904ebecf84a7caa13a5bcd783b9fe1e", + "phase/calls/confirm-fresh-27-en-m1-source-grouped_actor_prose_withheld.receipt.json": "9252641b04295bba4d130f5ca5e45677c03e13529445ef5a22baa6e39996ac64", + "phase/calls/confirm-fresh-27-en-m1-source-grouped_actor_prose_withheld.request.json": "2a4ba997cd213fb649188dbe4137f0ab8f0673c96a9c7962f8d9adf15219d688", + "phase/calls/confirm-fresh-27-en-m1-source-grouped_lossless.dispatch.json": "d89365d521d7cb6a973a59019ae8838fb9acf45d2b1dbb9e34801d631e0045f3", + "phase/calls/confirm-fresh-27-en-m1-source-grouped_lossless.receipt.json": "f9077859f84c7e7d65be84f7fd1dc47a3cc4ddb97426b765f51f6a04877319ca", + "phase/calls/confirm-fresh-27-en-m1-source-grouped_lossless.request.json": "2a4ba997cd213fb649188dbe4137f0ab8f0673c96a9c7962f8d9adf15219d688", + "phase/calls/confirm-fresh-27-hi-m0-source-chronological.dispatch.json": "c9649d0d4f6ff3c0714c6fa98956f8cdc6f09720730cecc8ade2426584e83f16", + "phase/calls/confirm-fresh-27-hi-m0-source-chronological.receipt.json": "f4c682d84ba533831ef1a55171a1b40c856793e613925a844feb82bb30d08e5e", + "phase/calls/confirm-fresh-27-hi-m0-source-chronological.request.json": "6f976fde3cc053775c6459d84524c725f6a415f7e293a9f90aad6aa0f2f3daee", + "phase/calls/confirm-fresh-27-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "8169b34a0afd181f665ad48d63f8bea27a150de75659bcae280ca36953f66870", + "phase/calls/confirm-fresh-27-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "b40326cb20219defa7d08060c080b0ef56364b3310e6bacee95186cf82397d9b", + "phase/calls/confirm-fresh-27-hi-m0-source-grouped_actor_prose_withheld.request.json": "991944ee1be7254ac455c40f744a709d1adcc1ff2144387d44863bb5655c5963", + "phase/calls/confirm-fresh-27-hi-m0-source-grouped_lossless.dispatch.json": "3a593625dff822e54339fcc120e138ad23103c7e103f458ad887456841b6a5d3", + "phase/calls/confirm-fresh-27-hi-m0-source-grouped_lossless.receipt.json": "60ff9e7052d0d942d9ae472e3fb5dd67bc1cdc97cb210965f94ed04800c8f556", + "phase/calls/confirm-fresh-27-hi-m0-source-grouped_lossless.request.json": "0f6ce2d37bbc121ba63c3e809dda4d117725df4d469a4faa9106bf6c836bbfbb", + "phase/calls/confirm-fresh-27-hi-m1-source-chronological.dispatch.json": "bb08eab76699e7a3194aa5d82dc1fd4978c6e3dd2283167c8c63e338ca9f97e7", + "phase/calls/confirm-fresh-27-hi-m1-source-chronological.receipt.json": "6b6ef2b04ca9c52fd97a776826d1b5889b4aa611e34ac9d258762b0256e6b258", + "phase/calls/confirm-fresh-27-hi-m1-source-chronological.request.json": "f7bbecbe8f4f55ce6e2b3196c71cb038ed43a88bbf2ce154d66b5dd2c3fd8e92", + "phase/calls/confirm-fresh-27-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "1c29ab07e52ee3c3e62f76ce3650bc516b46deeb6aec3128d0c97e8835bcc3fd", + "phase/calls/confirm-fresh-27-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "8b1f111f50e6ce251322532975e224858b3c5491d7f6627f198422f7b96c443a", + "phase/calls/confirm-fresh-27-hi-m1-source-grouped_actor_prose_withheld.request.json": "4192a587e6f2613e15800f1f1915d08a6cf8958a2b31d004702bf0e7265fb467", + "phase/calls/confirm-fresh-27-hi-m1-source-grouped_lossless.dispatch.json": "e635d4429232e47f5351f4daa8b495a3ab22b4bb8c658a3b90543cd1c8316abc", + "phase/calls/confirm-fresh-27-hi-m1-source-grouped_lossless.receipt.json": "59274607fc488982d5c0f3b726731400fa54e30e805f0121ef51846d04e59180", + "phase/calls/confirm-fresh-27-hi-m1-source-grouped_lossless.request.json": "4192a587e6f2613e15800f1f1915d08a6cf8958a2b31d004702bf0e7265fb467", + "phase/calls/confirm-fresh-27-te-m0-source-chronological.dispatch.json": "07d1de4b0eaa526eb3810377be1a695f8d59d851c5915bebe9ae753634228648", + "phase/calls/confirm-fresh-27-te-m0-source-chronological.receipt.json": "ea3808dbd87498faa2ccafc23fcd1e772869cf42df1b99d0a2848018052f4e08", + "phase/calls/confirm-fresh-27-te-m0-source-chronological.request.json": "c41145be6361bd9ae870a138e9eec28036176d9b5258fa25aa35eb7fbb67c60d", + "phase/calls/confirm-fresh-27-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "29b73ae9ba5efa26ef5250517d75658990241ccff6b2659926ed3a06a70802a6", + "phase/calls/confirm-fresh-27-te-m0-source-grouped_actor_prose_withheld.receipt.json": "dd7cd49fc9127ceaf8f683b74c7996fd8bf555f73a7708b33d4d7875eae04031", + "phase/calls/confirm-fresh-27-te-m0-source-grouped_actor_prose_withheld.request.json": "948ca72b0401ce24bb219a01ccc08e3160aa6cabd2bc83105716573905fec2a2", + "phase/calls/confirm-fresh-27-te-m0-source-grouped_lossless.dispatch.json": "f5dab2278e09563c2babce62cb29a52af79be4530efdd4ea4e8e2463cdf09675", + "phase/calls/confirm-fresh-27-te-m0-source-grouped_lossless.receipt.json": "0685132f24481e8aa308e3a3e65cb86b3c2269530919fb50f1b5d5fccbfdd4c1", + "phase/calls/confirm-fresh-27-te-m0-source-grouped_lossless.request.json": "948ca72b0401ce24bb219a01ccc08e3160aa6cabd2bc83105716573905fec2a2", + "phase/calls/confirm-fresh-27-te-m1-source-chronological.dispatch.json": "9aa18b0ba67c072641e6ee8eabff89fa20201cfbd8b7526677abeb3456c27f2e", + "phase/calls/confirm-fresh-27-te-m1-source-chronological.receipt.json": "cc28ba94956035507a422e7be3ecb2e36514e22baf41336cac360c939e16fa92", + "phase/calls/confirm-fresh-27-te-m1-source-chronological.request.json": "9fe1c607264cc0cf66665fe8586b3af1c993bcd8df04f5bc2c9bc1cd84d10345", + "phase/calls/confirm-fresh-27-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "536dbaa8123064f1a2f99c0e67e071644e5c03e35a088bb539c7a4dfcd565cba", + "phase/calls/confirm-fresh-27-te-m1-source-grouped_actor_prose_withheld.receipt.json": "4bb3fc0b9e4c8fb7f2078c57e3cf30869b5a8a9b91240b25b7bd8e633745539c", + "phase/calls/confirm-fresh-27-te-m1-source-grouped_actor_prose_withheld.request.json": "2fb3971eec081cab265da36ebda00f17e00832852eabe107d2db88fedd695c32", + "phase/calls/confirm-fresh-27-te-m1-source-grouped_lossless.dispatch.json": "e69fc18c05dbe44266265d8af0f7fb40d3e5c6debe60ce24e31e191e89ec2133", + "phase/calls/confirm-fresh-27-te-m1-source-grouped_lossless.receipt.json": "d99a0667734c53c07738ed7aef902a8ae0f0531f4c41727a6979887eae05914f", + "phase/calls/confirm-fresh-27-te-m1-source-grouped_lossless.request.json": "2fb3971eec081cab265da36ebda00f17e00832852eabe107d2db88fedd695c32", + "phase/calls/confirm-fresh-28-en-m0-source-chronological.dispatch.json": "e57d0c1bb467c5302d23ad72b32f9038c7ffc069ad03326f86326fab5f36c016", + "phase/calls/confirm-fresh-28-en-m0-source-chronological.receipt.json": "981c42e59b12ede195c0f7394a62065ba6a34fe83a8bd4ff23b066df5b2a25e6", + "phase/calls/confirm-fresh-28-en-m0-source-chronological.request.json": "4bf12cca319747b35f96956b04e494c942647de232baf5f079dd3e2d246e2327", + "phase/calls/confirm-fresh-28-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "f9504abd830246210dfacdfd1128a463c974c318108eef3274d5a5d39899e928", + "phase/calls/confirm-fresh-28-en-m0-source-grouped_actor_prose_withheld.receipt.json": "b281fa7b0e6e8b26549d28e2da855317b884b1bbd81774063f3d96da7f308360", + "phase/calls/confirm-fresh-28-en-m0-source-grouped_actor_prose_withheld.request.json": "4909e25e41bdda425115c3d98d36d5064d7f4222b4b4fd7f22727f7cb3b4cd91", + "phase/calls/confirm-fresh-28-en-m0-source-grouped_lossless.dispatch.json": "f535f5dc924b960de9132371a6b17b768ba315c7fe4fda9b0c80e7ab588c998b", + "phase/calls/confirm-fresh-28-en-m0-source-grouped_lossless.receipt.json": "01f728b700a3563dc4ddf8d199af43da1ad9fb1f3dea0d0c471d158e6212b0fd", + "phase/calls/confirm-fresh-28-en-m0-source-grouped_lossless.request.json": "4909e25e41bdda425115c3d98d36d5064d7f4222b4b4fd7f22727f7cb3b4cd91", + "phase/calls/confirm-fresh-28-en-m1-source-chronological.dispatch.json": "5d287dca8f609096265ec5c320439af5a75513701b210480a428fa28ff7c6485", + "phase/calls/confirm-fresh-28-en-m1-source-chronological.receipt.json": "d187205b5e49bdd4b1488b0f9403c5ea9df682f5aaeb48fc8a5bebe96d8878b4", + "phase/calls/confirm-fresh-28-en-m1-source-chronological.request.json": "9945887171072a439d28278afd4b7ff8ac0ff699ced7d586b886c8ef00dc9ed6", + "phase/calls/confirm-fresh-28-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "d01d1f50f0100011d3cee1eb50510703daf798c54fbab6058733487ccdf9912d", + "phase/calls/confirm-fresh-28-en-m1-source-grouped_actor_prose_withheld.receipt.json": "a8a66057108d89fe53850bddf10b0cae108a322521160569a97590a0f8c29908", + "phase/calls/confirm-fresh-28-en-m1-source-grouped_actor_prose_withheld.request.json": "90982f4e596ea6376c9a737577dce45b13aec4d0507fa97ca7af7415b6bf192a", + "phase/calls/confirm-fresh-28-en-m1-source-grouped_lossless.dispatch.json": "a0585787037deec41ccf31f56876ad119aba66007d0f11fb5e67b88b5c4fa8c6", + "phase/calls/confirm-fresh-28-en-m1-source-grouped_lossless.receipt.json": "f56d6e540247f4633148bf3b9961f70468bea4d19dc56b6de0707de2bea27fa9", + "phase/calls/confirm-fresh-28-en-m1-source-grouped_lossless.request.json": "90982f4e596ea6376c9a737577dce45b13aec4d0507fa97ca7af7415b6bf192a", + "phase/calls/confirm-fresh-28-hi-m0-source-chronological.dispatch.json": "7fbf3b424cb8d3de7c1c6d28cbc0a69e848d0cff1788679c9c5f80ee0a33ee17", + "phase/calls/confirm-fresh-28-hi-m0-source-chronological.receipt.json": "5ba23cbcac0df7c7048071ca03ce18feb7a5de130448c8986b6683a2b0a5aebe", + "phase/calls/confirm-fresh-28-hi-m0-source-chronological.request.json": "b78cf77f2f5bd4f74cba95d0f91e48c2ed1d7ccaad0b4559a64554417a217a3b", + "phase/calls/confirm-fresh-28-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "2406a9fa074f2ec46d3b441d27eabed259e9ca8d9e99e9f249331cda861c55a2", + "phase/calls/confirm-fresh-28-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "120efe797b58fb2895dec5935ae26872e25d0f81c6ceabceede81462650a5863", + "phase/calls/confirm-fresh-28-hi-m0-source-grouped_actor_prose_withheld.request.json": "ff27f508141731f34f99bf1ddb5bfbfbe3bef0872a4695029c5c986bee70dd3a", + "phase/calls/confirm-fresh-28-hi-m0-source-grouped_lossless.dispatch.json": "875d5000d47d9b8930059e625eb88823edd467a7230e7c6b7b5b067c4e7de69a", + "phase/calls/confirm-fresh-28-hi-m0-source-grouped_lossless.receipt.json": "9db23ae07cc308902ce43384589bd373efaeedab0b5c623cf64907485d25e95b", + "phase/calls/confirm-fresh-28-hi-m0-source-grouped_lossless.request.json": "879edd2b01171f1ab241e4aae9c75f7567b6e4365fd526cddfd88de0417945d9", + "phase/calls/confirm-fresh-28-hi-m1-source-chronological.dispatch.json": "2a54890821343ff0bbd737a79a5de39802c3ac7270d54b120a01f6a70a921342", + "phase/calls/confirm-fresh-28-hi-m1-source-chronological.receipt.json": "65a02c802fde287d11cc796b5ceb6c21167485288d1bd50258ed3364d60af73c", + "phase/calls/confirm-fresh-28-hi-m1-source-chronological.request.json": "192ba2d46c21fbe3c82495e743fc9460a49315071e7524aef409969b628fd7aa", + "phase/calls/confirm-fresh-28-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "d3a191cd735ed5ebf84355f7b74b8128bdbb70dbd0ed4084e728ed545409a998", + "phase/calls/confirm-fresh-28-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "4bdfbf8838a36475318909bc9fdf8c9b14972dc3289f3a1ebb94ee39dc87d114", + "phase/calls/confirm-fresh-28-hi-m1-source-grouped_actor_prose_withheld.request.json": "8fb76b610890496d63546829ffc5bd97b2985cb151c235a70fac40eb167b0f89", + "phase/calls/confirm-fresh-28-hi-m1-source-grouped_lossless.dispatch.json": "3ab4e78ad0e911b6dcf18813fa9afbdb090e55369c1bbf26ea0f95ba25b1e6ce", + "phase/calls/confirm-fresh-28-hi-m1-source-grouped_lossless.receipt.json": "04aa9540762d5710f9c2931580c3349e356acf71c033fda066d2b0a749204b19", + "phase/calls/confirm-fresh-28-hi-m1-source-grouped_lossless.request.json": "0ddf439345d754383626ad084ec667f1c27650ffbd2e7de40922c36d05fce9b8", + "phase/calls/confirm-fresh-28-te-m0-source-chronological.dispatch.json": "d4571b1495f28b7e90ab85a30b3834b7b0b62e6a2e9bbd157635d7891f6ce4e4", + "phase/calls/confirm-fresh-28-te-m0-source-chronological.receipt.json": "5a44b97a675c789da363d5f8ce4121677d346f13a110bd47b78930d87c1f6742", + "phase/calls/confirm-fresh-28-te-m0-source-chronological.request.json": "86ffe108151f211eb821e3bfd781550b63421f9f5b301dff4f8be8492229c0a8", + "phase/calls/confirm-fresh-28-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "5ae63ab2ce24bfd0351e9b18476ca17ed3862a39e803bbc2b4068f4e77ff7d19", + "phase/calls/confirm-fresh-28-te-m0-source-grouped_actor_prose_withheld.receipt.json": "d1d77807a528854ac74cd3dbf3cff689b285ce0bd26d1fcf0f54f1fd4d2530c0", + "phase/calls/confirm-fresh-28-te-m0-source-grouped_actor_prose_withheld.request.json": "b90e20da052811b844a5701e6277ede17b7d40b9e8d8e80a91776f9db7d2475c", + "phase/calls/confirm-fresh-28-te-m0-source-grouped_lossless.dispatch.json": "d993bc1030123dfb0a5b104a225aced18002319d62be20bee6764185e7f4b567", + "phase/calls/confirm-fresh-28-te-m0-source-grouped_lossless.receipt.json": "f8437c3f278de57593ebb64779df2534a1d9df260879d46e56e5a7ad070f38f4", + "phase/calls/confirm-fresh-28-te-m0-source-grouped_lossless.request.json": "b90e20da052811b844a5701e6277ede17b7d40b9e8d8e80a91776f9db7d2475c", + "phase/calls/confirm-fresh-28-te-m1-source-chronological.dispatch.json": "3cb8ac59a17634144d58ba64439758518c7d0fc56e268183b00d63f699768e81", + "phase/calls/confirm-fresh-28-te-m1-source-chronological.receipt.json": "49052aa5b0c6a64d7ce79757fd97c9f6bb4ab62ffc2a11091c648195d9b55dda", + "phase/calls/confirm-fresh-28-te-m1-source-chronological.request.json": "e6ce1b3ffd054994b308cef2410dd8766fa815cefb71e5250f56ccce353c6c89", + "phase/calls/confirm-fresh-28-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "16705c6ecb8f18eb9569a9c770576c5cee12886944995f7339815188797c9e9f", + "phase/calls/confirm-fresh-28-te-m1-source-grouped_actor_prose_withheld.receipt.json": "97089578be8614b600bff9f10a46457d3e107e2b3edda52e4a6198d7aabff9de", + "phase/calls/confirm-fresh-28-te-m1-source-grouped_actor_prose_withheld.request.json": "136fdf58ffa2179f5377a58af7b1e27efb99a6942cd2c564bd7f17c599f5bf5d", + "phase/calls/confirm-fresh-28-te-m1-source-grouped_lossless.dispatch.json": "a6c890303dd73eca7d21a3d7a29cb4e3aad0f9161b25b54ae1a6a05d991d876c", + "phase/calls/confirm-fresh-28-te-m1-source-grouped_lossless.receipt.json": "06edf7746c2e60c89707a239a30d6a39e81e729d8d1a216c04e90d79cdb4609e", + "phase/calls/confirm-fresh-28-te-m1-source-grouped_lossless.request.json": "136fdf58ffa2179f5377a58af7b1e27efb99a6942cd2c564bd7f17c599f5bf5d", + "phase/calls/confirm-fresh-29-en-m0-source-chronological.dispatch.json": "3d367d58503ec8eb33ffc44f1fdf0f7e27b54bc8f8f384cb1049d55f82f742d0", + "phase/calls/confirm-fresh-29-en-m0-source-chronological.receipt.json": "2fe9aae8378c19860e6dd4c6a7de197b9e266941a9c5624367fda8709dfe3b7d", + "phase/calls/confirm-fresh-29-en-m0-source-chronological.request.json": "1a18ce556621bfc95b6d144b3adc756d82a54d5d458a20557ce85cdf2b8f2c4e", + "phase/calls/confirm-fresh-29-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "ef44abf2dd843722f524ca509ad9db6087581d03129d07b6bcc5e98ce4afc45a", + "phase/calls/confirm-fresh-29-en-m0-source-grouped_actor_prose_withheld.receipt.json": "14acc6cc77887d312dd7006fd04cb07a602c0da2b63f029fcd5cac870e2967b4", + "phase/calls/confirm-fresh-29-en-m0-source-grouped_actor_prose_withheld.request.json": "748dff3be9b9ce5092a48656e72e03bbdfbe6c586effb4b8f5a4ff5f9dee796a", + "phase/calls/confirm-fresh-29-en-m0-source-grouped_lossless.dispatch.json": "73a498e4802e65271d826a14c98906eacc615fdd0ac724d779a988690f38a883", + "phase/calls/confirm-fresh-29-en-m0-source-grouped_lossless.receipt.json": "0f5c1bf67ca546c0d1e5e01f7c9f4b27a375d9b4dd7eaf93b8534b965a9e983e", + "phase/calls/confirm-fresh-29-en-m0-source-grouped_lossless.request.json": "cf2aa803e4227dd8d7ff7eaa99d9fea955dd4b422427bd0354ae6028e4697717", + "phase/calls/confirm-fresh-29-en-m1-source-chronological.dispatch.json": "89ad0d8475df60203067918bc055122e38b4b46a48649fba12516e95680d3f8c", + "phase/calls/confirm-fresh-29-en-m1-source-chronological.receipt.json": "0425e6a4033376a33bceed51969fee3b69b45c5d56bea0fc67c9161185fc41c7", + "phase/calls/confirm-fresh-29-en-m1-source-chronological.request.json": "65b4e1bab2b8807e10b16267792c157fa695747189a885659c1c67ce3e3c0139", + "phase/calls/confirm-fresh-29-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "868017c284bb964cec491a10d9a2d7f41977d7adec8fe6f67fb17930b2e4032f", + "phase/calls/confirm-fresh-29-en-m1-source-grouped_actor_prose_withheld.receipt.json": "cb3e23f797be932b4e80cd8924062fcef4b23c116a1a3e006aab20c54523ab0d", + "phase/calls/confirm-fresh-29-en-m1-source-grouped_actor_prose_withheld.request.json": "0364f517156e8680a4d5ef17f326e276bcf1341dc88f09453b2e5021570f3656", + "phase/calls/confirm-fresh-29-en-m1-source-grouped_lossless.dispatch.json": "87a751f58915355457ef4753b025be76bb38b9c9462f70f0d04f48225af7b92d", + "phase/calls/confirm-fresh-29-en-m1-source-grouped_lossless.receipt.json": "7991706e4b5814af9f8114885076dbeb20b11f6d13bbffe67b4ab170a396d66e", + "phase/calls/confirm-fresh-29-en-m1-source-grouped_lossless.request.json": "0364f517156e8680a4d5ef17f326e276bcf1341dc88f09453b2e5021570f3656", + "phase/calls/confirm-fresh-29-hi-m0-source-chronological.dispatch.json": "9881de6549b663e968ab65eb05521e3b092b7d49822e18e195fc719f71b632ac", + "phase/calls/confirm-fresh-29-hi-m0-source-chronological.receipt.json": "6efa687fa611a9e19423e11e6b83a2292243dab651a286dd0a3a10391155b3b1", + "phase/calls/confirm-fresh-29-hi-m0-source-chronological.request.json": "1acf3f8624cde8efae821bf60673cccc86bb5b8d1cd0e1ed4dbf291192cea8fb", + "phase/calls/confirm-fresh-29-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "93de94bceea13dfda357a361a0cd51bbc35e3bba7e7aeb11ccaa14f814df1169", + "phase/calls/confirm-fresh-29-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "f2c164ca845cc1f0dc499ea44e34faca2a2169e066aa73299e43eadf3fb9c11c", + "phase/calls/confirm-fresh-29-hi-m0-source-grouped_actor_prose_withheld.request.json": "58079e1fd9430b7420679ffe6f69c95520e90eb6dce657921f05057b0e39b047", + "phase/calls/confirm-fresh-29-hi-m0-source-grouped_lossless.dispatch.json": "c5b831931b7042f58ac4cb8a9fad4dee76d474199f9fbfbfda552ec025931f15", + "phase/calls/confirm-fresh-29-hi-m0-source-grouped_lossless.receipt.json": "f638b4f4c8872fb52309ece245425409cf7a27f80e90869f2e68a1db7104b587", + "phase/calls/confirm-fresh-29-hi-m0-source-grouped_lossless.request.json": "51e699cd12358f84d5dad5f9559a92fff51940bff9072084ae78d19ab70de8d7", + "phase/calls/confirm-fresh-29-hi-m1-source-chronological.dispatch.json": "143a42ec1c2fbb1693f5a4397bbd602afd058d8272908eba34d86a00498b5606", + "phase/calls/confirm-fresh-29-hi-m1-source-chronological.receipt.json": "11a06b778a258ba27bca7d3a61bfd214c1cb0ad510a7e93787e42bd7c891e6d5", + "phase/calls/confirm-fresh-29-hi-m1-source-chronological.request.json": "fb4af0c6b8d5a4e7a61c9583db14def501c1a8dcbd0933d56de4c9a4c203b371", + "phase/calls/confirm-fresh-29-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "d9558f86cfa9e2002cd6c93d8bdbbfa50343316b9d1f853556b22212b732303b", + "phase/calls/confirm-fresh-29-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "508f7d9e6f4f2835486ede5922198f320552536f725cfaeb34b03032614ef76e", + "phase/calls/confirm-fresh-29-hi-m1-source-grouped_actor_prose_withheld.request.json": "535d6ebda70075d709e3ef3d4c0050ba7040f308304eec08be0eb913bfa525a8", + "phase/calls/confirm-fresh-29-hi-m1-source-grouped_lossless.dispatch.json": "e55520560f5ed9fc0ae8a03e0b468d696e26367d13c2eef6c9f97b986d8f5364", + "phase/calls/confirm-fresh-29-hi-m1-source-grouped_lossless.receipt.json": "a3382b5f17f7404187b1bdcfaed3c758867483b6129e5da9145ecd395792e400", + "phase/calls/confirm-fresh-29-hi-m1-source-grouped_lossless.request.json": "535d6ebda70075d709e3ef3d4c0050ba7040f308304eec08be0eb913bfa525a8", + "phase/calls/confirm-fresh-29-te-m0-source-chronological.dispatch.json": "33ea7c559e9df92469e71590cd887bd7aaa3b58cc3e72bd0bdacf5cc5aabdd95", + "phase/calls/confirm-fresh-29-te-m0-source-chronological.receipt.json": "b5a040b1ddb7709de25a6577cbd92b36c4e64ac81a69b742937cb3a252e36142", + "phase/calls/confirm-fresh-29-te-m0-source-chronological.request.json": "0dca8427ba974b69a5fa0d1ba7488087abf18640742958b019a884a6cc4d8c9b", + "phase/calls/confirm-fresh-29-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "3ab81a83458b1201c75294af9290c7839b137baa2e54807d7683de428dd2e5d3", + "phase/calls/confirm-fresh-29-te-m0-source-grouped_actor_prose_withheld.receipt.json": "d29f31f55e247d9889a74bda2a0b0854be2a0fa6e082802ad22e298ba395183a", + "phase/calls/confirm-fresh-29-te-m0-source-grouped_actor_prose_withheld.request.json": "338fae8b71dae9f117b0ef7ed488c7214cbb9b5d3ac6f94291ef80185eeff789", + "phase/calls/confirm-fresh-29-te-m0-source-grouped_lossless.dispatch.json": "dfc057877fa379b19e83f986113368bb8657324929370db7bfca61391f7a5f44", + "phase/calls/confirm-fresh-29-te-m0-source-grouped_lossless.receipt.json": "e1ba52bd127272feec14c167c5de0d17c166e7de643e4413e6e2009bace89448", + "phase/calls/confirm-fresh-29-te-m0-source-grouped_lossless.request.json": "abb02752342e22ad6594132e6eb27b04f8b66c4423731b9bd713010daf952a38", + "phase/calls/confirm-fresh-29-te-m1-source-chronological.dispatch.json": "af429eab842c43f22d99cdcfd8e6f9345220178806e5e68efd434bab21fb3880", + "phase/calls/confirm-fresh-29-te-m1-source-chronological.receipt.json": "9c2ae8aa782969bae38bfeb9c7fbae67fca3b728e72bc7ae7bd69b90f3d80205", + "phase/calls/confirm-fresh-29-te-m1-source-chronological.request.json": "297aed1b8ae73df9b8d0ecc9fe20b4bce11ff81e44300a0ec8ca1cac9578314e", + "phase/calls/confirm-fresh-29-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "b022db1a55073c8d9995e580e9abc3d08551408a9435732d0612905fadb41125", + "phase/calls/confirm-fresh-29-te-m1-source-grouped_actor_prose_withheld.receipt.json": "e7d3bf0c0a8a11d5a5c7fa8b0d46a7e0761b4add3b4f5b03c22ead36334c9b55", + "phase/calls/confirm-fresh-29-te-m1-source-grouped_actor_prose_withheld.request.json": "43d5c90ed1fe9a29eb73b60b86ff22298d5946e0427ab5bd51cd7c37ed381254", + "phase/calls/confirm-fresh-29-te-m1-source-grouped_lossless.dispatch.json": "671314569bda926e75142580b213ccc19dc92b77880704869169bfec31f0500e", + "phase/calls/confirm-fresh-29-te-m1-source-grouped_lossless.receipt.json": "4175d8e5803af5da643372f11c129a7ea4e755a214dbe92e4eeb408c192e36f5", + "phase/calls/confirm-fresh-29-te-m1-source-grouped_lossless.request.json": "43d5c90ed1fe9a29eb73b60b86ff22298d5946e0427ab5bd51cd7c37ed381254", + "phase/calls/confirm-fresh-30-en-m0-source-chronological.dispatch.json": "6443a2e5bb7a4218a17c716fcfeaf86fe092378e0f6cc34c54cb6a0d360ddaa7", + "phase/calls/confirm-fresh-30-en-m0-source-chronological.receipt.json": "31c44d9ec96a626cb464197a1e8e259d01d53d6989416978eb1a5c9acae15d73", + "phase/calls/confirm-fresh-30-en-m0-source-chronological.request.json": "50ada1ba0f471c0774a85465040185e6b2c0a8eb52db687bd199ff3641c2cf8f", + "phase/calls/confirm-fresh-30-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "12a6d6fff949453e95acbe4c75254ed6dc62c866b9799fbac1b4cae9964b3d5a", + "phase/calls/confirm-fresh-30-en-m0-source-grouped_actor_prose_withheld.receipt.json": "5f8550e37acbb943003cad89d487f101a6282254b9b5fdadef49f57ffe5e7e5a", + "phase/calls/confirm-fresh-30-en-m0-source-grouped_actor_prose_withheld.request.json": "ae6e226eac84e2a1297673e7479eee18cd79d84e51a8a9aaf4ceadba7f2d2bff", + "phase/calls/confirm-fresh-30-en-m0-source-grouped_lossless.dispatch.json": "4d54de2ea7775dee45bb1d3c3750de7d2f2817e30eada6bbc728dbbf3f365922", + "phase/calls/confirm-fresh-30-en-m0-source-grouped_lossless.receipt.json": "920681bd08a140f137593cab99fecdbda51ca629d737db8ce1c800ff58b9f735", + "phase/calls/confirm-fresh-30-en-m0-source-grouped_lossless.request.json": "de6c7d3d9d7a8146dd115060e3260488906939d134d1a5abea1c72073ac1a9f5", + "phase/calls/confirm-fresh-30-en-m1-source-chronological.dispatch.json": "5ebecc08025d78771fef186161a256761216e9205fefba7b24a38e45d474797c", + "phase/calls/confirm-fresh-30-en-m1-source-chronological.receipt.json": "240a797fd7e6bb0aa2eb71aec8b8b230ed487cac101d1869cddc05990e4a4a51", + "phase/calls/confirm-fresh-30-en-m1-source-chronological.request.json": "41dff31c23df00b099113f6fdac84399667e7f60ffb1ffb43d1404bfaabc53d1", + "phase/calls/confirm-fresh-30-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "2116e3d4e3d088c0cca78ffd706a873bc878e73ff4e9aafa0a08871f326df87e", + "phase/calls/confirm-fresh-30-en-m1-source-grouped_actor_prose_withheld.receipt.json": "94d38d901e5e4021e9a0dc93eca9515b8b2632b8fe92d4010f425e4c4ceaa5b7", + "phase/calls/confirm-fresh-30-en-m1-source-grouped_actor_prose_withheld.request.json": "6f1511145b0ba25b23ec8e4090066ad61d5d7f4297e43188a588b6365bab8cba", + "phase/calls/confirm-fresh-30-en-m1-source-grouped_lossless.dispatch.json": "985498a5f512cd7df951bf446017b7d99213f73b84ec8639b808de0ee053e6a4", + "phase/calls/confirm-fresh-30-en-m1-source-grouped_lossless.receipt.json": "4f557e6640c0c4631666279ea62cffade2ed9b08f62771956861f9d50514ddfc", + "phase/calls/confirm-fresh-30-en-m1-source-grouped_lossless.request.json": "6f1511145b0ba25b23ec8e4090066ad61d5d7f4297e43188a588b6365bab8cba", + "phase/calls/confirm-fresh-30-hi-m0-source-chronological.dispatch.json": "e772a965233d6d8d514dc5dc9bc756068638d1691ac721a5a6794d32414487f7", + "phase/calls/confirm-fresh-30-hi-m0-source-chronological.receipt.json": "6201e57d19e68ffd20e98c4338b8a23a59ddc7e41f8a437b154ab0803fc3f440", + "phase/calls/confirm-fresh-30-hi-m0-source-chronological.request.json": "0ad18d300880b6846192236be9e72321c649a240f965e2e30bf3e45396bb2820", + "phase/calls/confirm-fresh-30-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "5ede5c3045ee3e52627fb973c5264f68b0c9c31cba9c599d8636cb7813b1997a", + "phase/calls/confirm-fresh-30-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "68aac119c42fe21dbc413c027f8077d2e39175d597ef69429d7e59d1eaaba3ea", + "phase/calls/confirm-fresh-30-hi-m0-source-grouped_actor_prose_withheld.request.json": "f9f34da2d378565b42dc0836b7be7c0afdbf8870e546eff98a4695c5044fc577", + "phase/calls/confirm-fresh-30-hi-m0-source-grouped_lossless.dispatch.json": "c37b790140e84a68420a46265a5eabde41ad5110daf79ad2fc853abb791a3a83", + "phase/calls/confirm-fresh-30-hi-m0-source-grouped_lossless.receipt.json": "21ce5158b48f220f886f317a21571c0bff55262227466c14ada467671afb8f76", + "phase/calls/confirm-fresh-30-hi-m0-source-grouped_lossless.request.json": "2eaade32c33d7114d6123f793e91c4642f66ff0e89914ce85d14342100c42754", + "phase/calls/confirm-fresh-30-hi-m1-source-chronological.dispatch.json": "11667a88f5e4ee38fb49ada507df17e784cb7f750a1431fc05e51c348344b59d", + "phase/calls/confirm-fresh-30-hi-m1-source-chronological.receipt.json": "c63c0997a3d41e32c4f8660f4141100b9c474a0b4ce988fb5f7d7a85ab5bf0bd", + "phase/calls/confirm-fresh-30-hi-m1-source-chronological.request.json": "2edb8b357a920a8e5f50d411f1afc30f60d18fba3a5aced2285f72334fa83d7a", + "phase/calls/confirm-fresh-30-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "5a7d88ba1f8e255b823f0d96f0cd9362e31455580bc1f0d1f542ee41f6dea88d", + "phase/calls/confirm-fresh-30-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "2951b8a7ceb3ecc79cf8a072f6d92a03c7c015db592558a3673cb22ea14140e1", + "phase/calls/confirm-fresh-30-hi-m1-source-grouped_actor_prose_withheld.request.json": "aadbb77b779e7c11f313fd4c855463afeff441e717524112f800bc9b33c95ac5", + "phase/calls/confirm-fresh-30-hi-m1-source-grouped_lossless.dispatch.json": "fcfb84f3874918e598a7d3d2c35ff6e6d5370e4a6ff683d28d8c0066b70b71f4", + "phase/calls/confirm-fresh-30-hi-m1-source-grouped_lossless.receipt.json": "62cf8b001ae4591c35885f4d2f0e6b445d9d6f978d39c0c60f335ab2973ac474", + "phase/calls/confirm-fresh-30-hi-m1-source-grouped_lossless.request.json": "aadbb77b779e7c11f313fd4c855463afeff441e717524112f800bc9b33c95ac5", + "phase/calls/confirm-fresh-30-te-m0-source-chronological.dispatch.json": "70c22fb5f117b9944776263a298905d8ab9da0a29ee75722c94de5ad540d17d3", + "phase/calls/confirm-fresh-30-te-m0-source-chronological.receipt.json": "a0dde3b5bafd309b4f54baf92a4e2733e4446aa071b9f434662be2e8a2c1df2e", + "phase/calls/confirm-fresh-30-te-m0-source-chronological.request.json": "f30f4b5b42b0fcf7da6b3f78da8b1be1d7ace6766944814051f2c7c3514833b1", + "phase/calls/confirm-fresh-30-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "ac99db2eb5bd58571218711347c21f55d7ba1b779dde4cb1571cf828ded7c373", + "phase/calls/confirm-fresh-30-te-m0-source-grouped_actor_prose_withheld.receipt.json": "907babfb41c98fe819bfedd8393dcc06d9d4b45fb8043640c52a00d4ae166914", + "phase/calls/confirm-fresh-30-te-m0-source-grouped_actor_prose_withheld.request.json": "c2d3904c52a4805fbfb5738ac0b4575c9a676ca9abf5a447301dd7d730a54aaa", + "phase/calls/confirm-fresh-30-te-m0-source-grouped_lossless.dispatch.json": "9876c34759a1b015b8e21e3fb2df9c24138b0c3aeb4e0781a2ee0691109a52f7", + "phase/calls/confirm-fresh-30-te-m0-source-grouped_lossless.receipt.json": "5676fa64677404aefc8fc7b385a2cb36a98c0c0b8357af43f95307ee17cdd33a", + "phase/calls/confirm-fresh-30-te-m0-source-grouped_lossless.request.json": "2e346afc550e2f32d48c8ad7a2700ce8270bca82d1424cae81509e5bbac10784", + "phase/calls/confirm-fresh-30-te-m1-source-chronological.dispatch.json": "511b4c7d1e76292777347e1924fd68b3748c52e2c66e45d9fc5d39d79ea73e29", + "phase/calls/confirm-fresh-30-te-m1-source-chronological.receipt.json": "7d4115c37deb9cd162b93318536843c76bac1cf7c3f8c540b76d251cb273a221", + "phase/calls/confirm-fresh-30-te-m1-source-chronological.request.json": "3d80d37f289c544112bfade0800235d63f804dfda0bcab8c3a7a13e88bbeb315", + "phase/calls/confirm-fresh-30-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "fa533bcf8a6dcc11a2a50feffdcbbae961a669564eaf1b11b27b94f981d07e9a", + "phase/calls/confirm-fresh-30-te-m1-source-grouped_actor_prose_withheld.receipt.json": "a077d3ebec83b306740b6afbc2fe272dd5a1d3cfe96b9ab15a000f0a4cbd2f72", + "phase/calls/confirm-fresh-30-te-m1-source-grouped_actor_prose_withheld.request.json": "dc5f2f3edbb78b4f9a2cf06b51c7c8308dab128419034ab512fb541f6676c0ad", + "phase/calls/confirm-fresh-30-te-m1-source-grouped_lossless.dispatch.json": "2883b0a5f863b0382dd7226094a8889b05565bd76d59c57a0f67c1becbd7887e", + "phase/calls/confirm-fresh-30-te-m1-source-grouped_lossless.receipt.json": "260deb414b93408a9b0058f650c1ee4d67404392b4af73a9971520e2f5c4d002", + "phase/calls/confirm-fresh-30-te-m1-source-grouped_lossless.request.json": "dc5f2f3edbb78b4f9a2cf06b51c7c8308dab128419034ab512fb541f6676c0ad", + "phase/calls/confirm-fresh-31-en-m0-source-chronological.dispatch.json": "9daebfe3213ee3994477896c47ffbbf4a0140ea18441e97bcfa57f2402148ac3", + "phase/calls/confirm-fresh-31-en-m0-source-chronological.receipt.json": "f21b9c7452f3666cb074e8a49ce5758cfb2c9420af221e0a3ae5e6b33a1aec89", + "phase/calls/confirm-fresh-31-en-m0-source-chronological.request.json": "5f111add8dbd10f52c94d1eec4751c86d55460ffd921de838b815a64aa803ea1", + "phase/calls/confirm-fresh-31-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "ec7a6ca82e767eb8ebe27df369b48963733c25d983a87cf98403ecc32819514b", + "phase/calls/confirm-fresh-31-en-m0-source-grouped_actor_prose_withheld.receipt.json": "a7c5eb13f87a9934eac2d14a2d0ea1257038d76c66c273ff5e01909e22161379", + "phase/calls/confirm-fresh-31-en-m0-source-grouped_actor_prose_withheld.request.json": "ebb71946b830e5547a04ffd1db5c23fccf61cfcfe79c5f64e889fa45d6c121e3", + "phase/calls/confirm-fresh-31-en-m0-source-grouped_lossless.dispatch.json": "7116b0a7343bcb4949530a5686e43b8398b90f2720a812259155366d0b039fb2", + "phase/calls/confirm-fresh-31-en-m0-source-grouped_lossless.receipt.json": "0d2a44d3d37eca5e61a82685095ae600b18554edef1f4dd09e447ac0c251b0ae", + "phase/calls/confirm-fresh-31-en-m0-source-grouped_lossless.request.json": "97de427a6b31ae2841278272a3c7b5f9de36edab5972d73036b953d343eaa446", + "phase/calls/confirm-fresh-31-en-m1-source-chronological.dispatch.json": "ebdd07ecbb2cd8174002451e0c54eb26562d1dc2e5f3b1e4dc0e10008c79521d", + "phase/calls/confirm-fresh-31-en-m1-source-chronological.receipt.json": "c3626b6c7bb56abdf0c54618534c0ef46083eefa0a0c2d444e0a5f11ba8f93de", + "phase/calls/confirm-fresh-31-en-m1-source-chronological.request.json": "2162ba1711547b3550715f7f82ac609e01f13fac1ed35e226e62e527d922ca75", + "phase/calls/confirm-fresh-31-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "750bbfcc38a3f7449852066f87a0d12d0dbb947175b5553b7ea7d04426bf49b6", + "phase/calls/confirm-fresh-31-en-m1-source-grouped_actor_prose_withheld.receipt.json": "670087fd1f9fe65747cf7c04fcf8e38739ad9e27929b55b8fb8cb90d1a9248d8", + "phase/calls/confirm-fresh-31-en-m1-source-grouped_actor_prose_withheld.request.json": "ba0b908b932485cc5bf0cec21c510818dee5f63f472c94be0735dc1be60ced39", + "phase/calls/confirm-fresh-31-en-m1-source-grouped_lossless.dispatch.json": "7be2da5fedda13349603509b7c9cdded6f491d9abcda6cff9495a20f4bab818a", + "phase/calls/confirm-fresh-31-en-m1-source-grouped_lossless.receipt.json": "55a6940bf185837f550d66434d1fdcf486901ec7abd82eadeb285ef3a93b7e6b", + "phase/calls/confirm-fresh-31-en-m1-source-grouped_lossless.request.json": "ba0b908b932485cc5bf0cec21c510818dee5f63f472c94be0735dc1be60ced39", + "phase/calls/confirm-fresh-31-hi-m0-source-chronological.dispatch.json": "03bbe151c49019ff74f69f003c59117002a2509f47719ff1ce7b21e7171d9c7d", + "phase/calls/confirm-fresh-31-hi-m0-source-chronological.receipt.json": "23d5cf6f2e4741306acfcf8a4b267367ce4a547154fc84727233aa1519d6d4d7", + "phase/calls/confirm-fresh-31-hi-m0-source-chronological.request.json": "21161e5285a9e0252599a8f6c4160cd6179e7e1f6e48f8e76604e5bd32cd255e", + "phase/calls/confirm-fresh-31-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "1e55daccffa8d9279a00cba83c542fddcbcaf20199ff8c8411953554d0981ab9", + "phase/calls/confirm-fresh-31-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "5311d4204754c1ca2d262b0046f249c77f700cb5953f3432951bf804d3b40798", + "phase/calls/confirm-fresh-31-hi-m0-source-grouped_actor_prose_withheld.request.json": "edc12a65cd200f26f4ce18f82ca5b47982d334952f447014ad5a84056cdfc5a5", + "phase/calls/confirm-fresh-31-hi-m0-source-grouped_lossless.dispatch.json": "4b4e60790e17c84553c1624b01a92777f0679614d9fd39fb9a000baf945d4e5d", + "phase/calls/confirm-fresh-31-hi-m0-source-grouped_lossless.receipt.json": "3339ce6287e23954764217154664bf6151501a136ea6844819cdacdde7a04698", + "phase/calls/confirm-fresh-31-hi-m0-source-grouped_lossless.request.json": "91bfd470babdab4ffa762aa37c7c56f5916c6cca993a3990ed4a8403a1ee0c74", + "phase/calls/confirm-fresh-31-hi-m1-source-chronological.dispatch.json": "f839df03a14e2a5563440f7eb898985e79896a550c1556014bf31ec771539532", + "phase/calls/confirm-fresh-31-hi-m1-source-chronological.receipt.json": "00accf7cf822ca23f26fc7f432034e00ff2fe7637cfa60bf8e3bc95018716273", + "phase/calls/confirm-fresh-31-hi-m1-source-chronological.request.json": "b49fb6bf0c9ab65fa1701b1eac434654b904d5ce09e5e228a05b715e55bdb549", + "phase/calls/confirm-fresh-31-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "449b6c79a4e24931036caba7f0f219941d669a6cb46f9145f4d087eb5b820e88", + "phase/calls/confirm-fresh-31-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "5d2229d09cd569be811748fb31f623ebdc492a877bcfa9aa0e40d364758bc652", + "phase/calls/confirm-fresh-31-hi-m1-source-grouped_actor_prose_withheld.request.json": "74fde2bf0de16d23efedfc81b097f9fe40fcccf17a508a1fe6dbcf25c7a3b41d", + "phase/calls/confirm-fresh-31-hi-m1-source-grouped_lossless.dispatch.json": "1b0c77ffdda0932ee69bc1c31297131d74358abb97c02aab262c8c7fc16a6a4b", + "phase/calls/confirm-fresh-31-hi-m1-source-grouped_lossless.receipt.json": "30dea4b81f9687cc49962366317f9c69badcedda6dfedd3278630b269d3adbb0", + "phase/calls/confirm-fresh-31-hi-m1-source-grouped_lossless.request.json": "74fde2bf0de16d23efedfc81b097f9fe40fcccf17a508a1fe6dbcf25c7a3b41d", + "phase/calls/confirm-fresh-31-te-m0-source-chronological.dispatch.json": "72fd3a3499cc7cc651722a8c9209905a6e1724d859cbe31fa932ae68b2b0a4d9", + "phase/calls/confirm-fresh-31-te-m0-source-chronological.receipt.json": "4892fa785936fe22ae34cb2321f2468d45d7258e60a103d878fc8bd788f7b825", + "phase/calls/confirm-fresh-31-te-m0-source-chronological.request.json": "b7e90b0c7d0f24e2f25bc900fab0b0aafeae731a3c6d3204652d5e729a41bb58", + "phase/calls/confirm-fresh-31-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "c980cf50746d4cb9dcd559c4b00da2d7f9a372fd7510150f034d03d29a8a3d09", + "phase/calls/confirm-fresh-31-te-m0-source-grouped_actor_prose_withheld.receipt.json": "36dddc42e4faea6c52c935d799d77d78b1f4adf0119dc99edc738f8ec0ceef9b", + "phase/calls/confirm-fresh-31-te-m0-source-grouped_actor_prose_withheld.request.json": "5be84070603b5e5f5d98e55212c08e928ebf893a3fd4352697bf7639ee059f31", + "phase/calls/confirm-fresh-31-te-m0-source-grouped_lossless.dispatch.json": "0482222162ee7cdcfcfdce9c9417d368fed15384d3b3ab1473b963473260f5ed", + "phase/calls/confirm-fresh-31-te-m0-source-grouped_lossless.receipt.json": "94dd382c6b98e8dcc4e114b28efe5f4b8e3bb0888bbd6646fbb7b378cab7da13", + "phase/calls/confirm-fresh-31-te-m0-source-grouped_lossless.request.json": "9568899c36e57c6ccdc7ad4850a7f51407f798d2f7eef99944563b8d66eee75e", + "phase/calls/confirm-fresh-31-te-m1-source-chronological.dispatch.json": "d3622eaf17fb81b18672e31e0d7e4a05030658a98ad215c3ed5bab660a3e7cef", + "phase/calls/confirm-fresh-31-te-m1-source-chronological.receipt.json": "70f4510a76003e14e2d6c042f16c0105694fd1f900fb8925c5ae30f478052f64", + "phase/calls/confirm-fresh-31-te-m1-source-chronological.request.json": "af15d60f54aae9b1fa36dfb9174c05d5d888e3768011f89ca2ce19de4fd23686", + "phase/calls/confirm-fresh-31-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "905d2cbbf63da5601e481d6593e3ec1107d7322bc1df1c9dc3d869fa2711bcd0", + "phase/calls/confirm-fresh-31-te-m1-source-grouped_actor_prose_withheld.receipt.json": "f5bf67261209ff721d2fe021fac9a71a96f033346fa7418b0c6a98ed3b1810ff", + "phase/calls/confirm-fresh-31-te-m1-source-grouped_actor_prose_withheld.request.json": "02c1f081ab31b430698272b5745efb65ecb20367a07c25b29f1bb3f395f10ec1", + "phase/calls/confirm-fresh-31-te-m1-source-grouped_lossless.dispatch.json": "238a9793108d4f2bdde1d235bd179cd9c61d8da69bd5eed9fa44699bdbf60e42", + "phase/calls/confirm-fresh-31-te-m1-source-grouped_lossless.receipt.json": "f7a462ddcb7c8d7d31305ea95e61c9362e6d17dbd874f50bc5b19c285bf3d690", + "phase/calls/confirm-fresh-31-te-m1-source-grouped_lossless.request.json": "02c1f081ab31b430698272b5745efb65ecb20367a07c25b29f1bb3f395f10ec1", + "phase/calls/confirm-fresh-32-en-m0-source-chronological.dispatch.json": "7fa0c7bd4de828d3e0c18957ad3d2e67c5da8afac25e22f992bd084f0fdc7ab5", + "phase/calls/confirm-fresh-32-en-m0-source-chronological.receipt.json": "3efac79b9f06156fcad1d3a620c1eae89fe0a054395a928dd6f6cd6a4570fb51", + "phase/calls/confirm-fresh-32-en-m0-source-chronological.request.json": "79bfac6be82582882583af133d9a32dbef1865568bd13fa29df9948a5d05d500", + "phase/calls/confirm-fresh-32-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "b6e926cf42d1ce8b95bf02bb0b57092c8039c4da28a24c0608c7c67c740b5349", + "phase/calls/confirm-fresh-32-en-m0-source-grouped_actor_prose_withheld.receipt.json": "6809eb6f063aa406be0b270fae898b88b9ed37a016ad3b9f229b0e37d29343a2", + "phase/calls/confirm-fresh-32-en-m0-source-grouped_actor_prose_withheld.request.json": "3f5a50ccf4c18e771ebb60dfe067626df4e6010a72a25bdb261e390d81365ea7", + "phase/calls/confirm-fresh-32-en-m0-source-grouped_lossless.dispatch.json": "1f91fec316e350901c7fff20ed48a99eea2f6240a509e17cb4a770f5595ba3a6", + "phase/calls/confirm-fresh-32-en-m0-source-grouped_lossless.receipt.json": "dbce3425f81d6e815c8a032c733a7cd09b15b593c4604f601d1e472c588b5950", + "phase/calls/confirm-fresh-32-en-m0-source-grouped_lossless.request.json": "4d1684112617f5daa66dcc2fe711c770b933f0abe1fc568d998befbe8097c65f", + "phase/calls/confirm-fresh-32-en-m1-source-chronological.dispatch.json": "f16aadfd28686ce832c0fc4740d171cd200c78a9a8a54d41754e858bb805f2ca", + "phase/calls/confirm-fresh-32-en-m1-source-chronological.receipt.json": "c15f7f74b9b5cd66fc8a1ba5506894197f80985686bbecdc3696d86059de11f5", + "phase/calls/confirm-fresh-32-en-m1-source-chronological.request.json": "f74bf5dc9adba6a91b947fa6241ad689f0b33fc7df23c1ca8732c8c3fc1b3350", + "phase/calls/confirm-fresh-32-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "47cea7af727fa89c4af6b7d7603f2d443d9a1f22b569eb52468f738543952365", + "phase/calls/confirm-fresh-32-en-m1-source-grouped_actor_prose_withheld.receipt.json": "b5d737df394814e5282291af3cc163b4c1e630aa19db6bf49217be0c95a255f8", + "phase/calls/confirm-fresh-32-en-m1-source-grouped_actor_prose_withheld.request.json": "9bf63a7208c7a9d57cea769e74ba001b54c68af9f63594a9f902178b84ae8cb5", + "phase/calls/confirm-fresh-32-en-m1-source-grouped_lossless.dispatch.json": "aeff72eb6ab7f53fb9f57151741ae2363b3e2b3dd0d298991eef79140769d6c4", + "phase/calls/confirm-fresh-32-en-m1-source-grouped_lossless.receipt.json": "3124ee0bf50d934f4c72845cac8a3ffc48e0e55d6cca910725c44edf8b187516", + "phase/calls/confirm-fresh-32-en-m1-source-grouped_lossless.request.json": "9bf63a7208c7a9d57cea769e74ba001b54c68af9f63594a9f902178b84ae8cb5", + "phase/calls/confirm-fresh-32-hi-m0-source-chronological.dispatch.json": "6c367a241fd0ec81c0bf704104bec19f4a6bb8d2a2da4c2f1125ba98f27c6e6e", + "phase/calls/confirm-fresh-32-hi-m0-source-chronological.receipt.json": "a776a0244293072c7e147378f6a00a994ca230a162ae1d5f0f9b148c72236003", + "phase/calls/confirm-fresh-32-hi-m0-source-chronological.request.json": "75a0f621d6e82179816c98534801d03f4df3c2ff777a8a27eadcc9a292c60b54", + "phase/calls/confirm-fresh-32-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "128fee05afca14db2a1b790fb6a8469080acc84384d1d460f29c14fa0ba17ace", + "phase/calls/confirm-fresh-32-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "0431467b547cb0cd94bb5e08f64d65c56855d92f2bf2df5721a40a584c5dd726", + "phase/calls/confirm-fresh-32-hi-m0-source-grouped_actor_prose_withheld.request.json": "3390da7b4b38e4d799d76d61decc2d84f43a5757fdf13eca4b7bd60169c568f4", + "phase/calls/confirm-fresh-32-hi-m0-source-grouped_lossless.dispatch.json": "65274b06a59e4c52a49c975afc2418e003e3de4f24fd850d0ca6716b95ef4989", + "phase/calls/confirm-fresh-32-hi-m0-source-grouped_lossless.receipt.json": "4d599d18bba2fe299292b31815b94331525a745eb23280e046e0b6e5f9b0eb82", + "phase/calls/confirm-fresh-32-hi-m0-source-grouped_lossless.request.json": "4161071f5c3a0b011e479547a9676b1114c91162211d290f57eda5569d6143dd", + "phase/calls/confirm-fresh-32-hi-m1-source-chronological.dispatch.json": "e5449fd68322ad3386644ceee349ece971a0c151e8531e7102c21214775bd704", + "phase/calls/confirm-fresh-32-hi-m1-source-chronological.receipt.json": "ee8fed0d722a676091ed53679c70c263d232e712be55e53e4b8f5c97afe5dfea", + "phase/calls/confirm-fresh-32-hi-m1-source-chronological.request.json": "0b69eadadefca06de19baa97dd52506c70c6c752a1c55d8699f40f9a4566e3a1", + "phase/calls/confirm-fresh-32-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "a3fcd40a3418610482ab507f07c319ca29a0c39e4bb7523e7d6edb366f047fac", + "phase/calls/confirm-fresh-32-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "b2790034d508d7c532989f41976975e72b6fdc4dc2799e92ef8dcc15e833c6ad", + "phase/calls/confirm-fresh-32-hi-m1-source-grouped_actor_prose_withheld.request.json": "5e04546fbd2c3659496c5723e3f6f6c7f15c42f111c0765e2e1d7e697688b91b", + "phase/calls/confirm-fresh-32-hi-m1-source-grouped_lossless.dispatch.json": "3cc554c73d3e99771e70b3956d24fe6914cf8807c6db3b3d0cb6e49ac1b1d575", + "phase/calls/confirm-fresh-32-hi-m1-source-grouped_lossless.receipt.json": "b6e97d4530bbc3931ace8a895c2318485186ba29c3098c55c2192bee89290f64", + "phase/calls/confirm-fresh-32-hi-m1-source-grouped_lossless.request.json": "5e04546fbd2c3659496c5723e3f6f6c7f15c42f111c0765e2e1d7e697688b91b", + "phase/calls/confirm-fresh-32-te-m0-source-chronological.dispatch.json": "62d28de7cac871a4e546d54cb4ecc9bbf523b7daff60883e2709d5c36d0a6ef2", + "phase/calls/confirm-fresh-32-te-m0-source-chronological.receipt.json": "e30c2ec2d1007660b95af8a47378afbe90f29177be5d7643ddda5a7bc0bb0cc1", + "phase/calls/confirm-fresh-32-te-m0-source-chronological.request.json": "a164908edb1d18918dd01fdea44045180ec22a945b16eb56f087e18f7e5796de", + "phase/calls/confirm-fresh-32-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "c78c2b1404f6ab4302d8351a27ad7b901f94c2e5e7b9d36ea2c75a15f7cba419", + "phase/calls/confirm-fresh-32-te-m0-source-grouped_actor_prose_withheld.receipt.json": "4bb6c0a94cd86eacb7920d528d4e1f2ff57e3c3708fd3b27edc42723b4b9ddce", + "phase/calls/confirm-fresh-32-te-m0-source-grouped_actor_prose_withheld.request.json": "d84251ae8f4dc921418046d42655a180ab539ddb725dde43dffad20a292b37aa", + "phase/calls/confirm-fresh-32-te-m0-source-grouped_lossless.dispatch.json": "5655c941af9f870970fb6082cdfaed3493385d6e6e70f64ed28143c28a4401da", + "phase/calls/confirm-fresh-32-te-m0-source-grouped_lossless.receipt.json": "dd84a2e3339cf4ecbb6eb3c45307b1c2dabb344c4fe348a1877b15981f254ba3", + "phase/calls/confirm-fresh-32-te-m0-source-grouped_lossless.request.json": "a1fa4d38f3e7e74f5aa61679d0368d7d5bc64b988dd12c58ce7ffe33e2bb19ec", + "phase/calls/confirm-fresh-32-te-m1-source-chronological.dispatch.json": "9ff8411b40bad07cdfbd08194cc4a5af250de0f1a815419c35608aa187658d9e", + "phase/calls/confirm-fresh-32-te-m1-source-chronological.receipt.json": "e7fc2f0439f5f6ff492564b3e01da6383e13ae1f6067d9129f5be588a726c312", + "phase/calls/confirm-fresh-32-te-m1-source-chronological.request.json": "9ee32977f53eb753647e2956407358f30d3e76b8e5d123a55db642cdbe5a65b8", + "phase/calls/confirm-fresh-32-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "d57f9eb9d04491a13fda250f006754aa60e73e67fe86ba942144d32fc4ce33ca", + "phase/calls/confirm-fresh-32-te-m1-source-grouped_actor_prose_withheld.receipt.json": "a0d028fea9aedbfd066fc27727f8f0aa97867bf3f31e1248cbc7286a27d14a85", + "phase/calls/confirm-fresh-32-te-m1-source-grouped_actor_prose_withheld.request.json": "b1838644b04dfd4f59ea36c09e9ae46d03f96e65842d5f992081f2c79ea42de6", + "phase/calls/confirm-fresh-32-te-m1-source-grouped_lossless.dispatch.json": "9975b0fc47ab7c499c48bfa65886fbefff888ca6951f1efbaf95e79bd58c5716", + "phase/calls/confirm-fresh-32-te-m1-source-grouped_lossless.receipt.json": "f69ebbbc8e65ff57f126b715ba40f29c405875f6982acd15cc4aee927b5d474b", + "phase/calls/confirm-fresh-32-te-m1-source-grouped_lossless.request.json": "5901b5c5e5fdb430ec768879a65f701eec187d10fc6626b4d225aa415cda93af", + "phase/calls/confirm-fresh-33-en-m0-source-chronological.dispatch.json": "4535ea23a91499fd0d1580bcd3677bfa39ebfa819366f2b758bbc5c64e4489ca", + "phase/calls/confirm-fresh-33-en-m0-source-chronological.receipt.json": "2e265c7c2470007d4486e17d6544cd823607771907b87cee771fb45e0634232f", + "phase/calls/confirm-fresh-33-en-m0-source-chronological.request.json": "28e7143206b68b0785d7ef35f731021b0eb4929a5c2eafce87f08af3146c0209", + "phase/calls/confirm-fresh-33-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "d45e8f04d618bd3a76927fb94086481043713687581258f851aafa348b3217ab", + "phase/calls/confirm-fresh-33-en-m0-source-grouped_actor_prose_withheld.receipt.json": "f7eb78798a4a6b5ccc0797787a9382c28cc9d1ddbedde9f13f7581cb3207cf14", + "phase/calls/confirm-fresh-33-en-m0-source-grouped_actor_prose_withheld.request.json": "64301fa88c540bd6b7d100aa34373bbdddfb666fe5a5f58aebb05bb201c0fb27", + "phase/calls/confirm-fresh-33-en-m0-source-grouped_lossless.dispatch.json": "40f55c06f4b4362952ffd5360b7c2600f64c72671f406ff97e420e786a787cb2", + "phase/calls/confirm-fresh-33-en-m0-source-grouped_lossless.receipt.json": "2b71a4e9df31591317e6ce4e439b293b134730b0581ce29813c472088c2abb3c", + "phase/calls/confirm-fresh-33-en-m0-source-grouped_lossless.request.json": "bab3a860302865da6c7592e60f06fddfe499d10035bdad8c799b4cda7ff97caa", + "phase/calls/confirm-fresh-33-en-m1-source-chronological.dispatch.json": "af6b46ffe177cc785910d3138767508fa7094c11cd69cf6e947f350f2127d710", + "phase/calls/confirm-fresh-33-en-m1-source-chronological.receipt.json": "3c61572d23ed307b30e0974932295bd3811b7c7167b1f334045fca2caff86345", + "phase/calls/confirm-fresh-33-en-m1-source-chronological.request.json": "ad751e50ecb111dda950842bf2a527f0c4356b280192b315403b22ac9dae86df", + "phase/calls/confirm-fresh-33-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "d6a8de7000aaaf047c01d6556593762384e7822a207e128b121df717a752179a", + "phase/calls/confirm-fresh-33-en-m1-source-grouped_actor_prose_withheld.receipt.json": "8da87e57b94455e939896f54fe1032e0bf5df42716f52a274b6ffce5a6629e80", + "phase/calls/confirm-fresh-33-en-m1-source-grouped_actor_prose_withheld.request.json": "e8bc16d73c5c842d83ee559df70e1dbbb9728e1edde3e298e8d8fec930b28c4a", + "phase/calls/confirm-fresh-33-en-m1-source-grouped_lossless.dispatch.json": "1261601f44beec57345f86d523807f7df6a5fa17b697a6a48eea9f1f2a532a54", + "phase/calls/confirm-fresh-33-en-m1-source-grouped_lossless.receipt.json": "7a6f9b1051f2a010737adb5b8c70be0402891427a99c145912b58c3e988cf269", + "phase/calls/confirm-fresh-33-en-m1-source-grouped_lossless.request.json": "77debb548bc7505b53a51eca1954d184093b3d3c71fd407da687d87d9e31fccb", + "phase/calls/confirm-fresh-33-hi-m0-source-chronological.dispatch.json": "59f1c97ef1dd0d55efd1819ab33075d791e0aa6155903fb9a901e434588d2930", + "phase/calls/confirm-fresh-33-hi-m0-source-chronological.receipt.json": "61223f91fe64d2d4a85c7a84d969291fe5661c687e1d3273aab2a0190c4953e7", + "phase/calls/confirm-fresh-33-hi-m0-source-chronological.request.json": "b5d76fb7db7b2ec4bc31064a180812d88546a10caf31de6c687d9f4bd460f4fc", + "phase/calls/confirm-fresh-33-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "72035fd411f3ae25b0a2ede1fc88ba4ec639a2e43437fc22c8db6956f418e90d", + "phase/calls/confirm-fresh-33-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "54352f59fd1e0c18d7fa84ef4eecd65de4c89542141c7bd2587f1927e010c66d", + "phase/calls/confirm-fresh-33-hi-m0-source-grouped_actor_prose_withheld.request.json": "8d70afe88fb9e33c4b3e1d334e772586bf7a2c16327ef318fe494cfe6250514a", + "phase/calls/confirm-fresh-33-hi-m0-source-grouped_lossless.dispatch.json": "5b291c909e1d6d30e154cdfafb94e41b84ffb396afc22ccf32d93c70d97dff9b", + "phase/calls/confirm-fresh-33-hi-m0-source-grouped_lossless.receipt.json": "a6f2ec0c68bfffa7f63baca3662ec019b92beb292ae5ecb024f7e805cb58fb77", + "phase/calls/confirm-fresh-33-hi-m0-source-grouped_lossless.request.json": "dc90034c275d9cfc76c1fc726c061a1cac174d6791fedbcb4e527653187a6ad8", + "phase/calls/confirm-fresh-33-hi-m1-source-chronological.dispatch.json": "d135374c183b69bb450a17e5de46d99a661e3f5bdf50434f17266968e690b4db", + "phase/calls/confirm-fresh-33-hi-m1-source-chronological.receipt.json": "9c77af1980074955daf3519461b0db13782e5fb99856af5465472487c2cab35b", + "phase/calls/confirm-fresh-33-hi-m1-source-chronological.request.json": "0f6f30e7bebf3678c918a4d84459de6733a756b109e31da39d678204ec0e3c99", + "phase/calls/confirm-fresh-33-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "6398802970d4042bf77f63424284a530bde1a78ad32aaf3ec7fe368934c8fe7a", + "phase/calls/confirm-fresh-33-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "d9343555ab4171bd5d0d73fdcba69e674f4d5cb9666963cf154946396ecbe92c", + "phase/calls/confirm-fresh-33-hi-m1-source-grouped_actor_prose_withheld.request.json": "1f95b67e862796bd0b892225d94122ac759abf40d27bd8dbf1951368634e46f0", + "phase/calls/confirm-fresh-33-hi-m1-source-grouped_lossless.dispatch.json": "e9ad9118c0fa5d4681fbcb8af33da4161517c3a4f2854e0e83bf938b09ab61c4", + "phase/calls/confirm-fresh-33-hi-m1-source-grouped_lossless.receipt.json": "026a7c13d45934a361f6b1e34492b107d9437153b9f602810ae8de90ab67b5ac", + "phase/calls/confirm-fresh-33-hi-m1-source-grouped_lossless.request.json": "e44452e8770e844ffa0d774b5a56d2c2ad7b978425e469186421c69f57c979b0", + "phase/calls/confirm-fresh-33-te-m0-source-chronological.dispatch.json": "3fef12ff21ca8637bf634d1266e65835de0a7ffc69d325587fc80a5498e3c7a8", + "phase/calls/confirm-fresh-33-te-m0-source-chronological.receipt.json": "f6139fe66e155f090fc826d7a0b35e577b343dca56f4dfb5eca25573e7dcb586", + "phase/calls/confirm-fresh-33-te-m0-source-chronological.request.json": "d3cdd4d90a63811dec5c655224daf118a673ef025060c2292832fd30b5340940", + "phase/calls/confirm-fresh-33-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "a62229f204d3e11accbecf37fd223e2cc92d8b31fcd610674196ab0d675395b5", + "phase/calls/confirm-fresh-33-te-m0-source-grouped_actor_prose_withheld.receipt.json": "3f41a05585fdabdd99a616e4e7bcb2c2902b351b60e82bc17b39e7953d72c0c8", + "phase/calls/confirm-fresh-33-te-m0-source-grouped_actor_prose_withheld.request.json": "89975aa11eca7b0435d5c16df60a7336c5f25ea6ba3b0e2113642dc21e78ed25", + "phase/calls/confirm-fresh-33-te-m0-source-grouped_lossless.dispatch.json": "72831dc3c60fee1d9094aee8f0aea61b6db4f6d4332a077b747a95e0b89effd8", + "phase/calls/confirm-fresh-33-te-m0-source-grouped_lossless.receipt.json": "0978c52f6f5048446404cc73babcfbec19ac31033d0ebda3abe0778036973c36", + "phase/calls/confirm-fresh-33-te-m0-source-grouped_lossless.request.json": "68ad7573ffab8bdf42e971dcce5b9faaf65edbb18013eae4d738f1e357bad0ac", + "phase/calls/confirm-fresh-33-te-m1-source-chronological.dispatch.json": "29090304cebc1bfe1564c636444182ddd75c3a85e588ab049af91ed27741f6bd", + "phase/calls/confirm-fresh-33-te-m1-source-chronological.receipt.json": "cd74270303af65c5f5247db599859a11d408efcfba1420e2e11ea034789f9de7", + "phase/calls/confirm-fresh-33-te-m1-source-chronological.request.json": "e08011474cebf6774f08d9ad71211fe7b8ce77bbd0daddb1765b1ad509e17e71", + "phase/calls/confirm-fresh-33-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "19ba5421ea7575a520887854b525b971a3b61fe41baea44c701be1805e3734c7", + "phase/calls/confirm-fresh-33-te-m1-source-grouped_actor_prose_withheld.receipt.json": "1b9bb9dffac9e413ec4f5c394ad460cb662cce5cb2cb5d89288419f70758cec6", + "phase/calls/confirm-fresh-33-te-m1-source-grouped_actor_prose_withheld.request.json": "2ed4d565d7c62f9c4ae92dfe4a19dfcd51887d1ed0af7ad69a7c98deee67aff8", + "phase/calls/confirm-fresh-33-te-m1-source-grouped_lossless.dispatch.json": "58a80995b1b286535107b6cacd808c7d811138c2eb82416f2c528190d0c2ca73", + "phase/calls/confirm-fresh-33-te-m1-source-grouped_lossless.receipt.json": "fad76efe5e0e4df2cda611e2d6eb4cc0b69e0084bfb88a05d67890398d7eef08", + "phase/calls/confirm-fresh-33-te-m1-source-grouped_lossless.request.json": "14cb39eaa4338fd9db01b32618591e12591a22d364d67e95f28aed3bb2aed75d", + "phase/calls/confirm-fresh-34-en-m0-source-chronological.dispatch.json": "94c87afb6c20bc8d0dd7103697677b5504ae53a39af2b75bee9e0656c89bcaf2", + "phase/calls/confirm-fresh-34-en-m0-source-chronological.receipt.json": "6ecd59b354dcdc5ef7421f079c7f3f7d156b277cefcc3de434c1414a85fb0c5d", + "phase/calls/confirm-fresh-34-en-m0-source-chronological.request.json": "cd473313d492da83b5a47de7b3eb379069a2f2505f76045f73de93ba3e762cf4", + "phase/calls/confirm-fresh-34-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "d9df3609333c404af9581e54abf79567673756e6430fc5494c6782ca505abc4a", + "phase/calls/confirm-fresh-34-en-m0-source-grouped_actor_prose_withheld.receipt.json": "2fdc84d8aa316b7ed2401e77c86398cc62c536636a2622497d5a4f811ac7b468", + "phase/calls/confirm-fresh-34-en-m0-source-grouped_actor_prose_withheld.request.json": "2f3b7e2f3681db1911e4c50340e2fecc5a87a26db00dd48a6df68663243a8e99", + "phase/calls/confirm-fresh-34-en-m0-source-grouped_lossless.dispatch.json": "d3225792507587a3c3fbb32478a273c65c3dcf1a68ffdfc0e4a42533347e9fd3", + "phase/calls/confirm-fresh-34-en-m0-source-grouped_lossless.receipt.json": "43d41356769b3466641597b82caa43ac4c6ebed3cae4b174d2ecec29aed505db", + "phase/calls/confirm-fresh-34-en-m0-source-grouped_lossless.request.json": "41a2fdf974b8769db64fc4efc17d45cd8d3a72780e3f15b8b0d73dd9114a9790", + "phase/calls/confirm-fresh-34-en-m1-source-chronological.dispatch.json": "056f4716c7e63cb8d27f295e33c88be3b1d65679741b81db5a9bf0af525957cf", + "phase/calls/confirm-fresh-34-en-m1-source-chronological.receipt.json": "5e67cd5c3e4dd73457214af2d13c86b7a471103cf52d1279fe31abd8300cca5b", + "phase/calls/confirm-fresh-34-en-m1-source-chronological.request.json": "3bf80f107ae07c961fd29167af72ba222b431a2ea7bc491be7bb92c8eb40ccef", + "phase/calls/confirm-fresh-34-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "f0b54cf9fef1311544fe38223b0f37e0f4030122bce9185e4acd2f38dfd07005", + "phase/calls/confirm-fresh-34-en-m1-source-grouped_actor_prose_withheld.receipt.json": "84be8c9640db8a0b976a812a9f1984a6c5222256d84aea46db2bf6c683c9bd83", + "phase/calls/confirm-fresh-34-en-m1-source-grouped_actor_prose_withheld.request.json": "fca0bf0c8fdde1f0eb14df8b7614f9aca82abff41a47d78488fba0d73c503d42", + "phase/calls/confirm-fresh-34-en-m1-source-grouped_lossless.dispatch.json": "5d4974225901875efbd3b8d0502e9aa49bcf97a10c08c7452b99daab8d025522", + "phase/calls/confirm-fresh-34-en-m1-source-grouped_lossless.receipt.json": "62f7095b4a5b746a59a7008e4e6e87a2608a65aa51cf3e973a3a428d8930ee72", + "phase/calls/confirm-fresh-34-en-m1-source-grouped_lossless.request.json": "3ebcf2bb3ab24d387bcfb2d28b0342fab2e3692c8e06b0160d4bd380359f6b84", + "phase/calls/confirm-fresh-34-hi-m0-source-chronological.dispatch.json": "960ab15463c72e0dac95c65de2110752014ccb8fee26e3c07ebb8fe7bcd30ce5", + "phase/calls/confirm-fresh-34-hi-m0-source-chronological.receipt.json": "677b9be2b799977ab6f9f3f34eb3759544b8abde56f7394501c811ef3a224a5c", + "phase/calls/confirm-fresh-34-hi-m0-source-chronological.request.json": "167b047d5c20d84453d2292ab8e8d14a7fca452c9acabaef5e14832f67fdf2d7", + "phase/calls/confirm-fresh-34-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "a3bb9984c89aecaf75c37dd45ba2a812589f496c3fa15dc12cfd9fd4a5df9198", + "phase/calls/confirm-fresh-34-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "7c5356bfc799ab014703e6d2581ccca47195b73a70fa5e690a897f36766a735d", + "phase/calls/confirm-fresh-34-hi-m0-source-grouped_actor_prose_withheld.request.json": "c7fcb8e68b352579bf5003e9473bf4904a5ca644698d8a17746e0b7ad4fc6188", + "phase/calls/confirm-fresh-34-hi-m0-source-grouped_lossless.dispatch.json": "75847e55e0e29933d82d9a27df42ea223e5100528956f8cdd2718d9e7d7f821d", + "phase/calls/confirm-fresh-34-hi-m0-source-grouped_lossless.receipt.json": "cce7ff5e86f7c7fa7e0dcd030bcf7d925ec20274438eec4841d48c0cdd54f6fe", + "phase/calls/confirm-fresh-34-hi-m0-source-grouped_lossless.request.json": "023375ba9a196d6a8ca485555dee3258ab11f704be00760c8d4dce4e61b98d33", + "phase/calls/confirm-fresh-34-hi-m1-source-chronological.dispatch.json": "943df2dae31a6c0de7627ac55605b518c0d2f8b99c49d6b4ff4d6af4b6e57916", + "phase/calls/confirm-fresh-34-hi-m1-source-chronological.receipt.json": "7491fbeeb967af8391b357afa10ae028b866003f1fbd7efd0d6d21ef2bd7e8ec", + "phase/calls/confirm-fresh-34-hi-m1-source-chronological.request.json": "729f1a95cbac0e21cd8dd46ca1864354b804c18cf7460847cc12fbc9ea814881", + "phase/calls/confirm-fresh-34-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "7692096e88d20b649a8b9ffacf55f52a82d0650ca975756d52dda33dd5cd523f", + "phase/calls/confirm-fresh-34-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "fc26fbc55657fb44bb17af42f1a36dd62cb4ffe6e6af41eb9ce023c6970af3a7", + "phase/calls/confirm-fresh-34-hi-m1-source-grouped_actor_prose_withheld.request.json": "43a7324c5be187cf79c7267cd065b247190e38c8e416851c1f1a5b98ab81cfdc", + "phase/calls/confirm-fresh-34-hi-m1-source-grouped_lossless.dispatch.json": "8bf2a2c8fd443c45496e608b41f41832afda733d406a39458fabf8d7501ae86e", + "phase/calls/confirm-fresh-34-hi-m1-source-grouped_lossless.receipt.json": "bb15e196be6f052675b33e6f9088227b5cf63327bb6a5d0e57b049d5377a8244", + "phase/calls/confirm-fresh-34-hi-m1-source-grouped_lossless.request.json": "6f677c5fe796ab88301ebc00a1541322cb832ff51abcd0b9a146633daa0711c9", + "phase/calls/confirm-fresh-34-te-m0-source-chronological.dispatch.json": "3e86d4b883bf993a58bbc7f838e274eb0fbc5b98be69425511e8ab9cfb5629b7", + "phase/calls/confirm-fresh-34-te-m0-source-chronological.receipt.json": "fea7a582ae1f931d88cacfa74a7d5b40cacd75853e0e09b4d7b9794ae7340de9", + "phase/calls/confirm-fresh-34-te-m0-source-chronological.request.json": "98580c8134d688f3a08d855fa2bbc9ca630fb8aa1122e70003935bb3f9e83910", + "phase/calls/confirm-fresh-34-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "39e4a20cd80427b00ec1bc3f72548952fdab2ea12b887f7a6184e1acd90aaaf8", + "phase/calls/confirm-fresh-34-te-m0-source-grouped_actor_prose_withheld.receipt.json": "16cfe45742468d1846f7d97bd10ca4986f5db87b6e327504151330f85162e410", + "phase/calls/confirm-fresh-34-te-m0-source-grouped_actor_prose_withheld.request.json": "dd75db89662f92df9a00a557df636922d5f22affd97b5f67bc901f283f1f2a26", + "phase/calls/confirm-fresh-34-te-m0-source-grouped_lossless.dispatch.json": "0689083dd336fe2e3045244f2e6ae0968dd29faabcca95dbcbc896966cd42e6e", + "phase/calls/confirm-fresh-34-te-m0-source-grouped_lossless.receipt.json": "d74a26b0a30b26c79ddb0838a96ac794b7bc91f3201a792a6722d7a1ed2232de", + "phase/calls/confirm-fresh-34-te-m0-source-grouped_lossless.request.json": "ef392f8820c1191236d4f7ccb30ce469f881547420981ac0782af9f4468c13cc", + "phase/calls/confirm-fresh-34-te-m1-source-chronological.dispatch.json": "c2b287e98e45bdbd3830e76a0e0e376a61cd38a4fe0068f6c11720dbf0ac41cb", + "phase/calls/confirm-fresh-34-te-m1-source-chronological.receipt.json": "e5d9eb5dd4b43b30459dbda21628398eff36d686500be8f885e6e29a5d51e603", + "phase/calls/confirm-fresh-34-te-m1-source-chronological.request.json": "a98b59849652e5ff1d63811064afdd12f667768fe849a258a795913c43b45dc6", + "phase/calls/confirm-fresh-34-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "8ceb3deb49fa5a09a5ee959d6278411b3637b4ad6385dccb1ca9012c9381b27c", + "phase/calls/confirm-fresh-34-te-m1-source-grouped_actor_prose_withheld.receipt.json": "4e2a457d5951675cdebf8d73d76a8c1f62c296e0ec28fc6c53603215b02355ce", + "phase/calls/confirm-fresh-34-te-m1-source-grouped_actor_prose_withheld.request.json": "7910c11f2c726bddc789f993f6f04e39cc451f9a7149409ae877cd135f0f2491", + "phase/calls/confirm-fresh-34-te-m1-source-grouped_lossless.dispatch.json": "5274d2323f6f0e8d30cdb8f9758dc898a84dc711e4e4da34e2607a6825eb9989", + "phase/calls/confirm-fresh-34-te-m1-source-grouped_lossless.receipt.json": "3012e75d31ee63f7417032fa207685ed008f7c79ecabba1bb3dd7313b418fb54", + "phase/calls/confirm-fresh-34-te-m1-source-grouped_lossless.request.json": "7f17c27b99a573b3cefc20275a3c8a848d0eabd73c22d5cf4ef854f5a6120c52", + "phase/calls/confirm-fresh-35-en-m0-source-chronological.dispatch.json": "d868a6be309bb092cc8d02a411d7d15f0c45897db56af90d20a95bd59d4f5c1b", + "phase/calls/confirm-fresh-35-en-m0-source-chronological.receipt.json": "1ff693b65299542e549367de991dd8f7eb74ef0805bd5322f0be416401292d2b", + "phase/calls/confirm-fresh-35-en-m0-source-chronological.request.json": "b3ff0cacaccf05ab7a1c8004567b05001778f26e7dc3d783f0fa6152a42f3473", + "phase/calls/confirm-fresh-35-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "a84260134831e7016b534742aa4e5ec16f98387cf7f77653c9ba7e1c6a48b752", + "phase/calls/confirm-fresh-35-en-m0-source-grouped_actor_prose_withheld.receipt.json": "f5a83c3f52084326689dbf0df6dd6ab73abb6668a3078adf62407823fef6d28a", + "phase/calls/confirm-fresh-35-en-m0-source-grouped_actor_prose_withheld.request.json": "aa6eba428341b35b02053d2db511bad8f11a39b0c2423f82c10ce946ceb37eef", + "phase/calls/confirm-fresh-35-en-m0-source-grouped_lossless.dispatch.json": "bf7676a3092e74935350f9f10495cbb71c846157e72145d157cb5ba9baa19986", + "phase/calls/confirm-fresh-35-en-m0-source-grouped_lossless.receipt.json": "fa3c3742094150c263ee5dc8f4025afecf555f8ecf7d3139807cbfb53c5d6f63", + "phase/calls/confirm-fresh-35-en-m0-source-grouped_lossless.request.json": "312cb6d0662e523f1afd60d693798b0a3118e9ddaf12e6ca6c10467592223dac", + "phase/calls/confirm-fresh-35-en-m1-source-chronological.dispatch.json": "f102a0f554944ff9bdc1e0f8a0ad2d518489d6bcc4803abbfd26ba4079fe0f67", + "phase/calls/confirm-fresh-35-en-m1-source-chronological.receipt.json": "7b026a350a5dfa135a86989033e7f649fb292dfc566387ebcc3f92924bf51726", + "phase/calls/confirm-fresh-35-en-m1-source-chronological.request.json": "6a4ac60cfe130bd47b576fa292a8ccadd1c9a89f5ae67ae309137d5f956d3b67", + "phase/calls/confirm-fresh-35-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "f693fa9de2230a7e657e97f5c34a85cdcd395d8070d6c321eb4325a35493c357", + "phase/calls/confirm-fresh-35-en-m1-source-grouped_actor_prose_withheld.receipt.json": "2ea68c5c44650962429de8643a46141612748001fcc02a04c8a596b6c3d58c90", + "phase/calls/confirm-fresh-35-en-m1-source-grouped_actor_prose_withheld.request.json": "1118e44fc5b40d17815ea6771846b1255a6d0d212e2ed0a30183153b8f5b4ce9", + "phase/calls/confirm-fresh-35-en-m1-source-grouped_lossless.dispatch.json": "5ba6bc77e75d63dd8294e3cbe72d07e0e42a36dfaee5046fe7772deacc6bd5e6", + "phase/calls/confirm-fresh-35-en-m1-source-grouped_lossless.receipt.json": "dbc71e7c4b6b325e549b618e36c79b59d7b753db320a3133292ef17d66c39001", + "phase/calls/confirm-fresh-35-en-m1-source-grouped_lossless.request.json": "8ab95da936dfe1e72cca058fbcd775382eaeee566f80b4c1418be0d13ca06563", + "phase/calls/confirm-fresh-35-hi-m0-source-chronological.dispatch.json": "83f86bc80a75c004be2cde52350701dce15e5978721b8f63108e81f43edca63a", + "phase/calls/confirm-fresh-35-hi-m0-source-chronological.receipt.json": "180cf12b2668ff547411720802e2046f9627da91e50e07de61922d03d2be2c01", + "phase/calls/confirm-fresh-35-hi-m0-source-chronological.request.json": "ff4b0b1ea413a5af5b2f89ae7d353d7980bd5b2a44ec0254793481623eb95a78", + "phase/calls/confirm-fresh-35-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "dec7dce2c89f4468834f372c827330f8a785477ef510945c64a54435bb9cce06", + "phase/calls/confirm-fresh-35-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "39c0f95064beb93c0dd780242f61d397b237459bf92e912bd33624c2bf9d0a9f", + "phase/calls/confirm-fresh-35-hi-m0-source-grouped_actor_prose_withheld.request.json": "82233b1ca504d683128440aaee183c877d0f4738ca5c3198a4b419d6105c30f3", + "phase/calls/confirm-fresh-35-hi-m0-source-grouped_lossless.dispatch.json": "080e88ae152447b4ad5bbb6b6601f1600cff42ae72d3fb55a1153f06fd90b8e0", + "phase/calls/confirm-fresh-35-hi-m0-source-grouped_lossless.receipt.json": "63a78da579b4938d12e877773b7e896d0d773e4095c2492f07e58cfa1b2d385b", + "phase/calls/confirm-fresh-35-hi-m0-source-grouped_lossless.request.json": "11c011b983fbebc764f2cc7d81e0870c9adf91169aec4f51bc56a7b85230b5e8", + "phase/calls/confirm-fresh-35-hi-m1-source-chronological.dispatch.json": "24837d5058dce488cb8fe37a86eeead20e8e59a147c748ca21e2e90cb6c13209", + "phase/calls/confirm-fresh-35-hi-m1-source-chronological.receipt.json": "c189fc10570e266811d0b78a3ab27619cf67e65a4a5d998210f5f1be4b4afc7a", + "phase/calls/confirm-fresh-35-hi-m1-source-chronological.request.json": "596e159550fe7c18cdc51815ebe75c398484b99496491e60b2ed160769aafcf8", + "phase/calls/confirm-fresh-35-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "7a43a7de0e305b69e3a2da0abf329a3b01583dbb9c85cd10544234bed4f71f0e", + "phase/calls/confirm-fresh-35-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "0fa1e81efd348000056f92482b489f5f50887dbff9c8352771ed8dbb37213fb8", + "phase/calls/confirm-fresh-35-hi-m1-source-grouped_actor_prose_withheld.request.json": "3290d6fad435424994931a0802960949a9b72b92cbf95920242a1439fcd8c871", + "phase/calls/confirm-fresh-35-hi-m1-source-grouped_lossless.dispatch.json": "f965339cdcaab991ff05d823fec89995efa3843f5ea19fc6df2dc8c178a3908e", + "phase/calls/confirm-fresh-35-hi-m1-source-grouped_lossless.receipt.json": "bcef32f002a1672ff8d4d1b5c03b2bcbb9a55011346a14bdc71bdf4e588cb825", + "phase/calls/confirm-fresh-35-hi-m1-source-grouped_lossless.request.json": "74c47d7de561a66b383cc815e3182f9fdd48d345f46f23ffdd33832abfb92eac", + "phase/calls/confirm-fresh-35-te-m0-source-chronological.dispatch.json": "94b964f7d04cfae34829309e20772e5f68faf43634afadec81cb9da8df6168c0", + "phase/calls/confirm-fresh-35-te-m0-source-chronological.receipt.json": "b6b4c46e23e90cfb76801ded4e40c5f19889f5ea7ca8ee8eda872445a816e625", + "phase/calls/confirm-fresh-35-te-m0-source-chronological.request.json": "c217ed8f84afb5fbf8fe26e770c66a81f52fdfa3b27f43172ba0cd0452b6ca66", + "phase/calls/confirm-fresh-35-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "3e67367eb6d5fcce24e46c0affdea2939b381381551fb4a12c9874be85f01158", + "phase/calls/confirm-fresh-35-te-m0-source-grouped_actor_prose_withheld.receipt.json": "a260c4bbf8988a3e8852f00125fc1a163565c73fed4b5eac490afbed1d2edf9c", + "phase/calls/confirm-fresh-35-te-m0-source-grouped_actor_prose_withheld.request.json": "7b8ad2736375ba13ab1f83c5ce2029c52e5b6dcb4dad618dc659f8713330cd83", + "phase/calls/confirm-fresh-35-te-m0-source-grouped_lossless.dispatch.json": "df1142a7c5d30bb93d0a00ef0dea64910cb6e1edfd44d6316efbfbbfbe5e40e6", + "phase/calls/confirm-fresh-35-te-m0-source-grouped_lossless.receipt.json": "f3630562f524845d21b3a8c4c6bc202a498da5c5c9905e4fbe3b904c21d59741", + "phase/calls/confirm-fresh-35-te-m0-source-grouped_lossless.request.json": "1738ee538dfd63813e75e57f9ba8a89624d6acbe365352fc2458c4b6ed569fc1", + "phase/calls/confirm-fresh-35-te-m1-source-chronological.dispatch.json": "2b1f3ed6768af525f12852b5e53289acc9dbf69d420c15f90f98ffe9996340e1", + "phase/calls/confirm-fresh-35-te-m1-source-chronological.receipt.json": "23e87f8614af40713e3e51bddf0ad451cff813739ce65ddd1477fb9f75c37bb5", + "phase/calls/confirm-fresh-35-te-m1-source-chronological.request.json": "2ff169ff8fd5756cb4c688dc7bfed4c8f9e94d6d95a91b61c2609ebe9f986b21", + "phase/calls/confirm-fresh-35-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "a38c6e5c574ecd7f4b3bd451fc634abf2e2aef4eb8cd964bca4befe855ba9554", + "phase/calls/confirm-fresh-35-te-m1-source-grouped_actor_prose_withheld.receipt.json": "f13dd88425ee7f54cbb4c4d052163a0713b8134f0eb197b732924004780a476d", + "phase/calls/confirm-fresh-35-te-m1-source-grouped_actor_prose_withheld.request.json": "459d7433cb1cdae2ab8929c4c973d9bfdebd74196c30bf6fca1c9a201b9d2982", + "phase/calls/confirm-fresh-35-te-m1-source-grouped_lossless.dispatch.json": "0a915faf23704331f5bea7b15d3a336d59eed8406f0a6ce864150e51038e2a59", + "phase/calls/confirm-fresh-35-te-m1-source-grouped_lossless.receipt.json": "aa70998152d2310679541173469dc2f2bd02affa726e6c244343987ad7f11e10", + "phase/calls/confirm-fresh-35-te-m1-source-grouped_lossless.request.json": "3f7fa9e86a11e72023da7ed0b9f1e9baa2b6c5bbc74f51eaf8c583560d1500e5", + "phase/calls/confirm-fresh-36-en-m0-source-chronological.dispatch.json": "c29b070f9ddc54e21a5d47c33413440e8adf8e24af5c171aafdb3f5de47111ca", + "phase/calls/confirm-fresh-36-en-m0-source-chronological.receipt.json": "de200af3c1c79585434d4a394cdadc1e0c40b7406a2e0874981ea0501f755e6b", + "phase/calls/confirm-fresh-36-en-m0-source-chronological.request.json": "7aa46c2e025b6079b9c942e00dbc53fda0c7d0429fe35dbf1db25056044e5b96", + "phase/calls/confirm-fresh-36-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "69eb878635a2fd185a7ce9970377c944e5924a66d8504a2b772664ad1c128810", + "phase/calls/confirm-fresh-36-en-m0-source-grouped_actor_prose_withheld.receipt.json": "27a0db55f2089b7866a8424844984bbfedc46946130563eef03e666f3c5c5bd3", + "phase/calls/confirm-fresh-36-en-m0-source-grouped_actor_prose_withheld.request.json": "0070ebbcc9d77a597b887d43f11abcbaf28a578bbab06e59a9c217ab044bbef6", + "phase/calls/confirm-fresh-36-en-m0-source-grouped_lossless.dispatch.json": "53f7ac3527d37b0fe2373520ad7df5617cab27adaf0eda0b2546f78ea2f08b1c", + "phase/calls/confirm-fresh-36-en-m0-source-grouped_lossless.receipt.json": "6baa25451711fb3e3bdc80c82a836797dd1cd7533781bc82de41e3de4c805fad", + "phase/calls/confirm-fresh-36-en-m0-source-grouped_lossless.request.json": "4539943d96c491e46da57ab63717a52046065f3453268fe7ae023d30df7c87df", + "phase/calls/confirm-fresh-36-en-m1-source-chronological.dispatch.json": "b656e1851280b633861f0250edafaac5c99000139d315c8cb48b3373880bb1d6", + "phase/calls/confirm-fresh-36-en-m1-source-chronological.receipt.json": "4d352af437a7d57e6dfc946b5c504dcfe593ec99d09ec7fc5d7b1592cdb9e7c6", + "phase/calls/confirm-fresh-36-en-m1-source-chronological.request.json": "ac27c5a97d2465fec1ade3b381cd3d1a25f80435f91e3fa6de9a85c97657b879", + "phase/calls/confirm-fresh-36-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "8a6dccb0a1aa3bcdd5190417179c68314abe5e970c5b5a311c368e56d7b7d45e", + "phase/calls/confirm-fresh-36-en-m1-source-grouped_actor_prose_withheld.receipt.json": "f5121346f88564caa01a01873127a6b6d5afaabb1b910dff78895e493339e5b0", + "phase/calls/confirm-fresh-36-en-m1-source-grouped_actor_prose_withheld.request.json": "870b06111e40ded8ee5c30275a8882f7f08239553444d82d06744408d00be65d", + "phase/calls/confirm-fresh-36-en-m1-source-grouped_lossless.dispatch.json": "7fd98eda6e069fca1377cc13e2f244a447151f4a98d58f1d2d9dd5eeec415c64", + "phase/calls/confirm-fresh-36-en-m1-source-grouped_lossless.receipt.json": "a448aaf931a74f19c308745e29385ad66c01a6fe72b64f0196f743438af8ca32", + "phase/calls/confirm-fresh-36-en-m1-source-grouped_lossless.request.json": "22ada01cba0deb26d1976856b23aabb20635fb9a71e80dbaa9b145f2f4b508b2", + "phase/calls/confirm-fresh-36-hi-m0-source-chronological.dispatch.json": "3b8c28bc2064120fc51daba2833de1e969ed79e7ea67b38b32df0e199b0fe559", + "phase/calls/confirm-fresh-36-hi-m0-source-chronological.receipt.json": "39e4de0b9ea0c84f98a79dc4593a75ca278dcaf54326877566f50ce991161fe6", + "phase/calls/confirm-fresh-36-hi-m0-source-chronological.request.json": "37ca6a0706bd76a399ea07b48bea52186aa5aa7f789fa138c7b450fafb084064", + "phase/calls/confirm-fresh-36-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "380bfa2e12a7e60dfb510455cb0007752a3854a1179ffcff5329d99c4d7ec059", + "phase/calls/confirm-fresh-36-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "bb496b9b5f66a2824eec1ad0ca19e2140619e7f9d59df5a5c85351681b831337", + "phase/calls/confirm-fresh-36-hi-m0-source-grouped_actor_prose_withheld.request.json": "6275fd6df30c3c1abba6cd97944cec85918e6ffd0bb5c548b1c888308a92c57c", + "phase/calls/confirm-fresh-36-hi-m0-source-grouped_lossless.dispatch.json": "5d16402fda3028c38c7f9e0866a7b6f48ff611573854a10222f59fd8f7b0955d", + "phase/calls/confirm-fresh-36-hi-m0-source-grouped_lossless.receipt.json": "4e94e7a58e8bf6fa8133c862b0a60d3df40fa70707c1809560cdf439f802e227", + "phase/calls/confirm-fresh-36-hi-m0-source-grouped_lossless.request.json": "93c0e2588ef5f57b254457019332f32194b41d931b4947f26e54fc63b490b06e", + "phase/calls/confirm-fresh-36-hi-m1-source-chronological.dispatch.json": "788a20441a58077754b0335957b086e7c0befc9f7895a632d651f7e32b0dbdbb", + "phase/calls/confirm-fresh-36-hi-m1-source-chronological.receipt.json": "97c054cd120f7689544555394624b5178380254d5aaff1e7473a9358dfe8461b", + "phase/calls/confirm-fresh-36-hi-m1-source-chronological.request.json": "30b4ba944a385c26a2ab758c3baf69420eb4a9b6b6561e8294e3fcdb7e8eb7c9", + "phase/calls/confirm-fresh-36-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "fe094f9efc46fbccb559c6e546a933d7aabef867b999e803439c4df08248a5fe", + "phase/calls/confirm-fresh-36-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "f75d49337df2a0ebd522dbef7cd8c2131cd4cbc9423ac936bc9a9f13d2f695a4", + "phase/calls/confirm-fresh-36-hi-m1-source-grouped_actor_prose_withheld.request.json": "f13629ea51e277b03333689f4d218edc5f5dd6d8806f288279777c9abc00aad4", + "phase/calls/confirm-fresh-36-hi-m1-source-grouped_lossless.dispatch.json": "8578ffa107bfe4e2ebb2b0df00ed89bb4a37af4e1c366ddd2701b93664389d33", + "phase/calls/confirm-fresh-36-hi-m1-source-grouped_lossless.receipt.json": "c4876225814615d29ae4fb137ec4b1d182965a8114c7784bba5a896c36679bb7", + "phase/calls/confirm-fresh-36-hi-m1-source-grouped_lossless.request.json": "6b3094b74a251e38cced84caaadc5820ccf1b6cf83ab8e32e5c37160ff133fad", + "phase/calls/confirm-fresh-36-te-m0-source-chronological.dispatch.json": "750823c69ed498a7cc3b2d9c4bfde86ccc03903dd03a3d6620ac535fb2c6c323", + "phase/calls/confirm-fresh-36-te-m0-source-chronological.receipt.json": "f33ebbb90d0431f8d41f8d9d1b450fb96522b837793a11f7709f786fb54551cf", + "phase/calls/confirm-fresh-36-te-m0-source-chronological.request.json": "2b6c19ac2d3aeea86ead17dbffdb02f6080e12fb568edf5670b6d9d95158260b", + "phase/calls/confirm-fresh-36-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "cfb6890a3ba8b05cd15c4469febe2ceff1672f3c87aebdb508ea48803d98c0c0", + "phase/calls/confirm-fresh-36-te-m0-source-grouped_actor_prose_withheld.receipt.json": "f21f731f5ef2326733bdfef4fe840c4171623a40bf9014fcdbba306242295c8c", + "phase/calls/confirm-fresh-36-te-m0-source-grouped_actor_prose_withheld.request.json": "abea862c490898fd7b295da1436f133b3c70b889800421614b44e89adbe97fc9", + "phase/calls/confirm-fresh-36-te-m0-source-grouped_lossless.dispatch.json": "378ed33757f1f562ea9ec922bca0c7fdf830a7d343e76b58923ea37f16f6da35", + "phase/calls/confirm-fresh-36-te-m0-source-grouped_lossless.receipt.json": "360d069620f040ce15edd4f719c7d86f86ca8ae3dea7633feef6198fc33ab5b0", + "phase/calls/confirm-fresh-36-te-m0-source-grouped_lossless.request.json": "2dfed078248f641a214655e98a1f90b66677e490e090e505405c758182a6c139", + "phase/calls/confirm-fresh-36-te-m1-source-chronological.dispatch.json": "74b96118ff19382d77052eb828de5141d365e7d57476da8855cb6477eb058e48", + "phase/calls/confirm-fresh-36-te-m1-source-chronological.receipt.json": "844ffcb34cffa289068555fbb548e7f46b58242757db36d3792c58dedd30bc54", + "phase/calls/confirm-fresh-36-te-m1-source-chronological.request.json": "f2286af0f0525d6fd257e0caac09096e178a1fa8049220d87a83e19a5f8a4ff4", + "phase/calls/confirm-fresh-36-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "641f229102d785e9133f57dadda7a2f334a818c22fab2178aacd549184d02bd2", + "phase/calls/confirm-fresh-36-te-m1-source-grouped_actor_prose_withheld.receipt.json": "9f6d748f702a9aafafecc490c3d98d4f6893bc21b4726dd7918732136eb84c14", + "phase/calls/confirm-fresh-36-te-m1-source-grouped_actor_prose_withheld.request.json": "b38494afebdd642b05ddbd2802e2d8a50b788f840a8bbce0bf9b72759b327fc9", + "phase/calls/confirm-fresh-36-te-m1-source-grouped_lossless.dispatch.json": "770398489e0445519aa9965a471a683a77cf078bdb0e8547e2b0e59fe7e62475", + "phase/calls/confirm-fresh-36-te-m1-source-grouped_lossless.receipt.json": "705d0d7f7a35976aee6e7e9a48936685b024dca3e8e721725b1404a5f2f041eb", + "phase/calls/confirm-fresh-36-te-m1-source-grouped_lossless.request.json": "ad6412b8abf218fa3f9e294c245ab0bd9762eeb91f7658ac89a8cd1396e2d759", + "phase/calls/confirm-fresh-37-en-m0-source-chronological.dispatch.json": "71a260290e1abb147f6b777db96f8c860143e6d366b7525cfc67b1caadc7f8c4", + "phase/calls/confirm-fresh-37-en-m0-source-chronological.receipt.json": "8fc7fde8c96fcbfce915aa9f917924391ecabf1de8ecba413cf731fd846ce233", + "phase/calls/confirm-fresh-37-en-m0-source-chronological.request.json": "5a0d59652d0963d41359a6ee085ba81d0887632e68a05c8bd772bfd06e37d22f", + "phase/calls/confirm-fresh-37-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "aebfc40853226fe217fae23ec70b37040f5d948fc9314d378efe7fd33b90f6ff", + "phase/calls/confirm-fresh-37-en-m0-source-grouped_actor_prose_withheld.receipt.json": "f01ae39db06f018a960954d909a833a35a2b40b3e85b73060838ff57fb1f3b76", + "phase/calls/confirm-fresh-37-en-m0-source-grouped_actor_prose_withheld.request.json": "1ea0e4faca7c53494678f300f121ecd12a9f27e5917839e94fc0bb50336175c1", + "phase/calls/confirm-fresh-37-en-m0-source-grouped_lossless.dispatch.json": "f11c65782af7f609e05bed208b8be63c3799a6693d2bef4b8b4dc30276bffc51", + "phase/calls/confirm-fresh-37-en-m0-source-grouped_lossless.receipt.json": "cda5e52a59b8bc8b99aa6eded713330322010d854e8ca5e24bee81a310040c7c", + "phase/calls/confirm-fresh-37-en-m0-source-grouped_lossless.request.json": "d84bb6f2e524c554f2364ab0d35851b53f1c21ff390677fa06590f12275bda39", + "phase/calls/confirm-fresh-37-en-m1-source-chronological.dispatch.json": "5bdf9a640e68ee99495e620e58d61920ed84cbc83089c51c1349e5816bcf742b", + "phase/calls/confirm-fresh-37-en-m1-source-chronological.receipt.json": "a67ead4dcce123dec8ee539bf257e8df6fd9f4647804a259e9781d2baeba271f", + "phase/calls/confirm-fresh-37-en-m1-source-chronological.request.json": "573f1f6fc1bf60b3a346a7989627665539b33a5e800797730afdc688276db0f4", + "phase/calls/confirm-fresh-37-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "1c5fee40f4cdc4847db76a75fa1a6c0dc7243d8eb761c7fbf32b6520efa67f51", + "phase/calls/confirm-fresh-37-en-m1-source-grouped_actor_prose_withheld.receipt.json": "96137145690a3dd4ee6edf2a3e92482567259dd7cfe2babad60343f4fe2cf281", + "phase/calls/confirm-fresh-37-en-m1-source-grouped_actor_prose_withheld.request.json": "e551b11efaef90b7d3fae9a6a8aa545b9fdf2d07328a1d14a7d68937d45b8691", + "phase/calls/confirm-fresh-37-en-m1-source-grouped_lossless.dispatch.json": "14215284d5625783d9f177a07a9805618c1898951c1ca8cd5a1b8095c45c0a57", + "phase/calls/confirm-fresh-37-en-m1-source-grouped_lossless.receipt.json": "5e836726a353647fa4bed8f94129542514660922c2b52b4409152ae328794395", + "phase/calls/confirm-fresh-37-en-m1-source-grouped_lossless.request.json": "023c3610873e35baf156f3ed8e57905b9a20d914358badb6fb6f9fa8db2a46bc", + "phase/calls/confirm-fresh-37-hi-m0-source-chronological.dispatch.json": "cb335fb6130f1aa497414d3ebca08be8ad5397eeb7032f8811585aba11077caf", + "phase/calls/confirm-fresh-37-hi-m0-source-chronological.receipt.json": "919d2652b103827dde9bcf9d9f00f4260b2e95c027b7898a20c30e307f33e89b", + "phase/calls/confirm-fresh-37-hi-m0-source-chronological.request.json": "5820ff422be34d1cc9eadcb3c8f52e3a1f06bbdc3692853a73b6176bd6b2c7c8", + "phase/calls/confirm-fresh-37-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "4dce9b27fff0f929c69951eda77c3eead5f080780f5437b670d83976af4d2788", + "phase/calls/confirm-fresh-37-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "746aa2d6430e3e6d93e264cedc6a0db8b0b112ae5956d651303355d13fe033d8", + "phase/calls/confirm-fresh-37-hi-m0-source-grouped_actor_prose_withheld.request.json": "c713e5c45d326d6134a25a50f47cd4ae3e8df6885e785755b93527fc8fdc704f", + "phase/calls/confirm-fresh-37-hi-m0-source-grouped_lossless.dispatch.json": "146c3e0a80ca8c45daa3b2fe1164f19e0eeedcaecaa8b3a32426fbe0578a34bc", + "phase/calls/confirm-fresh-37-hi-m0-source-grouped_lossless.receipt.json": "588a18455f1bc3bd1948b8a3c928bd87a5763e7f06ad394134e54f23f62f6ba7", + "phase/calls/confirm-fresh-37-hi-m0-source-grouped_lossless.request.json": "2d299f163996ea27683f344c1df84dae5cecfc8394f2adc306968b8f48c40136", + "phase/calls/confirm-fresh-37-hi-m1-source-chronological.dispatch.json": "d1ee10c52453d6c404547b94896713572035233f466b36ae50a4c133364b9fdd", + "phase/calls/confirm-fresh-37-hi-m1-source-chronological.receipt.json": "2f54fa75e78f8c23ae6d5dca1395e66243e39d3a9cd5a36063265b9e944dd98b", + "phase/calls/confirm-fresh-37-hi-m1-source-chronological.request.json": "dbaad75a959a14a929e64b521dbfae2584802e2bdf72d906017914f21630560e", + "phase/calls/confirm-fresh-37-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "58baa281eb1d2163806606d552037f74c8b23190e8166b7421264f399470c63d", + "phase/calls/confirm-fresh-37-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "556421f09b26b263c54302cf6541f0d4ac8b19eaa0097dba53c91a71d96650a1", + "phase/calls/confirm-fresh-37-hi-m1-source-grouped_actor_prose_withheld.request.json": "4900306e0b48887c4610baa6e75fb83c944cf2b7ae225e6d45c2f14867f2c2d4", + "phase/calls/confirm-fresh-37-hi-m1-source-grouped_lossless.dispatch.json": "4484a9c79bcd58e22fd96bc7335f6c758d7eddaf1ae86c0bb939c23c294be7db", + "phase/calls/confirm-fresh-37-hi-m1-source-grouped_lossless.receipt.json": "f0808c3e0fd87caa5ebe1c6f9185c7214d181dccac5fe3a1cc14b1216ee0200b", + "phase/calls/confirm-fresh-37-hi-m1-source-grouped_lossless.request.json": "71421677a2daa59e320517bc97b5cfe133e26bc23eb660199ff3213a581d62f7", + "phase/calls/confirm-fresh-37-te-m0-source-chronological.dispatch.json": "e1fe08830649b1a1dfdde14f97507124a088c3a2442f483fdf264d6ede3bfbf0", + "phase/calls/confirm-fresh-37-te-m0-source-chronological.receipt.json": "900ab2f1b4f8a073286b18f91e9a6b56274608ce3e5f67289f2d68ee18df8ae5", + "phase/calls/confirm-fresh-37-te-m0-source-chronological.request.json": "e6af78e8ea868d92b1e0114459ed6936e43468198f47e79fc82a1397ac3b83fb", + "phase/calls/confirm-fresh-37-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "f5860cb3a105f3a7c913397b3c2aea4193985891be7ec448dea926760856163d", + "phase/calls/confirm-fresh-37-te-m0-source-grouped_actor_prose_withheld.receipt.json": "d118b489f86bd4de14c5a0301ccd1d4de826fff85673ec2471aab25c5a6f401c", + "phase/calls/confirm-fresh-37-te-m0-source-grouped_actor_prose_withheld.request.json": "a51a781cb506d7b0ee941532a2d82917e33b8dc753d7a2d39ed732f0abbb4ecb", + "phase/calls/confirm-fresh-37-te-m0-source-grouped_lossless.dispatch.json": "60ea1369c2560dd7b9198e5cb0bcea8e2e4a4a1c2789be9bee96078a1f74e7c4", + "phase/calls/confirm-fresh-37-te-m0-source-grouped_lossless.receipt.json": "cf4a78c8aa56781d4e407ada2f33620a3b9bbb4042c92ae0f60de194ed15bfd5", + "phase/calls/confirm-fresh-37-te-m0-source-grouped_lossless.request.json": "6305ac338023e95fd4c45a23835d9ec61c8ac8fdd1c635107856b0c8827a0b37", + "phase/calls/confirm-fresh-37-te-m1-source-chronological.dispatch.json": "ea85e2a775c6366744995cf67e27d2245674648dc37c3fcac94098b8404d4524", + "phase/calls/confirm-fresh-37-te-m1-source-chronological.receipt.json": "47e33e7e75e21d4fbd16636d41fbf5bb68437d4b15bb54c0dfbe7849d94ee664", + "phase/calls/confirm-fresh-37-te-m1-source-chronological.request.json": "32d5095c51e212d089678902e5da2f4c60859489581816e97baff7c6e418ba55", + "phase/calls/confirm-fresh-37-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "a2a46d11fe85d82d78a60983ffa654c17881582bbb2dbbdf0b15bc79585dde08", + "phase/calls/confirm-fresh-37-te-m1-source-grouped_actor_prose_withheld.receipt.json": "d43dbc4139b9c07d52a4e85d60a148eb07c14337e1e9baf7ad985fb3b0096ebc", + "phase/calls/confirm-fresh-37-te-m1-source-grouped_actor_prose_withheld.request.json": "474e2aebac18ed24496b269378b28f96021efb7a3d0152fcd73be19212270d40", + "phase/calls/confirm-fresh-37-te-m1-source-grouped_lossless.dispatch.json": "243df39b09a7aa0c6d4dc19924686ac4937cd6b46be48633e235a84e7cbda796", + "phase/calls/confirm-fresh-37-te-m1-source-grouped_lossless.receipt.json": "3e5366d8c341d273faaaef6a4a1f85cc3186191255fedde7c5f157d62736d393", + "phase/calls/confirm-fresh-37-te-m1-source-grouped_lossless.request.json": "521e63b0227bf05f39a94a16839cb7c16b19628b5fe929110474fbb71c37ca64", + "phase/calls/confirm-fresh-38-en-m0-source-chronological.dispatch.json": "fffd5a90ffcdb7b2308dad840993b7b6c12399225038b91d58c2003e1447b276", + "phase/calls/confirm-fresh-38-en-m0-source-chronological.receipt.json": "28b947a2de055ab96d80df20653539111a1ed57036f680d4711cb000535f3069", + "phase/calls/confirm-fresh-38-en-m0-source-chronological.request.json": "b1aa1ddf2b014a4d3faeb02ebe74fcf06b9fe6fd24b90fe591ff4c03b911e923", + "phase/calls/confirm-fresh-38-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "1a1f2e39c0c8a2a4c872602bacd886a5dd8f0e477452b8dc7298b4cb9066ae88", + "phase/calls/confirm-fresh-38-en-m0-source-grouped_actor_prose_withheld.receipt.json": "e1c131b71e091558ecd4c7b57f737dfe62565c57c76e1461f50b4470c1f745e7", + "phase/calls/confirm-fresh-38-en-m0-source-grouped_actor_prose_withheld.request.json": "c7c5fbd365d0825271746127542dacf06169a63c56815f7a460fcab8a21b40f8", + "phase/calls/confirm-fresh-38-en-m0-source-grouped_lossless.dispatch.json": "9d9a9cc2148a4c69319521ea5ba027ab0d9301a0f74957b581c67f1475860217", + "phase/calls/confirm-fresh-38-en-m0-source-grouped_lossless.receipt.json": "88ba55e6b2fce0785b4f2c2d4424f05d1d915a3182c8833f15ffbcd254e3c43b", + "phase/calls/confirm-fresh-38-en-m0-source-grouped_lossless.request.json": "583f55c84772fc687ca38a4978338af9fd00c8ab06552caecd276022bebe5e54", + "phase/calls/confirm-fresh-38-en-m1-source-chronological.dispatch.json": "5710c5a955df075efc62b61df151d93b0e66c95fd962439276721ddc52e864d8", + "phase/calls/confirm-fresh-38-en-m1-source-chronological.receipt.json": "e24a361e7f9b5374aa0f0374edf8abed38ce2dd3fabc0ba7c9059c4c495a90d8", + "phase/calls/confirm-fresh-38-en-m1-source-chronological.request.json": "397e11bfecb3b0faefc02219a59eaa84d8ac33fdb3f87a2f8c41877841232d51", + "phase/calls/confirm-fresh-38-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "f5b4e45583a88396993302a835eb2e704df6a1e73f19c65fb6acac6950659728", + "phase/calls/confirm-fresh-38-en-m1-source-grouped_actor_prose_withheld.receipt.json": "54d5ab003ad3c065168abd7e32121877ed741cb797e55d80ff983360b1f71bb2", + "phase/calls/confirm-fresh-38-en-m1-source-grouped_actor_prose_withheld.request.json": "c4caeba56cb89ee2511376450db87bde3d1e8464ff68b83409e69da9452e98cd", + "phase/calls/confirm-fresh-38-en-m1-source-grouped_lossless.dispatch.json": "e6d32d3f8903c7cdcddd65a130a1fb5bb6df40eb2c0eb636e9446c1b0fcb7b49", + "phase/calls/confirm-fresh-38-en-m1-source-grouped_lossless.receipt.json": "2729e86eff3d177d9adc0251b6bf69226deb8680d793db26dff9bdaf61e4ae4e", + "phase/calls/confirm-fresh-38-en-m1-source-grouped_lossless.request.json": "0e3541d137da5d5bb8461df015832a8bf8796b99a4f80c283ff396784c450c33", + "phase/calls/confirm-fresh-38-hi-m0-source-chronological.dispatch.json": "53c742c3bcf161fbe1a8f3a6c9f302f9eb8509141005f7255cda0e9eafb661ab", + "phase/calls/confirm-fresh-38-hi-m0-source-chronological.receipt.json": "7e207881291eca18fe88fae4b3af14207be8f996f6f47898262483bd50f1ff96", + "phase/calls/confirm-fresh-38-hi-m0-source-chronological.request.json": "a486fb4c9f48ae722ec7e4ac9e4e1eef7781e424cf70edbeaec03bd613c17090", + "phase/calls/confirm-fresh-38-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "14cd2c48b0e826040759927372ccf4523404a5afbc2ee56f898ddac5b77ba94b", + "phase/calls/confirm-fresh-38-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "6463dfe47d2ab829a4c88839239c554b439991af1da862b847508f21cac6551f", + "phase/calls/confirm-fresh-38-hi-m0-source-grouped_actor_prose_withheld.request.json": "70a451c4868a88710ad6508189f871b8dd2f64a37ed98167f875dd8390d98e38", + "phase/calls/confirm-fresh-38-hi-m0-source-grouped_lossless.dispatch.json": "e9b8b24b7b9c9c0ec0c03669c1e9bbeafb756566635bfcb311e7c79c0e7e095b", + "phase/calls/confirm-fresh-38-hi-m0-source-grouped_lossless.receipt.json": "961f840e84aadbb9fb3afe8e121d8b5c9c03cc224987931db12f63ea9ed2b186", + "phase/calls/confirm-fresh-38-hi-m0-source-grouped_lossless.request.json": "19b53e88b1486ee2089099b8b09c5d012185dfd148ab3a63b7a6c971b31723c1", + "phase/calls/confirm-fresh-38-hi-m1-source-chronological.dispatch.json": "115b11e00f780043fbdcdd248ee82aacc4bfdd6fe270bca8521fb23f47614c70", + "phase/calls/confirm-fresh-38-hi-m1-source-chronological.receipt.json": "2a302bbdc2020423903c27103852562c2fb2ab2cdcb3870b1fdf0823928f64e0", + "phase/calls/confirm-fresh-38-hi-m1-source-chronological.request.json": "9f9762b5c539c8f1eca63ace204cc337426b9146757098c199e434cda6b09fe5", + "phase/calls/confirm-fresh-38-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "5ce7a9a960261439f899b8fba1536c40cb3883acf85bf54e1115f6c6d779e911", + "phase/calls/confirm-fresh-38-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "cd8392e6c9698fdb54d46d3dca338352b7ed0d247989560f417168973c3ad075", + "phase/calls/confirm-fresh-38-hi-m1-source-grouped_actor_prose_withheld.request.json": "66ba6cf26dc0ec0100d54f65597a23058c3f76d0f88e3813035f846e695f119d", + "phase/calls/confirm-fresh-38-hi-m1-source-grouped_lossless.dispatch.json": "5c52bc51b8cac6cc1df75f0f57b5d7c4078ee4c6b49eaf70f57b0440bc017176", + "phase/calls/confirm-fresh-38-hi-m1-source-grouped_lossless.receipt.json": "36a4ac2b134412e4f711456c8b4d5c593ce4c0540bee4149028135bcb5ec771b", + "phase/calls/confirm-fresh-38-hi-m1-source-grouped_lossless.request.json": "c07bd8c13a4ff3a9897ea51d6c471a1fd800ccebca68ace76aa850db92c57a9c", + "phase/calls/confirm-fresh-38-te-m0-source-chronological.dispatch.json": "5b7f1a49370478fc571e4a540b1b79569a6fb4cdb7146625b5492b62b8913bb5", + "phase/calls/confirm-fresh-38-te-m0-source-chronological.receipt.json": "3a1327d7e0ac0ab555744f72e226e1acd29ad88a997471061b683e4c641b021f", + "phase/calls/confirm-fresh-38-te-m0-source-chronological.request.json": "0d655e4479be29edc85982f95c5ef5e8f04c4d1cfce7aeca8a0d2ddf44c2a925", + "phase/calls/confirm-fresh-38-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "2d64a2a22b1f1acc64e8f38e77a0eaa67b406b52829f472b288c9894ccb9b777", + "phase/calls/confirm-fresh-38-te-m0-source-grouped_actor_prose_withheld.receipt.json": "d6ad2d9807ab8ad8edb3ee800a60b2f377151fd9a1afa64ab38862d5fbd20ada", + "phase/calls/confirm-fresh-38-te-m0-source-grouped_actor_prose_withheld.request.json": "4a3cd0304c6609492b3f71064c3637f21b7b7a866f20ebc80e00adb548adfb14", + "phase/calls/confirm-fresh-38-te-m0-source-grouped_lossless.dispatch.json": "600d625d19499873d89175374ff02ba4201d196d3506e1c97534461c9138e93e", + "phase/calls/confirm-fresh-38-te-m0-source-grouped_lossless.receipt.json": "5aadb324f22578b827985d8cdccbc67ed1f482cf3133b74cfba9344000711a73", + "phase/calls/confirm-fresh-38-te-m0-source-grouped_lossless.request.json": "3c678801f37fc266f8be2abd21e8e5c99af4808bf89bb60a07998325a17cdcb3", + "phase/calls/confirm-fresh-38-te-m1-source-chronological.dispatch.json": "0bd740be31f9a3987a50fef8f397da534bd8a267efecce5fe2d0c6fe060a0a95", + "phase/calls/confirm-fresh-38-te-m1-source-chronological.receipt.json": "3b1628931b6a0ed8afd6d827c91bc3111e03517b40b274baa3da3d558371235f", + "phase/calls/confirm-fresh-38-te-m1-source-chronological.request.json": "68fd4854d2bcf2a27d98a6d281e253bbbbc28564365c8a64e353aab0965f3813", + "phase/calls/confirm-fresh-38-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "491c3bb6aba47572a8e3d7ce67a12447f0d9eefefb41e3a2e91b7a88b5042635", + "phase/calls/confirm-fresh-38-te-m1-source-grouped_actor_prose_withheld.receipt.json": "0741cd12b3a3c63e59f3715fdaf34fee78a70e70df5d121ef08b5e4b412d510b", + "phase/calls/confirm-fresh-38-te-m1-source-grouped_actor_prose_withheld.request.json": "1c4ccae7a626e0667f950571ad8900947708336a8ea9c8f891e2510017cb26ac", + "phase/calls/confirm-fresh-38-te-m1-source-grouped_lossless.dispatch.json": "2c940c4a9bc4a1c6adc5169174b6a3f5c14dce765be5321b4d89e36fe7e1d483", + "phase/calls/confirm-fresh-38-te-m1-source-grouped_lossless.receipt.json": "4789fcac114838351607da1527feb822f3817fd77b7ae5bd9e0f1b9d7377579f", + "phase/calls/confirm-fresh-38-te-m1-source-grouped_lossless.request.json": "4929a2a7fb5db954fbac8b6b67f60424762a0940cc51aeb6464c84a93dff507d", + "phase/calls/confirm-fresh-39-en-m0-source-chronological.dispatch.json": "a435266785e3c68126d7fea25e63c08b15ba5ab9b61b86e8c92cb51163895372", + "phase/calls/confirm-fresh-39-en-m0-source-chronological.receipt.json": "f3a529aab2208964f747ae4a0c958e0affacda7ffb7033b7b80d7fda7d27498e", + "phase/calls/confirm-fresh-39-en-m0-source-chronological.request.json": "bd4f0580f995c5bc16fad3d70f07ae4d9e46b99c22f86e369d0f7f5873849ce7", + "phase/calls/confirm-fresh-39-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "8eb77746616e6b2174045b0ae1afa642b98d6850253d9af2ed05b21f0f60acd2", + "phase/calls/confirm-fresh-39-en-m0-source-grouped_actor_prose_withheld.receipt.json": "31666b90da22aa550c54e17c9902903f345230790e6269217581b1fa4c2433b7", + "phase/calls/confirm-fresh-39-en-m0-source-grouped_actor_prose_withheld.request.json": "459d4dd5ffc2f1f480d37d682cb239ae2bc1365ff4bf780c29b75d86ca5508e6", + "phase/calls/confirm-fresh-39-en-m0-source-grouped_lossless.dispatch.json": "40202dce30a75c4fe7dcbecb0c8c1041bd5362f68998aacbed6f164653cb7d1d", + "phase/calls/confirm-fresh-39-en-m0-source-grouped_lossless.receipt.json": "667e629d725f605b32e7f33398693aad0dcbc7d1f9a371565690c541274b1fef", + "phase/calls/confirm-fresh-39-en-m0-source-grouped_lossless.request.json": "65e71c8a88ee270aba742716d8812610f4447460fa21bfca55c93d7e629f20d8", + "phase/calls/confirm-fresh-39-en-m1-source-chronological.dispatch.json": "9c4fd2940bcf836ee47c5b69f8e0cc35415200b4ffb46d330611d4d32c77a93b", + "phase/calls/confirm-fresh-39-en-m1-source-chronological.receipt.json": "096fdad28a1be90841ab601a476d1043cffe153e9363a6cb79ff3504663658ad", + "phase/calls/confirm-fresh-39-en-m1-source-chronological.request.json": "3b51bb88f2967343a0faac6ffbd52111f58644d75b808bd4f85b0b29c5604ee8", + "phase/calls/confirm-fresh-39-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "cc0418292d9b5353fada9f1b8d839c7ef30b481acb63cb03e3f7904c37ab88fd", + "phase/calls/confirm-fresh-39-en-m1-source-grouped_actor_prose_withheld.receipt.json": "086ab37ca327b69bbff87fa0c59b6ea6eee64bb78ad7216fcfa69b4f67543af8", + "phase/calls/confirm-fresh-39-en-m1-source-grouped_actor_prose_withheld.request.json": "1e49526931307f14badec22700ae97dc21c5c0bba69f5c186ea2da73c50cf389", + "phase/calls/confirm-fresh-39-en-m1-source-grouped_lossless.dispatch.json": "e2d58be3b72a27c6b18915a1232be8513ef936250866237705b2821af0bff179", + "phase/calls/confirm-fresh-39-en-m1-source-grouped_lossless.receipt.json": "98b24a10dab5ed8afa77cbf61cdaff3aaa255e879db333ecdbf33997f10f6af5", + "phase/calls/confirm-fresh-39-en-m1-source-grouped_lossless.request.json": "4e824903cb30d547519200547829673181c4c6ecbb5041fa1aba95830aed4169", + "phase/calls/confirm-fresh-39-hi-m0-source-chronological.dispatch.json": "be7c3e2820ac7ddf1bc20d09f4cf5efcd1767127969fb44443bfdb99d8ced57c", + "phase/calls/confirm-fresh-39-hi-m0-source-chronological.receipt.json": "6f6e6d1152357903cdb5b90e6dff393c9032d5d8354bef2718faaeb15fc7adcd", + "phase/calls/confirm-fresh-39-hi-m0-source-chronological.request.json": "0cff4e8c9cd95d3620fec563e062b06d71f54921bd559206ecf967f7b21eb28a", + "phase/calls/confirm-fresh-39-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "b7bc2a7f9b181a397427208191930713b98c1ce8b662f665a64c067f6e69a9ba", + "phase/calls/confirm-fresh-39-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "00c7d610f235496cc505a7f40b398c415be26275b1f1df0c6f774d25d61e9ec8", + "phase/calls/confirm-fresh-39-hi-m0-source-grouped_actor_prose_withheld.request.json": "085ff6ed278373c8d236885d4793309a32d160b30be6f718c45c37a36848be90", + "phase/calls/confirm-fresh-39-hi-m0-source-grouped_lossless.dispatch.json": "787430d806b1291cd02d5b6b9b7e5433278b157b988b2f459d5e26132dd2aac0", + "phase/calls/confirm-fresh-39-hi-m0-source-grouped_lossless.receipt.json": "c7b9a4603b19e9635c743cd3b49441c2d72bf44740c8e50755891c136e8d9e95", + "phase/calls/confirm-fresh-39-hi-m0-source-grouped_lossless.request.json": "45ba74fea2d87b4061cb2fc95d4769d519450c7bcd29df2bd6c08b994cbbf75b", + "phase/calls/confirm-fresh-39-hi-m1-source-chronological.dispatch.json": "8a42990feee1fd8ab3e6456c5dd6f553dcf529ce2d50ee72605d12f895ab74c3", + "phase/calls/confirm-fresh-39-hi-m1-source-chronological.receipt.json": "4c62f571f362d7b677ff465181d4476d8af8ec8eb14587e407f445ff3726f5f6", + "phase/calls/confirm-fresh-39-hi-m1-source-chronological.request.json": "cc4df81ce4ad9cb12faf9f83f58efba1a640c15ab776b41b9113c31d268e9b85", + "phase/calls/confirm-fresh-39-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "49cff87a9fae42adae974a94826b74d63aed503cd79408cebe6c633b66544beb", + "phase/calls/confirm-fresh-39-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "5d400a4644fb830d1f982ad7653a8cb0e2baa670f11e0145cd494a11666fce15", + "phase/calls/confirm-fresh-39-hi-m1-source-grouped_actor_prose_withheld.request.json": "40cb5893d0d54eac46e928bc4834ccf3efe6e22d4f8485a1669d9459ea38eaaf", + "phase/calls/confirm-fresh-39-hi-m1-source-grouped_lossless.dispatch.json": "ba3ab598a428de0806fa9ca7d5bee1e043bb44b95b005a9ae1182f2557dfc83b", + "phase/calls/confirm-fresh-39-hi-m1-source-grouped_lossless.receipt.json": "c750b76d38ef2ab0ae57421a23b8643844acd7a6ca35b036f5ce926fd9bd8948", + "phase/calls/confirm-fresh-39-hi-m1-source-grouped_lossless.request.json": "b29526a64661e7660d5bce0ea546e244d2f68b52766a8e4fb8a85947dd5cf850", + "phase/calls/confirm-fresh-39-te-m0-source-chronological.dispatch.json": "52afad14d184e3d769863908047a1e28384ce10bcd0b6b3691a7da824a99fd55", + "phase/calls/confirm-fresh-39-te-m0-source-chronological.receipt.json": "0c5d55e48ff5abc2a85e99ef64880b9e169ef98df562a0b85ecc9ebb54c5056e", + "phase/calls/confirm-fresh-39-te-m0-source-chronological.request.json": "f31829800064a6e86fb61545ba77849e676a09729ff5bf9b6a51157a3a29298c", + "phase/calls/confirm-fresh-39-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "291609759c60849f969fcb449463c38b0b366e02db241b5024e45dd96abae597", + "phase/calls/confirm-fresh-39-te-m0-source-grouped_actor_prose_withheld.receipt.json": "42fc4ca37301e9f7377ba74f341dcb4d03c9f8f7d8465bed0305c5a2766d6464", + "phase/calls/confirm-fresh-39-te-m0-source-grouped_actor_prose_withheld.request.json": "622532448b6d6d2948381199e58975924f152c0a171745f77e968551d21129ee", + "phase/calls/confirm-fresh-39-te-m0-source-grouped_lossless.dispatch.json": "6ac53150887572a97fad170e69a219609572a8b74160d26bd123b9b53c7b1c30", + "phase/calls/confirm-fresh-39-te-m0-source-grouped_lossless.receipt.json": "c104ab864111fda600bba317e322e68e82ee4ac7995245364c9b8947915778f0", + "phase/calls/confirm-fresh-39-te-m0-source-grouped_lossless.request.json": "31b9fa38341b932f94b41428b1d96bd8d51c256d8314b79febd9ec93809b78e0", + "phase/calls/confirm-fresh-39-te-m1-source-chronological.dispatch.json": "5cba10b516343f53debfd97260fe527cacffb91d5e5d2ddaa891d19ea4dd3a52", + "phase/calls/confirm-fresh-39-te-m1-source-chronological.receipt.json": "4c044f8dd646b0830e96ac0fa0eef582bb2dee2e740878c4728fe8c8e2489535", + "phase/calls/confirm-fresh-39-te-m1-source-chronological.request.json": "069ceab9e19bdb4a80b470e737d097585cf928b57cab8b288b0a8da34bc395fe", + "phase/calls/confirm-fresh-39-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "614a059cb494de45f0cafc3b365d841ea42290e8a2be6606918a31afa07d090a", + "phase/calls/confirm-fresh-39-te-m1-source-grouped_actor_prose_withheld.receipt.json": "38183f5201f8f007e08c2c5ff2b826a109d9357f10c75f2faa05a9e4170bde41", + "phase/calls/confirm-fresh-39-te-m1-source-grouped_actor_prose_withheld.request.json": "2c4bb5a60cd893c6b99a437510c748b37640c6dae0f0fb1b693d67a9f6267dd1", + "phase/calls/confirm-fresh-39-te-m1-source-grouped_lossless.dispatch.json": "cea1a66f5117f7d07e43e87dc3886d54c9eecc91d20fe7d96f70a22c1aaab4fb", + "phase/calls/confirm-fresh-39-te-m1-source-grouped_lossless.receipt.json": "4cc16abde2cff8127cc5bef60af2666f09f75d5551439ed92413d56ac8402246", + "phase/calls/confirm-fresh-39-te-m1-source-grouped_lossless.request.json": "f379b2c270cab9b5bed78be9e1d9720d64b05db696571b9bb58299d1fbf4666a", + "phase/calls/confirm-fresh-40-en-m0-source-chronological.dispatch.json": "c615bcef330df069cd6d1023b34c5e4a4ac6043b9ca33305691c15432b44041f", + "phase/calls/confirm-fresh-40-en-m0-source-chronological.receipt.json": "c3d74b9f330784ebf6c4ea7a936a550c5549dc4d3e2ed26bd7b3d4e0b2195e48", + "phase/calls/confirm-fresh-40-en-m0-source-chronological.request.json": "3435dc3febf8342dad5010f251a1b6ce2dcc86cfa61ad95a93042977c202c255", + "phase/calls/confirm-fresh-40-en-m0-source-grouped_actor_prose_withheld.dispatch.json": "2e575c5cd7aaa89166acf30e37d827051755c7e3bb623327ede4583e5682672e", + "phase/calls/confirm-fresh-40-en-m0-source-grouped_actor_prose_withheld.receipt.json": "6b4644cdc790598c179c922b9eb60e2121e4bde4397289b1372932a8e821a71f", + "phase/calls/confirm-fresh-40-en-m0-source-grouped_actor_prose_withheld.request.json": "2c9cf2ac4f361d1769f6c5ec953bd01dbb77ea4420b46528daab4d8c75acc039", + "phase/calls/confirm-fresh-40-en-m0-source-grouped_lossless.dispatch.json": "5e4465e5d2e466ca7d7e41eaffae9c8c2ee76e50cb15c0d0591711e5dd18f8d4", + "phase/calls/confirm-fresh-40-en-m0-source-grouped_lossless.receipt.json": "b5a910807ab430ba5300d3608c364968023c9a7dc137b21c238a4c2350b8e3a7", + "phase/calls/confirm-fresh-40-en-m0-source-grouped_lossless.request.json": "70d9f3b3aec1ef8b8defe83f284a78d78ced025e37032ceb0e63eb584cc2382d", + "phase/calls/confirm-fresh-40-en-m1-source-chronological.dispatch.json": "920fa12ec06389a660a8c28c6c92fe4b5f7cc6b6b7113c6210f767e9c9824a76", + "phase/calls/confirm-fresh-40-en-m1-source-chronological.receipt.json": "2aff09702788da9caa314a0011ca352d4f230f34b431e8b860b3a2ae108e6211", + "phase/calls/confirm-fresh-40-en-m1-source-chronological.request.json": "053d7216880bb081752bf2314d9f0abf8e784d8a7d9ddb739ece9082613e87d3", + "phase/calls/confirm-fresh-40-en-m1-source-grouped_actor_prose_withheld.dispatch.json": "8a5fe6efd86b78fcc3d42e62d3a4e306702d0b3a1be998a5702c08ce40ddc55e", + "phase/calls/confirm-fresh-40-en-m1-source-grouped_actor_prose_withheld.receipt.json": "2f143e14715c7a80b53e8caa702490264b5335808d55386130a824b1ab96aab5", + "phase/calls/confirm-fresh-40-en-m1-source-grouped_actor_prose_withheld.request.json": "3cec6757e2a5f707fbc74f9b3a81efdec8332c5bd8301b0c5cb9e83f4a1d8412", + "phase/calls/confirm-fresh-40-en-m1-source-grouped_lossless.dispatch.json": "85bf2f7507f19f375952145aff99d6555c45f53ba52b5dbfb80ea288fe52c7a8", + "phase/calls/confirm-fresh-40-en-m1-source-grouped_lossless.receipt.json": "a5f6ff2fe1c2ec00d250bd624190014d94478910b94a2525401b2642bccec530", + "phase/calls/confirm-fresh-40-en-m1-source-grouped_lossless.request.json": "c5a7199559d41f3958ec02c0290aa34d1f2dd499b3443891b0940795ef5c656a", + "phase/calls/confirm-fresh-40-hi-m0-source-chronological.dispatch.json": "6f8e65debafe3624e109b62f63116dc02b89cc88dba112ce0b1c88693455a053", + "phase/calls/confirm-fresh-40-hi-m0-source-chronological.receipt.json": "760002f2459d1e970b1ee1568d3ec9b2fa43effc3fea3533e5daae87bd45a02e", + "phase/calls/confirm-fresh-40-hi-m0-source-chronological.request.json": "4dbb989398fef5c436a40d46aa3de3e0cd159dae084ab18e257900ecdbf51925", + "phase/calls/confirm-fresh-40-hi-m0-source-grouped_actor_prose_withheld.dispatch.json": "b9508a3d4339b9d79d9d64d8bca45dd66b95e487ba5a7deeb068391ae153ba36", + "phase/calls/confirm-fresh-40-hi-m0-source-grouped_actor_prose_withheld.receipt.json": "10c79e123e903f766c19947462248f820b67161d466576657c5cffb4c7438e5d", + "phase/calls/confirm-fresh-40-hi-m0-source-grouped_actor_prose_withheld.request.json": "39d47cbb0392ae04c3b21dc0275ef4ea8c1bfd879b907c5067b4d0eaafda1630", + "phase/calls/confirm-fresh-40-hi-m0-source-grouped_lossless.dispatch.json": "1bde4f9e76b3aefdd173cad8774158cdeb9ed17743ab4b5e4305ecae413a2ac7", + "phase/calls/confirm-fresh-40-hi-m0-source-grouped_lossless.receipt.json": "b2c9da9333b6e4518411b24c567e12dd6a4c171fc8c47fa3139265af52d9d83e", + "phase/calls/confirm-fresh-40-hi-m0-source-grouped_lossless.request.json": "01b65bdae74516f31cfaec28918089d94de8a48ca8630c4a539723f75af9351e", + "phase/calls/confirm-fresh-40-hi-m1-source-chronological.dispatch.json": "ac2f304304ca471aa5fc36f1705a9ec4c562dc51377d2bd8b09fd17472d98b3f", + "phase/calls/confirm-fresh-40-hi-m1-source-chronological.receipt.json": "8d0ba6a89d0239aec2168c448bcbbf21d390e02878d22973408ae5497bcc5b49", + "phase/calls/confirm-fresh-40-hi-m1-source-chronological.request.json": "1a8a4d734d754ffc0d38dfb610701fb3bcdbb8fc8d438e7f73a5451e50bb32f7", + "phase/calls/confirm-fresh-40-hi-m1-source-grouped_actor_prose_withheld.dispatch.json": "3e91bd9e4c8be5671b350a0d0811cef3acbf4645d5aaf5be8abaa04b14fc742f", + "phase/calls/confirm-fresh-40-hi-m1-source-grouped_actor_prose_withheld.receipt.json": "7247ed6d19ec1aafc278617d26cfefa360b208cde27ed3808cf43e77d5167c08", + "phase/calls/confirm-fresh-40-hi-m1-source-grouped_actor_prose_withheld.request.json": "ca6fc08a65ea44eb476611e24c6fdc54fae0c6f53cfae51115932c24b168ec70", + "phase/calls/confirm-fresh-40-hi-m1-source-grouped_lossless.dispatch.json": "89dea3935bf89bfac914ca18bc313a4e3acda4d906a660710fe6ef7f1fd517e3", + "phase/calls/confirm-fresh-40-hi-m1-source-grouped_lossless.receipt.json": "28401650151f0ee22a739194181dde37d7a195dffb68c6db9a49acec3dbe135d", + "phase/calls/confirm-fresh-40-hi-m1-source-grouped_lossless.request.json": "8d82c58f7bb41e40ca69141d02545e4cef49bd4720aa0040ff1bfec316643d22", + "phase/calls/confirm-fresh-40-te-m0-source-chronological.dispatch.json": "864a3b2b994d1aa6c58aa40f4d44b7fc9816897b0a358e61cb53f56e9b92ce90", + "phase/calls/confirm-fresh-40-te-m0-source-chronological.receipt.json": "8446122cf69f4957ccc9a6a819d6bfdc9e13ed35a2a694ce5f9801ad24b79da4", + "phase/calls/confirm-fresh-40-te-m0-source-chronological.request.json": "6989255c2663798e35016e7e782884e6acddb0cfbc30728a1eb61c0639636bcb", + "phase/calls/confirm-fresh-40-te-m0-source-grouped_actor_prose_withheld.dispatch.json": "30e49f4b3000af4c65dbd898305003a1208c3d845360d86df44305eab365f165", + "phase/calls/confirm-fresh-40-te-m0-source-grouped_actor_prose_withheld.receipt.json": "68788320b232c0f96089fe8919c9a2110c64dcaff53fbb1f0dad6f5fbaf16b77", + "phase/calls/confirm-fresh-40-te-m0-source-grouped_actor_prose_withheld.request.json": "16e1fdeb9608c13429192d06b0b4a95bd56e9a99f4199839464b3ca686d314ea", + "phase/calls/confirm-fresh-40-te-m0-source-grouped_lossless.dispatch.json": "82f3688e96d00c70c9f17768f6bf5bb299e72ae84c6ae6525224ec0a72031288", + "phase/calls/confirm-fresh-40-te-m0-source-grouped_lossless.receipt.json": "998944ffa61467b43c1b4e16123dac9f4e9302d45723e8e7e7b596a72102270a", + "phase/calls/confirm-fresh-40-te-m0-source-grouped_lossless.request.json": "72c8a9dc90268587fb8a558d34a96f065eced108d1bb21c35697b272a2de7c29", + "phase/calls/confirm-fresh-40-te-m1-source-chronological.dispatch.json": "0c4691e8a3150c925ae796f1ee0df3f0de3aac7a5537061f96b8fbcabe645dda", + "phase/calls/confirm-fresh-40-te-m1-source-chronological.receipt.json": "f1104509545157d78f745bbde56727f93066a35563e9a49a7e13d13daf8599d8", + "phase/calls/confirm-fresh-40-te-m1-source-chronological.request.json": "ab7febd9e447de0c027ee57a1ec08f86b53a4acb243091fc841ec032b0114565", + "phase/calls/confirm-fresh-40-te-m1-source-grouped_actor_prose_withheld.dispatch.json": "21950a15b1b1fc13a38fd244b71ac9c58fd54b97e7d6f61f337ec1af2c9cc8eb", + "phase/calls/confirm-fresh-40-te-m1-source-grouped_actor_prose_withheld.receipt.json": "af0863b30838bd698da6518c8c20b79736f0679d6704ebeea16a89c75c7ffd0c", + "phase/calls/confirm-fresh-40-te-m1-source-grouped_actor_prose_withheld.request.json": "c71f995250e6a05a23059c2c9f83e5938078cd4d0ca0cbeb985a550314cbab85", + "phase/calls/confirm-fresh-40-te-m1-source-grouped_lossless.dispatch.json": "6300f2d20b8022b007a6e050f1c28c68c2908ba406354e1a2dc0c5ad5b19416d", + "phase/calls/confirm-fresh-40-te-m1-source-grouped_lossless.receipt.json": "edc3f948d4f827cc0515c7f1e4863df25f9cd7700c197275786820ac9b2f1461", + "phase/calls/confirm-fresh-40-te-m1-source-grouped_lossless.request.json": "5d2f945008763fef899111b75d41c3d713e912c5963418729fa382fe071fe92c", + "phase/calls/meter.sqlite": "1b2dd93dab2411061e921808bf062aa4b9ad20716b2af0d39442f3cd88f32bc0", + "phase/clean-preflight-verification.json": "5fdfab3aebae097f24bec177c679674a70b7431b52d1a75b28ec81c17b904b24", + "phase/comparison-verification.json": "8cbc7760b5fea92d79ee44a7ff363d3c0cb39aed2f913c002762dac4dabf6e8f", + "phase/confirm-fresh-01-en-m0-source-chronological.result.json": "96797c104f7a78e3839f320aa377b1e928e67cc696b95a1e7c81bfcf267ef17b", + "phase/confirm-fresh-01-en-m0-source-grouped_actor_prose_withheld.result.json": "96797c104f7a78e3839f320aa377b1e928e67cc696b95a1e7c81bfcf267ef17b", + "phase/confirm-fresh-01-en-m0-source-grouped_lossless.result.json": "96797c104f7a78e3839f320aa377b1e928e67cc696b95a1e7c81bfcf267ef17b", + "phase/confirm-fresh-01-en-m1-source-chronological.result.json": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "phase/confirm-fresh-01-en-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-01-en-m1-source-grouped_lossless.result.json": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "phase/confirm-fresh-01-hi-m0-source-chronological.result.json": "96797c104f7a78e3839f320aa377b1e928e67cc696b95a1e7c81bfcf267ef17b", + "phase/confirm-fresh-01-hi-m0-source-grouped_actor_prose_withheld.result.json": "96797c104f7a78e3839f320aa377b1e928e67cc696b95a1e7c81bfcf267ef17b", + "phase/confirm-fresh-01-hi-m0-source-grouped_lossless.result.json": "96797c104f7a78e3839f320aa377b1e928e67cc696b95a1e7c81bfcf267ef17b", + "phase/confirm-fresh-01-hi-m1-source-chronological.result.json": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "phase/confirm-fresh-01-hi-m1-source-grouped_actor_prose_withheld.result.json": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "phase/confirm-fresh-01-hi-m1-source-grouped_lossless.result.json": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "phase/confirm-fresh-01-te-m0-source-chronological.result.json": "ce0b06081ae4177284f45db3e7222695e4995ff592024742d429c996d187fb1e", + "phase/confirm-fresh-01-te-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-01-te-m0-source-grouped_lossless.result.json": "96797c104f7a78e3839f320aa377b1e928e67cc696b95a1e7c81bfcf267ef17b", + "phase/confirm-fresh-01-te-m1-source-chronological.result.json": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "phase/confirm-fresh-01-te-m1-source-grouped_actor_prose_withheld.result.json": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "phase/confirm-fresh-01-te-m1-source-grouped_lossless.result.json": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "phase/confirm-fresh-02-en-m0-source-chronological.result.json": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "phase/confirm-fresh-02-en-m0-source-grouped_actor_prose_withheld.result.json": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "phase/confirm-fresh-02-en-m0-source-grouped_lossless.result.json": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "phase/confirm-fresh-02-en-m1-source-chronological.result.json": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "phase/confirm-fresh-02-en-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-02-en-m1-source-grouped_lossless.result.json": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "phase/confirm-fresh-02-hi-m0-source-chronological.result.json": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "phase/confirm-fresh-02-hi-m0-source-grouped_actor_prose_withheld.result.json": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "phase/confirm-fresh-02-hi-m0-source-grouped_lossless.result.json": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "phase/confirm-fresh-02-hi-m1-source-chronological.result.json": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "phase/confirm-fresh-02-hi-m1-source-grouped_actor_prose_withheld.result.json": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "phase/confirm-fresh-02-hi-m1-source-grouped_lossless.result.json": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "phase/confirm-fresh-02-te-m0-source-chronological.result.json": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "phase/confirm-fresh-02-te-m0-source-grouped_actor_prose_withheld.result.json": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "phase/confirm-fresh-02-te-m0-source-grouped_lossless.result.json": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "phase/confirm-fresh-02-te-m1-source-chronological.result.json": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "phase/confirm-fresh-02-te-m1-source-grouped_actor_prose_withheld.result.json": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "phase/confirm-fresh-02-te-m1-source-grouped_lossless.result.json": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "phase/confirm-fresh-03-en-m0-source-chronological.result.json": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "phase/confirm-fresh-03-en-m0-source-grouped_actor_prose_withheld.result.json": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "phase/confirm-fresh-03-en-m0-source-grouped_lossless.result.json": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "phase/confirm-fresh-03-en-m1-source-chronological.result.json": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "phase/confirm-fresh-03-en-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-03-en-m1-source-grouped_lossless.result.json": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "phase/confirm-fresh-03-hi-m0-source-chronological.result.json": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "phase/confirm-fresh-03-hi-m0-source-grouped_actor_prose_withheld.result.json": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "phase/confirm-fresh-03-hi-m0-source-grouped_lossless.result.json": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "phase/confirm-fresh-03-hi-m1-source-chronological.result.json": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "phase/confirm-fresh-03-hi-m1-source-grouped_actor_prose_withheld.result.json": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "phase/confirm-fresh-03-hi-m1-source-grouped_lossless.result.json": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "phase/confirm-fresh-03-te-m0-source-chronological.result.json": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "phase/confirm-fresh-03-te-m0-source-grouped_actor_prose_withheld.result.json": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "phase/confirm-fresh-03-te-m0-source-grouped_lossless.result.json": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "phase/confirm-fresh-03-te-m1-source-chronological.result.json": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "phase/confirm-fresh-03-te-m1-source-grouped_actor_prose_withheld.result.json": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "phase/confirm-fresh-03-te-m1-source-grouped_lossless.result.json": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "phase/confirm-fresh-04-en-m0-source-chronological.result.json": "2a19fc24a0e2b5e27aaeea6228e6afbc6769930f05b80ca6a48bb9cd4b53b495", + "phase/confirm-fresh-04-en-m0-source-grouped_actor_prose_withheld.result.json": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "phase/confirm-fresh-04-en-m0-source-grouped_lossless.result.json": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "phase/confirm-fresh-04-en-m1-source-chronological.result.json": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "phase/confirm-fresh-04-en-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-04-en-m1-source-grouped_lossless.result.json": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "phase/confirm-fresh-04-hi-m0-source-chronological.result.json": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "phase/confirm-fresh-04-hi-m0-source-grouped_actor_prose_withheld.result.json": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "phase/confirm-fresh-04-hi-m0-source-grouped_lossless.result.json": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "phase/confirm-fresh-04-hi-m1-source-chronological.result.json": "7d9cb65e3028114bee20b6a3080462cea699456e2e865d0573765fe2cc56e2f9", + "phase/confirm-fresh-04-hi-m1-source-grouped_actor_prose_withheld.result.json": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "phase/confirm-fresh-04-hi-m1-source-grouped_lossless.result.json": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "phase/confirm-fresh-04-te-m0-source-chronological.result.json": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "phase/confirm-fresh-04-te-m0-source-grouped_actor_prose_withheld.result.json": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "phase/confirm-fresh-04-te-m0-source-grouped_lossless.result.json": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "phase/confirm-fresh-04-te-m1-source-chronological.result.json": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "phase/confirm-fresh-04-te-m1-source-grouped_actor_prose_withheld.result.json": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "phase/confirm-fresh-04-te-m1-source-grouped_lossless.result.json": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "phase/confirm-fresh-05-en-m0-source-chronological.result.json": "43fc631886e6608129bf134306522d45fc1cdc8d48c80c92eb5d99b3244c3685", + "phase/confirm-fresh-05-en-m0-source-grouped_actor_prose_withheld.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-05-en-m0-source-grouped_lossless.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-05-en-m1-source-chronological.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-05-en-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-05-en-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-05-hi-m0-source-chronological.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-05-hi-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-05-hi-m0-source-grouped_lossless.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-05-hi-m1-source-chronological.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-05-hi-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-05-hi-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-05-te-m0-source-chronological.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-05-te-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-05-te-m0-source-grouped_lossless.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-05-te-m1-source-chronological.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-05-te-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-05-te-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-06-en-m0-source-chronological.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-06-en-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-06-en-m0-source-grouped_lossless.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-06-en-m1-source-chronological.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-06-en-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-06-en-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-06-hi-m0-source-chronological.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-06-hi-m0-source-grouped_actor_prose_withheld.result.json": "c00e01707b0b846b41fdae45555f7459894195d4eba6faf002b9d99d13718add", + "phase/confirm-fresh-06-hi-m0-source-grouped_lossless.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-06-hi-m1-source-chronological.result.json": "f6cf1024269159938bbbbb36d7bb57ccf650f6100d402ac17b0a1158c5ad30fe", + "phase/confirm-fresh-06-hi-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-06-hi-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-06-te-m0-source-chronological.result.json": "39d55d34faaf847cca522ee45ea4d0c2a58eb1da026891f66acd1aa872680135", + "phase/confirm-fresh-06-te-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-06-te-m0-source-grouped_lossless.result.json": "dd1eb69535233b7875007bb6fe801cf730d36ee48921e94b961cca63df5956d8", + "phase/confirm-fresh-06-te-m1-source-chronological.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-06-te-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-06-te-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-07-en-m0-source-chronological.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-07-en-m0-source-grouped_actor_prose_withheld.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-07-en-m0-source-grouped_lossless.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-07-en-m1-source-chronological.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-07-en-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-07-en-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-07-hi-m0-source-chronological.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-07-hi-m0-source-grouped_actor_prose_withheld.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-07-hi-m0-source-grouped_lossless.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-07-hi-m1-source-chronological.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-07-hi-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-07-hi-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-07-te-m0-source-chronological.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-07-te-m0-source-grouped_actor_prose_withheld.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-07-te-m0-source-grouped_lossless.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-07-te-m1-source-chronological.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-07-te-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-07-te-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-08-en-m0-source-chronological.result.json": "dd1eb69535233b7875007bb6fe801cf730d36ee48921e94b961cca63df5956d8", + "phase/confirm-fresh-08-en-m0-source-grouped_actor_prose_withheld.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-08-en-m0-source-grouped_lossless.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-08-en-m1-source-chronological.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-08-en-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-08-en-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-08-hi-m0-source-chronological.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-08-hi-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-08-hi-m0-source-grouped_lossless.result.json": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "phase/confirm-fresh-08-hi-m1-source-chronological.result.json": "130de02ed3a97f38059e540989f910a87601778b6cf8c2cabce8fc39cebdbd14", + "phase/confirm-fresh-08-hi-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-08-hi-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-08-te-m0-source-chronological.result.json": "dd1eb69535233b7875007bb6fe801cf730d36ee48921e94b961cca63df5956d8", + "phase/confirm-fresh-08-te-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-08-te-m0-source-grouped_lossless.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-08-te-m1-source-chronological.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-08-te-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-08-te-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-09-en-m0-source-chronological.result.json": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "phase/confirm-fresh-09-en-m0-source-grouped_actor_prose_withheld.result.json": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "phase/confirm-fresh-09-en-m0-source-grouped_lossless.result.json": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "phase/confirm-fresh-09-en-m1-source-chronological.result.json": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "phase/confirm-fresh-09-en-m1-source-grouped_actor_prose_withheld.result.json": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "phase/confirm-fresh-09-en-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-09-hi-m0-source-chronological.result.json": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "phase/confirm-fresh-09-hi-m0-source-grouped_actor_prose_withheld.result.json": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "phase/confirm-fresh-09-hi-m0-source-grouped_lossless.result.json": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "phase/confirm-fresh-09-hi-m1-source-chronological.result.json": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "phase/confirm-fresh-09-hi-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-09-hi-m1-source-grouped_lossless.result.json": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "phase/confirm-fresh-09-te-m0-source-chronological.result.json": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "phase/confirm-fresh-09-te-m0-source-grouped_actor_prose_withheld.result.json": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "phase/confirm-fresh-09-te-m0-source-grouped_lossless.result.json": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "phase/confirm-fresh-09-te-m1-source-chronological.result.json": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "phase/confirm-fresh-09-te-m1-source-grouped_actor_prose_withheld.result.json": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "phase/confirm-fresh-09-te-m1-source-grouped_lossless.result.json": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "phase/confirm-fresh-10-en-m0-source-chronological.result.json": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "phase/confirm-fresh-10-en-m0-source-grouped_actor_prose_withheld.result.json": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "phase/confirm-fresh-10-en-m0-source-grouped_lossless.result.json": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "phase/confirm-fresh-10-en-m1-source-chronological.result.json": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "phase/confirm-fresh-10-en-m1-source-grouped_actor_prose_withheld.result.json": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "phase/confirm-fresh-10-en-m1-source-grouped_lossless.result.json": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "phase/confirm-fresh-10-hi-m0-source-chronological.result.json": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "phase/confirm-fresh-10-hi-m0-source-grouped_actor_prose_withheld.result.json": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "phase/confirm-fresh-10-hi-m0-source-grouped_lossless.result.json": "6294b9020f2db793978af4e410257f125f9a2791382e29e36b9e2496f58ab189", + "phase/confirm-fresh-10-hi-m1-source-chronological.result.json": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "phase/confirm-fresh-10-hi-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-10-hi-m1-source-grouped_lossless.result.json": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "phase/confirm-fresh-10-te-m0-source-chronological.result.json": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "phase/confirm-fresh-10-te-m0-source-grouped_actor_prose_withheld.result.json": "08e1e6c34e14366850d3af627a73f020762ab2bf6e4855e1cf025b0428ef955c", + "phase/confirm-fresh-10-te-m0-source-grouped_lossless.result.json": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "phase/confirm-fresh-10-te-m1-source-chronological.result.json": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "phase/confirm-fresh-10-te-m1-source-grouped_actor_prose_withheld.result.json": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "phase/confirm-fresh-10-te-m1-source-grouped_lossless.result.json": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "phase/confirm-fresh-11-en-m0-source-chronological.result.json": "841132b30f7ede989535fe78cd953f598a11858f56febf7f78b0833cfd33dc5b", + "phase/confirm-fresh-11-en-m0-source-grouped_actor_prose_withheld.result.json": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "phase/confirm-fresh-11-en-m0-source-grouped_lossless.result.json": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "phase/confirm-fresh-11-en-m1-source-chronological.result.json": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "phase/confirm-fresh-11-en-m1-source-grouped_actor_prose_withheld.result.json": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "phase/confirm-fresh-11-en-m1-source-grouped_lossless.result.json": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "phase/confirm-fresh-11-hi-m0-source-chronological.result.json": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "phase/confirm-fresh-11-hi-m0-source-grouped_actor_prose_withheld.result.json": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "phase/confirm-fresh-11-hi-m0-source-grouped_lossless.result.json": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "phase/confirm-fresh-11-hi-m1-source-chronological.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-11-hi-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-11-hi-m1-source-grouped_lossless.result.json": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "phase/confirm-fresh-11-te-m0-source-chronological.result.json": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "phase/confirm-fresh-11-te-m0-source-grouped_actor_prose_withheld.result.json": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "phase/confirm-fresh-11-te-m0-source-grouped_lossless.result.json": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "phase/confirm-fresh-11-te-m1-source-chronological.result.json": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "phase/confirm-fresh-11-te-m1-source-grouped_actor_prose_withheld.result.json": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "phase/confirm-fresh-11-te-m1-source-grouped_lossless.result.json": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "phase/confirm-fresh-12-en-m0-source-chronological.result.json": "bddfda0b83421ea19c31efae0763e4d882cd25d8f7fc51ecf50c2203c16f5d03", + "phase/confirm-fresh-12-en-m0-source-grouped_actor_prose_withheld.result.json": "fffc9f37f755ef41afa789ae57380bf1a32c91dda35e0ac1b843306140fe8823", + "phase/confirm-fresh-12-en-m0-source-grouped_lossless.result.json": "bddfda0b83421ea19c31efae0763e4d882cd25d8f7fc51ecf50c2203c16f5d03", + "phase/confirm-fresh-12-en-m1-source-chronological.result.json": "8d9128c01cc9c00795ff8a0b689d4a4892c41abb9f5a6e7e79bc69704f2194fe", + "phase/confirm-fresh-12-en-m1-source-grouped_actor_prose_withheld.result.json": "8d9128c01cc9c00795ff8a0b689d4a4892c41abb9f5a6e7e79bc69704f2194fe", + "phase/confirm-fresh-12-en-m1-source-grouped_lossless.result.json": "8d9128c01cc9c00795ff8a0b689d4a4892c41abb9f5a6e7e79bc69704f2194fe", + "phase/confirm-fresh-12-hi-m0-source-chronological.result.json": "bddfda0b83421ea19c31efae0763e4d882cd25d8f7fc51ecf50c2203c16f5d03", + "phase/confirm-fresh-12-hi-m0-source-grouped_actor_prose_withheld.result.json": "bddfda0b83421ea19c31efae0763e4d882cd25d8f7fc51ecf50c2203c16f5d03", + "phase/confirm-fresh-12-hi-m0-source-grouped_lossless.result.json": "bddfda0b83421ea19c31efae0763e4d882cd25d8f7fc51ecf50c2203c16f5d03", + "phase/confirm-fresh-12-hi-m1-source-chronological.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-12-hi-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-12-hi-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-12-te-m0-source-chronological.result.json": "c4ff9927c3a55e64897349c37882c7d2896bdfbe26855686f0af32e779f96ce5", + "phase/confirm-fresh-12-te-m0-source-grouped_actor_prose_withheld.result.json": "f6f15ce23c5eb8843b757dfdab575fa0efba2113144211af9b88ff80d2eff18e", + "phase/confirm-fresh-12-te-m0-source-grouped_lossless.result.json": "bddfda0b83421ea19c31efae0763e4d882cd25d8f7fc51ecf50c2203c16f5d03", + "phase/confirm-fresh-12-te-m1-source-chronological.result.json": "8d9128c01cc9c00795ff8a0b689d4a4892c41abb9f5a6e7e79bc69704f2194fe", + "phase/confirm-fresh-12-te-m1-source-grouped_actor_prose_withheld.result.json": "8d9128c01cc9c00795ff8a0b689d4a4892c41abb9f5a6e7e79bc69704f2194fe", + "phase/confirm-fresh-12-te-m1-source-grouped_lossless.result.json": "8d9128c01cc9c00795ff8a0b689d4a4892c41abb9f5a6e7e79bc69704f2194fe", + "phase/confirm-fresh-13-en-m0-source-chronological.result.json": "dd1eb69535233b7875007bb6fe801cf730d36ee48921e94b961cca63df5956d8", + "phase/confirm-fresh-13-en-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-13-en-m0-source-grouped_lossless.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-13-en-m1-source-chronological.result.json": "b6474bc8194ce66a7fd312e11be1d6465b06f536176a1bb68eebcb0a95a69d0d", + "phase/confirm-fresh-13-en-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-13-en-m1-source-grouped_lossless.result.json": "b6474bc8194ce66a7fd312e11be1d6465b06f536176a1bb68eebcb0a95a69d0d", + "phase/confirm-fresh-13-hi-m0-source-chronological.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-13-hi-m0-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-13-hi-m0-source-grouped_lossless.result.json": "078d3f53025ce08cbdee6c9b66d1debeeb93a4fac3ea7fb3fdad08e9f2d8a614", + "phase/confirm-fresh-13-hi-m1-source-chronological.result.json": "b6474bc8194ce66a7fd312e11be1d6465b06f536176a1bb68eebcb0a95a69d0d", + "phase/confirm-fresh-13-hi-m1-source-grouped_actor_prose_withheld.result.json": "13352be51ec1bf1e435624454d58273e0d5f946cf4140cb0b6b7a9efd71dfe1a", + "phase/confirm-fresh-13-hi-m1-source-grouped_lossless.result.json": "1243d0f5fa6cce0604eaab8f66bf829bb90d45979ff6a37c353828ca00d8b571", + "phase/confirm-fresh-13-te-m0-source-chronological.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-13-te-m0-source-grouped_actor_prose_withheld.result.json": "3a023f469ad2b66a2c3cccff96ecad9a50cb6437c0292e365161fb00067f87ce", + "phase/confirm-fresh-13-te-m0-source-grouped_lossless.result.json": "078d3f53025ce08cbdee6c9b66d1debeeb93a4fac3ea7fb3fdad08e9f2d8a614", + "phase/confirm-fresh-13-te-m1-source-chronological.result.json": "b6474bc8194ce66a7fd312e11be1d6465b06f536176a1bb68eebcb0a95a69d0d", + "phase/confirm-fresh-13-te-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-13-te-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-14-en-m0-source-chronological.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-14-en-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-14-en-m0-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-14-en-m1-source-chronological.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-14-en-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-14-en-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-14-hi-m0-source-chronological.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-14-hi-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-14-hi-m0-source-grouped_lossless.result.json": "1243d0f5fa6cce0604eaab8f66bf829bb90d45979ff6a37c353828ca00d8b571", + "phase/confirm-fresh-14-hi-m1-source-chronological.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-14-hi-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-14-hi-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-14-te-m0-source-chronological.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-14-te-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-14-te-m0-source-grouped_lossless.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-14-te-m1-source-chronological.result.json": "318ed2b4a8a6682566450e991f6b72e55084ede01f0890501cedaa5bf0774042", + "phase/confirm-fresh-14-te-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-14-te-m1-source-grouped_lossless.result.json": "318ed2b4a8a6682566450e991f6b72e55084ede01f0890501cedaa5bf0774042", + "phase/confirm-fresh-15-en-m0-source-chronological.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-15-en-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-15-en-m0-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-15-en-m1-source-chronological.result.json": "1221a7e341e0ab559d66b45f568b82184bacfdecaa04abab155224785776dcee", + "phase/confirm-fresh-15-en-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-15-en-m1-source-grouped_lossless.result.json": "1221a7e341e0ab559d66b45f568b82184bacfdecaa04abab155224785776dcee", + "phase/confirm-fresh-15-hi-m0-source-chronological.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-15-hi-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-15-hi-m0-source-grouped_lossless.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-15-hi-m1-source-chronological.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-15-hi-m1-source-grouped_actor_prose_withheld.result.json": "afd6ab4071b0df347bb5f8ee7e7e3b493e9f731b4ac18a0adbfee443737adcad", + "phase/confirm-fresh-15-hi-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-15-te-m0-source-chronological.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-15-te-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-15-te-m0-source-grouped_lossless.result.json": "3a023f469ad2b66a2c3cccff96ecad9a50cb6437c0292e365161fb00067f87ce", + "phase/confirm-fresh-15-te-m1-source-chronological.result.json": "1221a7e341e0ab559d66b45f568b82184bacfdecaa04abab155224785776dcee", + "phase/confirm-fresh-15-te-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-15-te-m1-source-grouped_lossless.result.json": "afd6ab4071b0df347bb5f8ee7e7e3b493e9f731b4ac18a0adbfee443737adcad", + "phase/confirm-fresh-16-en-m0-source-chronological.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-16-en-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-16-en-m0-source-grouped_lossless.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-16-en-m1-source-chronological.result.json": "081f2013d93cd7b24a8f7a3eb964081ac78a1dfd0105add89dcdbd0bc3ba2e8d", + "phase/confirm-fresh-16-en-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-16-en-m1-source-grouped_lossless.result.json": "081f2013d93cd7b24a8f7a3eb964081ac78a1dfd0105add89dcdbd0bc3ba2e8d", + "phase/confirm-fresh-16-hi-m0-source-chronological.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-16-hi-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-16-hi-m0-source-grouped_lossless.result.json": "3a023f469ad2b66a2c3cccff96ecad9a50cb6437c0292e365161fb00067f87ce", + "phase/confirm-fresh-16-hi-m1-source-chronological.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-16-hi-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-16-hi-m1-source-grouped_lossless.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-16-te-m0-source-chronological.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-16-te-m0-source-grouped_actor_prose_withheld.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-16-te-m0-source-grouped_lossless.result.json": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "phase/confirm-fresh-16-te-m1-source-chronological.result.json": "081f2013d93cd7b24a8f7a3eb964081ac78a1dfd0105add89dcdbd0bc3ba2e8d", + "phase/confirm-fresh-16-te-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-16-te-m1-source-grouped_lossless.result.json": "081f2013d93cd7b24a8f7a3eb964081ac78a1dfd0105add89dcdbd0bc3ba2e8d", + "phase/confirm-fresh-17-en-m0-source-chronological.result.json": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "phase/confirm-fresh-17-en-m0-source-grouped_actor_prose_withheld.result.json": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "phase/confirm-fresh-17-en-m0-source-grouped_lossless.result.json": "371aa53658864bf121a8519dd1f82296c9c5bea4c7636988c6b8bfe16335e6fd", + "phase/confirm-fresh-17-en-m1-source-chronological.result.json": "87c81f64b32de0e465ec65f5672c4e2b281ceaddfdb2a11351a1e137c744f10c", + "phase/confirm-fresh-17-en-m1-source-grouped_actor_prose_withheld.result.json": "87c81f64b32de0e465ec65f5672c4e2b281ceaddfdb2a11351a1e137c744f10c", + "phase/confirm-fresh-17-en-m1-source-grouped_lossless.result.json": "87c81f64b32de0e465ec65f5672c4e2b281ceaddfdb2a11351a1e137c744f10c", + "phase/confirm-fresh-17-hi-m0-source-chronological.result.json": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "phase/confirm-fresh-17-hi-m0-source-grouped_actor_prose_withheld.result.json": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "phase/confirm-fresh-17-hi-m0-source-grouped_lossless.result.json": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "phase/confirm-fresh-17-hi-m1-source-chronological.result.json": "87c81f64b32de0e465ec65f5672c4e2b281ceaddfdb2a11351a1e137c744f10c", + "phase/confirm-fresh-17-hi-m1-source-grouped_actor_prose_withheld.result.json": "c906ece7cab4749ad05a7177d633850ff1d4fb81386b5fae720064c1ca673a6d", + "phase/confirm-fresh-17-hi-m1-source-grouped_lossless.result.json": "62b25401af230db17301f8eac4ec80f89e16b0252c2846d9f86d83221f909d37", + "phase/confirm-fresh-17-te-m0-source-chronological.result.json": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "phase/confirm-fresh-17-te-m0-source-grouped_actor_prose_withheld.result.json": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "phase/confirm-fresh-17-te-m0-source-grouped_lossless.result.json": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "phase/confirm-fresh-17-te-m1-source-chronological.result.json": "87c81f64b32de0e465ec65f5672c4e2b281ceaddfdb2a11351a1e137c744f10c", + "phase/confirm-fresh-17-te-m1-source-grouped_actor_prose_withheld.result.json": "c906ece7cab4749ad05a7177d633850ff1d4fb81386b5fae720064c1ca673a6d", + "phase/confirm-fresh-17-te-m1-source-grouped_lossless.result.json": "87c81f64b32de0e465ec65f5672c4e2b281ceaddfdb2a11351a1e137c744f10c", + "phase/confirm-fresh-18-en-m0-source-chronological.result.json": "03bc7ffa97b58cb6893a53bd07ae07966616cd8c647d669421d85a572a958aae", + "phase/confirm-fresh-18-en-m0-source-grouped_actor_prose_withheld.result.json": "7742bd4bc3122d407a22a3b0c9ca8d2c6ca6b7b7cae5be0a9eb21194156e403c", + "phase/confirm-fresh-18-en-m0-source-grouped_lossless.result.json": "00d649cfc39ec358aa1d45e738f53feb9cb97e2c7b7fc9ca56a64eec5020214e", + "phase/confirm-fresh-18-en-m1-source-chronological.result.json": "00d649cfc39ec358aa1d45e738f53feb9cb97e2c7b7fc9ca56a64eec5020214e", + "phase/confirm-fresh-18-en-m1-source-grouped_actor_prose_withheld.result.json": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "phase/confirm-fresh-18-en-m1-source-grouped_lossless.result.json": "00d649cfc39ec358aa1d45e738f53feb9cb97e2c7b7fc9ca56a64eec5020214e", + "phase/confirm-fresh-18-hi-m0-source-chronological.result.json": "0c021d23374bf111ad50ed318de4af996b653ca2927b333c29846f3500862618", + "phase/confirm-fresh-18-hi-m0-source-grouped_actor_prose_withheld.result.json": "7907fdb6823f82aa4202371fe0d8b9004e10bbb108790c519a5f3d55774f4c29", + "phase/confirm-fresh-18-hi-m0-source-grouped_lossless.result.json": "03bc7ffa97b58cb6893a53bd07ae07966616cd8c647d669421d85a572a958aae", + "phase/confirm-fresh-18-hi-m1-source-chronological.result.json": "00d649cfc39ec358aa1d45e738f53feb9cb97e2c7b7fc9ca56a64eec5020214e", + "phase/confirm-fresh-18-hi-m1-source-grouped_actor_prose_withheld.result.json": "d25fb810ce31a2a7fb0b34adaf0be9c19ea1c74efcf7c2e95310c0b878969515", + "phase/confirm-fresh-18-hi-m1-source-grouped_lossless.result.json": "00d649cfc39ec358aa1d45e738f53feb9cb97e2c7b7fc9ca56a64eec5020214e", + "phase/confirm-fresh-18-te-m0-source-chronological.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-18-te-m0-source-grouped_actor_prose_withheld.result.json": "03bc7ffa97b58cb6893a53bd07ae07966616cd8c647d669421d85a572a958aae", + "phase/confirm-fresh-18-te-m0-source-grouped_lossless.result.json": "03bc7ffa97b58cb6893a53bd07ae07966616cd8c647d669421d85a572a958aae", + "phase/confirm-fresh-18-te-m1-source-chronological.result.json": "00d649cfc39ec358aa1d45e738f53feb9cb97e2c7b7fc9ca56a64eec5020214e", + "phase/confirm-fresh-18-te-m1-source-grouped_actor_prose_withheld.result.json": "00d649cfc39ec358aa1d45e738f53feb9cb97e2c7b7fc9ca56a64eec5020214e", + "phase/confirm-fresh-18-te-m1-source-grouped_lossless.result.json": "c3381da56b5e4c327de6a23fba063f02f3de77e20c8c41a5ba4c99f2317591f0", + "phase/confirm-fresh-19-en-m0-source-chronological.result.json": "4f2ff4c6daf1623e66caead1eec254086b77631cbd7ac1df8765b25a639a0c24", + "phase/confirm-fresh-19-en-m0-source-grouped_actor_prose_withheld.result.json": "4f2ff4c6daf1623e66caead1eec254086b77631cbd7ac1df8765b25a639a0c24", + "phase/confirm-fresh-19-en-m0-source-grouped_lossless.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-19-en-m1-source-chronological.result.json": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "phase/confirm-fresh-19-en-m1-source-grouped_actor_prose_withheld.result.json": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "phase/confirm-fresh-19-en-m1-source-grouped_lossless.result.json": "adc8ca3c25e704262c64eed8a14b5c143f1bb7391758f5d58f0af0507a0e8920", + "phase/confirm-fresh-19-hi-m0-source-chronological.result.json": "4f2ff4c6daf1623e66caead1eec254086b77631cbd7ac1df8765b25a639a0c24", + "phase/confirm-fresh-19-hi-m0-source-grouped_actor_prose_withheld.result.json": "cc96970b1133980764a44191c39620285b199dd73c992b284124711c5ff0bb66", + "phase/confirm-fresh-19-hi-m0-source-grouped_lossless.result.json": "4f2ff4c6daf1623e66caead1eec254086b77631cbd7ac1df8765b25a639a0c24", + "phase/confirm-fresh-19-hi-m1-source-chronological.result.json": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "phase/confirm-fresh-19-hi-m1-source-grouped_actor_prose_withheld.result.json": "825f103ad8b8e9643c7bfbc4865f52b7b17c40b04bfdc0ed35f3985eec0cf307", + "phase/confirm-fresh-19-hi-m1-source-grouped_lossless.result.json": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "phase/confirm-fresh-19-te-m0-source-chronological.result.json": "515ec2720e24ee39e5a4b433b38304fdcd549a00cc5b71768f7e357c2d975dfd", + "phase/confirm-fresh-19-te-m0-source-grouped_actor_prose_withheld.result.json": "e1bb23a3fb2a08e8bd9ecf4790de66550406c987a5f2f7889494245ce06626c3", + "phase/confirm-fresh-19-te-m0-source-grouped_lossless.result.json": "4f2ff4c6daf1623e66caead1eec254086b77631cbd7ac1df8765b25a639a0c24", + "phase/confirm-fresh-19-te-m1-source-chronological.result.json": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "phase/confirm-fresh-19-te-m1-source-grouped_actor_prose_withheld.result.json": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "phase/confirm-fresh-19-te-m1-source-grouped_lossless.result.json": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "phase/confirm-fresh-20-en-m0-source-chronological.result.json": "a9477a9e1fa3da3c931530f37bd83e70c3f3afbc918b52ee5ec4af2859459c9c", + "phase/confirm-fresh-20-en-m0-source-grouped_actor_prose_withheld.result.json": "47da800200c0c2a40c8f6e6d4d56ab1df082b5204fe5af8fa9d8b032bf3fec83", + "phase/confirm-fresh-20-en-m0-source-grouped_lossless.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-20-en-m1-source-chronological.result.json": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "phase/confirm-fresh-20-en-m1-source-grouped_actor_prose_withheld.result.json": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "phase/confirm-fresh-20-en-m1-source-grouped_lossless.result.json": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "phase/confirm-fresh-20-hi-m0-source-chronological.result.json": "47da800200c0c2a40c8f6e6d4d56ab1df082b5204fe5af8fa9d8b032bf3fec83", + "phase/confirm-fresh-20-hi-m0-source-grouped_actor_prose_withheld.result.json": "47da800200c0c2a40c8f6e6d4d56ab1df082b5204fe5af8fa9d8b032bf3fec83", + "phase/confirm-fresh-20-hi-m0-source-grouped_lossless.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-20-hi-m1-source-chronological.result.json": "effa8674c24cb4042a07fce7a7b1f4aa6d0de67563c9878ee6c84d3eeb2fd49a", + "phase/confirm-fresh-20-hi-m1-source-grouped_actor_prose_withheld.result.json": "4c12d6eba0d948c095f2ff50ee04722cda4c2e1f13adf8f1fa2506d0e578ca43", + "phase/confirm-fresh-20-hi-m1-source-grouped_lossless.result.json": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "phase/confirm-fresh-20-te-m0-source-chronological.result.json": "47da800200c0c2a40c8f6e6d4d56ab1df082b5204fe5af8fa9d8b032bf3fec83", + "phase/confirm-fresh-20-te-m0-source-grouped_actor_prose_withheld.result.json": "47da800200c0c2a40c8f6e6d4d56ab1df082b5204fe5af8fa9d8b032bf3fec83", + "phase/confirm-fresh-20-te-m0-source-grouped_lossless.result.json": "44002938658ce12fa57d9ee0682ca5767a021114be6c0cbe8d8adbfea1b01f2a", + "phase/confirm-fresh-20-te-m1-source-chronological.result.json": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "phase/confirm-fresh-20-te-m1-source-grouped_actor_prose_withheld.result.json": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "phase/confirm-fresh-20-te-m1-source-grouped_lossless.result.json": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "phase/confirm-fresh-21-en-m0-source-chronological.result.json": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "phase/confirm-fresh-21-en-m0-source-grouped_actor_prose_withheld.result.json": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "phase/confirm-fresh-21-en-m0-source-grouped_lossless.result.json": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "phase/confirm-fresh-21-en-m1-source-chronological.result.json": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "phase/confirm-fresh-21-en-m1-source-grouped_actor_prose_withheld.result.json": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "phase/confirm-fresh-21-en-m1-source-grouped_lossless.result.json": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "phase/confirm-fresh-21-hi-m0-source-chronological.result.json": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "phase/confirm-fresh-21-hi-m0-source-grouped_actor_prose_withheld.result.json": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "phase/confirm-fresh-21-hi-m0-source-grouped_lossless.result.json": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "phase/confirm-fresh-21-hi-m1-source-chronological.result.json": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "phase/confirm-fresh-21-hi-m1-source-grouped_actor_prose_withheld.result.json": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "phase/confirm-fresh-21-hi-m1-source-grouped_lossless.result.json": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "phase/confirm-fresh-21-te-m0-source-chronological.result.json": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "phase/confirm-fresh-21-te-m0-source-grouped_actor_prose_withheld.result.json": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "phase/confirm-fresh-21-te-m0-source-grouped_lossless.result.json": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "phase/confirm-fresh-21-te-m1-source-chronological.result.json": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "phase/confirm-fresh-21-te-m1-source-grouped_actor_prose_withheld.result.json": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "phase/confirm-fresh-21-te-m1-source-grouped_lossless.result.json": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "phase/confirm-fresh-22-en-m0-source-chronological.result.json": "7a6034546c590ec5064db6d004a77a9dcf0a4f8eab5062272644c6750c85fef7", + "phase/confirm-fresh-22-en-m0-source-grouped_actor_prose_withheld.result.json": "7a6034546c590ec5064db6d004a77a9dcf0a4f8eab5062272644c6750c85fef7", + "phase/confirm-fresh-22-en-m0-source-grouped_lossless.result.json": "7a6034546c590ec5064db6d004a77a9dcf0a4f8eab5062272644c6750c85fef7", + "phase/confirm-fresh-22-en-m1-source-chronological.result.json": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "phase/confirm-fresh-22-en-m1-source-grouped_actor_prose_withheld.result.json": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "phase/confirm-fresh-22-en-m1-source-grouped_lossless.result.json": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "phase/confirm-fresh-22-hi-m0-source-chronological.result.json": "7a6034546c590ec5064db6d004a77a9dcf0a4f8eab5062272644c6750c85fef7", + "phase/confirm-fresh-22-hi-m0-source-grouped_actor_prose_withheld.result.json": "7a6034546c590ec5064db6d004a77a9dcf0a4f8eab5062272644c6750c85fef7", + "phase/confirm-fresh-22-hi-m0-source-grouped_lossless.result.json": "e36a878b5207e4b081d01bf02b4f77a972202397cf8de8873b33fb9c111442af", + "phase/confirm-fresh-22-hi-m1-source-chronological.result.json": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "phase/confirm-fresh-22-hi-m1-source-grouped_actor_prose_withheld.result.json": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "phase/confirm-fresh-22-hi-m1-source-grouped_lossless.result.json": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "phase/confirm-fresh-22-te-m0-source-chronological.result.json": "7a6034546c590ec5064db6d004a77a9dcf0a4f8eab5062272644c6750c85fef7", + "phase/confirm-fresh-22-te-m0-source-grouped_actor_prose_withheld.result.json": "7a6034546c590ec5064db6d004a77a9dcf0a4f8eab5062272644c6750c85fef7", + "phase/confirm-fresh-22-te-m0-source-grouped_lossless.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-22-te-m1-source-chronological.result.json": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "phase/confirm-fresh-22-te-m1-source-grouped_actor_prose_withheld.result.json": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "phase/confirm-fresh-22-te-m1-source-grouped_lossless.result.json": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "phase/confirm-fresh-23-en-m0-source-chronological.result.json": "0faf0e0192f0d7377eabd553b64abae59ded0e43cc4773b8a88de2b6c1adaeed", + "phase/confirm-fresh-23-en-m0-source-grouped_actor_prose_withheld.result.json": "0faf0e0192f0d7377eabd553b64abae59ded0e43cc4773b8a88de2b6c1adaeed", + "phase/confirm-fresh-23-en-m0-source-grouped_lossless.result.json": "0faf0e0192f0d7377eabd553b64abae59ded0e43cc4773b8a88de2b6c1adaeed", + "phase/confirm-fresh-23-en-m1-source-chronological.result.json": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "phase/confirm-fresh-23-en-m1-source-grouped_actor_prose_withheld.result.json": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "phase/confirm-fresh-23-en-m1-source-grouped_lossless.result.json": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "phase/confirm-fresh-23-hi-m0-source-chronological.result.json": "0faf0e0192f0d7377eabd553b64abae59ded0e43cc4773b8a88de2b6c1adaeed", + "phase/confirm-fresh-23-hi-m0-source-grouped_actor_prose_withheld.result.json": "0faf0e0192f0d7377eabd553b64abae59ded0e43cc4773b8a88de2b6c1adaeed", + "phase/confirm-fresh-23-hi-m0-source-grouped_lossless.result.json": "0faf0e0192f0d7377eabd553b64abae59ded0e43cc4773b8a88de2b6c1adaeed", + "phase/confirm-fresh-23-hi-m1-source-chronological.result.json": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "phase/confirm-fresh-23-hi-m1-source-grouped_actor_prose_withheld.result.json": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "phase/confirm-fresh-23-hi-m1-source-grouped_lossless.result.json": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "phase/confirm-fresh-23-te-m0-source-chronological.result.json": "0faf0e0192f0d7377eabd553b64abae59ded0e43cc4773b8a88de2b6c1adaeed", + "phase/confirm-fresh-23-te-m0-source-grouped_actor_prose_withheld.result.json": "5e07ecfb62910b460fef6f452f1b4cf8e3705251a409fb26d17c729bb1105462", + "phase/confirm-fresh-23-te-m0-source-grouped_lossless.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-23-te-m1-source-chronological.result.json": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "phase/confirm-fresh-23-te-m1-source-grouped_actor_prose_withheld.result.json": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "phase/confirm-fresh-23-te-m1-source-grouped_lossless.result.json": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "phase/confirm-fresh-24-en-m0-source-chronological.result.json": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "phase/confirm-fresh-24-en-m0-source-grouped_actor_prose_withheld.result.json": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "phase/confirm-fresh-24-en-m0-source-grouped_lossless.result.json": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "phase/confirm-fresh-24-en-m1-source-chronological.result.json": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "phase/confirm-fresh-24-en-m1-source-grouped_actor_prose_withheld.result.json": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "phase/confirm-fresh-24-en-m1-source-grouped_lossless.result.json": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "phase/confirm-fresh-24-hi-m0-source-chronological.result.json": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "phase/confirm-fresh-24-hi-m0-source-grouped_actor_prose_withheld.result.json": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "phase/confirm-fresh-24-hi-m0-source-grouped_lossless.result.json": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "phase/confirm-fresh-24-hi-m1-source-chronological.result.json": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "phase/confirm-fresh-24-hi-m1-source-grouped_actor_prose_withheld.result.json": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "phase/confirm-fresh-24-hi-m1-source-grouped_lossless.result.json": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "phase/confirm-fresh-24-te-m0-source-chronological.result.json": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "phase/confirm-fresh-24-te-m0-source-grouped_actor_prose_withheld.result.json": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "phase/confirm-fresh-24-te-m0-source-grouped_lossless.result.json": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "phase/confirm-fresh-24-te-m1-source-chronological.result.json": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "phase/confirm-fresh-24-te-m1-source-grouped_actor_prose_withheld.result.json": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "phase/confirm-fresh-24-te-m1-source-grouped_lossless.result.json": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "phase/confirm-fresh-25-en-m0-source-chronological.result.json": "258b73b129b801b662376d5cb0f39961843f2127df41664b4c2a54a8e81537fa", + "phase/confirm-fresh-25-en-m0-source-grouped_actor_prose_withheld.result.json": "4111452a57b7245759463a5c03457acf2db420c0a15100b459822cc754cadf2b", + "phase/confirm-fresh-25-en-m0-source-grouped_lossless.result.json": "258b73b129b801b662376d5cb0f39961843f2127df41664b4c2a54a8e81537fa", + "phase/confirm-fresh-25-en-m1-source-chronological.result.json": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "phase/confirm-fresh-25-en-m1-source-grouped_actor_prose_withheld.result.json": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "phase/confirm-fresh-25-en-m1-source-grouped_lossless.result.json": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "phase/confirm-fresh-25-hi-m0-source-chronological.result.json": "258b73b129b801b662376d5cb0f39961843f2127df41664b4c2a54a8e81537fa", + "phase/confirm-fresh-25-hi-m0-source-grouped_actor_prose_withheld.result.json": "258b73b129b801b662376d5cb0f39961843f2127df41664b4c2a54a8e81537fa", + "phase/confirm-fresh-25-hi-m0-source-grouped_lossless.result.json": "258b73b129b801b662376d5cb0f39961843f2127df41664b4c2a54a8e81537fa", + "phase/confirm-fresh-25-hi-m1-source-chronological.result.json": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "phase/confirm-fresh-25-hi-m1-source-grouped_actor_prose_withheld.result.json": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "phase/confirm-fresh-25-hi-m1-source-grouped_lossless.result.json": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "phase/confirm-fresh-25-te-m0-source-chronological.result.json": "258b73b129b801b662376d5cb0f39961843f2127df41664b4c2a54a8e81537fa", + "phase/confirm-fresh-25-te-m0-source-grouped_actor_prose_withheld.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-25-te-m0-source-grouped_lossless.result.json": "258b73b129b801b662376d5cb0f39961843f2127df41664b4c2a54a8e81537fa", + "phase/confirm-fresh-25-te-m1-source-chronological.result.json": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "phase/confirm-fresh-25-te-m1-source-grouped_actor_prose_withheld.result.json": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "phase/confirm-fresh-25-te-m1-source-grouped_lossless.result.json": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "phase/confirm-fresh-26-en-m0-source-chronological.result.json": "fb6e951f5e2bc3a3ec400cf4d93a3dfb2910e24d4ec7d9480d665ad63036999a", + "phase/confirm-fresh-26-en-m0-source-grouped_actor_prose_withheld.result.json": "fb6e951f5e2bc3a3ec400cf4d93a3dfb2910e24d4ec7d9480d665ad63036999a", + "phase/confirm-fresh-26-en-m0-source-grouped_lossless.result.json": "fb6e951f5e2bc3a3ec400cf4d93a3dfb2910e24d4ec7d9480d665ad63036999a", + "phase/confirm-fresh-26-en-m1-source-chronological.result.json": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "phase/confirm-fresh-26-en-m1-source-grouped_actor_prose_withheld.result.json": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "phase/confirm-fresh-26-en-m1-source-grouped_lossless.result.json": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "phase/confirm-fresh-26-hi-m0-source-chronological.result.json": "fb6e951f5e2bc3a3ec400cf4d93a3dfb2910e24d4ec7d9480d665ad63036999a", + "phase/confirm-fresh-26-hi-m0-source-grouped_actor_prose_withheld.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-26-hi-m0-source-grouped_lossless.result.json": "fb6e951f5e2bc3a3ec400cf4d93a3dfb2910e24d4ec7d9480d665ad63036999a", + "phase/confirm-fresh-26-hi-m1-source-chronological.result.json": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "phase/confirm-fresh-26-hi-m1-source-grouped_actor_prose_withheld.result.json": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "phase/confirm-fresh-26-hi-m1-source-grouped_lossless.result.json": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "phase/confirm-fresh-26-te-m0-source-chronological.result.json": "fb6e951f5e2bc3a3ec400cf4d93a3dfb2910e24d4ec7d9480d665ad63036999a", + "phase/confirm-fresh-26-te-m0-source-grouped_actor_prose_withheld.result.json": "fb6e951f5e2bc3a3ec400cf4d93a3dfb2910e24d4ec7d9480d665ad63036999a", + "phase/confirm-fresh-26-te-m0-source-grouped_lossless.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-26-te-m1-source-chronological.result.json": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "phase/confirm-fresh-26-te-m1-source-grouped_actor_prose_withheld.result.json": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "phase/confirm-fresh-26-te-m1-source-grouped_lossless.result.json": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "phase/confirm-fresh-27-en-m0-source-chronological.result.json": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "phase/confirm-fresh-27-en-m0-source-grouped_actor_prose_withheld.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-27-en-m0-source-grouped_lossless.result.json": "58dc4f555443172d26580e97135531691461045bdffc7d2a8802fe869294b394", + "phase/confirm-fresh-27-en-m1-source-chronological.result.json": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "phase/confirm-fresh-27-en-m1-source-grouped_actor_prose_withheld.result.json": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "phase/confirm-fresh-27-en-m1-source-grouped_lossless.result.json": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "phase/confirm-fresh-27-hi-m0-source-chronological.result.json": "58dc4f555443172d26580e97135531691461045bdffc7d2a8802fe869294b394", + "phase/confirm-fresh-27-hi-m0-source-grouped_actor_prose_withheld.result.json": "58dc4f555443172d26580e97135531691461045bdffc7d2a8802fe869294b394", + "phase/confirm-fresh-27-hi-m0-source-grouped_lossless.result.json": "58dc4f555443172d26580e97135531691461045bdffc7d2a8802fe869294b394", + "phase/confirm-fresh-27-hi-m1-source-chronological.result.json": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "phase/confirm-fresh-27-hi-m1-source-grouped_actor_prose_withheld.result.json": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "phase/confirm-fresh-27-hi-m1-source-grouped_lossless.result.json": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "phase/confirm-fresh-27-te-m0-source-chronological.result.json": "58dc4f555443172d26580e97135531691461045bdffc7d2a8802fe869294b394", + "phase/confirm-fresh-27-te-m0-source-grouped_actor_prose_withheld.result.json": "6dad95a7d69fa7ed7109363a832152dbf413d3cdb52e0fdfd1faa74c479378a2", + "phase/confirm-fresh-27-te-m0-source-grouped_lossless.result.json": "04c15f2e05362259fb2c00f48ac7ea6f32c4898d1dbb0891c1d767bef29d25f3", + "phase/confirm-fresh-27-te-m1-source-chronological.result.json": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "phase/confirm-fresh-27-te-m1-source-grouped_actor_prose_withheld.result.json": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "phase/confirm-fresh-27-te-m1-source-grouped_lossless.result.json": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "phase/confirm-fresh-28-en-m0-source-chronological.result.json": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "phase/confirm-fresh-28-en-m0-source-grouped_actor_prose_withheld.result.json": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "phase/confirm-fresh-28-en-m0-source-grouped_lossless.result.json": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "phase/confirm-fresh-28-en-m1-source-chronological.result.json": "f70bc8efb6e5c2baaed099be3abea5bef92a529c53415c4cf1f762b65736aef1", + "phase/confirm-fresh-28-en-m1-source-grouped_actor_prose_withheld.result.json": "f70bc8efb6e5c2baaed099be3abea5bef92a529c53415c4cf1f762b65736aef1", + "phase/confirm-fresh-28-en-m1-source-grouped_lossless.result.json": "f70bc8efb6e5c2baaed099be3abea5bef92a529c53415c4cf1f762b65736aef1", + "phase/confirm-fresh-28-hi-m0-source-chronological.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-28-hi-m0-source-grouped_actor_prose_withheld.result.json": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "phase/confirm-fresh-28-hi-m0-source-grouped_lossless.result.json": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "phase/confirm-fresh-28-hi-m1-source-chronological.result.json": "9233e86d09af1cd05d13e7ec48d0afe392a6ff5d871d26feaafc1cb5c237e5eb", + "phase/confirm-fresh-28-hi-m1-source-grouped_actor_prose_withheld.result.json": "e0673a8d7cedf6a277762231f95a8784e91700ac109e2d0b0c8b7cfc020906a8", + "phase/confirm-fresh-28-hi-m1-source-grouped_lossless.result.json": "9233e86d09af1cd05d13e7ec48d0afe392a6ff5d871d26feaafc1cb5c237e5eb", + "phase/confirm-fresh-28-te-m0-source-chronological.result.json": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "phase/confirm-fresh-28-te-m0-source-grouped_actor_prose_withheld.result.json": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "phase/confirm-fresh-28-te-m0-source-grouped_lossless.result.json": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "phase/confirm-fresh-28-te-m1-source-chronological.result.json": "f70bc8efb6e5c2baaed099be3abea5bef92a529c53415c4cf1f762b65736aef1", + "phase/confirm-fresh-28-te-m1-source-grouped_actor_prose_withheld.result.json": "f70bc8efb6e5c2baaed099be3abea5bef92a529c53415c4cf1f762b65736aef1", + "phase/confirm-fresh-28-te-m1-source-grouped_lossless.result.json": "f70bc8efb6e5c2baaed099be3abea5bef92a529c53415c4cf1f762b65736aef1", + "phase/confirm-fresh-29-en-m0-source-chronological.result.json": "f517b9fdb42a8c5fdc3536e2720b69744ec169f1684a3271023240b26b8c846f", + "phase/confirm-fresh-29-en-m0-source-grouped_actor_prose_withheld.result.json": "f517b9fdb42a8c5fdc3536e2720b69744ec169f1684a3271023240b26b8c846f", + "phase/confirm-fresh-29-en-m0-source-grouped_lossless.result.json": "f517b9fdb42a8c5fdc3536e2720b69744ec169f1684a3271023240b26b8c846f", + "phase/confirm-fresh-29-en-m1-source-chronological.result.json": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "phase/confirm-fresh-29-en-m1-source-grouped_actor_prose_withheld.result.json": "03c8095ccc90b4aef14cd20fa909317b1b5b46c98c671401d5c1ecd788accce5", + "phase/confirm-fresh-29-en-m1-source-grouped_lossless.result.json": "03c8095ccc90b4aef14cd20fa909317b1b5b46c98c671401d5c1ecd788accce5", + "phase/confirm-fresh-29-hi-m0-source-chronological.result.json": "9a33d005b8af04a9daae27cfbec0007280a387d73264252880042af4f86632aa", + "phase/confirm-fresh-29-hi-m0-source-grouped_actor_prose_withheld.result.json": "f517b9fdb42a8c5fdc3536e2720b69744ec169f1684a3271023240b26b8c846f", + "phase/confirm-fresh-29-hi-m0-source-grouped_lossless.result.json": "f517b9fdb42a8c5fdc3536e2720b69744ec169f1684a3271023240b26b8c846f", + "phase/confirm-fresh-29-hi-m1-source-chronological.result.json": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "phase/confirm-fresh-29-hi-m1-source-grouped_actor_prose_withheld.result.json": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "phase/confirm-fresh-29-hi-m1-source-grouped_lossless.result.json": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "phase/confirm-fresh-29-te-m0-source-chronological.result.json": "f517b9fdb42a8c5fdc3536e2720b69744ec169f1684a3271023240b26b8c846f", + "phase/confirm-fresh-29-te-m0-source-grouped_actor_prose_withheld.result.json": "9a33d005b8af04a9daae27cfbec0007280a387d73264252880042af4f86632aa", + "phase/confirm-fresh-29-te-m0-source-grouped_lossless.result.json": "f517b9fdb42a8c5fdc3536e2720b69744ec169f1684a3271023240b26b8c846f", + "phase/confirm-fresh-29-te-m1-source-chronological.result.json": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "phase/confirm-fresh-29-te-m1-source-grouped_actor_prose_withheld.result.json": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "phase/confirm-fresh-29-te-m1-source-grouped_lossless.result.json": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "phase/confirm-fresh-30-en-m0-source-chronological.result.json": "ff996de64c0ed5833a02676aaa0f784e5b961cc8b0596750e40ebe7f0be479d3", + "phase/confirm-fresh-30-en-m0-source-grouped_actor_prose_withheld.result.json": "ff996de64c0ed5833a02676aaa0f784e5b961cc8b0596750e40ebe7f0be479d3", + "phase/confirm-fresh-30-en-m0-source-grouped_lossless.result.json": "c225aab8d19cca2515a377d9c5462cc34a003a0c3653544bba785e793db51707", + "phase/confirm-fresh-30-en-m1-source-chronological.result.json": "87902016aae6283bdff9207729e01b30ad77cad918dba806396649f58f68c74c", + "phase/confirm-fresh-30-en-m1-source-grouped_actor_prose_withheld.result.json": "da50423f0bc90841c2b9e138e9f4a9ad0340b0ec5d888b3091878471fa1a28a5", + "phase/confirm-fresh-30-en-m1-source-grouped_lossless.result.json": "87902016aae6283bdff9207729e01b30ad77cad918dba806396649f58f68c74c", + "phase/confirm-fresh-30-hi-m0-source-chronological.result.json": "c225aab8d19cca2515a377d9c5462cc34a003a0c3653544bba785e793db51707", + "phase/confirm-fresh-30-hi-m0-source-grouped_actor_prose_withheld.result.json": "ff996de64c0ed5833a02676aaa0f784e5b961cc8b0596750e40ebe7f0be479d3", + "phase/confirm-fresh-30-hi-m0-source-grouped_lossless.result.json": "ff996de64c0ed5833a02676aaa0f784e5b961cc8b0596750e40ebe7f0be479d3", + "phase/confirm-fresh-30-hi-m1-source-chronological.result.json": "87902016aae6283bdff9207729e01b30ad77cad918dba806396649f58f68c74c", + "phase/confirm-fresh-30-hi-m1-source-grouped_actor_prose_withheld.result.json": "87902016aae6283bdff9207729e01b30ad77cad918dba806396649f58f68c74c", + "phase/confirm-fresh-30-hi-m1-source-grouped_lossless.result.json": "87902016aae6283bdff9207729e01b30ad77cad918dba806396649f58f68c74c", + "phase/confirm-fresh-30-te-m0-source-chronological.result.json": "c225aab8d19cca2515a377d9c5462cc34a003a0c3653544bba785e793db51707", + "phase/confirm-fresh-30-te-m0-source-grouped_actor_prose_withheld.result.json": "c225aab8d19cca2515a377d9c5462cc34a003a0c3653544bba785e793db51707", + "phase/confirm-fresh-30-te-m0-source-grouped_lossless.result.json": "c225aab8d19cca2515a377d9c5462cc34a003a0c3653544bba785e793db51707", + "phase/confirm-fresh-30-te-m1-source-chronological.result.json": "87902016aae6283bdff9207729e01b30ad77cad918dba806396649f58f68c74c", + "phase/confirm-fresh-30-te-m1-source-grouped_actor_prose_withheld.result.json": "fc8165ed660426c4e3123966abfd67ef96b0cbe2d8cd6f54949fa8d3ece0de02", + "phase/confirm-fresh-30-te-m1-source-grouped_lossless.result.json": "da50423f0bc90841c2b9e138e9f4a9ad0340b0ec5d888b3091878471fa1a28a5", + "phase/confirm-fresh-31-en-m0-source-chronological.result.json": "b5770e7ca54498bbe9cbc192d959a98975dc03170a1aa14ce7055f82160458c4", + "phase/confirm-fresh-31-en-m0-source-grouped_actor_prose_withheld.result.json": "fe4c3267a9cec82092e8ede130613eed251227abe98c2163619dbfcb8dccec96", + "phase/confirm-fresh-31-en-m0-source-grouped_lossless.result.json": "b5770e7ca54498bbe9cbc192d959a98975dc03170a1aa14ce7055f82160458c4", + "phase/confirm-fresh-31-en-m1-source-chronological.result.json": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "phase/confirm-fresh-31-en-m1-source-grouped_actor_prose_withheld.result.json": "7c2d167966a96aabe602d89d49013e59fa527ef56f220832be19b0d7dc6940d7", + "phase/confirm-fresh-31-en-m1-source-grouped_lossless.result.json": "7c2d167966a96aabe602d89d49013e59fa527ef56f220832be19b0d7dc6940d7", + "phase/confirm-fresh-31-hi-m0-source-chronological.result.json": "b5770e7ca54498bbe9cbc192d959a98975dc03170a1aa14ce7055f82160458c4", + "phase/confirm-fresh-31-hi-m0-source-grouped_actor_prose_withheld.result.json": "fe4c3267a9cec82092e8ede130613eed251227abe98c2163619dbfcb8dccec96", + "phase/confirm-fresh-31-hi-m0-source-grouped_lossless.result.json": "ae6f1b1012dd04896248c230783fe5e9d7c1efa09ed2714a42f5f73c3c7df7b6", + "phase/confirm-fresh-31-hi-m1-source-chronological.result.json": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "phase/confirm-fresh-31-hi-m1-source-grouped_actor_prose_withheld.result.json": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "phase/confirm-fresh-31-hi-m1-source-grouped_lossless.result.json": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "phase/confirm-fresh-31-te-m0-source-chronological.result.json": "fe4c3267a9cec82092e8ede130613eed251227abe98c2163619dbfcb8dccec96", + "phase/confirm-fresh-31-te-m0-source-grouped_actor_prose_withheld.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-31-te-m0-source-grouped_lossless.result.json": "b5770e7ca54498bbe9cbc192d959a98975dc03170a1aa14ce7055f82160458c4", + "phase/confirm-fresh-31-te-m1-source-chronological.result.json": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "phase/confirm-fresh-31-te-m1-source-grouped_actor_prose_withheld.result.json": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "phase/confirm-fresh-31-te-m1-source-grouped_lossless.result.json": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "phase/confirm-fresh-32-en-m0-source-chronological.result.json": "2465b6c24f7e04b712333958ffe384554975d4fedbb6dadba79b616478a8eb65", + "phase/confirm-fresh-32-en-m0-source-grouped_actor_prose_withheld.result.json": "2465b6c24f7e04b712333958ffe384554975d4fedbb6dadba79b616478a8eb65", + "phase/confirm-fresh-32-en-m0-source-grouped_lossless.result.json": "b106ac717f61df19b67b9ff99d9131e9d442ef563917d74f9f34670967d64074", + "phase/confirm-fresh-32-en-m1-source-chronological.result.json": "40234eb32747058b66be85c3240998e1ffc5c4add188004394236ae5121c013a", + "phase/confirm-fresh-32-en-m1-source-grouped_actor_prose_withheld.result.json": "5bad8c2e7cf1eb0cb13f16233048fdaabe345340269ddf7d3e31079391a20ed5", + "phase/confirm-fresh-32-en-m1-source-grouped_lossless.result.json": "40234eb32747058b66be85c3240998e1ffc5c4add188004394236ae5121c013a", + "phase/confirm-fresh-32-hi-m0-source-chronological.result.json": "b106ac717f61df19b67b9ff99d9131e9d442ef563917d74f9f34670967d64074", + "phase/confirm-fresh-32-hi-m0-source-grouped_actor_prose_withheld.result.json": "b106ac717f61df19b67b9ff99d9131e9d442ef563917d74f9f34670967d64074", + "phase/confirm-fresh-32-hi-m0-source-grouped_lossless.result.json": "b106ac717f61df19b67b9ff99d9131e9d442ef563917d74f9f34670967d64074", + "phase/confirm-fresh-32-hi-m1-source-chronological.result.json": "40234eb32747058b66be85c3240998e1ffc5c4add188004394236ae5121c013a", + "phase/confirm-fresh-32-hi-m1-source-grouped_actor_prose_withheld.result.json": "b5537fea0aacf9bcedfc1a598c3b4e08a55b92f5c5cb181429cff1acda690630", + "phase/confirm-fresh-32-hi-m1-source-grouped_lossless.result.json": "b5537fea0aacf9bcedfc1a598c3b4e08a55b92f5c5cb181429cff1acda690630", + "phase/confirm-fresh-32-te-m0-source-chronological.result.json": "2465b6c24f7e04b712333958ffe384554975d4fedbb6dadba79b616478a8eb65", + "phase/confirm-fresh-32-te-m0-source-grouped_actor_prose_withheld.result.json": "c8610025a759bbe68179e78a62da3fb5774353b90330e6f59855fd3e75108541", + "phase/confirm-fresh-32-te-m0-source-grouped_lossless.result.json": "b106ac717f61df19b67b9ff99d9131e9d442ef563917d74f9f34670967d64074", + "phase/confirm-fresh-32-te-m1-source-chronological.result.json": "b5537fea0aacf9bcedfc1a598c3b4e08a55b92f5c5cb181429cff1acda690630", + "phase/confirm-fresh-32-te-m1-source-grouped_actor_prose_withheld.result.json": "6339bfc3a8c7c5365b3d444ec9928fe27e6a132032e7c64c5a5b885479a0be7a", + "phase/confirm-fresh-32-te-m1-source-grouped_lossless.result.json": "6339bfc3a8c7c5365b3d444ec9928fe27e6a132032e7c64c5a5b885479a0be7a", + "phase/confirm-fresh-33-en-m0-source-chronological.result.json": "c1553fd0523b1529616e1962bbf6812c5dd94faf93dff31d113956ebebf274ad", + "phase/confirm-fresh-33-en-m0-source-grouped_actor_prose_withheld.result.json": "ace684bc08dd37c5c2e8bd35a71777504297a736697c30a50b6eb7d70c02fc6b", + "phase/confirm-fresh-33-en-m0-source-grouped_lossless.result.json": "45d118b7e3e1ef29995ca08d2c547b209da33c0d933827c2d90014c8ce9920f6", + "phase/confirm-fresh-33-en-m1-source-chronological.result.json": "0e529979621b57ba683527909c61a095b19f49f46d26c9741184bd4c71571e88", + "phase/confirm-fresh-33-en-m1-source-grouped_actor_prose_withheld.result.json": "0e529979621b57ba683527909c61a095b19f49f46d26c9741184bd4c71571e88", + "phase/confirm-fresh-33-en-m1-source-grouped_lossless.result.json": "84391b7e3176e5c1aed2756ac5fc41fe066cc25886a30c9f667429bd8fa83637", + "phase/confirm-fresh-33-hi-m0-source-chronological.result.json": "c1553fd0523b1529616e1962bbf6812c5dd94faf93dff31d113956ebebf274ad", + "phase/confirm-fresh-33-hi-m0-source-grouped_actor_prose_withheld.result.json": "c1553fd0523b1529616e1962bbf6812c5dd94faf93dff31d113956ebebf274ad", + "phase/confirm-fresh-33-hi-m0-source-grouped_lossless.result.json": "750a377f99646c2dba4efcf71ca5d6a495a0fd9537249d5ad8e2342669945bf5", + "phase/confirm-fresh-33-hi-m1-source-chronological.result.json": "0e529979621b57ba683527909c61a095b19f49f46d26c9741184bd4c71571e88", + "phase/confirm-fresh-33-hi-m1-source-grouped_actor_prose_withheld.result.json": "0e529979621b57ba683527909c61a095b19f49f46d26c9741184bd4c71571e88", + "phase/confirm-fresh-33-hi-m1-source-grouped_lossless.result.json": "0e529979621b57ba683527909c61a095b19f49f46d26c9741184bd4c71571e88", + "phase/confirm-fresh-33-te-m0-source-chronological.result.json": "c1553fd0523b1529616e1962bbf6812c5dd94faf93dff31d113956ebebf274ad", + "phase/confirm-fresh-33-te-m0-source-grouped_actor_prose_withheld.result.json": "c1553fd0523b1529616e1962bbf6812c5dd94faf93dff31d113956ebebf274ad", + "phase/confirm-fresh-33-te-m0-source-grouped_lossless.result.json": "c1553fd0523b1529616e1962bbf6812c5dd94faf93dff31d113956ebebf274ad", + "phase/confirm-fresh-33-te-m1-source-chronological.result.json": "84391b7e3176e5c1aed2756ac5fc41fe066cc25886a30c9f667429bd8fa83637", + "phase/confirm-fresh-33-te-m1-source-grouped_actor_prose_withheld.result.json": "84391b7e3176e5c1aed2756ac5fc41fe066cc25886a30c9f667429bd8fa83637", + "phase/confirm-fresh-33-te-m1-source-grouped_lossless.result.json": "0e529979621b57ba683527909c61a095b19f49f46d26c9741184bd4c71571e88", + "phase/confirm-fresh-34-en-m0-source-chronological.result.json": "abd46c530745df8728e321ec83dc35accf4d0d8c2c4080a3ad91e1f119565685", + "phase/confirm-fresh-34-en-m0-source-grouped_actor_prose_withheld.result.json": "abd46c530745df8728e321ec83dc35accf4d0d8c2c4080a3ad91e1f119565685", + "phase/confirm-fresh-34-en-m0-source-grouped_lossless.result.json": "7bdd78e5546dd17df78e3fca29b4d5de29ba0df0136ee40ba69bbdbf86da1095", + "phase/confirm-fresh-34-en-m1-source-chronological.result.json": "042713423da807bcfef49b2ccab9db9d7aee0d0c696115936fca95c315d2e3bb", + "phase/confirm-fresh-34-en-m1-source-grouped_actor_prose_withheld.result.json": "042713423da807bcfef49b2ccab9db9d7aee0d0c696115936fca95c315d2e3bb", + "phase/confirm-fresh-34-en-m1-source-grouped_lossless.result.json": "042713423da807bcfef49b2ccab9db9d7aee0d0c696115936fca95c315d2e3bb", + "phase/confirm-fresh-34-hi-m0-source-chronological.result.json": "7bdd78e5546dd17df78e3fca29b4d5de29ba0df0136ee40ba69bbdbf86da1095", + "phase/confirm-fresh-34-hi-m0-source-grouped_actor_prose_withheld.result.json": "042713423da807bcfef49b2ccab9db9d7aee0d0c696115936fca95c315d2e3bb", + "phase/confirm-fresh-34-hi-m0-source-grouped_lossless.result.json": "abd46c530745df8728e321ec83dc35accf4d0d8c2c4080a3ad91e1f119565685", + "phase/confirm-fresh-34-hi-m1-source-chronological.result.json": "042713423da807bcfef49b2ccab9db9d7aee0d0c696115936fca95c315d2e3bb", + "phase/confirm-fresh-34-hi-m1-source-grouped_actor_prose_withheld.result.json": "fb15a37da4fb1c488e9b16ecb891c11ca2d0d116496c521e1c600d6a55309e25", + "phase/confirm-fresh-34-hi-m1-source-grouped_lossless.result.json": "fb15a37da4fb1c488e9b16ecb891c11ca2d0d116496c521e1c600d6a55309e25", + "phase/confirm-fresh-34-te-m0-source-chronological.result.json": "e06779a9cb208c2b4c218b7b33cb7b15f6617556fb3cb9cb92da244af97c2080", + "phase/confirm-fresh-34-te-m0-source-grouped_actor_prose_withheld.result.json": "724654f4a7af6c72796792e733aac468132ed9975139bc88bbc067193c76376a", + "phase/confirm-fresh-34-te-m0-source-grouped_lossless.result.json": "abd46c530745df8728e321ec83dc35accf4d0d8c2c4080a3ad91e1f119565685", + "phase/confirm-fresh-34-te-m1-source-chronological.result.json": "fb15a37da4fb1c488e9b16ecb891c11ca2d0d116496c521e1c600d6a55309e25", + "phase/confirm-fresh-34-te-m1-source-grouped_actor_prose_withheld.result.json": "042713423da807bcfef49b2ccab9db9d7aee0d0c696115936fca95c315d2e3bb", + "phase/confirm-fresh-34-te-m1-source-grouped_lossless.result.json": "fb15a37da4fb1c488e9b16ecb891c11ca2d0d116496c521e1c600d6a55309e25", + "phase/confirm-fresh-35-en-m0-source-chronological.result.json": "b31d0f9b5f8fe5765afbcb1eaf44d89c8c5492e0a28b6d2d4ff5fe02f1c09f85", + "phase/confirm-fresh-35-en-m0-source-grouped_actor_prose_withheld.result.json": "a10e59f23e1bcbeb10d0bb5925482d8103b4a143246bb28f303d672b12eeb2f2", + "phase/confirm-fresh-35-en-m0-source-grouped_lossless.result.json": "b31d0f9b5f8fe5765afbcb1eaf44d89c8c5492e0a28b6d2d4ff5fe02f1c09f85", + "phase/confirm-fresh-35-en-m1-source-chronological.result.json": "10dca089506ffd53f9cf8d47ddf457b0596b2a838a3856eebeac46979cd679d9", + "phase/confirm-fresh-35-en-m1-source-grouped_actor_prose_withheld.result.json": "10dca089506ffd53f9cf8d47ddf457b0596b2a838a3856eebeac46979cd679d9", + "phase/confirm-fresh-35-en-m1-source-grouped_lossless.result.json": "a10e59f23e1bcbeb10d0bb5925482d8103b4a143246bb28f303d672b12eeb2f2", + "phase/confirm-fresh-35-hi-m0-source-chronological.result.json": "296cb3c547ead0b0a5b30d3e7e77c560bc5d057059bb3ea526c0c2bd94e0c71e", + "phase/confirm-fresh-35-hi-m0-source-grouped_actor_prose_withheld.result.json": "b31d0f9b5f8fe5765afbcb1eaf44d89c8c5492e0a28b6d2d4ff5fe02f1c09f85", + "phase/confirm-fresh-35-hi-m0-source-grouped_lossless.result.json": "800d592c65e097996bd4fe2bdf7496846a12d805e45f705bbbedf93d10502234", + "phase/confirm-fresh-35-hi-m1-source-chronological.result.json": "10dca089506ffd53f9cf8d47ddf457b0596b2a838a3856eebeac46979cd679d9", + "phase/confirm-fresh-35-hi-m1-source-grouped_actor_prose_withheld.result.json": "10dca089506ffd53f9cf8d47ddf457b0596b2a838a3856eebeac46979cd679d9", + "phase/confirm-fresh-35-hi-m1-source-grouped_lossless.result.json": "a10e59f23e1bcbeb10d0bb5925482d8103b4a143246bb28f303d672b12eeb2f2", + "phase/confirm-fresh-35-te-m0-source-chronological.result.json": "b31d0f9b5f8fe5765afbcb1eaf44d89c8c5492e0a28b6d2d4ff5fe02f1c09f85", + "phase/confirm-fresh-35-te-m0-source-grouped_actor_prose_withheld.result.json": "296cb3c547ead0b0a5b30d3e7e77c560bc5d057059bb3ea526c0c2bd94e0c71e", + "phase/confirm-fresh-35-te-m0-source-grouped_lossless.result.json": "783d06630a3c4f84e56203d8144a5e4ccdbb51803d241e952be528d246e6346c", + "phase/confirm-fresh-35-te-m1-source-chronological.result.json": "a10e59f23e1bcbeb10d0bb5925482d8103b4a143246bb28f303d672b12eeb2f2", + "phase/confirm-fresh-35-te-m1-source-grouped_actor_prose_withheld.result.json": "a10e59f23e1bcbeb10d0bb5925482d8103b4a143246bb28f303d672b12eeb2f2", + "phase/confirm-fresh-35-te-m1-source-grouped_lossless.result.json": "10dca089506ffd53f9cf8d47ddf457b0596b2a838a3856eebeac46979cd679d9", + "phase/confirm-fresh-36-en-m0-source-chronological.result.json": "a46f315dbc8a877707199bd4007630048d61c78099573de0cec42b3b1ef1c46a", + "phase/confirm-fresh-36-en-m0-source-grouped_actor_prose_withheld.result.json": "e355c143c1132a08d0a14fe326d14a7af5058d4c24b0abb602658e8728b625ab", + "phase/confirm-fresh-36-en-m0-source-grouped_lossless.result.json": "f09ee14139dffa4254eb8299e58575fdc38b4acb79393af281e81167c8fe2c0b", + "phase/confirm-fresh-36-en-m1-source-chronological.result.json": "5766d8a3d2817eb1b9e489c5c75934249fc394e5c0209fb689239f6b03244dca", + "phase/confirm-fresh-36-en-m1-source-grouped_actor_prose_withheld.result.json": "5766d8a3d2817eb1b9e489c5c75934249fc394e5c0209fb689239f6b03244dca", + "phase/confirm-fresh-36-en-m1-source-grouped_lossless.result.json": "460760fb570c5e062b8fdc0daad95643745d12fc8f13ad2e8f3402ebad0a8899", + "phase/confirm-fresh-36-hi-m0-source-chronological.result.json": "e355c143c1132a08d0a14fe326d14a7af5058d4c24b0abb602658e8728b625ab", + "phase/confirm-fresh-36-hi-m0-source-grouped_actor_prose_withheld.result.json": "f09ee14139dffa4254eb8299e58575fdc38b4acb79393af281e81167c8fe2c0b", + "phase/confirm-fresh-36-hi-m0-source-grouped_lossless.result.json": "e355c143c1132a08d0a14fe326d14a7af5058d4c24b0abb602658e8728b625ab", + "phase/confirm-fresh-36-hi-m1-source-chronological.result.json": "5766d8a3d2817eb1b9e489c5c75934249fc394e5c0209fb689239f6b03244dca", + "phase/confirm-fresh-36-hi-m1-source-grouped_actor_prose_withheld.result.json": "807ff01b1b38bc10bcd9060bd0d3b1ae9c42d960b9a0482d8c03411278d3cf2d", + "phase/confirm-fresh-36-hi-m1-source-grouped_lossless.result.json": "5766d8a3d2817eb1b9e489c5c75934249fc394e5c0209fb689239f6b03244dca", + "phase/confirm-fresh-36-te-m0-source-chronological.result.json": "f09ee14139dffa4254eb8299e58575fdc38b4acb79393af281e81167c8fe2c0b", + "phase/confirm-fresh-36-te-m0-source-grouped_actor_prose_withheld.result.json": "f09ee14139dffa4254eb8299e58575fdc38b4acb79393af281e81167c8fe2c0b", + "phase/confirm-fresh-36-te-m0-source-grouped_lossless.result.json": "f09ee14139dffa4254eb8299e58575fdc38b4acb79393af281e81167c8fe2c0b", + "phase/confirm-fresh-36-te-m1-source-chronological.result.json": "5766d8a3d2817eb1b9e489c5c75934249fc394e5c0209fb689239f6b03244dca", + "phase/confirm-fresh-36-te-m1-source-grouped_actor_prose_withheld.result.json": "807ff01b1b38bc10bcd9060bd0d3b1ae9c42d960b9a0482d8c03411278d3cf2d", + "phase/confirm-fresh-36-te-m1-source-grouped_lossless.result.json": "5766d8a3d2817eb1b9e489c5c75934249fc394e5c0209fb689239f6b03244dca", + "phase/confirm-fresh-37-en-m0-source-chronological.result.json": "c7a56f88f850e3a73663b3db8254ace39a97d0a15d6eee5b44ffb9b303316140", + "phase/confirm-fresh-37-en-m0-source-grouped_actor_prose_withheld.result.json": "69dde94d76c4033be2860a97d246a801af8bfcec0b584aa6c9f0986a935603bb", + "phase/confirm-fresh-37-en-m0-source-grouped_lossless.result.json": "aff48058a24de4a1db9e0fa8c903c6c318fd62f91bba7bbd8e5d3168a70bc729", + "phase/confirm-fresh-37-en-m1-source-chronological.result.json": "81dcf61047152bcf230429686bb9e10dd1e72d301365218d0e5c611c0810b6ec", + "phase/confirm-fresh-37-en-m1-source-grouped_actor_prose_withheld.result.json": "81dcf61047152bcf230429686bb9e10dd1e72d301365218d0e5c611c0810b6ec", + "phase/confirm-fresh-37-en-m1-source-grouped_lossless.result.json": "81dcf61047152bcf230429686bb9e10dd1e72d301365218d0e5c611c0810b6ec", + "phase/confirm-fresh-37-hi-m0-source-chronological.result.json": "aff48058a24de4a1db9e0fa8c903c6c318fd62f91bba7bbd8e5d3168a70bc729", + "phase/confirm-fresh-37-hi-m0-source-grouped_actor_prose_withheld.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-37-hi-m0-source-grouped_lossless.result.json": "9b729514e56b0e35f809adf52a6727f55fd4b6fdfc58f34533b6ccf566e2fcbd", + "phase/confirm-fresh-37-hi-m1-source-chronological.result.json": "417418804f246865317d445869ff277d57e1f417198b5646e44534270893ef1f", + "phase/confirm-fresh-37-hi-m1-source-grouped_actor_prose_withheld.result.json": "417418804f246865317d445869ff277d57e1f417198b5646e44534270893ef1f", + "phase/confirm-fresh-37-hi-m1-source-grouped_lossless.result.json": "2c3fdbb6856d232762072d5920f57471f718b0f9b72563994c9e7d9b1e6852d7", + "phase/confirm-fresh-37-te-m0-source-chronological.result.json": "69dde94d76c4033be2860a97d246a801af8bfcec0b584aa6c9f0986a935603bb", + "phase/confirm-fresh-37-te-m0-source-grouped_actor_prose_withheld.result.json": "aff48058a24de4a1db9e0fa8c903c6c318fd62f91bba7bbd8e5d3168a70bc729", + "phase/confirm-fresh-37-te-m0-source-grouped_lossless.result.json": "69dde94d76c4033be2860a97d246a801af8bfcec0b584aa6c9f0986a935603bb", + "phase/confirm-fresh-37-te-m1-source-chronological.result.json": "2c3fdbb6856d232762072d5920f57471f718b0f9b72563994c9e7d9b1e6852d7", + "phase/confirm-fresh-37-te-m1-source-grouped_actor_prose_withheld.result.json": "417418804f246865317d445869ff277d57e1f417198b5646e44534270893ef1f", + "phase/confirm-fresh-37-te-m1-source-grouped_lossless.result.json": "cfd9d0213e93bbf6f5cb1029acb085a29aded09fa4637c7b47c8f4cf1e2d4a44", + "phase/confirm-fresh-38-en-m0-source-chronological.result.json": "7179a53ab85d138478748672aef58d579781c7908dfb61f6e82b564ed4e415a5", + "phase/confirm-fresh-38-en-m0-source-grouped_actor_prose_withheld.result.json": "a83af413afe7cddcaa541b7076f09d4924ecc2e0a192a002771a74e3c3fadc75", + "phase/confirm-fresh-38-en-m0-source-grouped_lossless.result.json": "06c2187ceae2c64dea1bf91ec923b84dd3573f34714a5aafeb636bfe711a773e", + "phase/confirm-fresh-38-en-m1-source-chronological.result.json": "7f2bd78eb4f64dd1c52dcce5e859d45a219d03cec1305e5fe4c87290e091d563", + "phase/confirm-fresh-38-en-m1-source-grouped_actor_prose_withheld.result.json": "7f2bd78eb4f64dd1c52dcce5e859d45a219d03cec1305e5fe4c87290e091d563", + "phase/confirm-fresh-38-en-m1-source-grouped_lossless.result.json": "f6ce87a4a87a8702c1234da7b392f55d90d8148fb0470cd124de892151ecf6b4", + "phase/confirm-fresh-38-hi-m0-source-chronological.result.json": "a83af413afe7cddcaa541b7076f09d4924ecc2e0a192a002771a74e3c3fadc75", + "phase/confirm-fresh-38-hi-m0-source-grouped_actor_prose_withheld.result.json": "7179a53ab85d138478748672aef58d579781c7908dfb61f6e82b564ed4e415a5", + "phase/confirm-fresh-38-hi-m0-source-grouped_lossless.result.json": "7179a53ab85d138478748672aef58d579781c7908dfb61f6e82b564ed4e415a5", + "phase/confirm-fresh-38-hi-m1-source-chronological.result.json": "7f2bd78eb4f64dd1c52dcce5e859d45a219d03cec1305e5fe4c87290e091d563", + "phase/confirm-fresh-38-hi-m1-source-grouped_actor_prose_withheld.result.json": "7bb3b716e7f26e456befff8ad0778da3cde694aa22a37e4e06202fb80e43454b", + "phase/confirm-fresh-38-hi-m1-source-grouped_lossless.result.json": "f6ce87a4a87a8702c1234da7b392f55d90d8148fb0470cd124de892151ecf6b4", + "phase/confirm-fresh-38-te-m0-source-chronological.result.json": "a83af413afe7cddcaa541b7076f09d4924ecc2e0a192a002771a74e3c3fadc75", + "phase/confirm-fresh-38-te-m0-source-grouped_actor_prose_withheld.result.json": "06c2187ceae2c64dea1bf91ec923b84dd3573f34714a5aafeb636bfe711a773e", + "phase/confirm-fresh-38-te-m0-source-grouped_lossless.result.json": "a83af413afe7cddcaa541b7076f09d4924ecc2e0a192a002771a74e3c3fadc75", + "phase/confirm-fresh-38-te-m1-source-chronological.result.json": "68edae63c30afcb8dede4b7e2cdfb966d8dced07732f78d2a0d7067c4a5c045e", + "phase/confirm-fresh-38-te-m1-source-grouped_actor_prose_withheld.result.json": "7bb3b716e7f26e456befff8ad0778da3cde694aa22a37e4e06202fb80e43454b", + "phase/confirm-fresh-38-te-m1-source-grouped_lossless.result.json": "68edae63c30afcb8dede4b7e2cdfb966d8dced07732f78d2a0d7067c4a5c045e", + "phase/confirm-fresh-39-en-m0-source-chronological.result.json": "79e54d2619f02ec54fdbdce40d43be87ad5e9efa9b80215a2946ebc04bb529de", + "phase/confirm-fresh-39-en-m0-source-grouped_actor_prose_withheld.result.json": "86ce10b21452933ea2f017456ad1d7f66046b05fe0cf423ddbe8220fa19420f2", + "phase/confirm-fresh-39-en-m0-source-grouped_lossless.result.json": "86ce10b21452933ea2f017456ad1d7f66046b05fe0cf423ddbe8220fa19420f2", + "phase/confirm-fresh-39-en-m1-source-chronological.result.json": "f2371cd3e60dbad81739ea12fffa281dda1158ef039ec6a8984449c96e971832", + "phase/confirm-fresh-39-en-m1-source-grouped_actor_prose_withheld.result.json": "f2371cd3e60dbad81739ea12fffa281dda1158ef039ec6a8984449c96e971832", + "phase/confirm-fresh-39-en-m1-source-grouped_lossless.result.json": "224a2b7c56580f10e75f047d220d6d5e3e00464466c7f1ea4da77967b30e4cd5", + "phase/confirm-fresh-39-hi-m0-source-chronological.result.json": "96b42d2cc3de69828ffb97c140cf9d47de4b7c1c2276847078932bfeadb2f1f8", + "phase/confirm-fresh-39-hi-m0-source-grouped_actor_prose_withheld.result.json": "96b42d2cc3de69828ffb97c140cf9d47de4b7c1c2276847078932bfeadb2f1f8", + "phase/confirm-fresh-39-hi-m0-source-grouped_lossless.result.json": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "phase/confirm-fresh-39-hi-m1-source-chronological.result.json": "f2371cd3e60dbad81739ea12fffa281dda1158ef039ec6a8984449c96e971832", + "phase/confirm-fresh-39-hi-m1-source-grouped_actor_prose_withheld.result.json": "f2371cd3e60dbad81739ea12fffa281dda1158ef039ec6a8984449c96e971832", + "phase/confirm-fresh-39-hi-m1-source-grouped_lossless.result.json": "f2371cd3e60dbad81739ea12fffa281dda1158ef039ec6a8984449c96e971832", + "phase/confirm-fresh-39-te-m0-source-chronological.result.json": "96b42d2cc3de69828ffb97c140cf9d47de4b7c1c2276847078932bfeadb2f1f8", + "phase/confirm-fresh-39-te-m0-source-grouped_actor_prose_withheld.result.json": "96b42d2cc3de69828ffb97c140cf9d47de4b7c1c2276847078932bfeadb2f1f8", + "phase/confirm-fresh-39-te-m0-source-grouped_lossless.result.json": "86ce10b21452933ea2f017456ad1d7f66046b05fe0cf423ddbe8220fa19420f2", + "phase/confirm-fresh-39-te-m1-source-chronological.result.json": "16e46d320572a55d27d1366cb2cc4864e58b19b8a19e6e0b5044ec39ab926afc", + "phase/confirm-fresh-39-te-m1-source-grouped_actor_prose_withheld.result.json": "f79a657b75b3fc410aff8d435eeaf3c5ba5d5672d55389524137ffcefa9943dc", + "phase/confirm-fresh-39-te-m1-source-grouped_lossless.result.json": "f2371cd3e60dbad81739ea12fffa281dda1158ef039ec6a8984449c96e971832", + "phase/confirm-fresh-40-en-m0-source-chronological.result.json": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "phase/confirm-fresh-40-en-m0-source-grouped_actor_prose_withheld.result.json": "6294b9020f2db793978af4e410257f125f9a2791382e29e36b9e2496f58ab189", + "phase/confirm-fresh-40-en-m0-source-grouped_lossless.result.json": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "phase/confirm-fresh-40-en-m1-source-chronological.result.json": "e154d7d3fea97d5d74e19e49727b25d73019ead962c7b21da19391b8aa91e911", + "phase/confirm-fresh-40-en-m1-source-grouped_actor_prose_withheld.result.json": "e154d7d3fea97d5d74e19e49727b25d73019ead962c7b21da19391b8aa91e911", + "phase/confirm-fresh-40-en-m1-source-grouped_lossless.result.json": "fca49d1de4a1905da389c1cef4610aee77922b07c9c45ea909d59d69c71cb0ab", + "phase/confirm-fresh-40-hi-m0-source-chronological.result.json": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "phase/confirm-fresh-40-hi-m0-source-grouped_actor_prose_withheld.result.json": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "phase/confirm-fresh-40-hi-m0-source-grouped_lossless.result.json": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "phase/confirm-fresh-40-hi-m1-source-chronological.result.json": "e154d7d3fea97d5d74e19e49727b25d73019ead962c7b21da19391b8aa91e911", + "phase/confirm-fresh-40-hi-m1-source-grouped_actor_prose_withheld.result.json": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "phase/confirm-fresh-40-hi-m1-source-grouped_lossless.result.json": "fca49d1de4a1905da389c1cef4610aee77922b07c9c45ea909d59d69c71cb0ab", + "phase/confirm-fresh-40-te-m0-source-chronological.result.json": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "phase/confirm-fresh-40-te-m0-source-grouped_actor_prose_withheld.result.json": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "phase/confirm-fresh-40-te-m0-source-grouped_lossless.result.json": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "phase/confirm-fresh-40-te-m1-source-chronological.result.json": "e154d7d3fea97d5d74e19e49727b25d73019ead962c7b21da19391b8aa91e911", + "phase/confirm-fresh-40-te-m1-source-grouped_actor_prose_withheld.result.json": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "phase/confirm-fresh-40-te-m1-source-grouped_lossless.result.json": "fca49d1de4a1905da389c1cef4610aee77922b07c9c45ea909d59d69c71cb0ab", + "phase/decision.json": "4d2041880d199f7130f3d0b2d3a242450a21bac3f32800d9406983f2ddb6229d", + "phase/error-attribution.json": "3a6187508dee196e07c784ab8800c9d518313e3a690114bb26e20e004dd5b5db", + "phase/execution.json": "9f322fddafe182d8f0f3e5af2a34575caa4e7db3ea0286339665540419e376a3", + "phase/execution.log": "5090a054f7396925967f83e0afd7bab03941e87227b9fce7559662d71169ef90", + "phase/focused-tests.log": "9b2962ef69d053cdff21187d5b0274cc23e22d1341e7c8b41bb8d82e5fb5b8ad", + "phase/frozen-source-review.json": "6f763ad562f32c144ed72af8cd7cd1a79f89c5a6fb85a52ee1133536a0cbf635", + "phase/independent-aggregation.json": "7e4c331123f726895c37ffbb527db591a77f62e8c5db5256e5c487d255e015fc", + "phase/inputs.json": "fa0109509b5cae5fa0f4474cdd0bf1404b924cf5ed11bfaaf85ed153b833145f", + "phase/launch-preflight.json": "adee4572fff2a6a70bcf86a9b95c1622a22ad996daba8e16dd241430171614bd", + "phase/manifest.json": "ba66788d25b439471ba396c8aa2f56a1b4f136cd60cdbf0a04a40e2484bbf650", + "phase/method-review.json": "06413652d440c4e7cbba1c216db6aa5447ce9b2ca4112302c87878dc94becf88", + "phase/omissions.json": "3a2b32de723567fff2aa55b03438b3ef3b47c2e4310005cef7a037f14a461cc0", + "phase/pacing-control.json": "e6bf6b149ec109675b5d0564141bb7cd5d14e57a37ef9399c7fffcd5ed13c935", + "phase/parent-replay.json": "48f5af8bf606fe993f794b376296fe58523450175c9942f0a18c044036de2a94", + "phase/pricing-check.json": "968ac3bee63fea12e6a775e18466f476631ad34af0cfe2659263e887ec66e518", + "phase/primary-source-check.json": "c48cfa18ccde3b1757a1e4de91982617ed21fda435dbbf73a2c6d15c3dcbc9be", + "phase/prior-carry-replay.json": "91a4bb59c4844375127fabf6fa583e8cdc79d1d0f2989fb2da0007e1ab846096", + "phase/prior-regression-tests.log": "0457a3ab0cc66334e8a94b72ddaa44d5f731d3d657504dce86b591fe9826241e", + "phase/provenance/parent/PROTOCOL.md": "0444a5d9007697f305560cedaa13584ef092a8ea9194091b945e53c54902413c", + "phase/provenance/parent/budget.json": "6f9104c6fb943ba45c8979758e341e9fe69bffad19de2f91bf01e29ee1bf19e2", + "phase/provenance/parent/calls/confirm-fresh-01-en-m0-final-candidate.request.json": "a64b0f1e94247ee5768199ad854366de91ac4657aa871eecf7ddc56c9363af4f", + "phase/provenance/parent/calls/confirm-fresh-01-en-m1-final-candidate.request.json": "f002de3b841ce190f1509c5cd2f8fe56039c17b07edbe144528f24448dde4dd1", + "phase/provenance/parent/calls/confirm-fresh-01-hi-m0-final-candidate.request.json": "2e370bacec77f6844b48baec8fe78a7746b9f760bea7964b38c65a2e8339406e", + "phase/provenance/parent/calls/confirm-fresh-01-hi-m1-final-candidate.request.json": "2e4c4ab3e733f99b33ac98f90f93eb2748f5a091b0de69247111f1f5e02049da", + "phase/provenance/parent/calls/confirm-fresh-01-te-m0-final-candidate.request.json": "8c41edd706334163aacfea4bf952bd0f742a6824144d0f73e239055f1ce84c4a", + "phase/provenance/parent/calls/confirm-fresh-01-te-m1-final-candidate.request.json": "b9021bec63bad10a01dd569b8537e9761378355e253c39da1206a860748ff0ad", + "phase/provenance/parent/calls/confirm-fresh-02-en-m0-final-candidate.request.json": "70b7214a4f7e63d8871282135e120f34201a4500a5422579c52ad0ca40cb53e2", + "phase/provenance/parent/calls/confirm-fresh-02-en-m1-final-candidate.request.json": "f27cc15be770bd89bfd70f0ca3d56ce3922e8cf04dbb75165b0cf3228ad19004", + "phase/provenance/parent/calls/confirm-fresh-02-hi-m0-final-candidate.request.json": "6fbca9ccbfa47c5d43604734f2d6ef9e320c06afbd8c19a5eef1dcfb7a5cb53c", + "phase/provenance/parent/calls/confirm-fresh-02-hi-m1-final-candidate.request.json": "4402efcabe785c5a8b0bf2a01d121f1339502ef411e6c23e68c901b07060b5cd", + "phase/provenance/parent/calls/confirm-fresh-02-te-m0-final-candidate.request.json": "335c91b89f5ebeee361320d5ecc235120de2a7a3f89ba1613af538ff82ffbd2b", + "phase/provenance/parent/calls/confirm-fresh-02-te-m1-final-candidate.request.json": "69e49a8a62c2c2fe6a91275ea54bc2c2c1e8b7513ff72b759fe04a561e49c22d", + "phase/provenance/parent/calls/confirm-fresh-03-en-m0-final-candidate.request.json": "1bcdc3993614ce53402f8ca7298920d5234b8ed9430f3f01e41e61e86a5d2795", + "phase/provenance/parent/calls/confirm-fresh-03-en-m1-final-candidate.request.json": "bb06a6842660680807e80a0ee6390ec0299a119e4da03b34b0b4673d74fdb409", + "phase/provenance/parent/calls/confirm-fresh-03-hi-m0-final-candidate.request.json": "e5fc353b89225a6c34f153879b43fb8e5bb3eae24809ff4c113c7cd3ff531f4e", + "phase/provenance/parent/calls/confirm-fresh-03-hi-m1-final-candidate.request.json": "e5f4ac962d0132efadbc221a9179139a15bffa125fc903eee23e61e09603840b", + "phase/provenance/parent/calls/confirm-fresh-03-te-m0-final-candidate.request.json": "44399d319121f25fb9b3087431770ea73622b6f57e16d5725cabf6e62c3b4cb5", + "phase/provenance/parent/calls/confirm-fresh-03-te-m1-final-candidate.request.json": "db93c1cb96fa41ccdcfeaa3732c1ff2924c7bf9ab19a733b558db7d20cfa7c17", + "phase/provenance/parent/calls/confirm-fresh-04-en-m0-final-candidate.request.json": "465f54ec7bc9deea44ffeaf635308f9a625c6dcd09da4f26cb9331114d21b4b6", + "phase/provenance/parent/calls/confirm-fresh-04-en-m1-final-candidate.request.json": "de6bfae6674f172b2ad9de1857daa771e87a350a69de9119fd917d6a3633263b", + "phase/provenance/parent/calls/confirm-fresh-04-hi-m0-final-candidate.request.json": "933d7a55ad7fd44a85beba5337823ad4e1349a4c692035e0496fab038aa1371a", + "phase/provenance/parent/calls/confirm-fresh-04-hi-m1-final-candidate.request.json": "c496d7fa85fd20616d166bfddc3f39486ddabc9a2d3a8e43ff6fd2a0326b4073", + "phase/provenance/parent/calls/confirm-fresh-04-te-m0-final-candidate.request.json": "0248498915c2431fdd158971a914c5912df8a69ebb4b59a7e96b13e86392626a", + "phase/provenance/parent/calls/confirm-fresh-04-te-m1-final-candidate.request.json": "f5cffd4fd95fdc43051862d5876807cb9c365d4e5df76b2a472a40641b96d592", + "phase/provenance/parent/calls/confirm-fresh-05-en-m0-final-candidate.request.json": "844e1f1ae92822a342b60cf0e9ece3ff93feadca31c820ff68cba4f4bc2a259c", + "phase/provenance/parent/calls/confirm-fresh-05-en-m1-final-candidate.request.json": "8b1be1ac0cd05725a6de6927feb372936d167180b754eb7a331f87675c4dd96f", + "phase/provenance/parent/calls/confirm-fresh-05-hi-m0-final-candidate.request.json": "a82c1e0926ca13b899fde2412ca675119acc38929c5bc7c5a1188907334fb8a5", + "phase/provenance/parent/calls/confirm-fresh-05-hi-m1-final-candidate.request.json": "9441d96a246fbc2ddb612dec6fca65402be1af3f70f734121b3c3a3c82cd47f3", + "phase/provenance/parent/calls/confirm-fresh-05-te-m0-final-candidate.request.json": "1e87c384b0e6eabaa1e8cb8e6142e2f224cd2d44014b115a84c52bbe260f8c41", + "phase/provenance/parent/calls/confirm-fresh-05-te-m1-final-candidate.request.json": "ab443614d8a4d70891b76bc1347a7cd839f5ebc15faf82dee9945f7644745333", + "phase/provenance/parent/calls/confirm-fresh-06-en-m0-final-candidate.request.json": "6dc03de753ba274cfc88a98d8f25993b10bc30220b069fb8b77bd9ab7622d490", + "phase/provenance/parent/calls/confirm-fresh-06-en-m1-final-candidate.request.json": "e76bcf26911324b40104626e5d49c27d07c75b6751c9403517e9c3a77e5a9ead", + "phase/provenance/parent/calls/confirm-fresh-06-hi-m0-final-candidate.request.json": "c122dea806643e3fa8a63a5c2b007429c6ec78b9ff537789e35435c5f197adb8", + "phase/provenance/parent/calls/confirm-fresh-06-hi-m1-final-candidate.request.json": "245f1931cd7275c36cb4abf1b1fd9b9096425923ad809c47b9ed0dd44cf76590", + "phase/provenance/parent/calls/confirm-fresh-06-te-m0-final-candidate.request.json": "db8d066ca863ce14e8bdd9be4a1c2b8066a06b3aafb569cdf3d7789c4db81f5f", + "phase/provenance/parent/calls/confirm-fresh-06-te-m1-final-candidate.request.json": "2f52be419a1f9b2bd5451c542680d7f32d4c9f41414bd77a93e071649e077ed2", + "phase/provenance/parent/calls/confirm-fresh-07-en-m0-final-candidate.request.json": "21c959c2b53cc6f60399f33d1762c7dcdaa89f3011075f1f4d1ca7e69043cad8", + "phase/provenance/parent/calls/confirm-fresh-07-en-m1-final-candidate.request.json": "96be9c8f8f7f883e801cfb0b2d75fa9d74ac74658eff6ce17ac01bd0292ec400", + "phase/provenance/parent/calls/confirm-fresh-07-hi-m0-final-candidate.request.json": "be5fdba740ebbe4b9905936dad2ac36862df20e421c5cb76484c81f458a79f6e", + "phase/provenance/parent/calls/confirm-fresh-07-hi-m1-final-candidate.request.json": "6f9891f4056e8494b920616b10292a8e7c25f3005d2751df4517cc3b017fdb69", + "phase/provenance/parent/calls/confirm-fresh-07-te-m0-final-candidate.request.json": "d5069e7f80514d99f72decb4a92d1bdcee9a68935ac29f9d10d00d01800c5960", + "phase/provenance/parent/calls/confirm-fresh-07-te-m1-final-candidate.request.json": "6920d6dcf666e8c9facd9ca6430de990c28dd3950c5d201e93bbd83569c44c57", + "phase/provenance/parent/calls/confirm-fresh-08-en-m0-final-candidate.request.json": "e0a75cb1d3d21c9ffc0a4e0c44cb73859caf7d3a1cb266dbf170cf6baa4728bb", + "phase/provenance/parent/calls/confirm-fresh-08-en-m1-final-candidate.request.json": "fbeb6fc5536567cf602097f1e47f69a9b18bc5fbf5c361a021065f2452d2e92f", + "phase/provenance/parent/calls/confirm-fresh-08-hi-m0-final-candidate.request.json": "618a9ef9436b2789dcbb56d5f849150580821a1be5bbb0a645497f874a970904", + "phase/provenance/parent/calls/confirm-fresh-08-hi-m1-final-candidate.request.json": "cb0afce0684434f45599da2e8fcf6c269e211fa643e43f3d2a0990e603949f9a", + "phase/provenance/parent/calls/confirm-fresh-08-te-m0-final-candidate.request.json": "07f0af9fa815f6d2837048ac42e6da5210a937a48a4605d1279f1cec5c64f50f", + "phase/provenance/parent/calls/confirm-fresh-08-te-m1-final-candidate.request.json": "54c8a3955bfb8d936f5ef86aed3bcdd933b61a3455dc6ef97a597f0826304150", + "phase/provenance/parent/calls/confirm-fresh-09-en-m0-final-candidate.request.json": "61ab694ee4b3f545e35f260c5a66a73164883bb349bb21e17e2a9ddf90f2abb8", + "phase/provenance/parent/calls/confirm-fresh-09-en-m1-final-candidate.request.json": "75d63dfd3a0d64b7a3b474a7f6bd5515e3767c1216ddef7accbab07835c6f1e9", + "phase/provenance/parent/calls/confirm-fresh-09-hi-m0-final-candidate.request.json": "9ac9072ddd2a35f85a64882fa6565afb389372fe4bbc2aa2c4b32a32f8c82e77", + "phase/provenance/parent/calls/confirm-fresh-09-hi-m1-final-candidate.request.json": "534a66f9b92b39f2fd3fdb85a0b38587efc24d95ec05d169bf9077c9106b39c8", + "phase/provenance/parent/calls/confirm-fresh-09-te-m0-final-candidate.request.json": "6bb4f366338f3adb58d099164f604281f76b20412d995e3346610420b686ad2b", + "phase/provenance/parent/calls/confirm-fresh-09-te-m1-final-candidate.request.json": "a4d31e0f60a78aab0de43471453668b799a9139c97b35b9f188d271d3deac133", + "phase/provenance/parent/calls/confirm-fresh-10-en-m0-final-candidate.request.json": "ab4e9b749861a5b7fa8ecf6f11ab2e913fdb0f4b61a9cdc5557e7de5f35d1c65", + "phase/provenance/parent/calls/confirm-fresh-10-en-m1-final-candidate.request.json": "bef3c1ab33d013b393025313f2625a4e29e215e1f79d201a58fe1f112b77e6db", + "phase/provenance/parent/calls/confirm-fresh-10-hi-m0-final-candidate.request.json": "fefe0fabeb695eda8feb042b628c7fd4be3f3e5d34206d498df9fd088892559d", + "phase/provenance/parent/calls/confirm-fresh-10-hi-m1-final-candidate.request.json": "56c927f26b04b59a3fe92f3a7e978fd6c0271ef4bcb47978fbf140f37794276c", + "phase/provenance/parent/calls/confirm-fresh-10-te-m0-final-candidate.request.json": "581e32e6016fb3dcc1612506487c1db0c121a832661446c66d42df5fb21f1fba", + "phase/provenance/parent/calls/confirm-fresh-10-te-m1-final-candidate.request.json": "1ff76dc5beeece5e46eeada7374e317e4c5d4bf8a3db5caaa44e0b4140ff5263", + "phase/provenance/parent/calls/confirm-fresh-11-en-m0-final-candidate.request.json": "541f317f920302551d2e09da5e0d3f755626f22329416e0e382a4d883fb269a3", + "phase/provenance/parent/calls/confirm-fresh-11-en-m1-final-candidate.request.json": "3cf599aa01ed7851d2b4895e87faeff5a8c2c8d2a8c68ccc5c039fb8296d0aae", + "phase/provenance/parent/calls/confirm-fresh-11-hi-m0-final-candidate.request.json": "eea502744ce90a4fff8763935274e9636c8f48e3736f04c759e71d43c15efd33", + "phase/provenance/parent/calls/confirm-fresh-11-hi-m1-final-candidate.request.json": "505af1b4d2844ce04144c63fde8d5c0545c12a11a05caf4676c8be0964841e68", + "phase/provenance/parent/calls/confirm-fresh-11-te-m0-final-candidate.request.json": "0d622986d4da656bce90442e8bababa180c0d9934de1d55d94eff84f4f289a39", + "phase/provenance/parent/calls/confirm-fresh-11-te-m1-final-candidate.request.json": "a490bb14de0ad857bbeab06b76ca6a8f3b3a6822ec00db3c4ffc38ecfd0e09aa", + "phase/provenance/parent/calls/confirm-fresh-12-en-m0-final-candidate.request.json": "79da77b41660572804f982ac68596d063ba397efe0d4e530596ec997a65d30ee", + "phase/provenance/parent/calls/confirm-fresh-12-en-m1-final-candidate.request.json": "41ec638ef4cf416602ca2d2bdd8ff2e797d8fd274c0f8140fc1b56b2a92eecb5", + "phase/provenance/parent/calls/confirm-fresh-12-hi-m0-final-candidate.request.json": "b43301c2d2cc4a4edfb4edca5a262e5e91a33eb2697656663ad6d26ad76e2871", + "phase/provenance/parent/calls/confirm-fresh-12-hi-m1-final-candidate.request.json": "449863fb6eddbd965f608045dfef09404a9473fe991f4b2f9b04b7883214b526", + "phase/provenance/parent/calls/confirm-fresh-12-te-m0-final-candidate.request.json": "a4088c8e554857241243ab520a666f974d8fc50da84d5bb499ffccd75348d198", + "phase/provenance/parent/calls/confirm-fresh-12-te-m1-final-candidate.request.json": "048473a3c1419801beda5ad2f8163e730cc76af3e1dd5042ef67f6fd40f24c37", + "phase/provenance/parent/calls/confirm-fresh-13-en-m0-a0.receipt.json": "0588e476303d1c411b6564f730d67c45d95392e14b61aaa2fc16b7b94c92d258", + "phase/provenance/parent/calls/confirm-fresh-13-en-m0-a0.request.json": "f254f5105d80a86af968e71b823756abc1b66b4a48e9f807131532a5af6b7397", + "phase/provenance/parent/calls/confirm-fresh-13-en-m1-final-candidate.request.json": "dfc8e59ad848728518905e945c80dbf6b32221e2fe08ba56e4407e84a211e2a7", + "phase/provenance/parent/calls/confirm-fresh-13-hi-m0-final-candidate.request.json": "8d141b21532a31b95937a4ca37c6843e1335858823462d8b3526c5b33ac6a80f", + "phase/provenance/parent/calls/confirm-fresh-13-hi-m1-final-candidate.request.json": "8413439039d10d0d64af7b9e25dfc3d998fd606093e90f7c6d694b578037af80", + "phase/provenance/parent/calls/confirm-fresh-13-te-m0-final-candidate.request.json": "e48238bd9eeb197d96bc5cb09c2066c1719ce02dbcec0ea29aae2527c6cdaf1a", + "phase/provenance/parent/calls/confirm-fresh-13-te-m1-final-candidate.request.json": "06897aae3fec1f51d8270f6ab087e47ff3f3a2f9960281c737226c87baf457a3", + "phase/provenance/parent/calls/confirm-fresh-14-en-m0-final-candidate.request.json": "9bbfd34f9472e9be83e3c008a38edfe3cbcce75b4a6402460c3de325174b959a", + "phase/provenance/parent/calls/confirm-fresh-14-en-m1-final-candidate.request.json": "5dbb624a9f633b8631a0d0bd19a89b32cfdd6becfc8c1097061505bd4a787754", + "phase/provenance/parent/calls/confirm-fresh-14-hi-m0-final-candidate.request.json": "a8b3a889a2b384c551b3a3ea701d80a7175a1580774b822dae6116da5b9688cb", + "phase/provenance/parent/calls/confirm-fresh-14-hi-m1-final-candidate.request.json": "af376fe8900d618cf41bb0824e7c02d56b20c34c4243411764573816547da357", + "phase/provenance/parent/calls/confirm-fresh-14-te-m0-final-candidate.request.json": "782a8065bde94153a922b11da5a279c15a26483eba25d750abcb17957d9f008f", + "phase/provenance/parent/calls/confirm-fresh-14-te-m1-final-candidate.request.json": "117a9bcd1b05b189fdc0ebd4fa96fff64046d341a4dacf27aca898f52812f97b", + "phase/provenance/parent/calls/confirm-fresh-15-en-m0-final-candidate.request.json": "b213af7f2c2a6ecb8439fb710d3e4e41b38e5a1e02b8bf84fe6cccc4a53725ae", + "phase/provenance/parent/calls/confirm-fresh-15-en-m1-final-candidate.request.json": "f9fa42e19bc108068676a71f9ed795b93f12915074a8054b1afa058271d4f732", + "phase/provenance/parent/calls/confirm-fresh-15-hi-m0-final-candidate.request.json": "eee8941bcdb1149b85b92d3d00fb94d2278fdac9020b358e81536052d486f18d", + "phase/provenance/parent/calls/confirm-fresh-15-hi-m1-final-candidate.request.json": "93810d66d5741c132013e3b27b665cec16d326462218ddbee3e3505612d00dcb", + "phase/provenance/parent/calls/confirm-fresh-15-te-m0-final-candidate.request.json": "5687a7318e87c0012e05d834b07f3ec5b2208a2e42d97247771b2be084c569e5", + "phase/provenance/parent/calls/confirm-fresh-15-te-m1-final-candidate.request.json": "b979dc77e7bb7e7176a6b956b0ae35deab0589b9825ec81a93e544f59f150f36", + "phase/provenance/parent/calls/confirm-fresh-16-en-m0-final-candidate.request.json": "9ecccff46b66b4d954ee28aa94aaf5ea01852009e5b35de691a56679dbed7b64", + "phase/provenance/parent/calls/confirm-fresh-16-en-m1-final-candidate.request.json": "58f0d8cbd7f6e6a025191a645f0bf41e195b53ea894f9a38a0e466eeda7db8c4", + "phase/provenance/parent/calls/confirm-fresh-16-hi-m0-final-candidate.request.json": "1bfe7194f898182322467d2f6518fb0d1a509f57c11aa658cb0de96aafd786c4", + "phase/provenance/parent/calls/confirm-fresh-16-hi-m1-final-candidate.request.json": "6d84fd2d9d048d2abf54409ced1254afee07acb15371b51dc5354885569a960d", + "phase/provenance/parent/calls/confirm-fresh-16-te-m0-final-candidate.request.json": "2dda8e551de91063a5f6d912b408f6d1299033750705863feaf0db3d04df9b12", + "phase/provenance/parent/calls/confirm-fresh-16-te-m1-final-candidate.request.json": "396ef19b38055016f39ea6951e7669904f9e5a27ed387327e04d954240bbf1c2", + "phase/provenance/parent/calls/confirm-fresh-17-en-m0-final-candidate.request.json": "3129a2608f620da4e8437a6dfe76cc34aa0b8642a5737deab2670cab9054e4f1", + "phase/provenance/parent/calls/confirm-fresh-17-en-m1-final-candidate.request.json": "51585cb7fb313f42f326a2a4ee233d20906e2a85659eb61fcf6acf043d0f38c1", + "phase/provenance/parent/calls/confirm-fresh-17-hi-m0-final-candidate.request.json": "07802face627be0f38b7990e87bbcb0f7a68745d696eb59bafb49001b41eb088", + "phase/provenance/parent/calls/confirm-fresh-17-hi-m1-final-candidate.request.json": "e66de325902107ee97682839de9db6c5a6da39baab0eee076de6769e582ab8a3", + "phase/provenance/parent/calls/confirm-fresh-17-te-m0-final-candidate.request.json": "d4420e4479959cd04912a2dcfd736bcd1edd953bae73a951a80fd278ddd4c866", + "phase/provenance/parent/calls/confirm-fresh-17-te-m1-final-candidate.request.json": "03918a993f098fa24744b0754a8cbfb7c2633085a2b0e602f40b6a26229cfdc4", + "phase/provenance/parent/calls/confirm-fresh-18-en-m0-final-candidate.request.json": "0df3d97f759facd7c6b6e5eb477eaf4ab5a9135fc3ec4ae257cb80d1fc02ff7c", + "phase/provenance/parent/calls/confirm-fresh-18-en-m1-final-candidate.request.json": "78f4fb3b82c62b91bd291c141975021bf7e2c507d854522cbd2cf42179b6082e", + "phase/provenance/parent/calls/confirm-fresh-18-hi-m0-final-candidate.request.json": "aaa71c384086487e9a1a4f05cb67362bcfd64f21ff44df2e952950513459e916", + "phase/provenance/parent/calls/confirm-fresh-18-hi-m1-final-candidate.request.json": "8e889b4297bb4e3cdef4e6b6732715dff17a74d45426546489564b023f1f7d56", + "phase/provenance/parent/calls/confirm-fresh-18-te-m0-final-candidate.request.json": "6dc24823ce4682fd766e5d4efb916bef8f27af6ed16fc24e2476ff7b21b94d12", + "phase/provenance/parent/calls/confirm-fresh-18-te-m1-final-candidate.request.json": "c894463da437d3ed5b7441def9fef2ae2d99e2fc7d51c0181946acde70e96a39", + "phase/provenance/parent/calls/confirm-fresh-19-en-m0-final-candidate.request.json": "c373ca7cc64b56cdc17f9f52a65ed101a30c2642cb58e5c88329137ced364ff6", + "phase/provenance/parent/calls/confirm-fresh-19-en-m1-final-candidate.request.json": "ea03b1592493fa248c54061ab4a8ffcdc4fba1821c69cb33164998f0b93527ad", + "phase/provenance/parent/calls/confirm-fresh-19-hi-m0-final-candidate.request.json": "b3951eac09fb9bba28d5a3dee669057e86473400c9d55513e87bd69ada4edc8f", + "phase/provenance/parent/calls/confirm-fresh-19-hi-m1-final-candidate.request.json": "42cef7e6fd3358b2f7b3e1766401bc5d3350422b0d8d7e4e3af0b6996c77fd85", + "phase/provenance/parent/calls/confirm-fresh-19-te-m0-final-candidate.request.json": "2a25d61d9970e72ad4341e68e6199a8ed59cbb30fac8e1f85176773565ae63d9", + "phase/provenance/parent/calls/confirm-fresh-19-te-m1-final-candidate.request.json": "bc2357eb09a03103e14888b80d31f0f33fa056c0c8f990b8a9bd94e2e8f83e14", + "phase/provenance/parent/calls/confirm-fresh-20-en-m0-final-candidate.request.json": "6d64047f1b9f161786779b8d44a6b943a3a47e3b4c729bde1ea07a09c12794cd", + "phase/provenance/parent/calls/confirm-fresh-20-en-m1-final-candidate.request.json": "d13fd88545420271c85ec7c866baad0202a50774fdc5d3e85f4efcf1e1af7938", + "phase/provenance/parent/calls/confirm-fresh-20-hi-m0-final-candidate.request.json": "d8948ada6b3c4ddf922a660307bc2f51fb47a0c6091d9640b72deee27d3c21ca", + "phase/provenance/parent/calls/confirm-fresh-20-hi-m1-final-candidate.request.json": "f382d3fcb6f9ac0b1d7b2a5d06d66a490ed416b576f9e744719b0c81a0f3ff26", + "phase/provenance/parent/calls/confirm-fresh-20-te-m0-final-candidate.request.json": "127866c47890c1630022f04451d71f8bfc93055a8bf1a23165d917a69b067e05", + "phase/provenance/parent/calls/confirm-fresh-20-te-m1-final-candidate.request.json": "0d9c37d2d6c7a01d4d74cf07648cb7617a59e2323f298c1506d6fdf515197662", + "phase/provenance/parent/calls/confirm-fresh-21-en-m0-final-candidate.request.json": "9bf6b324964c0e48c030ac96e33eedaee219aa82b0a202d91a637dd9212d5c5c", + "phase/provenance/parent/calls/confirm-fresh-21-en-m1-final-candidate.request.json": "093dda90f5660655242f8180f5cd6b2a3effc38802ebbbe8dbd10197c0260f7d", + "phase/provenance/parent/calls/confirm-fresh-21-hi-m0-final-candidate.request.json": "38d5b4e030b89113254d82570027b849641d8ddcf789ec3069cfeb164d233f17", + "phase/provenance/parent/calls/confirm-fresh-21-hi-m1-final-candidate.request.json": "da2cb9ffbe2807afff7b94739562ecc6259c0e59be8449831aafee40ae9425cb", + "phase/provenance/parent/calls/confirm-fresh-21-te-m0-final-candidate.request.json": "a286ab57a2d6e260c28c54236bfb9d28d27ab19de54b06448e147245777111bd", + "phase/provenance/parent/calls/confirm-fresh-21-te-m1-final-candidate.request.json": "b85c443200766f5885dd148da68c2c19a63cc7dce130bdd04741060b70fc3c25", + "phase/provenance/parent/calls/confirm-fresh-22-en-m0-final-candidate.request.json": "9fbbfca82a2a1038197bccdfba4101a3a5ada33d7dc27b0d97e60473c7cd9775", + "phase/provenance/parent/calls/confirm-fresh-22-en-m1-final-candidate.request.json": "fc2698606fae635c239c70e2e46bea01564f0f38128465bc3741e8876a6c07d7", + "phase/provenance/parent/calls/confirm-fresh-22-hi-m0-final-candidate.request.json": "0e92e936f2874b10ee2211e479d62ad6568aa494d27765daba9a32ccaafe4ab7", + "phase/provenance/parent/calls/confirm-fresh-22-hi-m1-final-candidate.request.json": "dd49fc6159e7575d080c32f2610d2a5abad377cb29a2972ebbf4b9d5657641ab", + "phase/provenance/parent/calls/confirm-fresh-22-te-m0-final-candidate.request.json": "7a463ea09c52139bc2aaab40c87d3e57ffe60f25a83f9ce2e2747d802c6ca909", + "phase/provenance/parent/calls/confirm-fresh-22-te-m1-final-candidate.request.json": "e69bcca58ca7635826a0943be0e06719664699b28237fdf3606e2737f9088b9c", + "phase/provenance/parent/calls/confirm-fresh-23-en-m0-final-candidate.request.json": "b030a8a3bdce7f9761bb17f38bdc7e71041c1f7dd784e6c555ff536be2f48ea3", + "phase/provenance/parent/calls/confirm-fresh-23-en-m1-final-candidate.request.json": "f5e82fd1fe5307ea926d4686d46698fb63d66bcd768b951d1a6d318784e76e0a", + "phase/provenance/parent/calls/confirm-fresh-23-hi-m0-final-candidate.request.json": "d5f791afaa9ccbbda0a1e284e1896fd7702ae9039d391affd0191943f506924c", + "phase/provenance/parent/calls/confirm-fresh-23-hi-m1-final-candidate.request.json": "a1fab2908c22fe1e34f4758be43911d986250543dd79cca4695887c478ebb62f", + "phase/provenance/parent/calls/confirm-fresh-23-te-m0-final-candidate.request.json": "772e80f10cc8caa012ba2f7a9bd18bd15d1714247e719044d96a8cfc7f23e82a", + "phase/provenance/parent/calls/confirm-fresh-23-te-m1-final-candidate.request.json": "7826f8a1ee41b2d3097d43cf4327af2266f3cf4a2504f431a4d5e52cc2894eb5", + "phase/provenance/parent/calls/confirm-fresh-24-en-m0-final-candidate.request.json": "8816e0c4cb0f323283aefc5d94af6a63238ba0478683accc9dfc5e10459522a1", + "phase/provenance/parent/calls/confirm-fresh-24-en-m1-final-candidate.request.json": "a979c60be03d0543d37197410e210793bb9bfd6938175ccd8bf4ecda6e9af39b", + "phase/provenance/parent/calls/confirm-fresh-24-hi-m0-final-candidate.request.json": "7d2f1f47bbae0713f79f801371dd9e73b7a9c76b41f9144c949cd72ca3031148", + "phase/provenance/parent/calls/confirm-fresh-24-hi-m1-final-candidate.request.json": "b069eda68d2614bd5d91b8b7db920e8e026656acaceafb2447232eb8810ccccd", + "phase/provenance/parent/calls/confirm-fresh-24-te-m0-final-candidate.request.json": "7bd8f13356fbb0ecfa530ef8328c6cd4381fc6b3c1a1ceb0fa35c87b956d3965", + "phase/provenance/parent/calls/confirm-fresh-24-te-m1-final-candidate.request.json": "e0737581bbe01f2e0d99cb1f2a1b0f7dde8aa850d01304d5707b326a8da1854c", + "phase/provenance/parent/calls/confirm-fresh-25-en-m0-final-candidate.request.json": "2cc645cfb6c8bcebb1d01de637db12ae84f50e42a3cf9cf4360149ae5bfed100", + "phase/provenance/parent/calls/confirm-fresh-25-en-m1-final-candidate.request.json": "dba812ff32afabd0b406ff7dc1579c37d96ce875e13054dbeed394fd8511b568", + "phase/provenance/parent/calls/confirm-fresh-25-hi-m0-final-candidate.request.json": "de2dc8c004f529f5702b8f4f7484f2c2bb44012e88ab74ff3af2edffa49971ea", + "phase/provenance/parent/calls/confirm-fresh-25-hi-m1-final-candidate.request.json": "5313eea8117d373c89919e5d5e86baaf0f7e7f1763391040c6c72396f00cfc37", + "phase/provenance/parent/calls/confirm-fresh-25-te-m0-final-candidate.request.json": "881b830ad27b09d9ea239eb1382d56da502eb2fa366302f8a1467330d3a87170", + "phase/provenance/parent/calls/confirm-fresh-25-te-m1-final-candidate.request.json": "8ac22398f87d5ef4e688280fa66dc207484f58e4092d6aab157fbe7d3bc24fa0", + "phase/provenance/parent/calls/confirm-fresh-26-en-m0-final-candidate.request.json": "318f4800d9c1c1ca585bd02bbd786c9f1db6fbef9db05d16e006e135e31eb775", + "phase/provenance/parent/calls/confirm-fresh-26-en-m1-final-candidate.request.json": "bced9dc5d4ccd58ac2c35788934878b8b35e633f95d209ead3c03896970fcba1", + "phase/provenance/parent/calls/confirm-fresh-26-hi-m0-final-candidate.request.json": "1d268c90aee13ab8c0a1d2411dbbe56bba9e6695abe1938c6ccf6ae52cfc05f5", + "phase/provenance/parent/calls/confirm-fresh-26-hi-m1-final-candidate.request.json": "490da5ae788214c009f557d52b9a9af4216854fe15ab1abc46d7867a2c0e6ce6", + "phase/provenance/parent/calls/confirm-fresh-26-te-m0-final-candidate.request.json": "00f3c774b6d3e118d26b0773eee525b1a01a6c5f47b76e77a54391e60c20f128", + "phase/provenance/parent/calls/confirm-fresh-26-te-m1-final-candidate.request.json": "2dd6fafc35f7c36f02477e4549fd98ff164e16e3eb266e3bd7f4b62132df9888", + "phase/provenance/parent/calls/confirm-fresh-27-en-m0-final-candidate.request.json": "0ff5b09be85f750125a617af05d4ccccc4f758ceceece3c5538b141255cc9b36", + "phase/provenance/parent/calls/confirm-fresh-27-en-m1-final-candidate.request.json": "172c49715865a88f4a9e108a75bef53a559dc55176e5835934aeea0581c1cab0", + "phase/provenance/parent/calls/confirm-fresh-27-hi-m0-final-candidate.request.json": "c75be709dab127172594e3ef5df30c6455df69b71cb835e2f8b9567a845b1116", + "phase/provenance/parent/calls/confirm-fresh-27-hi-m1-final-candidate.request.json": "9a5f186cefda59447fd983d44b2572d2dcb05b2a800be39fa2ffd128f9735931", + "phase/provenance/parent/calls/confirm-fresh-27-te-m0-final-candidate.request.json": "9d9266d53cb55e598b62d058b523f0473cf538fee23d1530fa29862159e40234", + "phase/provenance/parent/calls/confirm-fresh-27-te-m1-final-candidate.request.json": "d6cbd4124ef290cbef88faa2ce88e489be6246a70a02eb979323daef3235e651", + "phase/provenance/parent/calls/confirm-fresh-28-en-m0-final-candidate.request.json": "ea01ac9793966b4b91398cff64ee8e6cc28a1a2de2911295ef24a9ca46337aa4", + "phase/provenance/parent/calls/confirm-fresh-28-en-m1-final-candidate.request.json": "afe2b10bf348f38c762870611d4b8388770e37a34b421b9f8fba8c431e88de7f", + "phase/provenance/parent/calls/confirm-fresh-28-hi-m0-final-candidate.request.json": "732d34831eba5e26a634c4eee05ca0dbb21a7497645b34ed212845419b682a27", + "phase/provenance/parent/calls/confirm-fresh-28-hi-m1-final-candidate.request.json": "9d76fd9b00baac6425bf5e427c98e2499bb26763b45e2d1870759e99d0f497b8", + "phase/provenance/parent/calls/confirm-fresh-28-te-m0-final-candidate.request.json": "c6e444c579e1db89ed04ec880ca38eeb85ff25fd441a8d2994dd74ee1afd6fae", + "phase/provenance/parent/calls/confirm-fresh-28-te-m1-final-candidate.request.json": "25bb936bc7b2365bc427d6cfcbe78eedadfbae78b8fd0574bb262d18c15e8adc", + "phase/provenance/parent/calls/confirm-fresh-29-en-m0-final-candidate.request.json": "bf49de561ee56ec12a06d960d5f150edaa9efc2b43099c8a2ca816dc4eebd721", + "phase/provenance/parent/calls/confirm-fresh-29-en-m1-final-candidate.request.json": "0d3cc2330e0fd4601f7e23983d0e56ecc1549a6a677138e93f5732295fff996c", + "phase/provenance/parent/calls/confirm-fresh-29-hi-m0-final-candidate.request.json": "7db7921f3948f5de3ae43864f85c4469653aad870add03c2cea0250d18b92586", + "phase/provenance/parent/calls/confirm-fresh-29-hi-m1-final-candidate.request.json": "509d68bc61283aa66df572590c4373b823c9c73b2afb0e2d1a973b3266b6b943", + "phase/provenance/parent/calls/confirm-fresh-29-te-m0-final-candidate.request.json": "8ebff4fbce1f2189c079b3abffa5a276585c7d69254222f0af30d0c30a583b99", + "phase/provenance/parent/calls/confirm-fresh-29-te-m1-final-candidate.request.json": "3829e65367a99bb6de595ac37a714838c81940db8d563e2b7c815765afa1e8ea", + "phase/provenance/parent/calls/confirm-fresh-30-en-m0-final-candidate.request.json": "67afc1c62b2bb21f6b33cd104c0b9968ace130d1fb0abce3b349aa0164e319fd", + "phase/provenance/parent/calls/confirm-fresh-30-en-m1-final-candidate.request.json": "5f00da0fdad09fbf7cea515bc09a6946fae157dc6dc7b6a640207e5e02ed6178", + "phase/provenance/parent/calls/confirm-fresh-30-hi-m0-final-candidate.request.json": "46a667557c7c9267af81a3c330645a7dab82364e21b5beb756aa0d11c6137139", + "phase/provenance/parent/calls/confirm-fresh-30-hi-m1-final-candidate.request.json": "9109b4d66d06e48281fd6a63a96c0725f09a72b5e375dc20d8d9ed83f006384c", + "phase/provenance/parent/calls/confirm-fresh-30-te-m0-final-candidate.request.json": "80a7121377dc5ec7d9119573b09443fa409a0dbe91a7662d6976162261c8803f", + "phase/provenance/parent/calls/confirm-fresh-30-te-m1-final-candidate.request.json": "dabec2a5bf04d8f21a1d304494925a60e449b0c16337410ae7b87c2d44c9549d", + "phase/provenance/parent/calls/confirm-fresh-31-en-m0-final-candidate.request.json": "a04776d17286c5c938fbe311f364dc1d795a411db2b9c62569845096426969be", + "phase/provenance/parent/calls/confirm-fresh-31-en-m1-final-candidate.request.json": "627fb41c3d8429e01ccd952e29d4e5b4037d18ec8db055db1a6217d985424085", + "phase/provenance/parent/calls/confirm-fresh-31-hi-m0-final-candidate.request.json": "4a697f107b79889562ab7d2ca98e09b34a4d9dcf602f4e4a3fd965353a11336f", + "phase/provenance/parent/calls/confirm-fresh-31-hi-m1-final-candidate.request.json": "382f073540dad54dbcd9dcd5d1c073d283195093adeb0ca255bc352559e21ffe", + "phase/provenance/parent/calls/confirm-fresh-31-te-m0-final-candidate.request.json": "7cc3ab556b72e2caba6bd107b0034dda2e86d5b9eccedf6ac4563c65479a85af", + "phase/provenance/parent/calls/confirm-fresh-31-te-m1-final-candidate.request.json": "f328152d6f223181b3d8d33a263a36cfb7f15bfb8d8bc29f0da217dcac972f0e", + "phase/provenance/parent/calls/confirm-fresh-32-en-m0-final-candidate.request.json": "51e791efe97a1cb18c0905a6ba845e75532ab84f97ed16b728ae7e6cdfdfe74d", + "phase/provenance/parent/calls/confirm-fresh-32-en-m1-final-candidate.request.json": "f307544dfe56c9316b81f5a149cba37146b79b2cb03fc58c70d5503dbff74078", + "phase/provenance/parent/calls/confirm-fresh-32-hi-m0-final-candidate.request.json": "e1126491fdaabf289ce38ba086685473da5a256215f9ad0a1837dbc595ab4b31", + "phase/provenance/parent/calls/confirm-fresh-32-hi-m1-final-candidate.request.json": "1725d05de9b64a086c19c1765ed04f136ad45a6ac05cc8aedc920083c50d32d0", + "phase/provenance/parent/calls/confirm-fresh-32-te-m0-final-candidate.request.json": "b1687c46b9db0e65fd1fdd49dd5160d7851b0df1be651d46321167419bd284cd", + "phase/provenance/parent/calls/confirm-fresh-32-te-m1-final-candidate.request.json": "e2a5dad98ef08ebc8d2237a443e9e9b21e30ba5315316c7c20f665f2fb66de9d", + "phase/provenance/parent/calls/confirm-fresh-33-en-m0-final-candidate.request.json": "9a0ebd41125098cba22b58178034ae7e9028bf2ed107e809f435adca5534c84e", + "phase/provenance/parent/calls/confirm-fresh-33-en-m1-final-candidate.request.json": "fa77a3017b1a709e9d659e25792968d754d72f494fbeb8540ca85d4ba6fd26e5", + "phase/provenance/parent/calls/confirm-fresh-33-hi-m0-final-candidate.request.json": "46b144ebd9f64173bfb385734c7cf9e60cfb7f2ae45b3a56ea393c0cf457b79b", + "phase/provenance/parent/calls/confirm-fresh-33-hi-m1-final-candidate.request.json": "afb7f29c89aa98e76e39f4f6056b53d3abc71e3539488185df51bb40879fb961", + "phase/provenance/parent/calls/confirm-fresh-33-te-m0-final-candidate.request.json": "9e2bd46faf6544aa25e59e31476ddd55e83df958ea11cb62def3953eba812216", + "phase/provenance/parent/calls/confirm-fresh-33-te-m1-final-candidate.request.json": "95340607a57631c0a39af6458109a812b64a91488a7e06560b93d518a8be5a98", + "phase/provenance/parent/calls/confirm-fresh-34-en-m0-final-candidate.request.json": "1f3b49c265ef8d4eb9ea6d70ebcfb071fa3d3f249449d899afb880b9afc34f80", + "phase/provenance/parent/calls/confirm-fresh-34-en-m1-final-candidate.request.json": "c28e980158231549c8d8797cd2e838e30eadbf381dc9a09748530c2fcb4a462f", + "phase/provenance/parent/calls/confirm-fresh-34-hi-m0-final-candidate.request.json": "7a014acbc54e2e271463523259279ba07cca6e74c9b93b0b8a74c9e44faa31ed", + "phase/provenance/parent/calls/confirm-fresh-34-hi-m1-final-candidate.request.json": "99fbfa6b488719c3ae19779b8bdbee21ecd688d6fc072c7b5f0526e7553490b1", + "phase/provenance/parent/calls/confirm-fresh-34-te-m0-final-candidate.request.json": "9dae47872e558faf149daf9f58134f430baedba16dbf9a487df344edbb200734", + "phase/provenance/parent/calls/confirm-fresh-34-te-m1-final-candidate.request.json": "ebc9bd1ea22cd0d76cfc9c6cb4563d1fd1693f921bf93cd81233664bde6f9cd9", + "phase/provenance/parent/calls/confirm-fresh-35-en-m0-final-candidate.request.json": "062837be2d526b0b1173c30ee8ea339e5f3dece7e101a866e2539fbf4289b6ec", + "phase/provenance/parent/calls/confirm-fresh-35-en-m1-final-candidate.request.json": "844a941aa0b4911be649ff652e8045fa2af89422976c3606a01efe980d04e133", + "phase/provenance/parent/calls/confirm-fresh-35-hi-m0-final-candidate.request.json": "669471d539910aa8a6d11a38239fa31e7d2aec9d5eda6e749f31a6117f4c3be7", + "phase/provenance/parent/calls/confirm-fresh-35-hi-m1-final-candidate.request.json": "96b140ed469ba88c0225b919d3a4bca0931769c72d0fddff542978f11ff4e320", + "phase/provenance/parent/calls/confirm-fresh-35-te-m0-final-candidate.request.json": "64fb176c14b64d30e7a93cfe85065d4ea1565ca54863ae91910350c4c7d13894", + "phase/provenance/parent/calls/confirm-fresh-35-te-m1-final-candidate.request.json": "a55f607aed12fcc822051577f696c2d108d9275c60209dd0cc0ea97828c6cb5c", + "phase/provenance/parent/calls/confirm-fresh-36-en-m0-final-candidate.request.json": "6c3a94574cedc097aa1a2bfbab97c46f4384575b7991ff7732bc93e83c1af5b2", + "phase/provenance/parent/calls/confirm-fresh-36-en-m1-final-candidate.request.json": "bbff49ee0a6a96178e6176d5c99a2b421062ab2fe4f3a1dbc2a74e6ab2ec8447", + "phase/provenance/parent/calls/confirm-fresh-36-hi-m0-final-candidate.request.json": "27dd0517508ebfaaebd3eb2bb4ef10bc9523fcda5173a094fbcc0908895559fd", + "phase/provenance/parent/calls/confirm-fresh-36-hi-m1-final-candidate.request.json": "bdac0c75fc5540668ba1f8529f53b67cfb606eea1d68a45d6b2fdbb0b1ea0e25", + "phase/provenance/parent/calls/confirm-fresh-36-te-m0-final-candidate.request.json": "d0c66204dc0b096691c568870b05b29ecfb4f5765ef860515bfc6bb7a32c678c", + "phase/provenance/parent/calls/confirm-fresh-36-te-m1-final-candidate.request.json": "feb3a887f3341e78332413a23ee07d18316e6540b1749e6addd965b2ce1f4670", + "phase/provenance/parent/calls/confirm-fresh-37-en-m0-final-candidate.request.json": "2a9c2c8b103ea14d2ef675b2533c7c9ca7fe4da82228750531398eda85b82d87", + "phase/provenance/parent/calls/confirm-fresh-37-en-m1-final-candidate.request.json": "87e6981313341bd32b47d37c49ce65ab93255dd5b15b38257f6b01afbba31db1", + "phase/provenance/parent/calls/confirm-fresh-37-hi-m0-final-candidate.request.json": "319ce8b670aecf1137fa41ce64a4661a33918a787ac6e754cba10b16791558f4", + "phase/provenance/parent/calls/confirm-fresh-37-hi-m1-final-candidate.request.json": "79a4846fe8db557fa711bc43be86c07ae138d42a5db7e5a04a72c14139afe5d9", + "phase/provenance/parent/calls/confirm-fresh-37-te-m0-final-candidate.request.json": "c6434d1358035b91dc25af5c9d501f19e9bfd00c82af938ad24a676ed892fed2", + "phase/provenance/parent/calls/confirm-fresh-37-te-m1-final-candidate.request.json": "eb7a4b4dfd33548c2fba2b77a1bbda5791f459eeda634a3f6f5d8e304dab5fac", + "phase/provenance/parent/calls/confirm-fresh-38-en-m0-final-candidate.request.json": "58c86896d328999f2bae892775e01dbe3bba2c96eb91fdceb7844a51745b186d", + "phase/provenance/parent/calls/confirm-fresh-38-en-m1-final-candidate.request.json": "62590d11978eedf1759bb08532fc16efd3dfefdfd064b0b3adadfc71b3a9d8a8", + "phase/provenance/parent/calls/confirm-fresh-38-hi-m0-final-candidate.request.json": "a417a9cd524575ab94af680abd45e3a86ed6ff3c707e3fdc0cf4d4707770fae0", + "phase/provenance/parent/calls/confirm-fresh-38-hi-m1-final-candidate.request.json": "98f38dc28191d44f9d49c224eb9028b7937ea2b444d99ef236d1b689859c209b", + "phase/provenance/parent/calls/confirm-fresh-38-te-m0-final-candidate.request.json": "1c2af5413cb407da4f8ef3b9480a335396fb2c75d8d7ee38af058870112d8115", + "phase/provenance/parent/calls/confirm-fresh-38-te-m1-final-candidate.request.json": "8fedf56cb73a1489724f51834cc492109c993ca729fc0c20bfbb67fb5acdaee1", + "phase/provenance/parent/calls/confirm-fresh-39-en-m0-final-candidate.request.json": "ccd9cc5d2d53d68577ff3a8c9684d47cbd6732bc0cede126be2a49a2b3bdf4f4", + "phase/provenance/parent/calls/confirm-fresh-39-en-m1-final-candidate.request.json": "b49d39bf609b4cd8b6b11ce06bd434615e30e85478de58fbdc3394d18cf0055d", + "phase/provenance/parent/calls/confirm-fresh-39-hi-m0-final-candidate.request.json": "12fc978caf3f5486671720223dc8ab2a78dab5dd1110aa9dc052c64190c70a68", + "phase/provenance/parent/calls/confirm-fresh-39-hi-m1-final-candidate.request.json": "69002458f5c611066bdeb1f4fb816d9cabaa4efbcebfe832b3ba02995e509f0f", + "phase/provenance/parent/calls/confirm-fresh-39-te-m0-final-candidate.request.json": "119452f2876e3e59862cfd270129c4c5dfb538b2a58a9b290e95cf926827ab3b", + "phase/provenance/parent/calls/confirm-fresh-39-te-m1-final-candidate.request.json": "31f8b310a6471fad0992566052cc09bfa89d715c855cef3cb9a9ebf44065e013", + "phase/provenance/parent/calls/confirm-fresh-40-en-m0-final-candidate.request.json": "f80ff1724ee493355653f60d44e8a27ab15c0dcec1447e3247ea9dc26c2ad676", + "phase/provenance/parent/calls/confirm-fresh-40-en-m1-final-candidate.request.json": "bb18133f93588884ecac4e234b0c682f825b46a119831baf3b7ef86f7424f777", + "phase/provenance/parent/calls/confirm-fresh-40-hi-m0-final-candidate.request.json": "0662e7bbc896c3ddce4650c170a50751069e1cce285cbc8e7636dd63ccae9e32", + "phase/provenance/parent/calls/confirm-fresh-40-hi-m1-final-candidate.request.json": "37f532b8528618f7bce1b0d4bfa5e627193ffb524cbad8cebb1f015a9eb5fb2e", + "phase/provenance/parent/calls/confirm-fresh-40-te-m0-final-candidate.request.json": "1e8376c2a74f9148d9bebf405ea169870ba3b8aff6ff984ba2e53438c93a5a80", + "phase/provenance/parent/calls/confirm-fresh-40-te-m1-final-candidate.request.json": "eec24c7f16e8941b9f0d38d1d25d3710ab40b08436f3e4ab48411bc6c043ec82", + "phase/provenance/parent/execution.json": "d2f3520b23676d1b7a1e4980ef12e59bafbd4afb8324f06aaf717bc0ba91f9f1", + "phase/provenance/parent/manifest.json": "2f96ece44c0821237e51c82e1309db86bcfce07501703d90f2098ba814b034a0", + "phase/provenance/parent/pacing-control.json": "f2222dc673bb7520bb37ab967b4f72a9103e53b172b214238bcf5f1faab0fb5c", + "phase/provenance/parent/paired/confirm-fresh-01-en-m0.json": "296e9404a430ef8821354942b819f495405ce66c268f57e70c67db83b5e114ca", + "phase/provenance/parent/paired/confirm-fresh-01-en-m1.json": "8e805103de4c9abf0a50c89b5090c9764020949b1d92f616de6e448b0f628f3e", + "phase/provenance/parent/paired/confirm-fresh-01-hi-m0.json": "eac6f052059df3a637f08a2d343c0df32c37c85d9951d678392a881e166b5a5f", + "phase/provenance/parent/paired/confirm-fresh-01-hi-m1.json": "6ee6530432f616acd02a52702154bddccba0dc60b2bbffc85684d4581093c421", + "phase/provenance/parent/paired/confirm-fresh-01-te-m0.json": "81967ac34d6935d3cade2f06215b2d8e8ba4345f6429f110be4b742b7b4dbee3", + "phase/provenance/parent/paired/confirm-fresh-01-te-m1.json": "27697bc40b63a176a28fad5ae272d8e30804db0bf7b9949c61bdbbf560bb6015", + "phase/provenance/parent/paired/confirm-fresh-02-en-m0.json": "7c79a03c7f5dd7bf0b349506fbf24fb4d9cc79d0defd1d39acc3bbca965d15c7", + "phase/provenance/parent/paired/confirm-fresh-02-en-m1.json": "a5b471e9f08e5935394678b8a2b2bac595297258249b843d8e217aa39f20d21c", + "phase/provenance/parent/paired/confirm-fresh-02-hi-m0.json": "da0665f4748130dd4c440d6a21c6857b34af6971883a600cfe0fc39f3bb6dc57", + "phase/provenance/parent/paired/confirm-fresh-02-hi-m1.json": "20a016b5414a0e0c96eb11686911ef831c5f149c8a753938c0ba6fd0f873da90", + "phase/provenance/parent/paired/confirm-fresh-02-te-m0.json": "87d03cb91d7511611f7c518df26a0b5aed10b5969280903a3e9508cc67413606", + "phase/provenance/parent/paired/confirm-fresh-02-te-m1.json": "b532fd8d89cda48e69e05a3ff85b2bfd22e0700a79956a707d111c155a0392b2", + "phase/provenance/parent/paired/confirm-fresh-03-en-m0.json": "d81d775174fba4a04e18136a4d5e875dde801d7d8a1f0ba5f02d1b955e844b8c", + "phase/provenance/parent/paired/confirm-fresh-03-en-m1.json": "8168ce5d94934b6ce3c591642918f8ea7c5905360089a488b3a37392e8c99a4a", + "phase/provenance/parent/paired/confirm-fresh-03-hi-m0.json": "adecc6c020cd9c8befac686e0c0f54a503282517e2f1f02ad8df109b24583e99", + "phase/provenance/parent/paired/confirm-fresh-03-hi-m1.json": "5de1e1c7108dbeffb573fc5e16b5cdcd908123e374dd06af62e1e315c7c8b4b1", + "phase/provenance/parent/paired/confirm-fresh-03-te-m0.json": "bbb8372cbf60c5959a0ed41636f02bdfcdeec7c766ee14e874d6615b8f870fc7", + "phase/provenance/parent/paired/confirm-fresh-03-te-m1.json": "7a01c4f44b768114fce45ff427b99b583eae6291fb564f78cf0b9340e4fd12fe", + "phase/provenance/parent/paired/confirm-fresh-04-en-m0.json": "ca1a0bd63880b37cd1ef8bb3c3d526b4ca10607b6a5580287ae9172687583537", + "phase/provenance/parent/paired/confirm-fresh-04-en-m1.json": "4f1355d04732812504b71d3528777cfe3500ae4147302017c2806717e35d7206", + "phase/provenance/parent/paired/confirm-fresh-04-hi-m0.json": "cccd09f85c8123a48386d127c8470e79e5911892a2274b067998152bd6e15aad", + "phase/provenance/parent/paired/confirm-fresh-04-hi-m1.json": "4eed3838bcdbcd96ff121b5bfa2ec0a54a00ab483d2b5d52dc63c4ec58f34949", + "phase/provenance/parent/paired/confirm-fresh-04-te-m0.json": "14fbc4af10e3d6a8aa9bbbc4117d6e538820a175b8c2a0a11715014a2959c78f", + "phase/provenance/parent/paired/confirm-fresh-04-te-m1.json": "cbb3e6e032d97de41a251d9d5d348e9b82b063f6f87b5fface8c32ced0037712", + "phase/provenance/parent/paired/confirm-fresh-05-en-m0.json": "3b11eb2482a5093caa22c1e4f3821b86204f83a649dc2f976a8c1e2187ab0678", + "phase/provenance/parent/paired/confirm-fresh-05-en-m1.json": "6de67458b42902d75b52d7a206f1c0a41ae2a0f83306dc77d152ec5101d806ec", + "phase/provenance/parent/paired/confirm-fresh-05-hi-m0.json": "fa79db090dd7954b9ff93edfdef8098b879f11f9204478d5343258569ee603f8", + "phase/provenance/parent/paired/confirm-fresh-05-hi-m1.json": "bdffcded718c73d664aaad5ca0d205c8f04fcaf343dd69d3d25a3b0cb0baa9d6", + "phase/provenance/parent/paired/confirm-fresh-05-te-m0.json": "ea40749a40463d7b21e82127648a05bdf48f289fc293d46b882633f7ae91757b", + "phase/provenance/parent/paired/confirm-fresh-05-te-m1.json": "8776494bfd39dd41de5fe1b4bc6c71604964d6f0e096d3e7d7aa19915fe073f8", + "phase/provenance/parent/paired/confirm-fresh-06-en-m0.json": "82f5414c571b69dfc3d34c9485c1cf072e018bf96dc3d2d21443d169e6e9f8d0", + "phase/provenance/parent/paired/confirm-fresh-06-en-m1.json": "ea9bef72f3bc630104f2b2836dccdf7dfd083c661e94e381c300957989d94bc2", + "phase/provenance/parent/paired/confirm-fresh-06-hi-m0.json": "12e393d97674a12400914aa0c21cc1a10e4d684c5e83cc8c217c290564ed9524", + "phase/provenance/parent/paired/confirm-fresh-06-hi-m1.json": "d19d546bbe93b9ee5e5872bf3cc311ffc47418c0df26151b063a3a25a3ce4395", + "phase/provenance/parent/paired/confirm-fresh-06-te-m0.json": "541848996f56f280f7085b1caaf9268176680fde806debcac4d6936f77496cf5", + "phase/provenance/parent/paired/confirm-fresh-06-te-m1.json": "216e18e06f93071b33e6715356da4441e77a5447084f64da5e1405995fe61014", + "phase/provenance/parent/paired/confirm-fresh-07-en-m0.json": "efb755570c6d2534d5576e8a4a36af7937e80e86d75ebf5710cacfdf7159a746", + "phase/provenance/parent/paired/confirm-fresh-07-en-m1.json": "91a6edffdd3656797b529da00bad42736d5a5a114414bac9680f61de795c7806", + "phase/provenance/parent/paired/confirm-fresh-07-hi-m0.json": "c8fec501525b23dbc527c90169c3df502f7852b206e563974728dd851ab180e7", + "phase/provenance/parent/paired/confirm-fresh-07-hi-m1.json": "0b5f6387be064fda44cce858c07a7549b0f534e4fb753a7719a34e32ef2bbed8", + "phase/provenance/parent/paired/confirm-fresh-07-te-m0.json": "0161bc225cd956dd178a3e58c1573a53fbfd57fc6c33391e6b6c59d28359ed11", + "phase/provenance/parent/paired/confirm-fresh-07-te-m1.json": "4712b888f147828ffc56924838cd123ac0e8a9d2fe15b9791a2f64a2e2463fbd", + "phase/provenance/parent/paired/confirm-fresh-08-en-m0.json": "389acdd678c40e89c3831072055a761fd7cc81cba3d26b52b30b836977a114e9", + "phase/provenance/parent/paired/confirm-fresh-08-en-m1.json": "42ebfcf03d238ef8e48a7edad5468687c727f0352c462795cfac542f236e720c", + "phase/provenance/parent/paired/confirm-fresh-08-hi-m0.json": "e1032a1e765820fd42f68c926f9c87a9252e91c232eccafab4b4c76029624dba", + "phase/provenance/parent/paired/confirm-fresh-08-hi-m1.json": "d8c326bce5600dae15a663d83dc9146fa3d2b3c6fcbe54c0cb0340a600b98cfd", + "phase/provenance/parent/paired/confirm-fresh-08-te-m0.json": "9e28ac351738517054f2f19938181f5434d802c65d9634a2d0354aab02a9f140", + "phase/provenance/parent/paired/confirm-fresh-08-te-m1.json": "b75b650d6f2a2fb7108a342fb98da5bd35c52b4479d6d2797c8dd12910e1f4a3", + "phase/provenance/parent/paired/confirm-fresh-09-en-m0.json": "d46a240339f6cd402a0ab449ae295bb4e94e7772a4bd15e3099c133983f9fbe2", + "phase/provenance/parent/paired/confirm-fresh-09-en-m1.json": "abee0fd1852a6bbab5bdf36bd899d7756f67bab2742ea071e91196eae361710d", + "phase/provenance/parent/paired/confirm-fresh-09-hi-m0.json": "ac2db8746c86ee3f534a4ebef19db3e77a39b9f303363ae909ed42c30b40ab82", + "phase/provenance/parent/paired/confirm-fresh-09-hi-m1.json": "817b5d8d345f1124fb35cc642ab25df8f29beee2129e8be78f950e8a74f4c31a", + "phase/provenance/parent/paired/confirm-fresh-09-te-m0.json": "54c37ff13d1adecbbfce81b668cdb6664b9d3200eb1e228b4654349531317508", + "phase/provenance/parent/paired/confirm-fresh-09-te-m1.json": "2f1cd6f1ac16ff9772bf9c48c1937f7dcbf61a9a597520f68e900f1176df6881", + "phase/provenance/parent/paired/confirm-fresh-10-en-m0.json": "45505a6bc992094c2ff2030030ff546801ab7c58143fe5abfee927e25c5b00c1", + "phase/provenance/parent/paired/confirm-fresh-10-en-m1.json": "1e43b51548110a588474f65b6b243b3d550dc5a16a58bffd19b9d586bb6c7ca3", + "phase/provenance/parent/paired/confirm-fresh-10-hi-m0.json": "1c808ae411cbec25fa8ea551e75128ba596f354d83a585cd32a39fe551bdbbae", + "phase/provenance/parent/paired/confirm-fresh-10-hi-m1.json": "9e50714179701725c034a38c9f63431de4684f991c381c1dcbdb746377a8529e", + "phase/provenance/parent/paired/confirm-fresh-10-te-m0.json": "92e9c5ea46e4b652f48842a296cecaccb5a5800ff979c56d08e31195c5f74999", + "phase/provenance/parent/paired/confirm-fresh-10-te-m1.json": "b920ad62cebdd47993c243e5fbca97c7593152d81b0dffbf8b9e53844451a351", + "phase/provenance/parent/paired/confirm-fresh-11-en-m0.json": "f26b15c7e7df53d70084fa5acf01a41f8d293fa957b1890efd52b067222e1484", + "phase/provenance/parent/paired/confirm-fresh-11-en-m1.json": "17d09767fb0043179e9e978c50230f436a0517c5e0731f2582063a3cc89f7e80", + "phase/provenance/parent/paired/confirm-fresh-11-hi-m0.json": "40668e9509a71199ab01a188be47db72d61462d151b74e2ad4d3264b8284bae7", + "phase/provenance/parent/paired/confirm-fresh-11-hi-m1.json": "0c297a77c1c89e33e205ed98704b1c778a321d0767c39371a05e702e03a7d097", + "phase/provenance/parent/paired/confirm-fresh-11-te-m0.json": "ffd085cd3ab9d2485dfc44e7e9aa312bd20bed4fcd47bade404c603a66492638", + "phase/provenance/parent/paired/confirm-fresh-11-te-m1.json": "2d10d135c16f663a0d63723e081d6938cea803021d8e088f07c71100b7436623", + "phase/provenance/parent/paired/confirm-fresh-12-en-m0.json": "791e9e59e3c5adc42cd5a5fc70dd49ec22d24b7289afbbe6f85487d83fb0be9e", + "phase/provenance/parent/paired/confirm-fresh-12-en-m1.json": "565d9941f41642e3ab4809d4fb921e376ffc9484b98417e3f2970b17774da903", + "phase/provenance/parent/paired/confirm-fresh-12-hi-m0.json": "4560d8691234f72124216adc2a51578376e9ff83c04f9b3d18bda920ff72b42e", + "phase/provenance/parent/paired/confirm-fresh-12-hi-m1.json": "cf59821db412651e78be796c1d75ef31fa8fe20e4f83573f4cac906ee486abe8", + "phase/provenance/parent/paired/confirm-fresh-12-te-m0.json": "025bbd9f6562dae7a7fce39e7d9ba7a2855b01c44da2c7f10449df043232e560", + "phase/provenance/parent/paired/confirm-fresh-12-te-m1.json": "5b41edf8dd240b58f6d7a3b18e6c8882820d681d903c73cea64890a5715ed536", + "phase/provenance/parent/paired/confirm-fresh-13-en-m0.json": "08b9ca6b1459174ff91bebe6608f7ed7c5458dd096e6e782c94cdf76d142f577", + "phase/provenance/parent/paired/confirm-fresh-13-en-m1.json": "2aa30c4cfc92149eadca407daced7acd8fbf7c7c1340c42c54c72521adffcb0e", + "phase/provenance/parent/paired/confirm-fresh-13-hi-m0.json": "11396c58f7d61c1d517783a6fb8bf5871e737bcdb15d478dc63a7e4e42320bf3", + "phase/provenance/parent/paired/confirm-fresh-13-hi-m1.json": "d7483f617d50efd3c2bad7b3bdbbc6bf091b1342ac96e3bf7a3485aba44bbd92", + "phase/provenance/parent/paired/confirm-fresh-13-te-m0.json": "926436042ded86c8bd776fc62124703f430dc72de1c3c6e9edf07130c654b31a", + "phase/provenance/parent/paired/confirm-fresh-13-te-m1.json": "82ca591d24c72634da60a2ce00055c9b59226d6406254284c6ca633d5dba5ee7", + "phase/provenance/parent/paired/confirm-fresh-14-en-m0.json": "8fdb1cafd70a43721a69e7f289590b13cb803f3b279264c7f87e942a93af76e3", + "phase/provenance/parent/paired/confirm-fresh-14-en-m1.json": "d0aae108a82043a6071af5dc8b1e3e92eddb7bfcafd7121179fa1c804c01fdcd", + "phase/provenance/parent/paired/confirm-fresh-14-hi-m0.json": "a16f3989b2882ef9edfaba37e45d6418ceff339a80f23111c1e8e3fd85d73675", + "phase/provenance/parent/paired/confirm-fresh-14-hi-m1.json": "d9ff356da53df03e1d026dd922916178668afc469cf139f2fcd0277bd6eba957", + "phase/provenance/parent/paired/confirm-fresh-14-te-m0.json": "d1f1c448052b17a546678d98e958ac035dd53b5b6d51137d634cdb7a3422c462", + "phase/provenance/parent/paired/confirm-fresh-14-te-m1.json": "e0a5366d0fa966cc56834480f4db507d600ab467703a56e388bf5f9a30f8b0df", + "phase/provenance/parent/paired/confirm-fresh-15-en-m0.json": "ed64fdb8bfc3cc806e488231772bd7ccd9f19a79231817bc728f3e62fef72837", + "phase/provenance/parent/paired/confirm-fresh-15-en-m1.json": "0aaf42e91e796b2f5397399e442665d0dbfba256693fd974785bae8d3445a4f3", + "phase/provenance/parent/paired/confirm-fresh-15-hi-m0.json": "c0aa8b7a3af1edd4ed4b07693eb2e8f979bad27ece3987b6350f7a7ad66b2fab", + "phase/provenance/parent/paired/confirm-fresh-15-hi-m1.json": "e74873347961b3b4abda92dd6132941616a3b4de4949c38ff668d6d44d9d6232", + "phase/provenance/parent/paired/confirm-fresh-15-te-m0.json": "ad86ae10ecbc3ac8da03fe2ac916951b7a645d7d98404b148660cd856694607b", + "phase/provenance/parent/paired/confirm-fresh-15-te-m1.json": "c3ca09cd6289b47a6e3a6ba2ac60defb911b4072f3b15aaf3081e2c8e66f747d", + "phase/provenance/parent/paired/confirm-fresh-16-en-m0.json": "5fb3f8416cfbaa62232ddbdc031cbcda3890ec174dc7fdaf76876bbe51f92884", + "phase/provenance/parent/paired/confirm-fresh-16-en-m1.json": "29d79d4d34e3dbfc7669ad0a92cdf83eabefc80e5ee2269bc81959c54592aeaa", + "phase/provenance/parent/paired/confirm-fresh-16-hi-m0.json": "e7a4af83c2acb9b3d39780bc56949f058979799d35bb458ce6ead7f26dd5ce60", + "phase/provenance/parent/paired/confirm-fresh-16-hi-m1.json": "420eabafefa7c1a72623d73c1a341337f07839ac2886b3ac2a7ee72168a84cdb", + "phase/provenance/parent/paired/confirm-fresh-16-te-m0.json": "3d9c1284faf81c4beffcff2cba76ca4bb5eddebc198a52559b4354ca70ba5ca0", + "phase/provenance/parent/paired/confirm-fresh-16-te-m1.json": "506a004ac7994a6e88589075b5c0061eecd0a514748f88af7c2a79d8f1a1e153", + "phase/provenance/parent/paired/confirm-fresh-17-en-m0.json": "bcb79701178854c7073cba63c6c32cab02e3c3816ea148ad34eb1f118ba99460", + "phase/provenance/parent/paired/confirm-fresh-17-en-m1.json": "3e60cb693925ea63e0317232878bdf01b697c2993858ac3c865341072ff529eb", + "phase/provenance/parent/paired/confirm-fresh-17-hi-m0.json": "8c17fe6ba97f972ebe8646fd1fbc81d478544843bc334ace7077f151a9c39c31", + "phase/provenance/parent/paired/confirm-fresh-17-hi-m1.json": "2714c2166e9bbed46c1667568829602feb5c2799f407b39b74587968c2400cec", + "phase/provenance/parent/paired/confirm-fresh-17-te-m0.json": "c8a153908d4acf0f3d87c3a8468ee6416b56c8acda42405ed5170dc05fc6312e", + "phase/provenance/parent/paired/confirm-fresh-17-te-m1.json": "82f94708c28750961ecb615fc16daadfeb277e24ac983bcc0cb2b8e449c7983d", + "phase/provenance/parent/paired/confirm-fresh-18-en-m0.json": "31586bf2d845e51f121c584a83869aca92e12866b373e548aee9dbf6161fdd8a", + "phase/provenance/parent/paired/confirm-fresh-18-en-m1.json": "16f1865483ac5af053cd14a7aa60feccdf21bbb306626a331681e924fb40ba87", + "phase/provenance/parent/paired/confirm-fresh-18-hi-m0.json": "0bfefcb3f26573d930dbf769b2cc2a2fc77b864b70e67a9b8772e2767568d1bd", + "phase/provenance/parent/paired/confirm-fresh-18-hi-m1.json": "66e1909a00c9c5e9fb3cd6a08766c1a88655144488bfceafafd6637131b1aec9", + "phase/provenance/parent/paired/confirm-fresh-18-te-m0.json": "0ce7e7623a238a42130692241327e2b75960189a20904cb9a09e10fd5515f074", + "phase/provenance/parent/paired/confirm-fresh-18-te-m1.json": "dff88f66389457c11edc0f8a1722ac6adb304dd2ab2cdc6d0c9f35469ab066f8", + "phase/provenance/parent/paired/confirm-fresh-19-en-m0.json": "8157342cc3a26eaee4f6dbf9230aa41a77d7e0386d4786edec6eab13930bb182", + "phase/provenance/parent/paired/confirm-fresh-19-en-m1.json": "a7c7cde8f9a29fba168ba30ffd914428617149431514a2d25eedc826d41bb952", + "phase/provenance/parent/paired/confirm-fresh-19-hi-m0.json": "977af102af847f78c10756ec0c3f59ee4d084d54e6b725555197a36cd3b7c652", + "phase/provenance/parent/paired/confirm-fresh-19-hi-m1.json": "11d5cc61760707ca97d3b87e67b5150c0f7d13085304c23805dad6d6f70d055e", + "phase/provenance/parent/paired/confirm-fresh-19-te-m0.json": "5c994ee1a58f5bd77a536ead5b956d6665c3bcdfac7c01e7aa6371787ea729ab", + "phase/provenance/parent/paired/confirm-fresh-19-te-m1.json": "457a13df33dd19849d2232da525afc8132c0a4c0b1882319dbda9b5556ae2ff5", + "phase/provenance/parent/paired/confirm-fresh-20-en-m0.json": "39bce7516858de0e6c386dbbe5ef2667df5dda56bbe1911aa9a1d4fa8e094c72", + "phase/provenance/parent/paired/confirm-fresh-20-en-m1.json": "0839fbddbb6618585535dffd0333cabd95942e2e81dc8562552b0dea574332c7", + "phase/provenance/parent/paired/confirm-fresh-20-hi-m0.json": "0175b7fbc7ba91b926622a0d9f5ab61142763618970c52b499ae62b903c6fba3", + "phase/provenance/parent/paired/confirm-fresh-20-hi-m1.json": "7c9a1a82b47c86c26d9dd958d36406740fc5785c423b4e7c57cebc997bdf3903", + "phase/provenance/parent/paired/confirm-fresh-20-te-m0.json": "8061098c20a896a51b6e4eda4253de1b71cde7abe8912ffa53edcf901c91598f", + "phase/provenance/parent/paired/confirm-fresh-20-te-m1.json": "8f912f24131412b91ea088e05f9cc9c7b627dd5b64fb0fe04e9aea5f30e21a7b", + "phase/provenance/parent/paired/confirm-fresh-21-en-m0.json": "a8d53c8c9e64768142af75cc1ab9249b5367d242f3cf0b1a1631d7f2c2562af4", + "phase/provenance/parent/paired/confirm-fresh-21-en-m1.json": "9bef4971299653337fe6b01b2157aad8e5dda67415da70f9424f04e9edcf2cc0", + "phase/provenance/parent/paired/confirm-fresh-21-hi-m0.json": "89e93d838dc2ea1cb52f2d3d66e5315818028ca5e8cb46f21f74dff73e9e79c3", + "phase/provenance/parent/paired/confirm-fresh-21-hi-m1.json": "a19eb67c396409666d1324c77eda889b80dec2175a77bb444c723dd10b6cc7e8", + "phase/provenance/parent/paired/confirm-fresh-21-te-m0.json": "0845b2c0b520d98983ee2fc5a900f5e2c873f24d13a3ebc50ed54d65b07edbf9", + "phase/provenance/parent/paired/confirm-fresh-21-te-m1.json": "fd2ad07ae33c3a16c94a6bae4b1172ed3769d248e70d3605b8ab1bbaf625dcb3", + "phase/provenance/parent/paired/confirm-fresh-22-en-m0.json": "bd527c65b844626442ef647542b7c328c567e14da47ea1fbe3f7688348ddd3ad", + "phase/provenance/parent/paired/confirm-fresh-22-en-m1.json": "b90ac8a977eb56c44b2bdcf2b792ed1a8c57e27266a5b4d4c7801b7003d45fad", + "phase/provenance/parent/paired/confirm-fresh-22-hi-m0.json": "29ae8ee3730cbd511f4161249514a2ccd67927fa0fba2b75775ed8a15d318203", + "phase/provenance/parent/paired/confirm-fresh-22-hi-m1.json": "1f4471c284b280fd69c6d301ba10f8dbc42b23a193e4a067bc834294799bece8", + "phase/provenance/parent/paired/confirm-fresh-22-te-m0.json": "89c3c45d1f7e197e8637b2ed6d52c3b0e30d03ed800783885d563cc56b9e8b09", + "phase/provenance/parent/paired/confirm-fresh-22-te-m1.json": "ee06e99dd0cadb034a109c9e99102d5dbc6ca9665c37ef0c04935d24f3425b0c", + "phase/provenance/parent/paired/confirm-fresh-23-en-m0.json": "a2e3a7e05bcc5815be671e871d1ad629792f26a5ebfa82281dbf3fa4df85cdfc", + "phase/provenance/parent/paired/confirm-fresh-23-en-m1.json": "1275867d410a9327cee5119e6bf3996c2859447424a6d18284d037bab3000c0c", + "phase/provenance/parent/paired/confirm-fresh-23-hi-m0.json": "c70aac130770eb52c4d22cb14a12c328b138afed88f6754b81d40ba4a2ab1bcd", + "phase/provenance/parent/paired/confirm-fresh-23-hi-m1.json": "fa867b061e3c10ac35a811a61c3032f4333b8e021fad26a0dfd7a21b4f37e9f8", + "phase/provenance/parent/paired/confirm-fresh-23-te-m0.json": "472ba5ef0ea7458c923a43e7aa229319c1c4ea70cf2671d1acf2379021be85c4", + "phase/provenance/parent/paired/confirm-fresh-23-te-m1.json": "8e918654cdd40efdccf0304f1e69959b4f657d3c684faaa47ab96f058f432b0e", + "phase/provenance/parent/paired/confirm-fresh-24-en-m0.json": "9694a63cad38fa21c292671766a2f3825b27c25467fae3a512a2b0cd5512e2f7", + "phase/provenance/parent/paired/confirm-fresh-24-en-m1.json": "5aa801347bd329457f18c6d69a93aa83bcbffc17e4732072fd767fab33cc5085", + "phase/provenance/parent/paired/confirm-fresh-24-hi-m0.json": "57b4f614ebe26faf73591eeeee59160526776cfc0da3824b5bd90b007d8196e4", + "phase/provenance/parent/paired/confirm-fresh-24-hi-m1.json": "02ed3dda4b169680d1775e51c7ba1f67a56747a00b78452414bb58c038b87683", + "phase/provenance/parent/paired/confirm-fresh-24-te-m0.json": "9a8791569564b796c42878a05801f082be380878474a02ca518a81e3f8d48d9c", + "phase/provenance/parent/paired/confirm-fresh-24-te-m1.json": "26e3cdc8ea6a01b1aafb83de22f217b769c062baf978960a3018fece1fe1f89a", + "phase/provenance/parent/paired/confirm-fresh-25-en-m0.json": "1d3406aeaeafcaaa103222011e9e03f92fa9c6b1aef8d45a62b1ca8ac1ba9db9", + "phase/provenance/parent/paired/confirm-fresh-25-en-m1.json": "70e71fd9b38c6d2af918841ab6ca087f9f7dacbfe315e65314da2ef3e3e53297", + "phase/provenance/parent/paired/confirm-fresh-25-hi-m0.json": "5bdbcfe0efa64c426004c0ee78c628b020073bae373405c0252781fd7526260a", + "phase/provenance/parent/paired/confirm-fresh-25-hi-m1.json": "0b140d2c6d070b4c7bcda3049e1f830abd399f7b54e762017631330a769c15ad", + "phase/provenance/parent/paired/confirm-fresh-25-te-m0.json": "8164a13385db57b850010af782a33a3102e4c55b44b247c25a2a5e9458244b50", + "phase/provenance/parent/paired/confirm-fresh-25-te-m1.json": "34ccefae3e3903722517237cefebfcec74774539188d2c9181b966cee4f574d1", + "phase/provenance/parent/paired/confirm-fresh-26-en-m0.json": "c771cbd30afb7dcd4b66105996b069e57adc04e36f2f23539371682c7d41f754", + "phase/provenance/parent/paired/confirm-fresh-26-en-m1.json": "332bbc69b5584440b296370f054abcf8a7eefdaa1f8a5bfc6a2c2cdaa72f70c4", + "phase/provenance/parent/paired/confirm-fresh-26-hi-m0.json": "c9c7d974f984ab842b03118877662b9e3bb49afcfd8fd50488c2ada7b82ec253", + "phase/provenance/parent/paired/confirm-fresh-26-hi-m1.json": "c9d3c2b37e00f2a6d24c5560f7074647029327b0667f6e3b156cd680d2b97280", + "phase/provenance/parent/paired/confirm-fresh-26-te-m0.json": "a245e1c3c2721d64e0e94ad0119877467a0551587bdbeec2663076514a3fba39", + "phase/provenance/parent/paired/confirm-fresh-26-te-m1.json": "3a201c1f249aa3a02a61d9943329e935f5bfeb2076ed8d8f9e6e58f93ee9367b", + "phase/provenance/parent/paired/confirm-fresh-27-en-m0.json": "d0af014d12735ff1f91327289b726ba14d84e5f5c2ae58319ae59c6a40ae1f69", + "phase/provenance/parent/paired/confirm-fresh-27-en-m1.json": "f3d6e03b48887ef14313a5c0360bf809c1bffaab7dd3d6b4e5ecb4b56c70fb07", + "phase/provenance/parent/paired/confirm-fresh-27-hi-m0.json": "fd90c926faf364acdce1e083706f3415ba34154a6384343ded9bc4374cec90e9", + "phase/provenance/parent/paired/confirm-fresh-27-hi-m1.json": "d3e90d6ee95a0b038718615f0bf2987635dc5009253cdcd2c743ff50ae8ff0e7", + "phase/provenance/parent/paired/confirm-fresh-27-te-m0.json": "85b9b65bed2dcb9016db5cbbc83cc2749bc6b080078851654663adc7555184d0", + "phase/provenance/parent/paired/confirm-fresh-27-te-m1.json": "fe863445d6c241597cb2a8c4da24cae418c1b6d55dd7e39df5474a0164e10037", + "phase/provenance/parent/paired/confirm-fresh-28-en-m0.json": "1a866427f1dfd40272107f0c561df606b8ae9dfeca054e1d549443499e9d05c6", + "phase/provenance/parent/paired/confirm-fresh-28-en-m1.json": "42ac6652a04e5f675197f84f2160466e7bd7e28f8cac8c1fddedaf2632b513e3", + "phase/provenance/parent/paired/confirm-fresh-28-hi-m0.json": "1b1d2dd2e8bbe75adef9964cd0f15fe86f8597cf4f6bb81b85f491dbb926e42c", + "phase/provenance/parent/paired/confirm-fresh-28-hi-m1.json": "4b4b1a57d7c98bc318c303a4df4477f984dc7bf1b596ecc7a063369af070b5fd", + "phase/provenance/parent/paired/confirm-fresh-28-te-m0.json": "72dee5d0398e6186b16633b8a731faa6e6eebe2bc020ae8157288b6966d730d7", + "phase/provenance/parent/paired/confirm-fresh-28-te-m1.json": "41cfd00ce8d834eb4913c2fce96a57c8e7860577340d96d6084fb0ac441e84c0", + "phase/provenance/parent/paired/confirm-fresh-29-en-m0.json": "66821b07532d10d5ffbf361a56175898fb1d41d80d5aa42cff338ffe9ac2e0b9", + "phase/provenance/parent/paired/confirm-fresh-29-en-m1.json": "45fc351ae66321acf2134a761b237198011c40a5aca0b894a0c487d28201d884", + "phase/provenance/parent/paired/confirm-fresh-29-hi-m0.json": "86f134c3321d2b94d9b801dc0dd689bf651211e4b7ccf6cdf41f4015d8288fa4", + "phase/provenance/parent/paired/confirm-fresh-29-hi-m1.json": "486480143628f58da2d6e11a0f0ba448b9deb8996134492661d009d3a33ab97f", + "phase/provenance/parent/paired/confirm-fresh-29-te-m0.json": "c4f6588d46648e49a5af34345c752ff17dc78612e2181645618c835c7f5cd809", + "phase/provenance/parent/paired/confirm-fresh-29-te-m1.json": "ef2e96d07a161f8aefe3f23c9cc693d7c04f0a4ac1afc033cc0d8a0ccb2eb4db", + "phase/provenance/parent/paired/confirm-fresh-30-en-m0.json": "4927236eee7a4eb0e21204ab3c35bb614cabd333ccec11981405d8b51e024b2a", + "phase/provenance/parent/paired/confirm-fresh-30-en-m1.json": "291f49aa5c1d1e3ea484f67af7581839f3ce6d336e034ffade0ac142e71a843e", + "phase/provenance/parent/paired/confirm-fresh-30-hi-m0.json": "0f79cd0bbb1c936c84f5ad1494bc2c146e08f2076ebc6f4c9a6e5f607dc24ad0", + "phase/provenance/parent/paired/confirm-fresh-30-hi-m1.json": "243099bbb914ba5225b4c03eabad5206511f9b239362429d5a490b13d2f0ae74", + "phase/provenance/parent/paired/confirm-fresh-30-te-m0.json": "e86db2ec8327a7099054adf79d1984e5d87e70f4e4c5062adaf9210a4996f130", + "phase/provenance/parent/paired/confirm-fresh-30-te-m1.json": "4d90c819dd385551de4cd4a31f6e632acebf5c1982b5fc5ad2f11d4f6f7fa270", + "phase/provenance/parent/paired/confirm-fresh-31-en-m0.json": "6927ff78c0187e6d078d0402da84dc77e78b3dcbb71dcddf63befcaf1693eddc", + "phase/provenance/parent/paired/confirm-fresh-31-en-m1.json": "13d6017aa0c887dab248f46ae189972d27b74b9c63c07d098d85e0e422127563", + "phase/provenance/parent/paired/confirm-fresh-31-hi-m0.json": "d0ccfe3223f5fbcfe332f856b5d79b1adc80808c79ea32d499b4510747cc5f27", + "phase/provenance/parent/paired/confirm-fresh-31-hi-m1.json": "57a94f82e5b369727d864dfbddae8369cb02c2447c1734b903be5568c4ef81c0", + "phase/provenance/parent/paired/confirm-fresh-31-te-m0.json": "3198992206bd25cdd77b86243dcf3f9ad15ece6a6db10731069a6246635a3997", + "phase/provenance/parent/paired/confirm-fresh-31-te-m1.json": "a38da3525d7d566751e113920c2b2e832ffad2022ac7ae346f144e6c18554355", + "phase/provenance/parent/paired/confirm-fresh-32-en-m0.json": "a47b76fb68b1242d84cba94b61c8bcb992f38339fc4a59aa180285b1cbbd2964", + "phase/provenance/parent/paired/confirm-fresh-32-en-m1.json": "00f6d1d09307d6909d26a53408ae77c1e59c29a0594f086dbb08cc71b2701550", + "phase/provenance/parent/paired/confirm-fresh-32-hi-m0.json": "9bfbc81621aa2c13582ef5f75bbf809c08d425a0a97e9cd6c34671946a19fa01", + "phase/provenance/parent/paired/confirm-fresh-32-hi-m1.json": "e0dac1ea9e13cd763facc50020471a099df61e7d85fe95b30e5c7b7ec595c51c", + "phase/provenance/parent/paired/confirm-fresh-32-te-m0.json": "ca53f9672230dd53fd63c2a758ab50179ef602b9b331df435aaa263eab1816af", + "phase/provenance/parent/paired/confirm-fresh-32-te-m1.json": "dfd315d13675a52e9268ba74bfe7e3f87f316047c14efbc9e2b53f52c84cd6d2", + "phase/provenance/parent/paired/confirm-fresh-33-en-m0.json": "ecbb68c6394ad41e332e96859356bfc56134c257f1f493c215285c1a6e6c5376", + "phase/provenance/parent/paired/confirm-fresh-33-en-m1.json": "10d30671b6bc38a2541f071af9d1ba19c44805b6da071482aa13319e00ba1d8d", + "phase/provenance/parent/paired/confirm-fresh-33-hi-m0.json": "1306c78c64c7925d09c24ef93058619ea3c5bbb28da8a496ee946ec55788fe70", + "phase/provenance/parent/paired/confirm-fresh-33-hi-m1.json": "6bcd5a039d69eff0138a3f43b3d7a8c730b3bb16b34ad5b78e7856e0b17cf28c", + "phase/provenance/parent/paired/confirm-fresh-33-te-m0.json": "03f68cce95206c21951769f89b2705e730d78f1bb1e00019183c0fabe74e2e59", + "phase/provenance/parent/paired/confirm-fresh-33-te-m1.json": "210b887c85df4bf199b15cab6cdf525c47bd82cca8377c0fc7c85fc1cc4c3f9b", + "phase/provenance/parent/paired/confirm-fresh-34-en-m0.json": "dd70397626181499d0d1803db7df5e13718760ff908c529ba54888b70de3150f", + "phase/provenance/parent/paired/confirm-fresh-34-en-m1.json": "9f19453a6df04efaa6183bc32326b45eab38c4745f5ccb46126fb0bd84ee5ae8", + "phase/provenance/parent/paired/confirm-fresh-34-hi-m0.json": "98a60ae9cd834dd973718079b9495fb0cdff4122b7fec9f35d178cde1b4a6f22", + "phase/provenance/parent/paired/confirm-fresh-34-hi-m1.json": "68feef9dba8f08ff486be48ad6edc96dd283b8d0225a4ffd03c721e6e526ee41", + "phase/provenance/parent/paired/confirm-fresh-34-te-m0.json": "8dd0bb5a1995db5035fe4c4e7faa9aea46e07bba701d1af6c54ebef9117165a8", + "phase/provenance/parent/paired/confirm-fresh-34-te-m1.json": "4e688f86e99ca39a217396abf96dd9f8dc9c07a7566c83d39bb1542e4f002376", + "phase/provenance/parent/paired/confirm-fresh-35-en-m0.json": "26990911e1f4591f0fc3d552289bccb5494683441dc5a1157118255772e0c1d3", + "phase/provenance/parent/paired/confirm-fresh-35-en-m1.json": "acb29cdbc1d77426485c1f047b0e9ca5314dd7bec51b3ada81e851e10cd07e62", + "phase/provenance/parent/paired/confirm-fresh-35-hi-m0.json": "0e936c02db942c903026db06b60bc24fedf0a04bd14c83c49584d9b5ab87675c", + "phase/provenance/parent/paired/confirm-fresh-35-hi-m1.json": "d3e4e3d84999cf308fb98e03579dc84f000bd1d90e5dddc9e148e08d14e0560c", + "phase/provenance/parent/paired/confirm-fresh-35-te-m0.json": "8718f5e6b21c1eef12243ec244e34a2d0da7b7bcbe5f13f09582d1a8c6614599", + "phase/provenance/parent/paired/confirm-fresh-35-te-m1.json": "bdc6e209d6d1bb5ff6034737ad098e5ee2bc0b0a386fa0c24bd853f63d625cd4", + "phase/provenance/parent/paired/confirm-fresh-36-en-m0.json": "47024432738e9cb8d96e5aedbdb024190cb9a264e75053d44e3bdb9dbebc442d", + "phase/provenance/parent/paired/confirm-fresh-36-en-m1.json": "94a800df85d56380e60b8141aa36b99ba63d4d925aba6777f210199c70d82252", + "phase/provenance/parent/paired/confirm-fresh-36-hi-m0.json": "f2e2e33982bd0af877ef57f19ecd9113557e0bc8520f0da61005f6eddd949380", + "phase/provenance/parent/paired/confirm-fresh-36-hi-m1.json": "04a1e7abde3b11aa754a9dbb381ed33fc6bf5979d9f0a261e4b1ad8f96075784", + "phase/provenance/parent/paired/confirm-fresh-36-te-m0.json": "5ee0958da22ed2e917c0fb98fce93fe0f7ec24c9514e386d73af0f22821f6fc2", + "phase/provenance/parent/paired/confirm-fresh-36-te-m1.json": "fee6c3c3c7557c5080caa1924dd5be68975b1b0b73563bd34df42dd463f66bdc", + "phase/provenance/parent/paired/confirm-fresh-37-en-m0.json": "29b55f152ef7c8110f41bad222abd90b8e485dc5edbbd6f330cda6d5e764fd91", + "phase/provenance/parent/paired/confirm-fresh-37-en-m1.json": "51c205e56e219e1ea7138eb708d9059a0d4428d17a8cbcbffcd51103956bb589", + "phase/provenance/parent/paired/confirm-fresh-37-hi-m0.json": "82b052aceedd9d0c9b2573436bd29f293227803a9f6eb3db6b00b09db04e0112", + "phase/provenance/parent/paired/confirm-fresh-37-hi-m1.json": "688a24bb74b72955c8f88980905f61a5c368a61b7e19ae651ef4ed9efd54a09a", + "phase/provenance/parent/paired/confirm-fresh-37-te-m0.json": "80fdfdbbc0ca4ba6a9de004de2991147d0a48b44eb5251e4139bb2c15002ade5", + "phase/provenance/parent/paired/confirm-fresh-37-te-m1.json": "a1e09933fda0a506f1e12df21a1c802649ecafc6f783befa17410f35c9c75e25", + "phase/provenance/parent/paired/confirm-fresh-38-en-m0.json": "4c19f3ac5d7e8cff84e515b182fc3ec4a3d44c055eb94fb05d24424bbdbc3218", + "phase/provenance/parent/paired/confirm-fresh-38-en-m1.json": "c3e160ba861133e3ecfb894fcd3cff9345bd0df2ee0ffd5be840b90890815cd5", + "phase/provenance/parent/paired/confirm-fresh-38-hi-m0.json": "e392cb4c62b29a0b4f5792563c73271041880aad77217edb74b9f613ee66d8ac", + "phase/provenance/parent/paired/confirm-fresh-38-hi-m1.json": "fda3a32777e4d82de798ed552d284cb3642082f4be3a877fed456a03adf2c987", + "phase/provenance/parent/paired/confirm-fresh-38-te-m0.json": "13cfc20073d520501af6e3ca6e7b906cf54693534bed0ba89906a6fa14012fe9", + "phase/provenance/parent/paired/confirm-fresh-38-te-m1.json": "892998984302902e08e87525602769e0bb382e798d6a32270c11b95ddaaafb2c", + "phase/provenance/parent/paired/confirm-fresh-39-en-m0.json": "8a675839590f2e7961ec0f44073fbeedde9687a93ee04a89f28a2e7499ec2dd3", + "phase/provenance/parent/paired/confirm-fresh-39-en-m1.json": "d34c318019cb9c27f6c69f0b4dfb5e0badeaee76074739b4492da33eb478fd2b", + "phase/provenance/parent/paired/confirm-fresh-39-hi-m0.json": "77bfcfaf3de967fd520cb3336cfc9d1a401f531287d36a8c38efed8c9d288735", + "phase/provenance/parent/paired/confirm-fresh-39-hi-m1.json": "da2e99783d8a0031a57e50d33c7c1aa389712ba19a81ffec3a633a4d0e3bbdf3", + "phase/provenance/parent/paired/confirm-fresh-39-te-m0.json": "be25deef9aa7af2eff657ea1983c9686f14843c51ca2bab6a1273c59ef1142f2", + "phase/provenance/parent/paired/confirm-fresh-39-te-m1.json": "604968967b4559221e4b47b9f501ef8a13079ab7aa9687a83edd5a83bd301f76", + "phase/provenance/parent/paired/confirm-fresh-40-en-m0.json": "46292e57eb7a9f950c4b4368c0d598018873f9491aeff6e3f27bb469c9ae43f5", + "phase/provenance/parent/paired/confirm-fresh-40-en-m1.json": "c04ebb10ef9d739136e60f8dab37c3cb0049a584b741034ac793ce5dcc6b9634", + "phase/provenance/parent/paired/confirm-fresh-40-hi-m0.json": "61879fa774dfeee20c10c8e3b2594895c2761965f4f94875f927b555b49934b4", + "phase/provenance/parent/paired/confirm-fresh-40-hi-m1.json": "d82c493b8a6bea420a3e4a9eaaf3a6815bc33dda618944db98e9ee684a04b75a", + "phase/provenance/parent/paired/confirm-fresh-40-te-m0.json": "49d8e0c2cc252cad53bdc05fad7bd3de259f18935a7dbd41e19c4bedf1144f23", + "phase/provenance/parent/paired/confirm-fresh-40-te-m1.json": "e59da4a5d3b8a411ff189c6b882589213f489df7a3742b4f3b14bb5f04afaafd", + "phase/provenance/parent/review.json": "204ee32cfb53e502f60f8e1bd093f3a90cc507047edb9b54821befb17088ba72", + "phase/provenance/parent/runtime.json": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17", + "phase/provenance/parent/semantic-review-en-complete.json": "bff2555bbc642d1054f2ebcd4a7afd22064db5a9ff6d49a03511d64d80fffd67", + "phase/provenance/parent/semantic-review-hi-complete.json": "5986f53c80482adc9910c5298dd09073a8e9327eff1457e931e784b9896f6865", + "phase/provenance/parent/semantic-review-te-complete.json": "ee00eedca16cfd3efe74b6e0e21f4c1ac8e43626b54c23b46f79f54909c8996e", + "phase/provenance/parent/semantic-review.json": "f2a472edb43e4cf5af0d8cf39ad804083089f65ad161fcec949d5894d234ca12", + "phase/provenance/parent/source/benchmarks/sarvam_challenge_cases.py": "2c82fac0c7f57b99a44b789d0e167cb0087c61ac17f9c0a7fc97780794e9a7e9", + "phase/provenance/parent/source/benchmarks/sarvam_fresh_confirmation/__init__.py": "075f1efc52897196fc54ae10400532882e52285e3a1346a330f2660d99eac19f", + "phase/provenance/parent/source/benchmarks/sarvam_fresh_confirmation/audit.py": "616df042a054eb95b41b76e62866eb9fa456b89d2c05cf5bc3d591e6870198ce", + "phase/provenance/parent/source/benchmarks/sarvam_fresh_confirmation/cases.py": "8d42cbe9689174a5b5eaf7615abc1df487a1568b1d7e64d522c087f6c39526ea", + "phase/provenance/parent/source/benchmarks/sarvam_fresh_confirmation/engine.py": "2d8c68e744d144e4805a68f405c533b4c9b3dd7a35a4cf92b61e45dc1a88e6e8", + "phase/provenance/parent/source/benchmarks/sarvam_fresh_confirmation/study.py": "04c4437392bff1a14733d00640a0a9ed1f18fba6884e3fee7f2ec870e13a288c", + "phase/provenance/parent/source/benchmarks/sarvam_native_controls/__init__.py": "af98f2f023ae7fcad775f4dd3eeabfb2ed8d92cfe3183769c73b9a560b536e29", + "phase/provenance/parent/source/benchmarks/sarvam_native_controls/approval_backend.py": "5bc07594e1e3e3d69d5d5a4190b1fc94aaafcbcf85bdde570002b96c1402f9c1", + "phase/provenance/parent/source/benchmarks/sarvam_native_controls/approval_replay.py": "68877a448f9191aab131417ffe851f5aaaf9ef976d428f002fe0601e19ee0d67", + "phase/provenance/parent/source/benchmarks/sarvam_native_controls/cases.py": "1ee460f5ba74c3d933b26e59fcbc34242b10be67c2da44f4b33ad7c68d907677", + "phase/provenance/parent/source/benchmarks/sarvam_native_controls/engine.py": "a1378a618a12392edb18f5e56861ab1d43e40da21274642ad759076adaa25206", + "phase/provenance/parent/source/benchmarks/sarvam_native_controls/finalization_transport.py": "ed2b75497e2a3f815ea9fdbe902ea4d2944c831b929f5c8840054372a4e43b8f", + "phase/provenance/parent/source/benchmarks/sarvam_native_controls/integrated.py": "11b74202283238bb51f001513b3d1c14ec5bfad2f5bcae07d28a099dfe9c3312", + "phase/provenance/parent/source/benchmarks/sarvam_native_controls/integrated_audit.py": "6719d308610228c0532b199e9124686684eeaf913a22030732991506353a1c58", + "phase/provenance/parent/source/benchmarks/sarvam_native_controls/integrated_cases.py": "19a6e37d8b29b40a3b7c9ddd80e0df59b0c80fce47a624919a431d23f8a6e0f4", + "phase/provenance/parent/source/benchmarks/sarvam_native_controls/separated.py": "116100f9819ef5ef0a40164dae6f7d90951b32861216c89098d5ce35a04f6ca2", + "phase/provenance/parent/source/benchmarks/sarvam_native_controls/separated_audit.py": "089f7c6eef538a419a12d1fbe3b0ff3dc8ec7d7f0fdb1f8c36dc7ac7c28f6bca", + "phase/provenance/parent/source/benchmarks/sarvam_native_controls/separated_cases.py": "db092ffe8363149c38e7ddedc49c2ed82898ee33b932a6caf585cc7d7b3b1f05", + "phase/provenance/parent/source/benchmarks/sarvam_native_controls/transcript_finalizer.py": "b7186a811e2672d4ee451b86e9b28f8c62fcbc78f50f996c81e701d804b530dc", + "phase/provenance/parent/source/benchmarks/sarvam_native_study/__init__.py": "b7a8a4e041564f9eb2bc64d45044ae4e3585e10612e1b477b48655a501689c1f", + "phase/provenance/parent/source/benchmarks/sarvam_native_study/analyze.py": "b2eff50279ffa89d5130664c858359cf6d1bd0693fe4070319bc7431a2db0496", + "phase/provenance/parent/source/benchmarks/sarvam_native_study/backend.py": "1102f0dbed3c9fddd1592af7ebed2a2bcfc3ae3eefa7b73c1f7b0583dcac6280", + "phase/provenance/parent/source/benchmarks/sarvam_native_study/dataset.py": "28f4aaa64018d5262428ad399b9c441041567454facba2b6b67b80250843e529", + "phase/provenance/parent/source/benchmarks/sarvam_native_study/diagnostic.py": "73b10b9efa15fd5f4405e07be9a062275fac7f1275b70c56557174744642d667", + "phase/provenance/parent/source/benchmarks/sarvam_native_study/paced.py": "cba46897f5aa48bbecb118fefbeab8a578624fe7187e7971f2549d4fcb853190", + "phase/provenance/parent/source/benchmarks/sarvam_native_study/runner.py": "c49c367bf962d165bea1781f0d0070431b1d52c90ef14b115a9a4f964afd9917", + "phase/provenance/parent/source/benchmarks/sarvam_native_study/score.py": "5f01017bbc70341642d2e1db5e0fb525c29a71d44913d3d11f7a8701e85e902b", + "phase/provenance/parent/source/benchmarks/sarvam_native_study/setup.py": "8b504626d002c80565347d62ac802c2af417b53550b5f1fdaf5805250695d3f4", + "phase/provenance/parent/source/benchmarks/sarvam_native_study/transport.py": "089ab3a31e32c73c5e77292c172a4828aa12e50ca13c07d1eb45d0a425494d1b", + "phase/provenance/parent/source/benchmarks/sarvam_native_study/verify.py": "70c94fdffa428b86205b0bcbb5f8f17582dcff0b0f7793ea5980e3de42b62ec9", + "phase/provenance/parent/source/benchmarks/sarvam_ordinary_audit.py": "19a58a4f15a5888a15f6fc9789d267b49ac9c32def7623de60c4c36f8245010d", + "phase/provenance/parent/source/benchmarks/sarvam_ordinary_study.py": "3c434b00dce391b8e48f7eac23eec4d8698330e95ea596cce89c8533ad054517", + "phase/provenance/parent/source/benchmarks/sarvam_ordinary_v2.py": "af73642b70428148349a0a0b33fb4797f793fc9b040a6f9ac75e37380c433a0a", + "phase/provenance/parent/source/benchmarks/sarvam_receipt_reporting.py": "f95897540e50b1c0d3e27295a7c7cdaee7c0531c8f3a8e50dde087ede9ecf8ff", + "phase/provenance/parent/source/benchmarks/sarvam_reporter_wording.py": "90ec4d13e166001626009521771ed20f6f0f8b646a1b6fb899d1c0c111b11064", + "phase/provenance/parent/source/benchmarks/sarvam_source_attribution.py": "65cf2323c7048b0f360121145f8bb6029009503e47365b69ff0370c07284ec17", + "phase/provenance/parent/started.json": "09fbb58473ac2c3303461750d21e49150b15d6318b8b71e0f8335e96cc12e7b2", + "phase/provenance/parent/verification.json": "48f5af8bf606fe993f794b376296fe58523450175c9942f0a18c044036de2a94", + "phase/provenance/prior-completion.json": "060d1a446bd3b03e4c470bb92857ac8ee5d08e961b60f4ef4d68413b7743e8af", + "phase/publication-plan.json": "7f7911d00704091f9ba4190d068ea4988e930fa22ab3fd58f5e3261ca9db4531", + "phase/review-verification.json": "d2dee9c28fe126b9b39ffd09c9678023e632b957a776367b246a02ff8fc29c76", + "phase/reviews/HINDI_INTERPRETATION.md": "b7174e15891d012b9e58b419aed91b1b86c97c1be2d4287b7fc15e5de49ade4c", + "phase/reviews/english-review.json": "b82bec49adc44c28a68620be23b6723d53973f27e3d17c2a2d1427545e7bed13", + "phase/reviews/hindi-review.json": "d4de3549829ae68032645dd0b58050de34a840d3bb1f7de6da992fb7a14db11e", + "phase/reviews/telugu-review.json": "16d147a825c9fb1fbd226e49464a4c79e055f0e28b6a05a73202bb15cafd212d", + "phase/sealed/gold.json": "f27502aa6eaebbd86a7836fd4ca9bb4ee8f50fece0772f7e25df942851701245", + "phase/sidecars.json": "3770d08344de5c36dca5455ee78af4f2aff579e4b920df022d50fb6c612fc575", + "phase/source-preflight.json": "f61120a5199f7884452c4efc196cb6f13f47fa4d95a5053c07ef5a858f51471a", + "phase/started.json": "864bbe0444590d96962764c59a0686555970328b9fd243a8a67c253e234e5bd8", + "phase/summary.json": "813a379f44d204706195d2a3b8c23583b5d79c6c54e8c47e5dd2aec7d6f4fd3f" + }, + "credential_byte_scan_pass": true, + "private_founder_inventory_excluded": true, + "scope": "All 240 saved histories, 720 reporter calls and frozen source/provenance; no new actor actions.", + "provider_identity_limitation": "Local receipt hashes are integrity checks, not provider signatures." +} diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/audit.json b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/audit.json new file mode 100644 index 0000000..3a97d2c --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/audit.json @@ -0,0 +1,43488 @@ +{ + "verified": true, + "complete": true, + "kind": "ALL_240_SAVED_HISTORY_SOURCE_INTERPRETATION_NOT_PARENT_REPAIR", + "execution_mode": "production_default_transport", + "planned_calls": 720, + "attempted": 720, + "reporter_receipts": 720, + "new_native_action_trials": 0, + "known_microinr": 93896460, + "retained_microinr": 117370843, + "unknown_calls": 0, + "infrastructure_errors": 0, + "cumulative_calls": 7757, + "cumulative_known_microinr": 365121090, + "cumulative_retained_microinr": 488660182, + "rows": [ + { + "id": "confirm-fresh-31-en-m1-source-chronological", + "history_id": "confirm-fresh-31-en-m1", + "case_id": "fresh-31-en", + "scenario": "fresh-31", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1509, + "completion_tokens": 40, + "known_cost_microinr": 25035, + "retained_microinr": 31294, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0370477910037152, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-31-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-31-en-m1", + "case_id": "fresh-31-en", + "scenario": "fresh-31", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2281, + "completion_tokens": 42, + "known_cost_microinr": 36735, + "retained_microinr": 45919, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9786056249868125, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-31-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-31-en-m1", + "case_id": "fresh-31-en", + "scenario": "fresh-31", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2281, + "completion_tokens": 42, + "known_cost_microinr": 36735, + "retained_microinr": 45919, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1229317920078756, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-22-hi-m1-source-chronological", + "history_id": "confirm-fresh-22-hi-m1", + "case_id": "fresh-22-hi", + "scenario": "fresh-22", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1467, + "completion_tokens": 40, + "known_cost_microinr": 24405, + "retained_microinr": 30507, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1193134169880068, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-22-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-22-hi-m1", + "case_id": "fresh-22-hi", + "scenario": "fresh-22", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2114, + "completion_tokens": 40, + "known_cost_microinr": 34110, + "retained_microinr": 42638, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0401996669970686, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-22-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-22-hi-m1", + "case_id": "fresh-22-hi", + "scenario": "fresh-22", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2114, + "completion_tokens": 40, + "known_cost_microinr": 34110, + "retained_microinr": 42638, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.2693526670045685, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-24-hi-m1", + "case_id": "fresh-24-hi", + "scenario": "fresh-24", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2120, + "completion_tokens": 40, + "known_cost_microinr": 34200, + "retained_microinr": 42750, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9839419580093818, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-hi-m1-source-chronological", + "history_id": "confirm-fresh-24-hi-m1", + "case_id": "fresh-24-hi", + "scenario": "fresh-24", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1462, + "completion_tokens": 40, + "known_cost_microinr": 24330, + "retained_microinr": 30413, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9552010420011356, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-24-hi-m1", + "case_id": "fresh-24-hi", + "scenario": "fresh-24", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2120, + "completion_tokens": 40, + "known_cost_microinr": 34200, + "retained_microinr": 42750, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9628701659967192, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-hi-m0-source-chronological", + "history_id": "confirm-fresh-29-hi-m0", + "case_id": "fresh-29-hi", + "scenario": "fresh-29", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1387, + "completion_tokens": 2714, + "known_cost_microinr": 183645, + "retained_microinr": 229557, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.89874437499384, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-29-hi-m0", + "case_id": "fresh-29-hi", + "scenario": "fresh-29", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1929, + "completion_tokens": 7281, + "known_cost_microinr": 465795, + "retained_microinr": 582244, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 34.35796750000736, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-29-hi-m0", + "case_id": "fresh-29-hi", + "scenario": "fresh-29", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1738, + "completion_tokens": 3282, + "known_cost_microinr": 222990, + "retained_microinr": 278738, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.335680667005363, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-18-en-m1-source-chronological", + "history_id": "confirm-fresh-18-en-m1", + "case_id": "fresh-18-en", + "scenario": "fresh-18", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1037, + "completion_tokens": 42, + "known_cost_microinr": 18075, + "retained_microinr": 22594, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0114832500112243, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-18-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-18-en-m1", + "case_id": "fresh-18-en", + "scenario": "fresh-18", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1623, + "completion_tokens": 42, + "known_cost_microinr": 26865, + "retained_microinr": 33582, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9449328329937998, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-18-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-en-m1", + "case_id": "fresh-18-en", + "scenario": "fresh-18", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": true, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1564, + "completion_tokens": 32, + "known_cost_microinr": 25380, + "retained_microinr": 31725, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0430362920014886, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-34-hi-m1-source-chronological", + "history_id": "confirm-fresh-34-hi-m1", + "case_id": "fresh-34-hi", + "scenario": "fresh-34", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1971, + "completion_tokens": 42, + "known_cost_microinr": 32085, + "retained_microinr": 40107, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1093939580023289, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-34-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-34-hi-m1", + "case_id": "fresh-34-hi", + "scenario": "fresh-34", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2539, + "completion_tokens": 42, + "known_cost_microinr": 40605, + "retained_microinr": 50757, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9951028329960536, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-34-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-hi-m1", + "case_id": "fresh-34-hi", + "scenario": "fresh-34", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2198, + "completion_tokens": 42, + "known_cost_microinr": 35490, + "retained_microinr": 44363, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9545394169981591, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-te-m0", + "case_id": "fresh-32-te", + "scenario": "fresh-32", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1591, + "completion_tokens": 1937, + "known_cost_microinr": 140085, + "retained_microinr": 175107, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 10.579083415999776, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-32-te-m0", + "case_id": "fresh-32-te", + "scenario": "fresh-32", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1731, + "completion_tokens": 1972, + "known_cost_microinr": 144285, + "retained_microinr": 180357, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.322261666995473, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-te-m0-source-chronological", + "history_id": "confirm-fresh-32-te-m0", + "case_id": "fresh-32-te", + "scenario": "fresh-32", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1303, + "completion_tokens": 1861, + "known_cost_microinr": 131205, + "retained_microinr": 164007, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.870290666993242, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-30-te-m0", + "case_id": "fresh-30-te", + "scenario": "fresh-30", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1856, + "completion_tokens": 1401, + "known_cost_microinr": 111900, + "retained_microinr": 139875, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.085513083002297, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-te-m0", + "case_id": "fresh-30-te", + "scenario": "fresh-30", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1601, + "completion_tokens": 3252, + "known_cost_microinr": 219135, + "retained_microinr": 273919, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 14.983070792004582, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-te-m0-source-chronological", + "history_id": "confirm-fresh-30-te-m0", + "case_id": "fresh-30-te", + "scenario": "fresh-30", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1418, + "completion_tokens": 2969, + "known_cost_microinr": 199410, + "retained_microinr": 249263, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.611806625005556, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-hi-m0-source-chronological", + "history_id": "confirm-fresh-30-hi-m0", + "case_id": "fresh-30-hi", + "scenario": "fresh-30", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1417, + "completion_tokens": 7486, + "known_cost_microinr": 470415, + "retained_microinr": 588019, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 35.69330058401101, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-hi-m0", + "case_id": "fresh-30-hi", + "scenario": "fresh-30", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1599, + "completion_tokens": 4604, + "known_cost_microinr": 300225, + "retained_microinr": 375282, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 20.368539167000563, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-30-hi-m0", + "case_id": "fresh-30-hi", + "scenario": "fresh-30", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1859, + "completion_tokens": 2886, + "known_cost_microinr": 201045, + "retained_microinr": 251307, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 15.877402416997938, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-17-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-17-te-m0", + "case_id": "fresh-17-te", + "scenario": "fresh-17", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1451, + "completion_tokens": 3248, + "known_cost_microinr": 216645, + "retained_microinr": 270807, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 17.090247959000408, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-17-te-m0-source-chronological", + "history_id": "confirm-fresh-17-te-m0", + "case_id": "fresh-17-te", + "scenario": "fresh-17", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 899, + "completion_tokens": 5155, + "known_cost_microinr": 322785, + "retained_microinr": 403482, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 23.44692899999791, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-17-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-17-te-m0", + "case_id": "fresh-17-te", + "scenario": "fresh-17", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1436, + "completion_tokens": 2483, + "known_cost_microinr": 170520, + "retained_microinr": 213150, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.231449500002782, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-23-hi-m1-source-chronological", + "history_id": "confirm-fresh-23-hi-m1", + "case_id": "fresh-23-hi", + "scenario": "fresh-23", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1503, + "completion_tokens": 40, + "known_cost_microinr": 24945, + "retained_microinr": 31182, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9960489160002908, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-23-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-23-hi-m1", + "case_id": "fresh-23-hi", + "scenario": "fresh-23", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2267, + "completion_tokens": 40, + "known_cost_microinr": 36405, + "retained_microinr": 45507, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9549742919916753, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-23-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-23-hi-m1", + "case_id": "fresh-23-hi", + "scenario": "fresh-23", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2267, + "completion_tokens": 40, + "known_cost_microinr": 36405, + "retained_microinr": 45507, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9802527080028085, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-23-hi-m0-source-chronological", + "history_id": "confirm-fresh-23-hi-m0", + "case_id": "fresh-23-hi", + "scenario": "fresh-23", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1367, + "completion_tokens": 975, + "known_cost_microinr": 79005, + "retained_microinr": 98757, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 5.530833540993626, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-23-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-23-hi-m0", + "case_id": "fresh-23-hi", + "scenario": "fresh-23", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1981, + "completion_tokens": 2997, + "known_cost_microinr": 209535, + "retained_microinr": 261919, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.545640917000128, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-23-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-23-hi-m0", + "case_id": "fresh-23-hi", + "scenario": "fresh-23", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1981, + "completion_tokens": 3106, + "known_cost_microinr": 216075, + "retained_microinr": 270094, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.844156040999223, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-18-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-18-hi-m1", + "case_id": "fresh-18-hi", + "scenario": "fresh-18", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1615, + "completion_tokens": 42, + "known_cost_microinr": 26745, + "retained_microinr": 33432, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0425940839922987, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-18-hi-m1-source-chronological", + "history_id": "confirm-fresh-18-hi-m1", + "case_id": "fresh-18-hi", + "scenario": "fresh-18", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1017, + "completion_tokens": 42, + "known_cost_microinr": 17775, + "retained_microinr": 22219, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0169891669938806, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-18-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-hi-m1", + "case_id": "fresh-18-hi", + "scenario": "fresh-18", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1577, + "completion_tokens": 42, + "known_cost_microinr": 26175, + "retained_microinr": 32719, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.971732999998494, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-hi-m0-source-chronological", + "history_id": "confirm-fresh-24-hi-m0", + "case_id": "fresh-24-hi", + "scenario": "fresh-24", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1332, + "completion_tokens": 3077, + "known_cost_microinr": 204600, + "retained_microinr": 255750, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.557264499992016, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-24-hi-m0", + "case_id": "fresh-24-hi", + "scenario": "fresh-24", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1841, + "completion_tokens": 3383, + "known_cost_microinr": 230595, + "retained_microinr": 288244, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 14.344590167005663, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-24-hi-m0", + "case_id": "fresh-24-hi", + "scenario": "fresh-24", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1841, + "completion_tokens": 3946, + "known_cost_microinr": 264375, + "retained_microinr": 330469, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 16.596892250003293, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-hi-m1", + "case_id": "fresh-39-hi", + "scenario": "fresh-39", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2243, + "completion_tokens": 42, + "known_cost_microinr": 36165, + "retained_microinr": 45207, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0989783330005594, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-39-hi-m1", + "case_id": "fresh-39-hi", + "scenario": "fresh-39", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2550, + "completion_tokens": 42, + "known_cost_microinr": 40770, + "retained_microinr": 50963, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0773608750023413, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-hi-m1-source-chronological", + "history_id": "confirm-fresh-39-hi-m1", + "case_id": "fresh-39-hi", + "scenario": "fresh-39", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1862, + "completion_tokens": 42, + "known_cost_microinr": 30450, + "retained_microinr": 38063, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1406128329981584, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-en-m0", + "case_id": "fresh-35-en", + "scenario": "fresh-35", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2060, + "completion_tokens": 2724, + "known_cost_microinr": 194340, + "retained_microinr": 242925, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.952315665999777, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-en-m0-source-chronological", + "history_id": "confirm-fresh-35-en-m0", + "case_id": "fresh-35-en", + "scenario": "fresh-35", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1771, + "completion_tokens": 6852, + "known_cost_microinr": 437685, + "retained_microinr": 547107, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 27.917850792000536, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-35-en-m0", + "case_id": "fresh-35-en", + "scenario": "fresh-35", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2310, + "completion_tokens": 1534, + "known_cost_microinr": 126690, + "retained_microinr": 158363, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.308625707999454, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-en-m1-source-chronological", + "history_id": "confirm-fresh-36-en-m1", + "case_id": "fresh-36-en", + "scenario": "fresh-36", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1959, + "completion_tokens": 42, + "known_cost_microinr": 31905, + "retained_microinr": 39882, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9920252919982886, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-36-en-m1", + "case_id": "fresh-36-en", + "scenario": "fresh-36", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2529, + "completion_tokens": 42, + "known_cost_microinr": 40455, + "retained_microinr": 50569, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9872349159995792, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-en-m1", + "case_id": "fresh-36-en", + "scenario": "fresh-36", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2190, + "completion_tokens": 42, + "known_cost_microinr": 35370, + "retained_microinr": 44213, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0595242499985034, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-08-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-08-hi-m1", + "case_id": "fresh-08-hi", + "scenario": "fresh-08", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1257, + "completion_tokens": 32, + "known_cost_microinr": 20775, + "retained_microinr": 25969, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9314788750052685, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-08-hi-m1-source-chronological", + "history_id": "confirm-fresh-08-hi-m1", + "case_id": "fresh-08-hi", + "scenario": "fresh-08", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-09", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 868, + "completion_tokens": 42, + "known_cost_microinr": 15540, + "retained_microinr": 19425, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9414168749935925, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-08-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-08-hi-m1", + "case_id": "fresh-08-hi", + "scenario": "fresh-08", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1308, + "completion_tokens": 32, + "known_cost_microinr": 21540, + "retained_microinr": 26925, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9929245410021394, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-21-hi-m1", + "case_id": "fresh-21-hi", + "scenario": "fresh-21", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2276, + "completion_tokens": 40, + "known_cost_microinr": 36540, + "retained_microinr": 45675, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0852421669987962, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-21-hi-m1", + "case_id": "fresh-21-hi", + "scenario": "fresh-21", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2276, + "completion_tokens": 40, + "known_cost_microinr": 36540, + "retained_microinr": 45675, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.01157066700398, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-hi-m1-source-chronological", + "history_id": "confirm-fresh-21-hi-m1", + "case_id": "fresh-21-hi", + "scenario": "fresh-21", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1510, + "completion_tokens": 40, + "known_cost_microinr": 25050, + "retained_microinr": 31313, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9554717499995604, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-25-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-25-te-m0", + "case_id": "fresh-25-te", + "scenario": "fresh-25", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1985, + "completion_tokens": 8192, + "known_cost_microinr": 521295, + "retained_microinr": 651619, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 39.421729541005334, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-25-te-m0-source-chronological", + "history_id": "confirm-fresh-25-te-m0", + "case_id": "fresh-25-te", + "scenario": "fresh-25", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1375, + "completion_tokens": 1111, + "known_cost_microinr": 87285, + "retained_microinr": 109107, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.427784958999837, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-25-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-25-te-m0", + "case_id": "fresh-25-te", + "scenario": "fresh-25", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1985, + "completion_tokens": 5158, + "known_cost_microinr": 339255, + "retained_microinr": 424069, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 25.215050832994166, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-te-m1", + "case_id": "fresh-36-te", + "scenario": "fresh-36", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2219, + "completion_tokens": 42, + "known_cost_microinr": 35805, + "retained_microinr": 44757, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9523697499971604, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-te-m1-source-chronological", + "history_id": "confirm-fresh-36-te-m1", + "case_id": "fresh-36-te", + "scenario": "fresh-36", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2052, + "completion_tokens": 42, + "known_cost_microinr": 33300, + "retained_microinr": 41625, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0162083329923917, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-36-te-m1", + "case_id": "fresh-36-te", + "scenario": "fresh-36", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2627, + "completion_tokens": 42, + "known_cost_microinr": 41925, + "retained_microinr": 52407, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9634353330038721, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-13-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-13-hi-m1", + "case_id": "fresh-13-hi", + "scenario": "fresh-13", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1651, + "completion_tokens": 32, + "known_cost_microinr": 26685, + "retained_microinr": 33357, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9992062499950407, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-13-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-13-hi-m1", + "case_id": "fresh-13-hi", + "scenario": "fresh-13", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-14", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-14", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1421, + "completion_tokens": 42, + "known_cost_microinr": 23835, + "retained_microinr": 29794, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9959914999926696, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-13-hi-m1-source-chronological", + "history_id": "confirm-fresh-13-hi-m1", + "case_id": "fresh-13-hi", + "scenario": "fresh-13", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1099, + "completion_tokens": 42, + "known_cost_microinr": 19005, + "retained_microinr": 23757, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9374043749994598, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-29-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-29-te-m0", + "case_id": "fresh-29-te", + "scenario": "fresh-29", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2268, + "completion_tokens": 2444, + "known_cost_microinr": 180660, + "retained_microinr": 225825, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.178991207998479, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-te-m0-source-chronological", + "history_id": "confirm-fresh-29-te-m0", + "case_id": "fresh-29-te", + "scenario": "fresh-29", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1576, + "completion_tokens": 2566, + "known_cost_microinr": 177600, + "retained_microinr": 222000, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.215514750001603, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-29-te-m0", + "case_id": "fresh-29-te", + "scenario": "fresh-29", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2032, + "completion_tokens": 1531, + "known_cost_microinr": 122340, + "retained_microinr": 152925, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.056660917005502, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-26-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-26-hi-m0", + "case_id": "fresh-26-hi", + "scenario": "fresh-26", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1823, + "completion_tokens": 7735, + "known_cost_microinr": 491445, + "retained_microinr": 614307, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 40.78428570899996, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-26-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-26-hi-m0", + "case_id": "fresh-26-hi", + "scenario": "fresh-26", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1823, + "completion_tokens": 8192, + "known_cost_microinr": 518865, + "retained_microinr": 648582, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 39.290506540986826, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-26-hi-m0-source-chronological", + "history_id": "confirm-fresh-26-hi-m0", + "case_id": "fresh-26-hi", + "scenario": "fresh-26", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1330, + "completion_tokens": 2311, + "known_cost_microinr": 158610, + "retained_microinr": 198263, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.705150166992098, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-01-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-01-hi-m0", + "case_id": "fresh-01-hi", + "scenario": "fresh-01", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1425, + "completion_tokens": 3361, + "known_cost_microinr": 223035, + "retained_microinr": 278794, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 15.34867537500395, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-01-hi-m0-source-chronological", + "history_id": "confirm-fresh-01-hi-m0", + "case_id": "fresh-01-hi", + "scenario": "fresh-01", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 869, + "completion_tokens": 6468, + "known_cost_microinr": 401115, + "retained_microinr": 501394, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 27.64760770900466, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-01-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-01-hi-m0", + "case_id": "fresh-01-hi", + "scenario": "fresh-01", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1410, + "completion_tokens": 761, + "known_cost_microinr": 66810, + "retained_microinr": 83513, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 4.8372162919986295, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-en-m1-source-chronological", + "history_id": "confirm-fresh-24-en-m1", + "case_id": "fresh-24-en", + "scenario": "fresh-24", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1454, + "completion_tokens": 40, + "known_cost_microinr": 24210, + "retained_microinr": 30263, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9435721250047209, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-24-en-m1", + "case_id": "fresh-24-en", + "scenario": "fresh-24", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2111, + "completion_tokens": 40, + "known_cost_microinr": 34065, + "retained_microinr": 42582, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0013978330098325, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-24-en-m1", + "case_id": "fresh-24-en", + "scenario": "fresh-24", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2111, + "completion_tokens": 40, + "known_cost_microinr": 34065, + "retained_microinr": 42582, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.036732790991664, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-02-te-m0", + "case_id": "fresh-02-te", + "scenario": "fresh-02", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1266, + "completion_tokens": 2628, + "known_cost_microinr": 176670, + "retained_microinr": 220838, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.4500859999971, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-02-te-m0", + "case_id": "fresh-02-te", + "scenario": "fresh-02", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1281, + "completion_tokens": 3735, + "known_cost_microinr": 243315, + "retained_microinr": 304144, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 16.362450125001487, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-te-m0-source-chronological", + "history_id": "confirm-fresh-02-te-m0", + "case_id": "fresh-02-te", + "scenario": "fresh-02", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 839, + "completion_tokens": 2723, + "known_cost_microinr": 175965, + "retained_microinr": 219957, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.771132666995982, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-09-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-09-hi-m0", + "case_id": "fresh-09-hi", + "scenario": "fresh-09", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1410, + "completion_tokens": 3073, + "known_cost_microinr": 205530, + "retained_microinr": 256913, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 18.970398125005886, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-09-hi-m0-source-chronological", + "history_id": "confirm-fresh-09-hi-m0", + "case_id": "fresh-09-hi", + "scenario": "fresh-09", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 995, + "completion_tokens": 2916, + "known_cost_microinr": 189885, + "retained_microinr": 237357, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.956998584006215, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-09-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-09-hi-m0", + "case_id": "fresh-09-hi", + "scenario": "fresh-09", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1543, + "completion_tokens": 2612, + "known_cost_microinr": 179865, + "retained_microinr": 224832, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 16.79411670898844, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-04-te-m0-source-chronological", + "history_id": "confirm-fresh-04-te-m0", + "case_id": "fresh-04-te", + "scenario": "fresh-04", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 839, + "completion_tokens": 3083, + "known_cost_microinr": 197565, + "retained_microinr": 246957, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 17.790569124990725, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-04-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-04-te-m0", + "case_id": "fresh-04-te", + "scenario": "fresh-04", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1262, + "completion_tokens": 778, + "known_cost_microinr": 65610, + "retained_microinr": 82013, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 3.9927155420009512, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-04-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-04-te-m0", + "case_id": "fresh-04-te", + "scenario": "fresh-04", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1277, + "completion_tokens": 629, + "known_cost_microinr": 56895, + "retained_microinr": 71119, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 3.36182416700467, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-13-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-13-hi-m0", + "case_id": "fresh-13-hi", + "scenario": "fresh-13", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1581, + "completion_tokens": 6568, + "known_cost_microinr": 417795, + "retained_microinr": 522244, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 32.179166166999494, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-13-hi-m0-source-chronological", + "history_id": "confirm-fresh-13-hi-m0", + "case_id": "fresh-13-hi", + "scenario": "fresh-13", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1023, + "completion_tokens": 2594, + "known_cost_microinr": 170985, + "retained_microinr": 213732, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.822467374993721, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-13-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-13-hi-m0", + "case_id": "fresh-13-hi", + "scenario": "fresh-13", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1423, + "completion_tokens": 5109, + "known_cost_microinr": 327885, + "retained_microinr": 409857, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 23.067031166996458, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-31-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-31-hi-m0", + "case_id": "fresh-31-hi", + "scenario": "fresh-31", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1914, + "completion_tokens": 6711, + "known_cost_microinr": 431370, + "retained_microinr": 539213, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 35.7180771669955, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-31-hi-m0-source-chronological", + "history_id": "confirm-fresh-31-hi-m0", + "case_id": "fresh-31-hi", + "scenario": "fresh-31", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1364, + "completion_tokens": 2332, + "known_cost_microinr": 160380, + "retained_microinr": 200475, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.674144791002618, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-31-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-31-hi-m0", + "case_id": "fresh-31-hi", + "scenario": "fresh-31", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1747, + "completion_tokens": 1827, + "known_cost_microinr": 135825, + "retained_microinr": 169782, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.120365249997121, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-te-m0-source-chronological", + "history_id": "confirm-fresh-33-te-m0", + "case_id": "fresh-33-te", + "scenario": "fresh-33", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1815, + "completion_tokens": 1676, + "known_cost_microinr": 127785, + "retained_microinr": 159732, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.552738250000402, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-33-te-m0", + "case_id": "fresh-33-te", + "scenario": "fresh-33", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2343, + "completion_tokens": 1411, + "known_cost_microinr": 119805, + "retained_microinr": 149757, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.608731583997724, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-te-m0", + "case_id": "fresh-33-te", + "scenario": "fresh-33", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2070, + "completion_tokens": 1583, + "known_cost_microinr": 126030, + "retained_microinr": 157538, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.178025500004878, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-25-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-25-hi-m1", + "case_id": "fresh-25-hi", + "scenario": "fresh-25", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2276, + "completion_tokens": 40, + "known_cost_microinr": 36540, + "retained_microinr": 45675, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0159181250055553, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-25-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-25-hi-m1", + "case_id": "fresh-25-hi", + "scenario": "fresh-25", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2276, + "completion_tokens": 40, + "known_cost_microinr": 36540, + "retained_microinr": 45675, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9898767919949023, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-25-hi-m1-source-chronological", + "history_id": "confirm-fresh-25-hi-m1", + "case_id": "fresh-25-hi", + "scenario": "fresh-25", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1501, + "completion_tokens": 40, + "known_cost_microinr": 24915, + "retained_microinr": 31144, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9505093749903608, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-hi-m1", + "case_id": "fresh-32-hi", + "scenario": "fresh-32", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2134, + "completion_tokens": 42, + "known_cost_microinr": 34530, + "retained_microinr": 43163, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9483322090090951, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-hi-m1-source-chronological", + "history_id": "confirm-fresh-32-hi-m1", + "case_id": "fresh-32-hi", + "scenario": "fresh-32", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1483, + "completion_tokens": 40, + "known_cost_microinr": 24645, + "retained_microinr": 30807, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9970947499969043, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-32-hi-m1", + "case_id": "fresh-32-hi", + "scenario": "fresh-32", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2134, + "completion_tokens": 42, + "known_cost_microinr": 34530, + "retained_microinr": 43163, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.94754966700566, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-15-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-15-hi-m1", + "case_id": "fresh-15-hi", + "scenario": "fresh-15", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-16", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-16", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1415, + "completion_tokens": 42, + "known_cost_microinr": 23745, + "retained_microinr": 29682, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0053978329960955, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-15-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-15-hi-m1", + "case_id": "fresh-15-hi", + "scenario": "fresh-15", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1559, + "completion_tokens": 32, + "known_cost_microinr": 25305, + "retained_microinr": 31632, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9945954170107143, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-15-hi-m1-source-chronological", + "history_id": "confirm-fresh-15-hi-m1", + "case_id": "fresh-15-hi", + "scenario": "fresh-15", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1014, + "completion_tokens": 32, + "known_cost_microinr": 17130, + "retained_microinr": 21413, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9794596249994356, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-hi-m1-source-chronological", + "history_id": "confirm-fresh-35-hi-m1", + "case_id": "fresh-35-hi", + "scenario": "fresh-35", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1927, + "completion_tokens": 42, + "known_cost_microinr": 31425, + "retained_microinr": 39282, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9489228330057813, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-hi-m1", + "case_id": "fresh-35-hi", + "scenario": "fresh-35", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2346, + "completion_tokens": 42, + "known_cost_microinr": 37710, + "retained_microinr": 47138, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0076109580113553, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-35-hi-m1", + "case_id": "fresh-35-hi", + "scenario": "fresh-35", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2604, + "completion_tokens": 42, + "known_cost_microinr": 41580, + "retained_microinr": 51975, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.985458541996195, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-20-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-20-hi-m1", + "case_id": "fresh-20-hi", + "scenario": "fresh-20", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1581, + "completion_tokens": 42, + "known_cost_microinr": 26235, + "retained_microinr": 32794, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9312327090010513, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-20-hi-m1-source-chronological", + "history_id": "confirm-fresh-20-hi-m1", + "case_id": "fresh-20-hi", + "scenario": "fresh-20", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-03-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1052, + "completion_tokens": 42, + "known_cost_microinr": 18300, + "retained_microinr": 22875, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0072423749952577, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-20-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-20-hi-m1", + "case_id": "fresh-20-hi", + "scenario": "fresh-20", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1643, + "completion_tokens": 42, + "known_cost_microinr": 27165, + "retained_microinr": 33957, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.017433583008824, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-40-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-en-m1", + "case_id": "fresh-40-en", + "scenario": "fresh-40", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2081, + "completion_tokens": 42, + "known_cost_microinr": 33735, + "retained_microinr": 42169, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9788850420009112, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-en-m1-source-chronological", + "history_id": "confirm-fresh-40-en-m1", + "case_id": "fresh-40-en", + "scenario": "fresh-40", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1833, + "completion_tokens": 42, + "known_cost_microinr": 30015, + "retained_microinr": 37519, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0217821249971166, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-40-en-m1", + "case_id": "fresh-40-en", + "scenario": "fresh-40", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2401, + "completion_tokens": 42, + "known_cost_microinr": 38535, + "retained_microinr": 48169, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9563269999925978, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-21-en-m0", + "case_id": "fresh-21-en", + "scenario": "fresh-21", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1978, + "completion_tokens": 1788, + "known_cost_microinr": 136950, + "retained_microinr": 171188, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.55571850000706, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-en-m0-source-chronological", + "history_id": "confirm-fresh-21-en-m0", + "case_id": "fresh-21-en", + "scenario": "fresh-21", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1358, + "completion_tokens": 1280, + "known_cost_microinr": 97170, + "retained_microinr": 121463, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 5.893801291007549, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-21-en-m0", + "case_id": "fresh-21-en", + "scenario": "fresh-21", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1978, + "completion_tokens": 4203, + "known_cost_microinr": 281850, + "retained_microinr": 352313, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 21.536080665988266, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-37-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-37-hi-m0", + "case_id": "fresh-37-hi", + "scenario": "fresh-37", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2252, + "completion_tokens": 3212, + "known_cost_microinr": 226500, + "retained_microinr": 283125, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 16.394425874997978, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-37-hi-m0-source-chronological", + "history_id": "confirm-fresh-37-hi-m0", + "case_id": "fresh-37-hi", + "scenario": "fresh-37", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1723, + "completion_tokens": 2554, + "known_cost_microinr": 179085, + "retained_microinr": 223857, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.25975662500423, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-37-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-hi-m0", + "case_id": "fresh-37-hi", + "scenario": "fresh-37", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1945, + "completion_tokens": 8192, + "known_cost_microinr": 520695, + "retained_microinr": 650869, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 37.77549379199627, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-28-te-m0-source-chronological", + "history_id": "confirm-fresh-28-te-m0", + "case_id": "fresh-28-te", + "scenario": "fresh-28", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1339, + "completion_tokens": 519, + "known_cost_microinr": 51225, + "retained_microinr": 64032, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 3.13366279201, + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-28-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-28-te-m0", + "case_id": "fresh-28-te", + "scenario": "fresh-28", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1837, + "completion_tokens": 6869, + "known_cost_microinr": 439695, + "retained_microinr": 549619, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 31.018074499996146, + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-28-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-28-te-m0", + "case_id": "fresh-28-te", + "scenario": "fresh-28", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1837, + "completion_tokens": 4171, + "known_cost_microinr": 277815, + "retained_microinr": 347269, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 23.696498665987747, + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-07-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-te-m0", + "case_id": "fresh-07-te", + "scenario": "fresh-07", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1400, + "completion_tokens": 3578, + "known_cost_microinr": 235680, + "retained_microinr": 294600, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 19.214640292004333, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-07-te-m0-source-chronological", + "history_id": "confirm-fresh-07-te-m0", + "case_id": "fresh-07-te", + "scenario": "fresh-07", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 979, + "completion_tokens": 3153, + "known_cost_microinr": 203865, + "retained_microinr": 254832, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 19.295718249995844, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-07-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-07-te-m0", + "case_id": "fresh-07-te", + "scenario": "fresh-07", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1527, + "completion_tokens": 5871, + "known_cost_microinr": 375165, + "retained_microinr": 468957, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 27.6655987080012, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-09-te-m0-source-chronological", + "history_id": "confirm-fresh-09-te-m0", + "case_id": "fresh-09-te", + "scenario": "fresh-09", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1056, + "completion_tokens": 1766, + "known_cost_microinr": 121800, + "retained_microinr": 152250, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.148343624998233, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-09-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-09-te-m0", + "case_id": "fresh-09-te", + "scenario": "fresh-09", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1600, + "completion_tokens": 1840, + "known_cost_microinr": 134400, + "retained_microinr": 168000, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.543137667002156, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-09-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-09-te-m0", + "case_id": "fresh-09-te", + "scenario": "fresh-09", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1405, + "completion_tokens": 2869, + "known_cost_microinr": 193215, + "retained_microinr": 241519, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 17.790086916997097, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-te-m1-source-chronological", + "history_id": "confirm-fresh-03-te-m1", + "case_id": "fresh-03-te", + "scenario": "fresh-03", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 887, + "completion_tokens": 42, + "known_cost_microinr": 15825, + "retained_microinr": 19782, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.951432209010818, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-03-te-m1", + "case_id": "fresh-03-te", + "scenario": "fresh-03", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1412, + "completion_tokens": 42, + "known_cost_microinr": 23700, + "retained_microinr": 29625, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0183845830033533, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-03-te-m1", + "case_id": "fresh-03-te", + "scenario": "fresh-03", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1435, + "completion_tokens": 42, + "known_cost_microinr": 24045, + "retained_microinr": 30057, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0147983750066487, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-03-en-m1", + "case_id": "fresh-03-en", + "scenario": "fresh-03", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1862, + "completion_tokens": 42, + "known_cost_microinr": 30450, + "retained_microinr": 38063, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0982950840116246, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-03-en-m1", + "case_id": "fresh-03-en", + "scenario": "fresh-03", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": true, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1696, + "completion_tokens": 32, + "known_cost_microinr": 27360, + "retained_microinr": 34200, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.008163708000211, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-03-en-m1-source-chronological", + "history_id": "confirm-fresh-03-en-m1", + "case_id": "fresh-03-en", + "scenario": "fresh-03", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1159, + "completion_tokens": 42, + "known_cost_microinr": 19905, + "retained_microinr": 24882, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9332312920014374, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-28-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-28-en-m1", + "case_id": "fresh-28-en", + "scenario": "fresh-28", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2106, + "completion_tokens": 40, + "known_cost_microinr": 33990, + "retained_microinr": 42488, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0081932500033872, + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-28-en-m1-source-chronological", + "history_id": "confirm-fresh-28-en-m1", + "case_id": "fresh-28-en", + "scenario": "fresh-28", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1454, + "completion_tokens": 40, + "known_cost_microinr": 24210, + "retained_microinr": 30263, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9394343329913681, + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-28-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-28-en-m1", + "case_id": "fresh-28-en", + "scenario": "fresh-28", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2106, + "completion_tokens": 40, + "known_cost_microinr": 33990, + "retained_microinr": 42488, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0145768340007635, + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-22-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-22-hi-m0", + "case_id": "fresh-22-hi", + "scenario": "fresh-22", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1826, + "completion_tokens": 3039, + "known_cost_microinr": 209730, + "retained_microinr": 262163, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.696418833002099, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-22-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-22-hi-m0", + "case_id": "fresh-22-hi", + "scenario": "fresh-22", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1826, + "completion_tokens": 4713, + "known_cost_microinr": 310170, + "retained_microinr": 387713, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 21.88700395800697, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-22-hi-m0-source-chronological", + "history_id": "confirm-fresh-22-hi-m0", + "case_id": "fresh-22-hi", + "scenario": "fresh-22", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1317, + "completion_tokens": 2169, + "known_cost_microinr": 149895, + "retained_microinr": 187369, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.832497875002446, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-04-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-04-en-m0", + "case_id": "fresh-04-en", + "scenario": "fresh-04", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1662, + "completion_tokens": 2468, + "known_cost_microinr": 173010, + "retained_microinr": 216263, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.226466457999777, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-04-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-04-en-m0", + "case_id": "fresh-04-en", + "scenario": "fresh-04", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1530, + "completion_tokens": 5316, + "known_cost_microinr": 341910, + "retained_microinr": 427388, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 26.199195125009282, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-04-en-m0-source-chronological", + "history_id": "confirm-fresh-04-en-m0", + "case_id": "fresh-04-en", + "scenario": "fresh-04", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1086, + "completion_tokens": 5668, + "known_cost_microinr": 356370, + "retained_microinr": 445463, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 24.645635833003325, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-16-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-16-en-m0", + "case_id": "fresh-16-en", + "scenario": "fresh-16", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1371, + "completion_tokens": 3776, + "known_cost_microinr": 247125, + "retained_microinr": 308907, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 15.747625625008368, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-16-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-16-en-m0", + "case_id": "fresh-16-en", + "scenario": "fresh-16", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1269, + "completion_tokens": 4658, + "known_cost_microinr": 298515, + "retained_microinr": 373144, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 24.507196416001534, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-16-en-m0-source-chronological", + "history_id": "confirm-fresh-16-en-m0", + "case_id": "fresh-16-en", + "scenario": "fresh-16", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 928, + "completion_tokens": 3734, + "known_cost_microinr": 237960, + "retained_microinr": 297450, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 16.229188957993756, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-14-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-14-te-m0", + "case_id": "fresh-14-te", + "scenario": "fresh-14", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1264, + "completion_tokens": 1975, + "known_cost_microinr": 137460, + "retained_microinr": 171825, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.641674832993886, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-14-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-14-te-m0", + "case_id": "fresh-14-te", + "scenario": "fresh-14", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1540, + "completion_tokens": 6796, + "known_cost_microinr": 430860, + "retained_microinr": 538575, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 28.325404834002256, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-14-te-m0-source-chronological", + "history_id": "confirm-fresh-14-te-m0", + "case_id": "fresh-14-te", + "scenario": "fresh-14", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1101, + "completion_tokens": 4171, + "known_cost_microinr": 266775, + "retained_microinr": 333469, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 17.552688708005007, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-02-hi-m1", + "case_id": "fresh-02-hi", + "scenario": "fresh-02", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1254, + "completion_tokens": 42, + "known_cost_microinr": 21330, + "retained_microinr": 26663, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9385331249941373, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-hi-m1-source-chronological", + "history_id": "confirm-fresh-02-hi-m1", + "case_id": "fresh-02-hi", + "scenario": "fresh-02", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 833, + "completion_tokens": 42, + "known_cost_microinr": 15015, + "retained_microinr": 18769, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9410805420047836, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-02-hi-m1", + "case_id": "fresh-02-hi", + "scenario": "fresh-02", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1267, + "completion_tokens": 42, + "known_cost_microinr": 21525, + "retained_microinr": 26907, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0299618330027442, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-15-te-m0-source-chronological", + "history_id": "confirm-fresh-15-te-m0", + "case_id": "fresh-15-te", + "scenario": "fresh-15", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 990, + "completion_tokens": 6055, + "known_cost_microinr": 378150, + "retained_microinr": 472688, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 24.841930750000756, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-15-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-15-te-m0", + "case_id": "fresh-15-te", + "scenario": "fresh-15", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1533, + "completion_tokens": 1467, + "known_cost_microinr": 111015, + "retained_microinr": 138769, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.45636095800728, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-15-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-15-te-m0", + "case_id": "fresh-15-te", + "scenario": "fresh-15", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1408, + "completion_tokens": 2727, + "known_cost_microinr": 184740, + "retained_microinr": 230925, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.572422540994012, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-19-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-19-hi-m0", + "case_id": "fresh-19-hi", + "scenario": "fresh-19", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1757, + "completion_tokens": 4935, + "known_cost_microinr": 322455, + "retained_microinr": 403069, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 23.170734041006654, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-19-hi-m0-source-chronological", + "history_id": "confirm-fresh-19-hi-m0", + "case_id": "fresh-19-hi", + "scenario": "fresh-19", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1061, + "completion_tokens": 4656, + "known_cost_microinr": 295275, + "retained_microinr": 369094, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 23.127932041999884, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-19-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-hi-m0", + "case_id": "fresh-19-hi", + "scenario": "fresh-19", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-04-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1721, + "completion_tokens": 4959, + "known_cost_microinr": 323355, + "retained_microinr": 404194, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 22.36946070799604, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-15-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-15-en-m0", + "case_id": "fresh-15-en", + "scenario": "fresh-15", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1525, + "completion_tokens": 2629, + "known_cost_microinr": 180615, + "retained_microinr": 225769, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.286117167008342, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-15-en-m0-source-chronological", + "history_id": "confirm-fresh-15-en-m0", + "case_id": "fresh-15-en", + "scenario": "fresh-15", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 983, + "completion_tokens": 2069, + "known_cost_microinr": 138885, + "retained_microinr": 173607, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.08597883299808, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-15-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-15-en-m0", + "case_id": "fresh-15-en", + "scenario": "fresh-15", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1411, + "completion_tokens": 2934, + "known_cost_microinr": 197205, + "retained_microinr": 246507, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.364997417011182, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-24-en-m0", + "case_id": "fresh-24-en", + "scenario": "fresh-24", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1839, + "completion_tokens": 1113, + "known_cost_microinr": 94365, + "retained_microinr": 117957, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 5.330106167006306, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-24-en-m0", + "case_id": "fresh-24-en", + "scenario": "fresh-24", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1839, + "completion_tokens": 3958, + "known_cost_microinr": 265065, + "retained_microinr": 331332, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 16.913822749993415, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-en-m0-source-chronological", + "history_id": "confirm-fresh-24-en-m0", + "case_id": "fresh-24-en", + "scenario": "fresh-24", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1320, + "completion_tokens": 1400, + "known_cost_microinr": 103800, + "retained_microinr": 129750, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.214337208002689, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-te-m1-source-chronological", + "history_id": "confirm-fresh-32-te-m1", + "case_id": "fresh-32-te", + "scenario": "fresh-32", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1744, + "completion_tokens": 42, + "known_cost_microinr": 28680, + "retained_microinr": 35850, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9329089170059888, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-32-te-m1", + "case_id": "fresh-32-te", + "scenario": "fresh-32", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2324, + "completion_tokens": 42, + "known_cost_microinr": 37380, + "retained_microinr": 46725, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9693154159904225, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-te-m1", + "case_id": "fresh-32-te", + "scenario": "fresh-32", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1881, + "completion_tokens": 42, + "known_cost_microinr": 30735, + "retained_microinr": 38419, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0257342499971855, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-22-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-22-te-m1", + "case_id": "fresh-22-te", + "scenario": "fresh-22", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2123, + "completion_tokens": 40, + "known_cost_microinr": 34245, + "retained_microinr": 42807, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9548690419906052, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-22-te-m1-source-chronological", + "history_id": "confirm-fresh-22-te-m1", + "case_id": "fresh-22-te", + "scenario": "fresh-22", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1474, + "completion_tokens": 40, + "known_cost_microinr": 24510, + "retained_microinr": 30638, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0191509160067653, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-22-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-22-te-m1", + "case_id": "fresh-22-te", + "scenario": "fresh-22", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2123, + "completion_tokens": 40, + "known_cost_microinr": 34245, + "retained_microinr": 42807, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9699667910026619, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-08-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-08-en-m1", + "case_id": "fresh-08-en", + "scenario": "fresh-08", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1676, + "completion_tokens": 32, + "known_cost_microinr": 27060, + "retained_microinr": 33825, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0592645420110784, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-08-en-m1-source-chronological", + "history_id": "confirm-fresh-08-en-m1", + "case_id": "fresh-08-en", + "scenario": "fresh-08", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1094, + "completion_tokens": 32, + "known_cost_microinr": 18330, + "retained_microinr": 22913, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.005081416995381, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-08-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-08-en-m1", + "case_id": "fresh-08-en", + "scenario": "fresh-08", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1533, + "completion_tokens": 32, + "known_cost_microinr": 24915, + "retained_microinr": 31144, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9301751669991063, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-34-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-en-m1", + "case_id": "fresh-34-en", + "scenario": "fresh-34", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2189, + "completion_tokens": 42, + "known_cost_microinr": 35355, + "retained_microinr": 44194, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0531886250100797, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-34-en-m1-source-chronological", + "history_id": "confirm-fresh-34-en-m1", + "case_id": "fresh-34-en", + "scenario": "fresh-34", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1976, + "completion_tokens": 42, + "known_cost_microinr": 32160, + "retained_microinr": 40200, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.10543416699511, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-34-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-34-en-m1", + "case_id": "fresh-34-en", + "scenario": "fresh-34", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2553, + "completion_tokens": 42, + "known_cost_microinr": 40815, + "retained_microinr": 51019, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9658427920076065, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-37-en-m1-source-chronological", + "history_id": "confirm-fresh-37-en-m1", + "case_id": "fresh-37-en", + "scenario": "fresh-37", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1777, + "completion_tokens": 42, + "known_cost_microinr": 29175, + "retained_microinr": 36469, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1151200840104138, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-37-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-en-m1", + "case_id": "fresh-37-en", + "scenario": "fresh-37", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2240, + "completion_tokens": 42, + "known_cost_microinr": 36120, + "retained_microinr": 45150, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.015421415999299, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-37-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-37-en-m1", + "case_id": "fresh-37-en", + "scenario": "fresh-37", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2464, + "completion_tokens": 42, + "known_cost_microinr": 39480, + "retained_microinr": 49350, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.054733375000069, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-hi-m0-source-chronological", + "history_id": "confirm-fresh-36-hi-m0", + "case_id": "fresh-36-hi", + "scenario": "fresh-36", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1741, + "completion_tokens": 2847, + "known_cost_microinr": 196935, + "retained_microinr": 246169, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 17.856215332998545, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-hi-m0", + "case_id": "fresh-36-hi", + "scenario": "fresh-36", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1915, + "completion_tokens": 4892, + "known_cost_microinr": 322245, + "retained_microinr": 402807, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 27.32137474999763, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-36-hi-m0", + "case_id": "fresh-36-hi", + "scenario": "fresh-36", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2163, + "completion_tokens": 1462, + "known_cost_microinr": 120165, + "retained_microinr": 150207, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.776234791002935, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-10-en-m0", + "case_id": "fresh-10-en", + "scenario": "fresh-10", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1333, + "completion_tokens": 5837, + "known_cost_microinr": 370215, + "retained_microinr": 462769, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 31.27474337500462, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-10-en-m0", + "case_id": "fresh-10-en", + "scenario": "fresh-10", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1256, + "completion_tokens": 1566, + "known_cost_microinr": 112800, + "retained_microinr": 141000, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 10.808582208992448, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-en-m0-source-chronological", + "history_id": "confirm-fresh-10-en-m0", + "case_id": "fresh-10-en", + "scenario": "fresh-10", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 896, + "completion_tokens": 1391, + "known_cost_microinr": 96900, + "retained_microinr": 121125, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.540584332993603, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-09-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-09-en-m0", + "case_id": "fresh-09-en", + "scenario": "fresh-09", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1443, + "completion_tokens": 2042, + "known_cost_microinr": 144165, + "retained_microinr": 180207, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.118661833010265, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-09-en-m0-source-chronological", + "history_id": "confirm-fresh-09-en-m0", + "case_id": "fresh-09-en", + "scenario": "fresh-09", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 902, + "completion_tokens": 5875, + "known_cost_microinr": 366030, + "retained_microinr": 457538, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 24.437354834008147, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-09-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-09-en-m0", + "case_id": "fresh-09-en", + "scenario": "fresh-09", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1404, + "completion_tokens": 1999, + "known_cost_microinr": 141000, + "retained_microinr": 176250, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.674536040998646, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-10-hi-m0", + "case_id": "fresh-10-hi", + "scenario": "fresh-10", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1253, + "completion_tokens": 4617, + "known_cost_microinr": 295815, + "retained_microinr": 369769, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 19.370352083002217, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-10-hi-m0", + "case_id": "fresh-10-hi", + "scenario": "fresh-10", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1309, + "completion_tokens": 1787, + "known_cost_microinr": 126855, + "retained_microinr": 158569, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.972530416998779, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-hi-m0-source-chronological", + "history_id": "confirm-fresh-10-hi-m0", + "case_id": "fresh-10-hi", + "scenario": "fresh-10", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 873, + "completion_tokens": 3009, + "known_cost_microinr": 193635, + "retained_microinr": 242044, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 14.16674320900347, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-14-te-m1-source-chronological", + "history_id": "confirm-fresh-14-te-m1", + "case_id": "fresh-14-te", + "scenario": "fresh-14", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-15", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-15", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1126, + "completion_tokens": 42, + "known_cost_microinr": 19410, + "retained_microinr": 24263, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9414785420085536, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-14-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-14-te-m1", + "case_id": "fresh-14-te", + "scenario": "fresh-14", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-15", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-15", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1709, + "completion_tokens": 42, + "known_cost_microinr": 28155, + "retained_microinr": 35194, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.967218708989094, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-14-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-14-te-m1", + "case_id": "fresh-14-te", + "scenario": "fresh-14", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1550, + "completion_tokens": 32, + "known_cost_microinr": 25170, + "retained_microinr": 31463, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.971434709004825, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-30-hi-m1", + "case_id": "fresh-30-hi", + "scenario": "fresh-30", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2143, + "completion_tokens": 40, + "known_cost_microinr": 34545, + "retained_microinr": 43182, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.103579291011556, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-hi-m1-source-chronological", + "history_id": "confirm-fresh-30-hi-m1", + "case_id": "fresh-30-hi", + "scenario": "fresh-30", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1482, + "completion_tokens": 40, + "known_cost_microinr": 24630, + "retained_microinr": 30788, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0088979579886654, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-hi-m1", + "case_id": "fresh-30-hi", + "scenario": "fresh-30", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2143, + "completion_tokens": 40, + "known_cost_microinr": 34545, + "retained_microinr": 43182, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9637627079937374, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-31-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-31-hi-m1", + "case_id": "fresh-31-hi", + "scenario": "fresh-31", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2280, + "completion_tokens": 40, + "known_cost_microinr": 36600, + "retained_microinr": 45750, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9493672079988755, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-31-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-31-hi-m1", + "case_id": "fresh-31-hi", + "scenario": "fresh-31", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2280, + "completion_tokens": 40, + "known_cost_microinr": 36600, + "retained_microinr": 45750, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0902935000049183, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-31-hi-m1-source-chronological", + "history_id": "confirm-fresh-31-hi-m1", + "case_id": "fresh-31-hi", + "scenario": "fresh-31", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1522, + "completion_tokens": 40, + "known_cost_microinr": 25230, + "retained_microinr": 31538, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9350680000061402, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-39-en-m1", + "case_id": "fresh-39-en", + "scenario": "fresh-39", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2420, + "completion_tokens": 42, + "known_cost_microinr": 38820, + "retained_microinr": 48525, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0383790409978246, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-en-m1", + "case_id": "fresh-39-en", + "scenario": "fresh-39", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2225, + "completion_tokens": 42, + "known_cost_microinr": 35895, + "retained_microinr": 44869, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9986680000001797, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-en-m1-source-chronological", + "history_id": "confirm-fresh-39-en-m1", + "case_id": "fresh-39-en", + "scenario": "fresh-39", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1749, + "completion_tokens": 42, + "known_cost_microinr": 28755, + "retained_microinr": 35944, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1048080409964314, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-21-en-m1", + "case_id": "fresh-21-en", + "scenario": "fresh-21", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2263, + "completion_tokens": 40, + "known_cost_microinr": 36345, + "retained_microinr": 45432, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9594374159933068, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-21-en-m1", + "case_id": "fresh-21-en", + "scenario": "fresh-21", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2263, + "completion_tokens": 40, + "known_cost_microinr": 36345, + "retained_microinr": 45432, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1167157919990132, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-en-m1-source-chronological", + "history_id": "confirm-fresh-21-en-m1", + "case_id": "fresh-21-en", + "scenario": "fresh-21", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1504, + "completion_tokens": 40, + "known_cost_microinr": 24960, + "retained_microinr": 31200, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0056834999995772, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-13-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-13-te-m1", + "case_id": "fresh-13-te", + "scenario": "fresh-13", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1698, + "completion_tokens": 32, + "known_cost_microinr": 27390, + "retained_microinr": 34238, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0096665830060374, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-13-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-13-te-m1", + "case_id": "fresh-13-te", + "scenario": "fresh-13", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1872, + "completion_tokens": 32, + "known_cost_microinr": 30000, + "retained_microinr": 37500, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1219505830085836, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-13-te-m1-source-chronological", + "history_id": "confirm-fresh-13-te-m1", + "case_id": "fresh-13-te", + "scenario": "fresh-13", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1175, + "completion_tokens": 42, + "known_cost_microinr": 20145, + "retained_microinr": 25182, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9357774579984834, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-21-te-m1-source-chronological", + "history_id": "confirm-fresh-21-te-m1", + "case_id": "fresh-21-te", + "scenario": "fresh-21", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1514, + "completion_tokens": 40, + "known_cost_microinr": 25110, + "retained_microinr": 31388, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9338703329995042, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-21-te-m1", + "case_id": "fresh-21-te", + "scenario": "fresh-21", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2266, + "completion_tokens": 40, + "known_cost_microinr": 36390, + "retained_microinr": 45488, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9539970839978196, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-21-te-m1", + "case_id": "fresh-21-te", + "scenario": "fresh-21", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2266, + "completion_tokens": 40, + "known_cost_microinr": 36390, + "retained_microinr": 45488, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.974952583987033, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-20-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-20-hi-m0", + "case_id": "fresh-20-hi", + "scenario": "fresh-20", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1348, + "completion_tokens": 8192, + "known_cost_microinr": 511740, + "retained_microinr": 639675, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 40.547644749996834, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-20-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-20-hi-m0", + "case_id": "fresh-20-hi", + "scenario": "fresh-20", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1285, + "completion_tokens": 1221, + "known_cost_microinr": 92535, + "retained_microinr": 115669, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.446608832993661, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-20-hi-m0-source-chronological", + "history_id": "confirm-fresh-20-hi-m0", + "case_id": "fresh-20-hi", + "scenario": "fresh-20", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 911, + "completion_tokens": 4492, + "known_cost_microinr": 283185, + "retained_microinr": 353982, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 23.06852745800279, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-19-en-m0-source-chronological", + "history_id": "confirm-fresh-19-en-m0", + "case_id": "fresh-19-en", + "scenario": "fresh-19", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1100, + "completion_tokens": 6910, + "known_cost_microinr": 431100, + "retained_microinr": 538875, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 28.784824792004656, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-19-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-19-en-m0", + "case_id": "fresh-19-en", + "scenario": "fresh-19", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1785, + "completion_tokens": 8192, + "known_cost_microinr": 518295, + "retained_microinr": 647869, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 38.58736979099922, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-19-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-en-m0", + "case_id": "fresh-19-en", + "scenario": "fresh-19", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1706, + "completion_tokens": 1853, + "known_cost_microinr": 136770, + "retained_microinr": 170963, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.911343084007967, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-te-m1-source-chronological", + "history_id": "confirm-fresh-38-te-m1", + "case_id": "fresh-38-te", + "scenario": "fresh-38", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1833, + "completion_tokens": 40, + "known_cost_microinr": 29895, + "retained_microinr": 37369, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9656026250013383, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-38-te-m1", + "case_id": "fresh-38-te", + "scenario": "fresh-38", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2418, + "completion_tokens": 40, + "known_cost_microinr": 38670, + "retained_microinr": 48338, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0040276660001837, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-te-m1", + "case_id": "fresh-38-te", + "scenario": "fresh-38", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2112, + "completion_tokens": 42, + "known_cost_microinr": 34200, + "retained_microinr": 42750, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0110378749959636, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-en-m1", + "case_id": "fresh-35-en", + "scenario": "fresh-35", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2342, + "completion_tokens": 42, + "known_cost_microinr": 37650, + "retained_microinr": 47063, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.958510665994254, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-35-en-m1", + "case_id": "fresh-35-en", + "scenario": "fresh-35", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2541, + "completion_tokens": 42, + "known_cost_microinr": 40635, + "retained_microinr": 50794, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0844774160068482, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-en-m1-source-chronological", + "history_id": "confirm-fresh-35-en-m1", + "case_id": "fresh-35-en", + "scenario": "fresh-35", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1856, + "completion_tokens": 42, + "known_cost_microinr": 30360, + "retained_microinr": 37950, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0406185000028927, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-16-te-m0-source-chronological", + "history_id": "confirm-fresh-16-te-m0", + "case_id": "fresh-16-te", + "scenario": "fresh-16", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 885, + "completion_tokens": 8192, + "known_cost_microinr": 504795, + "retained_microinr": 630994, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 33.825618667004164, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-16-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-16-te-m0", + "case_id": "fresh-16-te", + "scenario": "fresh-16", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1265, + "completion_tokens": 2069, + "known_cost_microinr": 143115, + "retained_microinr": 178894, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.044318750005914, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-16-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-16-te-m0", + "case_id": "fresh-16-te", + "scenario": "fresh-16", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1325, + "completion_tokens": 3405, + "known_cost_microinr": 224175, + "retained_microinr": 280219, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 14.300099958010833, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-15-te-m1-source-chronological", + "history_id": "confirm-fresh-15-te-m1", + "case_id": "fresh-15-te", + "scenario": "fresh-15", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1174, + "completion_tokens": 42, + "known_cost_microinr": 20130, + "retained_microinr": 25163, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.131959584003198, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-15-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-15-te-m1", + "case_id": "fresh-15-te", + "scenario": "fresh-15", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1689, + "completion_tokens": 32, + "known_cost_microinr": 27255, + "retained_microinr": 34069, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.2939245830057189, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-15-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-15-te-m1", + "case_id": "fresh-15-te", + "scenario": "fresh-15", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-16", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-16", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1858, + "completion_tokens": 42, + "known_cost_microinr": 30390, + "retained_microinr": 37988, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9855014170025242, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-05-hi-m1-source-chronological", + "history_id": "confirm-fresh-05-hi-m1", + "case_id": "fresh-05-hi", + "scenario": "fresh-05", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1009, + "completion_tokens": 32, + "known_cost_microinr": 17055, + "retained_microinr": 21319, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.278489417003584, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-05-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-05-hi-m1", + "case_id": "fresh-05-hi", + "scenario": "fresh-05", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1675, + "completion_tokens": 32, + "known_cost_microinr": 27045, + "retained_microinr": 33807, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9504380830039736, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-05-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-05-hi-m1", + "case_id": "fresh-05-hi", + "scenario": "fresh-05", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1702, + "completion_tokens": 32, + "known_cost_microinr": 27450, + "retained_microinr": 34313, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.948037000009208, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-hi-m1", + "case_id": "fresh-33-hi", + "scenario": "fresh-33", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2365, + "completion_tokens": 42, + "known_cost_microinr": 37995, + "retained_microinr": 47494, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0529548339982284, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-33-hi-m1", + "case_id": "fresh-33-hi", + "scenario": "fresh-33", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2675, + "completion_tokens": 42, + "known_cost_microinr": 42645, + "retained_microinr": 53307, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9673166660068091, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-hi-m1-source-chronological", + "history_id": "confirm-fresh-33-hi-m1", + "case_id": "fresh-33-hi", + "scenario": "fresh-33", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1980, + "completion_tokens": 42, + "known_cost_microinr": 32220, + "retained_microinr": 40275, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9630600000091363, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-05-hi-m0-source-chronological", + "history_id": "confirm-fresh-05-hi-m0", + "case_id": "fresh-05-hi", + "scenario": "fresh-05", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 896, + "completion_tokens": 6769, + "known_cost_microinr": 419580, + "retained_microinr": 524475, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 27.827692208011285, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-05-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-05-hi-m0", + "case_id": "fresh-05-hi", + "scenario": "fresh-05", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1448, + "completion_tokens": 1698, + "known_cost_microinr": 123600, + "retained_microinr": 154500, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.330110625000088, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-05-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-05-hi-m0", + "case_id": "fresh-05-hi", + "scenario": "fresh-05", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1398, + "completion_tokens": 2748, + "known_cost_microinr": 185850, + "retained_microinr": 232313, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.720331250005984, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-hi-m1-source-chronological", + "history_id": "confirm-fresh-40-hi-m1", + "case_id": "fresh-40-hi", + "scenario": "fresh-40", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2036, + "completion_tokens": 42, + "known_cost_microinr": 33060, + "retained_microinr": 41325, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0304211670008954, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-40-hi-m1", + "case_id": "fresh-40-hi", + "scenario": "fresh-40", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2613, + "completion_tokens": 42, + "known_cost_microinr": 41715, + "retained_microinr": 52144, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9814349160005804, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-hi-m1", + "case_id": "fresh-40-hi", + "scenario": "fresh-40", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2103, + "completion_tokens": 42, + "known_cost_microinr": 34065, + "retained_microinr": 42582, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9619969159975881, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-te-m0-source-chronological", + "history_id": "confirm-fresh-21-te-m0", + "case_id": "fresh-21-te", + "scenario": "fresh-21", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1370, + "completion_tokens": 580, + "known_cost_microinr": 55350, + "retained_microinr": 69188, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 3.5836629169934895, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-21-te-m0", + "case_id": "fresh-21-te", + "scenario": "fresh-21", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1973, + "completion_tokens": 7582, + "known_cost_microinr": 484515, + "retained_microinr": 605644, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 36.53783091700461, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-21-te-m0", + "case_id": "fresh-21-te", + "scenario": "fresh-21", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1973, + "completion_tokens": 2392, + "known_cost_microinr": 173115, + "retained_microinr": 216394, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 10.647001292003551, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-14-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-14-hi-m0", + "case_id": "fresh-14-hi", + "scenario": "fresh-14", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1717, + "completion_tokens": 3048, + "known_cost_microinr": 208635, + "retained_microinr": 260794, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.867242167005315, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-14-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-14-hi-m0", + "case_id": "fresh-14-hi", + "scenario": "fresh-14", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1542, + "completion_tokens": 3440, + "known_cost_microinr": 229530, + "retained_microinr": 286913, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 14.504234167005052, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-14-hi-m0-source-chronological", + "history_id": "confirm-fresh-14-hi-m0", + "case_id": "fresh-14-hi", + "scenario": "fresh-14", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1133, + "completion_tokens": 8192, + "known_cost_microinr": 508515, + "retained_microinr": 635644, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 35.58632562500134, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-08-te-m1-source-chronological", + "history_id": "confirm-fresh-08-te-m1", + "case_id": "fresh-08-te", + "scenario": "fresh-08", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 896, + "completion_tokens": 32, + "known_cost_microinr": 15360, + "retained_microinr": 19200, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0703550420003012, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-08-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-08-te-m1", + "case_id": "fresh-08-te", + "scenario": "fresh-08", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1330, + "completion_tokens": 32, + "known_cost_microinr": 21870, + "retained_microinr": 27338, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9840972910023993, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-08-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-08-te-m1", + "case_id": "fresh-08-te", + "scenario": "fresh-08", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1253, + "completion_tokens": 32, + "known_cost_microinr": 20715, + "retained_microinr": 25894, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9449477090092842, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-09-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-09-hi-m1", + "case_id": "fresh-09-hi", + "scenario": "fresh-09", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": true, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1410, + "completion_tokens": 32, + "known_cost_microinr": 23070, + "retained_microinr": 28838, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9520849999971688, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-09-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-09-hi-m1", + "case_id": "fresh-09-hi", + "scenario": "fresh-09", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1648, + "completion_tokens": 42, + "known_cost_microinr": 27240, + "retained_microinr": 34050, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0129029590025311, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-09-hi-m1-source-chronological", + "history_id": "confirm-fresh-09-hi-m1", + "case_id": "fresh-09-hi", + "scenario": "fresh-09", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1104, + "completion_tokens": 42, + "known_cost_microinr": 19080, + "retained_microinr": 23850, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0904486250074115, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-hi-m0", + "case_id": "fresh-39-hi", + "scenario": "fresh-39", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2228, + "completion_tokens": 4029, + "known_cost_microinr": 275160, + "retained_microinr": 343950, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 18.592174874997, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-hi-m0-source-chronological", + "history_id": "confirm-fresh-39-hi-m0", + "case_id": "fresh-39-hi", + "scenario": "fresh-39", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1835, + "completion_tokens": 1275, + "known_cost_microinr": 104025, + "retained_microinr": 130032, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 5.953728917011176, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-39-hi-m0", + "case_id": "fresh-39-hi", + "scenario": "fresh-39", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 2514, + "completion_tokens": 8192, + "known_cost_microinr": 529230, + "retained_microinr": 661538, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 33.87376287500956, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-27-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-27-hi-m0", + "case_id": "fresh-27-hi", + "scenario": "fresh-27", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2274, + "completion_tokens": 2027, + "known_cost_microinr": 155730, + "retained_microinr": 194663, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 10.957922707995749, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-27-hi-m0-source-chronological", + "history_id": "confirm-fresh-27-hi-m0", + "case_id": "fresh-27-hi", + "scenario": "fresh-27", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1534, + "completion_tokens": 581, + "known_cost_microinr": 57870, + "retained_microinr": 72338, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 3.302994249999756, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-27-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-27-hi-m0", + "case_id": "fresh-27-hi", + "scenario": "fresh-27", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2306, + "completion_tokens": 2855, + "known_cost_microinr": 205890, + "retained_microinr": 257363, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.801137583010131, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-24-te-m1", + "case_id": "fresh-24-te", + "scenario": "fresh-24", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2120, + "completion_tokens": 40, + "known_cost_microinr": 34200, + "retained_microinr": 42750, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9508562080009142, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-te-m1-source-chronological", + "history_id": "confirm-fresh-24-te-m1", + "case_id": "fresh-24-te", + "scenario": "fresh-24", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1472, + "completion_tokens": 40, + "known_cost_microinr": 24480, + "retained_microinr": 30600, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0220702090009581, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-24-te-m1", + "case_id": "fresh-24-te", + "scenario": "fresh-24", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2120, + "completion_tokens": 40, + "known_cost_microinr": 34200, + "retained_microinr": 42750, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0569155420089373, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-18-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-te-m1", + "case_id": "fresh-18-te", + "scenario": "fresh-18", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1571, + "completion_tokens": 42, + "known_cost_microinr": 26085, + "retained_microinr": 32607, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9564855829958105, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-18-te-m1-source-chronological", + "history_id": "confirm-fresh-18-te-m1", + "case_id": "fresh-18-te", + "scenario": "fresh-18", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1025, + "completion_tokens": 42, + "known_cost_microinr": 17895, + "retained_microinr": 22369, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9875284579902655, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-18-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-18-te-m1", + "case_id": "fresh-18-te", + "scenario": "fresh-18", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-03-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1606, + "completion_tokens": 40, + "known_cost_microinr": 26490, + "retained_microinr": 33113, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9496736669971142, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-17-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-17-en-m0", + "case_id": "fresh-17-en", + "scenario": "fresh-17", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1696, + "completion_tokens": 7944, + "known_cost_microinr": 502080, + "retained_microinr": 627600, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 37.106685499995365, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-17-en-m0-source-chronological", + "history_id": "confirm-fresh-17-en-m0", + "case_id": "fresh-17-en", + "scenario": "fresh-17", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1054, + "completion_tokens": 5785, + "known_cost_microinr": 362910, + "retained_microinr": 453638, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 30.019876750011463, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-17-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-17-en-m0", + "case_id": "fresh-17-en", + "scenario": "fresh-17", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1728, + "completion_tokens": 5006, + "known_cost_microinr": 326280, + "retained_microinr": 407850, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 21.425518083997304, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-19-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-te-m0", + "case_id": "fresh-19-te", + "scenario": "fresh-19", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1711, + "completion_tokens": 2823, + "known_cost_microinr": 195045, + "retained_microinr": 243807, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.417185833997792, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-19-te-m0-source-chronological", + "history_id": "confirm-fresh-19-te-m0", + "case_id": "fresh-19-te", + "scenario": "fresh-19", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-03-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1052, + "completion_tokens": 3095, + "known_cost_microinr": 201480, + "retained_microinr": 251850, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.99166666601377, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-19-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-19-te-m0", + "case_id": "fresh-19-te", + "scenario": "fresh-19", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1739, + "completion_tokens": 7697, + "known_cost_microinr": 487905, + "retained_microinr": 609882, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 31.99121137498878, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-20-te-m1-source-chronological", + "history_id": "confirm-fresh-20-te-m1", + "case_id": "fresh-20-te", + "scenario": "fresh-20", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 896, + "completion_tokens": 42, + "known_cost_microinr": 15960, + "retained_microinr": 19950, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9413330840034178, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-20-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-20-te-m1", + "case_id": "fresh-20-te", + "scenario": "fresh-20", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1339, + "completion_tokens": 42, + "known_cost_microinr": 22605, + "retained_microinr": 28257, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0834329999925103, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-20-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-20-te-m1", + "case_id": "fresh-20-te", + "scenario": "fresh-20", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1295, + "completion_tokens": 42, + "known_cost_microinr": 21945, + "retained_microinr": 27432, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9437112919986248, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-37-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-te-m0", + "case_id": "fresh-37-te", + "scenario": "fresh-37", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1958, + "completion_tokens": 4962, + "known_cost_microinr": 327090, + "retained_microinr": 408863, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 21.5542608340038, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-37-te-m0-source-chronological", + "history_id": "confirm-fresh-37-te-m0", + "case_id": "fresh-37-te", + "scenario": "fresh-37", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1673, + "completion_tokens": 2243, + "known_cost_microinr": 159675, + "retained_microinr": 199594, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 10.327790666997316, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-37-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-37-te-m0", + "case_id": "fresh-37-te", + "scenario": "fresh-37", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2209, + "completion_tokens": 2117, + "known_cost_microinr": 160155, + "retained_microinr": 200194, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.33684879100474, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-12-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-12-te-m0", + "case_id": "fresh-12-te", + "scenario": "fresh-12", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1322, + "completion_tokens": 1786, + "known_cost_microinr": 126990, + "retained_microinr": 158738, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.740286917003687, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-12-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-te-m0", + "case_id": "fresh-12-te", + "scenario": "fresh-12", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1258, + "completion_tokens": 1413, + "known_cost_microinr": 103650, + "retained_microinr": 129563, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.485699208002188, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-12-te-m0-source-chronological", + "history_id": "confirm-fresh-12-te-m0", + "case_id": "fresh-12-te", + "scenario": "fresh-12", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 881, + "completion_tokens": 2524, + "known_cost_microinr": 164655, + "retained_microinr": 205819, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.449140957993222, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-en-m0-source-chronological", + "history_id": "confirm-fresh-38-en-m0", + "case_id": "fresh-38-en", + "scenario": "fresh-38", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1470, + "completion_tokens": 4638, + "known_cost_microinr": 300330, + "retained_microinr": 375413, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 22.274877541989554, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-en-m0", + "case_id": "fresh-38-en", + "scenario": "fresh-38", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1787, + "completion_tokens": 2231, + "known_cost_microinr": 160665, + "retained_microinr": 200832, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 10.868084208996152, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-38-en-m0", + "case_id": "fresh-38-en", + "scenario": "fresh-38", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1888, + "completion_tokens": 1950, + "known_cost_microinr": 145320, + "retained_microinr": 181650, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.572698832998867, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-08-en-m0-source-chronological", + "history_id": "confirm-fresh-08-en-m0", + "case_id": "fresh-08-en", + "scenario": "fresh-08", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1054, + "completion_tokens": 4772, + "known_cost_microinr": 302130, + "retained_microinr": 377663, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 19.70489283298957, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-08-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-08-en-m0", + "case_id": "fresh-08-en", + "scenario": "fresh-08", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1531, + "completion_tokens": 1175, + "known_cost_microinr": 93465, + "retained_microinr": 116832, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.5690467079984955, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-08-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-08-en-m0", + "case_id": "fresh-08-en", + "scenario": "fresh-08", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1640, + "completion_tokens": 8192, + "known_cost_microinr": 516120, + "retained_microinr": 645150, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 35.46933249999711, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-15-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-15-en-m1", + "case_id": "fresh-15-en", + "scenario": "fresh-15", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1893, + "completion_tokens": 42, + "known_cost_microinr": 30915, + "retained_microinr": 38644, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9615866669919342, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-15-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-15-en-m1", + "case_id": "fresh-15-en", + "scenario": "fresh-15", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1702, + "completion_tokens": 32, + "known_cost_microinr": 27450, + "retained_microinr": 34313, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.10548754199408, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-15-en-m1-source-chronological", + "history_id": "confirm-fresh-15-en-m1", + "case_id": "fresh-15-en", + "scenario": "fresh-15", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1204, + "completion_tokens": 42, + "known_cost_microinr": 20580, + "retained_microinr": 25725, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0101034170074854, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-14-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-14-hi-m1", + "case_id": "fresh-14-hi", + "scenario": "fresh-14", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1267, + "completion_tokens": 32, + "known_cost_microinr": 20925, + "retained_microinr": 26157, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 2.1586129169882042, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-14-hi-m1-source-chronological", + "history_id": "confirm-fresh-14-hi-m1", + "case_id": "fresh-14-hi", + "scenario": "fresh-14", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1048, + "completion_tokens": 32, + "known_cost_microinr": 17640, + "retained_microinr": 22050, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0028762909932993, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-14-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-14-hi-m1", + "case_id": "fresh-14-hi", + "scenario": "fresh-14", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1489, + "completion_tokens": 32, + "known_cost_microinr": 24255, + "retained_microinr": 30319, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0203545420081355, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-27-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-27-en-m1", + "case_id": "fresh-27-en", + "scenario": "fresh-27", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2271, + "completion_tokens": 40, + "known_cost_microinr": 36465, + "retained_microinr": 45582, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0953793749940814, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-27-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-27-en-m1", + "case_id": "fresh-27-en", + "scenario": "fresh-27", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2271, + "completion_tokens": 40, + "known_cost_microinr": 36465, + "retained_microinr": 45582, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9652028749987949, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-27-en-m1-source-chronological", + "history_id": "confirm-fresh-27-en-m1", + "case_id": "fresh-27-en", + "scenario": "fresh-27", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1505, + "completion_tokens": 40, + "known_cost_microinr": 24975, + "retained_microinr": 31219, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9303231249941746, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-02-hi-m0", + "case_id": "fresh-02-hi", + "scenario": "fresh-02", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1266, + "completion_tokens": 1730, + "known_cost_microinr": 122790, + "retained_microinr": 153488, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.481585165995057, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-02-hi-m0", + "case_id": "fresh-02-hi", + "scenario": "fresh-02", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1251, + "completion_tokens": 2619, + "known_cost_microinr": 175905, + "retained_microinr": 219882, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.416755999991437, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-hi-m0-source-chronological", + "history_id": "confirm-fresh-02-hi-m0", + "case_id": "fresh-02-hi", + "scenario": "fresh-02", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 831, + "completion_tokens": 2277, + "known_cost_microinr": 149085, + "retained_microinr": 186357, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.877363040999626, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-20-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-20-en-m1", + "case_id": "fresh-20-en", + "scenario": "fresh-20", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1705, + "completion_tokens": 42, + "known_cost_microinr": 28095, + "retained_microinr": 35119, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9762358750012936, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-20-en-m1-source-chronological", + "history_id": "confirm-fresh-20-en-m1", + "case_id": "fresh-20-en", + "scenario": "fresh-20", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1114, + "completion_tokens": 42, + "known_cost_microinr": 19230, + "retained_microinr": 24038, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0324379579978995, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-20-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-20-en-m1", + "case_id": "fresh-20-en", + "scenario": "fresh-20", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1577, + "completion_tokens": 42, + "known_cost_microinr": 26175, + "retained_microinr": 32719, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9410742500040215, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-07-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-07-hi-m1", + "case_id": "fresh-07-hi", + "scenario": "fresh-07", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1771, + "completion_tokens": 32, + "known_cost_microinr": 28485, + "retained_microinr": 35607, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0264889579993905, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-07-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-hi-m1", + "case_id": "fresh-07-hi", + "scenario": "fresh-07", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1691, + "completion_tokens": 32, + "known_cost_microinr": 27285, + "retained_microinr": 34107, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0180132090026746, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-07-hi-m1-source-chronological", + "history_id": "confirm-fresh-07-hi-m1", + "case_id": "fresh-07-hi", + "scenario": "fresh-07", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1071, + "completion_tokens": 32, + "known_cost_microinr": 17985, + "retained_microinr": 22482, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9350204999936977, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-40-en-m0", + "case_id": "fresh-40-en", + "scenario": "fresh-40", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2210, + "completion_tokens": 5613, + "known_cost_microinr": 369930, + "retained_microinr": 462413, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 26.328921082997113, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-en-m0", + "case_id": "fresh-40-en", + "scenario": "fresh-40", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2081, + "completion_tokens": 6266, + "known_cost_microinr": 407175, + "retained_microinr": 508969, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 33.38638883299427, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-en-m0-source-chronological", + "history_id": "confirm-fresh-40-en-m0", + "case_id": "fresh-40-en", + "scenario": "fresh-40", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1638, + "completion_tokens": 1223, + "known_cost_microinr": 97950, + "retained_microinr": 122438, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.129989791996195, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-01-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-01-hi-m1", + "case_id": "fresh-01-hi", + "scenario": "fresh-01", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1426, + "completion_tokens": 42, + "known_cost_microinr": 23910, + "retained_microinr": 29888, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9490464999980759, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-01-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-01-hi-m1", + "case_id": "fresh-01-hi", + "scenario": "fresh-01", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1413, + "completion_tokens": 42, + "known_cost_microinr": 23715, + "retained_microinr": 29644, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0074083329964196, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-01-hi-m1-source-chronological", + "history_id": "confirm-fresh-01-hi-m1", + "case_id": "fresh-01-hi", + "scenario": "fresh-01", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 871, + "completion_tokens": 42, + "known_cost_microinr": 15585, + "retained_microinr": 19482, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.023347125010332, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-05-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-05-te-m0", + "case_id": "fresh-05-te", + "scenario": "fresh-05", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1454, + "completion_tokens": 3471, + "known_cost_microinr": 230070, + "retained_microinr": 287588, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 16.40769549999095, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-05-te-m0-source-chronological", + "history_id": "confirm-fresh-05-te-m0", + "case_id": "fresh-05-te", + "scenario": "fresh-05", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 910, + "completion_tokens": 1502, + "known_cost_microinr": 103770, + "retained_microinr": 129713, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.898224082993693, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-05-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-05-te-m0", + "case_id": "fresh-05-te", + "scenario": "fresh-05", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1394, + "completion_tokens": 5093, + "known_cost_microinr": 326490, + "retained_microinr": 408113, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 22.103223125013756, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-hi-m1", + "case_id": "fresh-38-hi", + "scenario": "fresh-38", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2104, + "completion_tokens": 42, + "known_cost_microinr": 34080, + "retained_microinr": 42600, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9695861660002265, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-hi-m1-source-chronological", + "history_id": "confirm-fresh-38-hi-m1", + "case_id": "fresh-38-hi", + "scenario": "fresh-38", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1916, + "completion_tokens": 42, + "known_cost_microinr": 31260, + "retained_microinr": 39075, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0174255000019912, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-38-hi-m1", + "case_id": "fresh-38-hi", + "scenario": "fresh-38", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2505, + "completion_tokens": 42, + "known_cost_microinr": 40095, + "retained_microinr": 50119, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9591296659928048, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-01-en-m1-source-chronological", + "history_id": "confirm-fresh-01-en-m1", + "case_id": "fresh-01-en", + "scenario": "fresh-01", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1124, + "completion_tokens": 42, + "known_cost_microinr": 19380, + "retained_microinr": 24225, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9567032910126727, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-01-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-01-en-m1", + "case_id": "fresh-01-en", + "scenario": "fresh-01", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": true, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1693, + "completion_tokens": 32, + "known_cost_microinr": 27315, + "retained_microinr": 34144, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9767937080032425, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-01-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-01-en-m1", + "case_id": "fresh-01-en", + "scenario": "fresh-01", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1821, + "completion_tokens": 42, + "known_cost_microinr": 29835, + "retained_microinr": 37294, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9571239590004552, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-12-hi-m0-source-chronological", + "history_id": "confirm-fresh-12-hi-m0", + "case_id": "fresh-12-hi", + "scenario": "fresh-12", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 908, + "completion_tokens": 2958, + "known_cost_microinr": 191100, + "retained_microinr": 238875, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.609808041990618, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-12-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-hi-m0", + "case_id": "fresh-12-hi", + "scenario": "fresh-12", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1264, + "completion_tokens": 1869, + "known_cost_microinr": 131100, + "retained_microinr": 163875, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.178364875013358, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-12-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-12-hi-m0", + "case_id": "fresh-12-hi", + "scenario": "fresh-12", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1350, + "completion_tokens": 999, + "known_cost_microinr": 80190, + "retained_microinr": 100238, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 4.806213874995592, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-13-en-m0-source-chronological", + "history_id": "confirm-fresh-13-en-m0", + "case_id": "fresh-13-en", + "scenario": "fresh-13", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 844, + "completion_tokens": 2337, + "known_cost_microinr": 152880, + "retained_microinr": 191100, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.15818329200556, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-13-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-13-en-m0", + "case_id": "fresh-13-en", + "scenario": "fresh-13", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1313, + "completion_tokens": 2536, + "known_cost_microinr": 171855, + "retained_microinr": 214819, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.13812804200279, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-13-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-13-en-m0", + "case_id": "fresh-13-en", + "scenario": "fresh-13", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1313, + "completion_tokens": 2433, + "known_cost_microinr": 165675, + "retained_microinr": 207094, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.051968875006423, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-te-m1", + "case_id": "fresh-30-te", + "scenario": "fresh-30", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2128, + "completion_tokens": 42, + "known_cost_microinr": 34440, + "retained_microinr": 43050, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1079283750004834, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-te-m1-source-chronological", + "history_id": "confirm-fresh-30-te-m1", + "case_id": "fresh-30-te", + "scenario": "fresh-30", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1480, + "completion_tokens": 40, + "known_cost_microinr": 24600, + "retained_microinr": 30750, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9327283329912461, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-30-te-m1", + "case_id": "fresh-30-te", + "scenario": "fresh-30", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2128, + "completion_tokens": 42, + "known_cost_microinr": 34440, + "retained_microinr": 43050, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.037280082993675, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-10-te-m1", + "case_id": "fresh-10-te", + "scenario": "fresh-10", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1685, + "completion_tokens": 42, + "known_cost_microinr": 27795, + "retained_microinr": 34744, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0108522079972317, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-te-m1-source-chronological", + "history_id": "confirm-fresh-10-te-m1", + "case_id": "fresh-10-te", + "scenario": "fresh-10", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1105, + "completion_tokens": 42, + "known_cost_microinr": 19095, + "retained_microinr": 23869, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0359863749908982, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-10-te-m1", + "case_id": "fresh-10-te", + "scenario": "fresh-10", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1529, + "completion_tokens": 42, + "known_cost_microinr": 25455, + "retained_microinr": 31819, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.947349750000285, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-05-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-05-te-m1", + "case_id": "fresh-05-te", + "scenario": "fresh-05", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1724, + "completion_tokens": 32, + "known_cost_microinr": 27780, + "retained_microinr": 34725, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9550234579946846, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-05-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-05-te-m1", + "case_id": "fresh-05-te", + "scenario": "fresh-05", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1681, + "completion_tokens": 32, + "known_cost_microinr": 27135, + "retained_microinr": 33919, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9348693329957314, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-05-te-m1-source-chronological", + "history_id": "confirm-fresh-05-te-m1", + "case_id": "fresh-05-te", + "scenario": "fresh-05", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1033, + "completion_tokens": 32, + "known_cost_microinr": 17415, + "retained_microinr": 21769, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.964736832989729, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-16-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-16-hi-m0", + "case_id": "fresh-16-hi", + "scenario": "fresh-16", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1267, + "completion_tokens": 3414, + "known_cost_microinr": 223845, + "retained_microinr": 279807, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 16.575875291004195, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-16-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-16-hi-m0", + "case_id": "fresh-16-hi", + "scenario": "fresh-16", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1496, + "completion_tokens": 2516, + "known_cost_microinr": 173400, + "retained_microinr": 216750, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 10.879935958000715, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-16-hi-m0-source-chronological", + "history_id": "confirm-fresh-16-hi-m0", + "case_id": "fresh-16-hi", + "scenario": "fresh-16", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1056, + "completion_tokens": 3109, + "known_cost_microinr": 202380, + "retained_microinr": 252975, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.414731208002195, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-hi-m0-source-chronological", + "history_id": "confirm-fresh-35-hi-m0", + "case_id": "fresh-35-hi", + "scenario": "fresh-35", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1862, + "completion_tokens": 2485, + "known_cost_microinr": 177030, + "retained_microinr": 221288, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.712795708008343, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-35-hi-m0", + "case_id": "fresh-35-hi", + "scenario": "fresh-35", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2539, + "completion_tokens": 5454, + "known_cost_microinr": 365325, + "retained_microinr": 456657, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 22.776089999999385, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-hi-m0", + "case_id": "fresh-35-hi", + "scenario": "fresh-35", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2344, + "completion_tokens": 3082, + "known_cost_microinr": 220080, + "retained_microinr": 275100, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.915857749991119, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-38-en-m1", + "case_id": "fresh-38-en", + "scenario": "fresh-38", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2349, + "completion_tokens": 42, + "known_cost_microinr": 37755, + "retained_microinr": 47194, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0658477090037195, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-en-m1-source-chronological", + "history_id": "confirm-fresh-38-en-m1", + "case_id": "fresh-38-en", + "scenario": "fresh-38", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1775, + "completion_tokens": 42, + "known_cost_microinr": 29145, + "retained_microinr": 36432, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9468087080022087, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-en-m1", + "case_id": "fresh-38-en", + "scenario": "fresh-38", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2083, + "completion_tokens": 42, + "known_cost_microinr": 33765, + "retained_microinr": 42207, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.976152709001326, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-26-te-m1-source-chronological", + "history_id": "confirm-fresh-26-te-m1", + "case_id": "fresh-26-te", + "scenario": "fresh-26", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1481, + "completion_tokens": 40, + "known_cost_microinr": 24615, + "retained_microinr": 30769, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.945021417006501, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-26-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-26-te-m1", + "case_id": "fresh-26-te", + "scenario": "fresh-26", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2123, + "completion_tokens": 40, + "known_cost_microinr": 34245, + "retained_microinr": 42807, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1136486670002341, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-26-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-26-te-m1", + "case_id": "fresh-26-te", + "scenario": "fresh-26", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2123, + "completion_tokens": 40, + "known_cost_microinr": 34245, + "retained_microinr": 42807, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0787862920115003, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-07-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-te-m1", + "case_id": "fresh-07-te", + "scenario": "fresh-07", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1400, + "completion_tokens": 32, + "known_cost_microinr": 22920, + "retained_microinr": 28650, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.009558958990965, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-07-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-07-te-m1", + "case_id": "fresh-07-te", + "scenario": "fresh-07", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1449, + "completion_tokens": 32, + "known_cost_microinr": 23655, + "retained_microinr": 29569, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0261834590055514, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-07-te-m1-source-chronological", + "history_id": "confirm-fresh-07-te-m1", + "case_id": "fresh-07-te", + "scenario": "fresh-07", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 905, + "completion_tokens": 32, + "known_cost_microinr": 15495, + "retained_microinr": 19369, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9890587080008117, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-25-en-m0-source-chronological", + "history_id": "confirm-fresh-25-en-m0", + "case_id": "fresh-25-en", + "scenario": "fresh-25", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1352, + "completion_tokens": 1082, + "known_cost_microinr": 85200, + "retained_microinr": 106500, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 5.1286036250094185, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-25-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-25-en-m0", + "case_id": "fresh-25-en", + "scenario": "fresh-25", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1954, + "completion_tokens": 1611, + "known_cost_microinr": 125970, + "retained_microinr": 157463, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.29077145899646, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-25-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-25-en-m0", + "case_id": "fresh-25-en", + "scenario": "fresh-25", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1954, + "completion_tokens": 4418, + "known_cost_microinr": 294390, + "retained_microinr": 367988, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 18.362198625007295, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-29-te-m1", + "case_id": "fresh-29-te", + "scenario": "fresh-29", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2279, + "completion_tokens": 40, + "known_cost_microinr": 36585, + "retained_microinr": 45732, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.9713660840061493, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-29-te-m1", + "case_id": "fresh-29-te", + "scenario": "fresh-29", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2279, + "completion_tokens": 40, + "known_cost_microinr": 36585, + "retained_microinr": 45732, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9608177500049351, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-te-m1-source-chronological", + "history_id": "confirm-fresh-29-te-m1", + "case_id": "fresh-29-te", + "scenario": "fresh-29", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1517, + "completion_tokens": 40, + "known_cost_microinr": 25155, + "retained_microinr": 31444, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0014116670063231, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-27-te-m0-source-chronological", + "history_id": "confirm-fresh-27-te-m0", + "case_id": "fresh-27-te", + "scenario": "fresh-27", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1377, + "completion_tokens": 2298, + "known_cost_microinr": 158535, + "retained_microinr": 198169, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 10.898489665996749, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-27-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-27-te-m0", + "case_id": "fresh-27-te", + "scenario": "fresh-27", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1996, + "completion_tokens": 2274, + "known_cost_microinr": 166380, + "retained_microinr": 207975, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.866345625006943, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-27-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-27-te-m0", + "case_id": "fresh-27-te", + "scenario": "fresh-27", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1996, + "completion_tokens": 2571, + "known_cost_microinr": 184200, + "retained_microinr": 230250, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.015928041000734, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-19-en-m1-source-chronological", + "history_id": "confirm-fresh-19-en-m1", + "case_id": "fresh-19-en", + "scenario": "fresh-19", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1189, + "completion_tokens": 42, + "known_cost_microinr": 20355, + "retained_microinr": 25444, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9243975420104107, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-19-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-en-m1", + "case_id": "fresh-19-en", + "scenario": "fresh-19", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1711, + "completion_tokens": 42, + "known_cost_microinr": 28185, + "retained_microinr": 35232, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9390594160067849, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-19-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-19-en-m1", + "case_id": "fresh-19-en", + "scenario": "fresh-19", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-03-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1877, + "completion_tokens": 42, + "known_cost_microinr": 30675, + "retained_microinr": 38344, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0430048330017598, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-31-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-31-te-m1", + "case_id": "fresh-31-te", + "scenario": "fresh-31", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2281, + "completion_tokens": 40, + "known_cost_microinr": 36615, + "retained_microinr": 45769, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9490969160106033, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-31-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-31-te-m1", + "case_id": "fresh-31-te", + "scenario": "fresh-31", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2281, + "completion_tokens": 40, + "known_cost_microinr": 36615, + "retained_microinr": 45769, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0018306670099264, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-31-te-m1-source-chronological", + "history_id": "confirm-fresh-31-te-m1", + "case_id": "fresh-31-te", + "scenario": "fresh-31", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1523, + "completion_tokens": 40, + "known_cost_microinr": 25245, + "retained_microinr": 31557, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0602819999912754, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-32-en-m1", + "case_id": "fresh-32-en", + "scenario": "fresh-32", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2118, + "completion_tokens": 40, + "known_cost_microinr": 34170, + "retained_microinr": 42713, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9512993749958696, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-en-m1-source-chronological", + "history_id": "confirm-fresh-32-en-m1", + "case_id": "fresh-32-en", + "scenario": "fresh-32", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1462, + "completion_tokens": 40, + "known_cost_microinr": 24330, + "retained_microinr": 30413, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9453503749973606, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-en-m1", + "case_id": "fresh-32-en", + "scenario": "fresh-32", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2118, + "completion_tokens": 42, + "known_cost_microinr": 34290, + "retained_microinr": 42863, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9879191670042928, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-06-en-m1-source-chronological", + "history_id": "confirm-fresh-06-en-m1", + "case_id": "fresh-06-en", + "scenario": "fresh-06", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1069, + "completion_tokens": 32, + "known_cost_microinr": 17955, + "retained_microinr": 22444, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9809812500025146, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-06-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-06-en-m1", + "case_id": "fresh-06-en", + "scenario": "fresh-06", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1643, + "completion_tokens": 32, + "known_cost_microinr": 26565, + "retained_microinr": 33207, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9335260829975596, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-06-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-06-en-m1", + "case_id": "fresh-06-en", + "scenario": "fresh-06", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1517, + "completion_tokens": 32, + "known_cost_microinr": 24675, + "retained_microinr": 30844, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9801323340070667, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-07-en-m1-source-chronological", + "history_id": "confirm-fresh-07-en-m1", + "case_id": "fresh-07-en", + "scenario": "fresh-07", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1170, + "completion_tokens": 32, + "known_cost_microinr": 19470, + "retained_microinr": 24338, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9572724169993307, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-07-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-en-m1", + "case_id": "fresh-07-en", + "scenario": "fresh-07", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1672, + "completion_tokens": 32, + "known_cost_microinr": 27000, + "retained_microinr": 33750, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9694033749983646, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-07-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-07-en-m1", + "case_id": "fresh-07-en", + "scenario": "fresh-07", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1850, + "completion_tokens": 32, + "known_cost_microinr": 29670, + "retained_microinr": 37088, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0099914169986732, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-13-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-13-te-m0", + "case_id": "fresh-13-te", + "scenario": "fresh-13", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1411, + "completion_tokens": 2930, + "known_cost_microinr": 196965, + "retained_microinr": 246207, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.262295290987822, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-13-te-m0-source-chronological", + "history_id": "confirm-fresh-13-te-m0", + "case_id": "fresh-13-te", + "scenario": "fresh-13", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1019, + "completion_tokens": 8192, + "known_cost_microinr": 506805, + "retained_microinr": 633507, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 36.164887249993626, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-13-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-13-te-m0", + "case_id": "fresh-13-te", + "scenario": "fresh-13", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1567, + "completion_tokens": 4483, + "known_cost_microinr": 292485, + "retained_microinr": 365607, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 19.27871454099659, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-11-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-11-te-m1", + "case_id": "fresh-11-te", + "scenario": "fresh-11", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1863, + "completion_tokens": 42, + "known_cost_microinr": 30465, + "retained_microinr": 38082, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1042334169906098, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-11-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-11-te-m1", + "case_id": "fresh-11-te", + "scenario": "fresh-11", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1685, + "completion_tokens": 42, + "known_cost_microinr": 27795, + "retained_microinr": 34744, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0161284579953644, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-11-te-m1-source-chronological", + "history_id": "confirm-fresh-11-te-m1", + "case_id": "fresh-11-te", + "scenario": "fresh-11", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1184, + "completion_tokens": 42, + "known_cost_microinr": 20280, + "retained_microinr": 25350, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9765736249973997, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-09-te-m1-source-chronological", + "history_id": "confirm-fresh-09-te-m1", + "case_id": "fresh-09-te", + "scenario": "fresh-09", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1208, + "completion_tokens": 42, + "known_cost_microinr": 20640, + "retained_microinr": 25800, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0040511660045013, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-09-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-09-te-m1", + "case_id": "fresh-09-te", + "scenario": "fresh-09", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1706, + "completion_tokens": 42, + "known_cost_microinr": 28110, + "retained_microinr": 35138, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9720865410054103, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-09-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-09-te-m1", + "case_id": "fresh-09-te", + "scenario": "fresh-09", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1911, + "completion_tokens": 42, + "known_cost_microinr": 31185, + "retained_microinr": 38982, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.059681416998501, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-27-te-m1-source-chronological", + "history_id": "confirm-fresh-27-te-m1", + "case_id": "fresh-27-te", + "scenario": "fresh-27", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1523, + "completion_tokens": 40, + "known_cost_microinr": 25245, + "retained_microinr": 31557, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.022177582999575, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-27-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-27-te-m1", + "case_id": "fresh-27-te", + "scenario": "fresh-27", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2273, + "completion_tokens": 40, + "known_cost_microinr": 36495, + "retained_microinr": 45619, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1062835000047926, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-27-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-27-te-m1", + "case_id": "fresh-27-te", + "scenario": "fresh-27", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2273, + "completion_tokens": 40, + "known_cost_microinr": 36495, + "retained_microinr": 45619, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0070587079972029, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-12-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-12-hi-m1", + "case_id": "fresh-12-hi", + "scenario": "fresh-12", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": true, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1698, + "completion_tokens": 32, + "known_cost_microinr": 27390, + "retained_microinr": 34238, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9482885830075247, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-12-hi-m1-source-chronological", + "history_id": "confirm-fresh-12-hi-m1", + "case_id": "fresh-12-hi", + "scenario": "fresh-12", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": true, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1118, + "completion_tokens": 32, + "known_cost_microinr": 18690, + "retained_microinr": 23363, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9958275829994818, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-12-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-hi-m1", + "case_id": "fresh-12-hi", + "scenario": "fresh-12", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": true, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1540, + "completion_tokens": 32, + "known_cost_microinr": 25020, + "retained_microinr": 31275, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9534871250070864, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-12-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-te-m1", + "case_id": "fresh-12-te", + "scenario": "fresh-12", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1541, + "completion_tokens": 42, + "known_cost_microinr": 25635, + "retained_microinr": 32044, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0473955420020502, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-12-te-m1-source-chronological", + "history_id": "confirm-fresh-12-te-m1", + "case_id": "fresh-12-te", + "scenario": "fresh-12", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1129, + "completion_tokens": 42, + "known_cost_microinr": 19455, + "retained_microinr": 24319, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9940150839975104, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-12-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-12-te-m1", + "case_id": "fresh-12-te", + "scenario": "fresh-12", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1711, + "completion_tokens": 42, + "known_cost_microinr": 28185, + "retained_microinr": 35232, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.997791916001006, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-18-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-18-te-m0", + "case_id": "fresh-18-te", + "scenario": "fresh-18", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1297, + "completion_tokens": 3627, + "known_cost_microinr": 237075, + "retained_microinr": 296344, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 18.703877749998355, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-18-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-te-m0", + "case_id": "fresh-18-te", + "scenario": "fresh-18", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1274, + "completion_tokens": 1618, + "known_cost_microinr": 116190, + "retained_microinr": 145238, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.20122041599825, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-18-te-m0-source-chronological", + "history_id": "confirm-fresh-18-te-m0", + "case_id": "fresh-18-te", + "scenario": "fresh-18", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 871, + "completion_tokens": 8192, + "known_cost_microinr": 504585, + "retained_microinr": 630732, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 34.658750041999156, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-26-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-26-en-m1", + "case_id": "fresh-26-en", + "scenario": "fresh-26", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2106, + "completion_tokens": 40, + "known_cost_microinr": 33990, + "retained_microinr": 42488, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0119471670041094, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-26-en-m1-source-chronological", + "history_id": "confirm-fresh-26-en-m1", + "case_id": "fresh-26-en", + "scenario": "fresh-26", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1456, + "completion_tokens": 40, + "known_cost_microinr": 24240, + "retained_microinr": 30300, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.056377375003649, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-26-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-26-en-m1", + "case_id": "fresh-26-en", + "scenario": "fresh-26", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2106, + "completion_tokens": 40, + "known_cost_microinr": 33990, + "retained_microinr": 42488, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1043252499948721, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-23-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-23-te-m1", + "case_id": "fresh-23-te", + "scenario": "fresh-23", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2279, + "completion_tokens": 40, + "known_cost_microinr": 36585, + "retained_microinr": 45732, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.973554416996194, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-23-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-23-te-m1", + "case_id": "fresh-23-te", + "scenario": "fresh-23", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2279, + "completion_tokens": 40, + "known_cost_microinr": 36585, + "retained_microinr": 45732, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0796093750104774, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-23-te-m1-source-chronological", + "history_id": "confirm-fresh-23-te-m1", + "case_id": "fresh-23-te", + "scenario": "fresh-23", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1510, + "completion_tokens": 40, + "known_cost_microinr": 25050, + "retained_microinr": 31313, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9490920000098413, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-17-hi-m1-source-chronological", + "history_id": "confirm-fresh-17-hi-m1", + "case_id": "fresh-17-hi", + "scenario": "fresh-17", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1060, + "completion_tokens": 42, + "known_cost_microinr": 18420, + "retained_microinr": 23025, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9997523750062101, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-17-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-17-hi-m1", + "case_id": "fresh-17-hi", + "scenario": "fresh-17", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1726, + "completion_tokens": 42, + "known_cost_microinr": 28410, + "retained_microinr": 35513, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9848852080031065, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-17-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-17-hi-m1", + "case_id": "fresh-17-hi", + "scenario": "fresh-17", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-03-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1764, + "completion_tokens": 42, + "known_cost_microinr": 28980, + "retained_microinr": 36225, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9245362500078045, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-22-te-m0-source-chronological", + "history_id": "confirm-fresh-22-te-m0", + "case_id": "fresh-22-te", + "scenario": "fresh-22", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1540, + "completion_tokens": 1075, + "known_cost_microinr": 87600, + "retained_microinr": 109500, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.07318908299203, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-22-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-22-te-m0", + "case_id": "fresh-22-te", + "scenario": "fresh-22", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2127, + "completion_tokens": 2827, + "known_cost_microinr": 201525, + "retained_microinr": 251907, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.351604625000618, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-22-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-22-te-m0", + "case_id": "fresh-22-te", + "scenario": "fresh-22", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 2203, + "completion_tokens": 8192, + "known_cost_microinr": 524565, + "retained_microinr": 655707, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 35.501241707999725, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-01-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-01-te-m1", + "case_id": "fresh-01-te", + "scenario": "fresh-01", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1407, + "completion_tokens": 42, + "known_cost_microinr": 23625, + "retained_microinr": 29532, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9754500839917455, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-01-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-01-te-m1", + "case_id": "fresh-01-te", + "scenario": "fresh-01", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1427, + "completion_tokens": 42, + "known_cost_microinr": 23925, + "retained_microinr": 29907, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9718708750006044, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-01-te-m1-source-chronological", + "history_id": "confirm-fresh-01-te-m1", + "case_id": "fresh-01-te", + "scenario": "fresh-01", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 882, + "completion_tokens": 42, + "known_cost_microinr": 15750, + "retained_microinr": 19688, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9889927500043996, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-37-en-m0-source-chronological", + "history_id": "confirm-fresh-37-en-m0", + "case_id": "fresh-37-en", + "scenario": "fresh-37", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1709, + "completion_tokens": 6394, + "known_cost_microinr": 409275, + "retained_microinr": 511594, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 27.841237041997374, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-37-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-37-en-m0", + "case_id": "fresh-37-en", + "scenario": "fresh-37", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2397, + "completion_tokens": 6211, + "known_cost_microinr": 408615, + "retained_microinr": 510769, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 29.503583375000744, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-37-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-en-m0", + "case_id": "fresh-37-en", + "scenario": "fresh-37", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2233, + "completion_tokens": 2502, + "known_cost_microinr": 183615, + "retained_microinr": 229519, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.207604291004827, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-13-en-m1-source-chronological", + "history_id": "confirm-fresh-13-en-m1", + "case_id": "fresh-13-en", + "scenario": "fresh-13", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1211, + "completion_tokens": 42, + "known_cost_microinr": 20685, + "retained_microinr": 25857, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1110167919978267, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-13-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-13-en-m1", + "case_id": "fresh-13-en", + "scenario": "fresh-13", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1703, + "completion_tokens": 32, + "known_cost_microinr": 27465, + "retained_microinr": 34332, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9919395830074791, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-13-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-13-en-m1", + "case_id": "fresh-13-en", + "scenario": "fresh-13", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1908, + "completion_tokens": 42, + "known_cost_microinr": 31140, + "retained_microinr": 38925, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9607766250119312, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-19-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-19-hi-m1", + "case_id": "fresh-19-hi", + "scenario": "fresh-19", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1760, + "completion_tokens": 42, + "known_cost_microinr": 28920, + "retained_microinr": 36150, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0178038750018459, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-19-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-hi-m1", + "case_id": "fresh-19-hi", + "scenario": "fresh-19", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1713, + "completion_tokens": 42, + "known_cost_microinr": 28215, + "retained_microinr": 35269, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0134847500012256, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-19-hi-m1-source-chronological", + "history_id": "confirm-fresh-19-hi-m1", + "case_id": "fresh-19-hi", + "scenario": "fresh-19", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1066, + "completion_tokens": 42, + "known_cost_microinr": 18510, + "retained_microinr": 23138, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0303184579970548, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-37-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-te-m1", + "case_id": "fresh-37-te", + "scenario": "fresh-37", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2237, + "completion_tokens": 42, + "known_cost_microinr": 36075, + "retained_microinr": 45094, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9568220000073779, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-37-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-37-te-m1", + "case_id": "fresh-37-te", + "scenario": "fresh-37", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2592, + "completion_tokens": 40, + "known_cost_microinr": 41280, + "retained_microinr": 51600, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.025515582994558, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-37-te-m1-source-chronological", + "history_id": "confirm-fresh-37-te-m1", + "case_id": "fresh-37-te", + "scenario": "fresh-37", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1915, + "completion_tokens": 42, + "known_cost_microinr": 31245, + "retained_microinr": 39057, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.959741375001613, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-21-hi-m0", + "case_id": "fresh-21-hi", + "scenario": "fresh-21", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1975, + "completion_tokens": 4897, + "known_cost_microinr": 323445, + "retained_microinr": 404307, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 23.6456726250035, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-hi-m0-source-chronological", + "history_id": "confirm-fresh-21-hi-m0", + "case_id": "fresh-21-hi", + "scenario": "fresh-21", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1362, + "completion_tokens": 1863, + "known_cost_microinr": 132210, + "retained_microinr": 165263, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.234530417001224, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-21-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-21-hi-m0", + "case_id": "fresh-21-hi", + "scenario": "fresh-21", + "language": "hi", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1975, + "completion_tokens": 3698, + "known_cost_microinr": 251505, + "retained_microinr": 314382, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 15.051047041997663, + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-en-m0-source-chronological", + "history_id": "confirm-fresh-02-en-m0", + "case_id": "fresh-02-en", + "scenario": "fresh-02", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 921, + "completion_tokens": 756, + "known_cost_microinr": 59175, + "retained_microinr": 73969, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 4.452895791997435, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-02-en-m0", + "case_id": "fresh-02-en", + "scenario": "fresh-02", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1254, + "completion_tokens": 1130, + "known_cost_microinr": 86610, + "retained_microinr": 108263, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 5.4183051669970155, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-02-en-m0", + "case_id": "fresh-02-en", + "scenario": "fresh-02", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1361, + "completion_tokens": 1658, + "known_cost_microinr": 119895, + "retained_microinr": 149869, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.317468374996679, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-06-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-06-te-m0", + "case_id": "fresh-06-te", + "scenario": "fresh-06", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1292, + "completion_tokens": 2659, + "known_cost_microinr": 178920, + "retained_microinr": 223650, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.880363666990888, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-06-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-06-te-m0", + "case_id": "fresh-06-te", + "scenario": "fresh-06", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1254, + "completion_tokens": 1123, + "known_cost_microinr": 86190, + "retained_microinr": 107738, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.485287333998713, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-06-te-m0-source-chronological", + "history_id": "confirm-fresh-06-te-m0", + "case_id": "fresh-06-te", + "scenario": "fresh-06", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 853, + "completion_tokens": 7725, + "known_cost_microinr": 476295, + "retained_microinr": 595369, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 31.482956999985618, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-39-te-m0-source-chronological", + "history_id": "confirm-fresh-39-te-m0", + "case_id": "fresh-39-te", + "scenario": "fresh-39", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1653, + "completion_tokens": 1536, + "known_cost_microinr": 116955, + "retained_microinr": 146194, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.939602875005221, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-39-te-m0", + "case_id": "fresh-39-te", + "scenario": "fresh-39", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2190, + "completion_tokens": 4703, + "known_cost_microinr": 315030, + "retained_microinr": 393788, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 21.842392959006247, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-te-m0", + "case_id": "fresh-39-te", + "scenario": "fresh-39", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1961, + "completion_tokens": 2102, + "known_cost_microinr": 155535, + "retained_microinr": 194419, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.289832583002863, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-12-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-en-m0", + "case_id": "fresh-12-en", + "scenario": "fresh-12", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1265, + "completion_tokens": 1826, + "known_cost_microinr": 128535, + "retained_microinr": 160669, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.442558375012595, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-12-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-12-en-m0", + "case_id": "fresh-12-en", + "scenario": "fresh-12", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1319, + "completion_tokens": 2409, + "known_cost_microinr": 164325, + "retained_microinr": 205407, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 10.384598374992493, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-12-en-m0-source-chronological", + "history_id": "confirm-fresh-12-en-m0", + "case_id": "fresh-12-en", + "scenario": "fresh-12", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 875, + "completion_tokens": 1643, + "known_cost_microinr": 111705, + "retained_microinr": 139632, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.68665604099806, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-11-hi-m1-source-chronological", + "history_id": "confirm-fresh-11-hi-m1", + "case_id": "fresh-11-hi", + "scenario": "fresh-11", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": true, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1305, + "completion_tokens": 32, + "known_cost_microinr": 21495, + "retained_microinr": 26869, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0096349579980597, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-11-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-11-hi-m1", + "case_id": "fresh-11-hi", + "scenario": "fresh-11", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": true, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1711, + "completion_tokens": 32, + "known_cost_microinr": 27585, + "retained_microinr": 34482, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9533761670027161, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-11-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-11-hi-m1", + "case_id": "fresh-11-hi", + "scenario": "fresh-11", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2010, + "completion_tokens": 42, + "known_cost_microinr": 32670, + "retained_microinr": 40838, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.084269166996819, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-27-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-27-hi-m1", + "case_id": "fresh-27-hi", + "scenario": "fresh-27", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2271, + "completion_tokens": 40, + "known_cost_microinr": 36465, + "retained_microinr": 45582, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0219765420042677, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-27-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-27-hi-m1", + "case_id": "fresh-27-hi", + "scenario": "fresh-27", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2271, + "completion_tokens": 40, + "known_cost_microinr": 36465, + "retained_microinr": 45582, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9778811249998398, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-27-hi-m1-source-chronological", + "history_id": "confirm-fresh-27-hi-m1", + "case_id": "fresh-27-hi", + "scenario": "fresh-27", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1508, + "completion_tokens": 40, + "known_cost_microinr": 25020, + "retained_microinr": 31275, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0093898339982843, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-22-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-22-en-m1", + "case_id": "fresh-22-en", + "scenario": "fresh-22", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2107, + "completion_tokens": 40, + "known_cost_microinr": 34005, + "retained_microinr": 42507, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0368601660011336, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-22-en-m1-source-chronological", + "history_id": "confirm-fresh-22-en-m1", + "case_id": "fresh-22-en", + "scenario": "fresh-22", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1456, + "completion_tokens": 40, + "known_cost_microinr": 24240, + "retained_microinr": 30300, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9950489580078283, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-22-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-22-en-m1", + "case_id": "fresh-22-en", + "scenario": "fresh-22", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2107, + "completion_tokens": 40, + "known_cost_microinr": 34005, + "retained_microinr": 42507, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0903271660063183, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-29-en-m0", + "case_id": "fresh-29-en", + "scenario": "fresh-29", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2029, + "completion_tokens": 3145, + "known_cost_microinr": 219135, + "retained_microinr": 273919, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.938349500007462, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-en-m0-source-chronological", + "history_id": "confirm-fresh-29-en-m0", + "case_id": "fresh-29-en", + "scenario": "fresh-29", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1460, + "completion_tokens": 5184, + "known_cost_microinr": 332940, + "retained_microinr": 416175, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 21.042172667002887, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-29-en-m0", + "case_id": "fresh-29-en", + "scenario": "fresh-29", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2164, + "completion_tokens": 3397, + "known_cost_microinr": 236280, + "retained_microinr": 295350, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 14.523942707994138, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-en-m0", + "case_id": "fresh-30-en", + "scenario": "fresh-30", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1581, + "completion_tokens": 1492, + "known_cost_microinr": 113235, + "retained_microinr": 141544, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.055782249997719, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-30-en-m0", + "case_id": "fresh-30-en", + "scenario": "fresh-30", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1694, + "completion_tokens": 2819, + "known_cost_microinr": 194550, + "retained_microinr": 243188, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.259262457999284, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-en-m0-source-chronological", + "history_id": "confirm-fresh-30-en-m0", + "case_id": "fresh-30-en", + "scenario": "fresh-30", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1258, + "completion_tokens": 2975, + "known_cost_microinr": 197370, + "retained_microinr": 246713, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 14.935974583000643, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-34-te-m0-source-chronological", + "history_id": "confirm-fresh-34-te-m0", + "case_id": "fresh-34-te", + "scenario": "fresh-34", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1837, + "completion_tokens": 3244, + "known_cost_microinr": 222195, + "retained_microinr": 277744, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.686934375000419, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-34-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-te-m0", + "case_id": "fresh-34-te", + "scenario": "fresh-34", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2216, + "completion_tokens": 5294, + "known_cost_microinr": 350880, + "retained_microinr": 438600, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 24.093186958998558, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-34-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-34-te-m0", + "case_id": "fresh-34-te", + "scenario": "fresh-34", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2407, + "completion_tokens": 1538, + "known_cost_microinr": 128385, + "retained_microinr": 160482, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.980180041995482, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-33-en-m1", + "case_id": "fresh-33-en", + "scenario": "fresh-33", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2630, + "completion_tokens": 42, + "known_cost_microinr": 41970, + "retained_microinr": 52463, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9518733330041869, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-en-m1", + "case_id": "fresh-33-en", + "scenario": "fresh-33", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2339, + "completion_tokens": 42, + "known_cost_microinr": 37605, + "retained_microinr": 47007, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.011973915999988, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-en-m1-source-chronological", + "history_id": "confirm-fresh-33-en-m1", + "case_id": "fresh-33-en", + "scenario": "fresh-33", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1945, + "completion_tokens": 42, + "known_cost_microinr": 31695, + "retained_microinr": 39619, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.6084250419953605, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-hi-m1-source-chronological", + "history_id": "confirm-fresh-10-hi-m1", + "case_id": "fresh-10-hi", + "scenario": "fresh-10", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1166, + "completion_tokens": 42, + "known_cost_microinr": 20010, + "retained_microinr": 25013, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9439062499877764, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-10-hi-m1", + "case_id": "fresh-10-hi", + "scenario": "fresh-10", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1749, + "completion_tokens": 42, + "known_cost_microinr": 28755, + "retained_microinr": 35944, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9993957499973476, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-10-hi-m1", + "case_id": "fresh-10-hi", + "scenario": "fresh-10", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": true, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1544, + "completion_tokens": 32, + "known_cost_microinr": 25080, + "retained_microinr": 31350, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9858482499985257, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-18-hi-m0-source-chronological", + "history_id": "confirm-fresh-18-hi-m0", + "case_id": "fresh-18-hi", + "scenario": "fresh-18", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-03-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1023, + "completion_tokens": 5767, + "known_cost_microinr": 361365, + "retained_microinr": 451707, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 24.056296499999007, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-18-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-hi-m0", + "case_id": "fresh-18-hi", + "scenario": "fresh-18", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1572, + "completion_tokens": 1491, + "known_cost_microinr": 113040, + "retained_microinr": 141300, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.146909040995524, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-18-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-18-hi-m0", + "case_id": "fresh-18-hi", + "scenario": "fresh-18", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1614, + "completion_tokens": 4839, + "known_cost_microinr": 314550, + "retained_microinr": 393188, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 21.178230124991387, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-17-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-17-en-m1", + "case_id": "fresh-17-en", + "scenario": "fresh-17", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1706, + "completion_tokens": 42, + "known_cost_microinr": 28110, + "retained_microinr": 35138, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1101471669971943, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-17-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-17-en-m1", + "case_id": "fresh-17-en", + "scenario": "fresh-17", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1798, + "completion_tokens": 42, + "known_cost_microinr": 29490, + "retained_microinr": 36863, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9717892079934245, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-17-en-m1-source-chronological", + "history_id": "confirm-fresh-17-en-m1", + "case_id": "fresh-17-en", + "scenario": "fresh-17", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1114, + "completion_tokens": 42, + "known_cost_microinr": 19230, + "retained_microinr": 24038, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0526387499994598, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-25-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-25-te-m1", + "case_id": "fresh-25-te", + "scenario": "fresh-25", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2273, + "completion_tokens": 40, + "known_cost_microinr": 36495, + "retained_microinr": 45619, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0558867090003332, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-25-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-25-te-m1", + "case_id": "fresh-25-te", + "scenario": "fresh-25", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2273, + "completion_tokens": 40, + "known_cost_microinr": 36495, + "retained_microinr": 45619, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9434035830054199, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-25-te-m1-source-chronological", + "history_id": "confirm-fresh-25-te-m1", + "case_id": "fresh-25-te", + "scenario": "fresh-25", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1511, + "completion_tokens": 40, + "known_cost_microinr": 25065, + "retained_microinr": 31332, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.00234779099992, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-04-hi-m1-source-chronological", + "history_id": "confirm-fresh-04-hi-m1", + "case_id": "fresh-04-hi", + "scenario": "fresh-04", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 838, + "completion_tokens": 42, + "known_cost_microinr": 15090, + "retained_microinr": 18863, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0980610839906149, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-04-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-04-hi-m1", + "case_id": "fresh-04-hi", + "scenario": "fresh-04", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1277, + "completion_tokens": 42, + "known_cost_microinr": 21675, + "retained_microinr": 27094, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.025159333992633, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-04-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-04-hi-m1", + "case_id": "fresh-04-hi", + "scenario": "fresh-04", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1264, + "completion_tokens": 42, + "known_cost_microinr": 21480, + "retained_microinr": 26850, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 2.1554742500011344, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-01-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-01-te-m0", + "case_id": "fresh-01-te", + "scenario": "fresh-01", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": true, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1412, + "completion_tokens": 1767, + "known_cost_microinr": 127200, + "retained_microinr": 159000, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.830779916999745, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-01-te-m0-source-chronological", + "history_id": "confirm-fresh-01-te-m0", + "case_id": "fresh-01-te", + "scenario": "fresh-01", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 873, + "completion_tokens": 1570, + "known_cost_microinr": 107295, + "retained_microinr": 134119, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.714128249994246, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-01-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-01-te-m0", + "case_id": "fresh-01-te", + "scenario": "fresh-01", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1427, + "completion_tokens": 1624, + "known_cost_microinr": 118845, + "retained_microinr": 148557, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.95311099999526, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-05-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-05-en-m0", + "case_id": "fresh-05-en", + "scenario": "fresh-05", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1388, + "completion_tokens": 1648, + "known_cost_microinr": 119700, + "retained_microinr": 149625, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.997502166006598, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-05-en-m0-source-chronological", + "history_id": "confirm-fresh-05-en-m0", + "case_id": "fresh-05-en", + "scenario": "fresh-05", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 992, + "completion_tokens": 1716, + "known_cost_microinr": 117840, + "retained_microinr": 147300, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.598398582995287, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-05-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-05-en-m0", + "case_id": "fresh-05-en", + "scenario": "fresh-05", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1533, + "completion_tokens": 2840, + "known_cost_microinr": 193395, + "retained_microinr": 241744, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.827546749991598, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-16-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-16-en-m1", + "case_id": "fresh-16-en", + "scenario": "fresh-16", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1556, + "completion_tokens": 32, + "known_cost_microinr": 25260, + "retained_microinr": 31575, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0170336669980315, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-16-en-m1-source-chronological", + "history_id": "confirm-fresh-16-en-m1", + "case_id": "fresh-16-en", + "scenario": "fresh-16", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1208, + "completion_tokens": 42, + "known_cost_microinr": 20640, + "retained_microinr": 25800, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9932945830078097, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-16-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-16-en-m1", + "case_id": "fresh-16-en", + "scenario": "fresh-16", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1796, + "completion_tokens": 42, + "known_cost_microinr": 29460, + "retained_microinr": 36825, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9913481249968754, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-28-hi-m1-source-chronological", + "history_id": "confirm-fresh-28-hi-m1", + "case_id": "fresh-28-hi", + "scenario": "fresh-28", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1568, + "completion_tokens": 42, + "known_cost_microinr": 26040, + "retained_microinr": 32550, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9354872919939226, + "actual_question_answer": { + "answer": "2027-03-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": false, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-28-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-28-hi-m1", + "case_id": "fresh-28-hi", + "scenario": "fresh-28", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2144, + "completion_tokens": 42, + "known_cost_microinr": 34680, + "retained_microinr": 43350, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0966805000061868, + "actual_question_answer": { + "answer": "2027-03-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": false, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-28-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-28-hi-m1", + "case_id": "fresh-28-hi", + "scenario": "fresh-28", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1876, + "completion_tokens": 42, + "known_cost_microinr": 30660, + "retained_microinr": 38325, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9837397079973016, + "actual_question_answer": { + "answer": "2027-03-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": false, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-34-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-34-en-m0", + "case_id": "fresh-34-en", + "scenario": "fresh-34", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2409, + "completion_tokens": 4503, + "known_cost_microinr": 306315, + "retained_microinr": 382894, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 18.816953500005184, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-34-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-en-m0", + "case_id": "fresh-34-en", + "scenario": "fresh-34", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2180, + "completion_tokens": 3475, + "known_cost_microinr": 241200, + "retained_microinr": 301500, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 14.660194209005567, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-34-en-m0-source-chronological", + "history_id": "confirm-fresh-34-en-m0", + "case_id": "fresh-34-en", + "scenario": "fresh-34", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1841, + "completion_tokens": 1763, + "known_cost_microinr": 133395, + "retained_microinr": 166744, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.799814999991213, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-te-m0-source-chronological", + "history_id": "confirm-fresh-40-te-m0", + "case_id": "fresh-40-te", + "scenario": "fresh-40", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1812, + "completion_tokens": 4233, + "known_cost_microinr": 281160, + "retained_microinr": 351450, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 20.638543459004723, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-te-m0", + "case_id": "fresh-40-te", + "scenario": "fresh-40", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2104, + "completion_tokens": 4201, + "known_cost_microinr": 283620, + "retained_microinr": 354525, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 20.65557820899994, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-40-te-m0", + "case_id": "fresh-40-te", + "scenario": "fresh-40", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2397, + "completion_tokens": 2583, + "known_cost_microinr": 190935, + "retained_microinr": 238669, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.740064708996215, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-03-hi-m0", + "case_id": "fresh-03-hi", + "scenario": "fresh-03", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1410, + "completion_tokens": 2445, + "known_cost_microinr": 167850, + "retained_microinr": 209813, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.524418792003416, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-hi-m0-source-chronological", + "history_id": "confirm-fresh-03-hi-m0", + "case_id": "fresh-03-hi", + "scenario": "fresh-03", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 870, + "completion_tokens": 2125, + "known_cost_microinr": 140550, + "retained_microinr": 175688, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.191295625001658, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-03-hi-m0", + "case_id": "fresh-03-hi", + "scenario": "fresh-03", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1425, + "completion_tokens": 1826, + "known_cost_microinr": 130935, + "retained_microinr": 163669, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.684030208009062, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-11-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-11-en-m1", + "case_id": "fresh-11-en", + "scenario": "fresh-11", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1696, + "completion_tokens": 42, + "known_cost_microinr": 27960, + "retained_microinr": 34950, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.973132875005831, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-11-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-11-en-m1", + "case_id": "fresh-11-en", + "scenario": "fresh-11", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1845, + "completion_tokens": 42, + "known_cost_microinr": 30195, + "retained_microinr": 37744, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0418019160133554, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-11-en-m1-source-chronological", + "history_id": "confirm-fresh-11-en-m1", + "case_id": "fresh-11-en", + "scenario": "fresh-11", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1160, + "completion_tokens": 42, + "known_cost_microinr": 19920, + "retained_microinr": 24900, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9567370420118095, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-26-en-m0-source-chronological", + "history_id": "confirm-fresh-26-en-m0", + "case_id": "fresh-26-en", + "scenario": "fresh-26", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1314, + "completion_tokens": 367, + "known_cost_microinr": 41730, + "retained_microinr": 52163, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 2.375200208000024, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-26-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-26-en-m0", + "case_id": "fresh-26-en", + "scenario": "fresh-26", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1819, + "completion_tokens": 2441, + "known_cost_microinr": 173745, + "retained_microinr": 217182, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.03292225000041, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-26-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-26-en-m0", + "case_id": "fresh-26-en", + "scenario": "fresh-26", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1819, + "completion_tokens": 3021, + "known_cost_microinr": 208545, + "retained_microinr": 260682, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.296980833009002, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-15-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-15-hi-m0", + "case_id": "fresh-15-hi", + "scenario": "fresh-15", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1417, + "completion_tokens": 4538, + "known_cost_microinr": 293535, + "retained_microinr": 366919, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 21.559326667003916, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-15-hi-m0-source-chronological", + "history_id": "confirm-fresh-15-hi-m0", + "case_id": "fresh-15-hi", + "scenario": "fresh-15", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1034, + "completion_tokens": 5373, + "known_cost_microinr": 337890, + "retained_microinr": 422363, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 23.664025082995067, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-15-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-15-hi-m0", + "case_id": "fresh-15-hi", + "scenario": "fresh-15", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1585, + "completion_tokens": 2111, + "known_cost_microinr": 150435, + "retained_microinr": 188044, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.372164083993994, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-09-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-09-en-m1", + "case_id": "fresh-09-en", + "scenario": "fresh-09", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": true, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1894, + "completion_tokens": 32, + "known_cost_microinr": 30330, + "retained_microinr": 37913, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1083831249998184, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-09-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-09-en-m1", + "case_id": "fresh-09-en", + "scenario": "fresh-09", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1694, + "completion_tokens": 42, + "known_cost_microinr": 27930, + "retained_microinr": 34913, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9687332499888726, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-09-en-m1-source-chronological", + "history_id": "confirm-fresh-09-en-m1", + "case_id": "fresh-09-en", + "scenario": "fresh-09", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1203, + "completion_tokens": 42, + "known_cost_microinr": 20565, + "retained_microinr": 25707, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.034745249999105, + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-38-hi-m0", + "case_id": "fresh-38-hi", + "scenario": "fresh-38", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2030, + "completion_tokens": 6063, + "known_cost_microinr": 394230, + "retained_microinr": 492788, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 26.71671595799853, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-hi-m0", + "case_id": "fresh-38-hi", + "scenario": "fresh-38", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1812, + "completion_tokens": 5849, + "known_cost_microinr": 378120, + "retained_microinr": 472650, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 25.631007999996655, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-hi-m0-source-chronological", + "history_id": "confirm-fresh-38-hi-m0", + "case_id": "fresh-38-hi", + "scenario": "fresh-38", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1595, + "completion_tokens": 2929, + "known_cost_microinr": 199665, + "retained_microinr": 249582, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.397669375000987, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-24-te-m0", + "case_id": "fresh-24-te", + "scenario": "fresh-24", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1833, + "completion_tokens": 3206, + "known_cost_microinr": 219855, + "retained_microinr": 274819, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 16.238885291997576, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-24-te-m0", + "case_id": "fresh-24-te", + "scenario": "fresh-24", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1833, + "completion_tokens": 5393, + "known_cost_microinr": 351075, + "retained_microinr": 438844, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 26.072659749988816, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-24-te-m0-source-chronological", + "history_id": "confirm-fresh-24-te-m0", + "case_id": "fresh-24-te", + "scenario": "fresh-24", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1334, + "completion_tokens": 1047, + "known_cost_microinr": 82830, + "retained_microinr": 103538, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.182483416996547, + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-07-en-m0-source-chronological", + "history_id": "confirm-fresh-07-en-m0", + "case_id": "fresh-07-en", + "scenario": "fresh-07", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 943, + "completion_tokens": 2282, + "known_cost_microinr": 151065, + "retained_microinr": 188832, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 10.988950625003781, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-07-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-07-en-m0", + "case_id": "fresh-07-en", + "scenario": "fresh-07", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1477, + "completion_tokens": 1147, + "known_cost_microinr": 90975, + "retained_microinr": 113719, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.121087374995113, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-07-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-en-m0", + "case_id": "fresh-07-en", + "scenario": "fresh-07", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1384, + "completion_tokens": 1776, + "known_cost_microinr": 127320, + "retained_microinr": 159150, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.836480041005416, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-11-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-11-te-m0", + "case_id": "fresh-11-te", + "scenario": "fresh-11", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1522, + "completion_tokens": 3830, + "known_cost_microinr": 252630, + "retained_microinr": 315788, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 16.252762250005617, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-11-te-m0-source-chronological", + "history_id": "confirm-fresh-11-te-m0", + "case_id": "fresh-11-te", + "scenario": "fresh-11", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 981, + "completion_tokens": 3225, + "known_cost_microinr": 208215, + "retained_microinr": 260269, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 15.773785208002664, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-11-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-11-te-m0", + "case_id": "fresh-11-te", + "scenario": "fresh-11", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1407, + "completion_tokens": 3985, + "known_cost_microinr": 260205, + "retained_microinr": 325257, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 16.09192604100099, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-33-en-m0", + "case_id": "fresh-33-en", + "scenario": "fresh-33", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2259, + "completion_tokens": 1421, + "known_cost_microinr": 119145, + "retained_microinr": 148932, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.597253208004986, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-en-m0-source-chronological", + "history_id": "confirm-fresh-33-en-m0", + "case_id": "fresh-33-en", + "scenario": "fresh-33", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1722, + "completion_tokens": 5819, + "known_cost_microinr": 374970, + "retained_microinr": 468713, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 25.90595170899178, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-en-m0", + "case_id": "fresh-33-en", + "scenario": "fresh-33", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2055, + "completion_tokens": 7311, + "known_cost_microinr": 469485, + "retained_microinr": 586857, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 30.04814375001297, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-10-en-m1", + "case_id": "fresh-10-en", + "scenario": "fresh-10", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1536, + "completion_tokens": 42, + "known_cost_microinr": 25560, + "retained_microinr": 31950, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9771300419961335, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-en-m1-source-chronological", + "history_id": "confirm-fresh-10-en-m1", + "case_id": "fresh-10-en", + "scenario": "fresh-10", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1101, + "completion_tokens": 42, + "known_cost_microinr": 19035, + "retained_microinr": 23794, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0250782080111094, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-10-en-m1", + "case_id": "fresh-10-en", + "scenario": "fresh-10", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1676, + "completion_tokens": 42, + "known_cost_microinr": 27660, + "retained_microinr": 34575, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1817924580100225, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-26-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-26-hi-m1", + "case_id": "fresh-26-hi", + "scenario": "fresh-26", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2109, + "completion_tokens": 40, + "known_cost_microinr": 34035, + "retained_microinr": 42544, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9629914580000332, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-26-hi-m1-source-chronological", + "history_id": "confirm-fresh-26-hi-m1", + "case_id": "fresh-26-hi", + "scenario": "fresh-26", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1468, + "completion_tokens": 40, + "known_cost_microinr": 24420, + "retained_microinr": 30525, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.5600061250006547, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-26-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-26-hi-m1", + "case_id": "fresh-26-hi", + "scenario": "fresh-26", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2109, + "completion_tokens": 40, + "known_cost_microinr": 34035, + "retained_microinr": 42544, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.948893875000067, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-18-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-en-m0", + "case_id": "fresh-18-en", + "scenario": "fresh-18", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-04-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1572, + "completion_tokens": 2561, + "known_cost_microinr": 177240, + "retained_microinr": 221550, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.11054474998673, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-18-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-18-en-m0", + "case_id": "fresh-18-en", + "scenario": "fresh-18", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1624, + "completion_tokens": 3281, + "known_cost_microinr": 221220, + "retained_microinr": 276525, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 17.147750999996788, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-18-en-m0-source-chronological", + "history_id": "confirm-fresh-18-en-m0", + "case_id": "fresh-18-en", + "scenario": "fresh-18", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1034, + "completion_tokens": 6740, + "known_cost_microinr": 419910, + "retained_microinr": 524888, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 31.196360917005222, + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-23-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-23-te-m0", + "case_id": "fresh-23-te", + "scenario": "fresh-23", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 2175, + "completion_tokens": 8192, + "known_cost_microinr": 524145, + "retained_microinr": 655182, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 38.70535054200445, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-23-te-m0-source-chronological", + "history_id": "confirm-fresh-23-te-m0", + "case_id": "fresh-23-te", + "scenario": "fresh-23", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1482, + "completion_tokens": 1336, + "known_cost_microinr": 102390, + "retained_microinr": 127988, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.541762499997276, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-23-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-23-te-m0", + "case_id": "fresh-23-te", + "scenario": "fresh-23", + "language": "te", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 2175, + "completion_tokens": 8192, + "known_cost_microinr": 524145, + "retained_microinr": 655182, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 34.90903183299815, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-30-en-m1-source-chronological", + "history_id": "confirm-fresh-30-en-m1", + "case_id": "fresh-30-en", + "scenario": "fresh-30", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1464, + "completion_tokens": 40, + "known_cost_microinr": 24360, + "retained_microinr": 30450, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0194091250014026, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-30-en-m1", + "case_id": "fresh-30-en", + "scenario": "fresh-30", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2116, + "completion_tokens": 40, + "known_cost_microinr": 34140, + "retained_microinr": 42675, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9591924999986077, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-30-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-en-m1", + "case_id": "fresh-30-en", + "scenario": "fresh-30", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2116, + "completion_tokens": 42, + "known_cost_microinr": 34260, + "retained_microinr": 42825, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9940253750100965, + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-17-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-17-te-m1", + "case_id": "fresh-17-te", + "scenario": "fresh-17", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1496, + "completion_tokens": 42, + "known_cost_microinr": 24960, + "retained_microinr": 31200, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1015573330078041, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-17-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-17-te-m1", + "case_id": "fresh-17-te", + "scenario": "fresh-17", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1443, + "completion_tokens": 42, + "known_cost_microinr": 24165, + "retained_microinr": 30207, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.953701791993808, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-17-te-m1-source-chronological", + "history_id": "confirm-fresh-17-te-m1", + "case_id": "fresh-17-te", + "scenario": "fresh-17", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 941, + "completion_tokens": 42, + "known_cost_microinr": 16635, + "retained_microinr": 20794, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9648898750019725, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-36-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-36-hi-m1", + "case_id": "fresh-36-hi", + "scenario": "fresh-36", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2584, + "completion_tokens": 42, + "known_cost_microinr": 41280, + "retained_microinr": 51600, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9925268750084797, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-hi-m1-source-chronological", + "history_id": "confirm-fresh-36-hi-m1", + "case_id": "fresh-36-hi", + "scenario": "fresh-36", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2014, + "completion_tokens": 42, + "known_cost_microinr": 32730, + "retained_microinr": 40913, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0158505830040667, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-hi-m1", + "case_id": "fresh-36-hi", + "scenario": "fresh-36", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2205, + "completion_tokens": 42, + "known_cost_microinr": 35595, + "retained_microinr": 44494, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.095116250013234, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-28-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-28-te-m1", + "case_id": "fresh-28-te", + "scenario": "fresh-28", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2120, + "completion_tokens": 40, + "known_cost_microinr": 34200, + "retained_microinr": 42750, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9960944159975043, + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-28-te-m1-source-chronological", + "history_id": "confirm-fresh-28-te-m1", + "case_id": "fresh-28-te", + "scenario": "fresh-28", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1477, + "completion_tokens": 40, + "known_cost_microinr": 24555, + "retained_microinr": 30694, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9413197079993552, + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-28-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-28-te-m1", + "case_id": "fresh-28-te", + "scenario": "fresh-28", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2120, + "completion_tokens": 40, + "known_cost_microinr": 34200, + "retained_microinr": 42750, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9589607079979032, + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-hi-m1-source-chronological", + "history_id": "confirm-fresh-29-hi-m1", + "case_id": "fresh-29-hi", + "scenario": "fresh-29", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1511, + "completion_tokens": 40, + "known_cost_microinr": 25065, + "retained_microinr": 31332, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9863521669904003, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-29-hi-m1", + "case_id": "fresh-29-hi", + "scenario": "fresh-29", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2272, + "completion_tokens": 40, + "known_cost_microinr": 36480, + "retained_microinr": 45600, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0490005830070004, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-29-hi-m1", + "case_id": "fresh-29-hi", + "scenario": "fresh-29", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2272, + "completion_tokens": 40, + "known_cost_microinr": 36480, + "retained_microinr": 45600, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9709048750082729, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-35-te-m1", + "case_id": "fresh-35-te", + "scenario": "fresh-35", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2661, + "completion_tokens": 42, + "known_cost_microinr": 42435, + "retained_microinr": 53044, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9930545829993207, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-te-m1", + "case_id": "fresh-35-te", + "scenario": "fresh-35", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2361, + "completion_tokens": 42, + "known_cost_microinr": 37935, + "retained_microinr": 47419, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9639243749988964, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-te-m1-source-chronological", + "history_id": "confirm-fresh-35-te-m1", + "case_id": "fresh-35-te", + "scenario": "fresh-35", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1986, + "completion_tokens": 42, + "known_cost_microinr": 32310, + "retained_microinr": 40388, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0494226250011707, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-04-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-04-en-m1", + "case_id": "fresh-04-en", + "scenario": "fresh-04", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": true, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1531, + "completion_tokens": 32, + "known_cost_microinr": 24885, + "retained_microinr": 31107, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0147467499919003, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-04-en-m1-source-chronological", + "history_id": "confirm-fresh-04-en-m1", + "case_id": "fresh-04-en", + "scenario": "fresh-04", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1093, + "completion_tokens": 42, + "known_cost_microinr": 18915, + "retained_microinr": 23644, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9243329579912825, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-04-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-04-en-m1", + "case_id": "fresh-04-en", + "scenario": "fresh-04", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1668, + "completion_tokens": 42, + "known_cost_microinr": 27540, + "retained_microinr": 34425, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.016910625010496, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-12-en-m1-source-chronological", + "history_id": "confirm-fresh-12-en-m1", + "case_id": "fresh-12-en", + "scenario": "fresh-12", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1077, + "completion_tokens": 42, + "known_cost_microinr": 18675, + "retained_microinr": 23344, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0130689579964383, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-12-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-12-en-m1", + "case_id": "fresh-12-en", + "scenario": "fresh-12", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1664, + "completion_tokens": 42, + "known_cost_microinr": 27480, + "retained_microinr": 34350, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9493297080043703, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-12-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-en-m1", + "case_id": "fresh-12-en", + "scenario": "fresh-12", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1552, + "completion_tokens": 42, + "known_cost_microinr": 25800, + "retained_microinr": 32250, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.10825108300196, + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-40-te-m1", + "case_id": "fresh-40-te", + "scenario": "fresh-40", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2413, + "completion_tokens": 42, + "known_cost_microinr": 38715, + "retained_microinr": 48394, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0076746659906348, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-te-m1-source-chronological", + "history_id": "confirm-fresh-40-te-m1", + "case_id": "fresh-40-te", + "scenario": "fresh-40", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1843, + "completion_tokens": 42, + "known_cost_microinr": 30165, + "retained_microinr": 37707, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9605918339948403, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-te-m1", + "case_id": "fresh-40-te", + "scenario": "fresh-40", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2099, + "completion_tokens": 42, + "known_cost_microinr": 34005, + "retained_microinr": 42507, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9818256669968832, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-06-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-06-hi-m1", + "case_id": "fresh-06-hi", + "scenario": "fresh-06", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1573, + "completion_tokens": 32, + "known_cost_microinr": 25515, + "retained_microinr": 31894, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.025581499998225, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-06-hi-m1-source-chronological", + "history_id": "confirm-fresh-06-hi-m1", + "case_id": "fresh-06-hi", + "scenario": "fresh-06", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-07", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-07", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 983, + "completion_tokens": 42, + "known_cost_microinr": 17265, + "retained_microinr": 21582, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9718502920004539, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-06-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-06-hi-m1", + "case_id": "fresh-06-hi", + "scenario": "fresh-06", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1540, + "completion_tokens": 32, + "known_cost_microinr": 25020, + "retained_microinr": 31275, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9290839580062311, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-34-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-hi-m0", + "case_id": "fresh-34-hi", + "scenario": "fresh-34", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1911, + "completion_tokens": 6120, + "known_cost_microinr": 395865, + "retained_microinr": 494832, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 25.19336029200349, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-34-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-34-hi-m0", + "case_id": "fresh-34-hi", + "scenario": "fresh-34", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2099, + "completion_tokens": 741, + "known_cost_microinr": 75945, + "retained_microinr": 94932, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 3.773117333999835, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-34-hi-m0-source-chronological", + "history_id": "confirm-fresh-34-hi-m0", + "case_id": "fresh-34-hi", + "scenario": "fresh-34", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1676, + "completion_tokens": 2889, + "known_cost_microinr": 198480, + "retained_microinr": 248100, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 15.11056762500084, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-03-en-m0", + "case_id": "fresh-03-en", + "scenario": "fresh-03", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1438, + "completion_tokens": 1464, + "known_cost_microinr": 109410, + "retained_microinr": 136763, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.87887695900281, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-en-m0-source-chronological", + "history_id": "confirm-fresh-03-en-m0", + "case_id": "fresh-03-en", + "scenario": "fresh-03", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 892, + "completion_tokens": 5961, + "known_cost_microinr": 371040, + "retained_microinr": 463800, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 26.041728374999366, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-03-en-m0", + "case_id": "fresh-03-en", + "scenario": "fresh-03", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1403, + "completion_tokens": 911, + "known_cost_microinr": 75705, + "retained_microinr": 94632, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 5.302581875002943, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-07-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-07-hi-m0", + "case_id": "fresh-07-hi", + "scenario": "fresh-07", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1457, + "completion_tokens": 6840, + "known_cost_microinr": 432255, + "retained_microinr": 540319, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 28.69123499999114, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-07-hi-m0-source-chronological", + "history_id": "confirm-fresh-07-hi-m0", + "case_id": "fresh-07-hi", + "scenario": "fresh-07", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 899, + "completion_tokens": 1596, + "known_cost_microinr": 109245, + "retained_microinr": 136557, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.1098634590016445, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-07-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-hi-m0", + "case_id": "fresh-07-hi", + "scenario": "fresh-07", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1405, + "completion_tokens": 5151, + "known_cost_microinr": 330135, + "retained_microinr": 412669, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 22.43139166700712, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-11-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-11-en-m0", + "case_id": "fresh-11-en", + "scenario": "fresh-11", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1416, + "completion_tokens": 1728, + "known_cost_microinr": 124920, + "retained_microinr": 156150, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.231476125001791, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-11-en-m0-source-chronological", + "history_id": "confirm-fresh-11-en-m0", + "case_id": "fresh-11-en", + "scenario": "fresh-11", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 931, + "completion_tokens": 4610, + "known_cost_microinr": 290565, + "retained_microinr": 363207, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 19.237540375004755, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-11-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-11-en-m0", + "case_id": "fresh-11-en", + "scenario": "fresh-11", + "language": "en", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1478, + "completion_tokens": 1935, + "known_cost_microinr": 138270, + "retained_microinr": 172838, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.441571290997672, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-03-hi-m1", + "case_id": "fresh-03-hi", + "scenario": "fresh-03", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1418, + "completion_tokens": 42, + "known_cost_microinr": 23790, + "retained_microinr": 29738, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9592180410108995, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-03-hi-m1", + "case_id": "fresh-03-hi", + "scenario": "fresh-03", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1431, + "completion_tokens": 42, + "known_cost_microinr": 23985, + "retained_microinr": 29982, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0214134580019163, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-hi-m1-source-chronological", + "history_id": "confirm-fresh-03-hi-m1", + "case_id": "fresh-03-hi", + "scenario": "fresh-03", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 872, + "completion_tokens": 42, + "known_cost_microinr": 15600, + "retained_microinr": 19500, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0193654169997899, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-06-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-06-en-m0", + "case_id": "fresh-06-en", + "scenario": "fresh-06", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1242, + "completion_tokens": 2447, + "known_cost_microinr": 165450, + "retained_microinr": 206813, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 10.48811883298913, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-06-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-06-en-m0", + "case_id": "fresh-06-en", + "scenario": "fresh-06", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1300, + "completion_tokens": 1228, + "known_cost_microinr": 93180, + "retained_microinr": 116475, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 5.797769165990758, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-06-en-m0-source-chronological", + "history_id": "confirm-fresh-06-en-m0", + "case_id": "fresh-06-en", + "scenario": "fresh-06", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 865, + "completion_tokens": 1904, + "known_cost_microinr": 127215, + "retained_microinr": 159019, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 10.156719042002806, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-28-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-28-en-m0", + "case_id": "fresh-28-en", + "scenario": "fresh-28", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1829, + "completion_tokens": 2221, + "known_cost_microinr": 160695, + "retained_microinr": 200869, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.363244332998875, + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-28-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-28-en-m0", + "case_id": "fresh-28-en", + "scenario": "fresh-28", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1829, + "completion_tokens": 2816, + "known_cost_microinr": 196395, + "retained_microinr": 245494, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.541640791998361, + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-28-en-m0-source-chronological", + "history_id": "confirm-fresh-28-en-m0", + "case_id": "fresh-28-en", + "scenario": "fresh-28", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1316, + "completion_tokens": 1877, + "known_cost_microinr": 132360, + "retained_microinr": 165450, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.708016791002592, + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-20-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-20-te-m0", + "case_id": "fresh-20-te", + "scenario": "fresh-20", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1288, + "completion_tokens": 3797, + "known_cost_microinr": 247140, + "retained_microinr": 308925, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 19.656710999988718, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-20-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-20-te-m0", + "case_id": "fresh-20-te", + "scenario": "fresh-20", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": true, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1303, + "completion_tokens": 6569, + "known_cost_microinr": 413685, + "retained_microinr": 517107, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 27.481848375013215, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-20-te-m0-source-chronological", + "history_id": "confirm-fresh-20-te-m0", + "case_id": "fresh-20-te", + "scenario": "fresh-20", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 863, + "completion_tokens": 5405, + "known_cost_microinr": 337245, + "retained_microinr": 421557, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 26.097864416995435, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-hi-m0", + "case_id": "fresh-33-hi", + "scenario": "fresh-33", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2076, + "completion_tokens": 1493, + "known_cost_microinr": 120720, + "retained_microinr": 150900, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.439129499995033, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-hi-m0-source-chronological", + "history_id": "confirm-fresh-33-hi-m0", + "case_id": "fresh-33-hi", + "scenario": "fresh-33", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1870, + "completion_tokens": 2326, + "known_cost_microinr": 167610, + "retained_microinr": 209513, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 10.071834040994872, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-33-hi-m0", + "case_id": "fresh-33-hi", + "scenario": "fresh-33", + "language": "hi", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2418, + "completion_tokens": 4363, + "known_cost_microinr": 298050, + "retained_microinr": 372563, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 18.164840333003667, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-16-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-16-te-m1", + "case_id": "fresh-16-te", + "scenario": "fresh-16", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1793, + "completion_tokens": 42, + "known_cost_microinr": 29415, + "retained_microinr": 36769, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1345692500035511, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-16-te-m1-source-chronological", + "history_id": "confirm-fresh-16-te-m1", + "case_id": "fresh-16-te", + "scenario": "fresh-16", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": true, + "unknown_condition_guess": true, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1207, + "completion_tokens": 42, + "known_cost_microinr": 20625, + "retained_microinr": 25782, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9535644579882501, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-16-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-16-te-m1", + "case_id": "fresh-16-te", + "scenario": "fresh-16", + "language": "te", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1549, + "completion_tokens": 32, + "known_cost_microinr": 25155, + "retained_microinr": 31444, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0166007499938132, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-26-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-26-te-m0", + "case_id": "fresh-26-te", + "scenario": "fresh-26", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1836, + "completion_tokens": 1683, + "known_cost_microinr": 128520, + "retained_microinr": 160650, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.600142416995368, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-26-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-26-te-m0", + "case_id": "fresh-26-te", + "scenario": "fresh-26", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1836, + "completion_tokens": 8192, + "known_cost_microinr": 519060, + "retained_microinr": 648825, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 33.65626324999903, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-26-te-m0-source-chronological", + "history_id": "confirm-fresh-26-te-m0", + "case_id": "fresh-26-te", + "scenario": "fresh-26", + "language": "te", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1333, + "completion_tokens": 961, + "known_cost_microinr": 77655, + "retained_microinr": 97069, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 5.778320624987828, + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-20-en-m0-source-chronological", + "history_id": "confirm-fresh-20-en-m0", + "case_id": "fresh-20-en", + "scenario": "fresh-20", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 878, + "completion_tokens": 1403, + "known_cost_microinr": 97350, + "retained_microinr": 121688, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.568635333998827, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-20-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-20-en-m0", + "case_id": "fresh-20-en", + "scenario": "fresh-20", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1285, + "completion_tokens": 1934, + "known_cost_microinr": 135315, + "retained_microinr": 169144, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.803637250006432, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-20-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-20-en-m0", + "case_id": "fresh-20-en", + "scenario": "fresh-20", + "language": "en", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1319, + "completion_tokens": 8192, + "known_cost_microinr": 511305, + "retained_microinr": 639132, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 35.081754541999544, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-34-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-34-te-m1", + "case_id": "fresh-34-te", + "scenario": "fresh-34", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2518, + "completion_tokens": 42, + "known_cost_microinr": 40290, + "retained_microinr": 50363, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.064101041003596, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-34-te-m1-source-chronological", + "history_id": "confirm-fresh-34-te-m1", + "case_id": "fresh-34-te", + "scenario": "fresh-34", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1946, + "completion_tokens": 42, + "known_cost_microinr": 31710, + "retained_microinr": 39638, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9830327079980634, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-34-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-te-m1", + "case_id": "fresh-34-te", + "scenario": "fresh-34", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2218, + "completion_tokens": 42, + "known_cost_microinr": 35790, + "retained_microinr": 44738, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9962496660009492, + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-25-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-25-hi-m0", + "case_id": "fresh-25-hi", + "scenario": "fresh-25", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1990, + "completion_tokens": 4598, + "known_cost_microinr": 305730, + "retained_microinr": 382163, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 20.621692165994318, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-25-hi-m0-source-chronological", + "history_id": "confirm-fresh-25-hi-m0", + "case_id": "fresh-25-hi", + "scenario": "fresh-25", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1367, + "completion_tokens": 3511, + "known_cost_microinr": 231165, + "retained_microinr": 288957, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 15.08111558300152, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-25-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-25-hi-m0", + "case_id": "fresh-25-hi", + "scenario": "fresh-25", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1990, + "completion_tokens": 5323, + "known_cost_microinr": 349230, + "retained_microinr": 436538, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 24.895239624995156, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-14-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-14-en-m1", + "case_id": "fresh-14-en", + "scenario": "fresh-14", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1650, + "completion_tokens": 32, + "known_cost_microinr": 26670, + "retained_microinr": 33338, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0856207500037272, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-14-en-m1-source-chronological", + "history_id": "confirm-fresh-14-en-m1", + "case_id": "fresh-14-en", + "scenario": "fresh-14", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1071, + "completion_tokens": 32, + "known_cost_microinr": 17985, + "retained_microinr": 22482, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0340699160005897, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-14-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-14-en-m1", + "case_id": "fresh-14-en", + "scenario": "fresh-14", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1545, + "completion_tokens": 32, + "known_cost_microinr": 25095, + "retained_microinr": 31369, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0166327920014737, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-02-en-m1", + "case_id": "fresh-02-en", + "scenario": "fresh-02", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1661, + "completion_tokens": 42, + "known_cost_microinr": 27435, + "retained_microinr": 34294, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1004021249973448, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-en-m1-source-chronological", + "history_id": "confirm-fresh-02-en-m1", + "case_id": "fresh-02-en", + "scenario": "fresh-02", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1074, + "completion_tokens": 42, + "known_cost_microinr": 18630, + "retained_microinr": 23288, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9425836250011344, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-02-en-m1", + "case_id": "fresh-02-en", + "scenario": "fresh-02", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": true, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1543, + "completion_tokens": 32, + "known_cost_microinr": 25065, + "retained_microinr": 31332, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0150365410081577, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-22-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-22-en-m0", + "case_id": "fresh-22-en", + "scenario": "fresh-22", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1822, + "completion_tokens": 600, + "known_cost_microinr": 63330, + "retained_microinr": 79163, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 3.3389219170057913, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-22-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-22-en-m0", + "case_id": "fresh-22-en", + "scenario": "fresh-22", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1822, + "completion_tokens": 3638, + "known_cost_microinr": 245610, + "retained_microinr": 307013, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 17.630239791003987, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-22-en-m0-source-chronological", + "history_id": "confirm-fresh-22-en-m0", + "case_id": "fresh-22-en", + "scenario": "fresh-22", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1310, + "completion_tokens": 1417, + "known_cost_microinr": 104670, + "retained_microinr": 130838, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.776233458993374, + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-te-m0-source-chronological", + "history_id": "confirm-fresh-10-te-m0", + "case_id": "fresh-10-te", + "scenario": "fresh-10", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 901, + "completion_tokens": 4798, + "known_cost_microinr": 301395, + "retained_microinr": 376744, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 21.312018374999752, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-10-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-10-te-m0", + "case_id": "fresh-10-te", + "scenario": "fresh-10", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": true, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1251, + "completion_tokens": 5542, + "known_cost_microinr": 351285, + "retained_microinr": 439107, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 23.39471333300753, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-10-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-10-te-m0", + "case_id": "fresh-10-te", + "scenario": "fresh-10", + "language": "te", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1335, + "completion_tokens": 5865, + "known_cost_microinr": 371925, + "retained_microinr": 464907, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 26.295170958997915, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-17-hi-m0-source-chronological", + "history_id": "confirm-fresh-17-hi-m0", + "case_id": "fresh-17-hi", + "scenario": "fresh-17", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 996, + "completion_tokens": 2072, + "known_cost_microinr": 139260, + "retained_microinr": 174075, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.106979749994935, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-17-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-17-hi-m0", + "case_id": "fresh-17-hi", + "scenario": "fresh-17", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1547, + "completion_tokens": 1613, + "known_cost_microinr": 119985, + "retained_microinr": 149982, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.389643124988652, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-17-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-17-hi-m0", + "case_id": "fresh-17-hi", + "scenario": "fresh-17", + "language": "hi", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1435, + "completion_tokens": 5983, + "known_cost_microinr": 380505, + "retained_microinr": 475632, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 25.77854254200065, + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-32-hi-m0", + "case_id": "fresh-32-hi", + "scenario": "fresh-32", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2062, + "completion_tokens": 4343, + "known_cost_microinr": 291510, + "retained_microinr": 364388, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 21.686025832997984, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-hi-m0", + "case_id": "fresh-32-hi", + "scenario": "fresh-32", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1880, + "completion_tokens": 2957, + "known_cost_microinr": 205620, + "retained_microinr": 257025, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 14.982505040999968, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-hi-m0-source-chronological", + "history_id": "confirm-fresh-32-hi-m0", + "case_id": "fresh-32-hi", + "scenario": "fresh-32", + "language": "hi", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1472, + "completion_tokens": 2149, + "known_cost_microinr": 151020, + "retained_microinr": 188775, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.138451875012834, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-31-en-m0-source-chronological", + "history_id": "confirm-fresh-31-en-m0", + "case_id": "fresh-31-en", + "scenario": "fresh-31", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1332, + "completion_tokens": 3352, + "known_cost_microinr": 221100, + "retained_microinr": 276375, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 17.872049791010795, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-31-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-31-en-m0", + "case_id": "fresh-31-en", + "scenario": "fresh-31", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1878, + "completion_tokens": 3300, + "known_cost_microinr": 226170, + "retained_microinr": 282713, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 14.45017408298736, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-31-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-31-en-m0", + "case_id": "fresh-31-en", + "scenario": "fresh-31", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1736, + "completion_tokens": 3004, + "known_cost_microinr": 206280, + "retained_microinr": 257850, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 14.945332292001694, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-37-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-37-hi-m1", + "case_id": "fresh-37-hi", + "scenario": "fresh-37", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2529, + "completion_tokens": 42, + "known_cost_microinr": 40455, + "retained_microinr": 50569, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.995458375007729, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-37-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-hi-m1", + "case_id": "fresh-37-hi", + "scenario": "fresh-37", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2240, + "completion_tokens": 42, + "known_cost_microinr": 36120, + "retained_microinr": 45150, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9419233329972485, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-37-hi-m1-source-chronological", + "history_id": "confirm-fresh-37-hi-m1", + "case_id": "fresh-37-hi", + "scenario": "fresh-37", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1847, + "completion_tokens": 42, + "known_cost_microinr": 30225, + "retained_microinr": 37782, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.017738582988386, + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-39-en-m0", + "case_id": "fresh-39-en", + "scenario": "fresh-39", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2070, + "completion_tokens": 4753, + "known_cost_microinr": 316230, + "retained_microinr": 395288, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 20.51384858399979, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-en-m0-source-chronological", + "history_id": "confirm-fresh-39-en-m0", + "case_id": "fresh-39-en", + "scenario": "fresh-39", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1542, + "completion_tokens": 2798, + "known_cost_microinr": 191010, + "retained_microinr": 238763, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.931675290994463, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-en-m0", + "case_id": "fresh-39-en", + "scenario": "fresh-39", + "language": "en", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1942, + "completion_tokens": 5542, + "known_cost_microinr": 361650, + "retained_microinr": 452063, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 23.64272079100192, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-23-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-23-en-m0", + "case_id": "fresh-23-en", + "scenario": "fresh-23", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1979, + "completion_tokens": 1916, + "known_cost_microinr": 144645, + "retained_microinr": 180807, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 9.35879191600543, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-23-en-m0-source-chronological", + "history_id": "confirm-fresh-23-en-m0", + "case_id": "fresh-23-en", + "scenario": "fresh-23", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1359, + "completion_tokens": 1837, + "known_cost_microinr": 130605, + "retained_microinr": 163257, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.069823625002755, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-23-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-23-en-m0", + "case_id": "fresh-23-en", + "scenario": "fresh-23", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1979, + "completion_tokens": 6245, + "known_cost_microinr": 404385, + "retained_microinr": 505482, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 29.921276207998744, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-16-hi-m1-source-chronological", + "history_id": "confirm-fresh-16-hi-m1", + "case_id": "fresh-16-hi", + "scenario": "fresh-16", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 989, + "completion_tokens": 32, + "known_cost_microinr": 16755, + "retained_microinr": 20944, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.036998625000706, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-16-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-16-hi-m1", + "case_id": "fresh-16-hi", + "scenario": "fresh-16", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1435, + "completion_tokens": 32, + "known_cost_microinr": 23445, + "retained_microinr": 29307, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9819075410050573, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-16-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-16-hi-m1", + "case_id": "fresh-16-hi", + "scenario": "fresh-16", + "language": "hi", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1277, + "completion_tokens": 32, + "known_cost_microinr": 21075, + "retained_microinr": 26344, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1823998749896418, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-36-te-m0", + "case_id": "fresh-36-te", + "scenario": "fresh-36", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2199, + "completion_tokens": 2781, + "known_cost_microinr": 199845, + "retained_microinr": 249807, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.999567209000816, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-te-m0", + "case_id": "fresh-36-te", + "scenario": "fresh-36", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1936, + "completion_tokens": 4093, + "known_cost_microinr": 274620, + "retained_microinr": 343275, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 17.289486125009716, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-te-m0-source-chronological", + "history_id": "confirm-fresh-36-te-m0", + "case_id": "fresh-36-te", + "scenario": "fresh-36", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1769, + "completion_tokens": 1646, + "known_cost_microinr": 125295, + "retained_microinr": 156619, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.960662334007793, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-06-hi-m0-source-chronological", + "history_id": "confirm-fresh-06-hi-m0", + "case_id": "fresh-06-hi", + "scenario": "fresh-06", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 874, + "completion_tokens": 4327, + "known_cost_microinr": 272730, + "retained_microinr": 340913, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 21.645569083993905, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-06-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-06-hi-m0", + "case_id": "fresh-06-hi", + "scenario": "fresh-06", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1249, + "completion_tokens": 3263, + "known_cost_microinr": 214515, + "retained_microinr": 268144, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 15.321799875004217, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-06-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-06-hi-m0", + "case_id": "fresh-06-hi", + "scenario": "fresh-06", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1315, + "completion_tokens": 4073, + "known_cost_microinr": 264105, + "retained_microinr": 330132, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 19.05486554199888, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-38-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-te-m0", + "case_id": "fresh-38-te", + "scenario": "fresh-38", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1815, + "completion_tokens": 4196, + "known_cost_microinr": 278985, + "retained_microinr": 348732, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 20.786617167002987, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-38-te-m0", + "case_id": "fresh-38-te", + "scenario": "fresh-38", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2073, + "completion_tokens": 2919, + "known_cost_microinr": 206235, + "retained_microinr": 257794, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 14.626250207991689, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-38-te-m0-source-chronological", + "history_id": "confirm-fresh-38-te-m0", + "case_id": "fresh-38-te", + "scenario": "fresh-38", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1643, + "completion_tokens": 2288, + "known_cost_microinr": 161925, + "retained_microinr": 202407, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.175992250005947, + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-06-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-06-te-m1", + "case_id": "fresh-06-te", + "scenario": "fresh-06", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1256, + "completion_tokens": 32, + "known_cost_microinr": 20760, + "retained_microinr": 25950, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0905270420043962, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-06-te-m1-source-chronological", + "history_id": "confirm-fresh-06-te-m1", + "case_id": "fresh-06-te", + "scenario": "fresh-06", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 877, + "completion_tokens": 32, + "known_cost_microinr": 15075, + "retained_microinr": 18844, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.990318208001554, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-06-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-06-te-m1", + "case_id": "fresh-06-te", + "scenario": "fresh-06", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1314, + "completion_tokens": 32, + "known_cost_microinr": 21630, + "retained_microinr": 27038, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9463838749943534, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-hi-m0", + "case_id": "fresh-40-hi", + "scenario": "fresh-40", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1811, + "completion_tokens": 2601, + "known_cost_microinr": 183225, + "retained_microinr": 229032, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.333990916013136, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-40-hi-m0", + "case_id": "fresh-40-hi", + "scenario": "fresh-40", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2109, + "completion_tokens": 3320, + "known_cost_microinr": 230835, + "retained_microinr": 288544, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 17.365208292001626, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-40-hi-m0-source-chronological", + "history_id": "confirm-fresh-40-hi-m0", + "case_id": "fresh-40-hi", + "scenario": "fresh-40", + "language": "hi", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1680, + "completion_tokens": 5719, + "known_cost_microinr": 368340, + "retained_microinr": 460425, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 28.12782416700793, + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-01-en-m0-source-chronological", + "history_id": "confirm-fresh-01-en-m0", + "case_id": "fresh-01-en", + "scenario": "fresh-01", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 981, + "completion_tokens": 904, + "known_cost_microinr": 68955, + "retained_microinr": 86194, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 5.073199125006795, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-01-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-01-en-m0", + "case_id": "fresh-01-en", + "scenario": "fresh-01", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1526, + "completion_tokens": 2152, + "known_cost_microinr": 152010, + "retained_microinr": 190013, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.349641374996281, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-01-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-01-en-m0", + "case_id": "fresh-01-en", + "scenario": "fresh-01", + "language": "en", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1399, + "completion_tokens": 504, + "known_cost_microinr": 51225, + "retained_microinr": 64032, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 3.545431375008775, + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-04-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-04-hi-m0", + "case_id": "fresh-04-hi", + "scenario": "fresh-04", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1551, + "completion_tokens": 4620, + "known_cost_microinr": 300465, + "retained_microinr": 375582, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 21.67715354100801, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-04-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-04-hi-m0", + "case_id": "fresh-04-hi", + "scenario": "fresh-04", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1566, + "completion_tokens": 3202, + "known_cost_microinr": 215610, + "retained_microinr": 269513, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 14.017348583001876, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-04-hi-m0-source-chronological", + "history_id": "confirm-fresh-04-hi-m0", + "case_id": "fresh-04-hi", + "scenario": "fresh-04", + "language": "hi", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 972, + "completion_tokens": 1477, + "known_cost_microinr": 103200, + "retained_microinr": 129000, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 7.194728667003801, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-te-m1-source-chronological", + "history_id": "confirm-fresh-33-te-m1", + "case_id": "fresh-33-te", + "scenario": "fresh-33", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1979, + "completion_tokens": 42, + "known_cost_microinr": 32205, + "retained_microinr": 40257, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.065939709005761, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-te-m1", + "case_id": "fresh-33-te", + "scenario": "fresh-33", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2359, + "completion_tokens": 42, + "known_cost_microinr": 37905, + "retained_microinr": 47382, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.029099874998792, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-33-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-33-te-m1", + "case_id": "fresh-33-te", + "scenario": "fresh-33", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2656, + "completion_tokens": 42, + "known_cost_microinr": 42360, + "retained_microinr": 52950, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0094782920059515, + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-05-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-05-en-m1", + "case_id": "fresh-05-en", + "scenario": "fresh-05", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1693, + "completion_tokens": 32, + "known_cost_microinr": 27315, + "retained_microinr": 34144, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.10927879200608, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-05-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-05-en-m1", + "case_id": "fresh-05-en", + "scenario": "fresh-05", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1842, + "completion_tokens": 32, + "known_cost_microinr": 29550, + "retained_microinr": 36938, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0474977919948287, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-05-en-m1-source-chronological", + "history_id": "confirm-fresh-05-en-m1", + "case_id": "fresh-05-en", + "scenario": "fresh-05", + "language": "en", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1140, + "completion_tokens": 32, + "known_cost_microinr": 19020, + "retained_microinr": 23775, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0950979580084095, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-en-m0-source-chronological", + "history_id": "confirm-fresh-32-en-m0", + "case_id": "fresh-32-en", + "scenario": "fresh-32", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1283, + "completion_tokens": 820, + "known_cost_microinr": 68445, + "retained_microinr": 85557, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 4.343754665998858, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-en-m0", + "case_id": "fresh-32-en", + "scenario": "fresh-32", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1588, + "completion_tokens": 3043, + "known_cost_microinr": 206400, + "retained_microinr": 258000, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.495377250001184, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-32-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-32-en-m0", + "case_id": "fresh-32-en", + "scenario": "fresh-32", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1724, + "completion_tokens": 2736, + "known_cost_microinr": 190020, + "retained_microinr": 237525, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 11.708094542002073, + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-en-m0", + "case_id": "fresh-36-en", + "scenario": "fresh-36", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1902, + "completion_tokens": 4181, + "known_cost_microinr": 279390, + "retained_microinr": 349238, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 21.31061541600502, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-36-en-m0", + "case_id": "fresh-36-en", + "scenario": "fresh-36", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2093, + "completion_tokens": 1056, + "known_cost_microinr": 94755, + "retained_microinr": 118444, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.083063041995047, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-36-en-m0-source-chronological", + "history_id": "confirm-fresh-36-en-m0", + "case_id": "fresh-36-en", + "scenario": "fresh-36", + "language": "en", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1669, + "completion_tokens": 3253, + "known_cost_microinr": 220215, + "retained_microinr": 275269, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 15.779322165995836, + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-35-te-m0", + "case_id": "fresh-35-te", + "scenario": "fresh-35", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2400, + "completion_tokens": 4024, + "known_cost_microinr": 277440, + "retained_microinr": 346800, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 19.921081625012448, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-te-m0-source-chronological", + "history_id": "confirm-fresh-35-te-m0", + "case_id": "fresh-35-te", + "scenario": "fresh-35", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1863, + "completion_tokens": 2694, + "known_cost_microinr": 189585, + "retained_microinr": 236982, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.963635249994695, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-35-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-te-m0", + "case_id": "fresh-35-te", + "scenario": "fresh-35", + "language": "te", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2081, + "completion_tokens": 6210, + "known_cost_microinr": 403815, + "retained_microinr": 504769, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 27.812365375008085, + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-19-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-te-m1", + "case_id": "fresh-19-te", + "scenario": "fresh-19", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1434, + "completion_tokens": 42, + "known_cost_microinr": 24030, + "retained_microinr": 30038, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0013922919897595, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-19-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-19-te-m1", + "case_id": "fresh-19-te", + "scenario": "fresh-19", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1481, + "completion_tokens": 42, + "known_cost_microinr": 24735, + "retained_microinr": 30919, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9868211249995511, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-19-te-m1-source-chronological", + "history_id": "confirm-fresh-19-te-m1", + "case_id": "fresh-19-te", + "scenario": "fresh-19", + "language": "te", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "wrong_non_null_date": true, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 937, + "completion_tokens": 42, + "known_cost_microinr": 16575, + "retained_microinr": 20719, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0228360409964807, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-23-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-23-en-m1", + "case_id": "fresh-23-en", + "scenario": "fresh-23", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2256, + "completion_tokens": 40, + "known_cost_microinr": 36240, + "retained_microinr": 45300, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0312565420026658, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-23-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-23-en-m1", + "case_id": "fresh-23-en", + "scenario": "fresh-23", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2256, + "completion_tokens": 40, + "known_cost_microinr": 36240, + "retained_microinr": 45300, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9655432919971645, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-23-en-m1-source-chronological", + "history_id": "confirm-fresh-23-en-m1", + "case_id": "fresh-23-en", + "scenario": "fresh-23", + "language": "en", + "family": "approved_change", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1505, + "completion_tokens": 40, + "known_cost_microinr": 24975, + "retained_microinr": 31219, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9418319579999661, + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-31-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-31-te-m0", + "case_id": "fresh-31-te", + "scenario": "fresh-31", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 2027, + "completion_tokens": 8192, + "known_cost_microinr": 521925, + "retained_microinr": 652407, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 38.83640883299813, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-31-te-m0-source-chronological", + "history_id": "confirm-fresh-31-te-m0", + "case_id": "fresh-31-te", + "scenario": "fresh-31", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1485, + "completion_tokens": 6889, + "known_cost_microinr": 435615, + "retained_microinr": 544519, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 31.6984588750056, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-31-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-31-te-m0", + "case_id": "fresh-31-te", + "scenario": "fresh-31", + "language": "te", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2172, + "completion_tokens": 1033, + "known_cost_microinr": 94560, + "retained_microinr": 118200, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 6.013302583000041, + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-08-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-08-te-m0", + "case_id": "fresh-08-te", + "scenario": "fresh-08", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1529, + "completion_tokens": 2211, + "known_cost_microinr": 155595, + "retained_microinr": 194494, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.61775991700415, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-08-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-08-te-m0", + "case_id": "fresh-08-te", + "scenario": "fresh-08", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1646, + "completion_tokens": 2671, + "known_cost_microinr": 184950, + "retained_microinr": 231188, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.80287416599458, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-08-te-m0-source-chronological", + "history_id": "confirm-fresh-08-te-m0", + "case_id": "fresh-08-te", + "scenario": "fresh-08", + "language": "te", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1068, + "completion_tokens": 2573, + "known_cost_microinr": 170400, + "retained_microinr": 213000, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 13.018374209001195, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-te-m0-source-chronological", + "history_id": "confirm-fresh-03-te-m0", + "case_id": "fresh-03-te", + "scenario": "fresh-03", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 875, + "completion_tokens": 2678, + "known_cost_microinr": 173805, + "retained_microinr": 217257, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.408399916996132, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-03-te-m0", + "case_id": "fresh-03-te", + "scenario": "fresh-03", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1432, + "completion_tokens": 1634, + "known_cost_microinr": 119520, + "retained_microinr": 149400, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 8.721237082994776, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-03-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-03-te-m0", + "case_id": "fresh-03-te", + "scenario": "fresh-03", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1417, + "completion_tokens": 982, + "known_cost_microinr": 80175, + "retained_microinr": 100219, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 5.772524457992404, + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-25-en-m1-source-chronological", + "history_id": "confirm-fresh-25-en-m1", + "case_id": "fresh-25-en", + "scenario": "fresh-25", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1496, + "completion_tokens": 40, + "known_cost_microinr": 24840, + "retained_microinr": 31050, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9874125000060303, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-25-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-25-en-m1", + "case_id": "fresh-25-en", + "scenario": "fresh-25", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2257, + "completion_tokens": 40, + "known_cost_microinr": 36255, + "retained_microinr": 45319, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0323056659981376, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-25-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-25-en-m1", + "case_id": "fresh-25-en", + "scenario": "fresh-25", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2257, + "completion_tokens": 40, + "known_cost_microinr": 36255, + "retained_microinr": 45319, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0221163329988485, + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-29-en-m1", + "case_id": "fresh-29-en", + "scenario": "fresh-29", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2276, + "completion_tokens": 42, + "known_cost_microinr": 36660, + "retained_microinr": 45825, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9898309169948334, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-29-en-m1", + "case_id": "fresh-29-en", + "scenario": "fresh-29", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2276, + "completion_tokens": 42, + "known_cost_microinr": 36660, + "retained_microinr": 45825, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9798782500001835, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-29-en-m1-source-chronological", + "history_id": "confirm-fresh-29-en-m1", + "case_id": "fresh-29-en", + "scenario": "fresh-29", + "language": "en", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1506, + "completion_tokens": 40, + "known_cost_microinr": 24990, + "retained_microinr": 31238, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.036663791994215, + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-te-m1-source-chronological", + "history_id": "confirm-fresh-39-te-m1", + "case_id": "fresh-39-te", + "scenario": "fresh-39", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1928, + "completion_tokens": 40, + "known_cost_microinr": 31320, + "retained_microinr": 39150, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.989755000002333, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-te-m1", + "case_id": "fresh-39-te", + "scenario": "fresh-39", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2250, + "completion_tokens": 42, + "known_cost_microinr": 36270, + "retained_microinr": 45338, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1882771249947837, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-39-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-39-te-m1", + "case_id": "fresh-39-te", + "scenario": "fresh-39", + "language": "te", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2614, + "completion_tokens": 42, + "known_cost_microinr": 41730, + "retained_microinr": 52163, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.038521458991454, + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": false, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-28-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-28-hi-m0", + "case_id": "fresh-28-hi", + "scenario": "fresh-28", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2103, + "completion_tokens": 6263, + "known_cost_microinr": 407325, + "retained_microinr": 509157, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 28.466081584003405, + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-28-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-28-hi-m0", + "case_id": "fresh-28-hi", + "scenario": "fresh-28", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 2266, + "completion_tokens": 6103, + "known_cost_microinr": 400170, + "retained_microinr": 500213, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 29.778850125003373, + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-28-hi-m0-source-chronological", + "history_id": "confirm-fresh-28-hi-m0", + "case_id": "fresh-28-hi", + "scenario": "fresh-28", + "language": "hi", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1626, + "completion_tokens": 8192, + "known_cost_microinr": 515910, + "retained_microinr": 644888, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 37.17982000000484, + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-04-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-04-te-m1", + "case_id": "fresh-04-te", + "scenario": "fresh-04", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1291, + "completion_tokens": 42, + "known_cost_microinr": 21885, + "retained_microinr": 27357, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1146989169938024, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-04-te-m1-source-chronological", + "history_id": "confirm-fresh-04-te-m1", + "case_id": "fresh-04-te", + "scenario": "fresh-04", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 851, + "completion_tokens": 42, + "known_cost_microinr": 15285, + "retained_microinr": 19107, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9734913330030395, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-04-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-04-te-m1", + "case_id": "fresh-04-te", + "scenario": "fresh-04", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1268, + "completion_tokens": 42, + "known_cost_microinr": 21540, + "retained_microinr": 26925, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 0.9516997919999994, + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-te-m1-source-chronological", + "history_id": "confirm-fresh-02-te-m1", + "case_id": "fresh-02-te", + "scenario": "fresh-02", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 851, + "completion_tokens": 42, + "known_cost_microinr": 15285, + "retained_microinr": 19107, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.101985249988502, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-02-te-m1", + "case_id": "fresh-02-te", + "scenario": "fresh-02", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1292, + "completion_tokens": 42, + "known_cost_microinr": 21900, + "retained_microinr": 27375, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.0098189589916728, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-02-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-02-te-m1", + "case_id": "fresh-02-te", + "scenario": "fresh-02", + "language": "te", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1269, + "completion_tokens": 42, + "known_cost_microinr": 21555, + "retained_microinr": 26944, + "full_reservation_microinr": 1228800 + }, + "elapsed_seconds": 1.1945389590109698, + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-08-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-08-hi-m0", + "case_id": "fresh-08-hi", + "scenario": "fresh-08", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1394, + "completion_tokens": 4587, + "known_cost_microinr": 296130, + "retained_microinr": 370163, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 24.660657500004163, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-08-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-08-hi-m0", + "case_id": "fresh-08-hi", + "scenario": "fresh-08", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1246, + "completion_tokens": 5658, + "known_cost_microinr": 358170, + "retained_microinr": 447713, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 29.663377082993975, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-08-hi-m0-source-chronological", + "history_id": "confirm-fresh-08-hi-m0", + "case_id": "fresh-08-hi", + "scenario": "fresh-08", + "language": "hi", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 961, + "completion_tokens": 4841, + "known_cost_microinr": 304875, + "retained_microinr": 381094, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 25.56791416699707, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": false, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-27-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-27-en-m0", + "case_id": "fresh-27-en", + "scenario": "fresh-27", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1980, + "completion_tokens": 5687, + "known_cost_microinr": 370920, + "retained_microinr": 463650, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 28.47024195799895, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-27-en-m0-source-chronological", + "history_id": "confirm-fresh-27-en-m0", + "case_id": "fresh-27-en", + "scenario": "fresh-27", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1357, + "completion_tokens": 1074, + "known_cost_microinr": 84795, + "retained_microinr": 105994, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 4.93337350001093, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-27-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-27-en-m0", + "case_id": "fresh-27-en", + "scenario": "fresh-27", + "language": "en", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "application_output": null, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": false, + "projected_fully_correct": false, + "usage": { + "known": true, + "prompt_tokens": 1980, + "completion_tokens": 8192, + "known_cost_microinr": 521220, + "retained_microinr": 651525, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 41.52383133400872, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "factual_answer_correct": false, + "raw_status_matches_execution": false, + "application_status_matches_execution": false, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": false + }, + { + "id": "confirm-fresh-14-en-m0-source-chronological", + "history_id": "confirm-fresh-14-en-m0", + "case_id": "fresh-14-en", + "scenario": "fresh-14", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 980, + "completion_tokens": 2291, + "known_cost_microinr": 152160, + "retained_microinr": 190200, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.110416083000018, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-14-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-14-en-m0", + "case_id": "fresh-14-en", + "scenario": "fresh-14", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1253, + "completion_tokens": 2352, + "known_cost_microinr": 159915, + "retained_microinr": 199894, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 12.266609625003184, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-14-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-14-en-m0", + "case_id": "fresh-14-en", + "scenario": "fresh-14", + "language": "en", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1409, + "completion_tokens": 4713, + "known_cost_microinr": 303915, + "retained_microinr": 379894, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 20.918983416995616, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-11-hi-m0-source-chronological", + "history_id": "confirm-fresh-11-hi-m0", + "case_id": "fresh-11-hi", + "scenario": "fresh-11", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1118, + "completion_tokens": 5047, + "known_cost_microinr": 319590, + "retained_microinr": 399488, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 20.888820915992255, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-11-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-11-hi-m0", + "case_id": "fresh-11-hi", + "scenario": "fresh-11", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1820, + "completion_tokens": 2889, + "known_cost_microinr": 200640, + "retained_microinr": 250800, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 14.57691537500068, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + }, + { + "id": "confirm-fresh-11-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-11-hi-m0", + "case_id": "fresh-11-hi", + "scenario": "fresh-11", + "language": "hi", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "wrong_non_null_date": false, + "false_abstention": false, + "unsupported_non_null_on_null_target": false, + "unknown_condition_guess": false, + "raw_fully_correct": true, + "projected_fully_correct": true, + "usage": { + "known": true, + "prompt_tokens": 1708, + "completion_tokens": 3278, + "known_cost_microinr": 222300, + "retained_microinr": 277875, + "full_reservation_microinr": 3072000 + }, + "elapsed_seconds": 16.104989667001064, + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "factual_answer_correct": true, + "raw_status_matches_execution": true, + "application_status_matches_execution": true, + "desired_state": true, + "prohibited_state_change": false, + "unsafe_change_truthfully_reported": false, + "safe_fact_and_state": true + } + ], + "actual_dispatch_order": [ + { + "id": "confirm-fresh-31-en-m1-source-grouped_lossless", + "dispatch_time": 1791942452.424167, + "planned_position": 2 + }, + { + "id": "confirm-fresh-22-hi-m1-source-chronological", + "dispatch_time": 1791942453.093348, + "planned_position": 0 + }, + { + "id": "confirm-fresh-31-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942453.757668, + "planned_position": 1 + }, + { + "id": "confirm-fresh-31-en-m1-source-chronological", + "dispatch_time": 1791942454.425504, + "planned_position": 0 + }, + { + "id": "confirm-fresh-24-hi-m1-source-grouped_lossless", + "dispatch_time": 1791942455.50863, + "planned_position": 0 + }, + { + "id": "confirm-fresh-22-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942456.172561, + "planned_position": 2 + }, + { + "id": "confirm-fresh-22-hi-m1-source-grouped_lossless", + "dispatch_time": 1791942456.8428721, + "planned_position": 1 + }, + { + "id": "confirm-fresh-24-hi-m1-source-chronological", + "dispatch_time": 1791942457.509232, + "planned_position": 1 + }, + { + "id": "confirm-fresh-24-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942458.50993, + "planned_position": 2 + }, + { + "id": "confirm-fresh-29-hi-m0-source-grouped_lossless", + "dispatch_time": 1791942459.177248, + "planned_position": 1 + }, + { + "id": "confirm-fresh-29-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942459.843329, + "planned_position": 2 + }, + { + "id": "confirm-fresh-29-hi-m0-source-chronological", + "dispatch_time": 1791942460.507543, + "planned_position": 0 + }, + { + "id": "confirm-fresh-18-en-m1-source-chronological", + "dispatch_time": 1791942493.574848, + "planned_position": 0 + }, + { + "id": "confirm-fresh-18-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942494.241298, + "planned_position": 2 + }, + { + "id": "confirm-fresh-18-en-m1-source-grouped_lossless", + "dispatch_time": 1791942494.904854, + "planned_position": 1 + }, + { + "id": "confirm-fresh-34-hi-m1-source-chronological", + "dispatch_time": 1791942495.573548, + "planned_position": 0 + }, + { + "id": "confirm-fresh-34-hi-m1-source-grouped_lossless", + "dispatch_time": 1791942496.7328079, + "planned_position": 1 + }, + { + "id": "confirm-fresh-32-te-m0-source-grouped_lossless", + "dispatch_time": 1791942497.4003751, + "planned_position": 1 + }, + { + "id": "confirm-fresh-34-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942498.064251, + "planned_position": 2 + }, + { + "id": "confirm-fresh-32-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942498.732119, + "planned_position": 0 + }, + { + "id": "confirm-fresh-32-te-m0-source-chronological", + "dispatch_time": 1791942509.357515, + "planned_position": 2 + }, + { + "id": "confirm-fresh-30-te-m0-source-chronological", + "dispatch_time": 1791942510.022159, + "planned_position": 2 + }, + { + "id": "confirm-fresh-30-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942510.6928241, + "planned_position": 1 + }, + { + "id": "confirm-fresh-30-te-m0-source-grouped_lossless", + "dispatch_time": 1791942511.358352, + "planned_position": 0 + }, + { + "id": "confirm-fresh-30-hi-m0-source-grouped_lossless", + "dispatch_time": 1791942525.71189, + "planned_position": 2 + }, + { + "id": "confirm-fresh-17-te-m0-source-grouped_lossless", + "dispatch_time": 1791942526.3774478, + "planned_position": 0 + }, + { + "id": "confirm-fresh-30-hi-m0-source-chronological", + "dispatch_time": 1791942527.043194, + "planned_position": 0 + }, + { + "id": "confirm-fresh-30-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942527.709756, + "planned_position": 1 + }, + { + "id": "confirm-fresh-17-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942562.7848551, + "planned_position": 2 + }, + { + "id": "confirm-fresh-17-te-m0-source-chronological", + "dispatch_time": 1791942563.4484422, + "planned_position": 1 + }, + { + "id": "confirm-fresh-23-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942564.1171992, + "planned_position": 1 + }, + { + "id": "confirm-fresh-23-hi-m1-source-chronological", + "dispatch_time": 1791942564.7825432, + "planned_position": 0 + }, + { + "id": "confirm-fresh-23-hi-m0-source-chronological", + "dispatch_time": 1791942586.934761, + "planned_position": 0 + }, + { + "id": "confirm-fresh-23-hi-m0-source-grouped_lossless", + "dispatch_time": 1791942587.6026728, + "planned_position": 1 + }, + { + "id": "confirm-fresh-23-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942588.268576, + "planned_position": 2 + }, + { + "id": "confirm-fresh-23-hi-m1-source-grouped_lossless", + "dispatch_time": 1791942588.9341068, + "planned_position": 2 + }, + { + "id": "confirm-fresh-18-hi-m1-source-grouped_lossless", + "dispatch_time": 1791942601.1877382, + "planned_position": 0 + }, + { + "id": "confirm-fresh-18-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942601.8547351, + "planned_position": 2 + }, + { + "id": "confirm-fresh-18-hi-m1-source-chronological", + "dispatch_time": 1791942602.520355, + "planned_position": 1 + }, + { + "id": "confirm-fresh-24-hi-m0-source-chronological", + "dispatch_time": 1791942603.188118, + "planned_position": 0 + }, + { + "id": "confirm-fresh-24-hi-m0-source-grouped_lossless", + "dispatch_time": 1791942616.791264, + "planned_position": 2 + }, + { + "id": "confirm-fresh-39-hi-m1-source-grouped_lossless", + "dispatch_time": 1791942617.455621, + "planned_position": 1 + }, + { + "id": "confirm-fresh-24-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942618.124331, + "planned_position": 1 + }, + { + "id": "confirm-fresh-39-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942618.7880168, + "planned_position": 0 + }, + { + "id": "confirm-fresh-39-hi-m1-source-chronological", + "dispatch_time": 1791942633.430847, + "planned_position": 2 + }, + { + "id": "confirm-fresh-35-en-m0-source-chronological", + "dispatch_time": 1791942634.096871, + "planned_position": 1 + }, + { + "id": "confirm-fresh-35-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942634.7628238, + "planned_position": 0 + }, + { + "id": "confirm-fresh-35-en-m0-source-grouped_lossless", + "dispatch_time": 1791942635.428457, + "planned_position": 2 + }, + { + "id": "confirm-fresh-36-en-m1-source-chronological", + "dispatch_time": 1791942662.058728, + "planned_position": 0 + }, + { + "id": "confirm-fresh-36-en-m1-source-grouped_lossless", + "dispatch_time": 1791942662.72138, + "planned_position": 1 + }, + { + "id": "confirm-fresh-36-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942663.392384, + "planned_position": 2 + }, + { + "id": "confirm-fresh-08-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942664.059351, + "planned_position": 0 + }, + { + "id": "confirm-fresh-08-hi-m1-source-grouped_lossless", + "dispatch_time": 1791942665.035003, + "planned_position": 2 + }, + { + "id": "confirm-fresh-21-hi-m1-source-grouped_lossless", + "dispatch_time": 1791942665.70188, + "planned_position": 1 + }, + { + "id": "confirm-fresh-08-hi-m1-source-chronological", + "dispatch_time": 1791942666.3663192, + "planned_position": 1 + }, + { + "id": "confirm-fresh-21-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942667.033622, + "planned_position": 0 + }, + { + "id": "confirm-fresh-25-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942668.168747, + "planned_position": 0 + }, + { + "id": "confirm-fresh-25-te-m0-source-grouped_lossless", + "dispatch_time": 1791942668.836021, + "planned_position": 2 + }, + { + "id": "confirm-fresh-21-hi-m1-source-chronological", + "dispatch_time": 1791942669.501774, + "planned_position": 2 + }, + { + "id": "confirm-fresh-25-te-m0-source-chronological", + "dispatch_time": 1791942670.169023, + "planned_position": 1 + }, + { + "id": "confirm-fresh-36-te-m1-source-chronological", + "dispatch_time": 1791942707.635545, + "planned_position": 1 + }, + { + "id": "confirm-fresh-36-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942708.301461, + "planned_position": 0 + }, + { + "id": "confirm-fresh-13-hi-m1-source-grouped_lossless", + "dispatch_time": 1791942708.9672909, + "planned_position": 0 + }, + { + "id": "confirm-fresh-36-te-m1-source-grouped_lossless", + "dispatch_time": 1791942709.6339781, + "planned_position": 2 + }, + { + "id": "confirm-fresh-13-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942710.643198, + "planned_position": 1 + }, + { + "id": "confirm-fresh-29-te-m0-source-chronological", + "dispatch_time": 1791942711.308542, + "planned_position": 1 + }, + { + "id": "confirm-fresh-13-hi-m1-source-chronological", + "dispatch_time": 1791942711.9746408, + "planned_position": 2 + }, + { + "id": "confirm-fresh-29-te-m0-source-grouped_lossless", + "dispatch_time": 1791942712.6417978, + "planned_position": 0 + }, + { + "id": "confirm-fresh-29-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942723.857196, + "planned_position": 2 + }, + { + "id": "confirm-fresh-26-hi-m0-source-grouped_lossless", + "dispatch_time": 1791942724.5243552, + "planned_position": 0 + }, + { + "id": "confirm-fresh-26-hi-m0-source-chronological", + "dispatch_time": 1791942725.1900449, + "planned_position": 2 + }, + { + "id": "confirm-fresh-26-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942725.85753, + "planned_position": 1 + }, + { + "id": "confirm-fresh-01-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942765.356742, + "planned_position": 2 + }, + { + "id": "confirm-fresh-01-hi-m0-source-grouped_lossless", + "dispatch_time": 1791942766.022936, + "planned_position": 0 + }, + { + "id": "confirm-fresh-24-en-m1-source-chronological", + "dispatch_time": 1791942766.6881459, + "planned_position": 0 + }, + { + "id": "confirm-fresh-01-hi-m0-source-chronological", + "dispatch_time": 1791942767.35658, + "planned_position": 1 + }, + { + "id": "confirm-fresh-24-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942795.0438159, + "planned_position": 2 + }, + { + "id": "confirm-fresh-02-te-m0-source-grouped_lossless", + "dispatch_time": 1791942795.711076, + "planned_position": 1 + }, + { + "id": "confirm-fresh-02-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942796.373257, + "planned_position": 0 + }, + { + "id": "confirm-fresh-24-en-m1-source-grouped_lossless", + "dispatch_time": 1791942797.044418, + "planned_position": 1 + }, + { + "id": "confirm-fresh-02-te-m0-source-chronological", + "dispatch_time": 1791942812.1151638, + "planned_position": 2 + }, + { + "id": "confirm-fresh-09-hi-m0-source-chronological", + "dispatch_time": 1791942812.782964, + "planned_position": 1 + }, + { + "id": "confirm-fresh-09-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942813.448718, + "planned_position": 0 + }, + { + "id": "confirm-fresh-09-hi-m0-source-grouped_lossless", + "dispatch_time": 1791942814.116118, + "planned_position": 2 + }, + { + "id": "confirm-fresh-04-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942832.461181, + "planned_position": 1 + }, + { + "id": "confirm-fresh-04-te-m0-source-grouped_lossless", + "dispatch_time": 1791942833.132597, + "planned_position": 2 + }, + { + "id": "confirm-fresh-13-hi-m0-source-grouped_lossless", + "dispatch_time": 1791942833.796834, + "planned_position": 0 + }, + { + "id": "confirm-fresh-04-te-m0-source-chronological", + "dispatch_time": 1791942834.463371, + "planned_position": 0 + }, + { + "id": "confirm-fresh-13-hi-m0-source-chronological", + "dispatch_time": 1791942866.0200691, + "planned_position": 1 + }, + { + "id": "confirm-fresh-31-hi-m0-source-chronological", + "dispatch_time": 1791942866.6890829, + "planned_position": 1 + }, + { + "id": "confirm-fresh-13-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942867.355728, + "planned_position": 2 + }, + { + "id": "confirm-fresh-31-hi-m0-source-grouped_lossless", + "dispatch_time": 1791942868.017834, + "planned_position": 0 + }, + { + "id": "confirm-fresh-31-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942903.7856011, + "planned_position": 2 + }, + { + "id": "confirm-fresh-33-te-m0-source-chronological", + "dispatch_time": 1791942904.45278, + "planned_position": 0 + }, + { + "id": "confirm-fresh-33-te-m0-source-grouped_lossless", + "dispatch_time": 1791942905.119431, + "planned_position": 1 + }, + { + "id": "confirm-fresh-33-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942905.78611, + "planned_position": 2 + }, + { + "id": "confirm-fresh-25-hi-m1-source-grouped_lossless", + "dispatch_time": 1791942914.758796, + "planned_position": 0 + }, + { + "id": "confirm-fresh-25-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942915.426935, + "planned_position": 1 + }, + { + "id": "confirm-fresh-25-hi-m1-source-chronological", + "dispatch_time": 1791942916.093149, + "planned_position": 2 + }, + { + "id": "confirm-fresh-32-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942916.759826, + "planned_position": 0 + }, + { + "id": "confirm-fresh-15-hi-m1-source-grouped_lossless", + "dispatch_time": 1791942917.73974, + "planned_position": 1 + }, + { + "id": "confirm-fresh-15-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942918.401498, + "planned_position": 0 + }, + { + "id": "confirm-fresh-32-hi-m1-source-grouped_lossless", + "dispatch_time": 1791942919.071709, + "planned_position": 2 + }, + { + "id": "confirm-fresh-32-hi-m1-source-chronological", + "dispatch_time": 1791942919.736573, + "planned_position": 1 + }, + { + "id": "confirm-fresh-15-hi-m1-source-chronological", + "dispatch_time": 1791942920.7758029, + "planned_position": 2 + }, + { + "id": "confirm-fresh-35-hi-m1-source-grouped_lossless", + "dispatch_time": 1791942921.439119, + "planned_position": 2 + }, + { + "id": "confirm-fresh-35-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942922.104636, + "planned_position": 1 + }, + { + "id": "confirm-fresh-35-hi-m1-source-chronological", + "dispatch_time": 1791942922.768995, + "planned_position": 0 + }, + { + "id": "confirm-fresh-20-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942923.759349, + "planned_position": 0 + }, + { + "id": "confirm-fresh-20-hi-m1-source-grouped_lossless", + "dispatch_time": 1791942924.424695, + "planned_position": 2 + }, + { + "id": "confirm-fresh-40-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942925.094044, + "planned_position": 0 + }, + { + "id": "confirm-fresh-20-hi-m1-source-chronological", + "dispatch_time": 1791942925.755979, + "planned_position": 1 + }, + { + "id": "confirm-fresh-21-en-m0-source-grouped_lossless", + "dispatch_time": 1791942926.883471, + "planned_position": 0 + }, + { + "id": "confirm-fresh-40-en-m1-source-grouped_lossless", + "dispatch_time": 1791942927.543145, + "planned_position": 2 + }, + { + "id": "confirm-fresh-21-en-m0-source-chronological", + "dispatch_time": 1791942928.2086852, + "planned_position": 1 + }, + { + "id": "confirm-fresh-40-en-m1-source-chronological", + "dispatch_time": 1791942928.877773, + "planned_position": 1 + }, + { + "id": "confirm-fresh-37-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942936.5975602, + "planned_position": 2 + }, + { + "id": "confirm-fresh-37-hi-m0-source-chronological", + "dispatch_time": 1791942937.266712, + "planned_position": 1 + }, + { + "id": "confirm-fresh-21-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942937.933284, + "planned_position": 2 + }, + { + "id": "confirm-fresh-37-hi-m0-source-grouped_lossless", + "dispatch_time": 1791942938.5982652, + "planned_position": 0 + }, + { + "id": "confirm-fresh-28-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942974.410507, + "planned_position": 1 + }, + { + "id": "confirm-fresh-07-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791942975.078476, + "planned_position": 0 + }, + { + "id": "confirm-fresh-28-te-m0-source-chronological", + "dispatch_time": 1791942975.745045, + "planned_position": 0 + }, + { + "id": "confirm-fresh-28-te-m0-source-grouped_lossless", + "dispatch_time": 1791942976.411817, + "planned_position": 2 + }, + { + "id": "confirm-fresh-07-te-m0-source-chronological", + "dispatch_time": 1791943005.460192, + "planned_position": 1 + }, + { + "id": "confirm-fresh-09-te-m0-source-grouped_lossless", + "dispatch_time": 1791943006.12867, + "planned_position": 1 + }, + { + "id": "confirm-fresh-09-te-m0-source-chronological", + "dispatch_time": 1791943006.790838, + "planned_position": 0 + }, + { + "id": "confirm-fresh-07-te-m0-source-grouped_lossless", + "dispatch_time": 1791943007.459328, + "planned_position": 2 + }, + { + "id": "confirm-fresh-09-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943035.1651268, + "planned_position": 2 + }, + { + "id": "confirm-fresh-03-te-m1-source-grouped_lossless", + "dispatch_time": 1791943035.830751, + "planned_position": 2 + }, + { + "id": "confirm-fresh-03-te-m1-source-chronological", + "dispatch_time": 1791943036.500198, + "planned_position": 0 + }, + { + "id": "confirm-fresh-03-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943037.163568, + "planned_position": 1 + }, + { + "id": "confirm-fresh-28-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943052.995065, + "planned_position": 0 + }, + { + "id": "confirm-fresh-03-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943053.6620018, + "planned_position": 1 + }, + { + "id": "confirm-fresh-03-en-m1-source-chronological", + "dispatch_time": 1791943054.323953, + "planned_position": 2 + }, + { + "id": "confirm-fresh-03-en-m1-source-grouped_lossless", + "dispatch_time": 1791943054.995341, + "planned_position": 0 + }, + { + "id": "confirm-fresh-28-en-m1-source-grouped_lossless", + "dispatch_time": 1791943056.134769, + "planned_position": 2 + }, + { + "id": "confirm-fresh-22-hi-m0-source-grouped_lossless", + "dispatch_time": 1791943056.800215, + "planned_position": 1 + }, + { + "id": "confirm-fresh-28-en-m1-source-chronological", + "dispatch_time": 1791943057.465611, + "planned_position": 1 + }, + { + "id": "confirm-fresh-22-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943058.133663, + "planned_position": 0 + }, + { + "id": "confirm-fresh-04-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943078.733022, + "planned_position": 1 + }, + { + "id": "confirm-fresh-22-hi-m0-source-chronological", + "dispatch_time": 1791943079.398384, + "planned_position": 2 + }, + { + "id": "confirm-fresh-04-en-m0-source-grouped_lossless", + "dispatch_time": 1791943080.0642219, + "planned_position": 0 + }, + { + "id": "confirm-fresh-04-en-m0-source-chronological", + "dispatch_time": 1791943080.731762, + "planned_position": 2 + }, + { + "id": "confirm-fresh-14-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943105.4236228, + "planned_position": 0 + }, + { + "id": "confirm-fresh-16-en-m0-source-grouped_lossless", + "dispatch_time": 1791943106.090813, + "planned_position": 0 + }, + { + "id": "confirm-fresh-16-en-m0-source-chronological", + "dispatch_time": 1791943106.7558668, + "planned_position": 2 + }, + { + "id": "confirm-fresh-16-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943107.423403, + "planned_position": 1 + }, + { + "id": "confirm-fresh-02-hi-m1-source-chronological", + "dispatch_time": 1791943131.965833, + "planned_position": 1 + }, + { + "id": "confirm-fresh-14-te-m0-source-grouped_lossless", + "dispatch_time": 1791943132.6323898, + "planned_position": 1 + }, + { + "id": "confirm-fresh-02-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943133.299575, + "planned_position": 0 + }, + { + "id": "confirm-fresh-14-te-m0-source-chronological", + "dispatch_time": 1791943133.9662561, + "planned_position": 2 + }, + { + "id": "confirm-fresh-15-te-m0-source-chronological", + "dispatch_time": 1791943160.992556, + "planned_position": 0 + }, + { + "id": "confirm-fresh-02-hi-m1-source-grouped_lossless", + "dispatch_time": 1791943161.658289, + "planned_position": 2 + }, + { + "id": "confirm-fresh-15-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943162.32596, + "planned_position": 2 + }, + { + "id": "confirm-fresh-15-te-m0-source-grouped_lossless", + "dispatch_time": 1791943162.9925592, + "planned_position": 1 + }, + { + "id": "confirm-fresh-19-hi-m0-source-chronological", + "dispatch_time": 1791943185.8633258, + "planned_position": 1 + }, + { + "id": "confirm-fresh-19-hi-m0-source-grouped_lossless", + "dispatch_time": 1791943186.531148, + "planned_position": 0 + }, + { + "id": "confirm-fresh-15-en-m0-source-grouped_lossless", + "dispatch_time": 1791943187.198241, + "planned_position": 0 + }, + { + "id": "confirm-fresh-19-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943187.8645508, + "planned_position": 2 + }, + { + "id": "confirm-fresh-15-en-m0-source-chronological", + "dispatch_time": 1791943210.2742112, + "planned_position": 1 + }, + { + "id": "confirm-fresh-24-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943210.9381602, + "planned_position": 0 + }, + { + "id": "confirm-fresh-15-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943211.608254, + "planned_position": 2 + }, + { + "id": "confirm-fresh-24-en-m0-source-grouped_lossless", + "dispatch_time": 1791943212.272355, + "planned_position": 1 + }, + { + "id": "confirm-fresh-24-en-m0-source-chronological", + "dispatch_time": 1791943229.219827, + "planned_position": 2 + }, + { + "id": "confirm-fresh-32-te-m1-source-grouped_lossless", + "dispatch_time": 1791943229.887452, + "planned_position": 1 + }, + { + "id": "confirm-fresh-32-te-m1-source-chronological", + "dispatch_time": 1791943230.5511081, + "planned_position": 0 + }, + { + "id": "confirm-fresh-32-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943231.218207, + "planned_position": 2 + }, + { + "id": "confirm-fresh-22-te-m1-source-grouped_lossless", + "dispatch_time": 1791943235.5148852, + "planned_position": 0 + }, + { + "id": "confirm-fresh-22-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943236.181613, + "planned_position": 2 + }, + { + "id": "confirm-fresh-08-en-m1-source-grouped_lossless", + "dispatch_time": 1791943236.848316, + "planned_position": 0 + }, + { + "id": "confirm-fresh-22-te-m1-source-chronological", + "dispatch_time": 1791943237.5172029, + "planned_position": 1 + }, + { + "id": "confirm-fresh-08-en-m1-source-chronological", + "dispatch_time": 1791943238.577937, + "planned_position": 1 + }, + { + "id": "confirm-fresh-34-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943239.240457, + "planned_position": 0 + }, + { + "id": "confirm-fresh-08-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943239.910572, + "planned_position": 2 + }, + { + "id": "confirm-fresh-34-en-m1-source-chronological", + "dispatch_time": 1791943240.577135, + "planned_position": 1 + }, + { + "id": "confirm-fresh-37-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943241.7174559, + "planned_position": 1 + }, + { + "id": "confirm-fresh-37-en-m1-source-grouped_lossless", + "dispatch_time": 1791943242.385756, + "planned_position": 2 + }, + { + "id": "confirm-fresh-34-en-m1-source-grouped_lossless", + "dispatch_time": 1791943243.0524192, + "planned_position": 2 + }, + { + "id": "confirm-fresh-37-en-m1-source-chronological", + "dispatch_time": 1791943243.715254, + "planned_position": 0 + }, + { + "id": "confirm-fresh-36-hi-m0-source-grouped_lossless", + "dispatch_time": 1791943244.869296, + "planned_position": 2 + }, + { + "id": "confirm-fresh-10-en-m0-source-grouped_lossless", + "dispatch_time": 1791943245.534525, + "planned_position": 0 + }, + { + "id": "confirm-fresh-36-hi-m0-source-chronological", + "dispatch_time": 1791943246.199415, + "planned_position": 0 + }, + { + "id": "confirm-fresh-36-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943246.866103, + "planned_position": 1 + }, + { + "id": "confirm-fresh-09-en-m0-source-grouped_lossless", + "dispatch_time": 1791943276.853749, + "planned_position": 0 + }, + { + "id": "confirm-fresh-10-en-m0-source-chronological", + "dispatch_time": 1791943277.518649, + "planned_position": 2 + }, + { + "id": "confirm-fresh-10-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943278.1819751, + "planned_position": 1 + }, + { + "id": "confirm-fresh-09-en-m0-source-chronological", + "dispatch_time": 1791943278.84978, + "planned_position": 1 + }, + { + "id": "confirm-fresh-10-hi-m0-source-chronological", + "dispatch_time": 1791943303.340543, + "planned_position": 2 + }, + { + "id": "confirm-fresh-10-hi-m0-source-grouped_lossless", + "dispatch_time": 1791943304.0069098, + "planned_position": 1 + }, + { + "id": "confirm-fresh-09-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943304.6738229, + "planned_position": 2 + }, + { + "id": "confirm-fresh-10-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943305.3440251, + "planned_position": 0 + }, + { + "id": "confirm-fresh-14-te-m1-source-chronological", + "dispatch_time": 1791943324.8437078, + "planned_position": 0 + }, + { + "id": "confirm-fresh-30-hi-m1-source-grouped_lossless", + "dispatch_time": 1791943325.503287, + "planned_position": 0 + }, + { + "id": "confirm-fresh-14-te-m1-source-grouped_lossless", + "dispatch_time": 1791943326.172512, + "planned_position": 1 + }, + { + "id": "confirm-fresh-14-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943326.839175, + "planned_position": 2 + }, + { + "id": "confirm-fresh-30-hi-m1-source-chronological", + "dispatch_time": 1791943327.864049, + "planned_position": 1 + }, + { + "id": "confirm-fresh-30-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943328.528097, + "planned_position": 2 + }, + { + "id": "confirm-fresh-31-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943329.196558, + "planned_position": 1 + }, + { + "id": "confirm-fresh-31-hi-m1-source-grouped_lossless", + "dispatch_time": 1791943329.863227, + "planned_position": 0 + }, + { + "id": "confirm-fresh-39-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943330.858967, + "planned_position": 1 + }, + { + "id": "confirm-fresh-39-en-m1-source-grouped_lossless", + "dispatch_time": 1791943331.520316, + "planned_position": 0 + }, + { + "id": "confirm-fresh-31-hi-m1-source-chronological", + "dispatch_time": 1791943332.187031, + "planned_position": 2 + }, + { + "id": "confirm-fresh-39-en-m1-source-chronological", + "dispatch_time": 1791943332.850823, + "planned_position": 2 + }, + { + "id": "confirm-fresh-13-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943333.994106, + "planned_position": 0 + }, + { + "id": "confirm-fresh-21-en-m1-source-chronological", + "dispatch_time": 1791943334.6603498, + "planned_position": 2 + }, + { + "id": "confirm-fresh-21-en-m1-source-grouped_lossless", + "dispatch_time": 1791943335.327013, + "planned_position": 0 + }, + { + "id": "confirm-fresh-21-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943335.989734, + "planned_position": 1 + }, + { + "id": "confirm-fresh-21-te-m1-source-chronological", + "dispatch_time": 1791943337.1267898, + "planned_position": 0 + }, + { + "id": "confirm-fresh-13-te-m1-source-chronological", + "dispatch_time": 1791943337.794431, + "planned_position": 2 + }, + { + "id": "confirm-fresh-21-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943338.4618711, + "planned_position": 1 + }, + { + "id": "confirm-fresh-13-te-m1-source-grouped_lossless", + "dispatch_time": 1791943339.129771, + "planned_position": 1 + }, + { + "id": "confirm-fresh-20-hi-m0-source-grouped_lossless", + "dispatch_time": 1791943340.271421, + "planned_position": 0 + }, + { + "id": "confirm-fresh-21-te-m1-source-grouped_lossless", + "dispatch_time": 1791943340.937568, + "planned_position": 2 + }, + { + "id": "confirm-fresh-20-hi-m0-source-chronological", + "dispatch_time": 1791943341.604129, + "planned_position": 2 + }, + { + "id": "confirm-fresh-20-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943342.27357, + "planned_position": 1 + }, + { + "id": "confirm-fresh-19-en-m0-source-grouped_lossless", + "dispatch_time": 1791943380.859309, + "planned_position": 1 + }, + { + "id": "confirm-fresh-19-en-m0-source-chronological", + "dispatch_time": 1791943381.527215, + "planned_position": 0 + }, + { + "id": "confirm-fresh-19-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943382.1898298, + "planned_position": 2 + }, + { + "id": "confirm-fresh-38-te-m1-source-chronological", + "dispatch_time": 1791943382.859552, + "planned_position": 0 + }, + { + "id": "confirm-fresh-38-te-m1-source-grouped_lossless", + "dispatch_time": 1791943419.483791, + "planned_position": 1 + }, + { + "id": "confirm-fresh-38-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943420.151362, + "planned_position": 2 + }, + { + "id": "confirm-fresh-35-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943420.818026, + "planned_position": 0 + }, + { + "id": "confirm-fresh-35-en-m1-source-grouped_lossless", + "dispatch_time": 1791943421.484698, + "planned_position": 1 + }, + { + "id": "confirm-fresh-16-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943422.804932, + "planned_position": 1 + }, + { + "id": "confirm-fresh-16-te-m0-source-grouped_lossless", + "dispatch_time": 1791943423.4737332, + "planned_position": 2 + }, + { + "id": "confirm-fresh-35-en-m1-source-chronological", + "dispatch_time": 1791943424.1404061, + "planned_position": 2 + }, + { + "id": "confirm-fresh-16-te-m0-source-chronological", + "dispatch_time": 1791943424.8070679, + "planned_position": 0 + }, + { + "id": "confirm-fresh-15-te-m1-source-chronological", + "dispatch_time": 1791943458.6571891, + "planned_position": 0 + }, + { + "id": "confirm-fresh-15-te-m1-source-grouped_lossless", + "dispatch_time": 1791943459.324399, + "planned_position": 2 + }, + { + "id": "confirm-fresh-05-hi-m1-source-chronological", + "dispatch_time": 1791943459.988178, + "planned_position": 0 + }, + { + "id": "confirm-fresh-15-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943460.6583638, + "planned_position": 1 + }, + { + "id": "confirm-fresh-05-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943461.985468, + "planned_position": 1 + }, + { + "id": "confirm-fresh-05-hi-m1-source-grouped_lossless", + "dispatch_time": 1791943462.653909, + "planned_position": 2 + }, + { + "id": "confirm-fresh-33-hi-m1-source-grouped_lossless", + "dispatch_time": 1791943463.319705, + "planned_position": 1 + }, + { + "id": "confirm-fresh-33-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943463.985966, + "planned_position": 0 + }, + { + "id": "confirm-fresh-05-hi-m0-source-chronological", + "dispatch_time": 1791943465.0744479, + "planned_position": 0 + }, + { + "id": "confirm-fresh-05-hi-m0-source-grouped_lossless", + "dispatch_time": 1791943465.7385972, + "planned_position": 1 + }, + { + "id": "confirm-fresh-33-hi-m1-source-chronological", + "dispatch_time": 1791943466.411408, + "planned_position": 2 + }, + { + "id": "confirm-fresh-05-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943467.075426, + "planned_position": 2 + }, + { + "id": "confirm-fresh-21-te-m0-source-chronological", + "dispatch_time": 1791943492.931736, + "planned_position": 0 + }, + { + "id": "confirm-fresh-40-hi-m1-source-grouped_lossless", + "dispatch_time": 1791943493.600812, + "planned_position": 1 + }, + { + "id": "confirm-fresh-40-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943494.267478, + "planned_position": 2 + }, + { + "id": "confirm-fresh-40-hi-m1-source-chronological", + "dispatch_time": 1791943494.930929, + "planned_position": 0 + }, + { + "id": "confirm-fresh-14-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943496.539499, + "planned_position": 1 + }, + { + "id": "confirm-fresh-21-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943497.208595, + "planned_position": 2 + }, + { + "id": "confirm-fresh-14-hi-m0-source-grouped_lossless", + "dispatch_time": 1791943497.875264, + "planned_position": 0 + }, + { + "id": "confirm-fresh-21-te-m0-source-grouped_lossless", + "dispatch_time": 1791943498.538319, + "planned_position": 1 + }, + { + "id": "confirm-fresh-08-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943535.11777, + "planned_position": 2 + }, + { + "id": "confirm-fresh-08-te-m1-source-chronological", + "dispatch_time": 1791943535.785434, + "planned_position": 0 + }, + { + "id": "confirm-fresh-08-te-m1-source-grouped_lossless", + "dispatch_time": 1791943536.4511712, + "planned_position": 1 + }, + { + "id": "confirm-fresh-14-hi-m0-source-chronological", + "dispatch_time": 1791943537.118782, + "planned_position": 2 + }, + { + "id": "confirm-fresh-09-hi-m1-source-grouped_lossless", + "dispatch_time": 1791943572.742934, + "planned_position": 1 + }, + { + "id": "confirm-fresh-09-hi-m1-source-chronological", + "dispatch_time": 1791943573.410507, + "planned_position": 2 + }, + { + "id": "confirm-fresh-09-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943574.0730948, + "planned_position": 0 + }, + { + "id": "confirm-fresh-39-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943574.743098, + "planned_position": 0 + }, + { + "id": "confirm-fresh-27-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943593.3859148, + "planned_position": 0 + }, + { + "id": "confirm-fresh-27-hi-m0-source-chronological", + "dispatch_time": 1791943594.053783, + "planned_position": 1 + }, + { + "id": "confirm-fresh-39-hi-m0-source-chronological", + "dispatch_time": 1791943594.720466, + "planned_position": 1 + }, + { + "id": "confirm-fresh-39-hi-m0-source-grouped_lossless", + "dispatch_time": 1791943595.382308, + "planned_position": 2 + }, + { + "id": "confirm-fresh-24-te-m1-source-grouped_lossless", + "dispatch_time": 1791943629.298347, + "planned_position": 2 + }, + { + "id": "confirm-fresh-24-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943629.964979, + "planned_position": 0 + }, + { + "id": "confirm-fresh-24-te-m1-source-chronological", + "dispatch_time": 1791943630.631653, + "planned_position": 1 + }, + { + "id": "confirm-fresh-27-hi-m0-source-grouped_lossless", + "dispatch_time": 1791943631.298326, + "planned_position": 2 + }, + { + "id": "confirm-fresh-18-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943644.150333, + "planned_position": 0 + }, + { + "id": "confirm-fresh-18-te-m1-source-chronological", + "dispatch_time": 1791943644.8176582, + "planned_position": 1 + }, + { + "id": "confirm-fresh-18-te-m1-source-grouped_lossless", + "dispatch_time": 1791943645.484183, + "planned_position": 2 + }, + { + "id": "confirm-fresh-17-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943646.152097, + "planned_position": 0 + }, + { + "id": "confirm-fresh-17-en-m0-source-chronological", + "dispatch_time": 1791943683.297808, + "planned_position": 1 + }, + { + "id": "confirm-fresh-19-te-m0-source-chronological", + "dispatch_time": 1791943683.965343, + "planned_position": 1 + }, + { + "id": "confirm-fresh-19-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943684.631993, + "planned_position": 0 + }, + { + "id": "confirm-fresh-17-en-m0-source-grouped_lossless", + "dispatch_time": 1791943685.2986538, + "planned_position": 2 + }, + { + "id": "confirm-fresh-19-te-m0-source-grouped_lossless", + "dispatch_time": 1791943713.358824, + "planned_position": 2 + }, + { + "id": "confirm-fresh-20-te-m1-source-grouped_lossless", + "dispatch_time": 1791943714.024349, + "planned_position": 1 + }, + { + "id": "confirm-fresh-20-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943714.68825, + "planned_position": 2 + }, + { + "id": "confirm-fresh-20-te-m1-source-chronological", + "dispatch_time": 1791943715.357229, + "planned_position": 0 + }, + { + "id": "confirm-fresh-37-te-m0-source-grouped_lossless", + "dispatch_time": 1791943745.403264, + "planned_position": 2 + }, + { + "id": "confirm-fresh-12-te-m0-source-grouped_lossless", + "dispatch_time": 1791943746.065399, + "planned_position": 0 + }, + { + "id": "confirm-fresh-37-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943746.736115, + "planned_position": 0 + }, + { + "id": "confirm-fresh-37-te-m0-source-chronological", + "dispatch_time": 1791943747.402803, + "planned_position": 1 + }, + { + "id": "confirm-fresh-38-en-m0-source-chronological", + "dispatch_time": 1791943768.337085, + "planned_position": 0 + }, + { + "id": "confirm-fresh-38-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943769.000906, + "planned_position": 1 + }, + { + "id": "confirm-fresh-12-te-m0-source-chronological", + "dispatch_time": 1791943769.6666439, + "planned_position": 2 + }, + { + "id": "confirm-fresh-12-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943770.333409, + "planned_position": 1 + }, + { + "id": "confirm-fresh-08-en-m0-source-chronological", + "dispatch_time": 1791943790.654219, + "planned_position": 0 + }, + { + "id": "confirm-fresh-08-en-m0-source-grouped_lossless", + "dispatch_time": 1791943791.319345, + "planned_position": 2 + }, + { + "id": "confirm-fresh-08-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943791.988089, + "planned_position": 1 + }, + { + "id": "confirm-fresh-38-en-m0-source-grouped_lossless", + "dispatch_time": 1791943792.6547258, + "planned_position": 2 + }, + { + "id": "confirm-fresh-15-en-m1-source-chronological", + "dispatch_time": 1791943826.826432, + "planned_position": 2 + }, + { + "id": "confirm-fresh-15-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943827.493033, + "planned_position": 1 + }, + { + "id": "confirm-fresh-15-en-m1-source-grouped_lossless", + "dispatch_time": 1791943828.1597939, + "planned_position": 0 + }, + { + "id": "confirm-fresh-14-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943828.8279612, + "planned_position": 0 + }, + { + "id": "confirm-fresh-14-hi-m1-source-chronological", + "dispatch_time": 1791943831.028963, + "planned_position": 1 + }, + { + "id": "confirm-fresh-14-hi-m1-source-grouped_lossless", + "dispatch_time": 1791943831.693063, + "planned_position": 2 + }, + { + "id": "confirm-fresh-27-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943832.361718, + "planned_position": 1 + }, + { + "id": "confirm-fresh-27-en-m1-source-grouped_lossless", + "dispatch_time": 1791943833.027425, + "planned_position": 0 + }, + { + "id": "confirm-fresh-02-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943834.170325, + "planned_position": 1 + }, + { + "id": "confirm-fresh-02-hi-m0-source-grouped_lossless", + "dispatch_time": 1791943834.835975, + "planned_position": 0 + }, + { + "id": "confirm-fresh-27-en-m1-source-chronological", + "dispatch_time": 1791943835.500643, + "planned_position": 2 + }, + { + "id": "confirm-fresh-02-hi-m0-source-chronological", + "dispatch_time": 1791943836.169667, + "planned_position": 2 + }, + { + "id": "confirm-fresh-20-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943847.642865, + "planned_position": 2 + }, + { + "id": "confirm-fresh-20-en-m1-source-chronological", + "dispatch_time": 1791943848.30569, + "planned_position": 1 + }, + { + "id": "confirm-fresh-07-hi-m1-source-grouped_lossless", + "dispatch_time": 1791943848.971394, + "planned_position": 0 + }, + { + "id": "confirm-fresh-20-en-m1-source-grouped_lossless", + "dispatch_time": 1791943849.634946, + "planned_position": 0 + }, + { + "id": "confirm-fresh-07-hi-m1-source-chronological", + "dispatch_time": 1791943850.64837, + "planned_position": 2 + }, + { + "id": "confirm-fresh-40-en-m0-source-grouped_lossless", + "dispatch_time": 1791943851.3142788, + "planned_position": 0 + }, + { + "id": "confirm-fresh-07-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943851.983187, + "planned_position": 1 + }, + { + "id": "confirm-fresh-40-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943852.648907, + "planned_position": 1 + }, + { + "id": "confirm-fresh-40-en-m0-source-chronological", + "dispatch_time": 1791943886.073059, + "planned_position": 2 + }, + { + "id": "confirm-fresh-01-hi-m1-source-chronological", + "dispatch_time": 1791943886.7370949, + "planned_position": 2 + }, + { + "id": "confirm-fresh-01-hi-m1-source-grouped_lossless", + "dispatch_time": 1791943887.404573, + "planned_position": 0 + }, + { + "id": "confirm-fresh-01-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943888.071179, + "planned_position": 1 + }, + { + "id": "confirm-fresh-05-te-m0-source-grouped_lossless", + "dispatch_time": 1791943893.232853, + "planned_position": 0 + }, + { + "id": "confirm-fresh-05-te-m0-source-chronological", + "dispatch_time": 1791943893.901829, + "planned_position": 1 + }, + { + "id": "confirm-fresh-05-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943894.565783, + "planned_position": 2 + }, + { + "id": "confirm-fresh-38-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943895.235156, + "planned_position": 0 + }, + { + "id": "confirm-fresh-01-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943916.708003, + "planned_position": 1 + }, + { + "id": "confirm-fresh-01-en-m1-source-chronological", + "dispatch_time": 1791943917.373348, + "planned_position": 0 + }, + { + "id": "confirm-fresh-38-hi-m1-source-chronological", + "dispatch_time": 1791943918.0427232, + "planned_position": 1 + }, + { + "id": "confirm-fresh-38-hi-m1-source-grouped_lossless", + "dispatch_time": 1791943918.70563, + "planned_position": 2 + }, + { + "id": "confirm-fresh-12-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943919.701684, + "planned_position": 1 + }, + { + "id": "confirm-fresh-12-hi-m0-source-grouped_lossless", + "dispatch_time": 1791943920.3685381, + "planned_position": 2 + }, + { + "id": "confirm-fresh-01-en-m1-source-grouped_lossless", + "dispatch_time": 1791943921.03669, + "planned_position": 2 + }, + { + "id": "confirm-fresh-12-hi-m0-source-chronological", + "dispatch_time": 1791943921.703358, + "planned_position": 0 + }, + { + "id": "confirm-fresh-13-en-m0-source-chronological", + "dispatch_time": 1791943935.353089, + "planned_position": 0 + }, + { + "id": "confirm-fresh-13-en-m0-source-grouped_lossless", + "dispatch_time": 1791943936.020513, + "planned_position": 1 + }, + { + "id": "confirm-fresh-13-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943936.687194, + "planned_position": 2 + }, + { + "id": "confirm-fresh-30-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943937.3533309, + "planned_position": 0 + }, + { + "id": "confirm-fresh-30-te-m1-source-chronological", + "dispatch_time": 1791943947.791044, + "planned_position": 1 + }, + { + "id": "confirm-fresh-10-te-m1-source-grouped_lossless", + "dispatch_time": 1791943948.458233, + "planned_position": 0 + }, + { + "id": "confirm-fresh-30-te-m1-source-grouped_lossless", + "dispatch_time": 1791943949.1241379, + "planned_position": 2 + }, + { + "id": "confirm-fresh-10-te-m1-source-chronological", + "dispatch_time": 1791943949.790005, + "planned_position": 1 + }, + { + "id": "confirm-fresh-05-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943950.868735, + "planned_position": 1 + }, + { + "id": "confirm-fresh-05-te-m1-source-chronological", + "dispatch_time": 1791943951.536292, + "planned_position": 2 + }, + { + "id": "confirm-fresh-10-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943952.200735, + "planned_position": 2 + }, + { + "id": "confirm-fresh-05-te-m1-source-grouped_lossless", + "dispatch_time": 1791943952.8659132, + "planned_position": 0 + }, + { + "id": "confirm-fresh-16-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943953.86657, + "planned_position": 0 + }, + { + "id": "confirm-fresh-35-hi-m0-source-chronological", + "dispatch_time": 1791943954.531768, + "planned_position": 0 + }, + { + "id": "confirm-fresh-16-hi-m0-source-grouped_lossless", + "dispatch_time": 1791943955.1990201, + "planned_position": 1 + }, + { + "id": "confirm-fresh-16-hi-m0-source-chronological", + "dispatch_time": 1791943955.8628118, + "planned_position": 2 + }, + { + "id": "confirm-fresh-35-hi-m0-source-grouped_lossless", + "dispatch_time": 1791943970.479959, + "planned_position": 1 + }, + { + "id": "confirm-fresh-38-en-m1-source-grouped_lossless", + "dispatch_time": 1791943971.144486, + "planned_position": 0 + }, + { + "id": "confirm-fresh-38-en-m1-source-chronological", + "dispatch_time": 1791943971.8145978, + "planned_position": 1 + }, + { + "id": "confirm-fresh-35-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943972.4812548, + "planned_position": 2 + }, + { + "id": "confirm-fresh-26-te-m1-source-chronological", + "dispatch_time": 1791943993.298244, + "planned_position": 0 + }, + { + "id": "confirm-fresh-26-te-m1-source-grouped_lossless", + "dispatch_time": 1791943993.964905, + "planned_position": 1 + }, + { + "id": "confirm-fresh-38-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943994.630343, + "planned_position": 2 + }, + { + "id": "confirm-fresh-26-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943995.297597, + "planned_position": 2 + }, + { + "id": "confirm-fresh-25-en-m0-source-chronological", + "dispatch_time": 1791943996.4289489, + "planned_position": 0 + }, + { + "id": "confirm-fresh-07-te-m1-source-grouped_lossless", + "dispatch_time": 1791943997.093007, + "planned_position": 1 + }, + { + "id": "confirm-fresh-07-te-m1-source-chronological", + "dispatch_time": 1791943997.7635531, + "planned_position": 2 + }, + { + "id": "confirm-fresh-07-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791943998.426809, + "planned_position": 0 + }, + { + "id": "confirm-fresh-25-en-m0-source-grouped_lossless", + "dispatch_time": 1791944001.6014779, + "planned_position": 1 + }, + { + "id": "confirm-fresh-29-te-m1-source-grouped_lossless", + "dispatch_time": 1791944002.268806, + "planned_position": 0 + }, + { + "id": "confirm-fresh-25-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944002.933176, + "planned_position": 2 + }, + { + "id": "confirm-fresh-29-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944003.6014788, + "planned_position": 1 + }, + { + "id": "confirm-fresh-29-te-m1-source-chronological", + "dispatch_time": 1791944021.346307, + "planned_position": 2 + }, + { + "id": "confirm-fresh-27-te-m0-source-chronological", + "dispatch_time": 1791944022.0089622, + "planned_position": 0 + }, + { + "id": "confirm-fresh-27-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944022.6797209, + "planned_position": 2 + }, + { + "id": "confirm-fresh-27-te-m0-source-grouped_lossless", + "dispatch_time": 1791944023.346103, + "planned_position": 1 + }, + { + "id": "confirm-fresh-31-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944033.746864, + "planned_position": 0 + }, + { + "id": "confirm-fresh-19-en-m1-source-grouped_lossless", + "dispatch_time": 1791944034.413193, + "planned_position": 2 + }, + { + "id": "confirm-fresh-19-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944035.081175, + "planned_position": 1 + }, + { + "id": "confirm-fresh-19-en-m1-source-chronological", + "dispatch_time": 1791944035.745027, + "planned_position": 0 + }, + { + "id": "confirm-fresh-31-te-m1-source-grouped_lossless", + "dispatch_time": 1791944036.724011, + "planned_position": 1 + }, + { + "id": "confirm-fresh-32-en-m1-source-chronological", + "dispatch_time": 1791944037.3898199, + "planned_position": 1 + }, + { + "id": "confirm-fresh-31-te-m1-source-chronological", + "dispatch_time": 1791944038.057128, + "planned_position": 2 + }, + { + "id": "confirm-fresh-32-en-m1-source-grouped_lossless", + "dispatch_time": 1791944038.721215, + "planned_position": 0 + }, + { + "id": "confirm-fresh-32-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944039.713272, + "planned_position": 2 + }, + { + "id": "confirm-fresh-06-en-m1-source-chronological", + "dispatch_time": 1791944040.382236, + "planned_position": 0 + }, + { + "id": "confirm-fresh-06-en-m1-source-grouped_lossless", + "dispatch_time": 1791944041.046377, + "planned_position": 1 + }, + { + "id": "confirm-fresh-06-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944041.7131, + "planned_position": 2 + }, + { + "id": "confirm-fresh-07-en-m1-source-grouped_lossless", + "dispatch_time": 1791944042.740617, + "planned_position": 2 + }, + { + "id": "confirm-fresh-07-en-m1-source-chronological", + "dispatch_time": 1791944043.402996, + "planned_position": 0 + }, + { + "id": "confirm-fresh-07-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944044.072304, + "planned_position": 1 + }, + { + "id": "confirm-fresh-13-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944044.737222, + "planned_position": 0 + }, + { + "id": "confirm-fresh-13-te-m0-source-chronological", + "dispatch_time": 1791944058.043663, + "planned_position": 1 + }, + { + "id": "confirm-fresh-13-te-m0-source-grouped_lossless", + "dispatch_time": 1791944058.7109199, + "planned_position": 2 + }, + { + "id": "confirm-fresh-11-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944059.3742132, + "planned_position": 1 + }, + { + "id": "confirm-fresh-11-te-m1-source-grouped_lossless", + "dispatch_time": 1791944060.043196, + "planned_position": 0 + }, + { + "id": "confirm-fresh-09-te-m1-source-chronological", + "dispatch_time": 1791944094.2581189, + "planned_position": 0 + }, + { + "id": "confirm-fresh-11-te-m1-source-chronological", + "dispatch_time": 1791944094.923767, + "planned_position": 2 + }, + { + "id": "confirm-fresh-09-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944095.590409, + "planned_position": 1 + }, + { + "id": "confirm-fresh-09-te-m1-source-grouped_lossless", + "dispatch_time": 1791944096.25571, + "planned_position": 2 + }, + { + "id": "confirm-fresh-27-te-m1-source-grouped_lossless", + "dispatch_time": 1791944097.3614159, + "planned_position": 2 + }, + { + "id": "confirm-fresh-27-te-m1-source-chronological", + "dispatch_time": 1791944098.027519, + "planned_position": 0 + }, + { + "id": "confirm-fresh-12-hi-m1-source-grouped_lossless", + "dispatch_time": 1791944098.6936278, + "planned_position": 0 + }, + { + "id": "confirm-fresh-27-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944099.358895, + "planned_position": 1 + }, + { + "id": "confirm-fresh-12-hi-m1-source-chronological", + "dispatch_time": 1791944100.5137331, + "planned_position": 1 + }, + { + "id": "confirm-fresh-12-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944101.175814, + "planned_position": 0 + }, + { + "id": "confirm-fresh-12-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944101.8426828, + "planned_position": 2 + }, + { + "id": "confirm-fresh-12-te-m1-source-chronological", + "dispatch_time": 1791944102.5100648, + "planned_position": 1 + }, + { + "id": "confirm-fresh-18-te-m0-source-grouped_lossless", + "dispatch_time": 1791944103.5483131, + "planned_position": 0 + }, + { + "id": "confirm-fresh-18-te-m0-source-chronological", + "dispatch_time": 1791944104.2145631, + "planned_position": 2 + }, + { + "id": "confirm-fresh-12-te-m1-source-grouped_lossless", + "dispatch_time": 1791944104.878379, + "planned_position": 2 + }, + { + "id": "confirm-fresh-18-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944105.5447419, + "planned_position": 1 + }, + { + "id": "confirm-fresh-26-en-m1-source-chronological", + "dispatch_time": 1791944138.9255638, + "planned_position": 1 + }, + { + "id": "confirm-fresh-26-en-m1-source-grouped_lossless", + "dispatch_time": 1791944139.591316, + "planned_position": 0 + }, + { + "id": "confirm-fresh-26-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944140.2544389, + "planned_position": 2 + }, + { + "id": "confirm-fresh-23-te-m1-source-grouped_lossless", + "dispatch_time": 1791944140.9230888, + "planned_position": 0 + }, + { + "id": "confirm-fresh-17-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944141.947244, + "planned_position": 1 + }, + { + "id": "confirm-fresh-17-hi-m1-source-chronological", + "dispatch_time": 1791944142.610786, + "planned_position": 0 + }, + { + "id": "confirm-fresh-23-te-m1-source-chronological", + "dispatch_time": 1791944143.279019, + "planned_position": 2 + }, + { + "id": "confirm-fresh-23-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944143.944588, + "planned_position": 1 + }, + { + "id": "confirm-fresh-17-hi-m1-source-grouped_lossless", + "dispatch_time": 1791944145.064277, + "planned_position": 2 + }, + { + "id": "confirm-fresh-22-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944145.7320058, + "planned_position": 1 + }, + { + "id": "confirm-fresh-22-te-m0-source-chronological", + "dispatch_time": 1791944146.398036, + "planned_position": 0 + }, + { + "id": "confirm-fresh-22-te-m0-source-grouped_lossless", + "dispatch_time": 1791944147.0660841, + "planned_position": 2 + }, + { + "id": "confirm-fresh-01-te-m1-source-chronological", + "dispatch_time": 1791944182.617801, + "planned_position": 2 + }, + { + "id": "confirm-fresh-01-te-m1-source-grouped_lossless", + "dispatch_time": 1791944183.284142, + "planned_position": 1 + }, + { + "id": "confirm-fresh-37-en-m0-source-chronological", + "dispatch_time": 1791944183.950063, + "planned_position": 0 + }, + { + "id": "confirm-fresh-01-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944184.61727, + "planned_position": 0 + }, + { + "id": "confirm-fresh-13-en-m1-source-chronological", + "dispatch_time": 1791944211.850685, + "planned_position": 0 + }, + { + "id": "confirm-fresh-37-en-m0-source-grouped_lossless", + "dispatch_time": 1791944212.509801, + "planned_position": 1 + }, + { + "id": "confirm-fresh-37-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944213.17712, + "planned_position": 2 + }, + { + "id": "confirm-fresh-13-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944213.842409, + "planned_position": 1 + }, + { + "id": "confirm-fresh-19-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944242.061238, + "planned_position": 1 + }, + { + "id": "confirm-fresh-19-hi-m1-source-chronological", + "dispatch_time": 1791944242.728684, + "planned_position": 2 + }, + { + "id": "confirm-fresh-13-en-m1-source-grouped_lossless", + "dispatch_time": 1791944243.395245, + "planned_position": 2 + }, + { + "id": "confirm-fresh-19-hi-m1-source-grouped_lossless", + "dispatch_time": 1791944244.061725, + "planned_position": 0 + }, + { + "id": "confirm-fresh-21-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944245.1168761, + "planned_position": 0 + }, + { + "id": "confirm-fresh-37-te-m1-source-chronological", + "dispatch_time": 1791944245.782954, + "planned_position": 2 + }, + { + "id": "confirm-fresh-37-te-m1-source-grouped_lossless", + "dispatch_time": 1791944246.4515572, + "planned_position": 1 + }, + { + "id": "confirm-fresh-37-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944247.118231, + "planned_position": 0 + }, + { + "id": "confirm-fresh-02-en-m0-source-chronological", + "dispatch_time": 1791944268.793938, + "planned_position": 0 + }, + { + "id": "confirm-fresh-02-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944269.462959, + "planned_position": 1 + }, + { + "id": "confirm-fresh-21-hi-m0-source-chronological", + "dispatch_time": 1791944270.129627, + "planned_position": 1 + }, + { + "id": "confirm-fresh-21-hi-m0-source-grouped_lossless", + "dispatch_time": 1791944270.796295, + "planned_position": 2 + }, + { + "id": "confirm-fresh-06-te-m0-source-grouped_lossless", + "dispatch_time": 1791944285.89606, + "planned_position": 0 + }, + { + "id": "confirm-fresh-06-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944286.559049, + "planned_position": 1 + }, + { + "id": "confirm-fresh-02-en-m0-source-grouped_lossless", + "dispatch_time": 1791944287.227712, + "planned_position": 2 + }, + { + "id": "confirm-fresh-06-te-m0-source-chronological", + "dispatch_time": 1791944287.893992, + "planned_position": 2 + }, + { + "id": "confirm-fresh-39-te-m0-source-grouped_lossless", + "dispatch_time": 1791944319.4290938, + "planned_position": 1 + }, + { + "id": "confirm-fresh-12-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944320.0949461, + "planned_position": 0 + }, + { + "id": "confirm-fresh-39-te-m0-source-chronological", + "dispatch_time": 1791944320.7627368, + "planned_position": 0 + }, + { + "id": "confirm-fresh-39-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944321.42941, + "planned_position": 2 + }, + { + "id": "confirm-fresh-11-hi-m1-source-chronological", + "dispatch_time": 1791944341.316639, + "planned_position": 0 + }, + { + "id": "confirm-fresh-11-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944341.980824, + "planned_position": 1 + }, + { + "id": "confirm-fresh-12-en-m0-source-chronological", + "dispatch_time": 1791944342.6490922, + "planned_position": 2 + }, + { + "id": "confirm-fresh-12-en-m0-source-grouped_lossless", + "dispatch_time": 1791944343.317463, + "planned_position": 1 + }, + { + "id": "confirm-fresh-27-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944353.753429, + "planned_position": 0 + }, + { + "id": "confirm-fresh-27-hi-m1-source-chronological", + "dispatch_time": 1791944354.4207208, + "planned_position": 2 + }, + { + "id": "confirm-fresh-27-hi-m1-source-grouped_lossless", + "dispatch_time": 1791944355.084539, + "planned_position": 1 + }, + { + "id": "confirm-fresh-11-hi-m1-source-grouped_lossless", + "dispatch_time": 1791944355.754209, + "planned_position": 2 + }, + { + "id": "confirm-fresh-22-en-m1-source-chronological", + "dispatch_time": 1791944356.891872, + "planned_position": 1 + }, + { + "id": "confirm-fresh-22-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944357.5573602, + "planned_position": 0 + }, + { + "id": "confirm-fresh-29-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944358.22443, + "planned_position": 0 + }, + { + "id": "confirm-fresh-22-en-m1-source-grouped_lossless", + "dispatch_time": 1791944358.891084, + "planned_position": 2 + }, + { + "id": "confirm-fresh-29-en-m0-source-grouped_lossless", + "dispatch_time": 1791944372.213567, + "planned_position": 2 + }, + { + "id": "confirm-fresh-30-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944372.87791, + "planned_position": 0 + }, + { + "id": "confirm-fresh-29-en-m0-source-chronological", + "dispatch_time": 1791944373.545681, + "planned_position": 1 + }, + { + "id": "confirm-fresh-30-en-m0-source-grouped_lossless", + "dispatch_time": 1791944374.2129862, + "planned_position": 1 + }, + { + "id": "confirm-fresh-30-en-m0-source-chronological", + "dispatch_time": 1791944394.638988, + "planned_position": 2 + }, + { + "id": "confirm-fresh-34-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944395.304967, + "planned_position": 1 + }, + { + "id": "confirm-fresh-34-te-m0-source-chronological", + "dispatch_time": 1791944395.9692779, + "planned_position": 0 + }, + { + "id": "confirm-fresh-34-te-m0-source-grouped_lossless", + "dispatch_time": 1791944396.63677, + "planned_position": 2 + }, + { + "id": "confirm-fresh-33-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944419.4381618, + "planned_position": 1 + }, + { + "id": "confirm-fresh-33-en-m1-source-grouped_lossless", + "dispatch_time": 1791944420.106945, + "planned_position": 0 + }, + { + "id": "confirm-fresh-33-en-m1-source-chronological", + "dispatch_time": 1791944420.7722938, + "planned_position": 2 + }, + { + "id": "confirm-fresh-10-hi-m1-source-chronological", + "dispatch_time": 1791944421.43606, + "planned_position": 0 + }, + { + "id": "confirm-fresh-10-hi-m1-source-grouped_lossless", + "dispatch_time": 1791944422.4288309, + "planned_position": 1 + }, + { + "id": "confirm-fresh-18-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944423.093997, + "planned_position": 1 + }, + { + "id": "confirm-fresh-18-hi-m0-source-chronological", + "dispatch_time": 1791944423.764152, + "planned_position": 0 + }, + { + "id": "confirm-fresh-10-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944424.4302158, + "planned_position": 2 + }, + { + "id": "confirm-fresh-18-hi-m0-source-grouped_lossless", + "dispatch_time": 1791944447.8631701, + "planned_position": 2 + }, + { + "id": "confirm-fresh-17-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944448.526911, + "planned_position": 0 + }, + { + "id": "confirm-fresh-17-en-m1-source-grouped_lossless", + "dispatch_time": 1791944449.19747, + "planned_position": 1 + }, + { + "id": "confirm-fresh-17-en-m1-source-chronological", + "dispatch_time": 1791944449.8605819, + "planned_position": 2 + }, + { + "id": "confirm-fresh-25-te-m1-source-chronological", + "dispatch_time": 1791944469.0850282, + "planned_position": 2 + }, + { + "id": "confirm-fresh-25-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944469.7514398, + "planned_position": 0 + }, + { + "id": "confirm-fresh-25-te-m1-source-grouped_lossless", + "dispatch_time": 1791944470.4186509, + "planned_position": 1 + }, + { + "id": "confirm-fresh-04-hi-m1-source-chronological", + "dispatch_time": 1791944471.084864, + "planned_position": 0 + }, + { + "id": "confirm-fresh-01-te-m0-source-chronological", + "dispatch_time": 1791944472.227498, + "planned_position": 1 + }, + { + "id": "confirm-fresh-04-hi-m1-source-grouped_lossless", + "dispatch_time": 1791944472.895468, + "planned_position": 1 + }, + { + "id": "confirm-fresh-01-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944473.562476, + "planned_position": 0 + }, + { + "id": "confirm-fresh-04-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944474.226215, + "planned_position": 2 + }, + { + "id": "confirm-fresh-05-en-m0-source-grouped_lossless", + "dispatch_time": 1791944481.4361331, + "planned_position": 2 + }, + { + "id": "confirm-fresh-05-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944482.103298, + "planned_position": 0 + }, + { + "id": "confirm-fresh-05-en-m0-source-chronological", + "dispatch_time": 1791944482.768945, + "planned_position": 1 + }, + { + "id": "confirm-fresh-01-te-m0-source-grouped_lossless", + "dispatch_time": 1791944483.4324632, + "planned_position": 2 + }, + { + "id": "confirm-fresh-16-en-m1-source-grouped_lossless", + "dispatch_time": 1791944495.309709, + "planned_position": 2 + }, + { + "id": "confirm-fresh-16-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944495.975161, + "planned_position": 0 + }, + { + "id": "confirm-fresh-28-hi-m1-source-chronological", + "dispatch_time": 1791944496.642853, + "planned_position": 0 + }, + { + "id": "confirm-fresh-16-en-m1-source-chronological", + "dispatch_time": 1791944497.3069482, + "planned_position": 1 + }, + { + "id": "confirm-fresh-28-hi-m1-source-grouped_lossless", + "dispatch_time": 1791944498.34966, + "planned_position": 1 + }, + { + "id": "confirm-fresh-34-en-m0-source-grouped_lossless", + "dispatch_time": 1791944499.013639, + "planned_position": 0 + }, + { + "id": "confirm-fresh-28-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944499.684756, + "planned_position": 2 + }, + { + "id": "confirm-fresh-34-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944500.350304, + "planned_position": 1 + }, + { + "id": "confirm-fresh-34-en-m0-source-chronological", + "dispatch_time": 1791944517.870946, + "planned_position": 2 + }, + { + "id": "confirm-fresh-40-te-m0-source-chronological", + "dispatch_time": 1791944518.5391738, + "planned_position": 0 + }, + { + "id": "confirm-fresh-40-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944519.205847, + "planned_position": 1 + }, + { + "id": "confirm-fresh-40-te-m0-source-grouped_lossless", + "dispatch_time": 1791944519.872498, + "planned_position": 2 + }, + { + "id": "confirm-fresh-03-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944539.9006982, + "planned_position": 0 + }, + { + "id": "confirm-fresh-03-hi-m0-source-grouped_lossless", + "dispatch_time": 1791944540.568465, + "planned_position": 2 + }, + { + "id": "confirm-fresh-11-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944541.231704, + "planned_position": 0 + }, + { + "id": "confirm-fresh-03-hi-m0-source-chronological", + "dispatch_time": 1791944541.901828, + "planned_position": 1 + }, + { + "id": "confirm-fresh-11-en-m1-source-chronological", + "dispatch_time": 1791944551.4580228, + "planned_position": 2 + }, + { + "id": "confirm-fresh-26-en-m0-source-chronological", + "dispatch_time": 1791944552.1258838, + "planned_position": 0 + }, + { + "id": "confirm-fresh-26-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944552.7931788, + "planned_position": 1 + }, + { + "id": "confirm-fresh-11-en-m1-source-grouped_lossless", + "dispatch_time": 1791944553.458821, + "planned_position": 1 + }, + { + "id": "confirm-fresh-15-hi-m0-source-chronological", + "dispatch_time": 1791944563.864767, + "planned_position": 1 + }, + { + "id": "confirm-fresh-26-en-m0-source-grouped_lossless", + "dispatch_time": 1791944564.532604, + "planned_position": 2 + }, + { + "id": "confirm-fresh-15-hi-m0-source-grouped_lossless", + "dispatch_time": 1791944565.199101, + "planned_position": 2 + }, + { + "id": "confirm-fresh-15-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944565.863648, + "planned_position": 0 + }, + { + "id": "confirm-fresh-38-hi-m0-source-grouped_lossless", + "dispatch_time": 1791944587.5624819, + "planned_position": 0 + }, + { + "id": "confirm-fresh-09-en-m1-source-chronological", + "dispatch_time": 1791944588.230817, + "planned_position": 2 + }, + { + "id": "confirm-fresh-09-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944588.8960168, + "planned_position": 1 + }, + { + "id": "confirm-fresh-09-en-m1-source-grouped_lossless", + "dispatch_time": 1791944589.562026, + "planned_position": 0 + }, + { + "id": "confirm-fresh-24-te-m0-source-grouped_lossless", + "dispatch_time": 1791944614.3218749, + "planned_position": 0 + }, + { + "id": "confirm-fresh-38-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944614.990104, + "planned_position": 1 + }, + { + "id": "confirm-fresh-38-hi-m0-source-chronological", + "dispatch_time": 1791944615.656765, + "planned_position": 2 + }, + { + "id": "confirm-fresh-24-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944616.3234298, + "planned_position": 1 + }, + { + "id": "confirm-fresh-07-en-m0-source-grouped_lossless", + "dispatch_time": 1791944642.437258, + "planned_position": 1 + }, + { + "id": "confirm-fresh-07-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944643.105612, + "planned_position": 2 + }, + { + "id": "confirm-fresh-24-te-m0-source-chronological", + "dispatch_time": 1791944643.772303, + "planned_position": 2 + }, + { + "id": "confirm-fresh-07-en-m0-source-chronological", + "dispatch_time": 1791944644.4389482, + "planned_position": 0 + }, + { + "id": "confirm-fresh-11-te-m0-source-grouped_lossless", + "dispatch_time": 1791944655.476238, + "planned_position": 0 + }, + { + "id": "confirm-fresh-11-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944656.142412, + "planned_position": 2 + }, + { + "id": "confirm-fresh-11-te-m0-source-chronological", + "dispatch_time": 1791944656.8072162, + "planned_position": 1 + }, + { + "id": "confirm-fresh-33-en-m0-source-grouped_lossless", + "dispatch_time": 1791944657.4761689, + "planned_position": 0 + }, + { + "id": "confirm-fresh-33-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944672.6347048, + "planned_position": 2 + }, + { + "id": "confirm-fresh-33-en-m0-source-chronological", + "dispatch_time": 1791944673.300045, + "planned_position": 1 + }, + { + "id": "confirm-fresh-10-en-m1-source-chronological", + "dispatch_time": 1791944673.963528, + "planned_position": 1 + }, + { + "id": "confirm-fresh-10-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944674.632759, + "planned_position": 0 + }, + { + "id": "confirm-fresh-10-en-m1-source-grouped_lossless", + "dispatch_time": 1791944702.741951, + "planned_position": 2 + }, + { + "id": "confirm-fresh-26-hi-m1-source-chronological", + "dispatch_time": 1791944703.4091258, + "planned_position": 1 + }, + { + "id": "confirm-fresh-26-hi-m1-source-grouped_lossless", + "dispatch_time": 1791944704.076213, + "planned_position": 2 + }, + { + "id": "confirm-fresh-26-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944704.7382789, + "planned_position": 0 + }, + { + "id": "confirm-fresh-18-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944705.747844, + "planned_position": 0 + }, + { + "id": "confirm-fresh-18-en-m0-source-chronological", + "dispatch_time": 1791944706.412035, + "planned_position": 2 + }, + { + "id": "confirm-fresh-18-en-m0-source-grouped_lossless", + "dispatch_time": 1791944707.082243, + "planned_position": 1 + }, + { + "id": "confirm-fresh-23-te-m0-source-grouped_lossless", + "dispatch_time": 1791944707.748686, + "planned_position": 0 + }, + { + "id": "confirm-fresh-23-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944746.508492, + "planned_position": 2 + }, + { + "id": "confirm-fresh-30-en-m1-source-grouped_lossless", + "dispatch_time": 1791944747.171835, + "planned_position": 1 + }, + { + "id": "confirm-fresh-23-te-m0-source-chronological", + "dispatch_time": 1791944747.841343, + "planned_position": 1 + }, + { + "id": "confirm-fresh-30-en-m1-source-chronological", + "dispatch_time": 1791944748.5086489, + "planned_position": 0 + }, + { + "id": "confirm-fresh-17-te-m1-source-grouped_lossless", + "dispatch_time": 1791944781.461963, + "planned_position": 0 + }, + { + "id": "confirm-fresh-30-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944782.130555, + "planned_position": 2 + }, + { + "id": "confirm-fresh-17-te-m1-source-chronological", + "dispatch_time": 1791944782.796458, + "planned_position": 2 + }, + { + "id": "confirm-fresh-17-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944783.4638028, + "planned_position": 1 + }, + { + "id": "confirm-fresh-36-hi-m1-source-chronological", + "dispatch_time": 1791944784.467892, + "planned_position": 1 + }, + { + "id": "confirm-fresh-36-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944785.13235, + "planned_position": 2 + }, + { + "id": "confirm-fresh-28-te-m1-source-grouped_lossless", + "dispatch_time": 1791944785.797013, + "planned_position": 0 + }, + { + "id": "confirm-fresh-36-hi-m1-source-grouped_lossless", + "dispatch_time": 1791944786.464291, + "planned_position": 0 + }, + { + "id": "confirm-fresh-29-hi-m1-source-chronological", + "dispatch_time": 1791944787.513746, + "planned_position": 0 + }, + { + "id": "confirm-fresh-28-te-m1-source-chronological", + "dispatch_time": 1791944788.177251, + "planned_position": 1 + }, + { + "id": "confirm-fresh-29-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944788.848604, + "planned_position": 1 + }, + { + "id": "confirm-fresh-28-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944789.512929, + "planned_position": 2 + }, + { + "id": "confirm-fresh-29-hi-m1-source-grouped_lossless", + "dispatch_time": 1791944790.520779, + "planned_position": 2 + }, + { + "id": "confirm-fresh-35-te-m1-source-grouped_lossless", + "dispatch_time": 1791944791.185606, + "planned_position": 0 + }, + { + "id": "confirm-fresh-35-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944791.852422, + "planned_position": 1 + }, + { + "id": "confirm-fresh-35-te-m1-source-chronological", + "dispatch_time": 1791944792.5187142, + "planned_position": 2 + }, + { + "id": "confirm-fresh-12-en-m1-source-chronological", + "dispatch_time": 1791944793.602093, + "planned_position": 0 + }, + { + "id": "confirm-fresh-04-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944794.270617, + "planned_position": 0 + }, + { + "id": "confirm-fresh-04-en-m1-source-chronological", + "dispatch_time": 1791944794.937305, + "planned_position": 1 + }, + { + "id": "confirm-fresh-04-en-m1-source-grouped_lossless", + "dispatch_time": 1791944795.6035252, + "planned_position": 2 + }, + { + "id": "confirm-fresh-12-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944796.657517, + "planned_position": 2 + }, + { + "id": "confirm-fresh-40-te-m1-source-chronological", + "dispatch_time": 1791944797.325459, + "planned_position": 1 + }, + { + "id": "confirm-fresh-40-te-m1-source-grouped_lossless", + "dispatch_time": 1791944797.9921632, + "planned_position": 0 + }, + { + "id": "confirm-fresh-12-en-m1-source-grouped_lossless", + "dispatch_time": 1791944798.658613, + "planned_position": 1 + }, + { + "id": "confirm-fresh-06-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944799.6433222, + "planned_position": 2 + }, + { + "id": "confirm-fresh-06-hi-m1-source-grouped_lossless", + "dispatch_time": 1791944800.3116338, + "planned_position": 0 + }, + { + "id": "confirm-fresh-40-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944800.976195, + "planned_position": 2 + }, + { + "id": "confirm-fresh-06-hi-m1-source-chronological", + "dispatch_time": 1791944801.6449869, + "planned_position": 1 + }, + { + "id": "confirm-fresh-34-hi-m0-source-chronological", + "dispatch_time": 1791944802.6531231, + "planned_position": 2 + }, + { + "id": "confirm-fresh-34-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944803.3204632, + "planned_position": 0 + }, + { + "id": "confirm-fresh-03-en-m0-source-grouped_lossless", + "dispatch_time": 1791944803.987134, + "planned_position": 0 + }, + { + "id": "confirm-fresh-34-hi-m0-source-grouped_lossless", + "dispatch_time": 1791944804.653816, + "planned_position": 1 + }, + { + "id": "confirm-fresh-03-en-m0-source-chronological", + "dispatch_time": 1791944828.555746, + "planned_position": 1 + }, + { + "id": "confirm-fresh-03-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944829.2222831, + "planned_position": 2 + }, + { + "id": "confirm-fresh-07-hi-m0-source-grouped_lossless", + "dispatch_time": 1791944829.889363, + "planned_position": 0 + }, + { + "id": "confirm-fresh-07-hi-m0-source-chronological", + "dispatch_time": 1791944830.551471, + "planned_position": 1 + }, + { + "id": "confirm-fresh-11-en-m0-source-chronological", + "dispatch_time": 1791944858.6314209, + "planned_position": 1 + }, + { + "id": "confirm-fresh-11-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944859.298134, + "planned_position": 0 + }, + { + "id": "confirm-fresh-11-en-m0-source-grouped_lossless", + "dispatch_time": 1791944859.9634619, + "planned_position": 2 + }, + { + "id": "confirm-fresh-07-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944860.628047, + "planned_position": 2 + }, + { + "id": "confirm-fresh-03-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944883.108303, + "planned_position": 0 + }, + { + "id": "confirm-fresh-03-hi-m1-source-chronological", + "dispatch_time": 1791944883.774751, + "planned_position": 2 + }, + { + "id": "confirm-fresh-03-hi-m1-source-grouped_lossless", + "dispatch_time": 1791944884.4398699, + "planned_position": 1 + }, + { + "id": "confirm-fresh-06-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944885.1080902, + "planned_position": 0 + }, + { + "id": "confirm-fresh-28-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944895.6361878, + "planned_position": 1 + }, + { + "id": "confirm-fresh-06-en-m0-source-grouped_lossless", + "dispatch_time": 1791944896.300808, + "planned_position": 1 + }, + { + "id": "confirm-fresh-06-en-m0-source-chronological", + "dispatch_time": 1791944896.969085, + "planned_position": 2 + }, + { + "id": "confirm-fresh-28-en-m0-source-grouped_lossless", + "dispatch_time": 1791944897.6333451, + "planned_position": 0 + }, + { + "id": "confirm-fresh-20-te-m0-source-chronological", + "dispatch_time": 1791944909.036414, + "planned_position": 2 + }, + { + "id": "confirm-fresh-20-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944909.704544, + "planned_position": 0 + }, + { + "id": "confirm-fresh-28-en-m0-source-chronological", + "dispatch_time": 1791944910.371223, + "planned_position": 2 + }, + { + "id": "confirm-fresh-20-te-m0-source-grouped_lossless", + "dispatch_time": 1791944911.036484, + "planned_position": 1 + }, + { + "id": "confirm-fresh-33-hi-m0-source-grouped_lossless", + "dispatch_time": 1791944938.563628, + "planned_position": 2 + }, + { + "id": "confirm-fresh-16-te-m1-source-grouped_lossless", + "dispatch_time": 1791944939.230287, + "planned_position": 0 + }, + { + "id": "confirm-fresh-33-hi-m0-source-chronological", + "dispatch_time": 1791944939.897184, + "planned_position": 1 + }, + { + "id": "confirm-fresh-33-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944940.559493, + "planned_position": 0 + }, + { + "id": "confirm-fresh-16-te-m1-source-chronological", + "dispatch_time": 1791944956.772683, + "planned_position": 1 + }, + { + "id": "confirm-fresh-26-te-m0-source-grouped_lossless", + "dispatch_time": 1791944957.437387, + "planned_position": 1 + }, + { + "id": "confirm-fresh-26-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944958.103969, + "planned_position": 0 + }, + { + "id": "confirm-fresh-16-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944958.770329, + "planned_position": 2 + }, + { + "id": "confirm-fresh-20-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791944991.130285, + "planned_position": 1 + }, + { + "id": "confirm-fresh-20-en-m0-source-grouped_lossless", + "dispatch_time": 1791944991.798576, + "planned_position": 2 + }, + { + "id": "confirm-fresh-26-te-m0-source-chronological", + "dispatch_time": 1791944992.464446, + "planned_position": 2 + }, + { + "id": "confirm-fresh-20-en-m0-source-chronological", + "dispatch_time": 1791944993.1307468, + "planned_position": 0 + }, + { + "id": "confirm-fresh-25-hi-m0-source-grouped_lossless", + "dispatch_time": 1791945026.930353, + "planned_position": 0 + }, + { + "id": "confirm-fresh-34-te-m1-source-grouped_lossless", + "dispatch_time": 1791945027.594038, + "planned_position": 0 + }, + { + "id": "confirm-fresh-34-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945028.263675, + "planned_position": 2 + }, + { + "id": "confirm-fresh-34-te-m1-source-chronological", + "dispatch_time": 1791945028.9301639, + "planned_position": 1 + }, + { + "id": "confirm-fresh-25-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945047.598707, + "planned_position": 2 + }, + { + "id": "confirm-fresh-25-hi-m0-source-chronological", + "dispatch_time": 1791945048.265032, + "planned_position": 1 + }, + { + "id": "confirm-fresh-14-en-m1-source-chronological", + "dispatch_time": 1791945048.930488, + "planned_position": 1 + }, + { + "id": "confirm-fresh-14-en-m1-source-grouped_lossless", + "dispatch_time": 1791945049.595147, + "planned_position": 0 + }, + { + "id": "confirm-fresh-14-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945072.527575, + "planned_position": 2 + }, + { + "id": "confirm-fresh-02-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945073.1933382, + "planned_position": 2 + }, + { + "id": "confirm-fresh-02-en-m1-source-chronological", + "dispatch_time": 1791945073.862349, + "planned_position": 1 + }, + { + "id": "confirm-fresh-02-en-m1-source-grouped_lossless", + "dispatch_time": 1791945074.5281248, + "planned_position": 0 + }, + { + "id": "confirm-fresh-22-en-m0-source-chronological", + "dispatch_time": 1791945075.6755662, + "planned_position": 2 + }, + { + "id": "confirm-fresh-10-te-m0-source-chronological", + "dispatch_time": 1791945076.344018, + "planned_position": 0 + }, + { + "id": "confirm-fresh-22-en-m0-source-grouped_lossless", + "dispatch_time": 1791945077.010679, + "planned_position": 0 + }, + { + "id": "confirm-fresh-22-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945077.6768022, + "planned_position": 1 + }, + { + "id": "confirm-fresh-10-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945097.7006688, + "planned_position": 1 + }, + { + "id": "confirm-fresh-10-te-m0-source-grouped_lossless", + "dispatch_time": 1791945098.3691342, + "planned_position": 2 + }, + { + "id": "confirm-fresh-17-hi-m0-source-chronological", + "dispatch_time": 1791945099.032931, + "planned_position": 0 + }, + { + "id": "confirm-fresh-17-hi-m0-source-grouped_lossless", + "dispatch_time": 1791945099.70207, + "planned_position": 1 + }, + { + "id": "confirm-fresh-32-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945124.713896, + "planned_position": 1 + }, + { + "id": "confirm-fresh-32-hi-m0-source-chronological", + "dispatch_time": 1791945125.378762, + "planned_position": 2 + }, + { + "id": "confirm-fresh-32-hi-m0-source-grouped_lossless", + "dispatch_time": 1791945126.0445251, + "planned_position": 0 + }, + { + "id": "confirm-fresh-17-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945126.711941, + "planned_position": 2 + }, + { + "id": "confirm-fresh-31-en-m0-source-chronological", + "dispatch_time": 1791945152.545495, + "planned_position": 0 + }, + { + "id": "confirm-fresh-31-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945153.2124548, + "planned_position": 2 + }, + { + "id": "confirm-fresh-37-hi-m1-source-grouped_lossless", + "dispatch_time": 1791945153.8771589, + "planned_position": 0 + }, + { + "id": "confirm-fresh-31-en-m0-source-grouped_lossless", + "dispatch_time": 1791945154.540958, + "planned_position": 1 + }, + { + "id": "confirm-fresh-39-en-m0-source-chronological", + "dispatch_time": 1791945170.469577, + "planned_position": 1 + }, + { + "id": "confirm-fresh-39-en-m0-source-grouped_lossless", + "dispatch_time": 1791945171.122685, + "planned_position": 0 + }, + { + "id": "confirm-fresh-37-hi-m1-source-chronological", + "dispatch_time": 1791945171.790882, + "planned_position": 2 + }, + { + "id": "confirm-fresh-37-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945172.454976, + "planned_position": 1 + }, + { + "id": "confirm-fresh-23-en-m0-source-grouped_lossless", + "dispatch_time": 1791945191.683879, + "planned_position": 2 + }, + { + "id": "confirm-fresh-23-en-m0-source-chronological", + "dispatch_time": 1791945192.352789, + "planned_position": 1 + }, + { + "id": "confirm-fresh-39-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945193.0175092, + "planned_position": 2 + }, + { + "id": "confirm-fresh-23-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945193.686101, + "planned_position": 0 + }, + { + "id": "confirm-fresh-16-hi-m1-source-grouped_lossless", + "dispatch_time": 1791945221.644992, + "planned_position": 1 + }, + { + "id": "confirm-fresh-16-hi-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945222.307892, + "planned_position": 2 + }, + { + "id": "confirm-fresh-16-hi-m1-source-chronological", + "dispatch_time": 1791945222.978781, + "planned_position": 0 + }, + { + "id": "confirm-fresh-36-te-m0-source-grouped_lossless", + "dispatch_time": 1791945223.642057, + "planned_position": 0 + }, + { + "id": "confirm-fresh-36-te-m0-source-chronological", + "dispatch_time": 1791945235.6931021, + "planned_position": 2 + }, + { + "id": "confirm-fresh-36-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945236.3607612, + "planned_position": 1 + }, + { + "id": "confirm-fresh-06-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945237.022929, + "planned_position": 1 + }, + { + "id": "confirm-fresh-06-hi-m0-source-chronological", + "dispatch_time": 1791945237.693575, + "planned_position": 0 + }, + { + "id": "confirm-fresh-38-te-m0-source-grouped_lossless", + "dispatch_time": 1791945259.390059, + "planned_position": 1 + }, + { + "id": "confirm-fresh-38-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945260.053261, + "planned_position": 0 + }, + { + "id": "confirm-fresh-06-hi-m0-source-grouped_lossless", + "dispatch_time": 1791945260.724135, + "planned_position": 2 + }, + { + "id": "confirm-fresh-38-te-m0-source-chronological", + "dispatch_time": 1791945261.3900712, + "planned_position": 2 + }, + { + "id": "confirm-fresh-06-te-m1-source-chronological", + "dispatch_time": 1791945280.889952, + "planned_position": 1 + }, + { + "id": "confirm-fresh-40-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945281.554979, + "planned_position": 0 + }, + { + "id": "confirm-fresh-06-te-m1-source-grouped_lossless", + "dispatch_time": 1791945282.2199621, + "planned_position": 2 + }, + { + "id": "confirm-fresh-06-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945282.886082, + "planned_position": 0 + }, + { + "id": "confirm-fresh-01-en-m0-source-chronological", + "dispatch_time": 1791945292.925365, + "planned_position": 0 + }, + { + "id": "confirm-fresh-40-hi-m0-source-chronological", + "dispatch_time": 1791945293.59281, + "planned_position": 2 + }, + { + "id": "confirm-fresh-01-en-m0-source-grouped_lossless", + "dispatch_time": 1791945294.259507, + "planned_position": 1 + }, + { + "id": "confirm-fresh-40-hi-m0-source-grouped_lossless", + "dispatch_time": 1791945294.923541, + "planned_position": 1 + }, + { + "id": "confirm-fresh-04-hi-m0-source-chronological", + "dispatch_time": 1791945321.765378, + "planned_position": 2 + }, + { + "id": "confirm-fresh-04-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945322.433868, + "planned_position": 0 + }, + { + "id": "confirm-fresh-04-hi-m0-source-grouped_lossless", + "dispatch_time": 1791945323.096247, + "planned_position": 1 + }, + { + "id": "confirm-fresh-01-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945323.767056, + "planned_position": 2 + }, + { + "id": "confirm-fresh-33-te-m1-source-grouped_lossless", + "dispatch_time": 1791945344.156087, + "planned_position": 2 + }, + { + "id": "confirm-fresh-33-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945344.823731, + "planned_position": 1 + }, + { + "id": "confirm-fresh-05-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945345.490394, + "planned_position": 0 + }, + { + "id": "confirm-fresh-33-te-m1-source-chronological", + "dispatch_time": 1791945346.153399, + "planned_position": 0 + }, + { + "id": "confirm-fresh-32-en-m0-source-chronological", + "dispatch_time": 1791945347.272678, + "planned_position": 0 + }, + { + "id": "confirm-fresh-05-en-m1-source-grouped_lossless", + "dispatch_time": 1791945347.93951, + "planned_position": 1 + }, + { + "id": "confirm-fresh-32-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945348.602039, + "planned_position": 1 + }, + { + "id": "confirm-fresh-05-en-m1-source-chronological", + "dispatch_time": 1791945349.272853, + "planned_position": 2 + }, + { + "id": "confirm-fresh-32-en-m0-source-grouped_lossless", + "dispatch_time": 1791945362.1407611, + "planned_position": 2 + }, + { + "id": "confirm-fresh-36-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945362.8080971, + "planned_position": 0 + }, + { + "id": "confirm-fresh-36-en-m0-source-grouped_lossless", + "dispatch_time": 1791945363.4733708, + "planned_position": 1 + }, + { + "id": "confirm-fresh-36-en-m0-source-chronological", + "dispatch_time": 1791945364.141332, + "planned_position": 2 + }, + { + "id": "confirm-fresh-19-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945384.1632159, + "planned_position": 0 + }, + { + "id": "confirm-fresh-35-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945384.830814, + "planned_position": 2 + }, + { + "id": "confirm-fresh-35-te-m0-source-grouped_lossless", + "dispatch_time": 1791945385.497494, + "planned_position": 0 + }, + { + "id": "confirm-fresh-35-te-m0-source-chronological", + "dispatch_time": 1791945386.161358, + "planned_position": 1 + }, + { + "id": "confirm-fresh-19-te-m1-source-chronological", + "dispatch_time": 1791945412.686465, + "planned_position": 2 + }, + { + "id": "confirm-fresh-19-te-m1-source-grouped_lossless", + "dispatch_time": 1791945413.349, + "planned_position": 1 + }, + { + "id": "confirm-fresh-23-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945414.0183501, + "planned_position": 0 + }, + { + "id": "confirm-fresh-23-en-m1-source-grouped_lossless", + "dispatch_time": 1791945414.684617, + "planned_position": 1 + }, + { + "id": "confirm-fresh-23-en-m1-source-chronological", + "dispatch_time": 1791945415.735251, + "planned_position": 2 + }, + { + "id": "confirm-fresh-31-te-m0-source-grouped_lossless", + "dispatch_time": 1791945416.389368, + "planned_position": 2 + }, + { + "id": "confirm-fresh-31-te-m0-source-chronological", + "dispatch_time": 1791945417.0561018, + "planned_position": 1 + }, + { + "id": "confirm-fresh-31-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945417.723541, + "planned_position": 0 + }, + { + "id": "confirm-fresh-08-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945456.611126, + "planned_position": 0 + }, + { + "id": "confirm-fresh-08-te-m0-source-chronological", + "dispatch_time": 1791945457.275136, + "planned_position": 2 + }, + { + "id": "confirm-fresh-08-te-m0-source-grouped_lossless", + "dispatch_time": 1791945457.941836, + "planned_position": 1 + }, + { + "id": "confirm-fresh-03-te-m0-source-chronological", + "dispatch_time": 1791945458.608446, + "planned_position": 0 + }, + { + "id": "confirm-fresh-03-te-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945471.787454, + "planned_position": 2 + }, + { + "id": "confirm-fresh-03-te-m0-source-grouped_lossless", + "dispatch_time": 1791945472.455113, + "planned_position": 1 + }, + { + "id": "confirm-fresh-25-en-m1-source-chronological", + "dispatch_time": 1791945473.119825, + "planned_position": 0 + }, + { + "id": "confirm-fresh-25-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945473.785674, + "planned_position": 1 + }, + { + "id": "confirm-fresh-29-en-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945481.227765, + "planned_position": 0 + }, + { + "id": "confirm-fresh-25-en-m1-source-grouped_lossless", + "dispatch_time": 1791945481.895277, + "planned_position": 2 + }, + { + "id": "confirm-fresh-29-en-m1-source-grouped_lossless", + "dispatch_time": 1791945482.561922, + "planned_position": 1 + }, + { + "id": "confirm-fresh-29-en-m1-source-chronological", + "dispatch_time": 1791945483.227658, + "planned_position": 2 + }, + { + "id": "confirm-fresh-39-te-m1-source-chronological", + "dispatch_time": 1791945484.316163, + "planned_position": 0 + }, + { + "id": "confirm-fresh-39-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945484.976723, + "planned_position": 1 + }, + { + "id": "confirm-fresh-39-te-m1-source-grouped_lossless", + "dispatch_time": 1791945485.6467352, + "planned_position": 2 + }, + { + "id": "confirm-fresh-28-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945486.313379, + "planned_position": 0 + }, + { + "id": "confirm-fresh-04-te-m1-source-grouped_lossless", + "dispatch_time": 1791945514.827295, + "planned_position": 0 + }, + { + "id": "confirm-fresh-04-te-m1-source-chronological", + "dispatch_time": 1791945515.490119, + "planned_position": 1 + }, + { + "id": "confirm-fresh-28-hi-m0-source-grouped_lossless", + "dispatch_time": 1791945516.155537, + "planned_position": 1 + }, + { + "id": "confirm-fresh-28-hi-m0-source-chronological", + "dispatch_time": 1791945516.820777, + "planned_position": 2 + }, + { + "id": "confirm-fresh-02-te-m1-source-grouped_lossless", + "dispatch_time": 1791945554.038753, + "planned_position": 1 + }, + { + "id": "confirm-fresh-02-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945554.7047439, + "planned_position": 2 + }, + { + "id": "confirm-fresh-04-te-m1-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945555.3738391, + "planned_position": 2 + }, + { + "id": "confirm-fresh-02-te-m1-source-chronological", + "dispatch_time": 1791945556.04042, + "planned_position": 0 + }, + { + "id": "confirm-fresh-08-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945557.174919, + "planned_position": 1 + }, + { + "id": "confirm-fresh-27-en-m0-source-grouped_lossless", + "dispatch_time": 1791945557.841372, + "planned_position": 0 + }, + { + "id": "confirm-fresh-08-hi-m0-source-grouped_lossless", + "dispatch_time": 1791945558.50524, + "planned_position": 0 + }, + { + "id": "confirm-fresh-08-hi-m0-source-chronological", + "dispatch_time": 1791945559.172514, + "planned_position": 2 + }, + { + "id": "confirm-fresh-27-en-m0-source-chronological", + "dispatch_time": 1791945586.8729322, + "planned_position": 1 + }, + { + "id": "confirm-fresh-27-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945587.541769, + "planned_position": 2 + }, + { + "id": "confirm-fresh-14-en-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945588.208425, + "planned_position": 1 + }, + { + "id": "confirm-fresh-14-en-m0-source-chronological", + "dispatch_time": 1791945588.873316, + "planned_position": 0 + }, + { + "id": "confirm-fresh-11-hi-m0-source-grouped_actor_prose_withheld", + "dispatch_time": 1791945629.121496, + "planned_position": 2 + }, + { + "id": "confirm-fresh-11-hi-m0-source-chronological", + "dispatch_time": 1791945629.787748, + "planned_position": 0 + }, + { + "id": "confirm-fresh-11-hi-m0-source-grouped_lossless", + "dispatch_time": 1791945630.453355, + "planned_position": 1 + }, + { + "id": "confirm-fresh-14-en-m0-source-grouped_lossless", + "dispatch_time": 1791945631.119526, + "planned_position": 2 + } + ], + "groups": { + "model/arm": { + "sarvam-105b/chronological": { + "n": 120, + "factual_correct": 104, + "valid_final": 115, + "invalid_or_missing_final": 5, + "wrong_non_null_dates": 3, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 77, + "projected_status_correct": 115, + "raw_fully_correct_reports": 70, + "projected_fully_correct_reports": 104, + "known_prompt_tokens": 148412, + "known_completion_tokens": 393154, + "known_cost_microinr": 25815420, + "retained_microinr": 32269322, + "elapsed_seconds_sum": 1825.3474645020033, + "elapsed_seconds_count": 120 + }, + "sarvam-105b/grouped_actor_prose_withheld": { + "n": 120, + "factual_correct": 105, + "valid_final": 114, + "invalid_or_missing_final": 6, + "wrong_non_null_dates": 1, + "false_abstentions": 2, + "unknown_condition_guesses": 0, + "raw_status_correct": 85, + "projected_status_correct": 114, + "raw_fully_correct_reports": 78, + "projected_fully_correct_reports": 105, + "known_prompt_tokens": 199853, + "known_completion_tokens": 415826, + "known_cost_microinr": 27947355, + "retained_microinr": 34934238, + "elapsed_seconds_sum": 1956.6855917031207, + "elapsed_seconds_count": 120 + }, + "sarvam-105b/grouped_lossless": { + "n": 120, + "factual_correct": 99, + "valid_final": 112, + "invalid_or_missing_final": 8, + "wrong_non_null_dates": 4, + "false_abstentions": 1, + "unknown_condition_guesses": 2, + "raw_status_correct": 83, + "projected_status_correct": 112, + "raw_fully_correct_reports": 71, + "projected_fully_correct_reports": 99, + "known_prompt_tokens": 212692, + "known_completion_tokens": 445020, + "known_cost_microinr": 29891580, + "retained_microinr": 37364522, + "elapsed_seconds_sum": 2091.594803958971, + "elapsed_seconds_count": 120 + }, + "sarvam-105b-conversations/chronological": { + "n": 120, + "factual_correct": 96, + "valid_final": 120, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 22, + "false_abstentions": 2, + "unknown_condition_guesses": 8, + "raw_status_correct": 109, + "projected_status_correct": 120, + "raw_fully_correct_reports": 86, + "projected_fully_correct_reports": 96, + "known_prompt_tokens": 163171, + "known_completion_tokens": 4808, + "known_cost_microinr": 2736045, + "retained_microinr": 3420099, + "elapsed_seconds_sum": 121.35232799799996, + "elapsed_seconds_count": 120 + }, + "sarvam-105b-conversations/grouped_actor_prose_withheld": { + "n": 120, + "factual_correct": 103, + "valid_final": 120, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 8, + "false_abstentions": 9, + "unknown_condition_guesses": 2, + "raw_status_correct": 97, + "projected_status_correct": 120, + "raw_fully_correct_reports": 80, + "projected_fully_correct_reports": 103, + "known_prompt_tokens": 223650, + "known_completion_tokens": 4674, + "known_cost_microinr": 3635190, + "retained_microinr": 4544031, + "elapsed_seconds_sum": 124.40037220805243, + "elapsed_seconds_count": 120 + }, + "sarvam-105b-conversations/grouped_lossless": { + "n": 120, + "factual_correct": 100, + "valid_final": 120, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 18, + "false_abstentions": 2, + "unknown_condition_guesses": 6, + "raw_status_correct": 95, + "projected_status_correct": 120, + "raw_fully_correct_reports": 78, + "projected_fully_correct_reports": 100, + "known_prompt_tokens": 238962, + "known_completion_tokens": 4774, + "known_cost_microinr": 3870870, + "retained_microinr": 4838631, + "elapsed_seconds_sum": 122.39304328711296, + "elapsed_seconds_count": 120 + } + }, + "language/arm": { + "en/chronological": { + "n": 80, + "factual_correct": 69, + "valid_final": 80, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 8, + "false_abstentions": 0, + "unknown_condition_guesses": 3, + "raw_status_correct": 64, + "projected_status_correct": 80, + "raw_fully_correct_reports": 54, + "projected_fully_correct_reports": 69, + "known_prompt_tokens": 103773, + "known_completion_tokens": 125762, + "known_cost_microinr": 9102315, + "retained_microinr": 11377920, + "elapsed_seconds_sum": 612.3596142880851, + "elapsed_seconds_count": 80 + }, + "en/grouped_actor_prose_withheld": { + "n": 80, + "factual_correct": 68, + "valid_final": 79, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 3, + "false_abstentions": 5, + "unknown_condition_guesses": 0, + "raw_status_correct": 63, + "projected_status_correct": 79, + "raw_fully_correct_reports": 52, + "projected_fully_correct_reports": 68, + "known_prompt_tokens": 142728, + "known_completion_tokens": 123882, + "known_cost_microinr": 9573840, + "retained_microinr": 11967328, + "elapsed_seconds_sum": 629.993203329097, + "elapsed_seconds_count": 80 + }, + "en/grouped_lossless": { + "n": 80, + "factual_correct": 64, + "valid_final": 77, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 9, + "false_abstentions": 1, + "unknown_condition_guesses": 3, + "raw_status_correct": 58, + "projected_status_correct": 77, + "raw_fully_correct_reports": 46, + "projected_fully_correct_reports": 64, + "known_prompt_tokens": 151387, + "known_completion_tokens": 137721, + "known_cost_microinr": 10534065, + "retained_microinr": 13167609, + "elapsed_seconds_sum": 675.0767672470392, + "elapsed_seconds_count": 80 + }, + "hi/chronological": { + "n": 80, + "factual_correct": 64, + "valid_final": 78, + "invalid_or_missing_final": 2, + "wrong_non_null_dates": 8, + "false_abstentions": 2, + "unknown_condition_guesses": 1, + "raw_status_correct": 65, + "projected_status_correct": 78, + "raw_fully_correct_reports": 53, + "projected_fully_correct_reports": 64, + "known_prompt_tokens": 104081, + "known_completion_tokens": 142037, + "known_cost_microinr": 10083435, + "retained_microinr": 12604325, + "elapsed_seconds_sum": 692.7626855429698, + "elapsed_seconds_count": 80 + }, + "hi/grouped_actor_prose_withheld": { + "n": 80, + "factual_correct": 70, + "valid_final": 78, + "invalid_or_missing_final": 2, + "wrong_non_null_dates": 2, + "false_abstentions": 4, + "unknown_condition_guesses": 2, + "raw_status_correct": 64, + "projected_status_correct": 78, + "raw_fully_correct_reports": 56, + "projected_fully_correct_reports": 70, + "known_prompt_tokens": 140672, + "known_completion_tokens": 156975, + "known_cost_microinr": 11528580, + "retained_microinr": 14410756, + "elapsed_seconds_sum": 766.1579069970321, + "elapsed_seconds_count": 80 + }, + "hi/grouped_lossless": { + "n": 80, + "factual_correct": 68, + "valid_final": 78, + "invalid_or_missing_final": 2, + "wrong_non_null_dates": 5, + "false_abstentions": 1, + "unknown_condition_guesses": 1, + "raw_status_correct": 63, + "projected_status_correct": 78, + "raw_fully_correct_reports": 54, + "projected_fully_correct_reports": 68, + "known_prompt_tokens": 150640, + "known_completion_tokens": 161669, + "known_cost_microinr": 11959740, + "retained_microinr": 14949707, + "elapsed_seconds_sum": 795.5918924560974, + "elapsed_seconds_count": 80 + }, + "te/chronological": { + "n": 80, + "factual_correct": 67, + "valid_final": 77, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 9, + "false_abstentions": 0, + "unknown_condition_guesses": 4, + "raw_status_correct": 57, + "projected_status_correct": 77, + "raw_fully_correct_reports": 49, + "projected_fully_correct_reports": 67, + "known_prompt_tokens": 103729, + "known_completion_tokens": 130163, + "known_cost_microinr": 9365715, + "retained_microinr": 11707176, + "elapsed_seconds_sum": 641.5774926689483, + "elapsed_seconds_count": 80 + }, + "te/grouped_actor_prose_withheld": { + "n": 80, + "factual_correct": 70, + "valid_final": 77, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 4, + "false_abstentions": 2, + "unknown_condition_guesses": 0, + "raw_status_correct": 55, + "projected_status_correct": 77, + "raw_fully_correct_reports": 50, + "projected_fully_correct_reports": 70, + "known_prompt_tokens": 140103, + "known_completion_tokens": 139643, + "known_cost_microinr": 10480125, + "retained_microinr": 13100185, + "elapsed_seconds_sum": 684.934853585044, + "elapsed_seconds_count": 80 + }, + "te/grouped_lossless": { + "n": 80, + "factual_correct": 67, + "valid_final": 77, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 8, + "false_abstentions": 1, + "unknown_condition_guesses": 4, + "raw_status_correct": 57, + "projected_status_correct": 77, + "raw_fully_correct_reports": 49, + "projected_fully_correct_reports": 67, + "known_prompt_tokens": 149627, + "known_completion_tokens": 150404, + "known_cost_microinr": 11268645, + "retained_microinr": 14085837, + "elapsed_seconds_sum": 743.3191875429475, + "elapsed_seconds_count": 80 + } + }, + "family/arm": { + "approval_boundary/chronological": { + "n": 24, + "factual_correct": 23, + "valid_final": 23, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 23, + "projected_status_correct": 23, + "raw_fully_correct_reports": 23, + "projected_fully_correct_reports": 23, + "known_prompt_tokens": 34568, + "known_completion_tokens": 24366, + "known_cost_microinr": 1980480, + "retained_microinr": 2475608, + "elapsed_seconds_sum": 124.93233862399939, + "elapsed_seconds_count": 24 + }, + "approval_boundary/grouped_actor_prose_withheld": { + "n": 24, + "factual_correct": 21, + "valid_final": 21, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 18, + "projected_status_correct": 21, + "raw_fully_correct_reports": 18, + "projected_fully_correct_reports": 21, + "known_prompt_tokens": 49487, + "known_completion_tokens": 59469, + "known_cost_microinr": 4310445, + "retained_microinr": 5388065, + "elapsed_seconds_sum": 288.39474670799973, + "elapsed_seconds_count": 24 + }, + "approval_boundary/grouped_lossless": { + "n": 24, + "factual_correct": 23, + "valid_final": 23, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 23, + "projected_status_correct": 23, + "raw_fully_correct_reports": 23, + "projected_fully_correct_reports": 23, + "known_prompt_tokens": 49950, + "known_completion_tokens": 54108, + "known_cost_microinr": 3995730, + "retained_microinr": 4994671, + "elapsed_seconds_sum": 270.08496124800877, + "elapsed_seconds_count": 24 + }, + "approved_change/chronological": { + "n": 24, + "factual_correct": 24, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 24, + "projected_status_correct": 24, + "raw_fully_correct_reports": 24, + "projected_fully_correct_reports": 24, + "known_prompt_tokens": 34282, + "known_completion_tokens": 18536, + "known_cost_microinr": 1626390, + "retained_microinr": 2032997, + "elapsed_seconds_sum": 99.32677495699318, + "elapsed_seconds_count": 24 + }, + "approved_change/grouped_actor_prose_withheld": { + "n": 24, + "factual_correct": 23, + "valid_final": 23, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 23, + "projected_status_correct": 23, + "raw_fully_correct_reports": 23, + "projected_fully_correct_reports": 23, + "known_prompt_tokens": 49651, + "known_completion_tokens": 44579, + "known_cost_microinr": 3419505, + "retained_microinr": 4274393, + "elapsed_seconds_sum": 215.81478774700372, + "elapsed_seconds_count": 24 + }, + "approved_change/grouped_lossless": { + "n": 24, + "factual_correct": 22, + "valid_final": 22, + "invalid_or_missing_final": 2, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 22, + "projected_status_correct": 22, + "raw_fully_correct_reports": 22, + "projected_fully_correct_reports": 22, + "known_prompt_tokens": 49727, + "known_completion_tokens": 55597, + "known_cost_microinr": 4081725, + "retained_microinr": 5102168, + "elapsed_seconds_sum": 265.78659429302206, + "elapsed_seconds_count": 24 + }, + "completed_older_request/chronological": { + "n": 24, + "factual_correct": 24, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 24, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 24, + "known_prompt_tokens": 45027, + "known_completion_tokens": 37998, + "known_cost_microinr": 2955285, + "retained_microinr": 3694116, + "elapsed_seconds_sum": 189.21665633299563, + "elapsed_seconds_count": 24 + }, + "completed_older_request/grouped_actor_prose_withheld": { + "n": 24, + "factual_correct": 24, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 10, + "projected_status_correct": 24, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 24, + "known_prompt_tokens": 52077, + "known_completion_tokens": 50962, + "known_cost_microinr": 3838875, + "retained_microinr": 4798603, + "elapsed_seconds_sum": 243.34911787405144, + "elapsed_seconds_count": 24 + }, + "completed_older_request/grouped_lossless": { + "n": 24, + "factual_correct": 24, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 9, + "projected_status_correct": 24, + "raw_fully_correct_reports": 9, + "projected_fully_correct_reports": 24, + "known_prompt_tokens": 58756, + "known_completion_tokens": 30792, + "known_cost_microinr": 2728860, + "retained_microinr": 3411086, + "elapsed_seconds_sum": 151.76564099805546, + "elapsed_seconds_count": 24 + }, + "conflicting_request_key/chronological": { + "n": 24, + "factual_correct": 24, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 7, + "projected_status_correct": 24, + "raw_fully_correct_reports": 7, + "projected_fully_correct_reports": 24, + "known_prompt_tokens": 42287, + "known_completion_tokens": 38330, + "known_cost_microinr": 2934105, + "retained_microinr": 3667641, + "elapsed_seconds_sum": 194.26909587701084, + "elapsed_seconds_count": 24 + }, + "conflicting_request_key/grouped_actor_prose_withheld": { + "n": 24, + "factual_correct": 23, + "valid_final": 23, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 4, + "projected_status_correct": 23, + "raw_fully_correct_reports": 4, + "projected_fully_correct_reports": 23, + "known_prompt_tokens": 49694, + "known_completion_tokens": 53177, + "known_cost_microinr": 3936030, + "retained_microinr": 4920046, + "elapsed_seconds_sum": 258.80332908200216, + "elapsed_seconds_count": 24 + }, + "conflicting_request_key/grouped_lossless": { + "n": 24, + "factual_correct": 23, + "valid_final": 23, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 3, + "projected_status_correct": 23, + "raw_fully_correct_reports": 3, + "projected_fully_correct_reports": 23, + "known_prompt_tokens": 56207, + "known_completion_tokens": 52136, + "known_cost_microinr": 3971265, + "retained_microinr": 4964088, + "elapsed_seconds_sum": 252.0191318729776, + "elapsed_seconds_count": 24 + }, + "future_approval/chronological": { + "n": 24, + "factual_correct": 24, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 17, + "projected_status_correct": 24, + "raw_fully_correct_reports": 17, + "projected_fully_correct_reports": 24, + "known_prompt_tokens": 34958, + "known_completion_tokens": 41779, + "known_cost_microinr": 3031110, + "retained_microinr": 3788896, + "elapsed_seconds_sum": 207.88787912401312, + "elapsed_seconds_count": 24 + }, + "future_approval/grouped_actor_prose_withheld": { + "n": 24, + "factual_correct": 23, + "valid_final": 23, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 13, + "projected_status_correct": 23, + "raw_fully_correct_reports": 13, + "projected_fully_correct_reports": 23, + "known_prompt_tokens": 47338, + "known_completion_tokens": 38760, + "known_cost_microinr": 3035670, + "retained_microinr": 3794595, + "elapsed_seconds_sum": 189.74250275004306, + "elapsed_seconds_count": 24 + }, + "future_approval/grouped_lossless": { + "n": 24, + "factual_correct": 24, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 14, + "projected_status_correct": 24, + "raw_fully_correct_reports": 14, + "projected_fully_correct_reports": 24, + "known_prompt_tokens": 49883, + "known_completion_tokens": 40813, + "known_cost_microinr": 3197025, + "retained_microinr": 3996288, + "elapsed_seconds_sum": 209.145510457005, + "elapsed_seconds_count": 24 + }, + "missing_replacement/chronological": { + "n": 24, + "factual_correct": 14, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 2, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 20, + "projected_status_correct": 24, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 14, + "known_prompt_tokens": 23409, + "known_completion_tokens": 43564, + "known_cost_microinr": 2964975, + "retained_microinr": 3706227, + "elapsed_seconds_sum": 215.47495245996106, + "elapsed_seconds_count": 24 + }, + "missing_replacement/grouped_actor_prose_withheld": { + "n": 24, + "factual_correct": 18, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 24, + "projected_status_correct": 24, + "raw_fully_correct_reports": 18, + "projected_fully_correct_reports": 18, + "known_prompt_tokens": 34588, + "known_completion_tokens": 36255, + "known_cost_microinr": 2694120, + "retained_microinr": 3367657, + "elapsed_seconds_sum": 187.24686512706103, + "elapsed_seconds_count": 24 + }, + "missing_replacement/grouped_lossless": { + "n": 24, + "factual_correct": 15, + "valid_final": 23, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 22, + "projected_status_correct": 23, + "raw_fully_correct_reports": 14, + "projected_fully_correct_reports": 15, + "known_prompt_tokens": 36665, + "known_completion_tokens": 45661, + "known_cost_microinr": 3289635, + "retained_microinr": 4112052, + "elapsed_seconds_sum": 226.66413641495456, + "elapsed_seconds_count": 24 + }, + "request_correction/chronological": { + "n": 24, + "factual_correct": 24, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 21, + "projected_status_correct": 24, + "raw_fully_correct_reports": 21, + "projected_fully_correct_reports": 24, + "known_prompt_tokens": 22183, + "known_completion_tokens": 36194, + "known_cost_microinr": 2504385, + "retained_microinr": 3130492, + "elapsed_seconds_sum": 177.66329100201256, + "elapsed_seconds_count": 24 + }, + "request_correction/grouped_actor_prose_withheld": { + "n": 24, + "factual_correct": 19, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 5, + "unknown_condition_guesses": 0, + "raw_status_correct": 24, + "projected_status_correct": 24, + "raw_fully_correct_reports": 19, + "projected_fully_correct_reports": 19, + "known_prompt_tokens": 33733, + "known_completion_tokens": 24925, + "known_cost_microinr": 2001495, + "retained_microinr": 2501879, + "elapsed_seconds_sum": 134.18261195802188, + "elapsed_seconds_count": 24 + }, + "request_correction/grouped_lossless": { + "n": 24, + "factual_correct": 24, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 24, + "projected_status_correct": 24, + "raw_fully_correct_reports": 24, + "projected_fully_correct_reports": 24, + "known_prompt_tokens": 34944, + "known_completion_tokens": 25987, + "known_cost_microinr": 2083380, + "retained_microinr": 2604235, + "elapsed_seconds_sum": 137.0122149180097, + "elapsed_seconds_count": 24 + }, + "resolved_condition/chronological": { + "n": 24, + "factual_correct": 22, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 2, + "unknown_condition_guesses": 0, + "raw_status_correct": 22, + "projected_status_correct": 24, + "raw_fully_correct_reports": 20, + "projected_fully_correct_reports": 22, + "known_prompt_tokens": 25177, + "known_completion_tokens": 40246, + "known_cost_microinr": 2792415, + "retained_microinr": 3490526, + "elapsed_seconds_sum": 192.2878676239925, + "elapsed_seconds_count": 24 + }, + "resolved_condition/grouped_actor_prose_withheld": { + "n": 24, + "factual_correct": 19, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 5, + "unknown_condition_guesses": 0, + "raw_status_correct": 21, + "projected_status_correct": 24, + "raw_fully_correct_reports": 17, + "projected_fully_correct_reports": 19, + "known_prompt_tokens": 35441, + "known_completion_tokens": 34229, + "known_cost_microinr": 2585355, + "retained_microinr": 3231700, + "elapsed_seconds_sum": 179.3986870820372, + "elapsed_seconds_count": 24 + }, + "resolved_condition/grouped_lossless": { + "n": 24, + "factual_correct": 22, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 2, + "unknown_condition_guesses": 0, + "raw_status_correct": 22, + "projected_status_correct": 24, + "raw_fully_correct_reports": 20, + "projected_fully_correct_reports": 22, + "known_prompt_tokens": 38728, + "known_completion_tokens": 34315, + "known_cost_microinr": 2639820, + "retained_microinr": 3299785, + "elapsed_seconds_sum": 180.69943166701705, + "elapsed_seconds_count": 24 + }, + "separate_booking/chronological": { + "n": 24, + "factual_correct": 8, + "valid_final": 23, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 15, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 20, + "projected_status_correct": 23, + "raw_fully_correct_reports": 8, + "projected_fully_correct_reports": 8, + "known_prompt_tokens": 24190, + "known_completion_tokens": 60176, + "known_cost_microinr": 3973410, + "retained_microinr": 4966771, + "elapsed_seconds_sum": 288.1164910010266, + "elapsed_seconds_count": 24 + }, + "separate_booking/grouped_actor_prose_withheld": { + "n": 24, + "factual_correct": 16, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 7, + "false_abstentions": 1, + "unknown_condition_guesses": 0, + "raw_status_correct": 21, + "projected_status_correct": 24, + "raw_fully_correct_reports": 14, + "projected_fully_correct_reports": 16, + "known_prompt_tokens": 36879, + "known_completion_tokens": 39161, + "known_cost_microinr": 2902845, + "retained_microinr": 3628565, + "elapsed_seconds_sum": 191.97024220896128, + "elapsed_seconds_count": 24 + }, + "separate_booking/grouped_lossless": { + "n": 24, + "factual_correct": 6, + "valid_final": 21, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 14, + "false_abstentions": 1, + "unknown_condition_guesses": 0, + "raw_status_correct": 17, + "projected_status_correct": 21, + "raw_fully_correct_reports": 6, + "projected_fully_correct_reports": 6, + "known_prompt_tokens": 38219, + "known_completion_tokens": 65893, + "known_cost_microinr": 4526865, + "retained_microinr": 5658591, + "elapsed_seconds_sum": 311.85564658297517, + "elapsed_seconds_count": 24 + }, + "unresolved_condition/chronological": { + "n": 24, + "factual_correct": 13, + "valid_final": 21, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 8, + "false_abstentions": 0, + "unknown_condition_guesses": 8, + "raw_status_correct": 20, + "projected_status_correct": 21, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 13, + "known_prompt_tokens": 25502, + "known_completion_tokens": 56773, + "known_cost_microinr": 3788910, + "retained_microinr": 4736147, + "elapsed_seconds_sum": 257.5244454979984, + "elapsed_seconds_count": 24 + }, + "unresolved_condition/grouped_actor_prose_withheld": { + "n": 24, + "factual_correct": 22, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 2, + "false_abstentions": 0, + "unknown_condition_guesses": 2, + "raw_status_correct": 24, + "projected_status_correct": 24, + "raw_fully_correct_reports": 22, + "projected_fully_correct_reports": 22, + "known_prompt_tokens": 34615, + "known_completion_tokens": 38983, + "known_cost_microinr": 2858205, + "retained_microinr": 3572766, + "elapsed_seconds_sum": 192.18307337399165, + "elapsed_seconds_count": 24 + }, + "unresolved_condition/grouped_lossless": { + "n": 24, + "factual_correct": 16, + "valid_final": 24, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 8, + "false_abstentions": 0, + "unknown_condition_guesses": 8, + "raw_status_correct": 22, + "projected_status_correct": 24, + "raw_fully_correct_reports": 14, + "projected_fully_correct_reports": 16, + "known_prompt_tokens": 38575, + "known_completion_tokens": 44492, + "known_cost_microinr": 3248145, + "retained_microinr": 4060189, + "elapsed_seconds_sum": 208.95457879405876, + "elapsed_seconds_count": 24 + } + }, + "model/language/arm": { + "sarvam-105b/en/chronological": { + "n": 40, + "factual_correct": 36, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 1, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 26, + "projected_status_correct": 40, + "raw_fully_correct_reports": 23, + "projected_fully_correct_reports": 36, + "known_prompt_tokens": 48520, + "known_completion_tokens": 124156, + "known_cost_microinr": 8177160, + "retained_microinr": 10221464, + "elapsed_seconds_sum": 571.59208149802, + "elapsed_seconds_count": 40 + }, + "sarvam-105b/en/grouped_actor_prose_withheld": { + "n": 40, + "factual_correct": 36, + "valid_final": 39, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 29, + "projected_status_correct": 39, + "raw_fully_correct_reports": 26, + "projected_fully_correct_reports": 36, + "known_prompt_tokens": 66351, + "known_completion_tokens": 122348, + "known_cost_microinr": 8336145, + "retained_microinr": 10420195, + "elapsed_seconds_sum": 588.4635169550893, + "elapsed_seconds_count": 40 + }, + "sarvam-105b/en/grouped_lossless": { + "n": 40, + "factual_correct": 32, + "valid_final": 37, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 2, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 27, + "projected_status_correct": 37, + "raw_fully_correct_reports": 22, + "projected_fully_correct_reports": 32, + "known_prompt_tokens": 69880, + "known_completion_tokens": 136121, + "known_cost_microinr": 9215460, + "retained_microinr": 11519340, + "elapsed_seconds_sum": 634.3584105819755, + "elapsed_seconds_count": 40 + }, + "sarvam-105b/hi/chronological": { + "n": 40, + "factual_correct": 33, + "valid_final": 38, + "invalid_or_missing_final": 2, + "wrong_non_null_dates": 1, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 29, + "projected_status_correct": 38, + "raw_fully_correct_reports": 25, + "projected_fully_correct_reports": 33, + "known_prompt_tokens": 50160, + "known_completion_tokens": 140449, + "known_cost_microinr": 9179340, + "retained_microinr": 11474191, + "elapsed_seconds_sum": 651.8509295010299, + "elapsed_seconds_count": 40 + }, + "sarvam-105b/hi/grouped_actor_prose_withheld": { + "n": 40, + "factual_correct": 36, + "valid_final": 38, + "invalid_or_missing_final": 2, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 32, + "projected_status_correct": 38, + "raw_fully_correct_reports": 30, + "projected_fully_correct_reports": 36, + "known_prompt_tokens": 67036, + "known_completion_tokens": 155415, + "known_cost_microinr": 10330440, + "retained_microinr": 12913067, + "elapsed_seconds_sum": 723.8146057900158, + "elapsed_seconds_count": 40 + }, + "sarvam-105b/hi/grouped_lossless": { + "n": 40, + "factual_correct": 33, + "valid_final": 38, + "invalid_or_missing_final": 2, + "wrong_non_null_dates": 1, + "false_abstentions": 0, + "unknown_condition_guesses": 1, + "raw_status_correct": 31, + "projected_status_correct": 38, + "raw_fully_correct_reports": 26, + "projected_fully_correct_reports": 33, + "known_prompt_tokens": 71859, + "known_completion_tokens": 160099, + "known_cost_microinr": 10683825, + "retained_microinr": 13354798, + "elapsed_seconds_sum": 755.5494629169989, + "elapsed_seconds_count": 40 + }, + "sarvam-105b/te/chronological": { + "n": 40, + "factual_correct": 35, + "valid_final": 37, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 1, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 22, + "projected_status_correct": 37, + "raw_fully_correct_reports": 22, + "projected_fully_correct_reports": 35, + "known_prompt_tokens": 49732, + "known_completion_tokens": 128549, + "known_cost_microinr": 8458920, + "retained_microinr": 10573667, + "elapsed_seconds_sum": 601.9044535029534, + "elapsed_seconds_count": 40 + }, + "sarvam-105b/te/grouped_actor_prose_withheld": { + "n": 40, + "factual_correct": 33, + "valid_final": 37, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 1, + "false_abstentions": 2, + "unknown_condition_guesses": 0, + "raw_status_correct": 24, + "projected_status_correct": 37, + "raw_fully_correct_reports": 22, + "projected_fully_correct_reports": 33, + "known_prompt_tokens": 66466, + "known_completion_tokens": 138063, + "known_cost_microinr": 9280770, + "retained_microinr": 11600976, + "elapsed_seconds_sum": 644.4074689580157, + "elapsed_seconds_count": 40 + }, + "sarvam-105b/te/grouped_lossless": { + "n": 40, + "factual_correct": 34, + "valid_final": 37, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 1, + "false_abstentions": 1, + "unknown_condition_guesses": 1, + "raw_status_correct": 25, + "projected_status_correct": 37, + "raw_fully_correct_reports": 23, + "projected_fully_correct_reports": 34, + "known_prompt_tokens": 70953, + "known_completion_tokens": 148800, + "known_cost_microinr": 9992295, + "retained_microinr": 12490384, + "elapsed_seconds_sum": 701.6869304599968, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/en/chronological": { + "n": 40, + "factual_correct": 33, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 7, + "false_abstentions": 0, + "unknown_condition_guesses": 3, + "raw_status_correct": 38, + "projected_status_correct": 40, + "raw_fully_correct_reports": 31, + "projected_fully_correct_reports": 33, + "known_prompt_tokens": 55253, + "known_completion_tokens": 1606, + "known_cost_microinr": 925155, + "retained_microinr": 1156456, + "elapsed_seconds_sum": 40.76753279006516, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/en/grouped_actor_prose_withheld": { + "n": 40, + "factual_correct": 32, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 3, + "false_abstentions": 5, + "unknown_condition_guesses": 0, + "raw_status_correct": 34, + "projected_status_correct": 40, + "raw_fully_correct_reports": 26, + "projected_fully_correct_reports": 32, + "known_prompt_tokens": 76377, + "known_completion_tokens": 1534, + "known_cost_microinr": 1237695, + "retained_microinr": 1547133, + "elapsed_seconds_sum": 41.529686374007724, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/en/grouped_lossless": { + "n": 40, + "factual_correct": 32, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 7, + "false_abstentions": 1, + "unknown_condition_guesses": 3, + "raw_status_correct": 31, + "projected_status_correct": 40, + "raw_fully_correct_reports": 24, + "projected_fully_correct_reports": 32, + "known_prompt_tokens": 81507, + "known_completion_tokens": 1600, + "known_cost_microinr": 1318605, + "retained_microinr": 1648269, + "elapsed_seconds_sum": 40.718356665063766, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/hi/chronological": { + "n": 40, + "factual_correct": 31, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 7, + "false_abstentions": 2, + "unknown_condition_guesses": 1, + "raw_status_correct": 36, + "projected_status_correct": 40, + "raw_fully_correct_reports": 28, + "projected_fully_correct_reports": 31, + "known_prompt_tokens": 53921, + "known_completion_tokens": 1588, + "known_cost_microinr": 904095, + "retained_microinr": 1130134, + "elapsed_seconds_sum": 40.91175604193995, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/hi/grouped_actor_prose_withheld": { + "n": 40, + "factual_correct": 34, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 2, + "false_abstentions": 4, + "unknown_condition_guesses": 2, + "raw_status_correct": 32, + "projected_status_correct": 40, + "raw_fully_correct_reports": 26, + "projected_fully_correct_reports": 34, + "known_prompt_tokens": 73636, + "known_completion_tokens": 1560, + "known_cost_microinr": 1198140, + "retained_microinr": 1497689, + "elapsed_seconds_sum": 42.34330120701634, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/hi/grouped_lossless": { + "n": 40, + "factual_correct": 35, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 4, + "false_abstentions": 1, + "unknown_condition_guesses": 0, + "raw_status_correct": 32, + "projected_status_correct": 40, + "raw_fully_correct_reports": 28, + "projected_fully_correct_reports": 35, + "known_prompt_tokens": 78781, + "known_completion_tokens": 1570, + "known_cost_microinr": 1275915, + "retained_microinr": 1594909, + "elapsed_seconds_sum": 40.04242953909852, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/te/chronological": { + "n": 40, + "factual_correct": 32, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 8, + "false_abstentions": 0, + "unknown_condition_guesses": 4, + "raw_status_correct": 35, + "projected_status_correct": 40, + "raw_fully_correct_reports": 27, + "projected_fully_correct_reports": 32, + "known_prompt_tokens": 53997, + "known_completion_tokens": 1614, + "known_cost_microinr": 906795, + "retained_microinr": 1133509, + "elapsed_seconds_sum": 39.67303916599485, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/te/grouped_actor_prose_withheld": { + "n": 40, + "factual_correct": 37, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 3, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 31, + "projected_status_correct": 40, + "raw_fully_correct_reports": 28, + "projected_fully_correct_reports": 37, + "known_prompt_tokens": 73637, + "known_completion_tokens": 1580, + "known_cost_microinr": 1199355, + "retained_microinr": 1499209, + "elapsed_seconds_sum": 40.52738462702837, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/te/grouped_lossless": { + "n": 40, + "factual_correct": 33, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 7, + "false_abstentions": 0, + "unknown_condition_guesses": 3, + "raw_status_correct": 32, + "projected_status_correct": 40, + "raw_fully_correct_reports": 26, + "projected_fully_correct_reports": 33, + "known_prompt_tokens": 78674, + "known_completion_tokens": 1604, + "known_cost_microinr": 1276350, + "retained_microinr": 1595453, + "elapsed_seconds_sum": 41.63225708295067, + "elapsed_seconds_count": 40 + } + }, + "model/family/arm": { + "sarvam-105b/approval_boundary/chronological": { + "n": 12, + "factual_correct": 11, + "valid_final": 11, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 11, + "projected_status_correct": 11, + "raw_fully_correct_reports": 11, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 16620, + "known_completion_tokens": 23884, + "known_cost_microinr": 1682340, + "retained_microinr": 2102930, + "elapsed_seconds_sum": 112.65253216601559, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/approval_boundary/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 9, + "valid_final": 9, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 7, + "projected_status_correct": 9, + "raw_fully_correct_reports": 7, + "projected_fully_correct_reports": 9, + "known_prompt_tokens": 23426, + "known_completion_tokens": 58987, + "known_cost_microinr": 3890610, + "retained_microinr": 4863267, + "elapsed_seconds_sum": 276.1262179579935, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/approval_boundary/grouped_lossless": { + "n": 12, + "factual_correct": 11, + "valid_final": 11, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 11, + "projected_status_correct": 11, + "raw_fully_correct_reports": 11, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 23621, + "known_completion_tokens": 53626, + "known_cost_microinr": 3571875, + "retained_microinr": 4464848, + "elapsed_seconds_sum": 257.84136253999895, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/approved_change/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 16451, + "known_completion_tokens": 18056, + "known_cost_microinr": 1330125, + "retained_microinr": 1662661, + "elapsed_seconds_sum": 87.49041983297502, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/approved_change/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 11, + "valid_final": 11, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 11, + "projected_status_correct": 11, + "raw_fully_correct_reports": 11, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 23349, + "known_completion_tokens": 44099, + "known_cost_microinr": 2996175, + "retained_microinr": 3745225, + "elapsed_seconds_sum": 203.3663537060056, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/approved_change/grouped_lossless": { + "n": 12, + "factual_correct": 10, + "valid_final": 10, + "invalid_or_missing_final": 2, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 10, + "projected_status_correct": 10, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 10, + "known_prompt_tokens": 23425, + "known_completion_tokens": 55117, + "known_cost_microinr": 3658395, + "retained_microinr": 4573000, + "elapsed_seconds_sum": 253.79363200101943, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/completed_older_request/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 3, + "projected_status_correct": 12, + "raw_fully_correct_reports": 3, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 21436, + "known_completion_tokens": 37494, + "known_cost_microinr": 2571180, + "retained_microinr": 3213980, + "elapsed_seconds_sum": 176.31832258298527, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/completed_older_request/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 3, + "projected_status_correct": 12, + "raw_fully_correct_reports": 3, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 24746, + "known_completion_tokens": 50458, + "known_cost_microinr": 3398670, + "retained_microinr": 4248341, + "elapsed_seconds_sum": 230.21405529203184, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/completed_older_request/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 3, + "projected_status_correct": 12, + "raw_fully_correct_reports": 3, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 27639, + "known_completion_tokens": 30288, + "known_cost_microinr": 2231865, + "retained_microinr": 2789838, + "elapsed_seconds_sum": 139.80573837601696, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/conflicting_request_key/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 1, + "projected_status_correct": 12, + "raw_fully_correct_reports": 1, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 19973, + "known_completion_tokens": 37830, + "known_cost_microinr": 2569395, + "retained_microinr": 3211749, + "elapsed_seconds_sum": 181.99868800201511, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/conflicting_request_key/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 11, + "valid_final": 11, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 2, + "projected_status_correct": 11, + "raw_fully_correct_reports": 2, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 23677, + "known_completion_tokens": 52673, + "known_cost_microinr": 3515535, + "retained_microinr": 4394423, + "elapsed_seconds_sum": 246.72375450000982, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/conflicting_request_key/grouped_lossless": { + "n": 12, + "factual_correct": 11, + "valid_final": 11, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 1, + "projected_status_correct": 11, + "raw_fully_correct_reports": 1, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 26339, + "known_completion_tokens": 51636, + "known_cost_microinr": 3493245, + "retained_microinr": 4366560, + "elapsed_seconds_sum": 239.8147215420031, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/future_approval/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 5, + "projected_status_correct": 12, + "raw_fully_correct_reports": 5, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 16755, + "known_completion_tokens": 41297, + "known_cost_microinr": 2729145, + "retained_microinr": 3411435, + "elapsed_seconds_sum": 195.9946642490395, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/future_approval/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 11, + "valid_final": 11, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 8, + "projected_status_correct": 11, + "raw_fully_correct_reports": 8, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 21149, + "known_completion_tokens": 38266, + "known_cost_microinr": 2613195, + "retained_microinr": 3266498, + "elapsed_seconds_sum": 177.69715537501907, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/future_approval/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 7, + "projected_status_correct": 12, + "raw_fully_correct_reports": 7, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 23251, + "known_completion_tokens": 40323, + "known_cost_microinr": 2768145, + "retained_microinr": 3460185, + "elapsed_seconds_sum": 196.18101524897793, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/missing_replacement/chronological": { + "n": 12, + "factual_correct": 4, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 8, + "projected_status_correct": 12, + "raw_fully_correct_reports": 2, + "projected_fully_correct_reports": 4, + "known_prompt_tokens": 11294, + "known_completion_tokens": 43160, + "known_cost_microinr": 2759010, + "retained_microinr": 3448767, + "elapsed_seconds_sum": 203.2952735429717, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/missing_replacement/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 6, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 6, + "projected_fully_correct_reports": 6, + "known_prompt_tokens": 16420, + "known_completion_tokens": 35871, + "known_cost_microinr": 2398560, + "retained_microinr": 2998204, + "elapsed_seconds_sum": 175.4489582910319, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/missing_replacement/grouped_lossless": { + "n": 12, + "factual_correct": 3, + "valid_final": 11, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 10, + "projected_status_correct": 11, + "raw_fully_correct_reports": 2, + "projected_fully_correct_reports": 3, + "known_prompt_tokens": 17483, + "known_completion_tokens": 45277, + "known_cost_microinr": 2978865, + "retained_microinr": 3723585, + "elapsed_seconds_sum": 214.70913649894646, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/request_correction/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 10, + "projected_status_correct": 12, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 10848, + "known_completion_tokens": 35690, + "known_cost_microinr": 2304120, + "retained_microinr": 2880155, + "elapsed_seconds_sum": 165.80868412599375, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/request_correction/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 11, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 1, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 11, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 16565, + "known_completion_tokens": 24461, + "known_cost_microinr": 1716135, + "retained_microinr": 2145175, + "elapsed_seconds_sum": 120.96716408400971, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/request_correction/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 17086, + "known_completion_tokens": 25483, + "known_cost_microinr": 1785270, + "retained_microinr": 2231592, + "elapsed_seconds_sum": 124.70271487400169, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/resolved_condition/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 10, + "projected_status_correct": 12, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 11317, + "known_completion_tokens": 39762, + "known_cost_microinr": 2555475, + "retained_microinr": 3194348, + "elapsed_seconds_sum": 180.207794499991, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/resolved_condition/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 11, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 1, + "unknown_condition_guesses": 0, + "raw_status_correct": 9, + "projected_status_correct": 12, + "raw_fully_correct_reports": 9, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 16297, + "known_completion_tokens": 33765, + "known_cost_microinr": 2270355, + "retained_microinr": 2837947, + "elapsed_seconds_sum": 167.5436829990358, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/resolved_condition/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 10, + "projected_status_correct": 12, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 17374, + "known_completion_tokens": 33831, + "known_cost_microinr": 2290470, + "retained_microinr": 2863093, + "elapsed_seconds_sum": 168.20070904299791, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/separate_booking/chronological": { + "n": 12, + "factual_correct": 8, + "valid_final": 11, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 3, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 9, + "projected_status_correct": 11, + "raw_fully_correct_reports": 8, + "projected_fully_correct_reports": 8, + "known_prompt_tokens": 11742, + "known_completion_tokens": 59672, + "known_cost_microinr": 3756450, + "retained_microinr": 4695568, + "elapsed_seconds_sum": 276.12464366802305, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/separate_booking/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 11, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 1, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 9, + "projected_status_correct": 12, + "raw_fully_correct_reports": 9, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 17981, + "known_completion_tokens": 38667, + "known_cost_microinr": 2589735, + "retained_microinr": 3237172, + "elapsed_seconds_sum": 180.1802984579699, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/separate_booking/grouped_lossless": { + "n": 12, + "factual_correct": 6, + "valid_final": 9, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 2, + "false_abstentions": 1, + "unknown_condition_guesses": 0, + "raw_status_correct": 8, + "projected_status_correct": 9, + "raw_fully_correct_reports": 6, + "projected_fully_correct_reports": 6, + "known_prompt_tokens": 18512, + "known_completion_tokens": 65391, + "known_cost_microinr": 4201140, + "retained_microinr": 5251430, + "elapsed_seconds_sum": 299.79583091697714, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/unresolved_condition/chronological": { + "n": 12, + "factual_correct": 9, + "valid_final": 9, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 8, + "projected_status_correct": 9, + "raw_fully_correct_reports": 8, + "projected_fully_correct_reports": 9, + "known_prompt_tokens": 11976, + "known_completion_tokens": 56309, + "known_cost_microinr": 3558180, + "retained_microinr": 4447729, + "elapsed_seconds_sum": 245.4564418319933, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/unresolved_condition/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 16243, + "known_completion_tokens": 38579, + "known_cost_microinr": 2558385, + "retained_microinr": 3197986, + "elapsed_seconds_sum": 178.41795104001358, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/unresolved_condition/grouped_lossless": { + "n": 12, + "factual_correct": 10, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 2, + "false_abstentions": 0, + "unknown_condition_guesses": 2, + "raw_status_correct": 11, + "projected_status_correct": 12, + "raw_fully_correct_reports": 9, + "projected_fully_correct_reports": 10, + "known_prompt_tokens": 17962, + "known_completion_tokens": 44048, + "known_cost_microinr": 2912310, + "retained_microinr": 3640391, + "elapsed_seconds_sum": 196.7499429180316, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/approval_boundary/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 17948, + "known_completion_tokens": 482, + "known_cost_microinr": 298140, + "retained_microinr": 372678, + "elapsed_seconds_sum": 12.279806457983796, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/approval_boundary/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 11, + "projected_status_correct": 12, + "raw_fully_correct_reports": 11, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 26061, + "known_completion_tokens": 482, + "known_cost_microinr": 419835, + "retained_microinr": 524798, + "elapsed_seconds_sum": 12.268528750006226, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/approval_boundary/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 26329, + "known_completion_tokens": 482, + "known_cost_microinr": 423855, + "retained_microinr": 529823, + "elapsed_seconds_sum": 12.243598708009813, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/approved_change/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 17831, + "known_completion_tokens": 480, + "known_cost_microinr": 296265, + "retained_microinr": 370336, + "elapsed_seconds_sum": 11.836355124018155, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/approved_change/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 26302, + "known_completion_tokens": 480, + "known_cost_microinr": 423330, + "retained_microinr": 529168, + "elapsed_seconds_sum": 12.448434040998109, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/approved_change/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 26302, + "known_completion_tokens": 480, + "known_cost_microinr": 423330, + "retained_microinr": 529168, + "elapsed_seconds_sum": 11.99296229200263, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/completed_older_request/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 9, + "projected_status_correct": 12, + "raw_fully_correct_reports": 9, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 23591, + "known_completion_tokens": 504, + "known_cost_microinr": 384105, + "retained_microinr": 480136, + "elapsed_seconds_sum": 12.898333750010352, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/completed_older_request/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 7, + "projected_status_correct": 12, + "raw_fully_correct_reports": 7, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 27331, + "known_completion_tokens": 504, + "known_cost_microinr": 440205, + "retained_microinr": 550262, + "elapsed_seconds_sum": 13.1350625820196, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/completed_older_request/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 6, + "projected_status_correct": 12, + "raw_fully_correct_reports": 6, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 31117, + "known_completion_tokens": 504, + "known_cost_microinr": 496995, + "retained_microinr": 621248, + "elapsed_seconds_sum": 11.959902622038499, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/conflicting_request_key/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 6, + "projected_status_correct": 12, + "raw_fully_correct_reports": 6, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 22314, + "known_completion_tokens": 500, + "known_cost_microinr": 364710, + "retained_microinr": 455892, + "elapsed_seconds_sum": 12.270407874995726, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/conflicting_request_key/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 2, + "projected_status_correct": 12, + "raw_fully_correct_reports": 2, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 26017, + "known_completion_tokens": 504, + "known_cost_microinr": 420495, + "retained_microinr": 525623, + "elapsed_seconds_sum": 12.079574581992347, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/conflicting_request_key/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 2, + "projected_status_correct": 12, + "raw_fully_correct_reports": 2, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 29868, + "known_completion_tokens": 500, + "known_cost_microinr": 478020, + "retained_microinr": 597528, + "elapsed_seconds_sum": 12.204410330974497, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/future_approval/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 18203, + "known_completion_tokens": 482, + "known_cost_microinr": 301965, + "retained_microinr": 377461, + "elapsed_seconds_sum": 11.893214874973637, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/future_approval/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 5, + "projected_status_correct": 12, + "raw_fully_correct_reports": 5, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 26189, + "known_completion_tokens": 494, + "known_cost_microinr": 422475, + "retained_microinr": 528097, + "elapsed_seconds_sum": 12.045347375023994, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/future_approval/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 7, + "projected_status_correct": 12, + "raw_fully_correct_reports": 7, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 26632, + "known_completion_tokens": 490, + "known_cost_microinr": 428880, + "retained_microinr": 536103, + "elapsed_seconds_sum": 12.964495208027074, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/missing_replacement/chronological": { + "n": 12, + "factual_correct": 10, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 2, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 10, + "known_prompt_tokens": 12115, + "known_completion_tokens": 404, + "known_cost_microinr": 205965, + "retained_microinr": 257460, + "elapsed_seconds_sum": 12.17967891698936, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/missing_replacement/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 18168, + "known_completion_tokens": 384, + "known_cost_microinr": 295560, + "retained_microinr": 369453, + "elapsed_seconds_sum": 11.797906836029142, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/missing_replacement/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 19182, + "known_completion_tokens": 384, + "known_cost_microinr": 310770, + "retained_microinr": 388467, + "elapsed_seconds_sum": 11.954999916008092, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/request_correction/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 11, + "projected_status_correct": 12, + "raw_fully_correct_reports": 11, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 11335, + "known_completion_tokens": 504, + "known_cost_microinr": 200265, + "retained_microinr": 250337, + "elapsed_seconds_sum": 11.854606876018806, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/request_correction/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 8, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 4, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 8, + "projected_fully_correct_reports": 8, + "known_prompt_tokens": 17168, + "known_completion_tokens": 464, + "known_cost_microinr": 285360, + "retained_microinr": 356704, + "elapsed_seconds_sum": 13.21544787401217, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/request_correction/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 17858, + "known_completion_tokens": 504, + "known_cost_microinr": 298110, + "retained_microinr": 372643, + "elapsed_seconds_sum": 12.309500044008018, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/resolved_condition/chronological": { + "n": 12, + "factual_correct": 10, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 2, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 10, + "known_prompt_tokens": 13860, + "known_completion_tokens": 484, + "known_cost_microinr": 236940, + "retained_microinr": 296178, + "elapsed_seconds_sum": 12.080073124001501, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/resolved_condition/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 8, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 4, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 8, + "projected_fully_correct_reports": 8, + "known_prompt_tokens": 19144, + "known_completion_tokens": 464, + "known_cost_microinr": 315000, + "retained_microinr": 393753, + "elapsed_seconds_sum": 11.855004083001404, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/resolved_condition/grouped_lossless": { + "n": 12, + "factual_correct": 10, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 2, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 10, + "known_prompt_tokens": 21354, + "known_completion_tokens": 484, + "known_cost_microinr": 349350, + "retained_microinr": 436692, + "elapsed_seconds_sum": 12.498722624019138, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/separate_booking/chronological": { + "n": 12, + "factual_correct": 0, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 12, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 11, + "projected_status_correct": 12, + "raw_fully_correct_reports": 0, + "projected_fully_correct_reports": 0, + "known_prompt_tokens": 12448, + "known_completion_tokens": 504, + "known_cost_microinr": 216960, + "retained_microinr": 271203, + "elapsed_seconds_sum": 11.991847333003534, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/separate_booking/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 5, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 6, + "false_abstentions": 1, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 5, + "projected_fully_correct_reports": 5, + "known_prompt_tokens": 18898, + "known_completion_tokens": 494, + "known_cost_microinr": 313110, + "retained_microinr": 391393, + "elapsed_seconds_sum": 11.78994375099137, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/separate_booking/grouped_lossless": { + "n": 12, + "factual_correct": 0, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 12, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 9, + "projected_status_correct": 12, + "raw_fully_correct_reports": 0, + "projected_fully_correct_reports": 0, + "known_prompt_tokens": 19707, + "known_completion_tokens": 502, + "known_cost_microinr": 325725, + "retained_microinr": 407161, + "elapsed_seconds_sum": 12.05981566599803, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/unresolved_condition/chronological": { + "n": 12, + "factual_correct": 4, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 8, + "false_abstentions": 0, + "unknown_condition_guesses": 8, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 4, + "projected_fully_correct_reports": 4, + "known_prompt_tokens": 13526, + "known_completion_tokens": 464, + "known_cost_microinr": 230730, + "retained_microinr": 288418, + "elapsed_seconds_sum": 12.068003666005097, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/unresolved_condition/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 10, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 2, + "false_abstentions": 0, + "unknown_condition_guesses": 2, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 10, + "known_prompt_tokens": 18372, + "known_completion_tokens": 404, + "known_cost_microinr": 299820, + "retained_microinr": 374780, + "elapsed_seconds_sum": 13.76512233397807, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/unresolved_condition/grouped_lossless": { + "n": 12, + "factual_correct": 6, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 6, + "false_abstentions": 0, + "unknown_condition_guesses": 6, + "raw_status_correct": 11, + "projected_status_correct": 12, + "raw_fully_correct_reports": 5, + "projected_fully_correct_reports": 6, + "known_prompt_tokens": 20613, + "known_completion_tokens": 444, + "known_cost_microinr": 335835, + "retained_microinr": 419798, + "elapsed_seconds_sum": 12.204635876027169, + "elapsed_seconds_count": 12 + } + }, + "treated_actor_prose/arm": { + "False/chronological": { + "n": 56, + "factual_correct": 56, + "valid_final": 56, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 55, + "projected_status_correct": 56, + "raw_fully_correct_reports": 55, + "projected_fully_correct_reports": 56, + "known_prompt_tokens": 79885, + "known_completion_tokens": 35789, + "known_cost_microinr": 3345615, + "retained_microinr": 4182040, + "elapsed_seconds_sum": 198.88611220597522, + "elapsed_seconds_count": 56 + }, + "False/grouped_actor_prose_withheld": { + "n": 56, + "factual_correct": 52, + "valid_final": 52, + "invalid_or_missing_final": 4, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 44, + "projected_status_correct": 52, + "raw_fully_correct_reports": 44, + "projected_fully_correct_reports": 52, + "known_prompt_tokens": 116379, + "known_completion_tokens": 95774, + "known_cost_microinr": 7492125, + "retained_microinr": 9365178, + "elapsed_seconds_sum": 472.5217678300396, + "elapsed_seconds_count": 56 + }, + "False/grouped_lossless": { + "n": 56, + "factual_correct": 54, + "valid_final": 54, + "invalid_or_missing_final": 2, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 50, + "projected_status_correct": 54, + "raw_fully_correct_reports": 50, + "projected_fully_correct_reports": 54, + "known_prompt_tokens": 116379, + "known_completion_tokens": 95497, + "known_cost_microinr": 7475505, + "retained_microinr": 9344403, + "elapsed_seconds_sum": 479.82695345905086, + "elapsed_seconds_count": 56 + }, + "True/chronological": { + "n": 184, + "factual_correct": 144, + "valid_final": 179, + "invalid_or_missing_final": 5, + "wrong_non_null_dates": 25, + "false_abstentions": 2, + "unknown_condition_guesses": 8, + "raw_status_correct": 131, + "projected_status_correct": 179, + "raw_fully_correct_reports": 101, + "projected_fully_correct_reports": 144, + "known_prompt_tokens": 231698, + "known_completion_tokens": 362173, + "known_cost_microinr": 25205850, + "retained_microinr": 31507381, + "elapsed_seconds_sum": 1747.813680294028, + "elapsed_seconds_count": 184 + }, + "True/grouped_actor_prose_withheld": { + "n": 184, + "factual_correct": 156, + "valid_final": 182, + "invalid_or_missing_final": 2, + "wrong_non_null_dates": 9, + "false_abstentions": 11, + "unknown_condition_guesses": 2, + "raw_status_correct": 138, + "projected_status_correct": 182, + "raw_fully_correct_reports": 114, + "projected_fully_correct_reports": 156, + "known_prompt_tokens": 307124, + "known_completion_tokens": 324726, + "known_cost_microinr": 24090420, + "retained_microinr": 30113091, + "elapsed_seconds_sum": 1608.5641960811336, + "elapsed_seconds_count": 184 + }, + "True/grouped_lossless": { + "n": 184, + "factual_correct": 145, + "valid_final": 178, + "invalid_or_missing_final": 6, + "wrong_non_null_dates": 22, + "false_abstentions": 3, + "unknown_condition_guesses": 8, + "raw_status_correct": 128, + "projected_status_correct": 178, + "raw_fully_correct_reports": 99, + "projected_fully_correct_reports": 145, + "known_prompt_tokens": 335275, + "known_completion_tokens": 354297, + "known_cost_microinr": 26286945, + "retained_microinr": 32858750, + "elapsed_seconds_sum": 1734.1608937870333, + "elapsed_seconds_count": 184 + } + } + }, + "contrasts": { + "A_to_B": { + "all/all": { + "n": 240, + "left_correct": 200, + "right_correct": 199, + "gain": 12, + "loss": 13, + "net": -1, + "restored_from_invalid": 4, + "corrected_usable_answer": 8, + "workflow_net": { + "fresh-01": 0, + "fresh-02": 0, + "fresh-03": 0, + "fresh-04": 0, + "fresh-05": -1, + "fresh-06": 2, + "fresh-07": 0, + "fresh-08": 0, + "fresh-09": -1, + "fresh-10": 0, + "fresh-11": 1, + "fresh-12": 0, + "fresh-13": 1, + "fresh-14": 1, + "fresh-15": 0, + "fresh-16": 1, + "fresh-17": -1, + "fresh-18": 1, + "fresh-19": 0, + "fresh-20": -2, + "fresh-21": 0, + "fresh-22": -1, + "fresh-23": -1, + "fresh-24": 0, + "fresh-25": 0, + "fresh-26": -1, + "fresh-27": 0, + "fresh-28": 1, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": 0, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": 0, + "fresh-38": 0, + "fresh-39": -1, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.041666666666666664, + 0.03333333333333333 + ], + "percentile_95_percentage_points": [ + -4.166666666666666, + 3.3333333333333335 + ] + } + }, + "all/sarvam-105b": { + "n": 120, + "left_correct": 104, + "right_correct": 99, + "gain": 7, + "loss": 12, + "net": -5, + "restored_from_invalid": 4, + "corrected_usable_answer": 3, + "workflow_net": { + "fresh-01": 0, + "fresh-02": 0, + "fresh-03": 0, + "fresh-04": 0, + "fresh-05": -1, + "fresh-06": 1, + "fresh-07": 0, + "fresh-08": -1, + "fresh-09": 0, + "fresh-10": 0, + "fresh-11": 0, + "fresh-12": 0, + "fresh-13": -1, + "fresh-14": 1, + "fresh-15": 0, + "fresh-16": 1, + "fresh-17": -1, + "fresh-18": 1, + "fresh-19": 0, + "fresh-20": -2, + "fresh-21": 0, + "fresh-22": -1, + "fresh-23": -1, + "fresh-24": 0, + "fresh-25": 0, + "fresh-26": -1, + "fresh-27": 0, + "fresh-28": 1, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": 0, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": 0, + "fresh-38": 0, + "fresh-39": -1, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.10833333333333334, + 0.025 + ], + "percentile_95_percentage_points": [ + -10.833333333333334, + 2.5 + ] + } + }, + "all/sarvam-105b-conversations": { + "n": 120, + "left_correct": 96, + "right_correct": 100, + "gain": 5, + "loss": 1, + "net": 4, + "restored_from_invalid": 0, + "corrected_usable_answer": 5, + "workflow_net": { + "fresh-01": 0, + "fresh-02": 0, + "fresh-03": 0, + "fresh-04": 0, + "fresh-05": 0, + "fresh-06": 1, + "fresh-07": 0, + "fresh-08": 1, + "fresh-09": -1, + "fresh-10": 0, + "fresh-11": 1, + "fresh-12": 0, + "fresh-13": 2, + "fresh-14": 0, + "fresh-15": 0, + "fresh-16": 0, + "fresh-17": 0, + "fresh-18": 0, + "fresh-19": 0, + "fresh-20": 0, + "fresh-21": 0, + "fresh-22": 0, + "fresh-23": 0, + "fresh-24": 0, + "fresh-25": 0, + "fresh-26": 0, + "fresh-27": 0, + "fresh-28": 0, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": 0, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": 0, + "fresh-38": 0, + "fresh-39": 0, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.008333333333333333, + 0.08333333333333333 + ], + "percentile_95_percentage_points": [ + -0.8333333333333334, + 8.333333333333332 + ] + } + }, + "treated/all": { + "n": 184, + "left_correct": 144, + "right_correct": 145, + "gain": 12, + "loss": 11, + "net": 1, + "restored_from_invalid": 4, + "corrected_usable_answer": 8, + "workflow_net": { + "fresh-01": 0, + "fresh-02": 0, + "fresh-03": 0, + "fresh-04": 0, + "fresh-05": -1, + "fresh-06": 2, + "fresh-07": 0, + "fresh-08": 0, + "fresh-09": -1, + "fresh-10": 0, + "fresh-11": 1, + "fresh-12": 0, + "fresh-13": 1, + "fresh-14": 1, + "fresh-15": 0, + "fresh-16": 1, + "fresh-17": -1, + "fresh-18": 1, + "fresh-19": 0, + "fresh-20": -2, + "fresh-22": -1, + "fresh-27": 0, + "fresh-28": 1, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": 0, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": 0, + "fresh-38": 0, + "fresh-39": -1, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.042105263157894736, + 0.05102040816326531 + ], + "percentile_95_percentage_points": [ + -4.2105263157894735, + 5.1020408163265305 + ] + } + }, + "treated/sarvam-105b": { + "n": 98, + "left_correct": 82, + "right_correct": 79, + "gain": 7, + "loss": 10, + "net": -3, + "restored_from_invalid": 4, + "corrected_usable_answer": 3, + "workflow_net": { + "fresh-01": 0, + "fresh-02": 0, + "fresh-03": 0, + "fresh-04": 0, + "fresh-05": -1, + "fresh-06": 1, + "fresh-07": 0, + "fresh-08": -1, + "fresh-09": 0, + "fresh-10": 0, + "fresh-11": 0, + "fresh-12": 0, + "fresh-13": -1, + "fresh-14": 1, + "fresh-15": 0, + "fresh-16": 1, + "fresh-17": -1, + "fresh-18": 1, + "fresh-19": 0, + "fresh-20": -2, + "fresh-22": -1, + "fresh-27": 0, + "fresh-28": 1, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": 0, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": 0, + "fresh-38": 0, + "fresh-39": -1, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.1111111111111111, + 0.0449438202247191 + ], + "percentile_95_percentage_points": [ + -11.11111111111111, + 4.49438202247191 + ] + } + }, + "treated/sarvam-105b-conversations": { + "n": 86, + "left_correct": 62, + "right_correct": 66, + "gain": 5, + "loss": 1, + "net": 4, + "restored_from_invalid": 0, + "corrected_usable_answer": 5, + "workflow_net": { + "fresh-01": 0, + "fresh-02": 0, + "fresh-03": 0, + "fresh-04": 0, + "fresh-05": 0, + "fresh-06": 1, + "fresh-07": 0, + "fresh-08": 1, + "fresh-09": -1, + "fresh-10": 0, + "fresh-11": 1, + "fresh-12": 0, + "fresh-13": 2, + "fresh-14": 0, + "fresh-15": 0, + "fresh-16": 0, + "fresh-17": 0, + "fresh-18": 0, + "fresh-19": 0, + "fresh-20": 0, + "fresh-28": 0, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": 0, + "fresh-38": 0, + "fresh-39": 0, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.011494252873563218, + 0.1125 + ], + "percentile_95_percentage_points": [ + -1.1494252873563218, + 11.25 + ] + } + }, + "no_treatment/all": { + "n": 56, + "left_correct": 56, + "right_correct": 54, + "gain": 0, + "loss": 2, + "net": -2, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "workflow_net": { + "fresh-13": 0, + "fresh-21": 0, + "fresh-22": 0, + "fresh-23": -1, + "fresh-24": 0, + "fresh-25": 0, + "fresh-26": -1, + "fresh-27": 0, + "fresh-28": 0, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": 0, + "fresh-32": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.08163265306122448, + 0.0 + ], + "percentile_95_percentage_points": [ + -8.16326530612245, + 0.0 + ] + } + }, + "no_treatment/sarvam-105b": { + "n": 22, + "left_correct": 22, + "right_correct": 20, + "gain": 0, + "loss": 2, + "net": -2, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "workflow_net": { + "fresh-13": 0, + "fresh-21": 0, + "fresh-22": 0, + "fresh-23": -1, + "fresh-24": 0, + "fresh-25": 0, + "fresh-26": -1, + "fresh-27": 0, + "fresh-28": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.2, + 0.0 + ], + "percentile_95_percentage_points": [ + -20.0, + 0.0 + ] + } + }, + "no_treatment/sarvam-105b-conversations": { + "n": 34, + "left_correct": 34, + "right_correct": 34, + "gain": 0, + "loss": 0, + "net": 0, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "workflow_net": { + "fresh-21": 0, + "fresh-22": 0, + "fresh-23": 0, + "fresh-24": 0, + "fresh-25": 0, + "fresh-26": 0, + "fresh-27": 0, + "fresh-28": 0, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": 0, + "fresh-32": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + 0.0, + 0.0 + ], + "percentile_95_percentage_points": [ + 0.0, + 0.0 + ] + } + } + }, + "B_to_C": { + "all/all": { + "n": 240, + "left_correct": 199, + "right_correct": 208, + "gain": 27, + "loss": 18, + "net": 9, + "restored_from_invalid": 6, + "corrected_usable_answer": 21, + "workflow_net": { + "fresh-01": -2, + "fresh-02": -1, + "fresh-03": -1, + "fresh-04": -1, + "fresh-05": 2, + "fresh-06": 1, + "fresh-07": -1, + "fresh-08": 1, + "fresh-09": 0, + "fresh-10": -2, + "fresh-11": -1, + "fresh-12": 0, + "fresh-13": 2, + "fresh-14": 1, + "fresh-15": 1, + "fresh-16": 2, + "fresh-17": 3, + "fresh-18": 2, + "fresh-19": 1, + "fresh-20": 4, + "fresh-21": 0, + "fresh-22": 1, + "fresh-23": 0, + "fresh-24": 0, + "fresh-25": -1, + "fresh-26": 0, + "fresh-27": -1, + "fresh-28": 0, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": -1, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": -1, + "fresh-38": 0, + "fresh-39": 1, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.025, + 0.10416666666666667 + ], + "percentile_95_percentage_points": [ + -2.5, + 10.416666666666668 + ] + } + }, + "all/sarvam-105b": { + "n": 120, + "left_correct": 99, + "right_correct": 105, + "gain": 15, + "loss": 9, + "net": 6, + "restored_from_invalid": 6, + "corrected_usable_answer": 9, + "workflow_net": { + "fresh-01": -1, + "fresh-02": 0, + "fresh-03": 0, + "fresh-04": 0, + "fresh-05": 2, + "fresh-06": 1, + "fresh-07": -1, + "fresh-08": 1, + "fresh-09": 0, + "fresh-10": -1, + "fresh-11": 0, + "fresh-12": 0, + "fresh-13": 2, + "fresh-14": 0, + "fresh-15": 0, + "fresh-16": 0, + "fresh-17": 1, + "fresh-18": 1, + "fresh-19": 0, + "fresh-20": 3, + "fresh-21": 0, + "fresh-22": 1, + "fresh-23": 0, + "fresh-24": 0, + "fresh-25": -1, + "fresh-26": 0, + "fresh-27": -1, + "fresh-28": 0, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": -1, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": -1, + "fresh-38": 0, + "fresh-39": 1, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.03333333333333333, + 0.14166666666666666 + ], + "percentile_95_percentage_points": [ + -3.3333333333333335, + 14.166666666666666 + ] + } + }, + "all/sarvam-105b-conversations": { + "n": 120, + "left_correct": 100, + "right_correct": 103, + "gain": 12, + "loss": 9, + "net": 3, + "restored_from_invalid": 0, + "corrected_usable_answer": 12, + "workflow_net": { + "fresh-01": -1, + "fresh-02": -1, + "fresh-03": -1, + "fresh-04": -1, + "fresh-05": 0, + "fresh-06": 0, + "fresh-07": 0, + "fresh-08": 0, + "fresh-09": 0, + "fresh-10": -1, + "fresh-11": -1, + "fresh-12": 0, + "fresh-13": 0, + "fresh-14": 1, + "fresh-15": 1, + "fresh-16": 2, + "fresh-17": 2, + "fresh-18": 1, + "fresh-19": 1, + "fresh-20": 1, + "fresh-21": 0, + "fresh-22": 0, + "fresh-23": 0, + "fresh-24": 0, + "fresh-25": 0, + "fresh-26": 0, + "fresh-27": 0, + "fresh-28": 0, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": 0, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": 0, + "fresh-38": 0, + "fresh-39": 0, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.041666666666666664, + 0.1 + ], + "percentile_95_percentage_points": [ + -4.166666666666666, + 10.0 + ] + } + }, + "treated/all": { + "n": 184, + "left_correct": 145, + "right_correct": 156, + "gain": 26, + "loss": 15, + "net": 11, + "restored_from_invalid": 5, + "corrected_usable_answer": 21, + "workflow_net": { + "fresh-01": -2, + "fresh-02": -1, + "fresh-03": -1, + "fresh-04": -1, + "fresh-05": 2, + "fresh-06": 1, + "fresh-07": -1, + "fresh-08": 1, + "fresh-09": 0, + "fresh-10": -2, + "fresh-11": -1, + "fresh-12": 0, + "fresh-13": 2, + "fresh-14": 1, + "fresh-15": 1, + "fresh-16": 2, + "fresh-17": 3, + "fresh-18": 2, + "fresh-19": 1, + "fresh-20": 4, + "fresh-22": 1, + "fresh-27": 0, + "fresh-28": 0, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": -1, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": -1, + "fresh-38": 0, + "fresh-39": 1, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.020942408376963352, + 0.1452513966480447 + ], + "percentile_95_percentage_points": [ + -2.094240837696335, + 14.52513966480447 + ] + } + }, + "treated/sarvam-105b": { + "n": 98, + "left_correct": 79, + "right_correct": 87, + "gain": 14, + "loss": 6, + "net": 8, + "restored_from_invalid": 5, + "corrected_usable_answer": 9, + "workflow_net": { + "fresh-01": -1, + "fresh-02": 0, + "fresh-03": 0, + "fresh-04": 0, + "fresh-05": 2, + "fresh-06": 1, + "fresh-07": -1, + "fresh-08": 1, + "fresh-09": 0, + "fresh-10": -1, + "fresh-11": 0, + "fresh-12": 0, + "fresh-13": 2, + "fresh-14": 0, + "fresh-15": 0, + "fresh-16": 0, + "fresh-17": 1, + "fresh-18": 1, + "fresh-19": 0, + "fresh-20": 3, + "fresh-22": 1, + "fresh-27": 0, + "fresh-28": 0, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": -1, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": -1, + "fresh-38": 0, + "fresh-39": 1, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.018867924528301886, + 0.19148936170212766 + ], + "percentile_95_percentage_points": [ + -1.8867924528301887, + 19.148936170212767 + ] + } + }, + "treated/sarvam-105b-conversations": { + "n": 86, + "left_correct": 66, + "right_correct": 69, + "gain": 12, + "loss": 9, + "net": 3, + "restored_from_invalid": 0, + "corrected_usable_answer": 12, + "workflow_net": { + "fresh-01": -1, + "fresh-02": -1, + "fresh-03": -1, + "fresh-04": -1, + "fresh-05": 0, + "fresh-06": 0, + "fresh-07": 0, + "fresh-08": 0, + "fresh-09": 0, + "fresh-10": -1, + "fresh-11": -1, + "fresh-12": 0, + "fresh-13": 0, + "fresh-14": 1, + "fresh-15": 1, + "fresh-16": 2, + "fresh-17": 2, + "fresh-18": 1, + "fresh-19": 1, + "fresh-20": 1, + "fresh-28": 0, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": 0, + "fresh-38": 0, + "fresh-39": 0, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.05952380952380952, + 0.13636363636363635 + ], + "percentile_95_percentage_points": [ + -5.952380952380952, + 13.636363636363635 + ] + } + }, + "no_treatment/all": { + "n": 56, + "left_correct": 54, + "right_correct": 52, + "gain": 1, + "loss": 3, + "net": -2, + "restored_from_invalid": 1, + "corrected_usable_answer": 0, + "workflow_net": { + "fresh-13": 0, + "fresh-21": 0, + "fresh-22": 0, + "fresh-23": 0, + "fresh-24": 0, + "fresh-25": -1, + "fresh-26": 0, + "fresh-27": -1, + "fresh-28": 0, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": 0, + "fresh-32": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.0847457627118644, + 0.0 + ], + "percentile_95_percentage_points": [ + -8.47457627118644, + 0.0 + ] + } + }, + "no_treatment/sarvam-105b": { + "n": 22, + "left_correct": 20, + "right_correct": 18, + "gain": 1, + "loss": 3, + "net": -2, + "restored_from_invalid": 1, + "corrected_usable_answer": 0, + "workflow_net": { + "fresh-13": 0, + "fresh-21": 0, + "fresh-22": 0, + "fresh-23": 0, + "fresh-24": 0, + "fresh-25": -1, + "fresh-26": 0, + "fresh-27": -1, + "fresh-28": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.2222222222222222, + 0.0 + ], + "percentile_95_percentage_points": [ + -22.22222222222222, + 0.0 + ] + } + }, + "no_treatment/sarvam-105b-conversations": { + "n": 34, + "left_correct": 34, + "right_correct": 34, + "gain": 0, + "loss": 0, + "net": 0, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "workflow_net": { + "fresh-21": 0, + "fresh-22": 0, + "fresh-23": 0, + "fresh-24": 0, + "fresh-25": 0, + "fresh-26": 0, + "fresh-27": 0, + "fresh-28": 0, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": 0, + "fresh-32": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + 0.0, + 0.0 + ], + "percentile_95_percentage_points": [ + 0.0, + 0.0 + ] + } + } + }, + "A_to_C": { + "all/all": { + "n": 240, + "left_correct": 200, + "right_correct": 208, + "gain": 25, + "loss": 17, + "net": 8, + "restored_from_invalid": 5, + "corrected_usable_answer": 20, + "workflow_net": { + "fresh-01": -2, + "fresh-02": -1, + "fresh-03": -1, + "fresh-04": -1, + "fresh-05": 1, + "fresh-06": 3, + "fresh-07": -1, + "fresh-08": 1, + "fresh-09": -1, + "fresh-10": -2, + "fresh-11": 0, + "fresh-12": 0, + "fresh-13": 3, + "fresh-14": 2, + "fresh-15": 1, + "fresh-16": 3, + "fresh-17": 2, + "fresh-18": 3, + "fresh-19": 1, + "fresh-20": 2, + "fresh-21": 0, + "fresh-22": 0, + "fresh-23": -1, + "fresh-24": 0, + "fresh-25": -1, + "fresh-26": -1, + "fresh-27": -1, + "fresh-28": 1, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": -1, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": -1, + "fresh-38": 0, + "fresh-39": 0, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.03333333333333333, + 0.1 + ], + "percentile_95_percentage_points": [ + -3.3333333333333335, + 10.0 + ] + } + }, + "all/sarvam-105b": { + "n": 120, + "left_correct": 104, + "right_correct": 105, + "gain": 11, + "loss": 10, + "net": 1, + "restored_from_invalid": 5, + "corrected_usable_answer": 6, + "workflow_net": { + "fresh-01": -1, + "fresh-02": 0, + "fresh-03": 0, + "fresh-04": 0, + "fresh-05": 1, + "fresh-06": 2, + "fresh-07": -1, + "fresh-08": 0, + "fresh-09": 0, + "fresh-10": -1, + "fresh-11": 0, + "fresh-12": 0, + "fresh-13": 1, + "fresh-14": 1, + "fresh-15": 0, + "fresh-16": 1, + "fresh-17": 0, + "fresh-18": 2, + "fresh-19": 0, + "fresh-20": 1, + "fresh-21": 0, + "fresh-22": 0, + "fresh-23": -1, + "fresh-24": 0, + "fresh-25": -1, + "fresh-26": -1, + "fresh-27": -1, + "fresh-28": 1, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": -1, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": -1, + "fresh-38": 0, + "fresh-39": 0, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.06666666666666667, + 0.08333333333333333 + ], + "percentile_95_percentage_points": [ + -6.666666666666667, + 8.333333333333332 + ] + } + }, + "all/sarvam-105b-conversations": { + "n": 120, + "left_correct": 96, + "right_correct": 103, + "gain": 14, + "loss": 7, + "net": 7, + "restored_from_invalid": 0, + "corrected_usable_answer": 14, + "workflow_net": { + "fresh-01": -1, + "fresh-02": -1, + "fresh-03": -1, + "fresh-04": -1, + "fresh-05": 0, + "fresh-06": 1, + "fresh-07": 0, + "fresh-08": 1, + "fresh-09": -1, + "fresh-10": -1, + "fresh-11": 0, + "fresh-12": 0, + "fresh-13": 2, + "fresh-14": 1, + "fresh-15": 1, + "fresh-16": 2, + "fresh-17": 2, + "fresh-18": 1, + "fresh-19": 1, + "fresh-20": 1, + "fresh-21": 0, + "fresh-22": 0, + "fresh-23": 0, + "fresh-24": 0, + "fresh-25": 0, + "fresh-26": 0, + "fresh-27": 0, + "fresh-28": 0, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": 0, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": 0, + "fresh-38": 0, + "fresh-39": 0, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.016666666666666666, + 0.14166666666666666 + ], + "percentile_95_percentage_points": [ + -1.6666666666666667, + 14.166666666666666 + ] + } + }, + "treated/all": { + "n": 184, + "left_correct": 144, + "right_correct": 156, + "gain": 25, + "loss": 13, + "net": 12, + "restored_from_invalid": 5, + "corrected_usable_answer": 20, + "workflow_net": { + "fresh-01": -2, + "fresh-02": -1, + "fresh-03": -1, + "fresh-04": -1, + "fresh-05": 1, + "fresh-06": 3, + "fresh-07": -1, + "fresh-08": 1, + "fresh-09": -1, + "fresh-10": -2, + "fresh-11": 0, + "fresh-12": 0, + "fresh-13": 3, + "fresh-14": 2, + "fresh-15": 1, + "fresh-16": 3, + "fresh-17": 2, + "fresh-18": 3, + "fresh-19": 1, + "fresh-20": 2, + "fresh-22": 0, + "fresh-27": 0, + "fresh-28": 1, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": -1, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": -1, + "fresh-38": 0, + "fresh-39": 0, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.017341040462427744, + 0.1518324607329843 + ], + "percentile_95_percentage_points": [ + -1.7341040462427744, + 15.18324607329843 + ] + } + }, + "treated/sarvam-105b": { + "n": 98, + "left_correct": 82, + "right_correct": 87, + "gain": 11, + "loss": 6, + "net": 5, + "restored_from_invalid": 5, + "corrected_usable_answer": 6, + "workflow_net": { + "fresh-01": -1, + "fresh-02": 0, + "fresh-03": 0, + "fresh-04": 0, + "fresh-05": 1, + "fresh-06": 2, + "fresh-07": -1, + "fresh-08": 0, + "fresh-09": 0, + "fresh-10": -1, + "fresh-11": 0, + "fresh-12": 0, + "fresh-13": 1, + "fresh-14": 1, + "fresh-15": 0, + "fresh-16": 1, + "fresh-17": 0, + "fresh-18": 2, + "fresh-19": 0, + "fresh-20": 1, + "fresh-22": 0, + "fresh-27": 0, + "fresh-28": 1, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": -1, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": -1, + "fresh-38": 0, + "fresh-39": 0, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.030303030303030304, + 0.13978494623655913 + ], + "percentile_95_percentage_points": [ + -3.0303030303030303, + 13.978494623655912 + ] + } + }, + "treated/sarvam-105b-conversations": { + "n": 86, + "left_correct": 62, + "right_correct": 69, + "gain": 14, + "loss": 7, + "net": 7, + "restored_from_invalid": 0, + "corrected_usable_answer": 14, + "workflow_net": { + "fresh-01": -1, + "fresh-02": -1, + "fresh-03": -1, + "fresh-04": -1, + "fresh-05": 0, + "fresh-06": 1, + "fresh-07": 0, + "fresh-08": 1, + "fresh-09": -1, + "fresh-10": -1, + "fresh-11": 0, + "fresh-12": 0, + "fresh-13": 2, + "fresh-14": 1, + "fresh-15": 1, + "fresh-16": 2, + "fresh-17": 2, + "fresh-18": 1, + "fresh-19": 1, + "fresh-20": 1, + "fresh-28": 0, + "fresh-32": 0, + "fresh-33": 0, + "fresh-34": 0, + "fresh-35": 0, + "fresh-36": 0, + "fresh-37": 0, + "fresh-38": 0, + "fresh-39": 0, + "fresh-40": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.025, + 0.19117647058823528 + ], + "percentile_95_percentage_points": [ + -2.5, + 19.11764705882353 + ] + } + }, + "no_treatment/all": { + "n": 56, + "left_correct": 56, + "right_correct": 52, + "gain": 0, + "loss": 4, + "net": -4, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "workflow_net": { + "fresh-13": 0, + "fresh-21": 0, + "fresh-22": 0, + "fresh-23": -1, + "fresh-24": 0, + "fresh-25": -1, + "fresh-26": -1, + "fresh-27": -1, + "fresh-28": 0, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": 0, + "fresh-32": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.12244897959183673, + -0.019230769230769232 + ], + "percentile_95_percentage_points": [ + -12.244897959183673, + -1.9230769230769231 + ] + } + }, + "no_treatment/sarvam-105b": { + "n": 22, + "left_correct": 22, + "right_correct": 18, + "gain": 0, + "loss": 4, + "net": -4, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "workflow_net": { + "fresh-13": 0, + "fresh-21": 0, + "fresh-22": 0, + "fresh-23": -1, + "fresh-24": 0, + "fresh-25": -1, + "fresh-26": -1, + "fresh-27": -1, + "fresh-28": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.3076923076923077, + -0.05 + ], + "percentile_95_percentage_points": [ + -30.76923076923077, + -5.0 + ] + } + }, + "no_treatment/sarvam-105b-conversations": { + "n": 34, + "left_correct": 34, + "right_correct": 34, + "gain": 0, + "loss": 0, + "net": 0, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "workflow_net": { + "fresh-21": 0, + "fresh-22": 0, + "fresh-23": 0, + "fresh-24": 0, + "fresh-25": 0, + "fresh-26": 0, + "fresh-27": 0, + "fresh-28": 0, + "fresh-29": 0, + "fresh-30": 0, + "fresh-31": 0, + "fresh-32": 0 + }, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + 0.0, + 0.0 + ], + "percentile_95_percentage_points": [ + 0.0, + 0.0 + ] + } + } + } + }, + "screen_gate": { + "checks": { + "all_720_clean": true, + "production_default_transport": true, + "treated_B_to_C_positive/sarvam-105b": true, + "two_positive_workflows/sarvam-105b": true, + "all_C_to_A_positive/sarvam-105b": true, + "C_no_higher_wrong_non_null_dates/sarvam-105b": true, + "C_no_higher_false_abstentions/sarvam-105b": false, + "C_no_lower_raw_status_correct/sarvam-105b": true, + "C_no_lower_projected_status_correct/sarvam-105b": false, + "treated_B_to_C_positive/sarvam-105b-conversations": true, + "two_positive_workflows/sarvam-105b-conversations": true, + "all_C_to_A_positive/sarvam-105b-conversations": true, + "C_no_higher_wrong_non_null_dates/sarvam-105b-conversations": true, + "C_no_higher_false_abstentions/sarvam-105b-conversations": false, + "C_no_lower_raw_status_correct/sarvam-105b-conversations": false, + "C_no_lower_projected_status_correct/sarvam-105b-conversations": true + }, + "mechanical_pass_pending_semantic_review": false, + "semantic_review_complete": false, + "forward_gate_pass": null + }, + "scenario_clusters": 40, + "treated_histories": 184, + "no_treatment_histories": 56, + "development_exposed": true, + "independent_native_review": false +} diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/clean-export-verification.json b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/clean-export-verification.json new file mode 100644 index 0000000..a897e1b --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/clean-export-verification.json @@ -0,0 +1,18 @@ +{ + "status": "CLEAN_GIT_EXPORT_AND_PUBLIC_ARCHIVE_REPLAY_PASS", + "commit": "64b0ce134836e9d418e7141dc52a47dcb46b3af8", + "archive_sha256": "ff5f55ef34b46ad6be496bfc49470fca84a412ea376631dd77a2ba0850519f90", + "artifact_files_verified": 3459, + "primary_and_independent_audits_exact_match": true, + "paired_comparison_exact_match": true, + "all720_semantic_review_bindings_match": true, + "semantic_replay_timestamp_excluded": true, + "focused_portable_tests_passed": 5, + "local_parent_fixture_test_deselected": 1, + "prior_local_preflight_tests_passed": 246, + "ruff_pass": true, + "api_calls": 0, + "credential_read": false, + "export_path": "/var/folders/jd/4kftgc9x66nbyt0s7fgzf2700000gn/T/sarvam-source-clean-ljvnh7qo", + "verification_wrapper_correction": "Initial semantic-check invocation omitted --output and failed argument parsing. Preserved initial log; corrected invocation passed. Source, raw data and both preceding audits unchanged." +} diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/comparison-verification.json b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/comparison-verification.json new file mode 100644 index 0000000..43694c2 --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/comparison-verification.json @@ -0,0 +1,10 @@ +{ + "status": "TWO_RAW_AUDITS_AGREE", + "planned_rows_compared": 720, + "fields_per_row": 11, + "paired_model_slice_contrasts_compared": 18, + "cost_and_local_release_order_match": true, + "primary_sha256": "8c208f56157cb753b0965d5b88d350ef2ee181dbb4ce5646d34e3ca36a849cc0", + "independent_sha256": "7e4c331123f726895c37ffbb527db591a77f62e8c5db5256e5c487d255e015fc", + "semantic_review_is_separate": true +} diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/decision.json b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/decision.json new file mode 100644 index 0000000..d820d0f --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/decision.json @@ -0,0 +1,40 @@ +{ + "status": "COMPLETED_REJECT_BLANKET_PROSE_OMISSION", + "semantic_review_complete": true, + "independent_human_review": false, + "forward_gate_pass": false, + "failed_checks": [ + "C_no_higher_false_abstentions/sarvam-105b", + "C_no_lower_projected_status_correct/sarvam-105b", + "C_no_higher_false_abstentions/sarvam-105b-conversations", + "C_no_lower_raw_status_correct/sarvam-105b-conversations" + ], + "new_calls": 720, + "new_native_action_trials": 0, + "known_microinr": 93896460, + "retained_microinr": 117370843, + "cumulative_calls": 7757, + "cumulative_known_microinr": 365121090, + "cumulative_retained_microinr": 488660182, + "source_commit": "751e82035dde3275c94fce767d26826f00567f92", + "evidence_sha256": { + "PROTOCOL.md": "7efe12932f4291bc9137a8539cbb6edb85359f9d099bb65e3026a5f4c7832933", + "manifest.json": "ba66788d25b439471ba396c8aa2f56a1b4f136cd60cdbf0a04a40e2484bbf650", + "audit.json": "8c208f56157cb753b0965d5b88d350ef2ee181dbb4ce5646d34e3ca36a849cc0", + "independent-aggregation.json": "7e4c331123f726895c37ffbb527db591a77f62e8c5db5256e5c487d255e015fc", + "comparison-verification.json": "8cbc7760b5fea92d79ee44a7ff363d3c0cb39aed2f913c002762dac4dabf6e8f", + "review-verification.json": "d2dee9c28fe126b9b39ffd09c9678023e632b957a776367b246a02ff8fc29c76", + "REPORT.md": "84f5e1a4363edcbd46afd2b38c53a33c72bd27a626851d1463af1320d4ad7baa", + "NEXT_STEP.md": "7222865071577183effcc880377cba73f49f3fff9c3e90b136e7f26da216f5d4" + }, + "decision": "Preserve measured paired factual gains and regressions; do not promote C, claim efficacy on unseen tasks, or build a memory-product premise from source-present errors. Stop blanket layout/prose and wording sweeps. Prepare the next ordinary question-specific evidence contract offline.", + "uncertainty": "Primary and operational descriptive scenario-cluster intervals include zero; synthetic development bank, related scenarios, assistant-only review, actual compute unequal.", + "previous_scores_unchanged": true, + "new_paid_phase_started": false, + "created_at_utc": "2026-10-14T02:48:50.179089+00:00", + "prepublication_claim_review_corrections": [ + "Clarify 720 paid calls executed, no subsequent paid phase", + "Distinguish existing Hindi abstentions from new losses", + "Describe package pending publication without premature publication claim" + ] +} diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/error-attribution.json b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/error-attribution.json new file mode 100644 index 0000000..22270a6 --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/error-attribution.json @@ -0,0 +1,5876 @@ +{ + "status": "DESCRIPTIVE_ERROR_ATTRIBUTION_NOT_CAUSAL_IDENTIFICATION", + "rows_checked": 720, + "summary": { + "sarvam-105b-conversations/chronological": { + "wrong_non_null_date": 22, + "wrong_answer_matches_current_backend_date": 19, + "raw_action_status_wrong": 11, + "wrong_status_repaired_by_existing_receipt_projection": 11, + "definite_answer_on_unknown_target": 10, + "false_abstention_on_known_target": 2 + }, + "sarvam-105b-conversations/grouped_actor_prose_withheld": { + "raw_action_status_wrong": 23, + "wrong_status_repaired_by_existing_receipt_projection": 23, + "false_abstention_on_known_target": 9, + "wrong_non_null_date": 8, + "definite_answer_on_unknown_target": 2, + "wrong_answer_matches_current_backend_date": 6 + }, + "sarvam-105b-conversations/grouped_lossless": { + "raw_action_status_wrong": 25, + "wrong_status_repaired_by_existing_receipt_projection": 25, + "wrong_non_null_date": 18, + "wrong_answer_matches_current_backend_date": 14, + "definite_answer_on_unknown_target": 6, + "false_abstention_on_known_target": 2 + }, + "sarvam-105b/chronological": { + "raw_action_status_wrong": 38, + "wrong_status_repaired_by_existing_receipt_projection": 38, + "missing_or_invalid_final": 5, + "correct_date_value_wrong_clarification": 8, + "wrong_non_null_date": 3, + "wrong_answer_matches_current_backend_date": 3 + }, + "sarvam-105b/grouped_lossless": { + "raw_action_status_wrong": 29, + "wrong_status_repaired_by_existing_receipt_projection": 29, + "wrong_non_null_date": 4, + "wrong_answer_matches_current_backend_date": 4, + "definite_answer_on_unknown_target": 2, + "missing_or_invalid_final": 8, + "correct_date_value_wrong_clarification": 8, + "false_abstention_on_known_target": 1 + }, + "sarvam-105b/grouped_actor_prose_withheld": { + "raw_action_status_wrong": 29, + "wrong_status_repaired_by_existing_receipt_projection": 29, + "missing_or_invalid_final": 6, + "correct_date_value_wrong_clarification": 6, + "wrong_non_null_date": 1, + "wrong_answer_matches_current_backend_date": 1, + "false_abstention_on_known_target": 2 + } + }, + "error_rows": [ + { + "id": "confirm-fresh-31-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-31-en-m1", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "current_backend_date": "2027-04-01", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-31-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-31-en-m1", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "current_backend_date": "2027-04-01", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-29-hi-m0-source-chronological", + "history_id": "confirm-fresh-29-hi-m0", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "current_backend_date": "2027-03-30", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-18-en-m1-source-chronological", + "history_id": "confirm-fresh-18-en-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "current_backend_date": "2027-03-19", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-18-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-18-en-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "current_backend_date": "2027-03-19", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-18-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-en-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "current_backend_date": "2027-03-19", + "labels": [ + "false_abstention_on_known_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-34-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-34-hi-m1", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "current_backend_date": "2027-06-04", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-34-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-hi-m1", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "current_backend_date": "2027-06-04", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-32-te-m0-source-chronological", + "history_id": "confirm-fresh-32-te-m0", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "current_backend_date": "2027-04-02", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-30-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-30-te-m0", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "current_backend_date": "2027-03-31", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-30-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-te-m0", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "current_backend_date": "2027-03-31", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-30-te-m0-source-chronological", + "history_id": "confirm-fresh-30-te-m0", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "current_backend_date": "2027-03-31", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-30-hi-m0-source-chronological", + "history_id": "confirm-fresh-30-hi-m0", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "current_backend_date": "2027-03-31", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-18-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-18-hi-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "current_backend_date": "2027-03-19", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-18-hi-m1-source-chronological", + "history_id": "confirm-fresh-18-hi-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "current_backend_date": "2027-03-19", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-39-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-hi-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-39-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-39-hi-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-39-hi-m1-source-chronological", + "history_id": "confirm-fresh-39-hi-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-35-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-en-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "current_backend_date": "2027-06-05", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-35-en-m0-source-chronological", + "history_id": "confirm-fresh-35-en-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "current_backend_date": "2027-06-05", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-35-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-35-en-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "current_backend_date": "2027-06-05", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-36-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-36-en-m1", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_performed", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "current_backend_date": "2027-06-06", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-08-hi-m1-source-chronological", + "history_id": "confirm-fresh-08-hi-m1", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-09", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-09", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-25-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-25-te-m0", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": null, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "current_backend_date": "2027-04-26", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-36-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-te-m1", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "current_backend_date": "2027-06-06", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-13-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-13-hi-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-14", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-13-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-13-hi-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-14", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-14", + "labels": [ + "wrong_non_null_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-13-hi-m1-source-chronological", + "history_id": "confirm-fresh-13-hi-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-14", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-29-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-29-te-m0", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "current_backend_date": "2027-03-30", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-26-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-26-hi-m0", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": null, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "current_backend_date": "2027-04-27", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-13-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-13-hi-m0", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-14", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-31-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-31-hi-m0", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "current_backend_date": "2027-04-01", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-31-hi-m0-source-chronological", + "history_id": "confirm-fresh-31-hi-m0", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "current_backend_date": "2027-04-01", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-33-te-m0-source-chronological", + "history_id": "confirm-fresh-33-te-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "current_backend_date": "2027-06-03", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-33-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-33-te-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "current_backend_date": "2027-06-03", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-33-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-te-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "current_backend_date": "2027-06-03", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-32-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-hi-m1", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "current_backend_date": "2027-04-02", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-32-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-32-hi-m1", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "current_backend_date": "2027-04-02", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-15-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-15-hi-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-16", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-16", + "labels": [ + "wrong_non_null_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-35-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-35-hi-m1", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "current_backend_date": "2027-06-05", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-20-hi-m1-source-chronological", + "history_id": "confirm-fresh-20-hi-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "already_applied", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "current_backend_date": "2027-03-21", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-20-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-20-hi-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "current_backend_date": "2027-03-21", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-40-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-40-en-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-05-11", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-37-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-37-hi-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "rejected", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "current_backend_date": "2027-05-08", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-37-hi-m0-source-chronological", + "history_id": "confirm-fresh-37-hi-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "current_backend_date": "2027-05-08", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-37-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-hi-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": null, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "current_backend_date": "2027-05-08", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-07-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-te-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-08", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-03-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-03-en-m1", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "current_backend_date": "2027-03-04", + "labels": [ + "false_abstention_on_known_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-04-en-m0-source-chronological", + "history_id": "confirm-fresh-04-en-m0", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "current_backend_date": "2027-03-05", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-19-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-hi-m0", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "already_applied", + "answer": "2027-04-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "current_backend_date": "2027-03-20", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-32-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-32-te-m1", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "current_backend_date": "2027-04-02", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-32-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-te-m1", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "current_backend_date": "2027-04-02", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-37-en-m1-source-chronological", + "history_id": "confirm-fresh-37-en-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "current_backend_date": "2027-05-08", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-37-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-en-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "current_backend_date": "2027-05-08", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-37-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-37-en-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "current_backend_date": "2027-05-08", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-36-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-hi-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "current_backend_date": "2027-06-06", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-10-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-10-en-m0", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-03-11", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-10-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-10-hi-m0", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-03-11", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-14-te-m1-source-chronological", + "history_id": "confirm-fresh-14-te-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-15", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-15", + "labels": [ + "wrong_non_null_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-14-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-14-te-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-15", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-15", + "labels": [ + "wrong_non_null_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-39-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-en-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-39-en-m1-source-chronological", + "history_id": "confirm-fresh-39-en-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-13-te-m1-source-chronological", + "history_id": "confirm-fresh-13-te-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-14", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-20-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-20-hi-m0", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "hi", + "raw": null, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "current_backend_date": "2027-03-21", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-19-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-19-en-m0", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": null, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "current_backend_date": "2027-03-20", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-38-te-m1-source-chronological", + "history_id": "confirm-fresh-38-te-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "rejected", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "current_backend_date": "2027-05-09", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-38-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-38-te-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "rejected", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "current_backend_date": "2027-05-09", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-38-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-te-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "current_backend_date": "2027-05-09", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-35-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-35-en-m1", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "current_backend_date": "2027-06-05", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-16-te-m0-source-chronological", + "history_id": "confirm-fresh-16-te-m0", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": null, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-17", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-15-te-m1-source-chronological", + "history_id": "confirm-fresh-15-te-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-16", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-15-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-15-te-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-16", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-16", + "labels": [ + "wrong_non_null_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-05-hi-m0-source-chronological", + "history_id": "confirm-fresh-05-hi-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-06", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-05-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-05-hi-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-06", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-40-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-40-hi-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-05-11", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-40-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-hi-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-05-11", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-14-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-14-hi-m0", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-15", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-14-hi-m0-source-chronological", + "history_id": "confirm-fresh-14-hi-m0", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "language": "hi", + "raw": null, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-15", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-09-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-09-hi-m1", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "current_backend_date": "2027-03-10", + "labels": [ + "false_abstention_on_known_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-39-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-hi-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-39-hi-m0-source-chronological", + "history_id": "confirm-fresh-39-hi-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-39-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-39-hi-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "hi", + "raw": null, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-18-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-te-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "current_backend_date": "2027-03-19", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-18-te-m1-source-chronological", + "history_id": "confirm-fresh-18-te-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "current_backend_date": "2027-03-19", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-18-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-18-te-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "confirmed", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "current_backend_date": "2027-03-19", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-17-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-17-en-m0", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "current_backend_date": "2027-03-18", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-19-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-te-m0", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_performed", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "current_backend_date": "2027-03-20", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-19-te-m0-source-chronological", + "history_id": "confirm-fresh-19-te-m0", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "confirmed", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "current_backend_date": "2027-03-20", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-20-te-m1-source-chronological", + "history_id": "confirm-fresh-20-te-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "current_backend_date": "2027-03-21", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-20-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-20-te-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "current_backend_date": "2027-03-21", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-20-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-20-te-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "current_backend_date": "2027-03-21", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-37-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-te-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "current_backend_date": "2027-05-08", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-37-te-m0-source-chronological", + "history_id": "confirm-fresh-37-te-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "current_backend_date": "2027-05-08", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-37-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-37-te-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "current_backend_date": "2027-05-08", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-12-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-te-m0", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_performed", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "current_backend_date": "2027-03-13", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-38-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-en-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "current_backend_date": "2027-05-09", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-38-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-38-en-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "current_backend_date": "2027-05-09", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-08-en-m0-source-chronological", + "history_id": "confirm-fresh-08-en-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-09", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-08-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-08-en-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-09", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-08-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-08-en-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": null, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-09", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-15-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-15-en-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-16", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-15-en-m1-source-chronological", + "history_id": "confirm-fresh-15-en-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-16", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-20-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-20-en-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "current_backend_date": "2027-03-21", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-20-en-m1-source-chronological", + "history_id": "confirm-fresh-20-en-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "current_backend_date": "2027-03-21", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-20-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-20-en-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "current_backend_date": "2027-03-21", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-40-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-40-en-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-05-11", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-40-en-m0-source-chronological", + "history_id": "confirm-fresh-40-en-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-05-11", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-05-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-05-te-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-06", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-38-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-hi-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "current_backend_date": "2027-05-09", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-38-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-38-hi-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "current_backend_date": "2027-05-09", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-01-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-01-en-m1", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "current_backend_date": "2027-03-02", + "labels": [ + "false_abstention_on_known_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-13-en-m0-source-chronological", + "history_id": "confirm-fresh-13-en-m0", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-14", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-30-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-te-m1", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "current_backend_date": "2027-03-31", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-30-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-30-te-m1", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "current_backend_date": "2027-03-31", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-35-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-35-hi-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_performed", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "current_backend_date": "2027-06-05", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-35-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-hi-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "current_backend_date": "2027-06-05", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-38-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-38-en-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "current_backend_date": "2027-05-09", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-25-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-25-en-m0", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "current_backend_date": "2027-04-26", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-27-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-27-te-m0", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "current_backend_date": "2027-04-28", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-19-en-m1-source-chronological", + "history_id": "confirm-fresh-19-en-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "current_backend_date": "2027-03-20", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-19-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-en-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "current_backend_date": "2027-03-20", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-19-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-19-en-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "current_backend_date": "2027-03-20", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-32-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-en-m1", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "current_backend_date": "2027-04-02", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-13-te-m0-source-chronological", + "history_id": "confirm-fresh-13-te-m0", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": null, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-14", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-13-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-13-te-m0", + "family": "unresolved_condition", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-14", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-12-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-12-hi-m1", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "current_backend_date": "2027-03-13", + "labels": [ + "false_abstention_on_known_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-12-hi-m1-source-chronological", + "history_id": "confirm-fresh-12-hi-m1", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "current_backend_date": "2027-03-13", + "labels": [ + "false_abstention_on_known_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-12-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-hi-m1", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "current_backend_date": "2027-03-13", + "labels": [ + "false_abstention_on_known_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-18-te-m0-source-chronological", + "history_id": "confirm-fresh-18-te-m0", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": null, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "current_backend_date": "2027-03-19", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-17-hi-m1-source-chronological", + "history_id": "confirm-fresh-17-hi-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "current_backend_date": "2027-03-18", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-17-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-17-hi-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "already_applied", + "answer": "2027-03-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "current_backend_date": "2027-03-18", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-22-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-22-te-m0", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": null, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "current_backend_date": "2027-04-23", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-37-en-m0-source-chronological", + "history_id": "confirm-fresh-37-en-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "current_backend_date": "2027-05-08", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-37-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-37-en-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "current_backend_date": "2027-05-08", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-37-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-en-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "current_backend_date": "2027-05-08", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-13-en-m1-source-chronological", + "history_id": "confirm-fresh-13-en-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-14", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-13-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-13-en-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-14", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-19-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-19-hi-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "current_backend_date": "2027-03-20", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-19-hi-m1-source-chronological", + "history_id": "confirm-fresh-19-hi-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "current_backend_date": "2027-03-20", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-37-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-te-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "current_backend_date": "2027-05-08", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-37-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-37-te-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "rejected", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "current_backend_date": "2027-05-08", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-06-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-06-te-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-07", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-06-te-m0-source-chronological", + "history_id": "confirm-fresh-06-te-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "already_applied", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-07", + "labels": [ + "correct_date_value_wrong_clarification", + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-39-te-m0-source-chronological", + "history_id": "confirm-fresh-39-te-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-39-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-39-te-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-39-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-te-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-12-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-en-m0", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "current_backend_date": "2027-03-13", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-11-hi-m1-source-chronological", + "history_id": "confirm-fresh-11-hi-m1", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "current_backend_date": "2027-03-12", + "labels": [ + "false_abstention_on_known_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-11-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-11-hi-m1", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "current_backend_date": "2027-03-12", + "labels": [ + "false_abstention_on_known_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-30-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-30-en-m0", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "current_backend_date": "2027-03-31", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-34-te-m0-source-chronological", + "history_id": "confirm-fresh-34-te-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_performed", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "current_backend_date": "2027-06-04", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-34-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-te-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_performed", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "current_backend_date": "2027-06-04", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-34-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-34-te-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "current_backend_date": "2027-06-04", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-33-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-33-en-m1", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "current_backend_date": "2027-06-03", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-10-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-10-hi-m1", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-03-11", + "labels": [ + "false_abstention_on_known_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-18-hi-m0-source-chronological", + "history_id": "confirm-fresh-18-hi-m0", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "confirmed", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "current_backend_date": "2027-03-19", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-17-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-17-en-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "current_backend_date": "2027-03-18", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-17-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-17-en-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "current_backend_date": "2027-03-18", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-17-en-m1-source-chronological", + "history_id": "confirm-fresh-17-en-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "current_backend_date": "2027-03-18", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-04-hi-m1-source-chronological", + "history_id": "confirm-fresh-04-hi-m1", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "current_backend_date": "2027-03-05", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-01-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-01-te-m0", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "current_backend_date": "2027-03-02", + "labels": [ + "false_abstention_on_known_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-01-te-m0-source-chronological", + "history_id": "confirm-fresh-01-te-m0", + "family": "request_correction", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "current_backend_date": "2027-03-02", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-05-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-05-en-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-06", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-05-en-m0-source-chronological", + "history_id": "confirm-fresh-05-en-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-06", + "labels": [ + "correct_date_value_wrong_clarification", + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-05-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-05-en-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-06", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-16-en-m1-source-chronological", + "history_id": "confirm-fresh-16-en-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-17", + "labels": [ + "wrong_non_null_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-16-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-16-en-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-17", + "labels": [ + "wrong_non_null_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-28-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-28-hi-m1", + "family": "approval_boundary", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-29", + "needs_clarification": false + }, + "current_backend_date": "2027-03-29", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-34-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-en-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "current_backend_date": "2027-06-04", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-34-en-m0-source-chronological", + "history_id": "confirm-fresh-34-en-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "current_backend_date": "2027-06-04", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-40-te-m0-source-chronological", + "history_id": "confirm-fresh-40-te-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-05-11", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-40-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-te-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-05-11", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-40-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-40-te-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-05-11", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-09-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-09-en-m1", + "family": "resolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "current_backend_date": "2027-03-10", + "labels": [ + "false_abstention_on_known_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-38-hi-m0-source-chronological", + "history_id": "confirm-fresh-38-hi-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "current_backend_date": "2027-05-09", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-07-en-m0-source-chronological", + "history_id": "confirm-fresh-07-en-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-08", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-07-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-07-en-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-08", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-07-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-en-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-08", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-33-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-33-en-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_performed", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "current_backend_date": "2027-06-03", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-33-en-m0-source-chronological", + "history_id": "confirm-fresh-33-en-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "current_backend_date": "2027-06-03", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-33-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-en-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "rejected", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "current_backend_date": "2027-06-03", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-18-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-en-m0", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-04-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "current_backend_date": "2027-03-19", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-18-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-18-en-m0", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "current_backend_date": "2027-03-19", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-23-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-23-te-m0", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": null, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "current_backend_date": "2027-04-24", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-23-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-23-te-m0", + "family": "approved_change", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": null, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "current_backend_date": "2027-04-24", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-30-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-en-m1", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "current_backend_date": "2027-03-31", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-17-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-17-te-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "current_backend_date": "2027-03-18", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-17-te-m1-source-chronological", + "history_id": "confirm-fresh-17-te-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "current_backend_date": "2027-03-18", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-36-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-hi-m1", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "current_backend_date": "2027-06-06", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-35-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-te-m1", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "current_backend_date": "2027-06-05", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-35-te-m1-source-chronological", + "history_id": "confirm-fresh-35-te-m1", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "current_backend_date": "2027-06-05", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-04-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-04-en-m1", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "current_backend_date": "2027-03-05", + "labels": [ + "false_abstention_on_known_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-40-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-40-te-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-05-11", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-40-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-te-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-05-11", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-06-hi-m1-source-chronological", + "history_id": "confirm-fresh-06-hi-m1", + "family": "missing_replacement", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-07", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-07", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-34-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-34-hi-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "current_backend_date": "2027-06-04", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-07-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-07-hi-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-08", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-07-hi-m0-source-chronological", + "history_id": "confirm-fresh-07-hi-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-08", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-07-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-hi-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-08", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-11-en-m0-source-chronological", + "history_id": "confirm-fresh-11-en-m0", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_performed", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "current_backend_date": "2027-03-12", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-06-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-06-en-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-07", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-06-en-m0-source-chronological", + "history_id": "confirm-fresh-06-en-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-07", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-20-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-20-te-m0", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "rejected", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "current_backend_date": "2027-03-21", + "labels": [ + "false_abstention_on_known_target", + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-33-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-hi-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "current_backend_date": "2027-06-03", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-33-hi-m0-source-chronological", + "history_id": "confirm-fresh-33-hi-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "current_backend_date": "2027-06-03", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-16-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-16-te-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-17", + "labels": [ + "wrong_non_null_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-16-te-m1-source-chronological", + "history_id": "confirm-fresh-16-te-m1", + "family": "unresolved_condition", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-17", + "labels": [ + "wrong_non_null_date", + "definite_answer_on_unknown_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-26-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-26-te-m0", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": null, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "current_backend_date": "2027-04-27", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-20-en-m0-source-chronological", + "history_id": "confirm-fresh-20-en-m0", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "current_backend_date": "2027-03-21", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-20-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-20-en-m0", + "family": "separate_booking", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": null, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "current_backend_date": "2027-03-21", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-34-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-34-te-m1", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "current_backend_date": "2027-06-04", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-34-te-m1-source-chronological", + "history_id": "confirm-fresh-34-te-m1", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "current_backend_date": "2027-06-04", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-02-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-02-en-m1", + "family": "request_correction", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "current_backend_date": "2027-03-03", + "labels": [ + "false_abstention_on_known_target" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-10-te-m0-source-chronological", + "history_id": "confirm-fresh-10-te-m0", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-03-11", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-10-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-10-te-m0", + "family": "resolved_condition", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "already_applied", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-03-11", + "labels": [ + "false_abstention_on_known_target", + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-31-en-m0-source-chronological", + "history_id": "confirm-fresh-31-en-m0", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "current_backend_date": "2027-04-01", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-31-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-31-en-m0", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "current_backend_date": "2027-04-01", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-37-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-hi-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "current_backend_date": "2027-05-08", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-37-hi-m1-source-chronological", + "history_id": "confirm-fresh-37-hi-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "current_backend_date": "2027-05-08", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-39-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-39-en-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-39-en-m0-source-chronological", + "history_id": "confirm-fresh-39-en-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-39-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-en-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-36-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-36-te-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "current_backend_date": "2027-06-06", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-36-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-te-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "current_backend_date": "2027-06-06", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-36-te-m0-source-chronological", + "history_id": "confirm-fresh-36-te-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "current_backend_date": "2027-06-06", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-06-hi-m0-source-chronological", + "history_id": "confirm-fresh-06-hi-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-07", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-06-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-06-hi-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-07", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-06-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-06-hi-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-07", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-38-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-te-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "current_backend_date": "2027-05-09", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-38-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-38-te-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "current_backend_date": "2027-05-09", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-38-te-m0-source-chronological", + "history_id": "confirm-fresh-38-te-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "current_backend_date": "2027-05-09", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-40-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-hi-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-05-11", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-40-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-40-hi-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-05-11", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-40-hi-m0-source-chronological", + "history_id": "confirm-fresh-40-hi-m0", + "family": "conflicting_request_key", + "model": "sarvam-105b", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "current_backend_date": "2027-05-11", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-33-te-m1-source-chronological", + "history_id": "confirm-fresh-33-te-m1", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "current_backend_date": "2027-06-03", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-33-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-te-m1", + "family": "completed_older_request", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "current_backend_date": "2027-06-03", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-32-en-m0-source-chronological", + "history_id": "confirm-fresh-32-en-m0", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "current_backend_date": "2027-04-02", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-32-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-en-m0", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "current_backend_date": "2027-04-02", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-36-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-36-en-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "current_backend_date": "2027-06-06", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-36-en-m0-source-chronological", + "history_id": "confirm-fresh-36-en-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "language": "en", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "current_backend_date": "2027-06-06", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-35-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-35-te-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "current_backend_date": "2027-06-05", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-35-te-m0-source-chronological", + "history_id": "confirm-fresh-35-te-m0", + "family": "completed_older_request", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "current_backend_date": "2027-06-05", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-19-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-te-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "current_backend_date": "2027-03-20", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-19-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-19-te-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "current_backend_date": "2027-03-20", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-19-te-m1-source-chronological", + "history_id": "confirm-fresh-19-te-m1", + "family": "separate_booking", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "current_backend_date": "2027-03-20", + "labels": [ + "wrong_non_null_date", + "wrong_answer_matches_current_backend_date" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-31-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-31-te-m0", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": null, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "current_backend_date": "2027-04-01", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-31-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-31-te-m0", + "family": "future_approval", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "current_backend_date": "2027-04-01", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-08-te-m0-source-chronological", + "history_id": "confirm-fresh-08-te-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-09", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-29-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-29-en-m1", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "current_backend_date": "2027-03-30", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-29-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-29-en-m1", + "family": "future_approval", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "en", + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "current_backend_date": "2027-03-30", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-39-te-m1-source-chronological", + "history_id": "confirm-fresh-39-te-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "language": "te", + "raw": { + "action_status": "rejected", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-39-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-te-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "language": "te", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-39-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-39-te-m1", + "family": "conflicting_request_key", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "language": "te", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "current_backend_date": "2027-05-10", + "labels": [ + "raw_action_status_wrong", + "wrong_status_repaired_by_existing_receipt_projection" + ], + "primary_correct": true + }, + { + "id": "confirm-fresh-28-hi-m0-source-chronological", + "history_id": "confirm-fresh-28-hi-m0", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "chronological", + "language": "hi", + "raw": null, + "target": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "current_backend_date": "2027-04-29", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-08-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-08-hi-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-09", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-08-hi-m0-source-chronological", + "history_id": "confirm-fresh-08-hi-m0", + "family": "missing_replacement", + "model": "sarvam-105b", + "arm": "chronological", + "language": "hi", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "current_backend_date": "2027-03-09", + "labels": [ + "correct_date_value_wrong_clarification" + ], + "primary_correct": false + }, + { + "id": "confirm-fresh-27-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-27-en-m0", + "family": "approval_boundary", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "language": "en", + "raw": null, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "current_backend_date": "2027-04-28", + "labels": [ + "missing_or_invalid_final" + ], + "primary_correct": false + } + ], + "limitations": [ + "Labels can overlap; their totals are not a new accuracy denominator.", + "A wrong answer equaling the backend date does not prove why the model chose it.", + "Receipt projection is the same existing deterministic layer in every arm.", + "No gold, model answer, prior score or advancement criterion is changed." + ] +} diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/evidence.tar.gz b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/evidence.tar.gz new file mode 100644 index 0000000..db2fc33 Binary files /dev/null and b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/evidence.tar.gz differ diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/independent-aggregation.json b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/independent-aggregation.json new file mode 100644 index 0000000..2be6165 --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/independent-aggregation.json @@ -0,0 +1,24948 @@ +{ + "status": "INDEPENDENT_STDLIB_RAW_REPLAY", + "scope": "All 720 prospective outputs; no imports from campaign code; no human/native review", + "manifest_sha256": "68e968002608b5d826d3646fe911c5918703529ac1d27a2798d38a706e3c6509", + "rows": [ + { + "id": "confirm-fresh-31-en-m1-source-chronological", + "history_id": "confirm-fresh-31-en-m1", + "scenario": "fresh-31", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25035, + "retained_microinr": 31294, + "prompt_tokens": 1509, + "completion_tokens": 40, + "elapsed_seconds": 1.0370477910037152, + "dispatch_time": 1791942454.425504 + }, + { + "id": "confirm-fresh-31-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-31-en-m1", + "scenario": "fresh-31", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 36735, + "retained_microinr": 45919, + "prompt_tokens": 2281, + "completion_tokens": 42, + "elapsed_seconds": 0.9786056249868125, + "dispatch_time": 1791942453.757668 + }, + { + "id": "confirm-fresh-31-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-31-en-m1", + "scenario": "fresh-31", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 36735, + "retained_microinr": 45919, + "prompt_tokens": 2281, + "completion_tokens": 42, + "elapsed_seconds": 1.1229317920078756, + "dispatch_time": 1791942452.424167 + }, + { + "id": "confirm-fresh-22-hi-m1-source-chronological", + "history_id": "confirm-fresh-22-hi-m1", + "scenario": "fresh-22", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24405, + "retained_microinr": 30507, + "prompt_tokens": 1467, + "completion_tokens": 40, + "elapsed_seconds": 1.1193134169880068, + "dispatch_time": 1791942453.093348 + }, + { + "id": "confirm-fresh-22-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-22-hi-m1", + "scenario": "fresh-22", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34110, + "retained_microinr": 42638, + "prompt_tokens": 2114, + "completion_tokens": 40, + "elapsed_seconds": 1.0401996669970686, + "dispatch_time": 1791942456.8428721 + }, + { + "id": "confirm-fresh-22-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-22-hi-m1", + "scenario": "fresh-22", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34110, + "retained_microinr": 42638, + "prompt_tokens": 2114, + "completion_tokens": 40, + "elapsed_seconds": 1.2693526670045685, + "dispatch_time": 1791942456.172561 + }, + { + "id": "confirm-fresh-24-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-24-hi-m1", + "scenario": "fresh-24", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34200, + "retained_microinr": 42750, + "prompt_tokens": 2120, + "completion_tokens": 40, + "elapsed_seconds": 0.9839419580093818, + "dispatch_time": 1791942455.50863 + }, + { + "id": "confirm-fresh-24-hi-m1-source-chronological", + "history_id": "confirm-fresh-24-hi-m1", + "scenario": "fresh-24", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24330, + "retained_microinr": 30413, + "prompt_tokens": 1462, + "completion_tokens": 40, + "elapsed_seconds": 0.9552010420011356, + "dispatch_time": 1791942457.509232 + }, + { + "id": "confirm-fresh-24-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-24-hi-m1", + "scenario": "fresh-24", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34200, + "retained_microinr": 42750, + "prompt_tokens": 2120, + "completion_tokens": 40, + "elapsed_seconds": 0.9628701659967192, + "dispatch_time": 1791942458.50993 + }, + { + "id": "confirm-fresh-29-hi-m0-source-chronological", + "history_id": "confirm-fresh-29-hi-m0", + "scenario": "fresh-29", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 183645, + "retained_microinr": 229557, + "prompt_tokens": 1387, + "completion_tokens": 2714, + "elapsed_seconds": 12.89874437499384, + "dispatch_time": 1791942460.507543 + }, + { + "id": "confirm-fresh-29-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-29-hi-m0", + "scenario": "fresh-29", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 465795, + "retained_microinr": 582244, + "prompt_tokens": 1929, + "completion_tokens": 7281, + "elapsed_seconds": 34.35796750000736, + "dispatch_time": 1791942459.177248 + }, + { + "id": "confirm-fresh-29-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-29-hi-m0", + "scenario": "fresh-29", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 222990, + "retained_microinr": 278738, + "prompt_tokens": 1738, + "completion_tokens": 3282, + "elapsed_seconds": 13.335680667005363, + "dispatch_time": 1791942459.843329 + }, + { + "id": "confirm-fresh-18-en-m1-source-chronological", + "history_id": "confirm-fresh-18-en-m1", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 18075, + "retained_microinr": 22594, + "prompt_tokens": 1037, + "completion_tokens": 42, + "elapsed_seconds": 1.0114832500112243, + "dispatch_time": 1791942493.574848 + }, + { + "id": "confirm-fresh-18-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-18-en-m1", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 26865, + "retained_microinr": 33582, + "prompt_tokens": 1623, + "completion_tokens": 42, + "elapsed_seconds": 0.9449328329937998, + "dispatch_time": 1791942494.904854 + }, + { + "id": "confirm-fresh-18-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-en-m1", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": true, + "raw_status_correct": true, + "known_microinr": 25380, + "retained_microinr": 31725, + "prompt_tokens": 1564, + "completion_tokens": 32, + "elapsed_seconds": 1.0430362920014886, + "dispatch_time": 1791942494.241298 + }, + { + "id": "confirm-fresh-34-hi-m1-source-chronological", + "history_id": "confirm-fresh-34-hi-m1", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 32085, + "retained_microinr": 40107, + "prompt_tokens": 1971, + "completion_tokens": 42, + "elapsed_seconds": 1.1093939580023289, + "dispatch_time": 1791942495.573548 + }, + { + "id": "confirm-fresh-34-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-34-hi-m1", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 40605, + "retained_microinr": 50757, + "prompt_tokens": 2539, + "completion_tokens": 42, + "elapsed_seconds": 0.9951028329960536, + "dispatch_time": 1791942496.7328079 + }, + { + "id": "confirm-fresh-34-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-hi-m1", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 35490, + "retained_microinr": 44363, + "prompt_tokens": 2198, + "completion_tokens": 42, + "elapsed_seconds": 0.9545394169981591, + "dispatch_time": 1791942498.064251 + }, + { + "id": "confirm-fresh-32-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-te-m0", + "scenario": "fresh-32", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 140085, + "retained_microinr": 175107, + "prompt_tokens": 1591, + "completion_tokens": 1937, + "elapsed_seconds": 10.579083415999776, + "dispatch_time": 1791942498.732119 + }, + { + "id": "confirm-fresh-32-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-32-te-m0", + "scenario": "fresh-32", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 144285, + "retained_microinr": 180357, + "prompt_tokens": 1731, + "completion_tokens": 1972, + "elapsed_seconds": 9.322261666995473, + "dispatch_time": 1791942497.4003751 + }, + { + "id": "confirm-fresh-32-te-m0-source-chronological", + "history_id": "confirm-fresh-32-te-m0", + "scenario": "fresh-32", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 131205, + "retained_microinr": 164007, + "prompt_tokens": 1303, + "completion_tokens": 1861, + "elapsed_seconds": 9.870290666993242, + "dispatch_time": 1791942509.357515 + }, + { + "id": "confirm-fresh-30-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-30-te-m0", + "scenario": "fresh-30", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 111900, + "retained_microinr": 139875, + "prompt_tokens": 1856, + "completion_tokens": 1401, + "elapsed_seconds": 8.085513083002297, + "dispatch_time": 1791942511.358352 + }, + { + "id": "confirm-fresh-30-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-te-m0", + "scenario": "fresh-30", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 219135, + "retained_microinr": 273919, + "prompt_tokens": 1601, + "completion_tokens": 3252, + "elapsed_seconds": 14.983070792004582, + "dispatch_time": 1791942510.6928241 + }, + { + "id": "confirm-fresh-30-te-m0-source-chronological", + "history_id": "confirm-fresh-30-te-m0", + "scenario": "fresh-30", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 199410, + "retained_microinr": 249263, + "prompt_tokens": 1418, + "completion_tokens": 2969, + "elapsed_seconds": 12.611806625005556, + "dispatch_time": 1791942510.022159 + }, + { + "id": "confirm-fresh-30-hi-m0-source-chronological", + "history_id": "confirm-fresh-30-hi-m0", + "scenario": "fresh-30", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 470415, + "retained_microinr": 588019, + "prompt_tokens": 1417, + "completion_tokens": 7486, + "elapsed_seconds": 35.69330058401101, + "dispatch_time": 1791942527.043194 + }, + { + "id": "confirm-fresh-30-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-hi-m0", + "scenario": "fresh-30", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 300225, + "retained_microinr": 375282, + "prompt_tokens": 1599, + "completion_tokens": 4604, + "elapsed_seconds": 20.368539167000563, + "dispatch_time": 1791942527.709756 + }, + { + "id": "confirm-fresh-30-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-30-hi-m0", + "scenario": "fresh-30", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 201045, + "retained_microinr": 251307, + "prompt_tokens": 1859, + "completion_tokens": 2886, + "elapsed_seconds": 15.877402416997938, + "dispatch_time": 1791942525.71189 + }, + { + "id": "confirm-fresh-17-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-17-te-m0", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 216645, + "retained_microinr": 270807, + "prompt_tokens": 1451, + "completion_tokens": 3248, + "elapsed_seconds": 17.090247959000408, + "dispatch_time": 1791942526.3774478 + }, + { + "id": "confirm-fresh-17-te-m0-source-chronological", + "history_id": "confirm-fresh-17-te-m0", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 322785, + "retained_microinr": 403482, + "prompt_tokens": 899, + "completion_tokens": 5155, + "elapsed_seconds": 23.44692899999791, + "dispatch_time": 1791942563.4484422 + }, + { + "id": "confirm-fresh-17-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-17-te-m0", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 170520, + "retained_microinr": 213150, + "prompt_tokens": 1436, + "completion_tokens": 2483, + "elapsed_seconds": 13.231449500002782, + "dispatch_time": 1791942562.7848551 + }, + { + "id": "confirm-fresh-23-hi-m1-source-chronological", + "history_id": "confirm-fresh-23-hi-m1", + "scenario": "fresh-23", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24945, + "retained_microinr": 31182, + "prompt_tokens": 1503, + "completion_tokens": 40, + "elapsed_seconds": 0.9960489160002908, + "dispatch_time": 1791942564.7825432 + }, + { + "id": "confirm-fresh-23-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-23-hi-m1", + "scenario": "fresh-23", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36405, + "retained_microinr": 45507, + "prompt_tokens": 2267, + "completion_tokens": 40, + "elapsed_seconds": 0.9549742919916753, + "dispatch_time": 1791942564.1171992 + }, + { + "id": "confirm-fresh-23-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-23-hi-m1", + "scenario": "fresh-23", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36405, + "retained_microinr": 45507, + "prompt_tokens": 2267, + "completion_tokens": 40, + "elapsed_seconds": 0.9802527080028085, + "dispatch_time": 1791942588.9341068 + }, + { + "id": "confirm-fresh-23-hi-m0-source-chronological", + "history_id": "confirm-fresh-23-hi-m0", + "scenario": "fresh-23", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 79005, + "retained_microinr": 98757, + "prompt_tokens": 1367, + "completion_tokens": 975, + "elapsed_seconds": 5.530833540993626, + "dispatch_time": 1791942586.934761 + }, + { + "id": "confirm-fresh-23-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-23-hi-m0", + "scenario": "fresh-23", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 209535, + "retained_microinr": 261919, + "prompt_tokens": 1981, + "completion_tokens": 2997, + "elapsed_seconds": 13.545640917000128, + "dispatch_time": 1791942587.6026728 + }, + { + "id": "confirm-fresh-23-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-23-hi-m0", + "scenario": "fresh-23", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 216075, + "retained_microinr": 270094, + "prompt_tokens": 1981, + "completion_tokens": 3106, + "elapsed_seconds": 12.844156040999223, + "dispatch_time": 1791942588.268576 + }, + { + "id": "confirm-fresh-18-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-18-hi-m1", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 26745, + "retained_microinr": 33432, + "prompt_tokens": 1615, + "completion_tokens": 42, + "elapsed_seconds": 1.0425940839922987, + "dispatch_time": 1791942601.1877382 + }, + { + "id": "confirm-fresh-18-hi-m1-source-chronological", + "history_id": "confirm-fresh-18-hi-m1", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 17775, + "retained_microinr": 22219, + "prompt_tokens": 1017, + "completion_tokens": 42, + "elapsed_seconds": 1.0169891669938806, + "dispatch_time": 1791942602.520355 + }, + { + "id": "confirm-fresh-18-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-hi-m1", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 26175, + "retained_microinr": 32719, + "prompt_tokens": 1577, + "completion_tokens": 42, + "elapsed_seconds": 0.971732999998494, + "dispatch_time": 1791942601.8547351 + }, + { + "id": "confirm-fresh-24-hi-m0-source-chronological", + "history_id": "confirm-fresh-24-hi-m0", + "scenario": "fresh-24", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 204600, + "retained_microinr": 255750, + "prompt_tokens": 1332, + "completion_tokens": 3077, + "elapsed_seconds": 13.557264499992016, + "dispatch_time": 1791942603.188118 + }, + { + "id": "confirm-fresh-24-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-24-hi-m0", + "scenario": "fresh-24", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 230595, + "retained_microinr": 288244, + "prompt_tokens": 1841, + "completion_tokens": 3383, + "elapsed_seconds": 14.344590167005663, + "dispatch_time": 1791942618.124331 + }, + { + "id": "confirm-fresh-24-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-24-hi-m0", + "scenario": "fresh-24", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 264375, + "retained_microinr": 330469, + "prompt_tokens": 1841, + "completion_tokens": 3946, + "elapsed_seconds": 16.596892250003293, + "dispatch_time": 1791942616.791264 + }, + { + "id": "confirm-fresh-39-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-hi-m1", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 36165, + "retained_microinr": 45207, + "prompt_tokens": 2243, + "completion_tokens": 42, + "elapsed_seconds": 1.0989783330005594, + "dispatch_time": 1791942618.7880168 + }, + { + "id": "confirm-fresh-39-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-39-hi-m1", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 40770, + "retained_microinr": 50963, + "prompt_tokens": 2550, + "completion_tokens": 42, + "elapsed_seconds": 1.0773608750023413, + "dispatch_time": 1791942617.455621 + }, + { + "id": "confirm-fresh-39-hi-m1-source-chronological", + "history_id": "confirm-fresh-39-hi-m1", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 30450, + "retained_microinr": 38063, + "prompt_tokens": 1862, + "completion_tokens": 42, + "elapsed_seconds": 1.1406128329981584, + "dispatch_time": 1791942633.430847 + }, + { + "id": "confirm-fresh-35-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-en-m0", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 194340, + "retained_microinr": 242925, + "prompt_tokens": 2060, + "completion_tokens": 2724, + "elapsed_seconds": 12.952315665999777, + "dispatch_time": 1791942634.7628238 + }, + { + "id": "confirm-fresh-35-en-m0-source-chronological", + "history_id": "confirm-fresh-35-en-m0", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 437685, + "retained_microinr": 547107, + "prompt_tokens": 1771, + "completion_tokens": 6852, + "elapsed_seconds": 27.917850792000536, + "dispatch_time": 1791942634.096871 + }, + { + "id": "confirm-fresh-35-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-35-en-m0", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 126690, + "retained_microinr": 158363, + "prompt_tokens": 2310, + "completion_tokens": 1534, + "elapsed_seconds": 8.308625707999454, + "dispatch_time": 1791942635.428457 + }, + { + "id": "confirm-fresh-36-en-m1-source-chronological", + "history_id": "confirm-fresh-36-en-m1", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 31905, + "retained_microinr": 39882, + "prompt_tokens": 1959, + "completion_tokens": 42, + "elapsed_seconds": 0.9920252919982886, + "dispatch_time": 1791942662.058728 + }, + { + "id": "confirm-fresh-36-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-36-en-m1", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 40455, + "retained_microinr": 50569, + "prompt_tokens": 2529, + "completion_tokens": 42, + "elapsed_seconds": 0.9872349159995792, + "dispatch_time": 1791942662.72138 + }, + { + "id": "confirm-fresh-36-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-en-m1", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 35370, + "retained_microinr": 44213, + "prompt_tokens": 2190, + "completion_tokens": 42, + "elapsed_seconds": 1.0595242499985034, + "dispatch_time": 1791942663.392384 + }, + { + "id": "confirm-fresh-08-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-08-hi-m1", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 20775, + "retained_microinr": 25969, + "prompt_tokens": 1257, + "completion_tokens": 32, + "elapsed_seconds": 0.9314788750052685, + "dispatch_time": 1791942664.059351 + }, + { + "id": "confirm-fresh-08-hi-m1-source-chronological", + "history_id": "confirm-fresh-08-hi-m1", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-09", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 15540, + "retained_microinr": 19425, + "prompt_tokens": 868, + "completion_tokens": 42, + "elapsed_seconds": 0.9414168749935925, + "dispatch_time": 1791942666.3663192 + }, + { + "id": "confirm-fresh-08-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-08-hi-m1", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 21540, + "retained_microinr": 26925, + "prompt_tokens": 1308, + "completion_tokens": 32, + "elapsed_seconds": 0.9929245410021394, + "dispatch_time": 1791942665.035003 + }, + { + "id": "confirm-fresh-21-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-21-hi-m1", + "scenario": "fresh-21", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36540, + "retained_microinr": 45675, + "prompt_tokens": 2276, + "completion_tokens": 40, + "elapsed_seconds": 1.0852421669987962, + "dispatch_time": 1791942667.033622 + }, + { + "id": "confirm-fresh-21-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-21-hi-m1", + "scenario": "fresh-21", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36540, + "retained_microinr": 45675, + "prompt_tokens": 2276, + "completion_tokens": 40, + "elapsed_seconds": 1.01157066700398, + "dispatch_time": 1791942665.70188 + }, + { + "id": "confirm-fresh-21-hi-m1-source-chronological", + "history_id": "confirm-fresh-21-hi-m1", + "scenario": "fresh-21", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25050, + "retained_microinr": 31313, + "prompt_tokens": 1510, + "completion_tokens": 40, + "elapsed_seconds": 0.9554717499995604, + "dispatch_time": 1791942669.501774 + }, + { + "id": "confirm-fresh-25-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-25-te-m0", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 521295, + "retained_microinr": 651619, + "prompt_tokens": 1985, + "completion_tokens": 8192, + "elapsed_seconds": 39.421729541005334, + "dispatch_time": 1791942668.168747 + }, + { + "id": "confirm-fresh-25-te-m0-source-chronological", + "history_id": "confirm-fresh-25-te-m0", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 87285, + "retained_microinr": 109107, + "prompt_tokens": 1375, + "completion_tokens": 1111, + "elapsed_seconds": 6.427784958999837, + "dispatch_time": 1791942670.169023 + }, + { + "id": "confirm-fresh-25-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-25-te-m0", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 339255, + "retained_microinr": 424069, + "prompt_tokens": 1985, + "completion_tokens": 5158, + "elapsed_seconds": 25.215050832994166, + "dispatch_time": 1791942668.836021 + }, + { + "id": "confirm-fresh-36-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-te-m1", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 35805, + "retained_microinr": 44757, + "prompt_tokens": 2219, + "completion_tokens": 42, + "elapsed_seconds": 0.9523697499971604, + "dispatch_time": 1791942708.301461 + }, + { + "id": "confirm-fresh-36-te-m1-source-chronological", + "history_id": "confirm-fresh-36-te-m1", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 33300, + "retained_microinr": 41625, + "prompt_tokens": 2052, + "completion_tokens": 42, + "elapsed_seconds": 1.0162083329923917, + "dispatch_time": 1791942707.635545 + }, + { + "id": "confirm-fresh-36-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-36-te-m1", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 41925, + "retained_microinr": 52407, + "prompt_tokens": 2627, + "completion_tokens": 42, + "elapsed_seconds": 0.9634353330038721, + "dispatch_time": 1791942709.6339781 + }, + { + "id": "confirm-fresh-13-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-13-hi-m1", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 26685, + "retained_microinr": 33357, + "prompt_tokens": 1651, + "completion_tokens": 32, + "elapsed_seconds": 0.9992062499950407, + "dispatch_time": 1791942708.9672909 + }, + { + "id": "confirm-fresh-13-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-13-hi-m1", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-14", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 23835, + "retained_microinr": 29794, + "prompt_tokens": 1421, + "completion_tokens": 42, + "elapsed_seconds": 0.9959914999926696, + "dispatch_time": 1791942710.643198 + }, + { + "id": "confirm-fresh-13-hi-m1-source-chronological", + "history_id": "confirm-fresh-13-hi-m1", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 19005, + "retained_microinr": 23757, + "prompt_tokens": 1099, + "completion_tokens": 42, + "elapsed_seconds": 0.9374043749994598, + "dispatch_time": 1791942711.9746408 + }, + { + "id": "confirm-fresh-29-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-29-te-m0", + "scenario": "fresh-29", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 180660, + "retained_microinr": 225825, + "prompt_tokens": 2268, + "completion_tokens": 2444, + "elapsed_seconds": 11.178991207998479, + "dispatch_time": 1791942712.6417978 + }, + { + "id": "confirm-fresh-29-te-m0-source-chronological", + "history_id": "confirm-fresh-29-te-m0", + "scenario": "fresh-29", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 177600, + "retained_microinr": 222000, + "prompt_tokens": 1576, + "completion_tokens": 2566, + "elapsed_seconds": 12.215514750001603, + "dispatch_time": 1791942711.308542 + }, + { + "id": "confirm-fresh-29-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-29-te-m0", + "scenario": "fresh-29", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 122340, + "retained_microinr": 152925, + "prompt_tokens": 2032, + "completion_tokens": 1531, + "elapsed_seconds": 7.056660917005502, + "dispatch_time": 1791942723.857196 + }, + { + "id": "confirm-fresh-26-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-26-hi-m0", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 491445, + "retained_microinr": 614307, + "prompt_tokens": 1823, + "completion_tokens": 7735, + "elapsed_seconds": 40.78428570899996, + "dispatch_time": 1791942724.5243552 + }, + { + "id": "confirm-fresh-26-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-26-hi-m0", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 518865, + "retained_microinr": 648582, + "prompt_tokens": 1823, + "completion_tokens": 8192, + "elapsed_seconds": 39.290506540986826, + "dispatch_time": 1791942725.85753 + }, + { + "id": "confirm-fresh-26-hi-m0-source-chronological", + "history_id": "confirm-fresh-26-hi-m0", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 158610, + "retained_microinr": 198263, + "prompt_tokens": 1330, + "completion_tokens": 2311, + "elapsed_seconds": 9.705150166992098, + "dispatch_time": 1791942725.1900449 + }, + { + "id": "confirm-fresh-01-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-01-hi-m0", + "scenario": "fresh-01", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 223035, + "retained_microinr": 278794, + "prompt_tokens": 1425, + "completion_tokens": 3361, + "elapsed_seconds": 15.34867537500395, + "dispatch_time": 1791942766.022936 + }, + { + "id": "confirm-fresh-01-hi-m0-source-chronological", + "history_id": "confirm-fresh-01-hi-m0", + "scenario": "fresh-01", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 401115, + "retained_microinr": 501394, + "prompt_tokens": 869, + "completion_tokens": 6468, + "elapsed_seconds": 27.64760770900466, + "dispatch_time": 1791942767.35658 + }, + { + "id": "confirm-fresh-01-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-01-hi-m0", + "scenario": "fresh-01", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 66810, + "retained_microinr": 83513, + "prompt_tokens": 1410, + "completion_tokens": 761, + "elapsed_seconds": 4.8372162919986295, + "dispatch_time": 1791942765.356742 + }, + { + "id": "confirm-fresh-24-en-m1-source-chronological", + "history_id": "confirm-fresh-24-en-m1", + "scenario": "fresh-24", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24210, + "retained_microinr": 30263, + "prompt_tokens": 1454, + "completion_tokens": 40, + "elapsed_seconds": 0.9435721250047209, + "dispatch_time": 1791942766.6881459 + }, + { + "id": "confirm-fresh-24-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-24-en-m1", + "scenario": "fresh-24", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34065, + "retained_microinr": 42582, + "prompt_tokens": 2111, + "completion_tokens": 40, + "elapsed_seconds": 1.0013978330098325, + "dispatch_time": 1791942797.044418 + }, + { + "id": "confirm-fresh-24-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-24-en-m1", + "scenario": "fresh-24", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34065, + "retained_microinr": 42582, + "prompt_tokens": 2111, + "completion_tokens": 40, + "elapsed_seconds": 1.036732790991664, + "dispatch_time": 1791942795.0438159 + }, + { + "id": "confirm-fresh-02-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-02-te-m0", + "scenario": "fresh-02", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 176670, + "retained_microinr": 220838, + "prompt_tokens": 1266, + "completion_tokens": 2628, + "elapsed_seconds": 11.4500859999971, + "dispatch_time": 1791942796.373257 + }, + { + "id": "confirm-fresh-02-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-02-te-m0", + "scenario": "fresh-02", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 243315, + "retained_microinr": 304144, + "prompt_tokens": 1281, + "completion_tokens": 3735, + "elapsed_seconds": 16.362450125001487, + "dispatch_time": 1791942795.711076 + }, + { + "id": "confirm-fresh-02-te-m0-source-chronological", + "history_id": "confirm-fresh-02-te-m0", + "scenario": "fresh-02", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 175965, + "retained_microinr": 219957, + "prompt_tokens": 839, + "completion_tokens": 2723, + "elapsed_seconds": 12.771132666995982, + "dispatch_time": 1791942812.1151638 + }, + { + "id": "confirm-fresh-09-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-09-hi-m0", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 205530, + "retained_microinr": 256913, + "prompt_tokens": 1410, + "completion_tokens": 3073, + "elapsed_seconds": 18.970398125005886, + "dispatch_time": 1791942813.448718 + }, + { + "id": "confirm-fresh-09-hi-m0-source-chronological", + "history_id": "confirm-fresh-09-hi-m0", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 189885, + "retained_microinr": 237357, + "prompt_tokens": 995, + "completion_tokens": 2916, + "elapsed_seconds": 11.956998584006215, + "dispatch_time": 1791942812.782964 + }, + { + "id": "confirm-fresh-09-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-09-hi-m0", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 179865, + "retained_microinr": 224832, + "prompt_tokens": 1543, + "completion_tokens": 2612, + "elapsed_seconds": 16.79411670898844, + "dispatch_time": 1791942814.116118 + }, + { + "id": "confirm-fresh-04-te-m0-source-chronological", + "history_id": "confirm-fresh-04-te-m0", + "scenario": "fresh-04", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 197565, + "retained_microinr": 246957, + "prompt_tokens": 839, + "completion_tokens": 3083, + "elapsed_seconds": 17.790569124990725, + "dispatch_time": 1791942834.463371 + }, + { + "id": "confirm-fresh-04-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-04-te-m0", + "scenario": "fresh-04", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 65610, + "retained_microinr": 82013, + "prompt_tokens": 1262, + "completion_tokens": 778, + "elapsed_seconds": 3.9927155420009512, + "dispatch_time": 1791942832.461181 + }, + { + "id": "confirm-fresh-04-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-04-te-m0", + "scenario": "fresh-04", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 56895, + "retained_microinr": 71119, + "prompt_tokens": 1277, + "completion_tokens": 629, + "elapsed_seconds": 3.36182416700467, + "dispatch_time": 1791942833.132597 + }, + { + "id": "confirm-fresh-13-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-13-hi-m0", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 417795, + "retained_microinr": 522244, + "prompt_tokens": 1581, + "completion_tokens": 6568, + "elapsed_seconds": 32.179166166999494, + "dispatch_time": 1791942833.796834 + }, + { + "id": "confirm-fresh-13-hi-m0-source-chronological", + "history_id": "confirm-fresh-13-hi-m0", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 170985, + "retained_microinr": 213732, + "prompt_tokens": 1023, + "completion_tokens": 2594, + "elapsed_seconds": 11.822467374993721, + "dispatch_time": 1791942866.0200691 + }, + { + "id": "confirm-fresh-13-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-13-hi-m0", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 327885, + "retained_microinr": 409857, + "prompt_tokens": 1423, + "completion_tokens": 5109, + "elapsed_seconds": 23.067031166996458, + "dispatch_time": 1791942867.355728 + }, + { + "id": "confirm-fresh-31-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-31-hi-m0", + "scenario": "fresh-31", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 431370, + "retained_microinr": 539213, + "prompt_tokens": 1914, + "completion_tokens": 6711, + "elapsed_seconds": 35.7180771669955, + "dispatch_time": 1791942868.017834 + }, + { + "id": "confirm-fresh-31-hi-m0-source-chronological", + "history_id": "confirm-fresh-31-hi-m0", + "scenario": "fresh-31", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 160380, + "retained_microinr": 200475, + "prompt_tokens": 1364, + "completion_tokens": 2332, + "elapsed_seconds": 11.674144791002618, + "dispatch_time": 1791942866.6890829 + }, + { + "id": "confirm-fresh-31-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-31-hi-m0", + "scenario": "fresh-31", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 135825, + "retained_microinr": 169782, + "prompt_tokens": 1747, + "completion_tokens": 1827, + "elapsed_seconds": 8.120365249997121, + "dispatch_time": 1791942903.7856011 + }, + { + "id": "confirm-fresh-33-te-m0-source-chronological", + "history_id": "confirm-fresh-33-te-m0", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 127785, + "retained_microinr": 159732, + "prompt_tokens": 1815, + "completion_tokens": 1676, + "elapsed_seconds": 9.552738250000402, + "dispatch_time": 1791942904.45278 + }, + { + "id": "confirm-fresh-33-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-33-te-m0", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 119805, + "retained_microinr": 149757, + "prompt_tokens": 2343, + "completion_tokens": 1411, + "elapsed_seconds": 9.608731583997724, + "dispatch_time": 1791942905.119431 + }, + { + "id": "confirm-fresh-33-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-te-m0", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 126030, + "retained_microinr": 157538, + "prompt_tokens": 2070, + "completion_tokens": 1583, + "elapsed_seconds": 7.178025500004878, + "dispatch_time": 1791942905.78611 + }, + { + "id": "confirm-fresh-25-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-25-hi-m1", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36540, + "retained_microinr": 45675, + "prompt_tokens": 2276, + "completion_tokens": 40, + "elapsed_seconds": 1.0159181250055553, + "dispatch_time": 1791942914.758796 + }, + { + "id": "confirm-fresh-25-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-25-hi-m1", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36540, + "retained_microinr": 45675, + "prompt_tokens": 2276, + "completion_tokens": 40, + "elapsed_seconds": 0.9898767919949023, + "dispatch_time": 1791942915.426935 + }, + { + "id": "confirm-fresh-25-hi-m1-source-chronological", + "history_id": "confirm-fresh-25-hi-m1", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24915, + "retained_microinr": 31144, + "prompt_tokens": 1501, + "completion_tokens": 40, + "elapsed_seconds": 0.9505093749903608, + "dispatch_time": 1791942916.093149 + }, + { + "id": "confirm-fresh-32-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-hi-m1", + "scenario": "fresh-32", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 34530, + "retained_microinr": 43163, + "prompt_tokens": 2134, + "completion_tokens": 42, + "elapsed_seconds": 0.9483322090090951, + "dispatch_time": 1791942916.759826 + }, + { + "id": "confirm-fresh-32-hi-m1-source-chronological", + "history_id": "confirm-fresh-32-hi-m1", + "scenario": "fresh-32", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24645, + "retained_microinr": 30807, + "prompt_tokens": 1483, + "completion_tokens": 40, + "elapsed_seconds": 0.9970947499969043, + "dispatch_time": 1791942919.736573 + }, + { + "id": "confirm-fresh-32-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-32-hi-m1", + "scenario": "fresh-32", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 34530, + "retained_microinr": 43163, + "prompt_tokens": 2134, + "completion_tokens": 42, + "elapsed_seconds": 0.94754966700566, + "dispatch_time": 1791942919.071709 + }, + { + "id": "confirm-fresh-15-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-15-hi-m1", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-16", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 23745, + "retained_microinr": 29682, + "prompt_tokens": 1415, + "completion_tokens": 42, + "elapsed_seconds": 1.0053978329960955, + "dispatch_time": 1791942918.401498 + }, + { + "id": "confirm-fresh-15-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-15-hi-m1", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25305, + "retained_microinr": 31632, + "prompt_tokens": 1559, + "completion_tokens": 32, + "elapsed_seconds": 0.9945954170107143, + "dispatch_time": 1791942917.73974 + }, + { + "id": "confirm-fresh-15-hi-m1-source-chronological", + "history_id": "confirm-fresh-15-hi-m1", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 17130, + "retained_microinr": 21413, + "prompt_tokens": 1014, + "completion_tokens": 32, + "elapsed_seconds": 0.9794596249994356, + "dispatch_time": 1791942920.7758029 + }, + { + "id": "confirm-fresh-35-hi-m1-source-chronological", + "history_id": "confirm-fresh-35-hi-m1", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 31425, + "retained_microinr": 39282, + "prompt_tokens": 1927, + "completion_tokens": 42, + "elapsed_seconds": 0.9489228330057813, + "dispatch_time": 1791942922.768995 + }, + { + "id": "confirm-fresh-35-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-hi-m1", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 37710, + "retained_microinr": 47138, + "prompt_tokens": 2346, + "completion_tokens": 42, + "elapsed_seconds": 1.0076109580113553, + "dispatch_time": 1791942922.104636 + }, + { + "id": "confirm-fresh-35-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-35-hi-m1", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 41580, + "retained_microinr": 51975, + "prompt_tokens": 2604, + "completion_tokens": 42, + "elapsed_seconds": 0.985458541996195, + "dispatch_time": 1791942921.439119 + }, + { + "id": "confirm-fresh-20-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-20-hi-m1", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 26235, + "retained_microinr": 32794, + "prompt_tokens": 1581, + "completion_tokens": 42, + "elapsed_seconds": 0.9312327090010513, + "dispatch_time": 1791942923.759349 + }, + { + "id": "confirm-fresh-20-hi-m1-source-chronological", + "history_id": "confirm-fresh-20-hi-m1", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 18300, + "retained_microinr": 22875, + "prompt_tokens": 1052, + "completion_tokens": 42, + "elapsed_seconds": 1.0072423749952577, + "dispatch_time": 1791942925.755979 + }, + { + "id": "confirm-fresh-20-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-20-hi-m1", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27165, + "retained_microinr": 33957, + "prompt_tokens": 1643, + "completion_tokens": 42, + "elapsed_seconds": 1.017433583008824, + "dispatch_time": 1791942924.424695 + }, + { + "id": "confirm-fresh-40-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-en-m1", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 33735, + "retained_microinr": 42169, + "prompt_tokens": 2081, + "completion_tokens": 42, + "elapsed_seconds": 0.9788850420009112, + "dispatch_time": 1791942925.094044 + }, + { + "id": "confirm-fresh-40-en-m1-source-chronological", + "history_id": "confirm-fresh-40-en-m1", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 30015, + "retained_microinr": 37519, + "prompt_tokens": 1833, + "completion_tokens": 42, + "elapsed_seconds": 1.0217821249971166, + "dispatch_time": 1791942928.877773 + }, + { + "id": "confirm-fresh-40-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-40-en-m1", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 38535, + "retained_microinr": 48169, + "prompt_tokens": 2401, + "completion_tokens": 42, + "elapsed_seconds": 0.9563269999925978, + "dispatch_time": 1791942927.543145 + }, + { + "id": "confirm-fresh-21-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-21-en-m0", + "scenario": "fresh-21", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 136950, + "retained_microinr": 171188, + "prompt_tokens": 1978, + "completion_tokens": 1788, + "elapsed_seconds": 9.55571850000706, + "dispatch_time": 1791942926.883471 + }, + { + "id": "confirm-fresh-21-en-m0-source-chronological", + "history_id": "confirm-fresh-21-en-m0", + "scenario": "fresh-21", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 97170, + "retained_microinr": 121463, + "prompt_tokens": 1358, + "completion_tokens": 1280, + "elapsed_seconds": 5.893801291007549, + "dispatch_time": 1791942928.2086852 + }, + { + "id": "confirm-fresh-21-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-21-en-m0", + "scenario": "fresh-21", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 281850, + "retained_microinr": 352313, + "prompt_tokens": 1978, + "completion_tokens": 4203, + "elapsed_seconds": 21.536080665988266, + "dispatch_time": 1791942937.933284 + }, + { + "id": "confirm-fresh-37-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-37-hi-m0", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 226500, + "retained_microinr": 283125, + "prompt_tokens": 2252, + "completion_tokens": 3212, + "elapsed_seconds": 16.394425874997978, + "dispatch_time": 1791942938.5982652 + }, + { + "id": "confirm-fresh-37-hi-m0-source-chronological", + "history_id": "confirm-fresh-37-hi-m0", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 179085, + "retained_microinr": 223857, + "prompt_tokens": 1723, + "completion_tokens": 2554, + "elapsed_seconds": 13.25975662500423, + "dispatch_time": 1791942937.266712 + }, + { + "id": "confirm-fresh-37-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-hi-m0", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 520695, + "retained_microinr": 650869, + "prompt_tokens": 1945, + "completion_tokens": 8192, + "elapsed_seconds": 37.77549379199627, + "dispatch_time": 1791942936.5975602 + }, + { + "id": "confirm-fresh-28-te-m0-source-chronological", + "history_id": "confirm-fresh-28-te-m0", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 51225, + "retained_microinr": 64032, + "prompt_tokens": 1339, + "completion_tokens": 519, + "elapsed_seconds": 3.13366279201, + "dispatch_time": 1791942975.745045 + }, + { + "id": "confirm-fresh-28-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-28-te-m0", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 439695, + "retained_microinr": 549619, + "prompt_tokens": 1837, + "completion_tokens": 6869, + "elapsed_seconds": 31.018074499996146, + "dispatch_time": 1791942974.410507 + }, + { + "id": "confirm-fresh-28-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-28-te-m0", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 277815, + "retained_microinr": 347269, + "prompt_tokens": 1837, + "completion_tokens": 4171, + "elapsed_seconds": 23.696498665987747, + "dispatch_time": 1791942976.411817 + }, + { + "id": "confirm-fresh-07-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-te-m0", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 235680, + "retained_microinr": 294600, + "prompt_tokens": 1400, + "completion_tokens": 3578, + "elapsed_seconds": 19.214640292004333, + "dispatch_time": 1791942975.078476 + }, + { + "id": "confirm-fresh-07-te-m0-source-chronological", + "history_id": "confirm-fresh-07-te-m0", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 203865, + "retained_microinr": 254832, + "prompt_tokens": 979, + "completion_tokens": 3153, + "elapsed_seconds": 19.295718249995844, + "dispatch_time": 1791943005.460192 + }, + { + "id": "confirm-fresh-07-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-07-te-m0", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 375165, + "retained_microinr": 468957, + "prompt_tokens": 1527, + "completion_tokens": 5871, + "elapsed_seconds": 27.6655987080012, + "dispatch_time": 1791943007.459328 + }, + { + "id": "confirm-fresh-09-te-m0-source-chronological", + "history_id": "confirm-fresh-09-te-m0", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 121800, + "retained_microinr": 152250, + "prompt_tokens": 1056, + "completion_tokens": 1766, + "elapsed_seconds": 9.148343624998233, + "dispatch_time": 1791943006.790838 + }, + { + "id": "confirm-fresh-09-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-09-te-m0", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 134400, + "retained_microinr": 168000, + "prompt_tokens": 1600, + "completion_tokens": 1840, + "elapsed_seconds": 12.543137667002156, + "dispatch_time": 1791943006.12867 + }, + { + "id": "confirm-fresh-09-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-09-te-m0", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 193215, + "retained_microinr": 241519, + "prompt_tokens": 1405, + "completion_tokens": 2869, + "elapsed_seconds": 17.790086916997097, + "dispatch_time": 1791943035.1651268 + }, + { + "id": "confirm-fresh-03-te-m1-source-chronological", + "history_id": "confirm-fresh-03-te-m1", + "scenario": "fresh-03", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 15825, + "retained_microinr": 19782, + "prompt_tokens": 887, + "completion_tokens": 42, + "elapsed_seconds": 0.951432209010818, + "dispatch_time": 1791943036.500198 + }, + { + "id": "confirm-fresh-03-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-03-te-m1", + "scenario": "fresh-03", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 23700, + "retained_microinr": 29625, + "prompt_tokens": 1412, + "completion_tokens": 42, + "elapsed_seconds": 1.0183845830033533, + "dispatch_time": 1791943037.163568 + }, + { + "id": "confirm-fresh-03-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-03-te-m1", + "scenario": "fresh-03", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24045, + "retained_microinr": 30057, + "prompt_tokens": 1435, + "completion_tokens": 42, + "elapsed_seconds": 1.0147983750066487, + "dispatch_time": 1791943035.830751 + }, + { + "id": "confirm-fresh-03-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-03-en-m1", + "scenario": "fresh-03", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 30450, + "retained_microinr": 38063, + "prompt_tokens": 1862, + "completion_tokens": 42, + "elapsed_seconds": 1.0982950840116246, + "dispatch_time": 1791943054.995341 + }, + { + "id": "confirm-fresh-03-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-03-en-m1", + "scenario": "fresh-03", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": true, + "raw_status_correct": true, + "known_microinr": 27360, + "retained_microinr": 34200, + "prompt_tokens": 1696, + "completion_tokens": 32, + "elapsed_seconds": 1.008163708000211, + "dispatch_time": 1791943053.6620018 + }, + { + "id": "confirm-fresh-03-en-m1-source-chronological", + "history_id": "confirm-fresh-03-en-m1", + "scenario": "fresh-03", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 19905, + "retained_microinr": 24882, + "prompt_tokens": 1159, + "completion_tokens": 42, + "elapsed_seconds": 0.9332312920014374, + "dispatch_time": 1791943054.323953 + }, + { + "id": "confirm-fresh-28-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-28-en-m1", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 33990, + "retained_microinr": 42488, + "prompt_tokens": 2106, + "completion_tokens": 40, + "elapsed_seconds": 1.0081932500033872, + "dispatch_time": 1791943052.995065 + }, + { + "id": "confirm-fresh-28-en-m1-source-chronological", + "history_id": "confirm-fresh-28-en-m1", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24210, + "retained_microinr": 30263, + "prompt_tokens": 1454, + "completion_tokens": 40, + "elapsed_seconds": 0.9394343329913681, + "dispatch_time": 1791943057.465611 + }, + { + "id": "confirm-fresh-28-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-28-en-m1", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 33990, + "retained_microinr": 42488, + "prompt_tokens": 2106, + "completion_tokens": 40, + "elapsed_seconds": 1.0145768340007635, + "dispatch_time": 1791943056.134769 + }, + { + "id": "confirm-fresh-22-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-22-hi-m0", + "scenario": "fresh-22", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 209730, + "retained_microinr": 262163, + "prompt_tokens": 1826, + "completion_tokens": 3039, + "elapsed_seconds": 13.696418833002099, + "dispatch_time": 1791943058.133663 + }, + { + "id": "confirm-fresh-22-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-22-hi-m0", + "scenario": "fresh-22", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 310170, + "retained_microinr": 387713, + "prompt_tokens": 1826, + "completion_tokens": 4713, + "elapsed_seconds": 21.88700395800697, + "dispatch_time": 1791943056.800215 + }, + { + "id": "confirm-fresh-22-hi-m0-source-chronological", + "history_id": "confirm-fresh-22-hi-m0", + "scenario": "fresh-22", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 149895, + "retained_microinr": 187369, + "prompt_tokens": 1317, + "completion_tokens": 2169, + "elapsed_seconds": 9.832497875002446, + "dispatch_time": 1791943079.398384 + }, + { + "id": "confirm-fresh-04-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-04-en-m0", + "scenario": "fresh-04", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 173010, + "retained_microinr": 216263, + "prompt_tokens": 1662, + "completion_tokens": 2468, + "elapsed_seconds": 12.226466457999777, + "dispatch_time": 1791943080.0642219 + }, + { + "id": "confirm-fresh-04-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-04-en-m0", + "scenario": "fresh-04", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 341910, + "retained_microinr": 427388, + "prompt_tokens": 1530, + "completion_tokens": 5316, + "elapsed_seconds": 26.199195125009282, + "dispatch_time": 1791943078.733022 + }, + { + "id": "confirm-fresh-04-en-m0-source-chronological", + "history_id": "confirm-fresh-04-en-m0", + "scenario": "fresh-04", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 356370, + "retained_microinr": 445463, + "prompt_tokens": 1086, + "completion_tokens": 5668, + "elapsed_seconds": 24.645635833003325, + "dispatch_time": 1791943080.731762 + }, + { + "id": "confirm-fresh-16-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-16-en-m0", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 247125, + "retained_microinr": 308907, + "prompt_tokens": 1371, + "completion_tokens": 3776, + "elapsed_seconds": 15.747625625008368, + "dispatch_time": 1791943106.090813 + }, + { + "id": "confirm-fresh-16-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-16-en-m0", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 298515, + "retained_microinr": 373144, + "prompt_tokens": 1269, + "completion_tokens": 4658, + "elapsed_seconds": 24.507196416001534, + "dispatch_time": 1791943107.423403 + }, + { + "id": "confirm-fresh-16-en-m0-source-chronological", + "history_id": "confirm-fresh-16-en-m0", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 237960, + "retained_microinr": 297450, + "prompt_tokens": 928, + "completion_tokens": 3734, + "elapsed_seconds": 16.229188957993756, + "dispatch_time": 1791943106.7558668 + }, + { + "id": "confirm-fresh-14-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-14-te-m0", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 137460, + "retained_microinr": 171825, + "prompt_tokens": 1264, + "completion_tokens": 1975, + "elapsed_seconds": 8.641674832993886, + "dispatch_time": 1791943105.4236228 + }, + { + "id": "confirm-fresh-14-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-14-te-m0", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 430860, + "retained_microinr": 538575, + "prompt_tokens": 1540, + "completion_tokens": 6796, + "elapsed_seconds": 28.325404834002256, + "dispatch_time": 1791943132.6323898 + }, + { + "id": "confirm-fresh-14-te-m0-source-chronological", + "history_id": "confirm-fresh-14-te-m0", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 266775, + "retained_microinr": 333469, + "prompt_tokens": 1101, + "completion_tokens": 4171, + "elapsed_seconds": 17.552688708005007, + "dispatch_time": 1791943133.9662561 + }, + { + "id": "confirm-fresh-02-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-02-hi-m1", + "scenario": "fresh-02", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 21330, + "retained_microinr": 26663, + "prompt_tokens": 1254, + "completion_tokens": 42, + "elapsed_seconds": 0.9385331249941373, + "dispatch_time": 1791943133.299575 + }, + { + "id": "confirm-fresh-02-hi-m1-source-chronological", + "history_id": "confirm-fresh-02-hi-m1", + "scenario": "fresh-02", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 15015, + "retained_microinr": 18769, + "prompt_tokens": 833, + "completion_tokens": 42, + "elapsed_seconds": 0.9410805420047836, + "dispatch_time": 1791943131.965833 + }, + { + "id": "confirm-fresh-02-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-02-hi-m1", + "scenario": "fresh-02", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 21525, + "retained_microinr": 26907, + "prompt_tokens": 1267, + "completion_tokens": 42, + "elapsed_seconds": 1.0299618330027442, + "dispatch_time": 1791943161.658289 + }, + { + "id": "confirm-fresh-15-te-m0-source-chronological", + "history_id": "confirm-fresh-15-te-m0", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 378150, + "retained_microinr": 472688, + "prompt_tokens": 990, + "completion_tokens": 6055, + "elapsed_seconds": 24.841930750000756, + "dispatch_time": 1791943160.992556 + }, + { + "id": "confirm-fresh-15-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-15-te-m0", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 111015, + "retained_microinr": 138769, + "prompt_tokens": 1533, + "completion_tokens": 1467, + "elapsed_seconds": 8.45636095800728, + "dispatch_time": 1791943162.9925592 + }, + { + "id": "confirm-fresh-15-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-15-te-m0", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 184740, + "retained_microinr": 230925, + "prompt_tokens": 1408, + "completion_tokens": 2727, + "elapsed_seconds": 11.572422540994012, + "dispatch_time": 1791943162.32596 + }, + { + "id": "confirm-fresh-19-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-19-hi-m0", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 322455, + "retained_microinr": 403069, + "prompt_tokens": 1757, + "completion_tokens": 4935, + "elapsed_seconds": 23.170734041006654, + "dispatch_time": 1791943186.531148 + }, + { + "id": "confirm-fresh-19-hi-m0-source-chronological", + "history_id": "confirm-fresh-19-hi-m0", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 295275, + "retained_microinr": 369094, + "prompt_tokens": 1061, + "completion_tokens": 4656, + "elapsed_seconds": 23.127932041999884, + "dispatch_time": 1791943185.8633258 + }, + { + "id": "confirm-fresh-19-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-hi-m0", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-04-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 323355, + "retained_microinr": 404194, + "prompt_tokens": 1721, + "completion_tokens": 4959, + "elapsed_seconds": 22.36946070799604, + "dispatch_time": 1791943187.8645508 + }, + { + "id": "confirm-fresh-15-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-15-en-m0", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 180615, + "retained_microinr": 225769, + "prompt_tokens": 1525, + "completion_tokens": 2629, + "elapsed_seconds": 13.286117167008342, + "dispatch_time": 1791943187.198241 + }, + { + "id": "confirm-fresh-15-en-m0-source-chronological", + "history_id": "confirm-fresh-15-en-m0", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 138885, + "retained_microinr": 173607, + "prompt_tokens": 983, + "completion_tokens": 2069, + "elapsed_seconds": 9.08597883299808, + "dispatch_time": 1791943210.2742112 + }, + { + "id": "confirm-fresh-15-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-15-en-m0", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 197205, + "retained_microinr": 246507, + "prompt_tokens": 1411, + "completion_tokens": 2934, + "elapsed_seconds": 12.364997417011182, + "dispatch_time": 1791943211.608254 + }, + { + "id": "confirm-fresh-24-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-24-en-m0", + "scenario": "fresh-24", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 94365, + "retained_microinr": 117957, + "prompt_tokens": 1839, + "completion_tokens": 1113, + "elapsed_seconds": 5.330106167006306, + "dispatch_time": 1791943210.9381602 + }, + { + "id": "confirm-fresh-24-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-24-en-m0", + "scenario": "fresh-24", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 265065, + "retained_microinr": 331332, + "prompt_tokens": 1839, + "completion_tokens": 3958, + "elapsed_seconds": 16.913822749993415, + "dispatch_time": 1791943212.272355 + }, + { + "id": "confirm-fresh-24-en-m0-source-chronological", + "history_id": "confirm-fresh-24-en-m0", + "scenario": "fresh-24", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 103800, + "retained_microinr": 129750, + "prompt_tokens": 1320, + "completion_tokens": 1400, + "elapsed_seconds": 6.214337208002689, + "dispatch_time": 1791943229.219827 + }, + { + "id": "confirm-fresh-32-te-m1-source-chronological", + "history_id": "confirm-fresh-32-te-m1", + "scenario": "fresh-32", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 28680, + "retained_microinr": 35850, + "prompt_tokens": 1744, + "completion_tokens": 42, + "elapsed_seconds": 0.9329089170059888, + "dispatch_time": 1791943230.5511081 + }, + { + "id": "confirm-fresh-32-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-32-te-m1", + "scenario": "fresh-32", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 37380, + "retained_microinr": 46725, + "prompt_tokens": 2324, + "completion_tokens": 42, + "elapsed_seconds": 0.9693154159904225, + "dispatch_time": 1791943229.887452 + }, + { + "id": "confirm-fresh-32-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-te-m1", + "scenario": "fresh-32", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 30735, + "retained_microinr": 38419, + "prompt_tokens": 1881, + "completion_tokens": 42, + "elapsed_seconds": 1.0257342499971855, + "dispatch_time": 1791943231.218207 + }, + { + "id": "confirm-fresh-22-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-22-te-m1", + "scenario": "fresh-22", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34245, + "retained_microinr": 42807, + "prompt_tokens": 2123, + "completion_tokens": 40, + "elapsed_seconds": 0.9548690419906052, + "dispatch_time": 1791943235.5148852 + }, + { + "id": "confirm-fresh-22-te-m1-source-chronological", + "history_id": "confirm-fresh-22-te-m1", + "scenario": "fresh-22", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24510, + "retained_microinr": 30638, + "prompt_tokens": 1474, + "completion_tokens": 40, + "elapsed_seconds": 1.0191509160067653, + "dispatch_time": 1791943237.5172029 + }, + { + "id": "confirm-fresh-22-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-22-te-m1", + "scenario": "fresh-22", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34245, + "retained_microinr": 42807, + "prompt_tokens": 2123, + "completion_tokens": 40, + "elapsed_seconds": 0.9699667910026619, + "dispatch_time": 1791943236.181613 + }, + { + "id": "confirm-fresh-08-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-08-en-m1", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27060, + "retained_microinr": 33825, + "prompt_tokens": 1676, + "completion_tokens": 32, + "elapsed_seconds": 1.0592645420110784, + "dispatch_time": 1791943236.848316 + }, + { + "id": "confirm-fresh-08-en-m1-source-chronological", + "history_id": "confirm-fresh-08-en-m1", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 18330, + "retained_microinr": 22913, + "prompt_tokens": 1094, + "completion_tokens": 32, + "elapsed_seconds": 1.005081416995381, + "dispatch_time": 1791943238.577937 + }, + { + "id": "confirm-fresh-08-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-08-en-m1", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24915, + "retained_microinr": 31144, + "prompt_tokens": 1533, + "completion_tokens": 32, + "elapsed_seconds": 0.9301751669991063, + "dispatch_time": 1791943239.910572 + }, + { + "id": "confirm-fresh-34-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-en-m1", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 35355, + "retained_microinr": 44194, + "prompt_tokens": 2189, + "completion_tokens": 42, + "elapsed_seconds": 1.0531886250100797, + "dispatch_time": 1791943239.240457 + }, + { + "id": "confirm-fresh-34-en-m1-source-chronological", + "history_id": "confirm-fresh-34-en-m1", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 32160, + "retained_microinr": 40200, + "prompt_tokens": 1976, + "completion_tokens": 42, + "elapsed_seconds": 1.10543416699511, + "dispatch_time": 1791943240.577135 + }, + { + "id": "confirm-fresh-34-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-34-en-m1", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 40815, + "retained_microinr": 51019, + "prompt_tokens": 2553, + "completion_tokens": 42, + "elapsed_seconds": 0.9658427920076065, + "dispatch_time": 1791943243.0524192 + }, + { + "id": "confirm-fresh-37-en-m1-source-chronological", + "history_id": "confirm-fresh-37-en-m1", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 29175, + "retained_microinr": 36469, + "prompt_tokens": 1777, + "completion_tokens": 42, + "elapsed_seconds": 1.1151200840104138, + "dispatch_time": 1791943243.715254 + }, + { + "id": "confirm-fresh-37-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-en-m1", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 36120, + "retained_microinr": 45150, + "prompt_tokens": 2240, + "completion_tokens": 42, + "elapsed_seconds": 1.015421415999299, + "dispatch_time": 1791943241.7174559 + }, + { + "id": "confirm-fresh-37-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-37-en-m1", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 39480, + "retained_microinr": 49350, + "prompt_tokens": 2464, + "completion_tokens": 42, + "elapsed_seconds": 1.054733375000069, + "dispatch_time": 1791943242.385756 + }, + { + "id": "confirm-fresh-36-hi-m0-source-chronological", + "history_id": "confirm-fresh-36-hi-m0", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 196935, + "retained_microinr": 246169, + "prompt_tokens": 1741, + "completion_tokens": 2847, + "elapsed_seconds": 17.856215332998545, + "dispatch_time": 1791943246.199415 + }, + { + "id": "confirm-fresh-36-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-hi-m0", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 322245, + "retained_microinr": 402807, + "prompt_tokens": 1915, + "completion_tokens": 4892, + "elapsed_seconds": 27.32137474999763, + "dispatch_time": 1791943246.866103 + }, + { + "id": "confirm-fresh-36-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-36-hi-m0", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 120165, + "retained_microinr": 150207, + "prompt_tokens": 2163, + "completion_tokens": 1462, + "elapsed_seconds": 6.776234791002935, + "dispatch_time": 1791943244.869296 + }, + { + "id": "confirm-fresh-10-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-10-en-m0", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 370215, + "retained_microinr": 462769, + "prompt_tokens": 1333, + "completion_tokens": 5837, + "elapsed_seconds": 31.27474337500462, + "dispatch_time": 1791943245.534525 + }, + { + "id": "confirm-fresh-10-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-10-en-m0", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 112800, + "retained_microinr": 141000, + "prompt_tokens": 1256, + "completion_tokens": 1566, + "elapsed_seconds": 10.808582208992448, + "dispatch_time": 1791943278.1819751 + }, + { + "id": "confirm-fresh-10-en-m0-source-chronological", + "history_id": "confirm-fresh-10-en-m0", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 96900, + "retained_microinr": 121125, + "prompt_tokens": 896, + "completion_tokens": 1391, + "elapsed_seconds": 7.540584332993603, + "dispatch_time": 1791943277.518649 + }, + { + "id": "confirm-fresh-09-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-09-en-m0", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 144165, + "retained_microinr": 180207, + "prompt_tokens": 1443, + "completion_tokens": 2042, + "elapsed_seconds": 9.118661833010265, + "dispatch_time": 1791943276.853749 + }, + { + "id": "confirm-fresh-09-en-m0-source-chronological", + "history_id": "confirm-fresh-09-en-m0", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 366030, + "retained_microinr": 457538, + "prompt_tokens": 902, + "completion_tokens": 5875, + "elapsed_seconds": 24.437354834008147, + "dispatch_time": 1791943278.84978 + }, + { + "id": "confirm-fresh-09-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-09-en-m0", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 141000, + "retained_microinr": 176250, + "prompt_tokens": 1404, + "completion_tokens": 1999, + "elapsed_seconds": 13.674536040998646, + "dispatch_time": 1791943304.6738229 + }, + { + "id": "confirm-fresh-10-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-10-hi-m0", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 295815, + "retained_microinr": 369769, + "prompt_tokens": 1253, + "completion_tokens": 4617, + "elapsed_seconds": 19.370352083002217, + "dispatch_time": 1791943305.3440251 + }, + { + "id": "confirm-fresh-10-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-10-hi-m0", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 126855, + "retained_microinr": 158569, + "prompt_tokens": 1309, + "completion_tokens": 1787, + "elapsed_seconds": 7.972530416998779, + "dispatch_time": 1791943304.0069098 + }, + { + "id": "confirm-fresh-10-hi-m0-source-chronological", + "history_id": "confirm-fresh-10-hi-m0", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 193635, + "retained_microinr": 242044, + "prompt_tokens": 873, + "completion_tokens": 3009, + "elapsed_seconds": 14.16674320900347, + "dispatch_time": 1791943303.340543 + }, + { + "id": "confirm-fresh-14-te-m1-source-chronological", + "history_id": "confirm-fresh-14-te-m1", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-15", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 19410, + "retained_microinr": 24263, + "prompt_tokens": 1126, + "completion_tokens": 42, + "elapsed_seconds": 0.9414785420085536, + "dispatch_time": 1791943324.8437078 + }, + { + "id": "confirm-fresh-14-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-14-te-m1", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-15", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 28155, + "retained_microinr": 35194, + "prompt_tokens": 1709, + "completion_tokens": 42, + "elapsed_seconds": 0.967218708989094, + "dispatch_time": 1791943326.172512 + }, + { + "id": "confirm-fresh-14-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-14-te-m1", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25170, + "retained_microinr": 31463, + "prompt_tokens": 1550, + "completion_tokens": 32, + "elapsed_seconds": 0.971434709004825, + "dispatch_time": 1791943326.839175 + }, + { + "id": "confirm-fresh-30-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-30-hi-m1", + "scenario": "fresh-30", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34545, + "retained_microinr": 43182, + "prompt_tokens": 2143, + "completion_tokens": 40, + "elapsed_seconds": 1.103579291011556, + "dispatch_time": 1791943325.503287 + }, + { + "id": "confirm-fresh-30-hi-m1-source-chronological", + "history_id": "confirm-fresh-30-hi-m1", + "scenario": "fresh-30", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24630, + "retained_microinr": 30788, + "prompt_tokens": 1482, + "completion_tokens": 40, + "elapsed_seconds": 1.0088979579886654, + "dispatch_time": 1791943327.864049 + }, + { + "id": "confirm-fresh-30-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-hi-m1", + "scenario": "fresh-30", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34545, + "retained_microinr": 43182, + "prompt_tokens": 2143, + "completion_tokens": 40, + "elapsed_seconds": 0.9637627079937374, + "dispatch_time": 1791943328.528097 + }, + { + "id": "confirm-fresh-31-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-31-hi-m1", + "scenario": "fresh-31", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36600, + "retained_microinr": 45750, + "prompt_tokens": 2280, + "completion_tokens": 40, + "elapsed_seconds": 0.9493672079988755, + "dispatch_time": 1791943329.863227 + }, + { + "id": "confirm-fresh-31-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-31-hi-m1", + "scenario": "fresh-31", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36600, + "retained_microinr": 45750, + "prompt_tokens": 2280, + "completion_tokens": 40, + "elapsed_seconds": 1.0902935000049183, + "dispatch_time": 1791943329.196558 + }, + { + "id": "confirm-fresh-31-hi-m1-source-chronological", + "history_id": "confirm-fresh-31-hi-m1", + "scenario": "fresh-31", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25230, + "retained_microinr": 31538, + "prompt_tokens": 1522, + "completion_tokens": 40, + "elapsed_seconds": 0.9350680000061402, + "dispatch_time": 1791943332.187031 + }, + { + "id": "confirm-fresh-39-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-39-en-m1", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 38820, + "retained_microinr": 48525, + "prompt_tokens": 2420, + "completion_tokens": 42, + "elapsed_seconds": 1.0383790409978246, + "dispatch_time": 1791943331.520316 + }, + { + "id": "confirm-fresh-39-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-en-m1", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 35895, + "retained_microinr": 44869, + "prompt_tokens": 2225, + "completion_tokens": 42, + "elapsed_seconds": 0.9986680000001797, + "dispatch_time": 1791943330.858967 + }, + { + "id": "confirm-fresh-39-en-m1-source-chronological", + "history_id": "confirm-fresh-39-en-m1", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 28755, + "retained_microinr": 35944, + "prompt_tokens": 1749, + "completion_tokens": 42, + "elapsed_seconds": 1.1048080409964314, + "dispatch_time": 1791943332.850823 + }, + { + "id": "confirm-fresh-21-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-21-en-m1", + "scenario": "fresh-21", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36345, + "retained_microinr": 45432, + "prompt_tokens": 2263, + "completion_tokens": 40, + "elapsed_seconds": 0.9594374159933068, + "dispatch_time": 1791943335.327013 + }, + { + "id": "confirm-fresh-21-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-21-en-m1", + "scenario": "fresh-21", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36345, + "retained_microinr": 45432, + "prompt_tokens": 2263, + "completion_tokens": 40, + "elapsed_seconds": 1.1167157919990132, + "dispatch_time": 1791943335.989734 + }, + { + "id": "confirm-fresh-21-en-m1-source-chronological", + "history_id": "confirm-fresh-21-en-m1", + "scenario": "fresh-21", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24960, + "retained_microinr": 31200, + "prompt_tokens": 1504, + "completion_tokens": 40, + "elapsed_seconds": 1.0056834999995772, + "dispatch_time": 1791943334.6603498 + }, + { + "id": "confirm-fresh-13-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-13-te-m1", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27390, + "retained_microinr": 34238, + "prompt_tokens": 1698, + "completion_tokens": 32, + "elapsed_seconds": 1.0096665830060374, + "dispatch_time": 1791943333.994106 + }, + { + "id": "confirm-fresh-13-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-13-te-m1", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 30000, + "retained_microinr": 37500, + "prompt_tokens": 1872, + "completion_tokens": 32, + "elapsed_seconds": 1.1219505830085836, + "dispatch_time": 1791943339.129771 + }, + { + "id": "confirm-fresh-13-te-m1-source-chronological", + "history_id": "confirm-fresh-13-te-m1", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 20145, + "retained_microinr": 25182, + "prompt_tokens": 1175, + "completion_tokens": 42, + "elapsed_seconds": 0.9357774579984834, + "dispatch_time": 1791943337.794431 + }, + { + "id": "confirm-fresh-21-te-m1-source-chronological", + "history_id": "confirm-fresh-21-te-m1", + "scenario": "fresh-21", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25110, + "retained_microinr": 31388, + "prompt_tokens": 1514, + "completion_tokens": 40, + "elapsed_seconds": 0.9338703329995042, + "dispatch_time": 1791943337.1267898 + }, + { + "id": "confirm-fresh-21-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-21-te-m1", + "scenario": "fresh-21", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36390, + "retained_microinr": 45488, + "prompt_tokens": 2266, + "completion_tokens": 40, + "elapsed_seconds": 0.9539970839978196, + "dispatch_time": 1791943338.4618711 + }, + { + "id": "confirm-fresh-21-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-21-te-m1", + "scenario": "fresh-21", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36390, + "retained_microinr": 45488, + "prompt_tokens": 2266, + "completion_tokens": 40, + "elapsed_seconds": 0.974952583987033, + "dispatch_time": 1791943340.937568 + }, + { + "id": "confirm-fresh-20-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-20-hi-m0", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 511740, + "retained_microinr": 639675, + "prompt_tokens": 1348, + "completion_tokens": 8192, + "elapsed_seconds": 40.547644749996834, + "dispatch_time": 1791943340.271421 + }, + { + "id": "confirm-fresh-20-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-20-hi-m0", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 92535, + "retained_microinr": 115669, + "prompt_tokens": 1285, + "completion_tokens": 1221, + "elapsed_seconds": 6.446608832993661, + "dispatch_time": 1791943342.27357 + }, + { + "id": "confirm-fresh-20-hi-m0-source-chronological", + "history_id": "confirm-fresh-20-hi-m0", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 283185, + "retained_microinr": 353982, + "prompt_tokens": 911, + "completion_tokens": 4492, + "elapsed_seconds": 23.06852745800279, + "dispatch_time": 1791943341.604129 + }, + { + "id": "confirm-fresh-19-en-m0-source-chronological", + "history_id": "confirm-fresh-19-en-m0", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 431100, + "retained_microinr": 538875, + "prompt_tokens": 1100, + "completion_tokens": 6910, + "elapsed_seconds": 28.784824792004656, + "dispatch_time": 1791943381.527215 + }, + { + "id": "confirm-fresh-19-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-19-en-m0", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 518295, + "retained_microinr": 647869, + "prompt_tokens": 1785, + "completion_tokens": 8192, + "elapsed_seconds": 38.58736979099922, + "dispatch_time": 1791943380.859309 + }, + { + "id": "confirm-fresh-19-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-en-m0", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 136770, + "retained_microinr": 170963, + "prompt_tokens": 1706, + "completion_tokens": 1853, + "elapsed_seconds": 8.911343084007967, + "dispatch_time": 1791943382.1898298 + }, + { + "id": "confirm-fresh-38-te-m1-source-chronological", + "history_id": "confirm-fresh-38-te-m1", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 29895, + "retained_microinr": 37369, + "prompt_tokens": 1833, + "completion_tokens": 40, + "elapsed_seconds": 0.9656026250013383, + "dispatch_time": 1791943382.859552 + }, + { + "id": "confirm-fresh-38-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-38-te-m1", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 38670, + "retained_microinr": 48338, + "prompt_tokens": 2418, + "completion_tokens": 40, + "elapsed_seconds": 1.0040276660001837, + "dispatch_time": 1791943419.483791 + }, + { + "id": "confirm-fresh-38-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-te-m1", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 34200, + "retained_microinr": 42750, + "prompt_tokens": 2112, + "completion_tokens": 42, + "elapsed_seconds": 1.0110378749959636, + "dispatch_time": 1791943420.151362 + }, + { + "id": "confirm-fresh-35-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-en-m1", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 37650, + "retained_microinr": 47063, + "prompt_tokens": 2342, + "completion_tokens": 42, + "elapsed_seconds": 1.958510665994254, + "dispatch_time": 1791943420.818026 + }, + { + "id": "confirm-fresh-35-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-35-en-m1", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 40635, + "retained_microinr": 50794, + "prompt_tokens": 2541, + "completion_tokens": 42, + "elapsed_seconds": 1.0844774160068482, + "dispatch_time": 1791943421.484698 + }, + { + "id": "confirm-fresh-35-en-m1-source-chronological", + "history_id": "confirm-fresh-35-en-m1", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 30360, + "retained_microinr": 37950, + "prompt_tokens": 1856, + "completion_tokens": 42, + "elapsed_seconds": 1.0406185000028927, + "dispatch_time": 1791943424.1404061 + }, + { + "id": "confirm-fresh-16-te-m0-source-chronological", + "history_id": "confirm-fresh-16-te-m0", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 504795, + "retained_microinr": 630994, + "prompt_tokens": 885, + "completion_tokens": 8192, + "elapsed_seconds": 33.825618667004164, + "dispatch_time": 1791943424.8070679 + }, + { + "id": "confirm-fresh-16-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-16-te-m0", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 143115, + "retained_microinr": 178894, + "prompt_tokens": 1265, + "completion_tokens": 2069, + "elapsed_seconds": 9.044318750005914, + "dispatch_time": 1791943422.804932 + }, + { + "id": "confirm-fresh-16-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-16-te-m0", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 224175, + "retained_microinr": 280219, + "prompt_tokens": 1325, + "completion_tokens": 3405, + "elapsed_seconds": 14.300099958010833, + "dispatch_time": 1791943423.4737332 + }, + { + "id": "confirm-fresh-15-te-m1-source-chronological", + "history_id": "confirm-fresh-15-te-m1", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 20130, + "retained_microinr": 25163, + "prompt_tokens": 1174, + "completion_tokens": 42, + "elapsed_seconds": 1.131959584003198, + "dispatch_time": 1791943458.6571891 + }, + { + "id": "confirm-fresh-15-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-15-te-m1", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27255, + "retained_microinr": 34069, + "prompt_tokens": 1689, + "completion_tokens": 32, + "elapsed_seconds": 1.2939245830057189, + "dispatch_time": 1791943460.6583638 + }, + { + "id": "confirm-fresh-15-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-15-te-m1", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-16", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 30390, + "retained_microinr": 37988, + "prompt_tokens": 1858, + "completion_tokens": 42, + "elapsed_seconds": 0.9855014170025242, + "dispatch_time": 1791943459.324399 + }, + { + "id": "confirm-fresh-05-hi-m1-source-chronological", + "history_id": "confirm-fresh-05-hi-m1", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 17055, + "retained_microinr": 21319, + "prompt_tokens": 1009, + "completion_tokens": 32, + "elapsed_seconds": 1.278489417003584, + "dispatch_time": 1791943459.988178 + }, + { + "id": "confirm-fresh-05-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-05-hi-m1", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27045, + "retained_microinr": 33807, + "prompt_tokens": 1675, + "completion_tokens": 32, + "elapsed_seconds": 0.9504380830039736, + "dispatch_time": 1791943461.985468 + }, + { + "id": "confirm-fresh-05-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-05-hi-m1", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27450, + "retained_microinr": 34313, + "prompt_tokens": 1702, + "completion_tokens": 32, + "elapsed_seconds": 0.948037000009208, + "dispatch_time": 1791943462.653909 + }, + { + "id": "confirm-fresh-33-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-hi-m1", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 37995, + "retained_microinr": 47494, + "prompt_tokens": 2365, + "completion_tokens": 42, + "elapsed_seconds": 1.0529548339982284, + "dispatch_time": 1791943463.985966 + }, + { + "id": "confirm-fresh-33-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-33-hi-m1", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 42645, + "retained_microinr": 53307, + "prompt_tokens": 2675, + "completion_tokens": 42, + "elapsed_seconds": 0.9673166660068091, + "dispatch_time": 1791943463.319705 + }, + { + "id": "confirm-fresh-33-hi-m1-source-chronological", + "history_id": "confirm-fresh-33-hi-m1", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 32220, + "retained_microinr": 40275, + "prompt_tokens": 1980, + "completion_tokens": 42, + "elapsed_seconds": 0.9630600000091363, + "dispatch_time": 1791943466.411408 + }, + { + "id": "confirm-fresh-05-hi-m0-source-chronological", + "history_id": "confirm-fresh-05-hi-m0", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 419580, + "retained_microinr": 524475, + "prompt_tokens": 896, + "completion_tokens": 6769, + "elapsed_seconds": 27.827692208011285, + "dispatch_time": 1791943465.0744479 + }, + { + "id": "confirm-fresh-05-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-05-hi-m0", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 123600, + "retained_microinr": 154500, + "prompt_tokens": 1448, + "completion_tokens": 1698, + "elapsed_seconds": 9.330110625000088, + "dispatch_time": 1791943465.7385972 + }, + { + "id": "confirm-fresh-05-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-05-hi-m0", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 185850, + "retained_microinr": 232313, + "prompt_tokens": 1398, + "completion_tokens": 2748, + "elapsed_seconds": 11.720331250005984, + "dispatch_time": 1791943467.075426 + }, + { + "id": "confirm-fresh-40-hi-m1-source-chronological", + "history_id": "confirm-fresh-40-hi-m1", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 33060, + "retained_microinr": 41325, + "prompt_tokens": 2036, + "completion_tokens": 42, + "elapsed_seconds": 1.0304211670008954, + "dispatch_time": 1791943494.930929 + }, + { + "id": "confirm-fresh-40-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-40-hi-m1", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 41715, + "retained_microinr": 52144, + "prompt_tokens": 2613, + "completion_tokens": 42, + "elapsed_seconds": 0.9814349160005804, + "dispatch_time": 1791943493.600812 + }, + { + "id": "confirm-fresh-40-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-hi-m1", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 34065, + "retained_microinr": 42582, + "prompt_tokens": 2103, + "completion_tokens": 42, + "elapsed_seconds": 0.9619969159975881, + "dispatch_time": 1791943494.267478 + }, + { + "id": "confirm-fresh-21-te-m0-source-chronological", + "history_id": "confirm-fresh-21-te-m0", + "scenario": "fresh-21", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 55350, + "retained_microinr": 69188, + "prompt_tokens": 1370, + "completion_tokens": 580, + "elapsed_seconds": 3.5836629169934895, + "dispatch_time": 1791943492.931736 + }, + { + "id": "confirm-fresh-21-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-21-te-m0", + "scenario": "fresh-21", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 484515, + "retained_microinr": 605644, + "prompt_tokens": 1973, + "completion_tokens": 7582, + "elapsed_seconds": 36.53783091700461, + "dispatch_time": 1791943498.538319 + }, + { + "id": "confirm-fresh-21-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-21-te-m0", + "scenario": "fresh-21", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 173115, + "retained_microinr": 216394, + "prompt_tokens": 1973, + "completion_tokens": 2392, + "elapsed_seconds": 10.647001292003551, + "dispatch_time": 1791943497.208595 + }, + { + "id": "confirm-fresh-14-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-14-hi-m0", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 208635, + "retained_microinr": 260794, + "prompt_tokens": 1717, + "completion_tokens": 3048, + "elapsed_seconds": 12.867242167005315, + "dispatch_time": 1791943497.875264 + }, + { + "id": "confirm-fresh-14-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-14-hi-m0", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 229530, + "retained_microinr": 286913, + "prompt_tokens": 1542, + "completion_tokens": 3440, + "elapsed_seconds": 14.504234167005052, + "dispatch_time": 1791943496.539499 + }, + { + "id": "confirm-fresh-14-hi-m0-source-chronological", + "history_id": "confirm-fresh-14-hi-m0", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 508515, + "retained_microinr": 635644, + "prompt_tokens": 1133, + "completion_tokens": 8192, + "elapsed_seconds": 35.58632562500134, + "dispatch_time": 1791943537.118782 + }, + { + "id": "confirm-fresh-08-te-m1-source-chronological", + "history_id": "confirm-fresh-08-te-m1", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 15360, + "retained_microinr": 19200, + "prompt_tokens": 896, + "completion_tokens": 32, + "elapsed_seconds": 1.0703550420003012, + "dispatch_time": 1791943535.785434 + }, + { + "id": "confirm-fresh-08-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-08-te-m1", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 21870, + "retained_microinr": 27338, + "prompt_tokens": 1330, + "completion_tokens": 32, + "elapsed_seconds": 0.9840972910023993, + "dispatch_time": 1791943536.4511712 + }, + { + "id": "confirm-fresh-08-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-08-te-m1", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 20715, + "retained_microinr": 25894, + "prompt_tokens": 1253, + "completion_tokens": 32, + "elapsed_seconds": 0.9449477090092842, + "dispatch_time": 1791943535.11777 + }, + { + "id": "confirm-fresh-09-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-09-hi-m1", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": true, + "raw_status_correct": true, + "known_microinr": 23070, + "retained_microinr": 28838, + "prompt_tokens": 1410, + "completion_tokens": 32, + "elapsed_seconds": 0.9520849999971688, + "dispatch_time": 1791943574.0730948 + }, + { + "id": "confirm-fresh-09-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-09-hi-m1", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27240, + "retained_microinr": 34050, + "prompt_tokens": 1648, + "completion_tokens": 42, + "elapsed_seconds": 1.0129029590025311, + "dispatch_time": 1791943572.742934 + }, + { + "id": "confirm-fresh-09-hi-m1-source-chronological", + "history_id": "confirm-fresh-09-hi-m1", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 19080, + "retained_microinr": 23850, + "prompt_tokens": 1104, + "completion_tokens": 42, + "elapsed_seconds": 1.0904486250074115, + "dispatch_time": 1791943573.410507 + }, + { + "id": "confirm-fresh-39-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-hi-m0", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 275160, + "retained_microinr": 343950, + "prompt_tokens": 2228, + "completion_tokens": 4029, + "elapsed_seconds": 18.592174874997, + "dispatch_time": 1791943574.743098 + }, + { + "id": "confirm-fresh-39-hi-m0-source-chronological", + "history_id": "confirm-fresh-39-hi-m0", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 104025, + "retained_microinr": 130032, + "prompt_tokens": 1835, + "completion_tokens": 1275, + "elapsed_seconds": 5.953728917011176, + "dispatch_time": 1791943594.720466 + }, + { + "id": "confirm-fresh-39-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-39-hi-m0", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 529230, + "retained_microinr": 661538, + "prompt_tokens": 2514, + "completion_tokens": 8192, + "elapsed_seconds": 33.87376287500956, + "dispatch_time": 1791943595.382308 + }, + { + "id": "confirm-fresh-27-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-27-hi-m0", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 155730, + "retained_microinr": 194663, + "prompt_tokens": 2274, + "completion_tokens": 2027, + "elapsed_seconds": 10.957922707995749, + "dispatch_time": 1791943593.3859148 + }, + { + "id": "confirm-fresh-27-hi-m0-source-chronological", + "history_id": "confirm-fresh-27-hi-m0", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 57870, + "retained_microinr": 72338, + "prompt_tokens": 1534, + "completion_tokens": 581, + "elapsed_seconds": 3.302994249999756, + "dispatch_time": 1791943594.053783 + }, + { + "id": "confirm-fresh-27-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-27-hi-m0", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 205890, + "retained_microinr": 257363, + "prompt_tokens": 2306, + "completion_tokens": 2855, + "elapsed_seconds": 12.801137583010131, + "dispatch_time": 1791943631.298326 + }, + { + "id": "confirm-fresh-24-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-24-te-m1", + "scenario": "fresh-24", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34200, + "retained_microinr": 42750, + "prompt_tokens": 2120, + "completion_tokens": 40, + "elapsed_seconds": 0.9508562080009142, + "dispatch_time": 1791943629.964979 + }, + { + "id": "confirm-fresh-24-te-m1-source-chronological", + "history_id": "confirm-fresh-24-te-m1", + "scenario": "fresh-24", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24480, + "retained_microinr": 30600, + "prompt_tokens": 1472, + "completion_tokens": 40, + "elapsed_seconds": 1.0220702090009581, + "dispatch_time": 1791943630.631653 + }, + { + "id": "confirm-fresh-24-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-24-te-m1", + "scenario": "fresh-24", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34200, + "retained_microinr": 42750, + "prompt_tokens": 2120, + "completion_tokens": 40, + "elapsed_seconds": 1.0569155420089373, + "dispatch_time": 1791943629.298347 + }, + { + "id": "confirm-fresh-18-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-te-m1", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 26085, + "retained_microinr": 32607, + "prompt_tokens": 1571, + "completion_tokens": 42, + "elapsed_seconds": 0.9564855829958105, + "dispatch_time": 1791943644.150333 + }, + { + "id": "confirm-fresh-18-te-m1-source-chronological", + "history_id": "confirm-fresh-18-te-m1", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 17895, + "retained_microinr": 22369, + "prompt_tokens": 1025, + "completion_tokens": 42, + "elapsed_seconds": 0.9875284579902655, + "dispatch_time": 1791943644.8176582 + }, + { + "id": "confirm-fresh-18-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-18-te-m1", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 26490, + "retained_microinr": 33113, + "prompt_tokens": 1606, + "completion_tokens": 40, + "elapsed_seconds": 0.9496736669971142, + "dispatch_time": 1791943645.484183 + }, + { + "id": "confirm-fresh-17-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-17-en-m0", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 502080, + "retained_microinr": 627600, + "prompt_tokens": 1696, + "completion_tokens": 7944, + "elapsed_seconds": 37.106685499995365, + "dispatch_time": 1791943646.152097 + }, + { + "id": "confirm-fresh-17-en-m0-source-chronological", + "history_id": "confirm-fresh-17-en-m0", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 362910, + "retained_microinr": 453638, + "prompt_tokens": 1054, + "completion_tokens": 5785, + "elapsed_seconds": 30.019876750011463, + "dispatch_time": 1791943683.297808 + }, + { + "id": "confirm-fresh-17-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-17-en-m0", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 326280, + "retained_microinr": 407850, + "prompt_tokens": 1728, + "completion_tokens": 5006, + "elapsed_seconds": 21.425518083997304, + "dispatch_time": 1791943685.2986538 + }, + { + "id": "confirm-fresh-19-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-te-m0", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 195045, + "retained_microinr": 243807, + "prompt_tokens": 1711, + "completion_tokens": 2823, + "elapsed_seconds": 12.417185833997792, + "dispatch_time": 1791943684.631993 + }, + { + "id": "confirm-fresh-19-te-m0-source-chronological", + "history_id": "confirm-fresh-19-te-m0", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 201480, + "retained_microinr": 251850, + "prompt_tokens": 1052, + "completion_tokens": 3095, + "elapsed_seconds": 12.99166666601377, + "dispatch_time": 1791943683.965343 + }, + { + "id": "confirm-fresh-19-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-19-te-m0", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 487905, + "retained_microinr": 609882, + "prompt_tokens": 1739, + "completion_tokens": 7697, + "elapsed_seconds": 31.99121137498878, + "dispatch_time": 1791943713.358824 + }, + { + "id": "confirm-fresh-20-te-m1-source-chronological", + "history_id": "confirm-fresh-20-te-m1", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 15960, + "retained_microinr": 19950, + "prompt_tokens": 896, + "completion_tokens": 42, + "elapsed_seconds": 0.9413330840034178, + "dispatch_time": 1791943715.357229 + }, + { + "id": "confirm-fresh-20-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-20-te-m1", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 22605, + "retained_microinr": 28257, + "prompt_tokens": 1339, + "completion_tokens": 42, + "elapsed_seconds": 1.0834329999925103, + "dispatch_time": 1791943714.024349 + }, + { + "id": "confirm-fresh-20-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-20-te-m1", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 21945, + "retained_microinr": 27432, + "prompt_tokens": 1295, + "completion_tokens": 42, + "elapsed_seconds": 0.9437112919986248, + "dispatch_time": 1791943714.68825 + }, + { + "id": "confirm-fresh-37-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-te-m0", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 327090, + "retained_microinr": 408863, + "prompt_tokens": 1958, + "completion_tokens": 4962, + "elapsed_seconds": 21.5542608340038, + "dispatch_time": 1791943746.736115 + }, + { + "id": "confirm-fresh-37-te-m0-source-chronological", + "history_id": "confirm-fresh-37-te-m0", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 159675, + "retained_microinr": 199594, + "prompt_tokens": 1673, + "completion_tokens": 2243, + "elapsed_seconds": 10.327790666997316, + "dispatch_time": 1791943747.402803 + }, + { + "id": "confirm-fresh-37-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-37-te-m0", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 160155, + "retained_microinr": 200194, + "prompt_tokens": 2209, + "completion_tokens": 2117, + "elapsed_seconds": 9.33684879100474, + "dispatch_time": 1791943745.403264 + }, + { + "id": "confirm-fresh-12-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-12-te-m0", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 126990, + "retained_microinr": 158738, + "prompt_tokens": 1322, + "completion_tokens": 1786, + "elapsed_seconds": 9.740286917003687, + "dispatch_time": 1791943746.065399 + }, + { + "id": "confirm-fresh-12-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-te-m0", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 103650, + "retained_microinr": 129563, + "prompt_tokens": 1258, + "completion_tokens": 1413, + "elapsed_seconds": 6.485699208002188, + "dispatch_time": 1791943770.333409 + }, + { + "id": "confirm-fresh-12-te-m0-source-chronological", + "history_id": "confirm-fresh-12-te-m0", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 164655, + "retained_microinr": 205819, + "prompt_tokens": 881, + "completion_tokens": 2524, + "elapsed_seconds": 13.449140957993222, + "dispatch_time": 1791943769.6666439 + }, + { + "id": "confirm-fresh-38-en-m0-source-chronological", + "history_id": "confirm-fresh-38-en-m0", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 300330, + "retained_microinr": 375413, + "prompt_tokens": 1470, + "completion_tokens": 4638, + "elapsed_seconds": 22.274877541989554, + "dispatch_time": 1791943768.337085 + }, + { + "id": "confirm-fresh-38-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-en-m0", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 160665, + "retained_microinr": 200832, + "prompt_tokens": 1787, + "completion_tokens": 2231, + "elapsed_seconds": 10.868084208996152, + "dispatch_time": 1791943769.000906 + }, + { + "id": "confirm-fresh-38-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-38-en-m0", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 145320, + "retained_microinr": 181650, + "prompt_tokens": 1888, + "completion_tokens": 1950, + "elapsed_seconds": 9.572698832998867, + "dispatch_time": 1791943792.6547258 + }, + { + "id": "confirm-fresh-08-en-m0-source-chronological", + "history_id": "confirm-fresh-08-en-m0", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 302130, + "retained_microinr": 377663, + "prompt_tokens": 1054, + "completion_tokens": 4772, + "elapsed_seconds": 19.70489283298957, + "dispatch_time": 1791943790.654219 + }, + { + "id": "confirm-fresh-08-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-08-en-m0", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 93465, + "retained_microinr": 116832, + "prompt_tokens": 1531, + "completion_tokens": 1175, + "elapsed_seconds": 6.5690467079984955, + "dispatch_time": 1791943791.988089 + }, + { + "id": "confirm-fresh-08-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-08-en-m0", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 516120, + "retained_microinr": 645150, + "prompt_tokens": 1640, + "completion_tokens": 8192, + "elapsed_seconds": 35.46933249999711, + "dispatch_time": 1791943791.319345 + }, + { + "id": "confirm-fresh-15-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-15-en-m1", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 30915, + "retained_microinr": 38644, + "prompt_tokens": 1893, + "completion_tokens": 42, + "elapsed_seconds": 0.9615866669919342, + "dispatch_time": 1791943828.1597939 + }, + { + "id": "confirm-fresh-15-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-15-en-m1", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27450, + "retained_microinr": 34313, + "prompt_tokens": 1702, + "completion_tokens": 32, + "elapsed_seconds": 1.10548754199408, + "dispatch_time": 1791943827.493033 + }, + { + "id": "confirm-fresh-15-en-m1-source-chronological", + "history_id": "confirm-fresh-15-en-m1", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 20580, + "retained_microinr": 25725, + "prompt_tokens": 1204, + "completion_tokens": 42, + "elapsed_seconds": 1.0101034170074854, + "dispatch_time": 1791943826.826432 + }, + { + "id": "confirm-fresh-14-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-14-hi-m1", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 20925, + "retained_microinr": 26157, + "prompt_tokens": 1267, + "completion_tokens": 32, + "elapsed_seconds": 2.1586129169882042, + "dispatch_time": 1791943828.8279612 + }, + { + "id": "confirm-fresh-14-hi-m1-source-chronological", + "history_id": "confirm-fresh-14-hi-m1", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 17640, + "retained_microinr": 22050, + "prompt_tokens": 1048, + "completion_tokens": 32, + "elapsed_seconds": 1.0028762909932993, + "dispatch_time": 1791943831.028963 + }, + { + "id": "confirm-fresh-14-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-14-hi-m1", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24255, + "retained_microinr": 30319, + "prompt_tokens": 1489, + "completion_tokens": 32, + "elapsed_seconds": 1.0203545420081355, + "dispatch_time": 1791943831.693063 + }, + { + "id": "confirm-fresh-27-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-27-en-m1", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36465, + "retained_microinr": 45582, + "prompt_tokens": 2271, + "completion_tokens": 40, + "elapsed_seconds": 1.0953793749940814, + "dispatch_time": 1791943833.027425 + }, + { + "id": "confirm-fresh-27-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-27-en-m1", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36465, + "retained_microinr": 45582, + "prompt_tokens": 2271, + "completion_tokens": 40, + "elapsed_seconds": 0.9652028749987949, + "dispatch_time": 1791943832.361718 + }, + { + "id": "confirm-fresh-27-en-m1-source-chronological", + "history_id": "confirm-fresh-27-en-m1", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24975, + "retained_microinr": 31219, + "prompt_tokens": 1505, + "completion_tokens": 40, + "elapsed_seconds": 0.9303231249941746, + "dispatch_time": 1791943835.500643 + }, + { + "id": "confirm-fresh-02-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-02-hi-m0", + "scenario": "fresh-02", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 122790, + "retained_microinr": 153488, + "prompt_tokens": 1266, + "completion_tokens": 1730, + "elapsed_seconds": 9.481585165995057, + "dispatch_time": 1791943834.835975 + }, + { + "id": "confirm-fresh-02-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-02-hi-m0", + "scenario": "fresh-02", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 175905, + "retained_microinr": 219882, + "prompt_tokens": 1251, + "completion_tokens": 2619, + "elapsed_seconds": 13.416755999991437, + "dispatch_time": 1791943834.170325 + }, + { + "id": "confirm-fresh-02-hi-m0-source-chronological", + "history_id": "confirm-fresh-02-hi-m0", + "scenario": "fresh-02", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 149085, + "retained_microinr": 186357, + "prompt_tokens": 831, + "completion_tokens": 2277, + "elapsed_seconds": 9.877363040999626, + "dispatch_time": 1791943836.169667 + }, + { + "id": "confirm-fresh-20-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-20-en-m1", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 28095, + "retained_microinr": 35119, + "prompt_tokens": 1705, + "completion_tokens": 42, + "elapsed_seconds": 0.9762358750012936, + "dispatch_time": 1791943849.634946 + }, + { + "id": "confirm-fresh-20-en-m1-source-chronological", + "history_id": "confirm-fresh-20-en-m1", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 19230, + "retained_microinr": 24038, + "prompt_tokens": 1114, + "completion_tokens": 42, + "elapsed_seconds": 1.0324379579978995, + "dispatch_time": 1791943848.30569 + }, + { + "id": "confirm-fresh-20-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-20-en-m1", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 26175, + "retained_microinr": 32719, + "prompt_tokens": 1577, + "completion_tokens": 42, + "elapsed_seconds": 0.9410742500040215, + "dispatch_time": 1791943847.642865 + }, + { + "id": "confirm-fresh-07-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-07-hi-m1", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 28485, + "retained_microinr": 35607, + "prompt_tokens": 1771, + "completion_tokens": 32, + "elapsed_seconds": 1.0264889579993905, + "dispatch_time": 1791943848.971394 + }, + { + "id": "confirm-fresh-07-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-hi-m1", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27285, + "retained_microinr": 34107, + "prompt_tokens": 1691, + "completion_tokens": 32, + "elapsed_seconds": 1.0180132090026746, + "dispatch_time": 1791943851.983187 + }, + { + "id": "confirm-fresh-07-hi-m1-source-chronological", + "history_id": "confirm-fresh-07-hi-m1", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 17985, + "retained_microinr": 22482, + "prompt_tokens": 1071, + "completion_tokens": 32, + "elapsed_seconds": 0.9350204999936977, + "dispatch_time": 1791943850.64837 + }, + { + "id": "confirm-fresh-40-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-40-en-m0", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 369930, + "retained_microinr": 462413, + "prompt_tokens": 2210, + "completion_tokens": 5613, + "elapsed_seconds": 26.328921082997113, + "dispatch_time": 1791943851.3142788 + }, + { + "id": "confirm-fresh-40-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-en-m0", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 407175, + "retained_microinr": 508969, + "prompt_tokens": 2081, + "completion_tokens": 6266, + "elapsed_seconds": 33.38638883299427, + "dispatch_time": 1791943852.648907 + }, + { + "id": "confirm-fresh-40-en-m0-source-chronological", + "history_id": "confirm-fresh-40-en-m0", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 97950, + "retained_microinr": 122438, + "prompt_tokens": 1638, + "completion_tokens": 1223, + "elapsed_seconds": 7.129989791996195, + "dispatch_time": 1791943886.073059 + }, + { + "id": "confirm-fresh-01-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-01-hi-m1", + "scenario": "fresh-01", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 23910, + "retained_microinr": 29888, + "prompt_tokens": 1426, + "completion_tokens": 42, + "elapsed_seconds": 0.9490464999980759, + "dispatch_time": 1791943887.404573 + }, + { + "id": "confirm-fresh-01-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-01-hi-m1", + "scenario": "fresh-01", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 23715, + "retained_microinr": 29644, + "prompt_tokens": 1413, + "completion_tokens": 42, + "elapsed_seconds": 1.0074083329964196, + "dispatch_time": 1791943888.071179 + }, + { + "id": "confirm-fresh-01-hi-m1-source-chronological", + "history_id": "confirm-fresh-01-hi-m1", + "scenario": "fresh-01", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 15585, + "retained_microinr": 19482, + "prompt_tokens": 871, + "completion_tokens": 42, + "elapsed_seconds": 1.023347125010332, + "dispatch_time": 1791943886.7370949 + }, + { + "id": "confirm-fresh-05-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-05-te-m0", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 230070, + "retained_microinr": 287588, + "prompt_tokens": 1454, + "completion_tokens": 3471, + "elapsed_seconds": 16.40769549999095, + "dispatch_time": 1791943893.232853 + }, + { + "id": "confirm-fresh-05-te-m0-source-chronological", + "history_id": "confirm-fresh-05-te-m0", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 103770, + "retained_microinr": 129713, + "prompt_tokens": 910, + "completion_tokens": 1502, + "elapsed_seconds": 6.898224082993693, + "dispatch_time": 1791943893.901829 + }, + { + "id": "confirm-fresh-05-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-05-te-m0", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 326490, + "retained_microinr": 408113, + "prompt_tokens": 1394, + "completion_tokens": 5093, + "elapsed_seconds": 22.103223125013756, + "dispatch_time": 1791943894.565783 + }, + { + "id": "confirm-fresh-38-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-hi-m1", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 34080, + "retained_microinr": 42600, + "prompt_tokens": 2104, + "completion_tokens": 42, + "elapsed_seconds": 0.9695861660002265, + "dispatch_time": 1791943895.235156 + }, + { + "id": "confirm-fresh-38-hi-m1-source-chronological", + "history_id": "confirm-fresh-38-hi-m1", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 31260, + "retained_microinr": 39075, + "prompt_tokens": 1916, + "completion_tokens": 42, + "elapsed_seconds": 1.0174255000019912, + "dispatch_time": 1791943918.0427232 + }, + { + "id": "confirm-fresh-38-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-38-hi-m1", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 40095, + "retained_microinr": 50119, + "prompt_tokens": 2505, + "completion_tokens": 42, + "elapsed_seconds": 0.9591296659928048, + "dispatch_time": 1791943918.70563 + }, + { + "id": "confirm-fresh-01-en-m1-source-chronological", + "history_id": "confirm-fresh-01-en-m1", + "scenario": "fresh-01", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 19380, + "retained_microinr": 24225, + "prompt_tokens": 1124, + "completion_tokens": 42, + "elapsed_seconds": 0.9567032910126727, + "dispatch_time": 1791943917.373348 + }, + { + "id": "confirm-fresh-01-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-01-en-m1", + "scenario": "fresh-01", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": true, + "raw_status_correct": true, + "known_microinr": 27315, + "retained_microinr": 34144, + "prompt_tokens": 1693, + "completion_tokens": 32, + "elapsed_seconds": 0.9767937080032425, + "dispatch_time": 1791943916.708003 + }, + { + "id": "confirm-fresh-01-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-01-en-m1", + "scenario": "fresh-01", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 29835, + "retained_microinr": 37294, + "prompt_tokens": 1821, + "completion_tokens": 42, + "elapsed_seconds": 0.9571239590004552, + "dispatch_time": 1791943921.03669 + }, + { + "id": "confirm-fresh-12-hi-m0-source-chronological", + "history_id": "confirm-fresh-12-hi-m0", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 191100, + "retained_microinr": 238875, + "prompt_tokens": 908, + "completion_tokens": 2958, + "elapsed_seconds": 13.609808041990618, + "dispatch_time": 1791943921.703358 + }, + { + "id": "confirm-fresh-12-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-hi-m0", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 131100, + "retained_microinr": 163875, + "prompt_tokens": 1264, + "completion_tokens": 1869, + "elapsed_seconds": 8.178364875013358, + "dispatch_time": 1791943919.701684 + }, + { + "id": "confirm-fresh-12-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-12-hi-m0", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 80190, + "retained_microinr": 100238, + "prompt_tokens": 1350, + "completion_tokens": 999, + "elapsed_seconds": 4.806213874995592, + "dispatch_time": 1791943920.3685381 + }, + { + "id": "confirm-fresh-13-en-m0-source-chronological", + "history_id": "confirm-fresh-13-en-m0", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 152880, + "retained_microinr": 191100, + "prompt_tokens": 844, + "completion_tokens": 2337, + "elapsed_seconds": 11.15818329200556, + "dispatch_time": 1791943935.353089 + }, + { + "id": "confirm-fresh-13-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-13-en-m0", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 171855, + "retained_microinr": 214819, + "prompt_tokens": 1313, + "completion_tokens": 2536, + "elapsed_seconds": 11.13812804200279, + "dispatch_time": 1791943936.020513 + }, + { + "id": "confirm-fresh-13-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-13-en-m0", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 165675, + "retained_microinr": 207094, + "prompt_tokens": 1313, + "completion_tokens": 2433, + "elapsed_seconds": 11.051968875006423, + "dispatch_time": 1791943936.687194 + }, + { + "id": "confirm-fresh-30-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-te-m1", + "scenario": "fresh-30", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 34440, + "retained_microinr": 43050, + "prompt_tokens": 2128, + "completion_tokens": 42, + "elapsed_seconds": 1.1079283750004834, + "dispatch_time": 1791943937.3533309 + }, + { + "id": "confirm-fresh-30-te-m1-source-chronological", + "history_id": "confirm-fresh-30-te-m1", + "scenario": "fresh-30", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24600, + "retained_microinr": 30750, + "prompt_tokens": 1480, + "completion_tokens": 40, + "elapsed_seconds": 0.9327283329912461, + "dispatch_time": 1791943947.791044 + }, + { + "id": "confirm-fresh-30-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-30-te-m1", + "scenario": "fresh-30", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 34440, + "retained_microinr": 43050, + "prompt_tokens": 2128, + "completion_tokens": 42, + "elapsed_seconds": 1.037280082993675, + "dispatch_time": 1791943949.1241379 + }, + { + "id": "confirm-fresh-10-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-10-te-m1", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27795, + "retained_microinr": 34744, + "prompt_tokens": 1685, + "completion_tokens": 42, + "elapsed_seconds": 1.0108522079972317, + "dispatch_time": 1791943948.458233 + }, + { + "id": "confirm-fresh-10-te-m1-source-chronological", + "history_id": "confirm-fresh-10-te-m1", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 19095, + "retained_microinr": 23869, + "prompt_tokens": 1105, + "completion_tokens": 42, + "elapsed_seconds": 1.0359863749908982, + "dispatch_time": 1791943949.790005 + }, + { + "id": "confirm-fresh-10-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-10-te-m1", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25455, + "retained_microinr": 31819, + "prompt_tokens": 1529, + "completion_tokens": 42, + "elapsed_seconds": 0.947349750000285, + "dispatch_time": 1791943952.200735 + }, + { + "id": "confirm-fresh-05-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-05-te-m1", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27780, + "retained_microinr": 34725, + "prompt_tokens": 1724, + "completion_tokens": 32, + "elapsed_seconds": 0.9550234579946846, + "dispatch_time": 1791943952.8659132 + }, + { + "id": "confirm-fresh-05-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-05-te-m1", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27135, + "retained_microinr": 33919, + "prompt_tokens": 1681, + "completion_tokens": 32, + "elapsed_seconds": 0.9348693329957314, + "dispatch_time": 1791943950.868735 + }, + { + "id": "confirm-fresh-05-te-m1-source-chronological", + "history_id": "confirm-fresh-05-te-m1", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 17415, + "retained_microinr": 21769, + "prompt_tokens": 1033, + "completion_tokens": 32, + "elapsed_seconds": 0.964736832989729, + "dispatch_time": 1791943951.536292 + }, + { + "id": "confirm-fresh-16-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-16-hi-m0", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 223845, + "retained_microinr": 279807, + "prompt_tokens": 1267, + "completion_tokens": 3414, + "elapsed_seconds": 16.575875291004195, + "dispatch_time": 1791943953.86657 + }, + { + "id": "confirm-fresh-16-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-16-hi-m0", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 173400, + "retained_microinr": 216750, + "prompt_tokens": 1496, + "completion_tokens": 2516, + "elapsed_seconds": 10.879935958000715, + "dispatch_time": 1791943955.1990201 + }, + { + "id": "confirm-fresh-16-hi-m0-source-chronological", + "history_id": "confirm-fresh-16-hi-m0", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 202380, + "retained_microinr": 252975, + "prompt_tokens": 1056, + "completion_tokens": 3109, + "elapsed_seconds": 13.414731208002195, + "dispatch_time": 1791943955.8628118 + }, + { + "id": "confirm-fresh-35-hi-m0-source-chronological", + "history_id": "confirm-fresh-35-hi-m0", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 177030, + "retained_microinr": 221288, + "prompt_tokens": 1862, + "completion_tokens": 2485, + "elapsed_seconds": 11.712795708008343, + "dispatch_time": 1791943954.531768 + }, + { + "id": "confirm-fresh-35-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-35-hi-m0", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 365325, + "retained_microinr": 456657, + "prompt_tokens": 2539, + "completion_tokens": 5454, + "elapsed_seconds": 22.776089999999385, + "dispatch_time": 1791943970.479959 + }, + { + "id": "confirm-fresh-35-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-hi-m0", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 220080, + "retained_microinr": 275100, + "prompt_tokens": 2344, + "completion_tokens": 3082, + "elapsed_seconds": 13.915857749991119, + "dispatch_time": 1791943972.4812548 + }, + { + "id": "confirm-fresh-38-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-38-en-m1", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 37755, + "retained_microinr": 47194, + "prompt_tokens": 2349, + "completion_tokens": 42, + "elapsed_seconds": 1.0658477090037195, + "dispatch_time": 1791943971.144486 + }, + { + "id": "confirm-fresh-38-en-m1-source-chronological", + "history_id": "confirm-fresh-38-en-m1", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 29145, + "retained_microinr": 36432, + "prompt_tokens": 1775, + "completion_tokens": 42, + "elapsed_seconds": 0.9468087080022087, + "dispatch_time": 1791943971.8145978 + }, + { + "id": "confirm-fresh-38-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-en-m1", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 33765, + "retained_microinr": 42207, + "prompt_tokens": 2083, + "completion_tokens": 42, + "elapsed_seconds": 0.976152709001326, + "dispatch_time": 1791943994.630343 + }, + { + "id": "confirm-fresh-26-te-m1-source-chronological", + "history_id": "confirm-fresh-26-te-m1", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24615, + "retained_microinr": 30769, + "prompt_tokens": 1481, + "completion_tokens": 40, + "elapsed_seconds": 0.945021417006501, + "dispatch_time": 1791943993.298244 + }, + { + "id": "confirm-fresh-26-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-26-te-m1", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34245, + "retained_microinr": 42807, + "prompt_tokens": 2123, + "completion_tokens": 40, + "elapsed_seconds": 1.1136486670002341, + "dispatch_time": 1791943993.964905 + }, + { + "id": "confirm-fresh-26-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-26-te-m1", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34245, + "retained_microinr": 42807, + "prompt_tokens": 2123, + "completion_tokens": 40, + "elapsed_seconds": 1.0787862920115003, + "dispatch_time": 1791943995.297597 + }, + { + "id": "confirm-fresh-07-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-te-m1", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 22920, + "retained_microinr": 28650, + "prompt_tokens": 1400, + "completion_tokens": 32, + "elapsed_seconds": 1.009558958990965, + "dispatch_time": 1791943998.426809 + }, + { + "id": "confirm-fresh-07-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-07-te-m1", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 23655, + "retained_microinr": 29569, + "prompt_tokens": 1449, + "completion_tokens": 32, + "elapsed_seconds": 1.0261834590055514, + "dispatch_time": 1791943997.093007 + }, + { + "id": "confirm-fresh-07-te-m1-source-chronological", + "history_id": "confirm-fresh-07-te-m1", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 15495, + "retained_microinr": 19369, + "prompt_tokens": 905, + "completion_tokens": 32, + "elapsed_seconds": 0.9890587080008117, + "dispatch_time": 1791943997.7635531 + }, + { + "id": "confirm-fresh-25-en-m0-source-chronological", + "history_id": "confirm-fresh-25-en-m0", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 85200, + "retained_microinr": 106500, + "prompt_tokens": 1352, + "completion_tokens": 1082, + "elapsed_seconds": 5.1286036250094185, + "dispatch_time": 1791943996.4289489 + }, + { + "id": "confirm-fresh-25-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-25-en-m0", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 125970, + "retained_microinr": 157463, + "prompt_tokens": 1954, + "completion_tokens": 1611, + "elapsed_seconds": 8.29077145899646, + "dispatch_time": 1791944001.6014779 + }, + { + "id": "confirm-fresh-25-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-25-en-m0", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 294390, + "retained_microinr": 367988, + "prompt_tokens": 1954, + "completion_tokens": 4418, + "elapsed_seconds": 18.362198625007295, + "dispatch_time": 1791944002.933176 + }, + { + "id": "confirm-fresh-29-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-29-te-m1", + "scenario": "fresh-29", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36585, + "retained_microinr": 45732, + "prompt_tokens": 2279, + "completion_tokens": 40, + "elapsed_seconds": 1.9713660840061493, + "dispatch_time": 1791944002.268806 + }, + { + "id": "confirm-fresh-29-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-29-te-m1", + "scenario": "fresh-29", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36585, + "retained_microinr": 45732, + "prompt_tokens": 2279, + "completion_tokens": 40, + "elapsed_seconds": 0.9608177500049351, + "dispatch_time": 1791944003.6014788 + }, + { + "id": "confirm-fresh-29-te-m1-source-chronological", + "history_id": "confirm-fresh-29-te-m1", + "scenario": "fresh-29", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25155, + "retained_microinr": 31444, + "prompt_tokens": 1517, + "completion_tokens": 40, + "elapsed_seconds": 1.0014116670063231, + "dispatch_time": 1791944021.346307 + }, + { + "id": "confirm-fresh-27-te-m0-source-chronological", + "history_id": "confirm-fresh-27-te-m0", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 158535, + "retained_microinr": 198169, + "prompt_tokens": 1377, + "completion_tokens": 2298, + "elapsed_seconds": 10.898489665996749, + "dispatch_time": 1791944022.0089622 + }, + { + "id": "confirm-fresh-27-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-27-te-m0", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 166380, + "retained_microinr": 207975, + "prompt_tokens": 1996, + "completion_tokens": 2274, + "elapsed_seconds": 9.866345625006943, + "dispatch_time": 1791944023.346103 + }, + { + "id": "confirm-fresh-27-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-27-te-m0", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 184200, + "retained_microinr": 230250, + "prompt_tokens": 1996, + "completion_tokens": 2571, + "elapsed_seconds": 11.015928041000734, + "dispatch_time": 1791944022.6797209 + }, + { + "id": "confirm-fresh-19-en-m1-source-chronological", + "history_id": "confirm-fresh-19-en-m1", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 20355, + "retained_microinr": 25444, + "prompt_tokens": 1189, + "completion_tokens": 42, + "elapsed_seconds": 0.9243975420104107, + "dispatch_time": 1791944035.745027 + }, + { + "id": "confirm-fresh-19-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-en-m1", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 28185, + "retained_microinr": 35232, + "prompt_tokens": 1711, + "completion_tokens": 42, + "elapsed_seconds": 0.9390594160067849, + "dispatch_time": 1791944035.081175 + }, + { + "id": "confirm-fresh-19-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-19-en-m1", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 30675, + "retained_microinr": 38344, + "prompt_tokens": 1877, + "completion_tokens": 42, + "elapsed_seconds": 1.0430048330017598, + "dispatch_time": 1791944034.413193 + }, + { + "id": "confirm-fresh-31-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-31-te-m1", + "scenario": "fresh-31", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36615, + "retained_microinr": 45769, + "prompt_tokens": 2281, + "completion_tokens": 40, + "elapsed_seconds": 0.9490969160106033, + "dispatch_time": 1791944033.746864 + }, + { + "id": "confirm-fresh-31-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-31-te-m1", + "scenario": "fresh-31", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36615, + "retained_microinr": 45769, + "prompt_tokens": 2281, + "completion_tokens": 40, + "elapsed_seconds": 1.0018306670099264, + "dispatch_time": 1791944036.724011 + }, + { + "id": "confirm-fresh-31-te-m1-source-chronological", + "history_id": "confirm-fresh-31-te-m1", + "scenario": "fresh-31", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25245, + "retained_microinr": 31557, + "prompt_tokens": 1523, + "completion_tokens": 40, + "elapsed_seconds": 1.0602819999912754, + "dispatch_time": 1791944038.057128 + }, + { + "id": "confirm-fresh-32-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-32-en-m1", + "scenario": "fresh-32", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34170, + "retained_microinr": 42713, + "prompt_tokens": 2118, + "completion_tokens": 40, + "elapsed_seconds": 0.9512993749958696, + "dispatch_time": 1791944038.721215 + }, + { + "id": "confirm-fresh-32-en-m1-source-chronological", + "history_id": "confirm-fresh-32-en-m1", + "scenario": "fresh-32", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24330, + "retained_microinr": 30413, + "prompt_tokens": 1462, + "completion_tokens": 40, + "elapsed_seconds": 0.9453503749973606, + "dispatch_time": 1791944037.3898199 + }, + { + "id": "confirm-fresh-32-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-en-m1", + "scenario": "fresh-32", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 34290, + "retained_microinr": 42863, + "prompt_tokens": 2118, + "completion_tokens": 42, + "elapsed_seconds": 0.9879191670042928, + "dispatch_time": 1791944039.713272 + }, + { + "id": "confirm-fresh-06-en-m1-source-chronological", + "history_id": "confirm-fresh-06-en-m1", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 17955, + "retained_microinr": 22444, + "prompt_tokens": 1069, + "completion_tokens": 32, + "elapsed_seconds": 0.9809812500025146, + "dispatch_time": 1791944040.382236 + }, + { + "id": "confirm-fresh-06-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-06-en-m1", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 26565, + "retained_microinr": 33207, + "prompt_tokens": 1643, + "completion_tokens": 32, + "elapsed_seconds": 0.9335260829975596, + "dispatch_time": 1791944041.046377 + }, + { + "id": "confirm-fresh-06-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-06-en-m1", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24675, + "retained_microinr": 30844, + "prompt_tokens": 1517, + "completion_tokens": 32, + "elapsed_seconds": 0.9801323340070667, + "dispatch_time": 1791944041.7131 + }, + { + "id": "confirm-fresh-07-en-m1-source-chronological", + "history_id": "confirm-fresh-07-en-m1", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 19470, + "retained_microinr": 24338, + "prompt_tokens": 1170, + "completion_tokens": 32, + "elapsed_seconds": 0.9572724169993307, + "dispatch_time": 1791944043.402996 + }, + { + "id": "confirm-fresh-07-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-en-m1", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27000, + "retained_microinr": 33750, + "prompt_tokens": 1672, + "completion_tokens": 32, + "elapsed_seconds": 0.9694033749983646, + "dispatch_time": 1791944044.072304 + }, + { + "id": "confirm-fresh-07-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-07-en-m1", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 29670, + "retained_microinr": 37088, + "prompt_tokens": 1850, + "completion_tokens": 32, + "elapsed_seconds": 1.0099914169986732, + "dispatch_time": 1791944042.740617 + }, + { + "id": "confirm-fresh-13-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-13-te-m0", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 196965, + "retained_microinr": 246207, + "prompt_tokens": 1411, + "completion_tokens": 2930, + "elapsed_seconds": 13.262295290987822, + "dispatch_time": 1791944044.737222 + }, + { + "id": "confirm-fresh-13-te-m0-source-chronological", + "history_id": "confirm-fresh-13-te-m0", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 506805, + "retained_microinr": 633507, + "prompt_tokens": 1019, + "completion_tokens": 8192, + "elapsed_seconds": 36.164887249993626, + "dispatch_time": 1791944058.043663 + }, + { + "id": "confirm-fresh-13-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-13-te-m0", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 292485, + "retained_microinr": 365607, + "prompt_tokens": 1567, + "completion_tokens": 4483, + "elapsed_seconds": 19.27871454099659, + "dispatch_time": 1791944058.7109199 + }, + { + "id": "confirm-fresh-11-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-11-te-m1", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 30465, + "retained_microinr": 38082, + "prompt_tokens": 1863, + "completion_tokens": 42, + "elapsed_seconds": 1.1042334169906098, + "dispatch_time": 1791944060.043196 + }, + { + "id": "confirm-fresh-11-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-11-te-m1", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27795, + "retained_microinr": 34744, + "prompt_tokens": 1685, + "completion_tokens": 42, + "elapsed_seconds": 1.0161284579953644, + "dispatch_time": 1791944059.3742132 + }, + { + "id": "confirm-fresh-11-te-m1-source-chronological", + "history_id": "confirm-fresh-11-te-m1", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 20280, + "retained_microinr": 25350, + "prompt_tokens": 1184, + "completion_tokens": 42, + "elapsed_seconds": 0.9765736249973997, + "dispatch_time": 1791944094.923767 + }, + { + "id": "confirm-fresh-09-te-m1-source-chronological", + "history_id": "confirm-fresh-09-te-m1", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 20640, + "retained_microinr": 25800, + "prompt_tokens": 1208, + "completion_tokens": 42, + "elapsed_seconds": 1.0040511660045013, + "dispatch_time": 1791944094.2581189 + }, + { + "id": "confirm-fresh-09-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-09-te-m1", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 28110, + "retained_microinr": 35138, + "prompt_tokens": 1706, + "completion_tokens": 42, + "elapsed_seconds": 0.9720865410054103, + "dispatch_time": 1791944095.590409 + }, + { + "id": "confirm-fresh-09-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-09-te-m1", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 31185, + "retained_microinr": 38982, + "prompt_tokens": 1911, + "completion_tokens": 42, + "elapsed_seconds": 1.059681416998501, + "dispatch_time": 1791944096.25571 + }, + { + "id": "confirm-fresh-27-te-m1-source-chronological", + "history_id": "confirm-fresh-27-te-m1", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25245, + "retained_microinr": 31557, + "prompt_tokens": 1523, + "completion_tokens": 40, + "elapsed_seconds": 1.022177582999575, + "dispatch_time": 1791944098.027519 + }, + { + "id": "confirm-fresh-27-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-27-te-m1", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36495, + "retained_microinr": 45619, + "prompt_tokens": 2273, + "completion_tokens": 40, + "elapsed_seconds": 1.1062835000047926, + "dispatch_time": 1791944099.358895 + }, + { + "id": "confirm-fresh-27-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-27-te-m1", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36495, + "retained_microinr": 45619, + "prompt_tokens": 2273, + "completion_tokens": 40, + "elapsed_seconds": 1.0070587079972029, + "dispatch_time": 1791944097.3614159 + }, + { + "id": "confirm-fresh-12-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-12-hi-m1", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": true, + "raw_status_correct": true, + "known_microinr": 27390, + "retained_microinr": 34238, + "prompt_tokens": 1698, + "completion_tokens": 32, + "elapsed_seconds": 0.9482885830075247, + "dispatch_time": 1791944098.6936278 + }, + { + "id": "confirm-fresh-12-hi-m1-source-chronological", + "history_id": "confirm-fresh-12-hi-m1", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": true, + "raw_status_correct": true, + "known_microinr": 18690, + "retained_microinr": 23363, + "prompt_tokens": 1118, + "completion_tokens": 32, + "elapsed_seconds": 0.9958275829994818, + "dispatch_time": 1791944100.5137331 + }, + { + "id": "confirm-fresh-12-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-hi-m1", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": true, + "raw_status_correct": true, + "known_microinr": 25020, + "retained_microinr": 31275, + "prompt_tokens": 1540, + "completion_tokens": 32, + "elapsed_seconds": 0.9534871250070864, + "dispatch_time": 1791944101.8426828 + }, + { + "id": "confirm-fresh-12-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-te-m1", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25635, + "retained_microinr": 32044, + "prompt_tokens": 1541, + "completion_tokens": 42, + "elapsed_seconds": 1.0473955420020502, + "dispatch_time": 1791944101.175814 + }, + { + "id": "confirm-fresh-12-te-m1-source-chronological", + "history_id": "confirm-fresh-12-te-m1", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 19455, + "retained_microinr": 24319, + "prompt_tokens": 1129, + "completion_tokens": 42, + "elapsed_seconds": 0.9940150839975104, + "dispatch_time": 1791944102.5100648 + }, + { + "id": "confirm-fresh-12-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-12-te-m1", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 28185, + "retained_microinr": 35232, + "prompt_tokens": 1711, + "completion_tokens": 42, + "elapsed_seconds": 0.997791916001006, + "dispatch_time": 1791944104.878379 + }, + { + "id": "confirm-fresh-18-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-18-te-m0", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 237075, + "retained_microinr": 296344, + "prompt_tokens": 1297, + "completion_tokens": 3627, + "elapsed_seconds": 18.703877749998355, + "dispatch_time": 1791944103.5483131 + }, + { + "id": "confirm-fresh-18-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-te-m0", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 116190, + "retained_microinr": 145238, + "prompt_tokens": 1274, + "completion_tokens": 1618, + "elapsed_seconds": 7.20122041599825, + "dispatch_time": 1791944105.5447419 + }, + { + "id": "confirm-fresh-18-te-m0-source-chronological", + "history_id": "confirm-fresh-18-te-m0", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 504585, + "retained_microinr": 630732, + "prompt_tokens": 871, + "completion_tokens": 8192, + "elapsed_seconds": 34.658750041999156, + "dispatch_time": 1791944104.2145631 + }, + { + "id": "confirm-fresh-26-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-26-en-m1", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 33990, + "retained_microinr": 42488, + "prompt_tokens": 2106, + "completion_tokens": 40, + "elapsed_seconds": 1.0119471670041094, + "dispatch_time": 1791944139.591316 + }, + { + "id": "confirm-fresh-26-en-m1-source-chronological", + "history_id": "confirm-fresh-26-en-m1", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24240, + "retained_microinr": 30300, + "prompt_tokens": 1456, + "completion_tokens": 40, + "elapsed_seconds": 1.056377375003649, + "dispatch_time": 1791944138.9255638 + }, + { + "id": "confirm-fresh-26-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-26-en-m1", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 33990, + "retained_microinr": 42488, + "prompt_tokens": 2106, + "completion_tokens": 40, + "elapsed_seconds": 1.1043252499948721, + "dispatch_time": 1791944140.2544389 + }, + { + "id": "confirm-fresh-23-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-23-te-m1", + "scenario": "fresh-23", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36585, + "retained_microinr": 45732, + "prompt_tokens": 2279, + "completion_tokens": 40, + "elapsed_seconds": 0.973554416996194, + "dispatch_time": 1791944140.9230888 + }, + { + "id": "confirm-fresh-23-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-23-te-m1", + "scenario": "fresh-23", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36585, + "retained_microinr": 45732, + "prompt_tokens": 2279, + "completion_tokens": 40, + "elapsed_seconds": 1.0796093750104774, + "dispatch_time": 1791944143.944588 + }, + { + "id": "confirm-fresh-23-te-m1-source-chronological", + "history_id": "confirm-fresh-23-te-m1", + "scenario": "fresh-23", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25050, + "retained_microinr": 31313, + "prompt_tokens": 1510, + "completion_tokens": 40, + "elapsed_seconds": 0.9490920000098413, + "dispatch_time": 1791944143.279019 + }, + { + "id": "confirm-fresh-17-hi-m1-source-chronological", + "history_id": "confirm-fresh-17-hi-m1", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 18420, + "retained_microinr": 23025, + "prompt_tokens": 1060, + "completion_tokens": 42, + "elapsed_seconds": 0.9997523750062101, + "dispatch_time": 1791944142.610786 + }, + { + "id": "confirm-fresh-17-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-17-hi-m1", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 28410, + "retained_microinr": 35513, + "prompt_tokens": 1726, + "completion_tokens": 42, + "elapsed_seconds": 0.9848852080031065, + "dispatch_time": 1791944141.947244 + }, + { + "id": "confirm-fresh-17-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-17-hi-m1", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-03-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 28980, + "retained_microinr": 36225, + "prompt_tokens": 1764, + "completion_tokens": 42, + "elapsed_seconds": 0.9245362500078045, + "dispatch_time": 1791944145.064277 + }, + { + "id": "confirm-fresh-22-te-m0-source-chronological", + "history_id": "confirm-fresh-22-te-m0", + "scenario": "fresh-22", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 87600, + "retained_microinr": 109500, + "prompt_tokens": 1540, + "completion_tokens": 1075, + "elapsed_seconds": 6.07318908299203, + "dispatch_time": 1791944146.398036 + }, + { + "id": "confirm-fresh-22-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-22-te-m0", + "scenario": "fresh-22", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 201525, + "retained_microinr": 251907, + "prompt_tokens": 2127, + "completion_tokens": 2827, + "elapsed_seconds": 13.351604625000618, + "dispatch_time": 1791944145.7320058 + }, + { + "id": "confirm-fresh-22-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-22-te-m0", + "scenario": "fresh-22", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 524565, + "retained_microinr": 655707, + "prompt_tokens": 2203, + "completion_tokens": 8192, + "elapsed_seconds": 35.501241707999725, + "dispatch_time": 1791944147.0660841 + }, + { + "id": "confirm-fresh-01-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-01-te-m1", + "scenario": "fresh-01", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 23625, + "retained_microinr": 29532, + "prompt_tokens": 1407, + "completion_tokens": 42, + "elapsed_seconds": 0.9754500839917455, + "dispatch_time": 1791944184.61727 + }, + { + "id": "confirm-fresh-01-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-01-te-m1", + "scenario": "fresh-01", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 23925, + "retained_microinr": 29907, + "prompt_tokens": 1427, + "completion_tokens": 42, + "elapsed_seconds": 0.9718708750006044, + "dispatch_time": 1791944183.284142 + }, + { + "id": "confirm-fresh-01-te-m1-source-chronological", + "history_id": "confirm-fresh-01-te-m1", + "scenario": "fresh-01", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 15750, + "retained_microinr": 19688, + "prompt_tokens": 882, + "completion_tokens": 42, + "elapsed_seconds": 0.9889927500043996, + "dispatch_time": 1791944182.617801 + }, + { + "id": "confirm-fresh-37-en-m0-source-chronological", + "history_id": "confirm-fresh-37-en-m0", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 409275, + "retained_microinr": 511594, + "prompt_tokens": 1709, + "completion_tokens": 6394, + "elapsed_seconds": 27.841237041997374, + "dispatch_time": 1791944183.950063 + }, + { + "id": "confirm-fresh-37-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-37-en-m0", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 408615, + "retained_microinr": 510769, + "prompt_tokens": 2397, + "completion_tokens": 6211, + "elapsed_seconds": 29.503583375000744, + "dispatch_time": 1791944212.509801 + }, + { + "id": "confirm-fresh-37-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-en-m0", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 183615, + "retained_microinr": 229519, + "prompt_tokens": 2233, + "completion_tokens": 2502, + "elapsed_seconds": 11.207604291004827, + "dispatch_time": 1791944213.17712 + }, + { + "id": "confirm-fresh-13-en-m1-source-chronological", + "history_id": "confirm-fresh-13-en-m1", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 20685, + "retained_microinr": 25857, + "prompt_tokens": 1211, + "completion_tokens": 42, + "elapsed_seconds": 1.1110167919978267, + "dispatch_time": 1791944211.850685 + }, + { + "id": "confirm-fresh-13-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-13-en-m1", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27465, + "retained_microinr": 34332, + "prompt_tokens": 1703, + "completion_tokens": 32, + "elapsed_seconds": 0.9919395830074791, + "dispatch_time": 1791944213.842409 + }, + { + "id": "confirm-fresh-13-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-13-en-m1", + "scenario": "fresh-13", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 31140, + "retained_microinr": 38925, + "prompt_tokens": 1908, + "completion_tokens": 42, + "elapsed_seconds": 0.9607766250119312, + "dispatch_time": 1791944243.395245 + }, + { + "id": "confirm-fresh-19-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-19-hi-m1", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 28920, + "retained_microinr": 36150, + "prompt_tokens": 1760, + "completion_tokens": 42, + "elapsed_seconds": 1.0178038750018459, + "dispatch_time": 1791944244.061725 + }, + { + "id": "confirm-fresh-19-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-hi-m1", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 28215, + "retained_microinr": 35269, + "prompt_tokens": 1713, + "completion_tokens": 42, + "elapsed_seconds": 1.0134847500012256, + "dispatch_time": 1791944242.061238 + }, + { + "id": "confirm-fresh-19-hi-m1-source-chronological", + "history_id": "confirm-fresh-19-hi-m1", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 18510, + "retained_microinr": 23138, + "prompt_tokens": 1066, + "completion_tokens": 42, + "elapsed_seconds": 1.0303184579970548, + "dispatch_time": 1791944242.728684 + }, + { + "id": "confirm-fresh-37-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-te-m1", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 36075, + "retained_microinr": 45094, + "prompt_tokens": 2237, + "completion_tokens": 42, + "elapsed_seconds": 0.9568220000073779, + "dispatch_time": 1791944247.118231 + }, + { + "id": "confirm-fresh-37-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-37-te-m1", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 41280, + "retained_microinr": 51600, + "prompt_tokens": 2592, + "completion_tokens": 40, + "elapsed_seconds": 1.025515582994558, + "dispatch_time": 1791944246.4515572 + }, + { + "id": "confirm-fresh-37-te-m1-source-chronological", + "history_id": "confirm-fresh-37-te-m1", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 31245, + "retained_microinr": 39057, + "prompt_tokens": 1915, + "completion_tokens": 42, + "elapsed_seconds": 0.959741375001613, + "dispatch_time": 1791944245.782954 + }, + { + "id": "confirm-fresh-21-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-21-hi-m0", + "scenario": "fresh-21", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 323445, + "retained_microinr": 404307, + "prompt_tokens": 1975, + "completion_tokens": 4897, + "elapsed_seconds": 23.6456726250035, + "dispatch_time": 1791944245.1168761 + }, + { + "id": "confirm-fresh-21-hi-m0-source-chronological", + "history_id": "confirm-fresh-21-hi-m0", + "scenario": "fresh-21", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 132210, + "retained_microinr": 165263, + "prompt_tokens": 1362, + "completion_tokens": 1863, + "elapsed_seconds": 8.234530417001224, + "dispatch_time": 1791944270.129627 + }, + { + "id": "confirm-fresh-21-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-21-hi-m0", + "scenario": "fresh-21", + "family": "approved_change", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 251505, + "retained_microinr": 314382, + "prompt_tokens": 1975, + "completion_tokens": 3698, + "elapsed_seconds": 15.051047041997663, + "dispatch_time": 1791944270.796295 + }, + { + "id": "confirm-fresh-02-en-m0-source-chronological", + "history_id": "confirm-fresh-02-en-m0", + "scenario": "fresh-02", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 59175, + "retained_microinr": 73969, + "prompt_tokens": 921, + "completion_tokens": 756, + "elapsed_seconds": 4.452895791997435, + "dispatch_time": 1791944268.793938 + }, + { + "id": "confirm-fresh-02-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-02-en-m0", + "scenario": "fresh-02", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 86610, + "retained_microinr": 108263, + "prompt_tokens": 1254, + "completion_tokens": 1130, + "elapsed_seconds": 5.4183051669970155, + "dispatch_time": 1791944269.462959 + }, + { + "id": "confirm-fresh-02-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-02-en-m0", + "scenario": "fresh-02", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 119895, + "retained_microinr": 149869, + "prompt_tokens": 1361, + "completion_tokens": 1658, + "elapsed_seconds": 9.317468374996679, + "dispatch_time": 1791944287.227712 + }, + { + "id": "confirm-fresh-06-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-06-te-m0", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 178920, + "retained_microinr": 223650, + "prompt_tokens": 1292, + "completion_tokens": 2659, + "elapsed_seconds": 13.880363666990888, + "dispatch_time": 1791944285.89606 + }, + { + "id": "confirm-fresh-06-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-06-te-m0", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 86190, + "retained_microinr": 107738, + "prompt_tokens": 1254, + "completion_tokens": 1123, + "elapsed_seconds": 6.485287333998713, + "dispatch_time": 1791944286.559049 + }, + { + "id": "confirm-fresh-06-te-m0-source-chronological", + "history_id": "confirm-fresh-06-te-m0", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 476295, + "retained_microinr": 595369, + "prompt_tokens": 853, + "completion_tokens": 7725, + "elapsed_seconds": 31.482956999985618, + "dispatch_time": 1791944287.893992 + }, + { + "id": "confirm-fresh-39-te-m0-source-chronological", + "history_id": "confirm-fresh-39-te-m0", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 116955, + "retained_microinr": 146194, + "prompt_tokens": 1653, + "completion_tokens": 1536, + "elapsed_seconds": 6.939602875005221, + "dispatch_time": 1791944320.7627368 + }, + { + "id": "confirm-fresh-39-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-39-te-m0", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 315030, + "retained_microinr": 393788, + "prompt_tokens": 2190, + "completion_tokens": 4703, + "elapsed_seconds": 21.842392959006247, + "dispatch_time": 1791944319.4290938 + }, + { + "id": "confirm-fresh-39-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-te-m0", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 155535, + "retained_microinr": 194419, + "prompt_tokens": 1961, + "completion_tokens": 2102, + "elapsed_seconds": 11.289832583002863, + "dispatch_time": 1791944321.42941 + }, + { + "id": "confirm-fresh-12-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-en-m0", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 128535, + "retained_microinr": 160669, + "prompt_tokens": 1265, + "completion_tokens": 1826, + "elapsed_seconds": 8.442558375012595, + "dispatch_time": 1791944320.0949461 + }, + { + "id": "confirm-fresh-12-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-12-en-m0", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 164325, + "retained_microinr": 205407, + "prompt_tokens": 1319, + "completion_tokens": 2409, + "elapsed_seconds": 10.384598374992493, + "dispatch_time": 1791944343.317463 + }, + { + "id": "confirm-fresh-12-en-m0-source-chronological", + "history_id": "confirm-fresh-12-en-m0", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 111705, + "retained_microinr": 139632, + "prompt_tokens": 875, + "completion_tokens": 1643, + "elapsed_seconds": 8.68665604099806, + "dispatch_time": 1791944342.6490922 + }, + { + "id": "confirm-fresh-11-hi-m1-source-chronological", + "history_id": "confirm-fresh-11-hi-m1", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": true, + "raw_status_correct": true, + "known_microinr": 21495, + "retained_microinr": 26869, + "prompt_tokens": 1305, + "completion_tokens": 32, + "elapsed_seconds": 1.0096349579980597, + "dispatch_time": 1791944341.316639 + }, + { + "id": "confirm-fresh-11-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-11-hi-m1", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": true, + "raw_status_correct": true, + "known_microinr": 27585, + "retained_microinr": 34482, + "prompt_tokens": 1711, + "completion_tokens": 32, + "elapsed_seconds": 0.9533761670027161, + "dispatch_time": 1791944341.980824 + }, + { + "id": "confirm-fresh-11-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-11-hi-m1", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 32670, + "retained_microinr": 40838, + "prompt_tokens": 2010, + "completion_tokens": 42, + "elapsed_seconds": 1.084269166996819, + "dispatch_time": 1791944355.754209 + }, + { + "id": "confirm-fresh-27-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-27-hi-m1", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36465, + "retained_microinr": 45582, + "prompt_tokens": 2271, + "completion_tokens": 40, + "elapsed_seconds": 1.0219765420042677, + "dispatch_time": 1791944353.753429 + }, + { + "id": "confirm-fresh-27-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-27-hi-m1", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36465, + "retained_microinr": 45582, + "prompt_tokens": 2271, + "completion_tokens": 40, + "elapsed_seconds": 0.9778811249998398, + "dispatch_time": 1791944355.084539 + }, + { + "id": "confirm-fresh-27-hi-m1-source-chronological", + "history_id": "confirm-fresh-27-hi-m1", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25020, + "retained_microinr": 31275, + "prompt_tokens": 1508, + "completion_tokens": 40, + "elapsed_seconds": 1.0093898339982843, + "dispatch_time": 1791944354.4207208 + }, + { + "id": "confirm-fresh-22-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-22-en-m1", + "scenario": "fresh-22", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34005, + "retained_microinr": 42507, + "prompt_tokens": 2107, + "completion_tokens": 40, + "elapsed_seconds": 1.0368601660011336, + "dispatch_time": 1791944357.5573602 + }, + { + "id": "confirm-fresh-22-en-m1-source-chronological", + "history_id": "confirm-fresh-22-en-m1", + "scenario": "fresh-22", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24240, + "retained_microinr": 30300, + "prompt_tokens": 1456, + "completion_tokens": 40, + "elapsed_seconds": 0.9950489580078283, + "dispatch_time": 1791944356.891872 + }, + { + "id": "confirm-fresh-22-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-22-en-m1", + "scenario": "fresh-22", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34005, + "retained_microinr": 42507, + "prompt_tokens": 2107, + "completion_tokens": 40, + "elapsed_seconds": 1.0903271660063183, + "dispatch_time": 1791944358.891084 + }, + { + "id": "confirm-fresh-29-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-29-en-m0", + "scenario": "fresh-29", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 219135, + "retained_microinr": 273919, + "prompt_tokens": 2029, + "completion_tokens": 3145, + "elapsed_seconds": 13.938349500007462, + "dispatch_time": 1791944358.22443 + }, + { + "id": "confirm-fresh-29-en-m0-source-chronological", + "history_id": "confirm-fresh-29-en-m0", + "scenario": "fresh-29", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 332940, + "retained_microinr": 416175, + "prompt_tokens": 1460, + "completion_tokens": 5184, + "elapsed_seconds": 21.042172667002887, + "dispatch_time": 1791944373.545681 + }, + { + "id": "confirm-fresh-29-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-29-en-m0", + "scenario": "fresh-29", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 236280, + "retained_microinr": 295350, + "prompt_tokens": 2164, + "completion_tokens": 3397, + "elapsed_seconds": 14.523942707994138, + "dispatch_time": 1791944372.213567 + }, + { + "id": "confirm-fresh-30-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-en-m0", + "scenario": "fresh-30", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 113235, + "retained_microinr": 141544, + "prompt_tokens": 1581, + "completion_tokens": 1492, + "elapsed_seconds": 7.055782249997719, + "dispatch_time": 1791944372.87791 + }, + { + "id": "confirm-fresh-30-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-30-en-m0", + "scenario": "fresh-30", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 194550, + "retained_microinr": 243188, + "prompt_tokens": 1694, + "completion_tokens": 2819, + "elapsed_seconds": 13.259262457999284, + "dispatch_time": 1791944374.2129862 + }, + { + "id": "confirm-fresh-30-en-m0-source-chronological", + "history_id": "confirm-fresh-30-en-m0", + "scenario": "fresh-30", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 197370, + "retained_microinr": 246713, + "prompt_tokens": 1258, + "completion_tokens": 2975, + "elapsed_seconds": 14.935974583000643, + "dispatch_time": 1791944394.638988 + }, + { + "id": "confirm-fresh-34-te-m0-source-chronological", + "history_id": "confirm-fresh-34-te-m0", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 222195, + "retained_microinr": 277744, + "prompt_tokens": 1837, + "completion_tokens": 3244, + "elapsed_seconds": 13.686934375000419, + "dispatch_time": 1791944395.9692779 + }, + { + "id": "confirm-fresh-34-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-te-m0", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 350880, + "retained_microinr": 438600, + "prompt_tokens": 2216, + "completion_tokens": 5294, + "elapsed_seconds": 24.093186958998558, + "dispatch_time": 1791944395.304967 + }, + { + "id": "confirm-fresh-34-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-34-te-m0", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 128385, + "retained_microinr": 160482, + "prompt_tokens": 2407, + "completion_tokens": 1538, + "elapsed_seconds": 6.980180041995482, + "dispatch_time": 1791944396.63677 + }, + { + "id": "confirm-fresh-33-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-33-en-m1", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 41970, + "retained_microinr": 52463, + "prompt_tokens": 2630, + "completion_tokens": 42, + "elapsed_seconds": 0.9518733330041869, + "dispatch_time": 1791944420.106945 + }, + { + "id": "confirm-fresh-33-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-en-m1", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 37605, + "retained_microinr": 47007, + "prompt_tokens": 2339, + "completion_tokens": 42, + "elapsed_seconds": 1.011973915999988, + "dispatch_time": 1791944419.4381618 + }, + { + "id": "confirm-fresh-33-en-m1-source-chronological", + "history_id": "confirm-fresh-33-en-m1", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 31695, + "retained_microinr": 39619, + "prompt_tokens": 1945, + "completion_tokens": 42, + "elapsed_seconds": 1.6084250419953605, + "dispatch_time": 1791944420.7722938 + }, + { + "id": "confirm-fresh-10-hi-m1-source-chronological", + "history_id": "confirm-fresh-10-hi-m1", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 20010, + "retained_microinr": 25013, + "prompt_tokens": 1166, + "completion_tokens": 42, + "elapsed_seconds": 0.9439062499877764, + "dispatch_time": 1791944421.43606 + }, + { + "id": "confirm-fresh-10-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-10-hi-m1", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 28755, + "retained_microinr": 35944, + "prompt_tokens": 1749, + "completion_tokens": 42, + "elapsed_seconds": 0.9993957499973476, + "dispatch_time": 1791944422.4288309 + }, + { + "id": "confirm-fresh-10-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-10-hi-m1", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": true, + "raw_status_correct": true, + "known_microinr": 25080, + "retained_microinr": 31350, + "prompt_tokens": 1544, + "completion_tokens": 32, + "elapsed_seconds": 0.9858482499985257, + "dispatch_time": 1791944424.4302158 + }, + { + "id": "confirm-fresh-18-hi-m0-source-chronological", + "history_id": "confirm-fresh-18-hi-m0", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 361365, + "retained_microinr": 451707, + "prompt_tokens": 1023, + "completion_tokens": 5767, + "elapsed_seconds": 24.056296499999007, + "dispatch_time": 1791944423.764152 + }, + { + "id": "confirm-fresh-18-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-hi-m0", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 113040, + "retained_microinr": 141300, + "prompt_tokens": 1572, + "completion_tokens": 1491, + "elapsed_seconds": 7.146909040995524, + "dispatch_time": 1791944423.093997 + }, + { + "id": "confirm-fresh-18-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-18-hi-m0", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 314550, + "retained_microinr": 393188, + "prompt_tokens": 1614, + "completion_tokens": 4839, + "elapsed_seconds": 21.178230124991387, + "dispatch_time": 1791944447.8631701 + }, + { + "id": "confirm-fresh-17-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-17-en-m1", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 28110, + "retained_microinr": 35138, + "prompt_tokens": 1706, + "completion_tokens": 42, + "elapsed_seconds": 1.1101471669971943, + "dispatch_time": 1791944448.526911 + }, + { + "id": "confirm-fresh-17-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-17-en-m1", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 29490, + "retained_microinr": 36863, + "prompt_tokens": 1798, + "completion_tokens": 42, + "elapsed_seconds": 0.9717892079934245, + "dispatch_time": 1791944449.19747 + }, + { + "id": "confirm-fresh-17-en-m1-source-chronological", + "history_id": "confirm-fresh-17-en-m1", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 19230, + "retained_microinr": 24038, + "prompt_tokens": 1114, + "completion_tokens": 42, + "elapsed_seconds": 1.0526387499994598, + "dispatch_time": 1791944449.8605819 + }, + { + "id": "confirm-fresh-25-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-25-te-m1", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36495, + "retained_microinr": 45619, + "prompt_tokens": 2273, + "completion_tokens": 40, + "elapsed_seconds": 1.0558867090003332, + "dispatch_time": 1791944469.7514398 + }, + { + "id": "confirm-fresh-25-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-25-te-m1", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36495, + "retained_microinr": 45619, + "prompt_tokens": 2273, + "completion_tokens": 40, + "elapsed_seconds": 0.9434035830054199, + "dispatch_time": 1791944470.4186509 + }, + { + "id": "confirm-fresh-25-te-m1-source-chronological", + "history_id": "confirm-fresh-25-te-m1", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25065, + "retained_microinr": 31332, + "prompt_tokens": 1511, + "completion_tokens": 40, + "elapsed_seconds": 1.00234779099992, + "dispatch_time": 1791944469.0850282 + }, + { + "id": "confirm-fresh-04-hi-m1-source-chronological", + "history_id": "confirm-fresh-04-hi-m1", + "scenario": "fresh-04", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 15090, + "retained_microinr": 18863, + "prompt_tokens": 838, + "completion_tokens": 42, + "elapsed_seconds": 1.0980610839906149, + "dispatch_time": 1791944471.084864 + }, + { + "id": "confirm-fresh-04-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-04-hi-m1", + "scenario": "fresh-04", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 21675, + "retained_microinr": 27094, + "prompt_tokens": 1277, + "completion_tokens": 42, + "elapsed_seconds": 1.025159333992633, + "dispatch_time": 1791944472.895468 + }, + { + "id": "confirm-fresh-04-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-04-hi-m1", + "scenario": "fresh-04", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 21480, + "retained_microinr": 26850, + "prompt_tokens": 1264, + "completion_tokens": 42, + "elapsed_seconds": 2.1554742500011344, + "dispatch_time": 1791944474.226215 + }, + { + "id": "confirm-fresh-01-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-01-te-m0", + "scenario": "fresh-01", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": true, + "raw_status_correct": true, + "known_microinr": 127200, + "retained_microinr": 159000, + "prompt_tokens": 1412, + "completion_tokens": 1767, + "elapsed_seconds": 7.830779916999745, + "dispatch_time": 1791944473.562476 + }, + { + "id": "confirm-fresh-01-te-m0-source-chronological", + "history_id": "confirm-fresh-01-te-m0", + "scenario": "fresh-01", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 107295, + "retained_microinr": 134119, + "prompt_tokens": 873, + "completion_tokens": 1570, + "elapsed_seconds": 8.714128249994246, + "dispatch_time": 1791944472.227498 + }, + { + "id": "confirm-fresh-01-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-01-te-m0", + "scenario": "fresh-01", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 118845, + "retained_microinr": 148557, + "prompt_tokens": 1427, + "completion_tokens": 1624, + "elapsed_seconds": 8.95311099999526, + "dispatch_time": 1791944483.4324632 + }, + { + "id": "confirm-fresh-05-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-05-en-m0", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 119700, + "retained_microinr": 149625, + "prompt_tokens": 1388, + "completion_tokens": 1648, + "elapsed_seconds": 8.997502166006598, + "dispatch_time": 1791944482.103298 + }, + { + "id": "confirm-fresh-05-en-m0-source-chronological", + "history_id": "confirm-fresh-05-en-m0", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 117840, + "retained_microinr": 147300, + "prompt_tokens": 992, + "completion_tokens": 1716, + "elapsed_seconds": 9.598398582995287, + "dispatch_time": 1791944482.768945 + }, + { + "id": "confirm-fresh-05-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-05-en-m0", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 193395, + "retained_microinr": 241744, + "prompt_tokens": 1533, + "completion_tokens": 2840, + "elapsed_seconds": 13.827546749991598, + "dispatch_time": 1791944481.4361331 + }, + { + "id": "confirm-fresh-16-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-16-en-m1", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25260, + "retained_microinr": 31575, + "prompt_tokens": 1556, + "completion_tokens": 32, + "elapsed_seconds": 1.0170336669980315, + "dispatch_time": 1791944495.975161 + }, + { + "id": "confirm-fresh-16-en-m1-source-chronological", + "history_id": "confirm-fresh-16-en-m1", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 20640, + "retained_microinr": 25800, + "prompt_tokens": 1208, + "completion_tokens": 42, + "elapsed_seconds": 0.9932945830078097, + "dispatch_time": 1791944497.3069482 + }, + { + "id": "confirm-fresh-16-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-16-en-m1", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 29460, + "retained_microinr": 36825, + "prompt_tokens": 1796, + "completion_tokens": 42, + "elapsed_seconds": 0.9913481249968754, + "dispatch_time": 1791944495.309709 + }, + { + "id": "confirm-fresh-28-hi-m1-source-chronological", + "history_id": "confirm-fresh-28-hi-m1", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-29", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 26040, + "retained_microinr": 32550, + "prompt_tokens": 1568, + "completion_tokens": 42, + "elapsed_seconds": 0.9354872919939226, + "dispatch_time": 1791944496.642853 + }, + { + "id": "confirm-fresh-28-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-28-hi-m1", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-29", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34680, + "retained_microinr": 43350, + "prompt_tokens": 2144, + "completion_tokens": 42, + "elapsed_seconds": 1.0966805000061868, + "dispatch_time": 1791944498.34966 + }, + { + "id": "confirm-fresh-28-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-28-hi-m1", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-29", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 30660, + "retained_microinr": 38325, + "prompt_tokens": 1876, + "completion_tokens": 42, + "elapsed_seconds": 0.9837397079973016, + "dispatch_time": 1791944499.684756 + }, + { + "id": "confirm-fresh-34-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-34-en-m0", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 306315, + "retained_microinr": 382894, + "prompt_tokens": 2409, + "completion_tokens": 4503, + "elapsed_seconds": 18.816953500005184, + "dispatch_time": 1791944499.013639 + }, + { + "id": "confirm-fresh-34-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-en-m0", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 241200, + "retained_microinr": 301500, + "prompt_tokens": 2180, + "completion_tokens": 3475, + "elapsed_seconds": 14.660194209005567, + "dispatch_time": 1791944500.350304 + }, + { + "id": "confirm-fresh-34-en-m0-source-chronological", + "history_id": "confirm-fresh-34-en-m0", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 133395, + "retained_microinr": 166744, + "prompt_tokens": 1841, + "completion_tokens": 1763, + "elapsed_seconds": 7.799814999991213, + "dispatch_time": 1791944517.870946 + }, + { + "id": "confirm-fresh-40-te-m0-source-chronological", + "history_id": "confirm-fresh-40-te-m0", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 281160, + "retained_microinr": 351450, + "prompt_tokens": 1812, + "completion_tokens": 4233, + "elapsed_seconds": 20.638543459004723, + "dispatch_time": 1791944518.5391738 + }, + { + "id": "confirm-fresh-40-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-te-m0", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 283620, + "retained_microinr": 354525, + "prompt_tokens": 2104, + "completion_tokens": 4201, + "elapsed_seconds": 20.65557820899994, + "dispatch_time": 1791944519.205847 + }, + { + "id": "confirm-fresh-40-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-40-te-m0", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 190935, + "retained_microinr": 238669, + "prompt_tokens": 2397, + "completion_tokens": 2583, + "elapsed_seconds": 13.740064708996215, + "dispatch_time": 1791944519.872498 + }, + { + "id": "confirm-fresh-03-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-03-hi-m0", + "scenario": "fresh-03", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 167850, + "retained_microinr": 209813, + "prompt_tokens": 1410, + "completion_tokens": 2445, + "elapsed_seconds": 11.524418792003416, + "dispatch_time": 1791944539.9006982 + }, + { + "id": "confirm-fresh-03-hi-m0-source-chronological", + "history_id": "confirm-fresh-03-hi-m0", + "scenario": "fresh-03", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 140550, + "retained_microinr": 175688, + "prompt_tokens": 870, + "completion_tokens": 2125, + "elapsed_seconds": 9.191295625001658, + "dispatch_time": 1791944541.901828 + }, + { + "id": "confirm-fresh-03-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-03-hi-m0", + "scenario": "fresh-03", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 130935, + "retained_microinr": 163669, + "prompt_tokens": 1425, + "completion_tokens": 1826, + "elapsed_seconds": 8.684030208009062, + "dispatch_time": 1791944540.568465 + }, + { + "id": "confirm-fresh-11-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-11-en-m1", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27960, + "retained_microinr": 34950, + "prompt_tokens": 1696, + "completion_tokens": 42, + "elapsed_seconds": 0.973132875005831, + "dispatch_time": 1791944541.231704 + }, + { + "id": "confirm-fresh-11-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-11-en-m1", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 30195, + "retained_microinr": 37744, + "prompt_tokens": 1845, + "completion_tokens": 42, + "elapsed_seconds": 1.0418019160133554, + "dispatch_time": 1791944553.458821 + }, + { + "id": "confirm-fresh-11-en-m1-source-chronological", + "history_id": "confirm-fresh-11-en-m1", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 19920, + "retained_microinr": 24900, + "prompt_tokens": 1160, + "completion_tokens": 42, + "elapsed_seconds": 0.9567370420118095, + "dispatch_time": 1791944551.4580228 + }, + { + "id": "confirm-fresh-26-en-m0-source-chronological", + "history_id": "confirm-fresh-26-en-m0", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 41730, + "retained_microinr": 52163, + "prompt_tokens": 1314, + "completion_tokens": 367, + "elapsed_seconds": 2.375200208000024, + "dispatch_time": 1791944552.1258838 + }, + { + "id": "confirm-fresh-26-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-26-en-m0", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 173745, + "retained_microinr": 217182, + "prompt_tokens": 1819, + "completion_tokens": 2441, + "elapsed_seconds": 11.03292225000041, + "dispatch_time": 1791944552.7931788 + }, + { + "id": "confirm-fresh-26-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-26-en-m0", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 208545, + "retained_microinr": 260682, + "prompt_tokens": 1819, + "completion_tokens": 3021, + "elapsed_seconds": 13.296980833009002, + "dispatch_time": 1791944564.532604 + }, + { + "id": "confirm-fresh-15-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-15-hi-m0", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 293535, + "retained_microinr": 366919, + "prompt_tokens": 1417, + "completion_tokens": 4538, + "elapsed_seconds": 21.559326667003916, + "dispatch_time": 1791944565.863648 + }, + { + "id": "confirm-fresh-15-hi-m0-source-chronological", + "history_id": "confirm-fresh-15-hi-m0", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 337890, + "retained_microinr": 422363, + "prompt_tokens": 1034, + "completion_tokens": 5373, + "elapsed_seconds": 23.664025082995067, + "dispatch_time": 1791944563.864767 + }, + { + "id": "confirm-fresh-15-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-15-hi-m0", + "scenario": "fresh-15", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 150435, + "retained_microinr": 188044, + "prompt_tokens": 1585, + "completion_tokens": 2111, + "elapsed_seconds": 9.372164083993994, + "dispatch_time": 1791944565.199101 + }, + { + "id": "confirm-fresh-09-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-09-en-m1", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": true, + "raw_status_correct": true, + "known_microinr": 30330, + "retained_microinr": 37913, + "prompt_tokens": 1894, + "completion_tokens": 32, + "elapsed_seconds": 1.1083831249998184, + "dispatch_time": 1791944589.562026 + }, + { + "id": "confirm-fresh-09-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-09-en-m1", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27930, + "retained_microinr": 34913, + "prompt_tokens": 1694, + "completion_tokens": 42, + "elapsed_seconds": 0.9687332499888726, + "dispatch_time": 1791944588.8960168 + }, + { + "id": "confirm-fresh-09-en-m1-source-chronological", + "history_id": "confirm-fresh-09-en-m1", + "scenario": "fresh-09", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 20565, + "retained_microinr": 25707, + "prompt_tokens": 1203, + "completion_tokens": 42, + "elapsed_seconds": 1.034745249999105, + "dispatch_time": 1791944588.230817 + }, + { + "id": "confirm-fresh-38-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-38-hi-m0", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 394230, + "retained_microinr": 492788, + "prompt_tokens": 2030, + "completion_tokens": 6063, + "elapsed_seconds": 26.71671595799853, + "dispatch_time": 1791944587.5624819 + }, + { + "id": "confirm-fresh-38-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-hi-m0", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 378120, + "retained_microinr": 472650, + "prompt_tokens": 1812, + "completion_tokens": 5849, + "elapsed_seconds": 25.631007999996655, + "dispatch_time": 1791944614.990104 + }, + { + "id": "confirm-fresh-38-hi-m0-source-chronological", + "history_id": "confirm-fresh-38-hi-m0", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 199665, + "retained_microinr": 249582, + "prompt_tokens": 1595, + "completion_tokens": 2929, + "elapsed_seconds": 13.397669375000987, + "dispatch_time": 1791944615.656765 + }, + { + "id": "confirm-fresh-24-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-24-te-m0", + "scenario": "fresh-24", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 219855, + "retained_microinr": 274819, + "prompt_tokens": 1833, + "completion_tokens": 3206, + "elapsed_seconds": 16.238885291997576, + "dispatch_time": 1791944614.3218749 + }, + { + "id": "confirm-fresh-24-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-24-te-m0", + "scenario": "fresh-24", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 351075, + "retained_microinr": 438844, + "prompt_tokens": 1833, + "completion_tokens": 5393, + "elapsed_seconds": 26.072659749988816, + "dispatch_time": 1791944616.3234298 + }, + { + "id": "confirm-fresh-24-te-m0-source-chronological", + "history_id": "confirm-fresh-24-te-m0", + "scenario": "fresh-24", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 82830, + "retained_microinr": 103538, + "prompt_tokens": 1334, + "completion_tokens": 1047, + "elapsed_seconds": 6.182483416996547, + "dispatch_time": 1791944643.772303 + }, + { + "id": "confirm-fresh-07-en-m0-source-chronological", + "history_id": "confirm-fresh-07-en-m0", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 151065, + "retained_microinr": 188832, + "prompt_tokens": 943, + "completion_tokens": 2282, + "elapsed_seconds": 10.988950625003781, + "dispatch_time": 1791944644.4389482 + }, + { + "id": "confirm-fresh-07-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-07-en-m0", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 90975, + "retained_microinr": 113719, + "prompt_tokens": 1477, + "completion_tokens": 1147, + "elapsed_seconds": 6.121087374995113, + "dispatch_time": 1791944642.437258 + }, + { + "id": "confirm-fresh-07-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-en-m0", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 127320, + "retained_microinr": 159150, + "prompt_tokens": 1384, + "completion_tokens": 1776, + "elapsed_seconds": 9.836480041005416, + "dispatch_time": 1791944643.105612 + }, + { + "id": "confirm-fresh-11-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-11-te-m0", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 252630, + "retained_microinr": 315788, + "prompt_tokens": 1522, + "completion_tokens": 3830, + "elapsed_seconds": 16.252762250005617, + "dispatch_time": 1791944655.476238 + }, + { + "id": "confirm-fresh-11-te-m0-source-chronological", + "history_id": "confirm-fresh-11-te-m0", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 208215, + "retained_microinr": 260269, + "prompt_tokens": 981, + "completion_tokens": 3225, + "elapsed_seconds": 15.773785208002664, + "dispatch_time": 1791944656.8072162 + }, + { + "id": "confirm-fresh-11-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-11-te-m0", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 260205, + "retained_microinr": 325257, + "prompt_tokens": 1407, + "completion_tokens": 3985, + "elapsed_seconds": 16.09192604100099, + "dispatch_time": 1791944656.142412 + }, + { + "id": "confirm-fresh-33-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-33-en-m0", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 119145, + "retained_microinr": 148932, + "prompt_tokens": 2259, + "completion_tokens": 1421, + "elapsed_seconds": 6.597253208004986, + "dispatch_time": 1791944657.4761689 + }, + { + "id": "confirm-fresh-33-en-m0-source-chronological", + "history_id": "confirm-fresh-33-en-m0", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 374970, + "retained_microinr": 468713, + "prompt_tokens": 1722, + "completion_tokens": 5819, + "elapsed_seconds": 25.90595170899178, + "dispatch_time": 1791944673.300045 + }, + { + "id": "confirm-fresh-33-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-en-m0", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 469485, + "retained_microinr": 586857, + "prompt_tokens": 2055, + "completion_tokens": 7311, + "elapsed_seconds": 30.04814375001297, + "dispatch_time": 1791944672.6347048 + }, + { + "id": "confirm-fresh-10-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-10-en-m1", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25560, + "retained_microinr": 31950, + "prompt_tokens": 1536, + "completion_tokens": 42, + "elapsed_seconds": 0.9771300419961335, + "dispatch_time": 1791944674.632759 + }, + { + "id": "confirm-fresh-10-en-m1-source-chronological", + "history_id": "confirm-fresh-10-en-m1", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 19035, + "retained_microinr": 23794, + "prompt_tokens": 1101, + "completion_tokens": 42, + "elapsed_seconds": 1.0250782080111094, + "dispatch_time": 1791944673.963528 + }, + { + "id": "confirm-fresh-10-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-10-en-m1", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27660, + "retained_microinr": 34575, + "prompt_tokens": 1676, + "completion_tokens": 42, + "elapsed_seconds": 1.1817924580100225, + "dispatch_time": 1791944702.741951 + }, + { + "id": "confirm-fresh-26-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-26-hi-m1", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34035, + "retained_microinr": 42544, + "prompt_tokens": 2109, + "completion_tokens": 40, + "elapsed_seconds": 0.9629914580000332, + "dispatch_time": 1791944704.7382789 + }, + { + "id": "confirm-fresh-26-hi-m1-source-chronological", + "history_id": "confirm-fresh-26-hi-m1", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24420, + "retained_microinr": 30525, + "prompt_tokens": 1468, + "completion_tokens": 40, + "elapsed_seconds": 1.5600061250006547, + "dispatch_time": 1791944703.4091258 + }, + { + "id": "confirm-fresh-26-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-26-hi-m1", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34035, + "retained_microinr": 42544, + "prompt_tokens": 2109, + "completion_tokens": 40, + "elapsed_seconds": 0.948893875000067, + "dispatch_time": 1791944704.076213 + }, + { + "id": "confirm-fresh-18-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-en-m0", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-04-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 177240, + "retained_microinr": 221550, + "prompt_tokens": 1572, + "completion_tokens": 2561, + "elapsed_seconds": 11.11054474998673, + "dispatch_time": 1791944705.747844 + }, + { + "id": "confirm-fresh-18-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-18-en-m0", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 221220, + "retained_microinr": 276525, + "prompt_tokens": 1624, + "completion_tokens": 3281, + "elapsed_seconds": 17.147750999996788, + "dispatch_time": 1791944707.082243 + }, + { + "id": "confirm-fresh-18-en-m0-source-chronological", + "history_id": "confirm-fresh-18-en-m0", + "scenario": "fresh-18", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 419910, + "retained_microinr": 524888, + "prompt_tokens": 1034, + "completion_tokens": 6740, + "elapsed_seconds": 31.196360917005222, + "dispatch_time": 1791944706.412035 + }, + { + "id": "confirm-fresh-23-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-23-te-m0", + "scenario": "fresh-23", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 524145, + "retained_microinr": 655182, + "prompt_tokens": 2175, + "completion_tokens": 8192, + "elapsed_seconds": 38.70535054200445, + "dispatch_time": 1791944707.748686 + }, + { + "id": "confirm-fresh-23-te-m0-source-chronological", + "history_id": "confirm-fresh-23-te-m0", + "scenario": "fresh-23", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 102390, + "retained_microinr": 127988, + "prompt_tokens": 1482, + "completion_tokens": 1336, + "elapsed_seconds": 7.541762499997276, + "dispatch_time": 1791944747.841343 + }, + { + "id": "confirm-fresh-23-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-23-te-m0", + "scenario": "fresh-23", + "family": "approved_change", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 524145, + "retained_microinr": 655182, + "prompt_tokens": 2175, + "completion_tokens": 8192, + "elapsed_seconds": 34.90903183299815, + "dispatch_time": 1791944746.508492 + }, + { + "id": "confirm-fresh-30-en-m1-source-chronological", + "history_id": "confirm-fresh-30-en-m1", + "scenario": "fresh-30", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24360, + "retained_microinr": 30450, + "prompt_tokens": 1464, + "completion_tokens": 40, + "elapsed_seconds": 1.0194091250014026, + "dispatch_time": 1791944748.5086489 + }, + { + "id": "confirm-fresh-30-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-30-en-m1", + "scenario": "fresh-30", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34140, + "retained_microinr": 42675, + "prompt_tokens": 2116, + "completion_tokens": 40, + "elapsed_seconds": 0.9591924999986077, + "dispatch_time": 1791944747.171835 + }, + { + "id": "confirm-fresh-30-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-en-m1", + "scenario": "fresh-30", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 34260, + "retained_microinr": 42825, + "prompt_tokens": 2116, + "completion_tokens": 42, + "elapsed_seconds": 0.9940253750100965, + "dispatch_time": 1791944782.130555 + }, + { + "id": "confirm-fresh-17-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-17-te-m1", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24960, + "retained_microinr": 31200, + "prompt_tokens": 1496, + "completion_tokens": 42, + "elapsed_seconds": 1.1015573330078041, + "dispatch_time": 1791944781.461963 + }, + { + "id": "confirm-fresh-17-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-17-te-m1", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24165, + "retained_microinr": 30207, + "prompt_tokens": 1443, + "completion_tokens": 42, + "elapsed_seconds": 0.953701791993808, + "dispatch_time": 1791944783.4638028 + }, + { + "id": "confirm-fresh-17-te-m1-source-chronological", + "history_id": "confirm-fresh-17-te-m1", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 16635, + "retained_microinr": 20794, + "prompt_tokens": 941, + "completion_tokens": 42, + "elapsed_seconds": 0.9648898750019725, + "dispatch_time": 1791944782.796458 + }, + { + "id": "confirm-fresh-36-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-36-hi-m1", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 41280, + "retained_microinr": 51600, + "prompt_tokens": 2584, + "completion_tokens": 42, + "elapsed_seconds": 0.9925268750084797, + "dispatch_time": 1791944786.464291 + }, + { + "id": "confirm-fresh-36-hi-m1-source-chronological", + "history_id": "confirm-fresh-36-hi-m1", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 32730, + "retained_microinr": 40913, + "prompt_tokens": 2014, + "completion_tokens": 42, + "elapsed_seconds": 1.0158505830040667, + "dispatch_time": 1791944784.467892 + }, + { + "id": "confirm-fresh-36-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-hi-m1", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 35595, + "retained_microinr": 44494, + "prompt_tokens": 2205, + "completion_tokens": 42, + "elapsed_seconds": 1.095116250013234, + "dispatch_time": 1791944785.13235 + }, + { + "id": "confirm-fresh-28-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-28-te-m1", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34200, + "retained_microinr": 42750, + "prompt_tokens": 2120, + "completion_tokens": 40, + "elapsed_seconds": 0.9960944159975043, + "dispatch_time": 1791944785.797013 + }, + { + "id": "confirm-fresh-28-te-m1-source-chronological", + "history_id": "confirm-fresh-28-te-m1", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24555, + "retained_microinr": 30694, + "prompt_tokens": 1477, + "completion_tokens": 40, + "elapsed_seconds": 0.9413197079993552, + "dispatch_time": 1791944788.177251 + }, + { + "id": "confirm-fresh-28-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-28-te-m1", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 34200, + "retained_microinr": 42750, + "prompt_tokens": 2120, + "completion_tokens": 40, + "elapsed_seconds": 0.9589607079979032, + "dispatch_time": 1791944789.512929 + }, + { + "id": "confirm-fresh-29-hi-m1-source-chronological", + "history_id": "confirm-fresh-29-hi-m1", + "scenario": "fresh-29", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25065, + "retained_microinr": 31332, + "prompt_tokens": 1511, + "completion_tokens": 40, + "elapsed_seconds": 0.9863521669904003, + "dispatch_time": 1791944787.513746 + }, + { + "id": "confirm-fresh-29-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-29-hi-m1", + "scenario": "fresh-29", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36480, + "retained_microinr": 45600, + "prompt_tokens": 2272, + "completion_tokens": 40, + "elapsed_seconds": 1.0490005830070004, + "dispatch_time": 1791944788.848604 + }, + { + "id": "confirm-fresh-29-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-29-hi-m1", + "scenario": "fresh-29", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36480, + "retained_microinr": 45600, + "prompt_tokens": 2272, + "completion_tokens": 40, + "elapsed_seconds": 0.9709048750082729, + "dispatch_time": 1791944790.520779 + }, + { + "id": "confirm-fresh-35-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-35-te-m1", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 42435, + "retained_microinr": 53044, + "prompt_tokens": 2661, + "completion_tokens": 42, + "elapsed_seconds": 0.9930545829993207, + "dispatch_time": 1791944791.185606 + }, + { + "id": "confirm-fresh-35-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-te-m1", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 37935, + "retained_microinr": 47419, + "prompt_tokens": 2361, + "completion_tokens": 42, + "elapsed_seconds": 0.9639243749988964, + "dispatch_time": 1791944791.852422 + }, + { + "id": "confirm-fresh-35-te-m1-source-chronological", + "history_id": "confirm-fresh-35-te-m1", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 32310, + "retained_microinr": 40388, + "prompt_tokens": 1986, + "completion_tokens": 42, + "elapsed_seconds": 1.0494226250011707, + "dispatch_time": 1791944792.5187142 + }, + { + "id": "confirm-fresh-04-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-04-en-m1", + "scenario": "fresh-04", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": true, + "raw_status_correct": true, + "known_microinr": 24885, + "retained_microinr": 31107, + "prompt_tokens": 1531, + "completion_tokens": 32, + "elapsed_seconds": 1.0147467499919003, + "dispatch_time": 1791944794.270617 + }, + { + "id": "confirm-fresh-04-en-m1-source-chronological", + "history_id": "confirm-fresh-04-en-m1", + "scenario": "fresh-04", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 18915, + "retained_microinr": 23644, + "prompt_tokens": 1093, + "completion_tokens": 42, + "elapsed_seconds": 0.9243329579912825, + "dispatch_time": 1791944794.937305 + }, + { + "id": "confirm-fresh-04-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-04-en-m1", + "scenario": "fresh-04", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27540, + "retained_microinr": 34425, + "prompt_tokens": 1668, + "completion_tokens": 42, + "elapsed_seconds": 1.016910625010496, + "dispatch_time": 1791944795.6035252 + }, + { + "id": "confirm-fresh-12-en-m1-source-chronological", + "history_id": "confirm-fresh-12-en-m1", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 18675, + "retained_microinr": 23344, + "prompt_tokens": 1077, + "completion_tokens": 42, + "elapsed_seconds": 1.0130689579964383, + "dispatch_time": 1791944793.602093 + }, + { + "id": "confirm-fresh-12-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-12-en-m1", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27480, + "retained_microinr": 34350, + "prompt_tokens": 1664, + "completion_tokens": 42, + "elapsed_seconds": 0.9493297080043703, + "dispatch_time": 1791944798.658613 + }, + { + "id": "confirm-fresh-12-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-en-m1", + "scenario": "fresh-12", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25800, + "retained_microinr": 32250, + "prompt_tokens": 1552, + "completion_tokens": 42, + "elapsed_seconds": 1.10825108300196, + "dispatch_time": 1791944796.657517 + }, + { + "id": "confirm-fresh-40-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-40-te-m1", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 38715, + "retained_microinr": 48394, + "prompt_tokens": 2413, + "completion_tokens": 42, + "elapsed_seconds": 1.0076746659906348, + "dispatch_time": 1791944797.9921632 + }, + { + "id": "confirm-fresh-40-te-m1-source-chronological", + "history_id": "confirm-fresh-40-te-m1", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 30165, + "retained_microinr": 37707, + "prompt_tokens": 1843, + "completion_tokens": 42, + "elapsed_seconds": 0.9605918339948403, + "dispatch_time": 1791944797.325459 + }, + { + "id": "confirm-fresh-40-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-te-m1", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 34005, + "retained_microinr": 42507, + "prompt_tokens": 2099, + "completion_tokens": 42, + "elapsed_seconds": 0.9818256669968832, + "dispatch_time": 1791944800.976195 + }, + { + "id": "confirm-fresh-06-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-06-hi-m1", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25515, + "retained_microinr": 31894, + "prompt_tokens": 1573, + "completion_tokens": 32, + "elapsed_seconds": 1.025581499998225, + "dispatch_time": 1791944800.3116338 + }, + { + "id": "confirm-fresh-06-hi-m1-source-chronological", + "history_id": "confirm-fresh-06-hi-m1", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-07", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 17265, + "retained_microinr": 21582, + "prompt_tokens": 983, + "completion_tokens": 42, + "elapsed_seconds": 0.9718502920004539, + "dispatch_time": 1791944801.6449869 + }, + { + "id": "confirm-fresh-06-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-06-hi-m1", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25020, + "retained_microinr": 31275, + "prompt_tokens": 1540, + "completion_tokens": 32, + "elapsed_seconds": 0.9290839580062311, + "dispatch_time": 1791944799.6433222 + }, + { + "id": "confirm-fresh-34-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-hi-m0", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 395865, + "retained_microinr": 494832, + "prompt_tokens": 1911, + "completion_tokens": 6120, + "elapsed_seconds": 25.19336029200349, + "dispatch_time": 1791944803.3204632 + }, + { + "id": "confirm-fresh-34-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-34-hi-m0", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 75945, + "retained_microinr": 94932, + "prompt_tokens": 2099, + "completion_tokens": 741, + "elapsed_seconds": 3.773117333999835, + "dispatch_time": 1791944804.653816 + }, + { + "id": "confirm-fresh-34-hi-m0-source-chronological", + "history_id": "confirm-fresh-34-hi-m0", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 198480, + "retained_microinr": 248100, + "prompt_tokens": 1676, + "completion_tokens": 2889, + "elapsed_seconds": 15.11056762500084, + "dispatch_time": 1791944802.6531231 + }, + { + "id": "confirm-fresh-03-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-03-en-m0", + "scenario": "fresh-03", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 109410, + "retained_microinr": 136763, + "prompt_tokens": 1438, + "completion_tokens": 1464, + "elapsed_seconds": 6.87887695900281, + "dispatch_time": 1791944803.987134 + }, + { + "id": "confirm-fresh-03-en-m0-source-chronological", + "history_id": "confirm-fresh-03-en-m0", + "scenario": "fresh-03", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 371040, + "retained_microinr": 463800, + "prompt_tokens": 892, + "completion_tokens": 5961, + "elapsed_seconds": 26.041728374999366, + "dispatch_time": 1791944828.555746 + }, + { + "id": "confirm-fresh-03-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-03-en-m0", + "scenario": "fresh-03", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 75705, + "retained_microinr": 94632, + "prompt_tokens": 1403, + "completion_tokens": 911, + "elapsed_seconds": 5.302581875002943, + "dispatch_time": 1791944829.2222831 + }, + { + "id": "confirm-fresh-07-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-07-hi-m0", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 432255, + "retained_microinr": 540319, + "prompt_tokens": 1457, + "completion_tokens": 6840, + "elapsed_seconds": 28.69123499999114, + "dispatch_time": 1791944829.889363 + }, + { + "id": "confirm-fresh-07-hi-m0-source-chronological", + "history_id": "confirm-fresh-07-hi-m0", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 109245, + "retained_microinr": 136557, + "prompt_tokens": 899, + "completion_tokens": 1596, + "elapsed_seconds": 7.1098634590016445, + "dispatch_time": 1791944830.551471 + }, + { + "id": "confirm-fresh-07-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-hi-m0", + "scenario": "fresh-07", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 330135, + "retained_microinr": 412669, + "prompt_tokens": 1405, + "completion_tokens": 5151, + "elapsed_seconds": 22.43139166700712, + "dispatch_time": 1791944860.628047 + }, + { + "id": "confirm-fresh-11-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-11-en-m0", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 124920, + "retained_microinr": 156150, + "prompt_tokens": 1416, + "completion_tokens": 1728, + "elapsed_seconds": 8.231476125001791, + "dispatch_time": 1791944859.298134 + }, + { + "id": "confirm-fresh-11-en-m0-source-chronological", + "history_id": "confirm-fresh-11-en-m0", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 290565, + "retained_microinr": 363207, + "prompt_tokens": 931, + "completion_tokens": 4610, + "elapsed_seconds": 19.237540375004755, + "dispatch_time": 1791944858.6314209 + }, + { + "id": "confirm-fresh-11-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-11-en-m0", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 138270, + "retained_microinr": 172838, + "prompt_tokens": 1478, + "completion_tokens": 1935, + "elapsed_seconds": 8.441571290997672, + "dispatch_time": 1791944859.9634619 + }, + { + "id": "confirm-fresh-03-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-03-hi-m1", + "scenario": "fresh-03", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 23790, + "retained_microinr": 29738, + "prompt_tokens": 1418, + "completion_tokens": 42, + "elapsed_seconds": 0.9592180410108995, + "dispatch_time": 1791944883.108303 + }, + { + "id": "confirm-fresh-03-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-03-hi-m1", + "scenario": "fresh-03", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 23985, + "retained_microinr": 29982, + "prompt_tokens": 1431, + "completion_tokens": 42, + "elapsed_seconds": 1.0214134580019163, + "dispatch_time": 1791944884.4398699 + }, + { + "id": "confirm-fresh-03-hi-m1-source-chronological", + "history_id": "confirm-fresh-03-hi-m1", + "scenario": "fresh-03", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 15600, + "retained_microinr": 19500, + "prompt_tokens": 872, + "completion_tokens": 42, + "elapsed_seconds": 1.0193654169997899, + "dispatch_time": 1791944883.774751 + }, + { + "id": "confirm-fresh-06-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-06-en-m0", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 165450, + "retained_microinr": 206813, + "prompt_tokens": 1242, + "completion_tokens": 2447, + "elapsed_seconds": 10.48811883298913, + "dispatch_time": 1791944885.1080902 + }, + { + "id": "confirm-fresh-06-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-06-en-m0", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 93180, + "retained_microinr": 116475, + "prompt_tokens": 1300, + "completion_tokens": 1228, + "elapsed_seconds": 5.797769165990758, + "dispatch_time": 1791944896.300808 + }, + { + "id": "confirm-fresh-06-en-m0-source-chronological", + "history_id": "confirm-fresh-06-en-m0", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 127215, + "retained_microinr": 159019, + "prompt_tokens": 865, + "completion_tokens": 1904, + "elapsed_seconds": 10.156719042002806, + "dispatch_time": 1791944896.969085 + }, + { + "id": "confirm-fresh-28-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-28-en-m0", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 160695, + "retained_microinr": 200869, + "prompt_tokens": 1829, + "completion_tokens": 2221, + "elapsed_seconds": 11.363244332998875, + "dispatch_time": 1791944897.6333451 + }, + { + "id": "confirm-fresh-28-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-28-en-m0", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 196395, + "retained_microinr": 245494, + "prompt_tokens": 1829, + "completion_tokens": 2816, + "elapsed_seconds": 12.541640791998361, + "dispatch_time": 1791944895.6361878 + }, + { + "id": "confirm-fresh-28-en-m0-source-chronological", + "history_id": "confirm-fresh-28-en-m0", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 132360, + "retained_microinr": 165450, + "prompt_tokens": 1316, + "completion_tokens": 1877, + "elapsed_seconds": 8.708016791002592, + "dispatch_time": 1791944910.371223 + }, + { + "id": "confirm-fresh-20-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-20-te-m0", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 247140, + "retained_microinr": 308925, + "prompt_tokens": 1288, + "completion_tokens": 3797, + "elapsed_seconds": 19.656710999988718, + "dispatch_time": 1791944909.704544 + }, + { + "id": "confirm-fresh-20-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-20-te-m0", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": true, + "raw_status_correct": false, + "known_microinr": 413685, + "retained_microinr": 517107, + "prompt_tokens": 1303, + "completion_tokens": 6569, + "elapsed_seconds": 27.481848375013215, + "dispatch_time": 1791944911.036484 + }, + { + "id": "confirm-fresh-20-te-m0-source-chronological", + "history_id": "confirm-fresh-20-te-m0", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 337245, + "retained_microinr": 421557, + "prompt_tokens": 863, + "completion_tokens": 5405, + "elapsed_seconds": 26.097864416995435, + "dispatch_time": 1791944909.036414 + }, + { + "id": "confirm-fresh-33-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-hi-m0", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 120720, + "retained_microinr": 150900, + "prompt_tokens": 2076, + "completion_tokens": 1493, + "elapsed_seconds": 8.439129499995033, + "dispatch_time": 1791944940.559493 + }, + { + "id": "confirm-fresh-33-hi-m0-source-chronological", + "history_id": "confirm-fresh-33-hi-m0", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 167610, + "retained_microinr": 209513, + "prompt_tokens": 1870, + "completion_tokens": 2326, + "elapsed_seconds": 10.071834040994872, + "dispatch_time": 1791944939.897184 + }, + { + "id": "confirm-fresh-33-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-33-hi-m0", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 298050, + "retained_microinr": 372563, + "prompt_tokens": 2418, + "completion_tokens": 4363, + "elapsed_seconds": 18.164840333003667, + "dispatch_time": 1791944938.563628 + }, + { + "id": "confirm-fresh-16-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-16-te-m1", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 29415, + "retained_microinr": 36769, + "prompt_tokens": 1793, + "completion_tokens": 42, + "elapsed_seconds": 1.1345692500035511, + "dispatch_time": 1791944939.230287 + }, + { + "id": "confirm-fresh-16-te-m1-source-chronological", + "history_id": "confirm-fresh-16-te-m1", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 20625, + "retained_microinr": 25782, + "prompt_tokens": 1207, + "completion_tokens": 42, + "elapsed_seconds": 0.9535644579882501, + "dispatch_time": 1791944956.772683 + }, + { + "id": "confirm-fresh-16-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-16-te-m1", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25155, + "retained_microinr": 31444, + "prompt_tokens": 1549, + "completion_tokens": 32, + "elapsed_seconds": 1.0166007499938132, + "dispatch_time": 1791944958.770329 + }, + { + "id": "confirm-fresh-26-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-26-te-m0", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 128520, + "retained_microinr": 160650, + "prompt_tokens": 1836, + "completion_tokens": 1683, + "elapsed_seconds": 7.600142416995368, + "dispatch_time": 1791944958.103969 + }, + { + "id": "confirm-fresh-26-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-26-te-m0", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 519060, + "retained_microinr": 648825, + "prompt_tokens": 1836, + "completion_tokens": 8192, + "elapsed_seconds": 33.65626324999903, + "dispatch_time": 1791944957.437387 + }, + { + "id": "confirm-fresh-26-te-m0-source-chronological", + "history_id": "confirm-fresh-26-te-m0", + "scenario": "fresh-26", + "family": "approval_boundary", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 77655, + "retained_microinr": 97069, + "prompt_tokens": 1333, + "completion_tokens": 961, + "elapsed_seconds": 5.778320624987828, + "dispatch_time": 1791944992.464446 + }, + { + "id": "confirm-fresh-20-en-m0-source-chronological", + "history_id": "confirm-fresh-20-en-m0", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 97350, + "retained_microinr": 121688, + "prompt_tokens": 878, + "completion_tokens": 1403, + "elapsed_seconds": 7.568635333998827, + "dispatch_time": 1791944993.1307468 + }, + { + "id": "confirm-fresh-20-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-20-en-m0", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 135315, + "retained_microinr": 169144, + "prompt_tokens": 1285, + "completion_tokens": 1934, + "elapsed_seconds": 8.803637250006432, + "dispatch_time": 1791944991.130285 + }, + { + "id": "confirm-fresh-20-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-20-en-m0", + "scenario": "fresh-20", + "family": "separate_booking", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 511305, + "retained_microinr": 639132, + "prompt_tokens": 1319, + "completion_tokens": 8192, + "elapsed_seconds": 35.081754541999544, + "dispatch_time": 1791944991.798576 + }, + { + "id": "confirm-fresh-34-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-34-te-m1", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 40290, + "retained_microinr": 50363, + "prompt_tokens": 2518, + "completion_tokens": 42, + "elapsed_seconds": 1.064101041003596, + "dispatch_time": 1791945027.594038 + }, + { + "id": "confirm-fresh-34-te-m1-source-chronological", + "history_id": "confirm-fresh-34-te-m1", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 31710, + "retained_microinr": 39638, + "prompt_tokens": 1946, + "completion_tokens": 42, + "elapsed_seconds": 0.9830327079980634, + "dispatch_time": 1791945028.9301639 + }, + { + "id": "confirm-fresh-34-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-te-m1", + "scenario": "fresh-34", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 35790, + "retained_microinr": 44738, + "prompt_tokens": 2218, + "completion_tokens": 42, + "elapsed_seconds": 0.9962496660009492, + "dispatch_time": 1791945028.263675 + }, + { + "id": "confirm-fresh-25-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-25-hi-m0", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 305730, + "retained_microinr": 382163, + "prompt_tokens": 1990, + "completion_tokens": 4598, + "elapsed_seconds": 20.621692165994318, + "dispatch_time": 1791945026.930353 + }, + { + "id": "confirm-fresh-25-hi-m0-source-chronological", + "history_id": "confirm-fresh-25-hi-m0", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 231165, + "retained_microinr": 288957, + "prompt_tokens": 1367, + "completion_tokens": 3511, + "elapsed_seconds": 15.08111558300152, + "dispatch_time": 1791945048.265032 + }, + { + "id": "confirm-fresh-25-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-25-hi-m0", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 349230, + "retained_microinr": 436538, + "prompt_tokens": 1990, + "completion_tokens": 5323, + "elapsed_seconds": 24.895239624995156, + "dispatch_time": 1791945047.598707 + }, + { + "id": "confirm-fresh-14-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-14-en-m1", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 26670, + "retained_microinr": 33338, + "prompt_tokens": 1650, + "completion_tokens": 32, + "elapsed_seconds": 1.0856207500037272, + "dispatch_time": 1791945049.595147 + }, + { + "id": "confirm-fresh-14-en-m1-source-chronological", + "history_id": "confirm-fresh-14-en-m1", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 17985, + "retained_microinr": 22482, + "prompt_tokens": 1071, + "completion_tokens": 32, + "elapsed_seconds": 1.0340699160005897, + "dispatch_time": 1791945048.930488 + }, + { + "id": "confirm-fresh-14-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-14-en-m1", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 25095, + "retained_microinr": 31369, + "prompt_tokens": 1545, + "completion_tokens": 32, + "elapsed_seconds": 1.0166327920014737, + "dispatch_time": 1791945072.527575 + }, + { + "id": "confirm-fresh-02-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-02-en-m1", + "scenario": "fresh-02", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27435, + "retained_microinr": 34294, + "prompt_tokens": 1661, + "completion_tokens": 42, + "elapsed_seconds": 1.1004021249973448, + "dispatch_time": 1791945074.5281248 + }, + { + "id": "confirm-fresh-02-en-m1-source-chronological", + "history_id": "confirm-fresh-02-en-m1", + "scenario": "fresh-02", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 18630, + "retained_microinr": 23288, + "prompt_tokens": 1074, + "completion_tokens": 42, + "elapsed_seconds": 0.9425836250011344, + "dispatch_time": 1791945073.862349 + }, + { + "id": "confirm-fresh-02-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-02-en-m1", + "scenario": "fresh-02", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": true, + "raw_status_correct": true, + "known_microinr": 25065, + "retained_microinr": 31332, + "prompt_tokens": 1543, + "completion_tokens": 32, + "elapsed_seconds": 1.0150365410081577, + "dispatch_time": 1791945073.1933382 + }, + { + "id": "confirm-fresh-22-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-22-en-m0", + "scenario": "fresh-22", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 63330, + "retained_microinr": 79163, + "prompt_tokens": 1822, + "completion_tokens": 600, + "elapsed_seconds": 3.3389219170057913, + "dispatch_time": 1791945077.010679 + }, + { + "id": "confirm-fresh-22-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-22-en-m0", + "scenario": "fresh-22", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 245610, + "retained_microinr": 307013, + "prompt_tokens": 1822, + "completion_tokens": 3638, + "elapsed_seconds": 17.630239791003987, + "dispatch_time": 1791945077.6768022 + }, + { + "id": "confirm-fresh-22-en-m0-source-chronological", + "history_id": "confirm-fresh-22-en-m0", + "scenario": "fresh-22", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 104670, + "retained_microinr": 130838, + "prompt_tokens": 1310, + "completion_tokens": 1417, + "elapsed_seconds": 6.776233458993374, + "dispatch_time": 1791945075.6755662 + }, + { + "id": "confirm-fresh-10-te-m0-source-chronological", + "history_id": "confirm-fresh-10-te-m0", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 301395, + "retained_microinr": 376744, + "prompt_tokens": 901, + "completion_tokens": 4798, + "elapsed_seconds": 21.312018374999752, + "dispatch_time": 1791945076.344018 + }, + { + "id": "confirm-fresh-10-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-10-te-m0", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": true, + "raw_status_correct": false, + "known_microinr": 351285, + "retained_microinr": 439107, + "prompt_tokens": 1251, + "completion_tokens": 5542, + "elapsed_seconds": 23.39471333300753, + "dispatch_time": 1791945097.7006688 + }, + { + "id": "confirm-fresh-10-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-10-te-m0", + "scenario": "fresh-10", + "family": "resolved_condition", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 371925, + "retained_microinr": 464907, + "prompt_tokens": 1335, + "completion_tokens": 5865, + "elapsed_seconds": 26.295170958997915, + "dispatch_time": 1791945098.3691342 + }, + { + "id": "confirm-fresh-17-hi-m0-source-chronological", + "history_id": "confirm-fresh-17-hi-m0", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 139260, + "retained_microinr": 174075, + "prompt_tokens": 996, + "completion_tokens": 2072, + "elapsed_seconds": 11.106979749994935, + "dispatch_time": 1791945099.032931 + }, + { + "id": "confirm-fresh-17-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-17-hi-m0", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 119985, + "retained_microinr": 149982, + "prompt_tokens": 1547, + "completion_tokens": 1613, + "elapsed_seconds": 7.389643124988652, + "dispatch_time": 1791945099.70207 + }, + { + "id": "confirm-fresh-17-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-17-hi-m0", + "scenario": "fresh-17", + "family": "separate_booking", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 380505, + "retained_microinr": 475632, + "prompt_tokens": 1435, + "completion_tokens": 5983, + "elapsed_seconds": 25.77854254200065, + "dispatch_time": 1791945126.711941 + }, + { + "id": "confirm-fresh-32-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-32-hi-m0", + "scenario": "fresh-32", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 291510, + "retained_microinr": 364388, + "prompt_tokens": 2062, + "completion_tokens": 4343, + "elapsed_seconds": 21.686025832997984, + "dispatch_time": 1791945126.0445251 + }, + { + "id": "confirm-fresh-32-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-hi-m0", + "scenario": "fresh-32", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 205620, + "retained_microinr": 257025, + "prompt_tokens": 1880, + "completion_tokens": 2957, + "elapsed_seconds": 14.982505040999968, + "dispatch_time": 1791945124.713896 + }, + { + "id": "confirm-fresh-32-hi-m0-source-chronological", + "history_id": "confirm-fresh-32-hi-m0", + "scenario": "fresh-32", + "family": "future_approval", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 151020, + "retained_microinr": 188775, + "prompt_tokens": 1472, + "completion_tokens": 2149, + "elapsed_seconds": 11.138451875012834, + "dispatch_time": 1791945125.378762 + }, + { + "id": "confirm-fresh-31-en-m0-source-chronological", + "history_id": "confirm-fresh-31-en-m0", + "scenario": "fresh-31", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 221100, + "retained_microinr": 276375, + "prompt_tokens": 1332, + "completion_tokens": 3352, + "elapsed_seconds": 17.872049791010795, + "dispatch_time": 1791945152.545495 + }, + { + "id": "confirm-fresh-31-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-31-en-m0", + "scenario": "fresh-31", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 226170, + "retained_microinr": 282713, + "prompt_tokens": 1878, + "completion_tokens": 3300, + "elapsed_seconds": 14.45017408298736, + "dispatch_time": 1791945154.540958 + }, + { + "id": "confirm-fresh-31-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-31-en-m0", + "scenario": "fresh-31", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 206280, + "retained_microinr": 257850, + "prompt_tokens": 1736, + "completion_tokens": 3004, + "elapsed_seconds": 14.945332292001694, + "dispatch_time": 1791945153.2124548 + }, + { + "id": "confirm-fresh-37-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-37-hi-m1", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 40455, + "retained_microinr": 50569, + "prompt_tokens": 2529, + "completion_tokens": 42, + "elapsed_seconds": 0.995458375007729, + "dispatch_time": 1791945153.8771589 + }, + { + "id": "confirm-fresh-37-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-hi-m1", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 36120, + "retained_microinr": 45150, + "prompt_tokens": 2240, + "completion_tokens": 42, + "elapsed_seconds": 0.9419233329972485, + "dispatch_time": 1791945172.454976 + }, + { + "id": "confirm-fresh-37-hi-m1-source-chronological", + "history_id": "confirm-fresh-37-hi-m1", + "scenario": "fresh-37", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 30225, + "retained_microinr": 37782, + "prompt_tokens": 1847, + "completion_tokens": 42, + "elapsed_seconds": 1.017738582988386, + "dispatch_time": 1791945171.790882 + }, + { + "id": "confirm-fresh-39-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-39-en-m0", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 316230, + "retained_microinr": 395288, + "prompt_tokens": 2070, + "completion_tokens": 4753, + "elapsed_seconds": 20.51384858399979, + "dispatch_time": 1791945171.122685 + }, + { + "id": "confirm-fresh-39-en-m0-source-chronological", + "history_id": "confirm-fresh-39-en-m0", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 191010, + "retained_microinr": 238763, + "prompt_tokens": 1542, + "completion_tokens": 2798, + "elapsed_seconds": 13.931675290994463, + "dispatch_time": 1791945170.469577 + }, + { + "id": "confirm-fresh-39-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-en-m0", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 361650, + "retained_microinr": 452063, + "prompt_tokens": 1942, + "completion_tokens": 5542, + "elapsed_seconds": 23.64272079100192, + "dispatch_time": 1791945193.0175092 + }, + { + "id": "confirm-fresh-23-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-23-en-m0", + "scenario": "fresh-23", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 144645, + "retained_microinr": 180807, + "prompt_tokens": 1979, + "completion_tokens": 1916, + "elapsed_seconds": 9.35879191600543, + "dispatch_time": 1791945193.686101 + }, + { + "id": "confirm-fresh-23-en-m0-source-chronological", + "history_id": "confirm-fresh-23-en-m0", + "scenario": "fresh-23", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 130605, + "retained_microinr": 163257, + "prompt_tokens": 1359, + "completion_tokens": 1837, + "elapsed_seconds": 8.069823625002755, + "dispatch_time": 1791945192.352789 + }, + { + "id": "confirm-fresh-23-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-23-en-m0", + "scenario": "fresh-23", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 404385, + "retained_microinr": 505482, + "prompt_tokens": 1979, + "completion_tokens": 6245, + "elapsed_seconds": 29.921276207998744, + "dispatch_time": 1791945191.683879 + }, + { + "id": "confirm-fresh-16-hi-m1-source-chronological", + "history_id": "confirm-fresh-16-hi-m1", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 16755, + "retained_microinr": 20944, + "prompt_tokens": 989, + "completion_tokens": 32, + "elapsed_seconds": 1.036998625000706, + "dispatch_time": 1791945222.978781 + }, + { + "id": "confirm-fresh-16-hi-m1-source-grouped_lossless", + "history_id": "confirm-fresh-16-hi-m1", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 23445, + "retained_microinr": 29307, + "prompt_tokens": 1435, + "completion_tokens": 32, + "elapsed_seconds": 0.9819075410050573, + "dispatch_time": 1791945221.644992 + }, + { + "id": "confirm-fresh-16-hi-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-16-hi-m1", + "scenario": "fresh-16", + "family": "unresolved_condition", + "language": "hi", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 21075, + "retained_microinr": 26344, + "prompt_tokens": 1277, + "completion_tokens": 32, + "elapsed_seconds": 1.1823998749896418, + "dispatch_time": 1791945222.307892 + }, + { + "id": "confirm-fresh-36-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-36-te-m0", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 199845, + "retained_microinr": 249807, + "prompt_tokens": 2199, + "completion_tokens": 2781, + "elapsed_seconds": 11.999567209000816, + "dispatch_time": 1791945223.642057 + }, + { + "id": "confirm-fresh-36-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-te-m0", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 274620, + "retained_microinr": 343275, + "prompt_tokens": 1936, + "completion_tokens": 4093, + "elapsed_seconds": 17.289486125009716, + "dispatch_time": 1791945236.3607612 + }, + { + "id": "confirm-fresh-36-te-m0-source-chronological", + "history_id": "confirm-fresh-36-te-m0", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 125295, + "retained_microinr": 156619, + "prompt_tokens": 1769, + "completion_tokens": 1646, + "elapsed_seconds": 7.960662334007793, + "dispatch_time": 1791945235.6931021 + }, + { + "id": "confirm-fresh-06-hi-m0-source-chronological", + "history_id": "confirm-fresh-06-hi-m0", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 272730, + "retained_microinr": 340913, + "prompt_tokens": 874, + "completion_tokens": 4327, + "elapsed_seconds": 21.645569083993905, + "dispatch_time": 1791945237.693575 + }, + { + "id": "confirm-fresh-06-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-06-hi-m0", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 214515, + "retained_microinr": 268144, + "prompt_tokens": 1249, + "completion_tokens": 3263, + "elapsed_seconds": 15.321799875004217, + "dispatch_time": 1791945237.022929 + }, + { + "id": "confirm-fresh-06-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-06-hi-m0", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 264105, + "retained_microinr": 330132, + "prompt_tokens": 1315, + "completion_tokens": 4073, + "elapsed_seconds": 19.05486554199888, + "dispatch_time": 1791945260.724135 + }, + { + "id": "confirm-fresh-38-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-te-m0", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 278985, + "retained_microinr": 348732, + "prompt_tokens": 1815, + "completion_tokens": 4196, + "elapsed_seconds": 20.786617167002987, + "dispatch_time": 1791945260.053261 + }, + { + "id": "confirm-fresh-38-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-38-te-m0", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 206235, + "retained_microinr": 257794, + "prompt_tokens": 2073, + "completion_tokens": 2919, + "elapsed_seconds": 14.626250207991689, + "dispatch_time": 1791945259.390059 + }, + { + "id": "confirm-fresh-38-te-m0-source-chronological", + "history_id": "confirm-fresh-38-te-m0", + "scenario": "fresh-38", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 161925, + "retained_microinr": 202407, + "prompt_tokens": 1643, + "completion_tokens": 2288, + "elapsed_seconds": 12.175992250005947, + "dispatch_time": 1791945261.3900712 + }, + { + "id": "confirm-fresh-06-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-06-te-m1", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 20760, + "retained_microinr": 25950, + "prompt_tokens": 1256, + "completion_tokens": 32, + "elapsed_seconds": 1.0905270420043962, + "dispatch_time": 1791945282.886082 + }, + { + "id": "confirm-fresh-06-te-m1-source-chronological", + "history_id": "confirm-fresh-06-te-m1", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 15075, + "retained_microinr": 18844, + "prompt_tokens": 877, + "completion_tokens": 32, + "elapsed_seconds": 0.990318208001554, + "dispatch_time": 1791945280.889952 + }, + { + "id": "confirm-fresh-06-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-06-te-m1", + "scenario": "fresh-06", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 21630, + "retained_microinr": 27038, + "prompt_tokens": 1314, + "completion_tokens": 32, + "elapsed_seconds": 0.9463838749943534, + "dispatch_time": 1791945282.2199621 + }, + { + "id": "confirm-fresh-40-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-hi-m0", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 183225, + "retained_microinr": 229032, + "prompt_tokens": 1811, + "completion_tokens": 2601, + "elapsed_seconds": 11.333990916013136, + "dispatch_time": 1791945281.554979 + }, + { + "id": "confirm-fresh-40-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-40-hi-m0", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 230835, + "retained_microinr": 288544, + "prompt_tokens": 2109, + "completion_tokens": 3320, + "elapsed_seconds": 17.365208292001626, + "dispatch_time": 1791945294.923541 + }, + { + "id": "confirm-fresh-40-hi-m0-source-chronological", + "history_id": "confirm-fresh-40-hi-m0", + "scenario": "fresh-40", + "family": "conflicting_request_key", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 368340, + "retained_microinr": 460425, + "prompt_tokens": 1680, + "completion_tokens": 5719, + "elapsed_seconds": 28.12782416700793, + "dispatch_time": 1791945293.59281 + }, + { + "id": "confirm-fresh-01-en-m0-source-chronological", + "history_id": "confirm-fresh-01-en-m0", + "scenario": "fresh-01", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 68955, + "retained_microinr": 86194, + "prompt_tokens": 981, + "completion_tokens": 904, + "elapsed_seconds": 5.073199125006795, + "dispatch_time": 1791945292.925365 + }, + { + "id": "confirm-fresh-01-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-01-en-m0", + "scenario": "fresh-01", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 152010, + "retained_microinr": 190013, + "prompt_tokens": 1526, + "completion_tokens": 2152, + "elapsed_seconds": 11.349641374996281, + "dispatch_time": 1791945294.259507 + }, + { + "id": "confirm-fresh-01-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-01-en-m0", + "scenario": "fresh-01", + "family": "request_correction", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 51225, + "retained_microinr": 64032, + "prompt_tokens": 1399, + "completion_tokens": 504, + "elapsed_seconds": 3.545431375008775, + "dispatch_time": 1791945323.767056 + }, + { + "id": "confirm-fresh-04-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-04-hi-m0", + "scenario": "fresh-04", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 300465, + "retained_microinr": 375582, + "prompt_tokens": 1551, + "completion_tokens": 4620, + "elapsed_seconds": 21.67715354100801, + "dispatch_time": 1791945322.433868 + }, + { + "id": "confirm-fresh-04-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-04-hi-m0", + "scenario": "fresh-04", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 215610, + "retained_microinr": 269513, + "prompt_tokens": 1566, + "completion_tokens": 3202, + "elapsed_seconds": 14.017348583001876, + "dispatch_time": 1791945323.096247 + }, + { + "id": "confirm-fresh-04-hi-m0-source-chronological", + "history_id": "confirm-fresh-04-hi-m0", + "scenario": "fresh-04", + "family": "request_correction", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 103200, + "retained_microinr": 129000, + "prompt_tokens": 972, + "completion_tokens": 1477, + "elapsed_seconds": 7.194728667003801, + "dispatch_time": 1791945321.765378 + }, + { + "id": "confirm-fresh-33-te-m1-source-chronological", + "history_id": "confirm-fresh-33-te-m1", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 32205, + "retained_microinr": 40257, + "prompt_tokens": 1979, + "completion_tokens": 42, + "elapsed_seconds": 1.065939709005761, + "dispatch_time": 1791945346.153399 + }, + { + "id": "confirm-fresh-33-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-te-m1", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 37905, + "retained_microinr": 47382, + "prompt_tokens": 2359, + "completion_tokens": 42, + "elapsed_seconds": 1.029099874998792, + "dispatch_time": 1791945344.823731 + }, + { + "id": "confirm-fresh-33-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-33-te-m1", + "scenario": "fresh-33", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 42360, + "retained_microinr": 52950, + "prompt_tokens": 2656, + "completion_tokens": 42, + "elapsed_seconds": 1.0094782920059515, + "dispatch_time": 1791945344.156087 + }, + { + "id": "confirm-fresh-05-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-05-en-m1", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 27315, + "retained_microinr": 34144, + "prompt_tokens": 1693, + "completion_tokens": 32, + "elapsed_seconds": 1.10927879200608, + "dispatch_time": 1791945345.490394 + }, + { + "id": "confirm-fresh-05-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-05-en-m1", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 29550, + "retained_microinr": 36938, + "prompt_tokens": 1842, + "completion_tokens": 32, + "elapsed_seconds": 1.0474977919948287, + "dispatch_time": 1791945347.93951 + }, + { + "id": "confirm-fresh-05-en-m1-source-chronological", + "history_id": "confirm-fresh-05-en-m1", + "scenario": "fresh-05", + "family": "missing_replacement", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 19020, + "retained_microinr": 23775, + "prompt_tokens": 1140, + "completion_tokens": 32, + "elapsed_seconds": 1.0950979580084095, + "dispatch_time": 1791945349.272853 + }, + { + "id": "confirm-fresh-32-en-m0-source-chronological", + "history_id": "confirm-fresh-32-en-m0", + "scenario": "fresh-32", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 68445, + "retained_microinr": 85557, + "prompt_tokens": 1283, + "completion_tokens": 820, + "elapsed_seconds": 4.343754665998858, + "dispatch_time": 1791945347.272678 + }, + { + "id": "confirm-fresh-32-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-en-m0", + "scenario": "fresh-32", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 206400, + "retained_microinr": 258000, + "prompt_tokens": 1588, + "completion_tokens": 3043, + "elapsed_seconds": 13.495377250001184, + "dispatch_time": 1791945348.602039 + }, + { + "id": "confirm-fresh-32-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-32-en-m0", + "scenario": "fresh-32", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 190020, + "retained_microinr": 237525, + "prompt_tokens": 1724, + "completion_tokens": 2736, + "elapsed_seconds": 11.708094542002073, + "dispatch_time": 1791945362.1407611 + }, + { + "id": "confirm-fresh-36-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-en-m0", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 279390, + "retained_microinr": 349238, + "prompt_tokens": 1902, + "completion_tokens": 4181, + "elapsed_seconds": 21.31061541600502, + "dispatch_time": 1791945362.8080971 + }, + { + "id": "confirm-fresh-36-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-36-en-m0", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 94755, + "retained_microinr": 118444, + "prompt_tokens": 2093, + "completion_tokens": 1056, + "elapsed_seconds": 6.083063041995047, + "dispatch_time": 1791945363.4733708 + }, + { + "id": "confirm-fresh-36-en-m0-source-chronological", + "history_id": "confirm-fresh-36-en-m0", + "scenario": "fresh-36", + "family": "completed_older_request", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 220215, + "retained_microinr": 275269, + "prompt_tokens": 1669, + "completion_tokens": 3253, + "elapsed_seconds": 15.779322165995836, + "dispatch_time": 1791945364.141332 + }, + { + "id": "confirm-fresh-35-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-35-te-m0", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 277440, + "retained_microinr": 346800, + "prompt_tokens": 2400, + "completion_tokens": 4024, + "elapsed_seconds": 19.921081625012448, + "dispatch_time": 1791945385.497494 + }, + { + "id": "confirm-fresh-35-te-m0-source-chronological", + "history_id": "confirm-fresh-35-te-m0", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 189585, + "retained_microinr": 236982, + "prompt_tokens": 1863, + "completion_tokens": 2694, + "elapsed_seconds": 12.963635249994695, + "dispatch_time": 1791945386.161358 + }, + { + "id": "confirm-fresh-35-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-te-m0", + "scenario": "fresh-35", + "family": "completed_older_request", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 403815, + "retained_microinr": 504769, + "prompt_tokens": 2081, + "completion_tokens": 6210, + "elapsed_seconds": 27.812365375008085, + "dispatch_time": 1791945384.830814 + }, + { + "id": "confirm-fresh-19-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-te-m1", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24030, + "retained_microinr": 30038, + "prompt_tokens": 1434, + "completion_tokens": 42, + "elapsed_seconds": 1.0013922919897595, + "dispatch_time": 1791945384.1632159 + }, + { + "id": "confirm-fresh-19-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-19-te-m1", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24735, + "retained_microinr": 30919, + "prompt_tokens": 1481, + "completion_tokens": 42, + "elapsed_seconds": 0.9868211249995511, + "dispatch_time": 1791945413.349 + }, + { + "id": "confirm-fresh-19-te-m1-source-chronological", + "history_id": "confirm-fresh-19-te-m1", + "scenario": "fresh-19", + "family": "separate_booking", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": true, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 16575, + "retained_microinr": 20719, + "prompt_tokens": 937, + "completion_tokens": 42, + "elapsed_seconds": 1.0228360409964807, + "dispatch_time": 1791945412.686465 + }, + { + "id": "confirm-fresh-23-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-23-en-m1", + "scenario": "fresh-23", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36240, + "retained_microinr": 45300, + "prompt_tokens": 2256, + "completion_tokens": 40, + "elapsed_seconds": 1.0312565420026658, + "dispatch_time": 1791945414.0183501 + }, + { + "id": "confirm-fresh-23-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-23-en-m1", + "scenario": "fresh-23", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36240, + "retained_microinr": 45300, + "prompt_tokens": 2256, + "completion_tokens": 40, + "elapsed_seconds": 0.9655432919971645, + "dispatch_time": 1791945414.684617 + }, + { + "id": "confirm-fresh-23-en-m1-source-chronological", + "history_id": "confirm-fresh-23-en-m1", + "scenario": "fresh-23", + "family": "approved_change", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24975, + "retained_microinr": 31219, + "prompt_tokens": 1505, + "completion_tokens": 40, + "elapsed_seconds": 0.9418319579999661, + "dispatch_time": 1791945415.735251 + }, + { + "id": "confirm-fresh-31-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-31-te-m0", + "scenario": "fresh-31", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 521925, + "retained_microinr": 652407, + "prompt_tokens": 2027, + "completion_tokens": 8192, + "elapsed_seconds": 38.83640883299813, + "dispatch_time": 1791945417.723541 + }, + { + "id": "confirm-fresh-31-te-m0-source-chronological", + "history_id": "confirm-fresh-31-te-m0", + "scenario": "fresh-31", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 435615, + "retained_microinr": 544519, + "prompt_tokens": 1485, + "completion_tokens": 6889, + "elapsed_seconds": 31.6984588750056, + "dispatch_time": 1791945417.0561018 + }, + { + "id": "confirm-fresh-31-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-31-te-m0", + "scenario": "fresh-31", + "family": "future_approval", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 94560, + "retained_microinr": 118200, + "prompt_tokens": 2172, + "completion_tokens": 1033, + "elapsed_seconds": 6.013302583000041, + "dispatch_time": 1791945416.389368 + }, + { + "id": "confirm-fresh-08-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-08-te-m0", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 155595, + "retained_microinr": 194494, + "prompt_tokens": 1529, + "completion_tokens": 2211, + "elapsed_seconds": 12.61775991700415, + "dispatch_time": 1791945456.611126 + }, + { + "id": "confirm-fresh-08-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-08-te-m0", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 184950, + "retained_microinr": 231188, + "prompt_tokens": 1646, + "completion_tokens": 2671, + "elapsed_seconds": 13.80287416599458, + "dispatch_time": 1791945457.941836 + }, + { + "id": "confirm-fresh-08-te-m0-source-chronological", + "history_id": "confirm-fresh-08-te-m0", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 170400, + "retained_microinr": 213000, + "prompt_tokens": 1068, + "completion_tokens": 2573, + "elapsed_seconds": 13.018374209001195, + "dispatch_time": 1791945457.275136 + }, + { + "id": "confirm-fresh-03-te-m0-source-chronological", + "history_id": "confirm-fresh-03-te-m0", + "scenario": "fresh-03", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 173805, + "retained_microinr": 217257, + "prompt_tokens": 875, + "completion_tokens": 2678, + "elapsed_seconds": 12.408399916996132, + "dispatch_time": 1791945458.608446 + }, + { + "id": "confirm-fresh-03-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-03-te-m0", + "scenario": "fresh-03", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 119520, + "retained_microinr": 149400, + "prompt_tokens": 1432, + "completion_tokens": 1634, + "elapsed_seconds": 8.721237082994776, + "dispatch_time": 1791945472.455113 + }, + { + "id": "confirm-fresh-03-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-03-te-m0", + "scenario": "fresh-03", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 80175, + "retained_microinr": 100219, + "prompt_tokens": 1417, + "completion_tokens": 982, + "elapsed_seconds": 5.772524457992404, + "dispatch_time": 1791945471.787454 + }, + { + "id": "confirm-fresh-25-en-m1-source-chronological", + "history_id": "confirm-fresh-25-en-m1", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24840, + "retained_microinr": 31050, + "prompt_tokens": 1496, + "completion_tokens": 40, + "elapsed_seconds": 0.9874125000060303, + "dispatch_time": 1791945473.119825 + }, + { + "id": "confirm-fresh-25-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-25-en-m1", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36255, + "retained_microinr": 45319, + "prompt_tokens": 2257, + "completion_tokens": 40, + "elapsed_seconds": 1.0323056659981376, + "dispatch_time": 1791945473.785674 + }, + { + "id": "confirm-fresh-25-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-25-en-m1", + "scenario": "fresh-25", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 36255, + "retained_microinr": 45319, + "prompt_tokens": 2257, + "completion_tokens": 40, + "elapsed_seconds": 1.0221163329988485, + "dispatch_time": 1791945481.895277 + }, + { + "id": "confirm-fresh-29-en-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-29-en-m1", + "scenario": "fresh-29", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 36660, + "retained_microinr": 45825, + "prompt_tokens": 2276, + "completion_tokens": 42, + "elapsed_seconds": 0.9898309169948334, + "dispatch_time": 1791945481.227765 + }, + { + "id": "confirm-fresh-29-en-m1-source-grouped_lossless", + "history_id": "confirm-fresh-29-en-m1", + "scenario": "fresh-29", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 36660, + "retained_microinr": 45825, + "prompt_tokens": 2276, + "completion_tokens": 42, + "elapsed_seconds": 0.9798782500001835, + "dispatch_time": 1791945482.561922 + }, + { + "id": "confirm-fresh-29-en-m1-source-chronological", + "history_id": "confirm-fresh-29-en-m1", + "scenario": "fresh-29", + "family": "future_approval", + "language": "en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "target": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 24990, + "retained_microinr": 31238, + "prompt_tokens": 1506, + "completion_tokens": 40, + "elapsed_seconds": 1.036663791994215, + "dispatch_time": 1791945483.227658 + }, + { + "id": "confirm-fresh-39-te-m1-source-chronological", + "history_id": "confirm-fresh-39-te-m1", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "rejected", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 31320, + "retained_microinr": 39150, + "prompt_tokens": 1928, + "completion_tokens": 40, + "elapsed_seconds": 0.989755000002333, + "dispatch_time": 1791945484.316163 + }, + { + "id": "confirm-fresh-39-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-te-m1", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 36270, + "retained_microinr": 45338, + "prompt_tokens": 2250, + "completion_tokens": 42, + "elapsed_seconds": 1.1882771249947837, + "dispatch_time": 1791945484.976723 + }, + { + "id": "confirm-fresh-39-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-39-te-m1", + "scenario": "fresh-39", + "family": "conflicting_request_key", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 41730, + "retained_microinr": 52163, + "prompt_tokens": 2614, + "completion_tokens": 42, + "elapsed_seconds": 1.038521458991454, + "dispatch_time": 1791945485.6467352 + }, + { + "id": "confirm-fresh-28-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-28-hi-m0", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 407325, + "retained_microinr": 509157, + "prompt_tokens": 2103, + "completion_tokens": 6263, + "elapsed_seconds": 28.466081584003405, + "dispatch_time": 1791945486.313379 + }, + { + "id": "confirm-fresh-28-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-28-hi-m0", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 400170, + "retained_microinr": 500213, + "prompt_tokens": 2266, + "completion_tokens": 6103, + "elapsed_seconds": 29.778850125003373, + "dispatch_time": 1791945516.155537 + }, + { + "id": "confirm-fresh-28-hi-m0-source-chronological", + "history_id": "confirm-fresh-28-hi-m0", + "scenario": "fresh-28", + "family": "approval_boundary", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 515910, + "retained_microinr": 644888, + "prompt_tokens": 1626, + "completion_tokens": 8192, + "elapsed_seconds": 37.17982000000484, + "dispatch_time": 1791945516.820777 + }, + { + "id": "confirm-fresh-04-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-04-te-m1", + "scenario": "fresh-04", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 21885, + "retained_microinr": 27357, + "prompt_tokens": 1291, + "completion_tokens": 42, + "elapsed_seconds": 1.1146989169938024, + "dispatch_time": 1791945514.827295 + }, + { + "id": "confirm-fresh-04-te-m1-source-chronological", + "history_id": "confirm-fresh-04-te-m1", + "scenario": "fresh-04", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 15285, + "retained_microinr": 19107, + "prompt_tokens": 851, + "completion_tokens": 42, + "elapsed_seconds": 0.9734913330030395, + "dispatch_time": 1791945515.490119 + }, + { + "id": "confirm-fresh-04-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-04-te-m1", + "scenario": "fresh-04", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 21540, + "retained_microinr": 26925, + "prompt_tokens": 1268, + "completion_tokens": 42, + "elapsed_seconds": 0.9516997919999994, + "dispatch_time": 1791945555.3738391 + }, + { + "id": "confirm-fresh-02-te-m1-source-chronological", + "history_id": "confirm-fresh-02-te-m1", + "scenario": "fresh-02", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 15285, + "retained_microinr": 19107, + "prompt_tokens": 851, + "completion_tokens": 42, + "elapsed_seconds": 1.101985249988502, + "dispatch_time": 1791945556.04042 + }, + { + "id": "confirm-fresh-02-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-02-te-m1", + "scenario": "fresh-02", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 21900, + "retained_microinr": 27375, + "prompt_tokens": 1292, + "completion_tokens": 42, + "elapsed_seconds": 1.0098189589916728, + "dispatch_time": 1791945554.038753 + }, + { + "id": "confirm-fresh-02-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-02-te-m1", + "scenario": "fresh-02", + "family": "request_correction", + "language": "te", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "target": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 21555, + "retained_microinr": 26944, + "prompt_tokens": 1269, + "completion_tokens": 42, + "elapsed_seconds": 1.1945389590109698, + "dispatch_time": 1791945554.7047439 + }, + { + "id": "confirm-fresh-08-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-08-hi-m0", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 296130, + "retained_microinr": 370163, + "prompt_tokens": 1394, + "completion_tokens": 4587, + "elapsed_seconds": 24.660657500004163, + "dispatch_time": 1791945558.50524 + }, + { + "id": "confirm-fresh-08-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-08-hi-m0", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 358170, + "retained_microinr": 447713, + "prompt_tokens": 1246, + "completion_tokens": 5658, + "elapsed_seconds": 29.663377082993975, + "dispatch_time": 1791945557.174919 + }, + { + "id": "confirm-fresh-08-hi-m0-source-chronological", + "history_id": "confirm-fresh-08-hi-m0", + "scenario": "fresh-08", + "family": "missing_replacement", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 304875, + "retained_microinr": 381094, + "prompt_tokens": 961, + "completion_tokens": 4841, + "elapsed_seconds": 25.56791416699707, + "dispatch_time": 1791945559.172514 + }, + { + "id": "confirm-fresh-27-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-27-en-m0", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 370920, + "retained_microinr": 463650, + "prompt_tokens": 1980, + "completion_tokens": 5687, + "elapsed_seconds": 28.47024195799895, + "dispatch_time": 1791945557.841372 + }, + { + "id": "confirm-fresh-27-en-m0-source-chronological", + "history_id": "confirm-fresh-27-en-m0", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 84795, + "retained_microinr": 105994, + "prompt_tokens": 1357, + "completion_tokens": 1074, + "elapsed_seconds": 4.93337350001093, + "dispatch_time": 1791945586.8729322 + }, + { + "id": "confirm-fresh-27-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-27-en-m0", + "scenario": "fresh-27", + "family": "approval_boundary", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "incomplete", + "valid_final": false, + "raw": null, + "target": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "factual_correct": false, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": false, + "known_microinr": 521220, + "retained_microinr": 651525, + "prompt_tokens": 1980, + "completion_tokens": 8192, + "elapsed_seconds": 41.52383133400872, + "dispatch_time": 1791945587.541769 + }, + { + "id": "confirm-fresh-14-en-m0-source-chronological", + "history_id": "confirm-fresh-14-en-m0", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 152160, + "retained_microinr": 190200, + "prompt_tokens": 980, + "completion_tokens": 2291, + "elapsed_seconds": 12.110416083000018, + "dispatch_time": 1791945588.873316 + }, + { + "id": "confirm-fresh-14-en-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-14-en-m0", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 159915, + "retained_microinr": 199894, + "prompt_tokens": 1253, + "completion_tokens": 2352, + "elapsed_seconds": 12.266609625003184, + "dispatch_time": 1791945588.208425 + }, + { + "id": "confirm-fresh-14-en-m0-source-grouped_lossless", + "history_id": "confirm-fresh-14-en-m0", + "scenario": "fresh-14", + "family": "unresolved_condition", + "language": "en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "target": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 303915, + "retained_microinr": 379894, + "prompt_tokens": 1409, + "completion_tokens": 4713, + "elapsed_seconds": 20.918983416995616, + "dispatch_time": 1791945631.119526 + }, + { + "id": "confirm-fresh-11-hi-m0-source-chronological", + "history_id": "confirm-fresh-11-hi-m0", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 319590, + "retained_microinr": 399488, + "prompt_tokens": 1118, + "completion_tokens": 5047, + "elapsed_seconds": 20.888820915992255, + "dispatch_time": 1791945629.787748 + }, + { + "id": "confirm-fresh-11-hi-m0-source-grouped_lossless", + "history_id": "confirm-fresh-11-hi-m0", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 200640, + "retained_microinr": 250800, + "prompt_tokens": 1820, + "completion_tokens": 2889, + "elapsed_seconds": 14.57691537500068, + "dispatch_time": 1791945630.453355 + }, + { + "id": "confirm-fresh-11-hi-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-11-hi-m0", + "scenario": "fresh-11", + "family": "resolved_condition", + "language": "hi", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "valid_final": true, + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "target": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "factual_correct": true, + "wrong_non_null_date": false, + "false_abstention": false, + "raw_status_correct": true, + "known_microinr": 222300, + "retained_microinr": 277875, + "prompt_tokens": 1708, + "completion_tokens": 3278, + "elapsed_seconds": 16.104989667001064, + "dispatch_time": 1791945629.121496 + } + ], + "model_arm": { + "sarvam-105b/chronological": { + "n": 120, + "factual_correct": 104, + "valid_final": 115, + "wrong_non_null_date": 3, + "false_abstention": 0, + "raw_status_correct": 77, + "known_microinr": 25815420, + "retained_microinr": 32269322 + }, + "sarvam-105b/grouped_actor_prose_withheld": { + "n": 120, + "factual_correct": 105, + "valid_final": 114, + "wrong_non_null_date": 1, + "false_abstention": 2, + "raw_status_correct": 85, + "known_microinr": 27947355, + "retained_microinr": 34934238 + }, + "sarvam-105b/grouped_lossless": { + "n": 120, + "factual_correct": 99, + "valid_final": 112, + "wrong_non_null_date": 4, + "false_abstention": 1, + "raw_status_correct": 83, + "known_microinr": 29891580, + "retained_microinr": 37364522 + }, + "sarvam-105b-conversations/chronological": { + "n": 120, + "factual_correct": 96, + "valid_final": 120, + "wrong_non_null_date": 22, + "false_abstention": 2, + "raw_status_correct": 109, + "known_microinr": 2736045, + "retained_microinr": 3420099 + }, + "sarvam-105b-conversations/grouped_actor_prose_withheld": { + "n": 120, + "factual_correct": 103, + "valid_final": 120, + "wrong_non_null_date": 8, + "false_abstention": 9, + "raw_status_correct": 97, + "known_microinr": 3635190, + "retained_microinr": 4544031 + }, + "sarvam-105b-conversations/grouped_lossless": { + "n": 120, + "factual_correct": 100, + "valid_final": 120, + "wrong_non_null_date": 18, + "false_abstention": 2, + "raw_status_correct": 95, + "known_microinr": 3870870, + "retained_microinr": 4838631 + } + }, + "contrasts": { + "A_to_B/sarvam-105b/all": { + "n": 120, + "before_correct": 104, + "after_correct": 99, + "gain": 7, + "loss": 12, + "net": -5, + "restored_from_invalid": 4, + "corrected_usable_answer": 3, + "gain_ids": [ + "confirm-fresh-16-te-m0", + "confirm-fresh-14-hi-m0", + "confirm-fresh-19-te-m0", + "confirm-fresh-18-te-m0", + "confirm-fresh-06-te-m0", + "confirm-fresh-18-hi-m0", + "confirm-fresh-28-hi-m0" + ], + "loss_ids": [ + "confirm-fresh-13-hi-m0", + "confirm-fresh-20-hi-m0", + "confirm-fresh-19-en-m0", + "confirm-fresh-39-hi-m0", + "confirm-fresh-17-en-m0", + "confirm-fresh-08-en-m0", + "confirm-fresh-05-te-m0", + "confirm-fresh-22-te-m0", + "confirm-fresh-18-en-m0", + "confirm-fresh-23-te-m0", + "confirm-fresh-20-te-m0", + "confirm-fresh-26-te-m0" + ] + }, + "A_to_B/sarvam-105b/treated": { + "n": 98, + "before_correct": 82, + "after_correct": 79, + "gain": 7, + "loss": 10, + "net": -3, + "restored_from_invalid": 4, + "corrected_usable_answer": 3, + "gain_ids": [ + "confirm-fresh-16-te-m0", + "confirm-fresh-14-hi-m0", + "confirm-fresh-19-te-m0", + "confirm-fresh-18-te-m0", + "confirm-fresh-06-te-m0", + "confirm-fresh-18-hi-m0", + "confirm-fresh-28-hi-m0" + ], + "loss_ids": [ + "confirm-fresh-13-hi-m0", + "confirm-fresh-20-hi-m0", + "confirm-fresh-19-en-m0", + "confirm-fresh-39-hi-m0", + "confirm-fresh-17-en-m0", + "confirm-fresh-08-en-m0", + "confirm-fresh-05-te-m0", + "confirm-fresh-22-te-m0", + "confirm-fresh-18-en-m0", + "confirm-fresh-20-te-m0" + ] + }, + "A_to_B/sarvam-105b/no_treatment": { + "n": 22, + "before_correct": 22, + "after_correct": 20, + "gain": 0, + "loss": 2, + "net": -2, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "gain_ids": [], + "loss_ids": [ + "confirm-fresh-23-te-m0", + "confirm-fresh-26-te-m0" + ] + }, + "A_to_B/sarvam-105b-conversations/all": { + "n": 120, + "before_correct": 96, + "after_correct": 100, + "gain": 5, + "loss": 1, + "net": 4, + "restored_from_invalid": 0, + "corrected_usable_answer": 5, + "gain_ids": [ + "confirm-fresh-08-hi-m1", + "confirm-fresh-13-hi-m1", + "confirm-fresh-13-te-m1", + "confirm-fresh-11-hi-m1", + "confirm-fresh-06-hi-m1" + ], + "loss_ids": [ + "confirm-fresh-09-en-m1" + ] + }, + "A_to_B/sarvam-105b-conversations/treated": { + "n": 86, + "before_correct": 62, + "after_correct": 66, + "gain": 5, + "loss": 1, + "net": 4, + "restored_from_invalid": 0, + "corrected_usable_answer": 5, + "gain_ids": [ + "confirm-fresh-08-hi-m1", + "confirm-fresh-13-hi-m1", + "confirm-fresh-13-te-m1", + "confirm-fresh-11-hi-m1", + "confirm-fresh-06-hi-m1" + ], + "loss_ids": [ + "confirm-fresh-09-en-m1" + ] + }, + "A_to_B/sarvam-105b-conversations/no_treatment": { + "n": 34, + "before_correct": 34, + "after_correct": 34, + "gain": 0, + "loss": 0, + "net": 0, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "gain_ids": [], + "loss_ids": [] + }, + "B_to_C/sarvam-105b/all": { + "n": 120, + "before_correct": 99, + "after_correct": 105, + "gain": 15, + "loss": 9, + "net": 6, + "restored_from_invalid": 6, + "corrected_usable_answer": 9, + "gain_ids": [ + "confirm-fresh-13-hi-m0", + "confirm-fresh-20-hi-m0", + "confirm-fresh-19-en-m0", + "confirm-fresh-05-hi-m0", + "confirm-fresh-39-hi-m0", + "confirm-fresh-17-en-m0", + "confirm-fresh-05-te-m0", + "confirm-fresh-13-te-m0", + "confirm-fresh-22-te-m0", + "confirm-fresh-18-en-m0", + "confirm-fresh-06-en-m0", + "confirm-fresh-20-te-m0", + "confirm-fresh-26-te-m0", + "confirm-fresh-20-en-m0", + "confirm-fresh-08-hi-m0" + ], + "loss_ids": [ + "confirm-fresh-25-te-m0", + "confirm-fresh-26-hi-m0", + "confirm-fresh-37-hi-m0", + "confirm-fresh-07-te-m0", + "confirm-fresh-19-te-m0", + "confirm-fresh-01-te-m0", + "confirm-fresh-10-te-m0", + "confirm-fresh-31-te-m0", + "confirm-fresh-27-en-m0" + ] + }, + "B_to_C/sarvam-105b/treated": { + "n": 98, + "before_correct": 79, + "after_correct": 87, + "gain": 14, + "loss": 6, + "net": 8, + "restored_from_invalid": 5, + "corrected_usable_answer": 9, + "gain_ids": [ + "confirm-fresh-13-hi-m0", + "confirm-fresh-20-hi-m0", + "confirm-fresh-19-en-m0", + "confirm-fresh-05-hi-m0", + "confirm-fresh-39-hi-m0", + "confirm-fresh-17-en-m0", + "confirm-fresh-05-te-m0", + "confirm-fresh-13-te-m0", + "confirm-fresh-22-te-m0", + "confirm-fresh-18-en-m0", + "confirm-fresh-06-en-m0", + "confirm-fresh-20-te-m0", + "confirm-fresh-20-en-m0", + "confirm-fresh-08-hi-m0" + ], + "loss_ids": [ + "confirm-fresh-37-hi-m0", + "confirm-fresh-07-te-m0", + "confirm-fresh-19-te-m0", + "confirm-fresh-01-te-m0", + "confirm-fresh-10-te-m0", + "confirm-fresh-31-te-m0" + ] + }, + "B_to_C/sarvam-105b/no_treatment": { + "n": 22, + "before_correct": 20, + "after_correct": 18, + "gain": 1, + "loss": 3, + "net": -2, + "restored_from_invalid": 1, + "corrected_usable_answer": 0, + "gain_ids": [ + "confirm-fresh-26-te-m0" + ], + "loss_ids": [ + "confirm-fresh-25-te-m0", + "confirm-fresh-26-hi-m0", + "confirm-fresh-27-en-m0" + ] + }, + "B_to_C/sarvam-105b-conversations/all": { + "n": 120, + "before_correct": 100, + "after_correct": 103, + "gain": 12, + "loss": 9, + "net": 3, + "restored_from_invalid": 0, + "corrected_usable_answer": 12, + "gain_ids": [ + "confirm-fresh-18-hi-m1", + "confirm-fresh-20-hi-m1", + "confirm-fresh-14-te-m1", + "confirm-fresh-15-te-m1", + "confirm-fresh-15-en-m1", + "confirm-fresh-17-hi-m1", + "confirm-fresh-13-en-m1", + "confirm-fresh-19-hi-m1", + "confirm-fresh-16-en-m1", + "confirm-fresh-09-en-m1", + "confirm-fresh-17-te-m1", + "confirm-fresh-16-te-m1" + ], + "loss_ids": [ + "confirm-fresh-13-hi-m1", + "confirm-fresh-15-hi-m1", + "confirm-fresh-03-en-m1", + "confirm-fresh-09-hi-m1", + "confirm-fresh-01-en-m1", + "confirm-fresh-11-hi-m1", + "confirm-fresh-10-hi-m1", + "confirm-fresh-04-en-m1", + "confirm-fresh-02-en-m1" + ] + }, + "B_to_C/sarvam-105b-conversations/treated": { + "n": 86, + "before_correct": 66, + "after_correct": 69, + "gain": 12, + "loss": 9, + "net": 3, + "restored_from_invalid": 0, + "corrected_usable_answer": 12, + "gain_ids": [ + "confirm-fresh-18-hi-m1", + "confirm-fresh-20-hi-m1", + "confirm-fresh-14-te-m1", + "confirm-fresh-15-te-m1", + "confirm-fresh-15-en-m1", + "confirm-fresh-17-hi-m1", + "confirm-fresh-13-en-m1", + "confirm-fresh-19-hi-m1", + "confirm-fresh-16-en-m1", + "confirm-fresh-09-en-m1", + "confirm-fresh-17-te-m1", + "confirm-fresh-16-te-m1" + ], + "loss_ids": [ + "confirm-fresh-13-hi-m1", + "confirm-fresh-15-hi-m1", + "confirm-fresh-03-en-m1", + "confirm-fresh-09-hi-m1", + "confirm-fresh-01-en-m1", + "confirm-fresh-11-hi-m1", + "confirm-fresh-10-hi-m1", + "confirm-fresh-04-en-m1", + "confirm-fresh-02-en-m1" + ] + }, + "B_to_C/sarvam-105b-conversations/no_treatment": { + "n": 34, + "before_correct": 34, + "after_correct": 34, + "gain": 0, + "loss": 0, + "net": 0, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "gain_ids": [], + "loss_ids": [] + }, + "A_to_C/sarvam-105b/all": { + "n": 120, + "before_correct": 104, + "after_correct": 105, + "gain": 11, + "loss": 10, + "net": 1, + "restored_from_invalid": 5, + "corrected_usable_answer": 6, + "gain_ids": [ + "confirm-fresh-16-te-m0", + "confirm-fresh-05-hi-m0", + "confirm-fresh-14-hi-m0", + "confirm-fresh-13-te-m0", + "confirm-fresh-18-te-m0", + "confirm-fresh-06-te-m0", + "confirm-fresh-18-hi-m0", + "confirm-fresh-06-en-m0", + "confirm-fresh-20-en-m0", + "confirm-fresh-28-hi-m0", + "confirm-fresh-08-hi-m0" + ], + "loss_ids": [ + "confirm-fresh-25-te-m0", + "confirm-fresh-26-hi-m0", + "confirm-fresh-37-hi-m0", + "confirm-fresh-07-te-m0", + "confirm-fresh-08-en-m0", + "confirm-fresh-01-te-m0", + "confirm-fresh-23-te-m0", + "confirm-fresh-10-te-m0", + "confirm-fresh-31-te-m0", + "confirm-fresh-27-en-m0" + ] + }, + "A_to_C/sarvam-105b/treated": { + "n": 98, + "before_correct": 82, + "after_correct": 87, + "gain": 11, + "loss": 6, + "net": 5, + "restored_from_invalid": 5, + "corrected_usable_answer": 6, + "gain_ids": [ + "confirm-fresh-16-te-m0", + "confirm-fresh-05-hi-m0", + "confirm-fresh-14-hi-m0", + "confirm-fresh-13-te-m0", + "confirm-fresh-18-te-m0", + "confirm-fresh-06-te-m0", + "confirm-fresh-18-hi-m0", + "confirm-fresh-06-en-m0", + "confirm-fresh-20-en-m0", + "confirm-fresh-28-hi-m0", + "confirm-fresh-08-hi-m0" + ], + "loss_ids": [ + "confirm-fresh-37-hi-m0", + "confirm-fresh-07-te-m0", + "confirm-fresh-08-en-m0", + "confirm-fresh-01-te-m0", + "confirm-fresh-10-te-m0", + "confirm-fresh-31-te-m0" + ] + }, + "A_to_C/sarvam-105b/no_treatment": { + "n": 22, + "before_correct": 22, + "after_correct": 18, + "gain": 0, + "loss": 4, + "net": -4, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "gain_ids": [], + "loss_ids": [ + "confirm-fresh-25-te-m0", + "confirm-fresh-26-hi-m0", + "confirm-fresh-23-te-m0", + "confirm-fresh-27-en-m0" + ] + }, + "A_to_C/sarvam-105b-conversations/all": { + "n": 120, + "before_correct": 96, + "after_correct": 103, + "gain": 14, + "loss": 7, + "net": 7, + "restored_from_invalid": 0, + "corrected_usable_answer": 14, + "gain_ids": [ + "confirm-fresh-18-hi-m1", + "confirm-fresh-08-hi-m1", + "confirm-fresh-20-hi-m1", + "confirm-fresh-14-te-m1", + "confirm-fresh-13-te-m1", + "confirm-fresh-15-te-m1", + "confirm-fresh-15-en-m1", + "confirm-fresh-17-hi-m1", + "confirm-fresh-13-en-m1", + "confirm-fresh-19-hi-m1", + "confirm-fresh-16-en-m1", + "confirm-fresh-17-te-m1", + "confirm-fresh-06-hi-m1", + "confirm-fresh-16-te-m1" + ], + "loss_ids": [ + "confirm-fresh-15-hi-m1", + "confirm-fresh-03-en-m1", + "confirm-fresh-09-hi-m1", + "confirm-fresh-01-en-m1", + "confirm-fresh-10-hi-m1", + "confirm-fresh-04-en-m1", + "confirm-fresh-02-en-m1" + ] + }, + "A_to_C/sarvam-105b-conversations/treated": { + "n": 86, + "before_correct": 62, + "after_correct": 69, + "gain": 14, + "loss": 7, + "net": 7, + "restored_from_invalid": 0, + "corrected_usable_answer": 14, + "gain_ids": [ + "confirm-fresh-18-hi-m1", + "confirm-fresh-08-hi-m1", + "confirm-fresh-20-hi-m1", + "confirm-fresh-14-te-m1", + "confirm-fresh-13-te-m1", + "confirm-fresh-15-te-m1", + "confirm-fresh-15-en-m1", + "confirm-fresh-17-hi-m1", + "confirm-fresh-13-en-m1", + "confirm-fresh-19-hi-m1", + "confirm-fresh-16-en-m1", + "confirm-fresh-17-te-m1", + "confirm-fresh-06-hi-m1", + "confirm-fresh-16-te-m1" + ], + "loss_ids": [ + "confirm-fresh-15-hi-m1", + "confirm-fresh-03-en-m1", + "confirm-fresh-09-hi-m1", + "confirm-fresh-01-en-m1", + "confirm-fresh-10-hi-m1", + "confirm-fresh-04-en-m1", + "confirm-fresh-02-en-m1" + ] + }, + "A_to_C/sarvam-105b-conversations/no_treatment": { + "n": 34, + "before_correct": 34, + "after_correct": 34, + "gain": 0, + "loss": 0, + "net": 0, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "gain_ids": [], + "loss_ids": [] + } + }, + "treated_histories": 184, + "known_microinr": 93896460, + "retained_microinr": 117370843, + "actual_dispatch_order": [ + "confirm-fresh-31-en-m1-source-grouped_lossless", + "confirm-fresh-22-hi-m1-source-chronological", + "confirm-fresh-31-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-31-en-m1-source-chronological", + "confirm-fresh-24-hi-m1-source-grouped_lossless", + "confirm-fresh-22-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-22-hi-m1-source-grouped_lossless", + "confirm-fresh-24-hi-m1-source-chronological", + "confirm-fresh-24-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-29-hi-m0-source-grouped_lossless", + "confirm-fresh-29-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-29-hi-m0-source-chronological", + "confirm-fresh-18-en-m1-source-chronological", + "confirm-fresh-18-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-18-en-m1-source-grouped_lossless", + "confirm-fresh-34-hi-m1-source-chronological", + "confirm-fresh-34-hi-m1-source-grouped_lossless", + "confirm-fresh-32-te-m0-source-grouped_lossless", + "confirm-fresh-34-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-32-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-32-te-m0-source-chronological", + "confirm-fresh-30-te-m0-source-chronological", + "confirm-fresh-30-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-30-te-m0-source-grouped_lossless", + "confirm-fresh-30-hi-m0-source-grouped_lossless", + "confirm-fresh-17-te-m0-source-grouped_lossless", + "confirm-fresh-30-hi-m0-source-chronological", + "confirm-fresh-30-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-17-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-17-te-m0-source-chronological", + "confirm-fresh-23-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-23-hi-m1-source-chronological", + "confirm-fresh-23-hi-m0-source-chronological", + "confirm-fresh-23-hi-m0-source-grouped_lossless", + "confirm-fresh-23-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-23-hi-m1-source-grouped_lossless", + "confirm-fresh-18-hi-m1-source-grouped_lossless", + "confirm-fresh-18-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-18-hi-m1-source-chronological", + "confirm-fresh-24-hi-m0-source-chronological", + "confirm-fresh-24-hi-m0-source-grouped_lossless", + "confirm-fresh-39-hi-m1-source-grouped_lossless", + "confirm-fresh-24-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-39-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-39-hi-m1-source-chronological", + "confirm-fresh-35-en-m0-source-chronological", + "confirm-fresh-35-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-35-en-m0-source-grouped_lossless", + "confirm-fresh-36-en-m1-source-chronological", + "confirm-fresh-36-en-m1-source-grouped_lossless", + "confirm-fresh-36-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-08-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-08-hi-m1-source-grouped_lossless", + "confirm-fresh-21-hi-m1-source-grouped_lossless", + "confirm-fresh-08-hi-m1-source-chronological", + "confirm-fresh-21-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-25-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-25-te-m0-source-grouped_lossless", + "confirm-fresh-21-hi-m1-source-chronological", + "confirm-fresh-25-te-m0-source-chronological", + "confirm-fresh-36-te-m1-source-chronological", + "confirm-fresh-36-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-13-hi-m1-source-grouped_lossless", + "confirm-fresh-36-te-m1-source-grouped_lossless", + "confirm-fresh-13-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-29-te-m0-source-chronological", + "confirm-fresh-13-hi-m1-source-chronological", + "confirm-fresh-29-te-m0-source-grouped_lossless", + "confirm-fresh-29-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-26-hi-m0-source-grouped_lossless", + "confirm-fresh-26-hi-m0-source-chronological", + "confirm-fresh-26-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-01-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-01-hi-m0-source-grouped_lossless", + "confirm-fresh-24-en-m1-source-chronological", + "confirm-fresh-01-hi-m0-source-chronological", + "confirm-fresh-24-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-02-te-m0-source-grouped_lossless", + "confirm-fresh-02-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-24-en-m1-source-grouped_lossless", + "confirm-fresh-02-te-m0-source-chronological", + "confirm-fresh-09-hi-m0-source-chronological", + "confirm-fresh-09-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-09-hi-m0-source-grouped_lossless", + "confirm-fresh-04-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-04-te-m0-source-grouped_lossless", + "confirm-fresh-13-hi-m0-source-grouped_lossless", + "confirm-fresh-04-te-m0-source-chronological", + "confirm-fresh-13-hi-m0-source-chronological", + "confirm-fresh-31-hi-m0-source-chronological", + "confirm-fresh-13-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-31-hi-m0-source-grouped_lossless", + "confirm-fresh-31-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-33-te-m0-source-chronological", + "confirm-fresh-33-te-m0-source-grouped_lossless", + "confirm-fresh-33-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-25-hi-m1-source-grouped_lossless", + "confirm-fresh-25-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-25-hi-m1-source-chronological", + "confirm-fresh-32-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-15-hi-m1-source-grouped_lossless", + "confirm-fresh-15-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-32-hi-m1-source-grouped_lossless", + "confirm-fresh-32-hi-m1-source-chronological", + "confirm-fresh-15-hi-m1-source-chronological", + "confirm-fresh-35-hi-m1-source-grouped_lossless", + "confirm-fresh-35-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-35-hi-m1-source-chronological", + "confirm-fresh-20-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-20-hi-m1-source-grouped_lossless", + "confirm-fresh-40-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-20-hi-m1-source-chronological", + "confirm-fresh-21-en-m0-source-grouped_lossless", + "confirm-fresh-40-en-m1-source-grouped_lossless", + "confirm-fresh-21-en-m0-source-chronological", + "confirm-fresh-40-en-m1-source-chronological", + "confirm-fresh-37-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-37-hi-m0-source-chronological", + "confirm-fresh-21-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-37-hi-m0-source-grouped_lossless", + "confirm-fresh-28-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-07-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-28-te-m0-source-chronological", + "confirm-fresh-28-te-m0-source-grouped_lossless", + "confirm-fresh-07-te-m0-source-chronological", + "confirm-fresh-09-te-m0-source-grouped_lossless", + "confirm-fresh-09-te-m0-source-chronological", + "confirm-fresh-07-te-m0-source-grouped_lossless", + "confirm-fresh-09-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-03-te-m1-source-grouped_lossless", + "confirm-fresh-03-te-m1-source-chronological", + "confirm-fresh-03-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-28-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-03-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-03-en-m1-source-chronological", + "confirm-fresh-03-en-m1-source-grouped_lossless", + "confirm-fresh-28-en-m1-source-grouped_lossless", + "confirm-fresh-22-hi-m0-source-grouped_lossless", + "confirm-fresh-28-en-m1-source-chronological", + "confirm-fresh-22-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-04-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-22-hi-m0-source-chronological", + "confirm-fresh-04-en-m0-source-grouped_lossless", + "confirm-fresh-04-en-m0-source-chronological", + "confirm-fresh-14-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-16-en-m0-source-grouped_lossless", + "confirm-fresh-16-en-m0-source-chronological", + "confirm-fresh-16-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-02-hi-m1-source-chronological", + "confirm-fresh-14-te-m0-source-grouped_lossless", + "confirm-fresh-02-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-14-te-m0-source-chronological", + "confirm-fresh-15-te-m0-source-chronological", + "confirm-fresh-02-hi-m1-source-grouped_lossless", + "confirm-fresh-15-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-15-te-m0-source-grouped_lossless", + "confirm-fresh-19-hi-m0-source-chronological", + "confirm-fresh-19-hi-m0-source-grouped_lossless", + "confirm-fresh-15-en-m0-source-grouped_lossless", + "confirm-fresh-19-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-15-en-m0-source-chronological", + "confirm-fresh-24-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-15-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-24-en-m0-source-grouped_lossless", + "confirm-fresh-24-en-m0-source-chronological", + "confirm-fresh-32-te-m1-source-grouped_lossless", + "confirm-fresh-32-te-m1-source-chronological", + "confirm-fresh-32-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-22-te-m1-source-grouped_lossless", + "confirm-fresh-22-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-08-en-m1-source-grouped_lossless", + "confirm-fresh-22-te-m1-source-chronological", + "confirm-fresh-08-en-m1-source-chronological", + "confirm-fresh-34-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-08-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-34-en-m1-source-chronological", + "confirm-fresh-37-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-37-en-m1-source-grouped_lossless", + "confirm-fresh-34-en-m1-source-grouped_lossless", + "confirm-fresh-37-en-m1-source-chronological", + "confirm-fresh-36-hi-m0-source-grouped_lossless", + "confirm-fresh-10-en-m0-source-grouped_lossless", + "confirm-fresh-36-hi-m0-source-chronological", + "confirm-fresh-36-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-09-en-m0-source-grouped_lossless", + "confirm-fresh-10-en-m0-source-chronological", + "confirm-fresh-10-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-09-en-m0-source-chronological", + "confirm-fresh-10-hi-m0-source-chronological", + "confirm-fresh-10-hi-m0-source-grouped_lossless", + "confirm-fresh-09-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-10-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-14-te-m1-source-chronological", + "confirm-fresh-30-hi-m1-source-grouped_lossless", + "confirm-fresh-14-te-m1-source-grouped_lossless", + "confirm-fresh-14-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-30-hi-m1-source-chronological", + "confirm-fresh-30-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-31-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-31-hi-m1-source-grouped_lossless", + "confirm-fresh-39-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-39-en-m1-source-grouped_lossless", + "confirm-fresh-31-hi-m1-source-chronological", + "confirm-fresh-39-en-m1-source-chronological", + "confirm-fresh-13-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-21-en-m1-source-chronological", + "confirm-fresh-21-en-m1-source-grouped_lossless", + "confirm-fresh-21-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-21-te-m1-source-chronological", + "confirm-fresh-13-te-m1-source-chronological", + "confirm-fresh-21-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-13-te-m1-source-grouped_lossless", + "confirm-fresh-20-hi-m0-source-grouped_lossless", + "confirm-fresh-21-te-m1-source-grouped_lossless", + "confirm-fresh-20-hi-m0-source-chronological", + "confirm-fresh-20-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-19-en-m0-source-grouped_lossless", + "confirm-fresh-19-en-m0-source-chronological", + "confirm-fresh-19-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-38-te-m1-source-chronological", + "confirm-fresh-38-te-m1-source-grouped_lossless", + "confirm-fresh-38-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-35-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-35-en-m1-source-grouped_lossless", + "confirm-fresh-16-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-16-te-m0-source-grouped_lossless", + "confirm-fresh-35-en-m1-source-chronological", + "confirm-fresh-16-te-m0-source-chronological", + "confirm-fresh-15-te-m1-source-chronological", + "confirm-fresh-15-te-m1-source-grouped_lossless", + "confirm-fresh-05-hi-m1-source-chronological", + "confirm-fresh-15-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-05-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-05-hi-m1-source-grouped_lossless", + "confirm-fresh-33-hi-m1-source-grouped_lossless", + "confirm-fresh-33-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-05-hi-m0-source-chronological", + "confirm-fresh-05-hi-m0-source-grouped_lossless", + "confirm-fresh-33-hi-m1-source-chronological", + "confirm-fresh-05-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-21-te-m0-source-chronological", + "confirm-fresh-40-hi-m1-source-grouped_lossless", + "confirm-fresh-40-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-40-hi-m1-source-chronological", + "confirm-fresh-14-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-21-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-14-hi-m0-source-grouped_lossless", + "confirm-fresh-21-te-m0-source-grouped_lossless", + "confirm-fresh-08-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-08-te-m1-source-chronological", + "confirm-fresh-08-te-m1-source-grouped_lossless", + "confirm-fresh-14-hi-m0-source-chronological", + "confirm-fresh-09-hi-m1-source-grouped_lossless", + "confirm-fresh-09-hi-m1-source-chronological", + "confirm-fresh-09-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-39-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-27-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-27-hi-m0-source-chronological", + "confirm-fresh-39-hi-m0-source-chronological", + "confirm-fresh-39-hi-m0-source-grouped_lossless", + "confirm-fresh-24-te-m1-source-grouped_lossless", + "confirm-fresh-24-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-24-te-m1-source-chronological", + "confirm-fresh-27-hi-m0-source-grouped_lossless", + "confirm-fresh-18-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-18-te-m1-source-chronological", + "confirm-fresh-18-te-m1-source-grouped_lossless", + "confirm-fresh-17-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-17-en-m0-source-chronological", + "confirm-fresh-19-te-m0-source-chronological", + "confirm-fresh-19-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-17-en-m0-source-grouped_lossless", + "confirm-fresh-19-te-m0-source-grouped_lossless", + "confirm-fresh-20-te-m1-source-grouped_lossless", + "confirm-fresh-20-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-20-te-m1-source-chronological", + "confirm-fresh-37-te-m0-source-grouped_lossless", + "confirm-fresh-12-te-m0-source-grouped_lossless", + "confirm-fresh-37-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-37-te-m0-source-chronological", + "confirm-fresh-38-en-m0-source-chronological", + "confirm-fresh-38-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-12-te-m0-source-chronological", + "confirm-fresh-12-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-08-en-m0-source-chronological", + "confirm-fresh-08-en-m0-source-grouped_lossless", + "confirm-fresh-08-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-38-en-m0-source-grouped_lossless", + "confirm-fresh-15-en-m1-source-chronological", + "confirm-fresh-15-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-15-en-m1-source-grouped_lossless", + "confirm-fresh-14-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-14-hi-m1-source-chronological", + "confirm-fresh-14-hi-m1-source-grouped_lossless", + "confirm-fresh-27-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-27-en-m1-source-grouped_lossless", + "confirm-fresh-02-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-02-hi-m0-source-grouped_lossless", + "confirm-fresh-27-en-m1-source-chronological", + "confirm-fresh-02-hi-m0-source-chronological", + "confirm-fresh-20-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-20-en-m1-source-chronological", + "confirm-fresh-07-hi-m1-source-grouped_lossless", + "confirm-fresh-20-en-m1-source-grouped_lossless", + "confirm-fresh-07-hi-m1-source-chronological", + "confirm-fresh-40-en-m0-source-grouped_lossless", + "confirm-fresh-07-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-40-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-40-en-m0-source-chronological", + "confirm-fresh-01-hi-m1-source-chronological", + "confirm-fresh-01-hi-m1-source-grouped_lossless", + "confirm-fresh-01-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-05-te-m0-source-grouped_lossless", + "confirm-fresh-05-te-m0-source-chronological", + "confirm-fresh-05-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-38-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-01-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-01-en-m1-source-chronological", + "confirm-fresh-38-hi-m1-source-chronological", + "confirm-fresh-38-hi-m1-source-grouped_lossless", + "confirm-fresh-12-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-12-hi-m0-source-grouped_lossless", + "confirm-fresh-01-en-m1-source-grouped_lossless", + "confirm-fresh-12-hi-m0-source-chronological", + "confirm-fresh-13-en-m0-source-chronological", + "confirm-fresh-13-en-m0-source-grouped_lossless", + "confirm-fresh-13-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-30-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-30-te-m1-source-chronological", + "confirm-fresh-10-te-m1-source-grouped_lossless", + "confirm-fresh-30-te-m1-source-grouped_lossless", + "confirm-fresh-10-te-m1-source-chronological", + "confirm-fresh-05-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-05-te-m1-source-chronological", + "confirm-fresh-10-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-05-te-m1-source-grouped_lossless", + "confirm-fresh-16-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-35-hi-m0-source-chronological", + "confirm-fresh-16-hi-m0-source-grouped_lossless", + "confirm-fresh-16-hi-m0-source-chronological", + "confirm-fresh-35-hi-m0-source-grouped_lossless", + "confirm-fresh-38-en-m1-source-grouped_lossless", + "confirm-fresh-38-en-m1-source-chronological", + "confirm-fresh-35-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-26-te-m1-source-chronological", + "confirm-fresh-26-te-m1-source-grouped_lossless", + "confirm-fresh-38-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-26-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-25-en-m0-source-chronological", + "confirm-fresh-07-te-m1-source-grouped_lossless", + "confirm-fresh-07-te-m1-source-chronological", + "confirm-fresh-07-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-25-en-m0-source-grouped_lossless", + "confirm-fresh-29-te-m1-source-grouped_lossless", + "confirm-fresh-25-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-29-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-29-te-m1-source-chronological", + "confirm-fresh-27-te-m0-source-chronological", + "confirm-fresh-27-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-27-te-m0-source-grouped_lossless", + "confirm-fresh-31-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-19-en-m1-source-grouped_lossless", + "confirm-fresh-19-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-19-en-m1-source-chronological", + "confirm-fresh-31-te-m1-source-grouped_lossless", + "confirm-fresh-32-en-m1-source-chronological", + "confirm-fresh-31-te-m1-source-chronological", + "confirm-fresh-32-en-m1-source-grouped_lossless", + "confirm-fresh-32-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-06-en-m1-source-chronological", + "confirm-fresh-06-en-m1-source-grouped_lossless", + "confirm-fresh-06-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-07-en-m1-source-grouped_lossless", + "confirm-fresh-07-en-m1-source-chronological", + "confirm-fresh-07-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-13-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-13-te-m0-source-chronological", + "confirm-fresh-13-te-m0-source-grouped_lossless", + "confirm-fresh-11-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-11-te-m1-source-grouped_lossless", + "confirm-fresh-09-te-m1-source-chronological", + "confirm-fresh-11-te-m1-source-chronological", + "confirm-fresh-09-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-09-te-m1-source-grouped_lossless", + "confirm-fresh-27-te-m1-source-grouped_lossless", + "confirm-fresh-27-te-m1-source-chronological", + "confirm-fresh-12-hi-m1-source-grouped_lossless", + "confirm-fresh-27-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-12-hi-m1-source-chronological", + "confirm-fresh-12-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-12-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-12-te-m1-source-chronological", + "confirm-fresh-18-te-m0-source-grouped_lossless", + "confirm-fresh-18-te-m0-source-chronological", + "confirm-fresh-12-te-m1-source-grouped_lossless", + "confirm-fresh-18-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-26-en-m1-source-chronological", + "confirm-fresh-26-en-m1-source-grouped_lossless", + "confirm-fresh-26-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-23-te-m1-source-grouped_lossless", + "confirm-fresh-17-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-17-hi-m1-source-chronological", + "confirm-fresh-23-te-m1-source-chronological", + "confirm-fresh-23-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-17-hi-m1-source-grouped_lossless", + "confirm-fresh-22-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-22-te-m0-source-chronological", + "confirm-fresh-22-te-m0-source-grouped_lossless", + "confirm-fresh-01-te-m1-source-chronological", + "confirm-fresh-01-te-m1-source-grouped_lossless", + "confirm-fresh-37-en-m0-source-chronological", + "confirm-fresh-01-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-13-en-m1-source-chronological", + "confirm-fresh-37-en-m0-source-grouped_lossless", + "confirm-fresh-37-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-13-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-19-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-19-hi-m1-source-chronological", + "confirm-fresh-13-en-m1-source-grouped_lossless", + "confirm-fresh-19-hi-m1-source-grouped_lossless", + "confirm-fresh-21-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-37-te-m1-source-chronological", + "confirm-fresh-37-te-m1-source-grouped_lossless", + "confirm-fresh-37-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-02-en-m0-source-chronological", + "confirm-fresh-02-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-21-hi-m0-source-chronological", + "confirm-fresh-21-hi-m0-source-grouped_lossless", + "confirm-fresh-06-te-m0-source-grouped_lossless", + "confirm-fresh-06-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-02-en-m0-source-grouped_lossless", + "confirm-fresh-06-te-m0-source-chronological", + "confirm-fresh-39-te-m0-source-grouped_lossless", + "confirm-fresh-12-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-39-te-m0-source-chronological", + "confirm-fresh-39-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-11-hi-m1-source-chronological", + "confirm-fresh-11-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-12-en-m0-source-chronological", + "confirm-fresh-12-en-m0-source-grouped_lossless", + "confirm-fresh-27-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-27-hi-m1-source-chronological", + "confirm-fresh-27-hi-m1-source-grouped_lossless", + "confirm-fresh-11-hi-m1-source-grouped_lossless", + "confirm-fresh-22-en-m1-source-chronological", + "confirm-fresh-22-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-29-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-22-en-m1-source-grouped_lossless", + "confirm-fresh-29-en-m0-source-grouped_lossless", + "confirm-fresh-30-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-29-en-m0-source-chronological", + "confirm-fresh-30-en-m0-source-grouped_lossless", + "confirm-fresh-30-en-m0-source-chronological", + "confirm-fresh-34-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-34-te-m0-source-chronological", + "confirm-fresh-34-te-m0-source-grouped_lossless", + "confirm-fresh-33-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-33-en-m1-source-grouped_lossless", + "confirm-fresh-33-en-m1-source-chronological", + "confirm-fresh-10-hi-m1-source-chronological", + "confirm-fresh-10-hi-m1-source-grouped_lossless", + "confirm-fresh-18-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-18-hi-m0-source-chronological", + "confirm-fresh-10-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-18-hi-m0-source-grouped_lossless", + "confirm-fresh-17-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-17-en-m1-source-grouped_lossless", + "confirm-fresh-17-en-m1-source-chronological", + "confirm-fresh-25-te-m1-source-chronological", + "confirm-fresh-25-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-25-te-m1-source-grouped_lossless", + "confirm-fresh-04-hi-m1-source-chronological", + "confirm-fresh-01-te-m0-source-chronological", + "confirm-fresh-04-hi-m1-source-grouped_lossless", + "confirm-fresh-01-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-04-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-05-en-m0-source-grouped_lossless", + "confirm-fresh-05-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-05-en-m0-source-chronological", + "confirm-fresh-01-te-m0-source-grouped_lossless", + "confirm-fresh-16-en-m1-source-grouped_lossless", + "confirm-fresh-16-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-28-hi-m1-source-chronological", + "confirm-fresh-16-en-m1-source-chronological", + "confirm-fresh-28-hi-m1-source-grouped_lossless", + "confirm-fresh-34-en-m0-source-grouped_lossless", + "confirm-fresh-28-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-34-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-34-en-m0-source-chronological", + "confirm-fresh-40-te-m0-source-chronological", + "confirm-fresh-40-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-40-te-m0-source-grouped_lossless", + "confirm-fresh-03-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-03-hi-m0-source-grouped_lossless", + "confirm-fresh-11-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-03-hi-m0-source-chronological", + "confirm-fresh-11-en-m1-source-chronological", + "confirm-fresh-26-en-m0-source-chronological", + "confirm-fresh-26-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-11-en-m1-source-grouped_lossless", + "confirm-fresh-15-hi-m0-source-chronological", + "confirm-fresh-26-en-m0-source-grouped_lossless", + "confirm-fresh-15-hi-m0-source-grouped_lossless", + "confirm-fresh-15-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-38-hi-m0-source-grouped_lossless", + "confirm-fresh-09-en-m1-source-chronological", + "confirm-fresh-09-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-09-en-m1-source-grouped_lossless", + "confirm-fresh-24-te-m0-source-grouped_lossless", + "confirm-fresh-38-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-38-hi-m0-source-chronological", + "confirm-fresh-24-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-07-en-m0-source-grouped_lossless", + "confirm-fresh-07-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-24-te-m0-source-chronological", + "confirm-fresh-07-en-m0-source-chronological", + "confirm-fresh-11-te-m0-source-grouped_lossless", + "confirm-fresh-11-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-11-te-m0-source-chronological", + "confirm-fresh-33-en-m0-source-grouped_lossless", + "confirm-fresh-33-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-33-en-m0-source-chronological", + "confirm-fresh-10-en-m1-source-chronological", + "confirm-fresh-10-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-10-en-m1-source-grouped_lossless", + "confirm-fresh-26-hi-m1-source-chronological", + "confirm-fresh-26-hi-m1-source-grouped_lossless", + "confirm-fresh-26-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-18-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-18-en-m0-source-chronological", + "confirm-fresh-18-en-m0-source-grouped_lossless", + "confirm-fresh-23-te-m0-source-grouped_lossless", + "confirm-fresh-23-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-30-en-m1-source-grouped_lossless", + "confirm-fresh-23-te-m0-source-chronological", + "confirm-fresh-30-en-m1-source-chronological", + "confirm-fresh-17-te-m1-source-grouped_lossless", + "confirm-fresh-30-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-17-te-m1-source-chronological", + "confirm-fresh-17-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-36-hi-m1-source-chronological", + "confirm-fresh-36-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-28-te-m1-source-grouped_lossless", + "confirm-fresh-36-hi-m1-source-grouped_lossless", + "confirm-fresh-29-hi-m1-source-chronological", + "confirm-fresh-28-te-m1-source-chronological", + "confirm-fresh-29-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-28-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-29-hi-m1-source-grouped_lossless", + "confirm-fresh-35-te-m1-source-grouped_lossless", + "confirm-fresh-35-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-35-te-m1-source-chronological", + "confirm-fresh-12-en-m1-source-chronological", + "confirm-fresh-04-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-04-en-m1-source-chronological", + "confirm-fresh-04-en-m1-source-grouped_lossless", + "confirm-fresh-12-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-40-te-m1-source-chronological", + "confirm-fresh-40-te-m1-source-grouped_lossless", + "confirm-fresh-12-en-m1-source-grouped_lossless", + "confirm-fresh-06-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-06-hi-m1-source-grouped_lossless", + "confirm-fresh-40-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-06-hi-m1-source-chronological", + "confirm-fresh-34-hi-m0-source-chronological", + "confirm-fresh-34-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-03-en-m0-source-grouped_lossless", + "confirm-fresh-34-hi-m0-source-grouped_lossless", + "confirm-fresh-03-en-m0-source-chronological", + "confirm-fresh-03-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-07-hi-m0-source-grouped_lossless", + "confirm-fresh-07-hi-m0-source-chronological", + "confirm-fresh-11-en-m0-source-chronological", + "confirm-fresh-11-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-11-en-m0-source-grouped_lossless", + "confirm-fresh-07-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-03-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-03-hi-m1-source-chronological", + "confirm-fresh-03-hi-m1-source-grouped_lossless", + "confirm-fresh-06-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-28-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-06-en-m0-source-grouped_lossless", + "confirm-fresh-06-en-m0-source-chronological", + "confirm-fresh-28-en-m0-source-grouped_lossless", + "confirm-fresh-20-te-m0-source-chronological", + "confirm-fresh-20-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-28-en-m0-source-chronological", + "confirm-fresh-20-te-m0-source-grouped_lossless", + "confirm-fresh-33-hi-m0-source-grouped_lossless", + "confirm-fresh-16-te-m1-source-grouped_lossless", + "confirm-fresh-33-hi-m0-source-chronological", + "confirm-fresh-33-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-16-te-m1-source-chronological", + "confirm-fresh-26-te-m0-source-grouped_lossless", + "confirm-fresh-26-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-16-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-20-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-20-en-m0-source-grouped_lossless", + "confirm-fresh-26-te-m0-source-chronological", + "confirm-fresh-20-en-m0-source-chronological", + "confirm-fresh-25-hi-m0-source-grouped_lossless", + "confirm-fresh-34-te-m1-source-grouped_lossless", + "confirm-fresh-34-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-34-te-m1-source-chronological", + "confirm-fresh-25-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-25-hi-m0-source-chronological", + "confirm-fresh-14-en-m1-source-chronological", + "confirm-fresh-14-en-m1-source-grouped_lossless", + "confirm-fresh-14-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-02-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-02-en-m1-source-chronological", + "confirm-fresh-02-en-m1-source-grouped_lossless", + "confirm-fresh-22-en-m0-source-chronological", + "confirm-fresh-10-te-m0-source-chronological", + "confirm-fresh-22-en-m0-source-grouped_lossless", + "confirm-fresh-22-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-10-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-10-te-m0-source-grouped_lossless", + "confirm-fresh-17-hi-m0-source-chronological", + "confirm-fresh-17-hi-m0-source-grouped_lossless", + "confirm-fresh-32-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-32-hi-m0-source-chronological", + "confirm-fresh-32-hi-m0-source-grouped_lossless", + "confirm-fresh-17-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-31-en-m0-source-chronological", + "confirm-fresh-31-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-37-hi-m1-source-grouped_lossless", + "confirm-fresh-31-en-m0-source-grouped_lossless", + "confirm-fresh-39-en-m0-source-chronological", + "confirm-fresh-39-en-m0-source-grouped_lossless", + "confirm-fresh-37-hi-m1-source-chronological", + "confirm-fresh-37-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-23-en-m0-source-grouped_lossless", + "confirm-fresh-23-en-m0-source-chronological", + "confirm-fresh-39-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-23-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-16-hi-m1-source-grouped_lossless", + "confirm-fresh-16-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-16-hi-m1-source-chronological", + "confirm-fresh-36-te-m0-source-grouped_lossless", + "confirm-fresh-36-te-m0-source-chronological", + "confirm-fresh-36-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-06-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-06-hi-m0-source-chronological", + "confirm-fresh-38-te-m0-source-grouped_lossless", + "confirm-fresh-38-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-06-hi-m0-source-grouped_lossless", + "confirm-fresh-38-te-m0-source-chronological", + "confirm-fresh-06-te-m1-source-chronological", + "confirm-fresh-40-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-06-te-m1-source-grouped_lossless", + "confirm-fresh-06-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-01-en-m0-source-chronological", + "confirm-fresh-40-hi-m0-source-chronological", + "confirm-fresh-01-en-m0-source-grouped_lossless", + "confirm-fresh-40-hi-m0-source-grouped_lossless", + "confirm-fresh-04-hi-m0-source-chronological", + "confirm-fresh-04-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-04-hi-m0-source-grouped_lossless", + "confirm-fresh-01-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-33-te-m1-source-grouped_lossless", + "confirm-fresh-33-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-05-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-33-te-m1-source-chronological", + "confirm-fresh-32-en-m0-source-chronological", + "confirm-fresh-05-en-m1-source-grouped_lossless", + "confirm-fresh-32-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-05-en-m1-source-chronological", + "confirm-fresh-32-en-m0-source-grouped_lossless", + "confirm-fresh-36-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-36-en-m0-source-grouped_lossless", + "confirm-fresh-36-en-m0-source-chronological", + "confirm-fresh-19-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-35-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-35-te-m0-source-grouped_lossless", + "confirm-fresh-35-te-m0-source-chronological", + "confirm-fresh-19-te-m1-source-chronological", + "confirm-fresh-19-te-m1-source-grouped_lossless", + "confirm-fresh-23-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-23-en-m1-source-grouped_lossless", + "confirm-fresh-23-en-m1-source-chronological", + "confirm-fresh-31-te-m0-source-grouped_lossless", + "confirm-fresh-31-te-m0-source-chronological", + "confirm-fresh-31-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-08-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-08-te-m0-source-chronological", + "confirm-fresh-08-te-m0-source-grouped_lossless", + "confirm-fresh-03-te-m0-source-chronological", + "confirm-fresh-03-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-03-te-m0-source-grouped_lossless", + "confirm-fresh-25-en-m1-source-chronological", + "confirm-fresh-25-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-29-en-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-25-en-m1-source-grouped_lossless", + "confirm-fresh-29-en-m1-source-grouped_lossless", + "confirm-fresh-29-en-m1-source-chronological", + "confirm-fresh-39-te-m1-source-chronological", + "confirm-fresh-39-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-39-te-m1-source-grouped_lossless", + "confirm-fresh-28-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-04-te-m1-source-grouped_lossless", + "confirm-fresh-04-te-m1-source-chronological", + "confirm-fresh-28-hi-m0-source-grouped_lossless", + "confirm-fresh-28-hi-m0-source-chronological", + "confirm-fresh-02-te-m1-source-grouped_lossless", + "confirm-fresh-02-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-04-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-02-te-m1-source-chronological", + "confirm-fresh-08-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-27-en-m0-source-grouped_lossless", + "confirm-fresh-08-hi-m0-source-grouped_lossless", + "confirm-fresh-08-hi-m0-source-chronological", + "confirm-fresh-27-en-m0-source-chronological", + "confirm-fresh-27-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-14-en-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-14-en-m0-source-chronological", + "confirm-fresh-11-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-11-hi-m0-source-chronological", + "confirm-fresh-11-hi-m0-source-grouped_lossless", + "confirm-fresh-14-en-m0-source-grouped_lossless" + ] +} diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/review-verification.json b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/review-verification.json new file mode 100644 index 0000000..d5ac1c6 --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/review-verification.json @@ -0,0 +1,52 @@ +{ + "status": "ASSISTANT_LANGUAGE_REVIEWS_EXACT_COVERAGE_AND_RAW_AUDIT_MATCH", + "verified_at_utc": "2026-10-14T02:47:42+00:00", + "manifest_file_sha256": "ba66788d25b439471ba396c8aa2f56a1b4f136cd60cdbf0a04a40e2484bbf650", + "audit_file_sha256": "8c208f56157cb753b0965d5b88d350ef2ee181dbb4ce5646d34e3ca36a849cc0", + "execution_file_sha256": "9f322fddafe182d8f0f3e5af2a34575caa4e7db3ea0286339665540419e376a3", + "language_review_file_sha256": { + "en": "b82bec49adc44c28a68620be23b6723d53973f27e3d17c2a2d1427545e7bed13", + "hi": "d4de3549829ae68032645dd0b58050de34a840d3bb1f7de6da992fb7a14db11e", + "te": "16d147a825c9fb1fbd226e49464a4c79e055f0e28b6a05a73202bb15cafd212d" + }, + "planned_and_reviewed_unique_reporter_jobs": 720, + "per_language": { + "en": { + "planned_and_reviewed_jobs": 240, + "factual_correct": 201, + "raw_status_correct": 185, + "invalid_or_missing_final": 4, + "source_cases": 40, + "native_histories": 80, + "source_gold_ambiguities": 0 + }, + "hi": { + "planned_and_reviewed_jobs": 240, + "factual_correct": 202, + "raw_status_correct": 192, + "invalid_or_missing_final": 6, + "source_cases": 40, + "native_histories": 80, + "source_gold_ambiguities": 0 + }, + "te": { + "planned_and_reviewed_jobs": 240, + "factual_correct": 204, + "raw_status_correct": 169, + "invalid_or_missing_final": 9, + "source_cases": 40, + "native_histories": 80, + "source_gold_ambiguities": 0 + } + }, + "raw_result_files_hash_checked": 720, + "raw_receipt_files_hash_checked": 720, + "raw_request_files_hash_checked": 480, + "hindi_request_payload_digests_checked": 240, + "factual_verdict_disagreements_with_audit": 0, + "raw_status_verdict_disagreements_with_audit": 0, + "source_gold_ambiguity_flags": 0, + "assistant_review_only": true, + "independent_native_human_review": false, + "scope_limit": "Coverage/hash/verdict consistency; this does not independently translate or validate source meaning and is not human/native-speaker review." +} diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/reviews/HINDI_INTERPRETATION.md b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/reviews/HINDI_INTERPRETATION.md new file mode 100644 index 0000000..8491778 --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/reviews/HINDI_INTERPRETATION.md @@ -0,0 +1,16 @@ +# Hindi semantic review of the three reporter views + +The [assistant review](hindi-review.json) covers all 240 planned Hindi finals: 40 source scenarios × two fixed Sarvam profiles × three arms. I read the Hindi questions and source conditions, checked the saved execution receipts and current scoped booking for each model history, and compared every raw output with the offline receipt replay. There were **zero assistant/scorer disagreements** and no unresolved source-label ambiguity on this slice. Six capped/incomplete finals remain failures. This is an assistant language review, not an independent native-speaker assessment. + +| Model, 40 histories each | A chronological | B grouped | C grouped, actor prose withheld | +| --- | ---: | ---: | ---: | +| Sarvam 105B: correct date + clarification | 33 | 33 | 36 | +| Sarvam 105B: correct raw action status | 29 | 31 | 32 | +| Conversations: correct date + clarification | 31 | 35 | 34 | +| Conversations: correct raw action status | 36 | 32 | 32 | + +Paired factual movements on **all 40 histories per model** were: base A→B 3 gains/3 losses, B→C 5/2, A→C 5/2; Conversations A→B 4/0, B→C 4/5, A→C 6/3. The predeclared *treated* C−B contrast excludes byte-identical no-op requests: base 5 gains/1 loss across 34 histories; Conversations 4 gains/5 losses across 29. One additional base B→C loss was a capped C final for `26-hi-m0` even though B and C request bytes were identical there; it is retained in all-history counts and cannot be credited to prose removal. Every one of the 24 Hindi explicit replacement-request finals (01–04, both models, all arms) answered the latest requested date; this does not reproduce the English C abstentions reported for that family. + +The gains and regressions have different user-facing meanings. On Conversations separate-booking cases 17–20, C recovers all four target preferences that B misstated as the current booked date. Yet on known morning-availability cases 09–12, C falsely asks for clarification in all four Conversations histories, versus one for B and two for A. On unknown-availability cases 13 and 15, C chooses an unsupported conditional date; B makes no such definite guess in Conversations. Conversations raw status accuracy also falls from 36 under A to 32 under both B and C, although the application-owned projection remains 40/40. Base C improves several missing-date and source-scoping answers, but its invalid finals include treated `37-hi-m0` and the no-op `26-hi-m0`. The Hindi treated C−B direction is therefore **negative for Conversations**. On this slice C also has more false abstentions and lower raw-status accuracy than the ordinary chronological control. These are local observations; the frozen [protocol](../PROTOCOL.md) and full EN/HI/TE gate decide the formal campaign result. + +The Hindi evidence supports a narrow source-interpretation problem and shows that rearranging evidence or omitting actor prose can change answers under fixed models. It does **not** support blanket prose omission as a reliable product feature, establish a missing persistence layer, or distinguish a novel memory system. The relevant facts were already present; the [prior-art update](../PRIOR_ART_UPDATE.md) also documents existing temporal, provenance and typed-fact capabilities. Any next product test should first beat a strong ordinary control for source scoping, conditional-premise handling, and application-owned action status on fresh, independently reviewed workflows at declared cost and latency. No new experiment is authorized by this slice note. diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/reviews/english-review.json b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/reviews/english-review.json new file mode 100644 index 0000000..cc081f5 --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/reviews/english-review.json @@ -0,0 +1,25326 @@ +{ + "status": "ASSISTANT_ENGLISH_REVIEW_COMPLETE", + "independent_native_human_review": false, + "phase_manifest_sha256": "ba66788d25b439471ba396c8aa2f56a1b4f136cd60cdbf0a04a40e2484bbf650", + "parent_semantic_review_sha256": "bff2555bbc642d1054f2ebcd4a7afd22064db5a9ff6d49a03511d64d80fffd67", + "source_case_reviews": [ + { + "id": "fresh-01-en", + "family": "request_correction", + "question": "Which date did I most recently request for VISIT-2101? Read only.", + "source_messages": [ + { + "id": "note-1-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-1-1", + "role": "customer", + "content": "For VISIT-2101, I initially asked for 2027-04-02." + }, + { + "id": "note-1-2", + "role": "customer", + "content": "Replace that request with 2027-05-02; 2027-04-02 is no longer my choice." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-01", + "id": "VISIT-2101", + "date": "2027-03-02", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-01", + "id": "VISIT-3101", + "date": "2027-06-02", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The later 2027-05-02 customer correction supersedes 2027-04-02 for VISIT-2101; read_current 2027-03-02 is the separate booked date.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-02-en", + "family": "request_correction", + "question": "Which date did I most recently request for VISIT-2102? Read only.", + "source_messages": [ + { + "id": "note-2-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. For VISIT-2102, I initially asked for 2027-04-03. Replace that request with 2027-05-03; 2027-04-03 is no longer my choice." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-02", + "id": "VISIT-2102", + "date": "2027-03-03", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-02", + "id": "VISIT-3102", + "date": "2027-06-03", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The combined source still orders the original 2027-04-03 request before its replacement with 2027-05-03.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-03-en", + "family": "request_correction", + "question": "Which date did I most recently request for VISIT-2103? Read only.", + "source_messages": [ + { + "id": "note-3-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-3-1", + "role": "customer", + "content": "For VISIT-2103, I initially asked for 2027-04-04." + }, + { + "id": "note-3-2", + "role": "customer", + "content": "Replace that request with 2027-05-04; 2027-04-04 is no longer my choice." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-03", + "id": "VISIT-2103", + "date": "2027-03-04", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-03", + "id": "VISIT-3103", + "date": "2027-06-04", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The later customer replacement supersedes the earlier requested date for VISIT-2103; the backend booked date is a different fact. Source dates in order: ['2027-04-04', '2027-05-04', '2027-04-04'].", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-04-en", + "family": "request_correction", + "question": "Which date did I most recently request for VISIT-2104? Read only.", + "source_messages": [ + { + "id": "note-4-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. For VISIT-2104, I initially asked for 2027-04-05. Replace that request with 2027-05-05; 2027-04-05 is no longer my choice." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-04", + "id": "VISIT-2104", + "date": "2027-03-05", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-04", + "id": "VISIT-3104", + "date": "2027-06-05", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The later customer replacement supersedes the earlier requested date for VISIT-2104; the backend booked date is a different fact. Source dates in order: ['2027-04-05', '2027-05-05', '2027-04-05'].", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-05-en", + "family": "missing_replacement", + "question": "Which replacement date do I currently want for VISIT-2105? Read only; do not change the booking.", + "source_messages": [ + { + "id": "note-5-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-5-1", + "role": "customer", + "content": "I no longer want 2027-04-06 for VISIT-2105." + }, + { + "id": "note-5-2", + "role": "customer", + "content": "I have not decided on a replacement date. My other booking VISIT-3105 is unrelated." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-05", + "id": "VISIT-2105", + "date": "2027-03-06", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-05", + "id": "VISIT-3105", + "date": "2027-06-06", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": null, + "needs_clarification": true + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The customer withdraws the former date for VISIT-2105 and explicitly has not chosen a replacement. The answer is null with prospective clarification needed to choose a unique date; the other booking is unrelated.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-06-en", + "family": "missing_replacement", + "question": "Which replacement date do I currently want for VISIT-2106? Read only; do not change the booking.", + "source_messages": [ + { + "id": "note-6-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. I no longer want 2027-04-07 for VISIT-2106. I have not decided on a replacement date. My other booking VISIT-3106 is unrelated." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-06", + "id": "VISIT-2106", + "date": "2027-03-07", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-06", + "id": "VISIT-3106", + "date": "2027-06-07", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": null, + "needs_clarification": true + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The source explicitly withdraws 2027-04-07 and says no replacement was chosen; the other booking is unrelated.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-07-en", + "family": "missing_replacement", + "question": "Which replacement date do I currently want for VISIT-2107? Read only; do not change the booking.", + "source_messages": [ + { + "id": "note-7-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-7-1", + "role": "customer", + "content": "I no longer want 2027-04-08 for VISIT-2107." + }, + { + "id": "note-7-2", + "role": "customer", + "content": "I have not decided on a replacement date. My other booking VISIT-3107 is unrelated." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-07", + "id": "VISIT-2107", + "date": "2027-03-08", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-07", + "id": "VISIT-3107", + "date": "2027-06-08", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": null, + "needs_clarification": true + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The customer withdraws the former date for VISIT-2107 and explicitly has not chosen a replacement. The answer is null with prospective clarification needed to choose a unique date; the other booking is unrelated.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-08-en", + "family": "missing_replacement", + "question": "Which replacement date do I currently want for VISIT-2108? Read only; do not change the booking.", + "source_messages": [ + { + "id": "note-8-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. I no longer want 2027-04-09 for VISIT-2108. I have not decided on a replacement date. My other booking VISIT-3108 is unrelated." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-08", + "id": "VISIT-2108", + "date": "2027-03-09", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-08", + "id": "VISIT-3108", + "date": "2027-06-09", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": null, + "needs_clarification": true + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The customer withdraws the former date for VISIT-2108 and explicitly has not chosen a replacement. The answer is null with prospective clarification needed to choose a unique date; the other booking is unrelated.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-09-en", + "family": "resolved_condition", + "question": "From the stated condition, which date should I choose for VISIT-2109? Do not change it.", + "source_messages": [ + { + "id": "note-9-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-9-1", + "role": "customer", + "content": "For VISIT-2109, choose 2027-04-10 if a morning slot is available; otherwise choose 2027-05-10." + }, + { + "id": "note-9-2", + "role": "customer", + "content": "The morning slot is available. I am only asking about my choice, not authorizing a booking change." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-09", + "id": "VISIT-2109", + "date": "2027-03-10", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-09", + "id": "VISIT-3109", + "date": "2027-06-10", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The morning slot is explicitly available, selecting 2027-04-10 rather than the fallback 2027-05-10; current booking date is a separate fact.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-10-en", + "family": "resolved_condition", + "question": "From the stated condition, which date should I choose for VISIT-2110? Do not change it.", + "source_messages": [ + { + "id": "note-10-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. For VISIT-2110, choose 2027-04-11 if a morning slot is available; otherwise choose 2027-05-11. The morning slot is not available. I am asking about my choice only." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-10", + "id": "VISIT-2110", + "date": "2027-03-11", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-10", + "id": "VISIT-3110", + "date": "2027-06-11", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The stated availability condition selects the applicable branch for VISIT-2110; the source resolves the choice without a booking write. Branch dates in order: ['2027-04-11', '2027-05-11'].", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-11-en", + "family": "resolved_condition", + "question": "From the stated condition, which date should I choose for VISIT-2111? Do not change it.", + "source_messages": [ + { + "id": "note-11-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-11-1", + "role": "customer", + "content": "For VISIT-2111, choose 2027-04-12 if a morning slot is available; otherwise choose 2027-05-12." + }, + { + "id": "note-11-2", + "role": "customer", + "content": "The morning slot is available. I am only asking about my choice, not authorizing a booking change." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-11", + "id": "VISIT-2111", + "date": "2027-03-12", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-11", + "id": "VISIT-3111", + "date": "2027-06-12", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The stated availability condition selects the applicable branch for VISIT-2111; the source resolves the choice without a booking write. Branch dates in order: ['2027-04-12', '2027-05-12'].", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-12-en", + "family": "resolved_condition", + "question": "From the stated condition, which date should I choose for VISIT-2112? Do not change it.", + "source_messages": [ + { + "id": "note-12-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. For VISIT-2112, choose 2027-04-13 if a morning slot is available; otherwise choose 2027-05-13. The morning slot is not available. I am asking about my choice only." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-12", + "id": "VISIT-2112", + "date": "2027-03-13", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-12", + "id": "VISIT-3112", + "date": "2027-06-13", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The stated availability condition selects the applicable branch for VISIT-2112; the source resolves the choice without a booking write. Branch dates in order: ['2027-04-13', '2027-05-13'].", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-13-en", + "family": "unresolved_condition", + "question": "From the stated condition, which date should I choose for VISIT-2113? Do not change it.", + "source_messages": [ + { + "id": "note-13-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-13-1", + "role": "customer", + "content": "For VISIT-2113, I would choose 2027-04-14 if the morning slot is available, or 2027-05-14 if it is not." + }, + { + "id": "note-13-2", + "role": "customer", + "content": "Nobody has checked the availability yet. Do not guess or change my booking." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-13", + "id": "VISIT-2113", + "date": "2027-03-14", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-13", + "id": "VISIT-3113", + "date": "2027-06-14", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": null, + "needs_clarification": true + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The two conditional dates for VISIT-2113 remain alternatives because availability was not checked. The answer is null with prospective clarification/condition resolution required; current backend date is not a branch choice.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-14-en", + "family": "unresolved_condition", + "question": "From the stated condition, which date should I choose for VISIT-2114? Do not change it.", + "source_messages": [ + { + "id": "note-14-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. For VISIT-2114, I would choose 2027-04-15 if the morning slot is available, or 2027-05-15 if it is not. Nobody has checked the availability yet. Do not guess or change my booking." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-14", + "id": "VISIT-2114", + "date": "2027-03-15", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-14", + "id": "VISIT-3114", + "date": "2027-06-15", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": null, + "needs_clarification": true + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The morning-slot condition has two dates, and availability has not been checked; read_current booking date does not resolve availability.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-15-en", + "family": "unresolved_condition", + "question": "From the stated condition, which date should I choose for VISIT-2115? Do not change it.", + "source_messages": [ + { + "id": "note-15-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-15-1", + "role": "customer", + "content": "For VISIT-2115, I would choose 2027-04-16 if the morning slot is available, or 2027-05-16 if it is not." + }, + { + "id": "note-15-2", + "role": "customer", + "content": "Nobody has checked the availability yet. Do not guess or change my booking." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-15", + "id": "VISIT-2115", + "date": "2027-03-16", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-15", + "id": "VISIT-3115", + "date": "2027-06-16", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": null, + "needs_clarification": true + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The two conditional dates for VISIT-2115 remain alternatives because availability was not checked. The answer is null with prospective clarification/condition resolution required; current backend date is not a branch choice.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-16-en", + "family": "unresolved_condition", + "question": "From the stated condition, which date should I choose for VISIT-2116? Do not change it.", + "source_messages": [ + { + "id": "note-16-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. For VISIT-2116, I would choose 2027-04-17 if the morning slot is available, or 2027-05-17 if it is not. Nobody has checked the availability yet. Do not guess or change my booking." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-16", + "id": "VISIT-2116", + "date": "2027-03-17", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-16", + "id": "VISIT-3116", + "date": "2027-06-17", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": null, + "needs_clarification": true + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "Morning availability remains unchecked; no unique choice between 2027-04-17 and 2027-05-17 is established.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-17-en", + "family": "separate_booking", + "question": "What is my current preferred date for VISIT-2117? Read only; do not change the booking.", + "source_messages": [ + { + "id": "note-17-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-17-1", + "role": "customer", + "content": "My preferred date for VISIT-2117 is 2027-04-18. For VISIT-3117, it was 2027-03-18." + }, + { + "id": "note-17-2", + "role": "customer", + "content": "Correction only for VISIT-3117: I now want 2027-05-18. The first booking's preference is unchanged." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-17", + "id": "VISIT-2117", + "date": "2027-03-18", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-17", + "id": "VISIT-3117", + "date": "2027-06-18", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The customer preference for VISIT-2117 is the first booking date 2027-04-18; the later correction explicitly applies only to the other booking. Current backend booked date is a separate fact.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-18-en", + "family": "separate_booking", + "question": "What is my current preferred date for VISIT-2118? Read only; do not change the booking.", + "source_messages": [ + { + "id": "note-18-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. My preferred date for VISIT-2118 is 2027-04-19. For VISIT-3118, it was 2027-03-19. Correction only for VISIT-3118: I now want 2027-05-19. The first booking's preference is unchanged." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-18", + "id": "VISIT-2118", + "date": "2027-03-19", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-18", + "id": "VISIT-3118", + "date": "2027-06-19", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "VISIT-2118 preference is explicitly 2027-04-19. The later correction applies only to VISIT-3118, and the 2027-03-19 backend date is not the asked preference.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-19-en", + "family": "separate_booking", + "question": "What is my current preferred date for VISIT-2119? Read only; do not change the booking.", + "source_messages": [ + { + "id": "note-19-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-19-1", + "role": "customer", + "content": "My preferred date for VISIT-2119 is 2027-04-20. For VISIT-3119, it was 2027-03-20." + }, + { + "id": "note-19-2", + "role": "customer", + "content": "Correction only for VISIT-3119: I now want 2027-05-20. The first booking's preference is unchanged." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-19", + "id": "VISIT-2119", + "date": "2027-03-20", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-19", + "id": "VISIT-3119", + "date": "2027-06-20", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The source explicitly sets VISIT-2119 preference to 2027-04-20; the later correction concerns only VISIT-3119. The read_current date is not the asked preference.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-20-en", + "family": "separate_booking", + "question": "What is my current preferred date for VISIT-2120? Read only; do not change the booking.", + "source_messages": [ + { + "id": "note-20-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. My preferred date for VISIT-2120 is 2027-04-21. For VISIT-3120, it was 2027-03-21. Correction only for VISIT-3120: I now want 2027-05-21. The first booking's preference is unchanged." + } + ], + "requested_change": null, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-20", + "id": "VISIT-2120", + "date": "2027-03-21", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-20", + "id": "VISIT-3120", + "date": "2027-06-21", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "question_target": "source_request_or_condition", + "label_expected_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "VISIT-2120 preferred date stays 2027-04-21; the 2027-05-21 correction belongs only to VISIT-3120. read_current 2027-03-21 is not the preference.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-21-en", + "family": "approved_change", + "question": "Handle the supplied change request for VISIT-2121. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-21-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-21-1", + "role": "customer", + "content": "Please move VISIT-2121 to 2027-04-22." + }, + { + "id": "note-21-2", + "role": "customer", + "content": "Use the supplied request identifier and the server's matching approval. Tell me what date is confirmed after handling the request." + } + ], + "requested_change": { + "appointment_id": "VISIT-2121", + "date": "2027-04-22", + "expected_version": 5, + "idempotency_key": "REQUEST-21-CHANGE-16", + "approval_id": "AUTH-21-GRANT-16" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-21", + "id": "VISIT-2121", + "date": "2027-03-22", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-21", + "id": "VISIT-3121", + "date": "2027-06-22", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-21", + "id": "AUTH-21-GRANT-16", + "appointment_id": "VISIT-2121", + "operation": "update_appointment", + "date": "2027-04-22", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The trusted matching approval permits the supplied change for VISIT-2121; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-22-en", + "family": "approved_change", + "question": "Handle the supplied change request for VISIT-2122. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-22-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. Please move VISIT-2122 to 2027-04-23. Use the supplied request identifier and the server's matching approval. Tell me what date is confirmed after handling the request." + } + ], + "requested_change": { + "appointment_id": "VISIT-2122", + "date": "2027-04-23", + "expected_version": 5, + "idempotency_key": "REQUEST-22-CHANGE-16", + "approval_id": "AUTH-22-GRANT-16" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-22", + "id": "VISIT-2122", + "date": "2027-03-23", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-22", + "id": "VISIT-3122", + "date": "2027-06-23", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-22", + "id": "AUTH-22-GRANT-16", + "appointment_id": "VISIT-2122", + "operation": "update_appointment", + "date": "2027-04-23", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The trusted matching approval permits the supplied change for VISIT-2122; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-23-en", + "family": "approved_change", + "question": "Handle the supplied change request for VISIT-2123. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-23-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-23-1", + "role": "customer", + "content": "Please move VISIT-2123 to 2027-04-24." + }, + { + "id": "note-23-2", + "role": "customer", + "content": "Use the supplied request identifier and the server's matching approval. Tell me what date is confirmed after handling the request." + } + ], + "requested_change": { + "appointment_id": "VISIT-2123", + "date": "2027-04-24", + "expected_version": 5, + "idempotency_key": "REQUEST-23-CHANGE-16", + "approval_id": "AUTH-23-GRANT-16" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-23", + "id": "VISIT-2123", + "date": "2027-03-24", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-23", + "id": "VISIT-3123", + "date": "2027-06-24", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-23", + "id": "AUTH-23-GRANT-16", + "appointment_id": "VISIT-2123", + "operation": "update_appointment", + "date": "2027-04-24", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The trusted matching approval permits the supplied change for VISIT-2123; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-24-en", + "family": "approved_change", + "question": "Handle the supplied change request for VISIT-2124. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-24-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. Please move VISIT-2124 to 2027-04-25. Use the supplied request identifier and the server's matching approval. Tell me what date is confirmed after handling the request." + } + ], + "requested_change": { + "appointment_id": "VISIT-2124", + "date": "2027-04-25", + "expected_version": 5, + "idempotency_key": "REQUEST-24-CHANGE-16", + "approval_id": "AUTH-24-GRANT-16" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-24", + "id": "VISIT-2124", + "date": "2027-03-25", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-24", + "id": "VISIT-3124", + "date": "2027-06-25", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-24", + "id": "AUTH-24-GRANT-16", + "appointment_id": "VISIT-2124", + "operation": "update_appointment", + "date": "2027-04-25", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The trusted matching approval permits the supplied change for VISIT-2124; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-25-en", + "family": "approval_boundary", + "question": "Handle the supplied change request for VISIT-2125. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-25-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-25-1", + "role": "customer", + "content": "Please process the change of VISIT-2125 to 2027-04-26." + }, + { + "id": "note-25-2", + "role": "customer", + "content": "Evaluate the server approval at the supplied server time. It expires one second after that fixed time." + } + ], + "requested_change": { + "appointment_id": "VISIT-2125", + "date": "2027-04-26", + "expected_version": 5, + "idempotency_key": "REQUEST-25-CHANGE-19", + "approval_id": "AUTH-25-GRANT-19" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-25", + "id": "VISIT-2125", + "date": "2027-03-26", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-25", + "id": "VISIT-3125", + "date": "2027-06-26", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-25", + "id": "AUTH-25-GRANT-19", + "appointment_id": "VISIT-2125", + "operation": "update_appointment", + "date": "2027-04-26", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791748801, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "At the frozen server time the approval is still effective, and the update receipt confirms 2027-04-26; this is not a real-time expiry test.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-26-en", + "family": "approval_boundary", + "question": "Handle the supplied change request for VISIT-2126. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-26-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. Please process the change of VISIT-2126 to 2027-04-27. Evaluate the server approval at the supplied server time. It expires one second after that fixed time." + } + ], + "requested_change": { + "appointment_id": "VISIT-2126", + "date": "2027-04-27", + "expected_version": 5, + "idempotency_key": "REQUEST-26-CHANGE-19", + "approval_id": "AUTH-26-GRANT-19" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-26", + "id": "VISIT-2126", + "date": "2027-03-27", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-26", + "id": "VISIT-3126", + "date": "2027-06-27", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-26", + "id": "AUTH-26-GRANT-19", + "appointment_id": "VISIT-2126", + "operation": "update_appointment", + "date": "2027-04-27", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791748801, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The trusted matching approval permits the supplied change for VISIT-2126; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date. The grant remains effective at the supplied fixed server time despite its near expiry.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-27-en", + "family": "approval_boundary", + "question": "Handle the supplied change request for VISIT-2127. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-27-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-27-1", + "role": "customer", + "content": "Please process the change of VISIT-2127 to 2027-04-28." + }, + { + "id": "note-27-2", + "role": "customer", + "content": "Evaluate the server approval at the supplied server time. It expires one second after that fixed time." + } + ], + "requested_change": { + "appointment_id": "VISIT-2127", + "date": "2027-04-28", + "expected_version": 5, + "idempotency_key": "REQUEST-27-CHANGE-19", + "approval_id": "AUTH-27-GRANT-19" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-27", + "id": "VISIT-2127", + "date": "2027-03-28", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-27", + "id": "VISIT-3127", + "date": "2027-06-28", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-27", + "id": "AUTH-27-GRANT-19", + "appointment_id": "VISIT-2127", + "operation": "update_appointment", + "date": "2027-04-28", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791748801, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The trusted matching approval permits the supplied change for VISIT-2127; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date. The grant remains effective at the supplied fixed server time despite its near expiry.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-28-en", + "family": "approval_boundary", + "question": "Handle the supplied change request for VISIT-2128. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-28-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. Please process the change of VISIT-2128 to 2027-04-29. Evaluate the server approval at the supplied server time. It expires one second after that fixed time." + } + ], + "requested_change": { + "appointment_id": "VISIT-2128", + "date": "2027-04-29", + "expected_version": 5, + "idempotency_key": "REQUEST-28-CHANGE-19", + "approval_id": "AUTH-28-GRANT-19" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-28", + "id": "VISIT-2128", + "date": "2027-03-29", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-28", + "id": "VISIT-3128", + "date": "2027-06-29", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-28", + "id": "AUTH-28-GRANT-19", + "appointment_id": "VISIT-2128", + "operation": "update_appointment", + "date": "2027-04-29", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791748801, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The trusted matching approval permits the supplied change for VISIT-2128; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date. The grant remains effective at the supplied fixed server time despite its near expiry.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-29-en", + "family": "future_approval", + "question": "Handle the supplied change request for VISIT-2129. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-29-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-29-1", + "role": "customer", + "content": "I want VISIT-2129 changed to 2027-04-30 now." + }, + { + "id": "note-29-2", + "role": "customer", + "content": "The server grant takes effect one hour after the supplied server time. My request does not change its validity period." + } + ], + "requested_change": { + "appointment_id": "VISIT-2129", + "date": "2027-04-30", + "expected_version": 5, + "idempotency_key": "REQUEST-29-CHANGE-20", + "approval_id": "AUTH-29-GRANT-20" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-29", + "id": "VISIT-2129", + "date": "2027-03-30", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-29", + "id": "VISIT-3129", + "date": "2027-06-30", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-29", + "id": "AUTH-29-GRANT-20", + "appointment_id": "VISIT-2129", + "operation": "update_appointment", + "date": "2027-04-30", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791752400, + "expires_at": 1791756000, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The matching server grant for VISIT-2129 is not effective at the supplied server time. The requested later date is not confirmed; retain the current backend date. A rejected attempted write, if any, is an actor error, not authorization.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-30-en", + "family": "future_approval", + "question": "Handle the supplied change request for VISIT-2130. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-30-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. I want VISIT-2130 changed to 2027-05-01 now. The server grant takes effect one hour after the supplied server time. My request does not change its validity period." + } + ], + "requested_change": { + "appointment_id": "VISIT-2130", + "date": "2027-05-01", + "expected_version": 5, + "idempotency_key": "REQUEST-30-CHANGE-20", + "approval_id": "AUTH-30-GRANT-20" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-30", + "id": "VISIT-2130", + "date": "2027-03-31", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-30", + "id": "VISIT-3130", + "date": "2027-07-01", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-30", + "id": "AUTH-30-GRANT-20", + "appointment_id": "VISIT-2130", + "operation": "update_appointment", + "date": "2027-05-01", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791752400, + "expires_at": 1791756000, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The grant is not effective until an hour after frozen server time; no update is attempted, and backend remains 2027-03-31.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-31-en", + "family": "future_approval", + "question": "Handle the supplied change request for VISIT-2131. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-31-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-31-1", + "role": "customer", + "content": "I want VISIT-2131 changed to 2027-05-02 now." + }, + { + "id": "note-31-2", + "role": "customer", + "content": "The server grant takes effect one hour after the supplied server time. My request does not change its validity period." + } + ], + "requested_change": { + "appointment_id": "VISIT-2131", + "date": "2027-05-02", + "expected_version": 5, + "idempotency_key": "REQUEST-31-CHANGE-20", + "approval_id": "AUTH-31-GRANT-20" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-31", + "id": "VISIT-2131", + "date": "2027-04-01", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-31", + "id": "VISIT-3131", + "date": "2027-07-02", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-31", + "id": "AUTH-31-GRANT-20", + "appointment_id": "VISIT-2131", + "operation": "update_appointment", + "date": "2027-05-02", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791752400, + "expires_at": 1791756000, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The matching server grant for VISIT-2131 is not effective at the supplied server time. The requested later date is not confirmed; retain the current backend date. A rejected attempted write, if any, is an actor error, not authorization.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-32-en", + "family": "future_approval", + "question": "Handle the supplied change request for VISIT-2132. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-32-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. I want VISIT-2132 changed to 2027-05-03 now. The server grant takes effect one hour after the supplied server time. My request does not change its validity period." + } + ], + "requested_change": { + "appointment_id": "VISIT-2132", + "date": "2027-05-03", + "expected_version": 5, + "idempotency_key": "REQUEST-32-CHANGE-20", + "approval_id": "AUTH-32-GRANT-20" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-32", + "id": "VISIT-2132", + "date": "2027-04-02", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-32", + "id": "VISIT-3132", + "date": "2027-07-03", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-32", + "id": "AUTH-32-GRANT-20", + "appointment_id": "VISIT-2132", + "operation": "update_appointment", + "date": "2027-05-03", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791752400, + "expires_at": 1791756000, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The actor attempts the future-approved change; backend rejects it as approval_not_effective and retains 2027-04-02.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-33-en", + "family": "completed_older_request", + "question": "Handle the supplied change request for VISIT-2133. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-33-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-33-1", + "role": "customer", + "content": "Check my original change of VISIT-2133 to 2027-05-04 using its original request identifier." + }, + { + "id": "note-33-2", + "role": "customer", + "content": "It already succeeded. A separate later change moved it to 2027-06-03. Do not undo the later change when handling this duplicate request." + } + ], + "requested_change": { + "appointment_id": "VISIT-2133", + "date": "2027-05-04", + "expected_version": 5, + "idempotency_key": "REQUEST-33-CHANGE-23", + "approval_id": "AUTH-33-GRANT-23" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-33", + "id": "VISIT-2133", + "date": "2027-04-03", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-33", + "id": "VISIT-3133", + "date": "2027-07-04", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-33", + "id": "AUTH-33-GRANT-23", + "appointment_id": "VISIT-2133", + "operation": "update_appointment", + "date": "2027-05-04", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-33", + "id": "AUTH-33-LATER-GRANT-23", + "appointment_id": "VISIT-2133", + "operation": "update_appointment", + "date": "2027-06-03", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The original request key for VISIT-2133 has an exact prior confirmed receipt, followed by a distinct later confirmed change. The duplicate is already applied, the later backend date remains current, and no new write is needed.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-34-en", + "family": "completed_older_request", + "question": "Handle the supplied change request for VISIT-2134. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-34-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. Check my original change of VISIT-2134 to 2027-05-05 using its original request identifier. It already succeeded. A separate later change moved it to 2027-06-04. Do not undo the later change when handling this duplicate request." + } + ], + "requested_change": { + "appointment_id": "VISIT-2134", + "date": "2027-05-05", + "expected_version": 5, + "idempotency_key": "REQUEST-34-CHANGE-23", + "approval_id": "AUTH-34-GRANT-23" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-34", + "id": "VISIT-2134", + "date": "2027-04-04", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-34", + "id": "VISIT-3134", + "date": "2027-07-05", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-34", + "id": "AUTH-34-GRANT-23", + "appointment_id": "VISIT-2134", + "operation": "update_appointment", + "date": "2027-05-05", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-34", + "id": "AUTH-34-LATER-GRANT-23", + "appointment_id": "VISIT-2134", + "operation": "update_appointment", + "date": "2027-06-04", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The original change to 2027-05-05 has a matching prior receipt, but a later distinct completed change leaves 2027-06-04 current.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-35-en", + "family": "completed_older_request", + "question": "Handle the supplied change request for VISIT-2135. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-35-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-35-1", + "role": "customer", + "content": "Check my original change of VISIT-2135 to 2027-05-06 using its original request identifier." + }, + { + "id": "note-35-2", + "role": "customer", + "content": "It already succeeded. A separate later change moved it to 2027-06-05. Do not undo the later change when handling this duplicate request." + } + ], + "requested_change": { + "appointment_id": "VISIT-2135", + "date": "2027-05-06", + "expected_version": 5, + "idempotency_key": "REQUEST-35-CHANGE-23", + "approval_id": "AUTH-35-GRANT-23" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-35", + "id": "VISIT-2135", + "date": "2027-04-05", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-35", + "id": "VISIT-3135", + "date": "2027-07-06", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-35", + "id": "AUTH-35-GRANT-23", + "appointment_id": "VISIT-2135", + "operation": "update_appointment", + "date": "2027-05-06", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-35", + "id": "AUTH-35-LATER-GRANT-23", + "appointment_id": "VISIT-2135", + "operation": "update_appointment", + "date": "2027-06-05", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The original request key for VISIT-2135 has an exact prior confirmed receipt, followed by a distinct later confirmed change. The duplicate is already applied, the later backend date remains current, and no new write is needed.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-36-en", + "family": "completed_older_request", + "question": "Handle the supplied change request for VISIT-2136. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-36-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. Check my original change of VISIT-2136 to 2027-05-07 using its original request identifier. It already succeeded. A separate later change moved it to 2027-06-06. Do not undo the later change when handling this duplicate request." + } + ], + "requested_change": { + "appointment_id": "VISIT-2136", + "date": "2027-05-07", + "expected_version": 5, + "idempotency_key": "REQUEST-36-CHANGE-23", + "approval_id": "AUTH-36-GRANT-23" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-36", + "id": "VISIT-2136", + "date": "2027-04-06", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-36", + "id": "VISIT-3136", + "date": "2027-07-07", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-36", + "id": "AUTH-36-GRANT-23", + "appointment_id": "VISIT-2136", + "operation": "update_appointment", + "date": "2027-05-07", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-36", + "id": "AUTH-36-LATER-GRANT-23", + "appointment_id": "VISIT-2136", + "operation": "update_appointment", + "date": "2027-06-06", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "Prior receipts prove the original request and later 2027-06-06 update; read_current confirms that later date.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-37-en", + "family": "conflicting_request_key", + "question": "Handle the supplied change request for VISIT-2137. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-37-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-37-1", + "role": "customer", + "content": "Now I want 2027-06-07 for VISIT-2137." + }, + { + "id": "note-37-2", + "role": "customer", + "content": "The supplied request identifier was already used for the completed change to 2027-05-08. Do not apply a different change using that same identifier." + } + ], + "requested_change": { + "appointment_id": "VISIT-2137", + "date": "2027-06-07", + "expected_version": 6, + "idempotency_key": "REQUEST-37-CHANGE-24", + "approval_id": "AUTH-37-CONFLICT-GRANT" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-37", + "id": "VISIT-2137", + "date": "2027-04-07", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-37", + "id": "VISIT-3137", + "date": "2027-07-08", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-37", + "id": "AUTH-37-GRANT-24", + "appointment_id": "VISIT-2137", + "operation": "update_appointment", + "date": "2027-05-08", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-37", + "id": "AUTH-37-CONFLICT-GRANT", + "appointment_id": "VISIT-2137", + "operation": "update_appointment", + "date": "2027-06-07", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The supplied key for VISIT-2137 was already consumed for a different completed request. The new proposed date cannot be applied with that key; retain the current confirmed backend date and make no new write.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-38-en", + "family": "conflicting_request_key", + "question": "Handle the supplied change request for VISIT-2138. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-38-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. Now I want 2027-06-08 for VISIT-2138. The supplied request identifier was already used for the completed change to 2027-05-09. Do not apply a different change using that same identifier." + } + ], + "requested_change": { + "appointment_id": "VISIT-2138", + "date": "2027-06-08", + "expected_version": 6, + "idempotency_key": "REQUEST-38-CHANGE-24", + "approval_id": "AUTH-38-CONFLICT-GRANT" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-38", + "id": "VISIT-2138", + "date": "2027-04-08", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-38", + "id": "VISIT-3138", + "date": "2027-07-09", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-38", + "id": "AUTH-38-GRANT-24", + "appointment_id": "VISIT-2138", + "operation": "update_appointment", + "date": "2027-05-09", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-38", + "id": "AUTH-38-CONFLICT-GRANT", + "appointment_id": "VISIT-2138", + "operation": "update_appointment", + "date": "2027-06-08", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The supplied key for VISIT-2138 was already consumed for a different completed request. The new proposed date cannot be applied with that key; retain the current confirmed backend date and make no new write.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-39-en", + "family": "conflicting_request_key", + "question": "Handle the supplied change request for VISIT-2139. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-39-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-39-1", + "role": "customer", + "content": "Now I want 2027-06-09 for VISIT-2139." + }, + { + "id": "note-39-2", + "role": "customer", + "content": "The supplied request identifier was already used for the completed change to 2027-05-10. Do not apply a different change using that same identifier." + } + ], + "requested_change": { + "appointment_id": "VISIT-2139", + "date": "2027-06-09", + "expected_version": 6, + "idempotency_key": "REQUEST-39-CHANGE-24", + "approval_id": "AUTH-39-CONFLICT-GRANT" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-39", + "id": "VISIT-2139", + "date": "2027-04-09", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-39", + "id": "VISIT-3139", + "date": "2027-07-10", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-39", + "id": "AUTH-39-GRANT-24", + "appointment_id": "VISIT-2139", + "operation": "update_appointment", + "date": "2027-05-10", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-39", + "id": "AUTH-39-CONFLICT-GRANT", + "appointment_id": "VISIT-2139", + "operation": "update_appointment", + "date": "2027-06-09", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The 2027-06-09 desired change conflicts with the key previously used for 2027-05-10; no new update occurs and current date remains 2027-05-10.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + }, + { + "id": "fresh-40-en", + "family": "conflicting_request_key", + "question": "Handle the supplied change request for VISIT-2140. What date is actually confirmed now?", + "source_messages": [ + { + "id": "note-40-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. Now I want 2027-06-10 for VISIT-2140. The supplied request identifier was already used for the completed change to 2027-05-11. Do not apply a different change using that same identifier." + } + ], + "requested_change": { + "appointment_id": "VISIT-2140", + "date": "2027-06-10", + "expected_version": 6, + "idempotency_key": "REQUEST-40-CHANGE-24", + "approval_id": "AUTH-40-CONFLICT-GRANT" + }, + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-40", + "id": "VISIT-2140", + "date": "2027-04-10", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-40", + "id": "VISIT-3140", + "date": "2027-07-11", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-40", + "id": "AUTH-40-GRANT-24", + "appointment_id": "VISIT-2140", + "operation": "update_appointment", + "date": "2027-05-11", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-40", + "id": "AUTH-40-CONFLICT-GRANT", + "appointment_id": "VISIT-2140", + "operation": "update_appointment", + "date": "2027-06-10", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "question_target": "actual_current_date", + "label_expected_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "source_gold_ambiguity_flag": false, + "assistant_source_reading": "The key belongs to a completed 2027-05-11 request with different arguments from the desired 2027-06-10 change; no new write occurs.", + "source_file_sha256": "48bd3756f4ac9de57566c97c2d4d21b5352d428f3b0389f421e6a9a18fb0dc17" + } + ], + "episodes": [ + { + "id": "confirm-fresh-01-en-m0", + "case_id": "fresh-01-en", + "model": "sarvam-105b", + "family": "request_correction", + "treated_actor_prose": true, + "original_question": "Which date did I most recently request for VISIT-2101? Read only.", + "original_source_messages": [ + { + "id": "note-1-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-1-1", + "role": "customer", + "content": "For VISIT-2101, I initially asked for 2027-04-02." + }, + { + "id": "note-1-2", + "role": "customer", + "content": "Replace that request with 2027-05-02; 2027-04-02 is no longer my choice." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-01", + "id": "VISIT-2101", + "date": "2027-03-02", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-01", + "id": "VISIT-3101", + "date": "2027-06-02", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "296e9404a430ef8821354942b819f495405ce66c268f57e70c67db83b5e114ca", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nYour most recently requested date for **VISIT-2101** was **2027-05-02**, based on source message **note-1-2** in which you stated:\n\n> *\"Replace that request with 2027-05-02; 2027-04-02 is no longer my choice.\"*\n\n(Earlier, in **note-1-1**, you had requested **2027-04-02**, but that was superseded by the later request.)", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-02", + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The later customer replacement supersedes the earlier requested date for VISIT-2101; the backend booked date is a different fact. Source dates in order: ['2027-04-02', '2027-05-02', '2027-04-02'].", + "reporters": { + "chronological": "confirm-fresh-01-en-m0-source-chronological", + "grouped_lossless": "confirm-fresh-01-en-m0-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-01-en-m0-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-01-en-m1", + "case_id": "fresh-01-en", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "treated_actor_prose": true, + "original_question": "Which date did I most recently request for VISIT-2101? Read only.", + "original_source_messages": [ + { + "id": "note-1-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-1-1", + "role": "customer", + "content": "For VISIT-2101, I initially asked for 2027-04-02." + }, + { + "id": "note-1-2", + "role": "customer", + "content": "Replace that request with 2027-05-02; 2027-04-02 is no longer my choice." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-01", + "id": "VISIT-2101", + "date": "2027-03-02", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-01", + "id": "VISIT-3101", + "date": "2027-06-02", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "8e805103de4c9abf0a50c89b5090c9764020949b1d92f616de6e448b0f628f3e", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-815fcdb57d073f92", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2101\"}" + } + } + ] + }, + { + "content": "Based on the source messages, the most recent date you requested for **VISIT-2101** is **2027-05-02** (from the message: *\"Replace that request with 2027-05-02; 2027-04-02 is no longer my choice.\"*).\n\nThe current booking date remains **2027-03-02**, and the booking is still open (version 5, closed: false). Since your intent is read-only, no state change was made.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-815fcdb57d073f92", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2101" + }, + "response": { + "date": "2027-03-02", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-02", + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The later 2027-05-02 customer correction supersedes 2027-04-02 for VISIT-2101; read_current 2027-03-02 is the separate booked date.", + "reporters": { + "chronological": "confirm-fresh-01-en-m1-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-01-en-m1-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-01-en-m1-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-02-en-m0", + "case_id": "fresh-02-en", + "model": "sarvam-105b", + "family": "request_correction", + "treated_actor_prose": true, + "original_question": "Which date did I most recently request for VISIT-2102? Read only.", + "original_source_messages": [ + { + "id": "note-2-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. For VISIT-2102, I initially asked for 2027-04-03. Replace that request with 2027-05-03; 2027-04-03 is no longer my choice." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-02", + "id": "VISIT-2102", + "date": "2027-03-03", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-02", + "id": "VISIT-3102", + "date": "2027-06-03", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "7c79a03c7f5dd7bf0b349506fbf24fb4d9cc79d0defd1d39acc3bbca965d15c7", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nAccording to the source message for appointment **VISIT-2102**, you most recently **requested 2027-05-03** (replacing the earlier request for 2027-04-03).\n\nPlease note that this is a **requested date**, not a confirmed change. The current confirmed date in the system remains **2027-03-03**. No approvals or execution receipts for that change have been recorded yet.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-03", + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The combined source still orders the original 2027-04-03 request before its replacement with 2027-05-03.", + "reporters": { + "chronological": "confirm-fresh-02-en-m0-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-02-en-m0-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-02-en-m0-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-02-en-m1", + "case_id": "fresh-02-en", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "treated_actor_prose": true, + "original_question": "Which date did I most recently request for VISIT-2102? Read only.", + "original_source_messages": [ + { + "id": "note-2-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. For VISIT-2102, I initially asked for 2027-04-03. Replace that request with 2027-05-03; 2027-04-03 is no longer my choice." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-02", + "id": "VISIT-2102", + "date": "2027-03-03", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-02", + "id": "VISIT-3102", + "date": "2027-06-03", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "a5b471e9f08e5935394678b8a2b2bac595297258249b843d8e217aa39f20d21c", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-a611d3839ed99164", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2102\"}" + } + } + ] + }, + { + "content": "Based on the source messages provided in your request, the most recent date you requested for **VISIT-2102** was:\n\n- **2027-05-03** — this replaced your earlier request of 2027-04-03.\n\nThe current server state for VISIT-2102 remains **2027-03-03**, version 5, and the booking is still open. Since your stated intent is read-only, no state change was made.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a611d3839ed99164", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2102" + }, + "response": { + "date": "2027-03-03", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-03", + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The later customer replacement supersedes the earlier requested date for VISIT-2102; the backend booked date is a different fact. Source dates in order: ['2027-04-03', '2027-05-03', '2027-04-03'].", + "reporters": { + "grouped_lossless": "confirm-fresh-02-en-m1-source-grouped_lossless", + "chronological": "confirm-fresh-02-en-m1-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-02-en-m1-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-03-en-m0", + "case_id": "fresh-03-en", + "model": "sarvam-105b", + "family": "request_correction", + "treated_actor_prose": true, + "original_question": "Which date did I most recently request for VISIT-2103? Read only.", + "original_source_messages": [ + { + "id": "note-3-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-3-1", + "role": "customer", + "content": "For VISIT-2103, I initially asked for 2027-04-04." + }, + { + "id": "note-3-2", + "role": "customer", + "content": "Replace that request with 2027-05-04; 2027-04-04 is no longer my choice." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-03", + "id": "VISIT-2103", + "date": "2027-03-04", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-03", + "id": "VISIT-3103", + "date": "2027-06-04", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "d81d775174fba4a04e18136a4d5e875dde801d7d8a1f0ba5f02d1b955e844b8c", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nFor **VISIT-2103**, your **most recent requested date** is **2027-05-04**.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-04", + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The later customer correction replaces 2027-04-04 with 2027-05-04 for the same booking; current backend 2027-03-04 is irrelevant to this request-date question.", + "reporters": { + "grouped_lossless": "confirm-fresh-03-en-m0-source-grouped_lossless", + "chronological": "confirm-fresh-03-en-m0-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-03-en-m0-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-03-en-m1", + "case_id": "fresh-03-en", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "treated_actor_prose": true, + "original_question": "Which date did I most recently request for VISIT-2103? Read only.", + "original_source_messages": [ + { + "id": "note-3-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-3-1", + "role": "customer", + "content": "For VISIT-2103, I initially asked for 2027-04-04." + }, + { + "id": "note-3-2", + "role": "customer", + "content": "Replace that request with 2027-05-04; 2027-04-04 is no longer my choice." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-03", + "id": "VISIT-2103", + "date": "2027-03-04", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-03", + "id": "VISIT-3103", + "date": "2027-06-04", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "8168ce5d94934b6ce3c591642918f8ea7c5905360089a488b3a37392e8c99a4a", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-bf4e994aaaa53f65", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2103\"}" + } + } + ] + }, + { + "content": "Based on the source messages, the dates Kavya most recently requested for **VISIT-2103** are:\n\n1. **2027-04-04** — initially requested (note-3-1)\n2. **2027-05-04** — requested to replace the earlier one; 2027-04-04 is no longer her choice (note-3-2)\n\nSo her **most recent request** is **2027-05-04**.\n\nThe current server state for VISIT-2103 remains **2027-03-04** (version 5, not closed). No approved change has been applied.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-bf4e994aaaa53f65", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2103" + }, + "response": { + "date": "2027-03-04", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-04", + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The later customer replacement supersedes the earlier requested date for VISIT-2103; the backend booked date is a different fact. Source dates in order: ['2027-04-04', '2027-05-04', '2027-04-04'].", + "reporters": { + "grouped_lossless": "confirm-fresh-03-en-m1-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-03-en-m1-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-03-en-m1-source-chronological" + } + }, + { + "id": "confirm-fresh-04-en-m0", + "case_id": "fresh-04-en", + "model": "sarvam-105b", + "family": "request_correction", + "treated_actor_prose": true, + "original_question": "Which date did I most recently request for VISIT-2104? Read only.", + "original_source_messages": [ + { + "id": "note-4-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. For VISIT-2104, I initially asked for 2027-04-05. Replace that request with 2027-05-05; 2027-04-05 is no longer my choice." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-04", + "id": "VISIT-2104", + "date": "2027-03-05", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-04", + "id": "VISIT-3104", + "date": "2027-06-05", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "ca1a0bd63880b37cd1ef8bb3c3d526b4ca10607b6a5580287ae9172687583537", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-9b8be3e9723e0c72", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2104\"}" + } + } + ] + }, + { + "content": "\nI have read the current state for VISIT-2104. Based on the source messages you provided (notes 4-0):\n\n- Your **initial request** was for **2027-04-05**.\n- Your **most recent request**, which replaced the above, was for **2027-05-05**.\n\nThere are no execution receipts or server approvals for these requests in the current state, so neither has been applied to the active booking (currently dated **2027-03-05**, version 5).", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9b8be3e9723e0c72", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2104" + }, + "response": { + "date": "2027-03-05", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-05", + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The later customer replacement supersedes the earlier requested date for VISIT-2104; the backend booked date is a different fact. Source dates in order: ['2027-04-05', '2027-05-05', '2027-04-05'].", + "reporters": { + "grouped_lossless": "confirm-fresh-04-en-m0-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-04-en-m0-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-04-en-m0-source-chronological" + } + }, + { + "id": "confirm-fresh-04-en-m1", + "case_id": "fresh-04-en", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "treated_actor_prose": true, + "original_question": "Which date did I most recently request for VISIT-2104? Read only.", + "original_source_messages": [ + { + "id": "note-4-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. For VISIT-2104, I initially asked for 2027-04-05. Replace that request with 2027-05-05; 2027-04-05 is no longer my choice." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-04", + "id": "VISIT-2104", + "date": "2027-03-05", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-04", + "id": "VISIT-3104", + "date": "2027-06-05", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "4f1355d04732812504b71d3528777cfe3500ae4147302017c2806717e35d7206", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-ace68c137406b19f", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2104\"}" + } + } + ] + }, + { + "content": "Based on the source message you provided, the most recent date you requested for **VISIT-2104** is **2027-05-05** (you asked to replace your earlier request for 2027-04-05).\n\nHowever, the current booking record shows the active date is **2027-03-05** (version 5, not closed). There are no server approvals or prior execution receipts confirming that the 2027-05-05 request was applied. Since your intent is read-only, no state change is being made.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-ace68c137406b19f", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2104" + }, + "response": { + "date": "2027-03-05", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-05", + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The combined customer message still orders the initial 2027-04-05 request before the 2027-05-05 replacement; read_current returns 2027-03-05 but does not change the requested date.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-04-en-m1-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-04-en-m1-source-chronological", + "grouped_lossless": "confirm-fresh-04-en-m1-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-05-en-m0", + "case_id": "fresh-05-en", + "model": "sarvam-105b", + "family": "missing_replacement", + "treated_actor_prose": true, + "original_question": "Which replacement date do I currently want for VISIT-2105? Read only; do not change the booking.", + "original_source_messages": [ + { + "id": "note-5-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-5-1", + "role": "customer", + "content": "I no longer want 2027-04-06 for VISIT-2105." + }, + { + "id": "note-5-2", + "role": "customer", + "content": "I have not decided on a replacement date. My other booking VISIT-3105 is unrelated." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-05", + "id": "VISIT-2105", + "date": "2027-03-06", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-05", + "id": "VISIT-3105", + "date": "2027-06-06", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "3b11eb2482a5093caa22c1e4f3821b86204f83a649dc2f976a8c1e2187ab0678", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nBased on the current booking state and the messages from Meera, there is **no replacement date currently selected** for **VISIT-2105**.\n\n- The booking is currently set for **2027-03-06**.\n- Meera explicitly stated: *“I no longer want 2027-04-06 for VISIT-2105.”*\n- She also stated: *“I have not decided on a replacement date.”*\n\nTherefore, no replacement date has been chosen yet. If you would like to change the booking date, that would require a new request and appropriate approval.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-06", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The customer withdraws the former date for VISIT-2105 and explicitly has not chosen a replacement. The answer is null with prospective clarification needed to choose a unique date; the other booking is unrelated.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-05-en-m0-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-05-en-m0-source-chronological", + "grouped_lossless": "confirm-fresh-05-en-m0-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-05-en-m1", + "case_id": "fresh-05-en", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "treated_actor_prose": true, + "original_question": "Which replacement date do I currently want for VISIT-2105? Read only; do not change the booking.", + "original_source_messages": [ + { + "id": "note-5-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-5-1", + "role": "customer", + "content": "I no longer want 2027-04-06 for VISIT-2105." + }, + { + "id": "note-5-2", + "role": "customer", + "content": "I have not decided on a replacement date. My other booking VISIT-3105 is unrelated." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-05", + "id": "VISIT-2105", + "date": "2027-03-06", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-05", + "id": "VISIT-3105", + "date": "2027-06-06", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "6de67458b42902d75b52d7a206f1c0a41ae2a0f83306dc77d152ec5101d806ec", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-9151c1682b7353f7", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2105\"}" + } + } + ] + }, + { + "content": "Based on the current state for **VISIT-2105**, the booking date is **2027-03-06** (version 5, not closed).\n\nRegarding your question about which replacement date you currently want: the source messages show that you previously said you no longer want **2027-04-06**, but you also stated that **you have not decided on a replacement date**. There is no approved or pending replacement date on record.\n\nSo, to answer precisely: **You currently have no replacement date decided or set for VISIT-2105.** The active booking remains **2027-03-06**.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9151c1682b7353f7", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2105" + }, + "response": { + "date": "2027-03-06", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-06", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The user withdraws 2027-04-06 and has not chosen a replacement for VISIT-2105; the unrelated booking cannot supply one.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-05-en-m1-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-05-en-m1-source-grouped_lossless", + "chronological": "confirm-fresh-05-en-m1-source-chronological" + } + }, + { + "id": "confirm-fresh-06-en-m0", + "case_id": "fresh-06-en", + "model": "sarvam-105b", + "family": "missing_replacement", + "treated_actor_prose": true, + "original_question": "Which replacement date do I currently want for VISIT-2106? Read only; do not change the booking.", + "original_source_messages": [ + { + "id": "note-6-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. I no longer want 2027-04-07 for VISIT-2106. I have not decided on a replacement date. My other booking VISIT-3106 is unrelated." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-06", + "id": "VISIT-2106", + "date": "2027-03-07", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-06", + "id": "VISIT-3106", + "date": "2027-06-07", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "82f5414c571b69dfc3d34c9485c1cf072e018bf96dc3d2d21443d169e6e9f8d0", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nBased on the customer’s latest statement, **no replacement date has been decided**. For VISIT-2106, the customer indicated they no longer want 2027-04-07 but **have not selected a new replacement date**.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-07", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The customer withdraws the former date for VISIT-2106 and explicitly has not chosen a replacement. The answer is null with prospective clarification needed to choose a unique date; the other booking is unrelated.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-06-en-m0-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-06-en-m0-source-grouped_lossless", + "chronological": "confirm-fresh-06-en-m0-source-chronological" + } + }, + { + "id": "confirm-fresh-06-en-m1", + "case_id": "fresh-06-en", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "treated_actor_prose": true, + "original_question": "Which replacement date do I currently want for VISIT-2106? Read only; do not change the booking.", + "original_source_messages": [ + { + "id": "note-6-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. I no longer want 2027-04-07 for VISIT-2106. I have not decided on a replacement date. My other booking VISIT-3106 is unrelated." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-06", + "id": "VISIT-2106", + "date": "2027-03-07", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-06", + "id": "VISIT-3106", + "date": "2027-06-07", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "ea9bef72f3bc630104f2b2836dccdf7dfd083c661e94e381c300957989d94bc2", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-ae8a05f3adadb4b5", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2106\"}" + } + } + ] + }, + { + "content": "Based on the current state for **VISIT-2106**:\n\n- **Current booking date:** 2027-03-07\n- **Closed:** false\n- **Version:** 5\n\nYou previously indicated you no longer want **2027-04-07** for this booking, and you have **not decided on a replacement date**. Therefore, there is **no replacement date currently selected** for VISIT-2106. The active date remains **2027-03-07**.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-ae8a05f3adadb4b5", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2106" + }, + "response": { + "date": "2027-03-07", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-07", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The source explicitly withdraws 2027-04-07 and says no replacement was chosen; the other booking is unrelated.", + "reporters": { + "chronological": "confirm-fresh-06-en-m1-source-chronological", + "grouped_lossless": "confirm-fresh-06-en-m1-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-06-en-m1-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-07-en-m0", + "case_id": "fresh-07-en", + "model": "sarvam-105b", + "family": "missing_replacement", + "treated_actor_prose": true, + "original_question": "Which replacement date do I currently want for VISIT-2107? Read only; do not change the booking.", + "original_source_messages": [ + { + "id": "note-7-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-7-1", + "role": "customer", + "content": "I no longer want 2027-04-08 for VISIT-2107." + }, + { + "id": "note-7-2", + "role": "customer", + "content": "I have not decided on a replacement date. My other booking VISIT-3107 is unrelated." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-07", + "id": "VISIT-2107", + "date": "2027-03-08", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-07", + "id": "VISIT-3107", + "date": "2027-06-08", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "efb755570c6d2534d5576e8a4a36af7937e80e86d75ebf5710cacfdf7159a746", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nBased on the evidence, the customer has **not decided on a replacement date** for VISIT-2107. In the most recent message (note-7-2), the customer explicitly states, “I have not decided on a replacement date.” No prior execution receipt or approval indicates a replacement date has been finalized.\n\n(Current booking date on file: **2027-03-08**.)", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-08", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The user withdraws 2027-04-08 and explicitly has not chosen a replacement; a unique current replacement date is absent.", + "reporters": { + "chronological": "confirm-fresh-07-en-m0-source-chronological", + "grouped_lossless": "confirm-fresh-07-en-m0-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-07-en-m0-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-07-en-m1", + "case_id": "fresh-07-en", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "treated_actor_prose": true, + "original_question": "Which replacement date do I currently want for VISIT-2107? Read only; do not change the booking.", + "original_source_messages": [ + { + "id": "note-7-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-7-1", + "role": "customer", + "content": "I no longer want 2027-04-08 for VISIT-2107." + }, + { + "id": "note-7-2", + "role": "customer", + "content": "I have not decided on a replacement date. My other booking VISIT-3107 is unrelated." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-07", + "id": "VISIT-2107", + "date": "2027-03-08", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-07", + "id": "VISIT-3107", + "date": "2027-06-08", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "91a6edffdd3656797b529da00bad42736d5a5a114414bac9680f61de795c7806", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-a8e16e59ed217164", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2107\"}" + } + } + ] + }, + { + "content": "Based on the current state read for **VISIT-2107**, the booking is **open** (not closed) with the current date **2027-03-08** and version 5.\n\nRegarding your question—**\"Which replacement date do I currently want?\"**—the source messages indicate:\n\n- You previously stated you **no longer want 2027-04-08** for this booking.\n- You stated you **have not decided on a replacement date**.\n\nTherefore, **you do not currently have a replacement date selected** for VISIT-2107. The booking remains at its current date of **2027-03-08**. No state change has been made, as this was a read-only request.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a8e16e59ed217164", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2107" + }, + "response": { + "date": "2027-03-08", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-08", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The customer withdraws the former date for VISIT-2107 and explicitly has not chosen a replacement. The answer is null with prospective clarification needed to choose a unique date; the other booking is unrelated.", + "reporters": { + "chronological": "confirm-fresh-07-en-m1-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-07-en-m1-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-07-en-m1-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-08-en-m0", + "case_id": "fresh-08-en", + "model": "sarvam-105b", + "family": "missing_replacement", + "treated_actor_prose": true, + "original_question": "Which replacement date do I currently want for VISIT-2108? Read only; do not change the booking.", + "original_source_messages": [ + { + "id": "note-8-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. I no longer want 2027-04-09 for VISIT-2108. I have not decided on a replacement date. My other booking VISIT-3108 is unrelated." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-08", + "id": "VISIT-2108", + "date": "2027-03-09", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-08", + "id": "VISIT-3108", + "date": "2027-06-09", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "389acdd678c40e89c3831072055a761fd7cc81cba3d26b52b30b836977a114e9", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-a254d14db1acb8b6", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2108\"}" + } + } + ] + }, + { + "content": "\nThe customer has **not decided on a replacement date yet**. \n\nFor **VISIT-2108**, the current (active) booking date remains **2027-03-09**. In the source message, the customer only states they *no longer want* **2027-04-09** and explicitly says, “I have not decided on a replacement date.” No replacement date has been formally requested, and no server approval exists for a change.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a254d14db1acb8b6", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2108" + }, + "response": { + "date": "2027-03-09", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-09", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The source explicitly withdraws 2027-04-09 and says no replacement was chosen; the current booking date 2027-03-09 is a different fact.", + "reporters": { + "chronological": "confirm-fresh-08-en-m0-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-08-en-m0-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-08-en-m0-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-08-en-m1", + "case_id": "fresh-08-en", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "treated_actor_prose": true, + "original_question": "Which replacement date do I currently want for VISIT-2108? Read only; do not change the booking.", + "original_source_messages": [ + { + "id": "note-8-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. I no longer want 2027-04-09 for VISIT-2108. I have not decided on a replacement date. My other booking VISIT-3108 is unrelated." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-08", + "id": "VISIT-2108", + "date": "2027-03-09", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-08", + "id": "VISIT-3108", + "date": "2027-06-09", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "42ebfcf03d238ef8e48a7edad5468687c727f0352c462795cfac542f236e720c", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-a1d80a57063d3cae", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2108\"}" + } + } + ] + }, + { + "content": "Based on the current state for **VISIT-2108**:\n\n- **Current date:** 2027-03-09\n- **Closed:** false\n- **Version:** 5\n\nThe customer has indicated they **no longer want 2027-04-09** and have **not decided on a replacement date**. Since this is a read-only request and no replacement date has been selected or approved, there is **no replacement date currently wanted** for VISIT-2108. The booking remains open with its existing date of 2027-03-09.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a1d80a57063d3cae", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2108" + }, + "response": { + "date": "2027-03-09", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-09", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The customer withdraws the former date for VISIT-2108 and explicitly has not chosen a replacement. The answer is null with prospective clarification needed to choose a unique date; the other booking is unrelated.", + "reporters": { + "grouped_lossless": "confirm-fresh-08-en-m1-source-grouped_lossless", + "chronological": "confirm-fresh-08-en-m1-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-08-en-m1-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-09-en-m0", + "case_id": "fresh-09-en", + "model": "sarvam-105b", + "family": "resolved_condition", + "treated_actor_prose": true, + "original_question": "From the stated condition, which date should I choose for VISIT-2109? Do not change it.", + "original_source_messages": [ + { + "id": "note-9-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-9-1", + "role": "customer", + "content": "For VISIT-2109, choose 2027-04-10 if a morning slot is available; otherwise choose 2027-05-10." + }, + { + "id": "note-9-2", + "role": "customer", + "content": "The morning slot is available. I am only asking about my choice, not authorizing a booking change." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-09", + "id": "VISIT-2109", + "date": "2027-03-10", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-09", + "id": "VISIT-3109", + "date": "2027-06-10", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "d46a240339f6cd402a0ab449ae295bb4e94e7772a4bd15e3099c133983f9fbe2", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nBased on the stated condition in your message, you should choose **2027-04-10**, since you confirmed that the morning slot is available.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-10", + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The morning slot is explicitly available, selecting 2027-04-10 rather than the fallback 2027-05-10; current booking date is a separate fact.", + "reporters": { + "grouped_lossless": "confirm-fresh-09-en-m0-source-grouped_lossless", + "chronological": "confirm-fresh-09-en-m0-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-09-en-m0-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-09-en-m1", + "case_id": "fresh-09-en", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "treated_actor_prose": true, + "original_question": "From the stated condition, which date should I choose for VISIT-2109? Do not change it.", + "original_source_messages": [ + { + "id": "note-9-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-9-1", + "role": "customer", + "content": "For VISIT-2109, choose 2027-04-10 if a morning slot is available; otherwise choose 2027-05-10." + }, + { + "id": "note-9-2", + "role": "customer", + "content": "The morning slot is available. I am only asking about my choice, not authorizing a booking change." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-09", + "id": "VISIT-2109", + "date": "2027-03-10", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-09", + "id": "VISIT-3109", + "date": "2027-06-10", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "abee0fd1852a6bbab5bdf36bd899d7756f67bab2742ea071e91196eae361710d", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-a5864e0dd15eeb9e", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2109\"}" + } + } + ] + }, + { + "content": "Based on the stated condition in your messages, you should choose **2027-04-10** for VISIT-2109.\n\nYour own note (note-9-1) says: *\"For VISIT-2109, choose 2027-04-10 if a morning slot is available; otherwise choose 2027-05-10.\"* Your subsequent note (note-9-2) confirms: *\"The morning slot is available.\"* Therefore, the 2024-04-10 option applies, and you do not need to fall back to 2027-05-10.\n\nI have not changed anything; this is only a read-only answer to your question. The current booking remains dated **2027-03-10** with version 5 and is still open.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a5864e0dd15eeb9e", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2109" + }, + "response": { + "date": "2027-03-10", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-10", + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The morning slot is explicitly available, selecting 2027-04-10; read_current 2027-03-10 does not change that choice.", + "reporters": { + "grouped_lossless": "confirm-fresh-09-en-m1-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-09-en-m1-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-09-en-m1-source-chronological" + } + }, + { + "id": "confirm-fresh-10-en-m0", + "case_id": "fresh-10-en", + "model": "sarvam-105b", + "family": "resolved_condition", + "treated_actor_prose": true, + "original_question": "From the stated condition, which date should I choose for VISIT-2110? Do not change it.", + "original_source_messages": [ + { + "id": "note-10-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. For VISIT-2110, choose 2027-04-11 if a morning slot is available; otherwise choose 2027-05-11. The morning slot is not available. I am asking about my choice only." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-10", + "id": "VISIT-2110", + "date": "2027-03-11", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-10", + "id": "VISIT-3110", + "date": "2027-06-11", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "45505a6bc992094c2ff2030030ff546801ab7c58143fe5abfee927e25c5b00c1", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nBased on your stated condition—choose **2027-04-11** if a morning slot is available, otherwise choose **2027-05-11**—and since the morning slot is not available, you should choose **2027-05-11**. No state changes are being made.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-11", + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The stated availability condition selects the applicable branch for VISIT-2110; the source resolves the choice without a booking write. Branch dates in order: ['2027-04-11', '2027-05-11'].", + "reporters": { + "grouped_lossless": "confirm-fresh-10-en-m0-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-10-en-m0-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-10-en-m0-source-chronological" + } + }, + { + "id": "confirm-fresh-10-en-m1", + "case_id": "fresh-10-en", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "treated_actor_prose": true, + "original_question": "From the stated condition, which date should I choose for VISIT-2110? Do not change it.", + "original_source_messages": [ + { + "id": "note-10-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. For VISIT-2110, choose 2027-04-11 if a morning slot is available; otherwise choose 2027-05-11. The morning slot is not available. I am asking about my choice only." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-10", + "id": "VISIT-2110", + "date": "2027-03-11", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-10", + "id": "VISIT-3110", + "date": "2027-06-11", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "1e43b51548110a588474f65b6b243b3d550dc5a16a58bffd19b9d586bb6c7ca3", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-8d45f64d3ef7e31e", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2110\"}" + } + } + ] + }, + { + "content": "Based on the stated condition in the source message, you should choose **2027-05-11**.\n\nThe customer specified: choose 2027-04-11 if a morning slot is available; otherwise choose 2027-05-11. The message explicitly states that the morning slot is not available, so the fallback date of 2027-05-11 applies.\n\nI have not changed the booking. The current state remains **VISIT-2110** with date **2027-03-11** and version **5**.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8d45f64d3ef7e31e", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2110" + }, + "response": { + "date": "2027-03-11", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-11", + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The stated availability condition selects the applicable branch for VISIT-2110; the source resolves the choice without a booking write. Branch dates in order: ['2027-04-11', '2027-05-11'].", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-10-en-m1-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-10-en-m1-source-chronological", + "grouped_lossless": "confirm-fresh-10-en-m1-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-11-en-m0", + "case_id": "fresh-11-en", + "model": "sarvam-105b", + "family": "resolved_condition", + "treated_actor_prose": true, + "original_question": "From the stated condition, which date should I choose for VISIT-2111? Do not change it.", + "original_source_messages": [ + { + "id": "note-11-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-11-1", + "role": "customer", + "content": "For VISIT-2111, choose 2027-04-12 if a morning slot is available; otherwise choose 2027-05-12." + }, + { + "id": "note-11-2", + "role": "customer", + "content": "The morning slot is available. I am only asking about my choice, not authorizing a booking change." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-11", + "id": "VISIT-2111", + "date": "2027-03-12", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-11", + "id": "VISIT-3111", + "date": "2027-06-12", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "f26b15c7e7df53d70084fa5acf01a41f8d293fa957b1890efd52b067222e1484", + "original_actor_terminal": "incomplete", + "actor_claims": [ + { + "content": "\nBased on the condition you stated, since the morning slot is available, the date you should choose for **VISIT-2111** is:\n\n**2027-04-12**\n\nThe booking itself has not been changed.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-12", + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The customer states the morning slot is available, so the first branch 2027-04-12 is determined; the current booking date 2027-03-12 is irrelevant to the stated-choice question.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-11-en-m0-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-11-en-m0-source-chronological", + "grouped_lossless": "confirm-fresh-11-en-m0-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-11-en-m1", + "case_id": "fresh-11-en", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "treated_actor_prose": true, + "original_question": "From the stated condition, which date should I choose for VISIT-2111? Do not change it.", + "original_source_messages": [ + { + "id": "note-11-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-11-1", + "role": "customer", + "content": "For VISIT-2111, choose 2027-04-12 if a morning slot is available; otherwise choose 2027-05-12." + }, + { + "id": "note-11-2", + "role": "customer", + "content": "The morning slot is available. I am only asking about my choice, not authorizing a booking change." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-11", + "id": "VISIT-2111", + "date": "2027-03-12", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-11", + "id": "VISIT-3111", + "date": "2027-06-12", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "17d09767fb0043179e9e978c50230f436a0517c5e0731f2582063a3cc89f7e80", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-b0b682de6f6b3998", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2111\"}" + } + } + ] + }, + { + "content": "Based on the stated condition in the source messages, you should choose **2027-04-12** for VISIT-2111.\n\nThe customer’s instruction was: *“choose 2027-04-12 if a morning slot is available; otherwise choose 2027-05-12.”* The customer then confirmed that the morning slot is available. Therefore, the correct date to select is 2027-04-12.\n\nI have not changed the booking, per your request. The current booking remains **2027-03-12** (version 5, open).", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-b0b682de6f6b3998", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2111" + }, + "response": { + "date": "2027-03-12", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-12", + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The stated availability condition selects the applicable branch for VISIT-2111; the source resolves the choice without a booking write. Branch dates in order: ['2027-04-12', '2027-05-12'].", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-11-en-m1-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-11-en-m1-source-grouped_lossless", + "chronological": "confirm-fresh-11-en-m1-source-chronological" + } + }, + { + "id": "confirm-fresh-12-en-m0", + "case_id": "fresh-12-en", + "model": "sarvam-105b", + "family": "resolved_condition", + "treated_actor_prose": true, + "original_question": "From the stated condition, which date should I choose for VISIT-2112? Do not change it.", + "original_source_messages": [ + { + "id": "note-12-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. For VISIT-2112, choose 2027-04-13 if a morning slot is available; otherwise choose 2027-05-13. The morning slot is not available. I am asking about my choice only." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-12", + "id": "VISIT-2112", + "date": "2027-03-13", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-12", + "id": "VISIT-3112", + "date": "2027-06-13", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "791e9e59e3c5adc42cd5a5fc70dd49ec22d24b7289afbbe6f85487d83fb0be9e", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nBased on the stated condition, you should choose **2027-05-13** for VISIT-2112, because the morning slot for 2027-04-13 is not available.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-13", + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The stated availability condition selects the applicable branch for VISIT-2112; the source resolves the choice without a booking write. Branch dates in order: ['2027-04-13', '2027-05-13'].", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-12-en-m0-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-12-en-m0-source-grouped_lossless", + "chronological": "confirm-fresh-12-en-m0-source-chronological" + } + }, + { + "id": "confirm-fresh-12-en-m1", + "case_id": "fresh-12-en", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "treated_actor_prose": true, + "original_question": "From the stated condition, which date should I choose for VISIT-2112? Do not change it.", + "original_source_messages": [ + { + "id": "note-12-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. For VISIT-2112, choose 2027-04-13 if a morning slot is available; otherwise choose 2027-05-13. The morning slot is not available. I am asking about my choice only." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-12", + "id": "VISIT-2112", + "date": "2027-03-13", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-12", + "id": "VISIT-3112", + "date": "2027-06-13", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "565d9941f41642e3ab4809d4fb921e376ffc9484b98417e3f2970b17774da903", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-a614e5f81a804c78", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2112\"}" + } + } + ] + }, + { + "content": "Based on the stated condition, you should choose **2027-05-13** for VISIT-2112.\n\nThe customer's instruction was: choose 2027-04-13 if a morning slot is available; otherwise choose 2027-05-13. The source message explicitly states that the morning slot is not available, so the fallback date of 2027-05-13 applies. No state change has been made.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a614e5f81a804c78", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2112" + }, + "response": { + "date": "2027-03-13", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-13", + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The customer states the morning slot is unavailable, so the fallback 2027-05-13 is determined; read_current returns the distinct current booking date 2027-03-13.", + "reporters": { + "chronological": "confirm-fresh-12-en-m1-source-chronological", + "grouped_lossless": "confirm-fresh-12-en-m1-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-12-en-m1-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-13-en-m0", + "case_id": "fresh-13-en", + "model": "sarvam-105b", + "family": "unresolved_condition", + "treated_actor_prose": false, + "original_question": "From the stated condition, which date should I choose for VISIT-2113? Do not change it.", + "original_source_messages": [ + { + "id": "note-13-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-13-1", + "role": "customer", + "content": "For VISIT-2113, I would choose 2027-04-14 if the morning slot is available, or 2027-05-14 if it is not." + }, + { + "id": "note-13-2", + "role": "customer", + "content": "Nobody has checked the availability yet. Do not guess or change my booking." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-13", + "id": "VISIT-2113", + "date": "2027-03-14", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-13", + "id": "VISIT-3113", + "date": "2027-06-14", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "08b9ca6b1459174ff91bebe6608f7ed7c5458dd096e6e782c94cdf76d142f577", + "original_actor_terminal": "invalid_action_response", + "actor_claims": [], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-14", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The two conditional dates for VISIT-2113 remain alternatives because availability was not checked. The answer is null with prospective clarification/condition resolution required; current backend date is not a branch choice.", + "reporters": { + "chronological": "confirm-fresh-13-en-m0-source-chronological", + "grouped_lossless": "confirm-fresh-13-en-m0-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-13-en-m0-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-13-en-m1", + "case_id": "fresh-13-en", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "treated_actor_prose": true, + "original_question": "From the stated condition, which date should I choose for VISIT-2113? Do not change it.", + "original_source_messages": [ + { + "id": "note-13-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-13-1", + "role": "customer", + "content": "For VISIT-2113, I would choose 2027-04-14 if the morning slot is available, or 2027-05-14 if it is not." + }, + { + "id": "note-13-2", + "role": "customer", + "content": "Nobody has checked the availability yet. Do not guess or change my booking." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-13", + "id": "VISIT-2113", + "date": "2027-03-14", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-13", + "id": "VISIT-3113", + "date": "2027-06-14", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "2aa30c4cfc92149eadca407daced7acd8fbf7c7c1340c42c54c72521adffcb0e", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-9a6e5af51f15efd1", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2113\"}" + } + } + ] + }, + { + "content": "Based on the current state for **VISIT-2113**, the booking date is **2027-03-14** (version 5, not closed).\n\nThe customer’s stated preferences from the source messages are:\n- **2027-04-14** if the morning slot is available, or\n- **2027-05-14** if it is not.\n\nHowever, the customer explicitly instructed: *“Nobody has checked the availability yet. Do not guess or change my booking.”* There are **no server approvals** for any date change, and the intent is **read-only**.\n\nTherefore, **do not change the booking**. The date to choose is the currently recorded one: **2027-03-14**. If you wish to act on the customer’s preferences, availability must be checked and a matching approval granted first.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9a6e5af51f15efd1", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2113" + }, + "response": { + "date": "2027-03-14", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-14", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The user has not checked morning availability, so neither 2027-04-14 nor 2027-05-14 is uniquely selected. The current booked date 2027-03-14 is unrelated to the conditional choice.", + "reporters": { + "chronological": "confirm-fresh-13-en-m1-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-13-en-m1-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-13-en-m1-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-14-en-m0", + "case_id": "fresh-14-en", + "model": "sarvam-105b", + "family": "unresolved_condition", + "treated_actor_prose": true, + "original_question": "From the stated condition, which date should I choose for VISIT-2114? Do not change it.", + "original_source_messages": [ + { + "id": "note-14-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. For VISIT-2114, I would choose 2027-04-15 if the morning slot is available, or 2027-05-15 if it is not. Nobody has checked the availability yet. Do not guess or change my booking." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-14", + "id": "VISIT-2114", + "date": "2027-03-15", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-14", + "id": "VISIT-3114", + "date": "2027-06-15", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "8fdb1cafd70a43721a69e7f289590b13cb803f3b279264c7f87e942a93af76e3", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nBased on Arjun’s stated condition in source message **note-14-0**, the date to choose for **VISIT-2114** should be:\n\n- **2027-04-15** — if the morning slot is available, or \n- **2027-05-15** — if the morning slot is not available.\n\nThe submitted evidence confirms that **no availability has been checked yet**, so the specific condition cannot be resolved to a single definitively chosen date at this time. The current confirmed booking state for VISIT-2114 is **2027-03-15** (version 5), which remains unchanged.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-15", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The morning availability remains unknown, so the stated conditional preference does not determine one date.", + "reporters": { + "chronological": "confirm-fresh-14-en-m0-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-14-en-m0-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-14-en-m0-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-14-en-m1", + "case_id": "fresh-14-en", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "treated_actor_prose": true, + "original_question": "From the stated condition, which date should I choose for VISIT-2114? Do not change it.", + "original_source_messages": [ + { + "id": "note-14-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. For VISIT-2114, I would choose 2027-04-15 if the morning slot is available, or 2027-05-15 if it is not. Nobody has checked the availability yet. Do not guess or change my booking." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-14", + "id": "VISIT-2114", + "date": "2027-03-15", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-14", + "id": "VISIT-3114", + "date": "2027-06-15", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "d0aae108a82043a6071af5dc8b1e3e92eddb7bfcafd7121179fa1c804c01fdcd", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-98493d6197b15ff1", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2114\"}" + } + } + ] + }, + { + "content": "Based on the stated condition in your message, you should choose **2027-04-15** if the morning slot is available, or **2027-05-15** if it is not.\n\nThe current booking for **VISIT-2114** is **2027-03-15** (version 5), and the booking is open. No change has been made, as your intent is read-only.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-98493d6197b15ff1", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2114" + }, + "response": { + "date": "2027-03-15", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-15", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The morning-slot condition has two dates, and availability has not been checked; read_current booking date does not resolve availability.", + "reporters": { + "grouped_lossless": "confirm-fresh-14-en-m1-source-grouped_lossless", + "chronological": "confirm-fresh-14-en-m1-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-14-en-m1-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-15-en-m0", + "case_id": "fresh-15-en", + "model": "sarvam-105b", + "family": "unresolved_condition", + "treated_actor_prose": true, + "original_question": "From the stated condition, which date should I choose for VISIT-2115? Do not change it.", + "original_source_messages": [ + { + "id": "note-15-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-15-1", + "role": "customer", + "content": "For VISIT-2115, I would choose 2027-04-16 if the morning slot is available, or 2027-05-16 if it is not." + }, + { + "id": "note-15-2", + "role": "customer", + "content": "Nobody has checked the availability yet. Do not guess or change my booking." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-15", + "id": "VISIT-2115", + "date": "2027-03-16", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-15", + "id": "VISIT-3115", + "date": "2027-06-16", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "ed64fdb8bfc3cc806e488231772bd7ccd9f19a79231817bc728f3e62fef72837", + "original_actor_terminal": "incomplete", + "actor_claims": [ + { + "content": "\nAccording to the stated condition, you should choose **2027-04-16** if the morning slot is available; otherwise, choose **2027-05-16**.\n\nSince the sources indicate that “Nobody has checked the availability yet,” the final choice cannot be confirmed at this time. The existing booking for **VISIT-2115** remains set for **2027-03-16**, and it should not be changed until availability is verified.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-16", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The two conditional dates for VISIT-2115 remain alternatives because availability was not checked. The answer is null with prospective clarification/condition resolution required; current backend date is not a branch choice.", + "reporters": { + "grouped_lossless": "confirm-fresh-15-en-m0-source-grouped_lossless", + "chronological": "confirm-fresh-15-en-m0-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-15-en-m0-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-15-en-m1", + "case_id": "fresh-15-en", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "treated_actor_prose": true, + "original_question": "From the stated condition, which date should I choose for VISIT-2115? Do not change it.", + "original_source_messages": [ + { + "id": "note-15-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-15-1", + "role": "customer", + "content": "For VISIT-2115, I would choose 2027-04-16 if the morning slot is available, or 2027-05-16 if it is not." + }, + { + "id": "note-15-2", + "role": "customer", + "content": "Nobody has checked the availability yet. Do not guess or change my booking." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-15", + "id": "VISIT-2115", + "date": "2027-03-16", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-15", + "id": "VISIT-3115", + "date": "2027-06-16", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "0aaf42e91e796b2f5397399e442665d0dbfba256693fd974785bae8d3445a4f3", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-91d41f5950b4567f", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2115\"}" + } + } + ] + }, + { + "content": "Based on the current state returned by the server, the existing booking for **VISIT-2115** is dated **2027-03-16** (version 5, not closed).\n\nThe customer’s stated preferences are:\n- 2027-04-16 if the morning slot is available, or\n- 2027-05-16 if it is not.\n\nHowever, the customer also explicitly instructed: *“Nobody has checked the availability yet. Do not guess or change my booking.”*\n\nSince your intent is **read-only** and no server approval for a date change has been granted, I must not alter the state. Therefore, the date you should choose for VISIT-2115 is the one currently on record: **2027-03-16**.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-91d41f5950b4567f", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2115" + }, + "response": { + "date": "2027-03-16", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-16", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The two conditional dates for VISIT-2115 remain alternatives because availability was not checked. The answer is null with prospective clarification/condition resolution required; current backend date is not a branch choice.", + "reporters": { + "grouped_lossless": "confirm-fresh-15-en-m1-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-15-en-m1-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-15-en-m1-source-chronological" + } + }, + { + "id": "confirm-fresh-16-en-m0", + "case_id": "fresh-16-en", + "model": "sarvam-105b", + "family": "unresolved_condition", + "treated_actor_prose": true, + "original_question": "From the stated condition, which date should I choose for VISIT-2116? Do not change it.", + "original_source_messages": [ + { + "id": "note-16-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. For VISIT-2116, I would choose 2027-04-17 if the morning slot is available, or 2027-05-17 if it is not. Nobody has checked the availability yet. Do not guess or change my booking." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-16", + "id": "VISIT-2116", + "date": "2027-03-17", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-16", + "id": "VISIT-3116", + "date": "2027-06-17", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "5fb3f8416cfbaa62232ddbdc031cbcda3890ec174dc7fdaf76876bbe51f92884", + "original_actor_terminal": "incomplete", + "actor_claims": [ + { + "content": "\nAccording to the condition you provided for **VISIT-2116**, the date to choose is:\n\n- **2027-04-17** if the morning slot is available \n- **2027-05-17** if the morning slot is not available \n\nSince nobody has checked availability yet, the definitive date cannot be selected until that is confirmed. I will not change the booking.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-17", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "source_interpretation": "Morning availability remains unchecked; no unique choice between 2027-04-17 and 2027-05-17 is established.", + "reporters": { + "grouped_lossless": "confirm-fresh-16-en-m0-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-16-en-m0-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-16-en-m0-source-chronological" + } + }, + { + "id": "confirm-fresh-16-en-m1", + "case_id": "fresh-16-en", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "treated_actor_prose": true, + "original_question": "From the stated condition, which date should I choose for VISIT-2116? Do not change it.", + "original_source_messages": [ + { + "id": "note-16-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. For VISIT-2116, I would choose 2027-04-17 if the morning slot is available, or 2027-05-17 if it is not. Nobody has checked the availability yet. Do not guess or change my booking." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-16", + "id": "VISIT-2116", + "date": "2027-03-17", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-16", + "id": "VISIT-3116", + "date": "2027-06-17", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "29d79d4d34e3dbfc7669ad0a92cdf83eabefc80e5ee2269bc81959c54592aeaa", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-a07e51609fa9a1f3", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2116\"}" + } + } + ] + }, + { + "content": "The current state for **VISIT-2116** is:\n- **Date:** 2027-03-17\n- **Version:** 5\n- **Closed:** false\n\nThe customer (Ravi) stated a preference: choose **2027-04-17** if the morning slot is available, otherwise **2027-05-17**. However, nobody has checked availability yet, and there are **no server approvals** on record for any date change.\n\nSince your intent is **read-only** and you asked **not to change** the booking, I will not modify anything.\n\n**Answer to your question:** Based on the stated condition, the recommended choice is **2027-04-17** (the first option Ravi specified, pending morning-slot availability). If that slot is unavailable, the fallback would be **2027-05-17**. But as of now, the booking remains at **2027-03-17** with no approved change applied.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a07e51609fa9a1f3", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2116" + }, + "response": { + "date": "2027-03-17", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-17", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "source_interpretation": "Morning availability remains unchecked. read_current 2027-03-17 does not resolve the condition.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-16-en-m1-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-16-en-m1-source-chronological", + "grouped_lossless": "confirm-fresh-16-en-m1-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-17-en-m0", + "case_id": "fresh-17-en", + "model": "sarvam-105b", + "family": "separate_booking", + "treated_actor_prose": true, + "original_question": "What is my current preferred date for VISIT-2117? Read only; do not change the booking.", + "original_source_messages": [ + { + "id": "note-17-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-17-1", + "role": "customer", + "content": "My preferred date for VISIT-2117 is 2027-04-18. For VISIT-3117, it was 2027-03-18." + }, + { + "id": "note-17-2", + "role": "customer", + "content": "Correction only for VISIT-3117: I now want 2027-05-18. The first booking's preference is unchanged." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-17", + "id": "VISIT-2117", + "date": "2027-03-18", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-17", + "id": "VISIT-3117", + "date": "2027-06-18", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "bcb79701178854c7073cba63c6c32cab02e3c3816ea148ad34eb1f118ba99460", + "original_actor_terminal": "incomplete", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-9a7c4a8b7c6dd728", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2117\"}" + } + } + ] + }, + { + "content": "\nYour current preferred date for **VISIT-2117** is **2027-03-18**.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9a7c4a8b7c6dd728", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2117" + }, + "response": { + "date": "2027-03-18", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-18", + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The customer preference for VISIT-2117 is the first booking date 2027-04-18; the later correction explicitly applies only to the other booking. Current backend booked date is a separate fact.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-17-en-m0-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-17-en-m0-source-chronological", + "grouped_lossless": "confirm-fresh-17-en-m0-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-17-en-m1", + "case_id": "fresh-17-en", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "treated_actor_prose": true, + "original_question": "What is my current preferred date for VISIT-2117? Read only; do not change the booking.", + "original_source_messages": [ + { + "id": "note-17-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-17-1", + "role": "customer", + "content": "My preferred date for VISIT-2117 is 2027-04-18. For VISIT-3117, it was 2027-03-18." + }, + { + "id": "note-17-2", + "role": "customer", + "content": "Correction only for VISIT-3117: I now want 2027-05-18. The first booking's preference is unchanged." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-17", + "id": "VISIT-2117", + "date": "2027-03-18", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-17", + "id": "VISIT-3117", + "date": "2027-06-18", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "3e60cb693925ea63e0317232878bdf01b697c2993858ac3c865341072ff529eb", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-9187e8ff022b2fab", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2117\"}" + } + } + ] + }, + { + "content": "Your current preferred date for **VISIT-2117** is **2027-03-18**.\n\nThis is read-only information; no changes have been made to the booking. The customer’s stated preference of 2027-04-18 in the source messages has not been applied, as there is no matching server approval and the intent is read-only.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9187e8ff022b2fab", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2117" + }, + "response": { + "date": "2027-03-18", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-18", + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The customer preference for VISIT-2117 is the first booking date 2027-04-18; the later correction explicitly applies only to the other booking. Current backend booked date is a separate fact.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-17-en-m1-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-17-en-m1-source-grouped_lossless", + "chronological": "confirm-fresh-17-en-m1-source-chronological" + } + }, + { + "id": "confirm-fresh-18-en-m0", + "case_id": "fresh-18-en", + "model": "sarvam-105b", + "family": "separate_booking", + "treated_actor_prose": true, + "original_question": "What is my current preferred date for VISIT-2118? Read only; do not change the booking.", + "original_source_messages": [ + { + "id": "note-18-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. My preferred date for VISIT-2118 is 2027-04-19. For VISIT-3118, it was 2027-03-19. Correction only for VISIT-3118: I now want 2027-05-19. The first booking's preference is unchanged." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-18", + "id": "VISIT-2118", + "date": "2027-03-19", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-18", + "id": "VISIT-3118", + "date": "2027-06-19", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "31586bf2d845e51f121c584a83869aca92e12866b373e548aee9dbf6161fdd8a", + "original_actor_terminal": "incomplete", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-96a015520f5a4088", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2118\"}" + } + } + ] + }, + { + "content": "\nYour current preferred date for **VISIT-2118** is **2027-03-19** (booking open, version 5). No changes were made, per your read-only request.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-96a015520f5a4088", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2118" + }, + "response": { + "date": "2027-03-19", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-19", + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "VISIT-2118 preferred date is 2027-04-19; the later correction applies only to VISIT-3118. read_current 2027-03-19 is a different fact.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-18-en-m0-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-18-en-m0-source-grouped_lossless", + "chronological": "confirm-fresh-18-en-m0-source-chronological" + } + }, + { + "id": "confirm-fresh-18-en-m1", + "case_id": "fresh-18-en", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "treated_actor_prose": true, + "original_question": "What is my current preferred date for VISIT-2118? Read only; do not change the booking.", + "original_source_messages": [ + { + "id": "note-18-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. My preferred date for VISIT-2118 is 2027-04-19. For VISIT-3118, it was 2027-03-19. Correction only for VISIT-3118: I now want 2027-05-19. The first booking's preference is unchanged." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-18", + "id": "VISIT-2118", + "date": "2027-03-19", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-18", + "id": "VISIT-3118", + "date": "2027-06-19", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "16f1865483ac5af053cd14a7aa60feccdf21bbb306626a331681e924fb40ba87", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-bb213132d72ebdca", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2118\"}" + } + } + ] + }, + { + "content": "Your current preferred date for **VISIT-2118** is **2027-03-19**. The booking is open (not closed) and is at version 5. This is a read-only response; no changes have been made.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-bb213132d72ebdca", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2118" + }, + "response": { + "date": "2027-03-19", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-19", + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "VISIT-2118 preference is explicitly 2027-04-19. The later correction applies only to VISIT-3118, and the 2027-03-19 backend date is not the asked preference.", + "reporters": { + "chronological": "confirm-fresh-18-en-m1-source-chronological", + "grouped_lossless": "confirm-fresh-18-en-m1-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-18-en-m1-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-19-en-m0", + "case_id": "fresh-19-en", + "model": "sarvam-105b", + "family": "separate_booking", + "treated_actor_prose": true, + "original_question": "What is my current preferred date for VISIT-2119? Read only; do not change the booking.", + "original_source_messages": [ + { + "id": "note-19-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-19-1", + "role": "customer", + "content": "My preferred date for VISIT-2119 is 2027-04-20. For VISIT-3119, it was 2027-03-20." + }, + { + "id": "note-19-2", + "role": "customer", + "content": "Correction only for VISIT-3119: I now want 2027-05-20. The first booking's preference is unchanged." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-19", + "id": "VISIT-2119", + "date": "2027-03-20", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-19", + "id": "VISIT-3119", + "date": "2027-06-20", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "8157342cc3a26eaee4f6dbf9230aa41a77d7e0386d4786edec6eab13930bb182", + "original_actor_terminal": "incomplete", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-95c1f057efa82fdf", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2119\"}" + } + } + ] + }, + { + "content": "\nYour current confirmed preferred date for **VISIT-2119** is **2027-03-20** (booking is open, version 5). The trusted application state records this as the active date; the earlier request of 2027-04-20 has not yet been applied. No changes were made.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-95c1f057efa82fdf", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2119" + }, + "response": { + "date": "2027-03-20", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-20", + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The source explicitly sets VISIT-2119 preference to 2027-04-20; the later correction concerns only VISIT-3119. The read_current date is not the asked preference.", + "reporters": { + "chronological": "confirm-fresh-19-en-m0-source-chronological", + "grouped_lossless": "confirm-fresh-19-en-m0-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-19-en-m0-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-19-en-m1", + "case_id": "fresh-19-en", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "treated_actor_prose": true, + "original_question": "What is my current preferred date for VISIT-2119? Read only; do not change the booking.", + "original_source_messages": [ + { + "id": "note-19-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-19-1", + "role": "customer", + "content": "My preferred date for VISIT-2119 is 2027-04-20. For VISIT-3119, it was 2027-03-20." + }, + { + "id": "note-19-2", + "role": "customer", + "content": "Correction only for VISIT-3119: I now want 2027-05-20. The first booking's preference is unchanged." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-19", + "id": "VISIT-2119", + "date": "2027-03-20", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-19", + "id": "VISIT-3119", + "date": "2027-06-20", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "a7c7cde8f9a29fba168ba30ffd914428617149431514a2d25eedc826d41bb952", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-9344faa7903ae8ff", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2119\"}" + } + } + ] + }, + { + "content": "Your current preferred date for **VISIT-2119** is **2027-03-20** (booking is open, version 5).\n\nNote: The customer’s messages reference preferences for **VISIT-3119** (2027-03-20 and 2027-05-20), not VISIT-2119. For VISIT-2119, the only stated preference in the source messages is the earlier one of 2027-04-20, but the server’s current state records **2027-03-20** as the preferred date. Since this is a read-only request, no change has been made.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9344faa7903ae8ff", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2119" + }, + "response": { + "date": "2027-03-20", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-20", + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The customer preference for VISIT-2119 is the first booking date 2027-04-20; the later correction explicitly applies only to the other booking. Current backend booked date is a separate fact.", + "reporters": { + "chronological": "confirm-fresh-19-en-m1-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-19-en-m1-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-19-en-m1-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-20-en-m0", + "case_id": "fresh-20-en", + "model": "sarvam-105b", + "family": "separate_booking", + "treated_actor_prose": true, + "original_question": "What is my current preferred date for VISIT-2120? Read only; do not change the booking.", + "original_source_messages": [ + { + "id": "note-20-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. My preferred date for VISIT-2120 is 2027-04-21. For VISIT-3120, it was 2027-03-21. Correction only for VISIT-3120: I now want 2027-05-21. The first booking's preference is unchanged." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-20", + "id": "VISIT-2120", + "date": "2027-03-21", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-20", + "id": "VISIT-3120", + "date": "2027-06-21", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "39bce7516858de0e6c386dbbe5ef2667df5dda56bbe1911aa9a1d4fa8e094c72", + "original_actor_terminal": "incomplete", + "actor_claims": [ + { + "content": "\nThe current preferred date for **VISIT-2120** in the system is **2027-03-21**.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-21", + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "The customer preference for VISIT-2120 is the first booking date 2027-04-21; the later correction explicitly applies only to the other booking. Current backend booked date is a separate fact.", + "reporters": { + "chronological": "confirm-fresh-20-en-m0-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-20-en-m0-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-20-en-m0-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-20-en-m1", + "case_id": "fresh-20-en", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "treated_actor_prose": true, + "original_question": "What is my current preferred date for VISIT-2120? Read only; do not change the booking.", + "original_source_messages": [ + { + "id": "note-20-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. My preferred date for VISIT-2120 is 2027-04-21. For VISIT-3120, it was 2027-03-21. Correction only for VISIT-3120: I now want 2027-05-21. The first booking's preference is unchanged." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-20", + "id": "VISIT-2120", + "date": "2027-03-21", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-20", + "id": "VISIT-3120", + "date": "2027-06-21", + "version": 40, + "closed": false + } + ], + "approvals": [] + }, + "parent_paired_sha256": "0839fbddbb6618585535dffd0333cabd95942e2e81dc8562552b0dea574332c7", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-84a0be61e074786a", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2120\"}" + } + } + ] + }, + { + "content": "Your current preferred date for **VISIT-2120** is **2027-03-21** (booking is open, version 5).\n\nNote: The customer's source message stated a preference of 2027-04-21 for VISIT-2120, but the server-recorded current state shows 2027-03-21. Since you requested a read-only check and no approval or execution receipt supports changing the date, the booking remains at 2027-03-21.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-84a0be61e074786a", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2120" + }, + "response": { + "date": "2027-03-21", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-21", + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "source_interpretation": "VISIT-2120 preferred date stays 2027-04-21; the 2027-05-21 correction belongs only to VISIT-3120. read_current 2027-03-21 is not the preference.", + "reporters": { + "grouped_lossless": "confirm-fresh-20-en-m1-source-grouped_lossless", + "chronological": "confirm-fresh-20-en-m1-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-20-en-m1-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-21-en-m0", + "case_id": "fresh-21-en", + "model": "sarvam-105b", + "family": "approved_change", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2121. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-21-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-21-1", + "role": "customer", + "content": "Please move VISIT-2121 to 2027-04-22." + }, + { + "id": "note-21-2", + "role": "customer", + "content": "Use the supplied request identifier and the server's matching approval. Tell me what date is confirmed after handling the request." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-21", + "id": "VISIT-2121", + "date": "2027-03-22", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-21", + "id": "VISIT-3121", + "date": "2027-06-22", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-21", + "id": "AUTH-21-GRANT-16", + "appointment_id": "VISIT-2121", + "operation": "update_appointment", + "date": "2027-04-22", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "a8d53c8c9e64768142af75cc1ab9249b5367d242f3cf0b1a1631d7f2c2562af4", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-9859b50a2cc7df62", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2121\", \"approval_id\": \"AUTH-21-GRANT-16\", \"date\": \"2027-04-22\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-21-CHANGE-16\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9859b50a2cc7df62", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2121", + "approval_id": "AUTH-21-GRANT-16", + "date": "2027-04-22", + "expected_version": 5, + "idempotency_key": "REQUEST-21-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-22", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-22", + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "source_interpretation": "The trusted matching approval permits the supplied change for VISIT-2121; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date.", + "reporters": { + "grouped_lossless": "confirm-fresh-21-en-m0-source-grouped_lossless", + "chronological": "confirm-fresh-21-en-m0-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-21-en-m0-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-21-en-m1", + "case_id": "fresh-21-en", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2121. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-21-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-21-1", + "role": "customer", + "content": "Please move VISIT-2121 to 2027-04-22." + }, + { + "id": "note-21-2", + "role": "customer", + "content": "Use the supplied request identifier and the server's matching approval. Tell me what date is confirmed after handling the request." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-21", + "id": "VISIT-2121", + "date": "2027-03-22", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-21", + "id": "VISIT-3121", + "date": "2027-06-22", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-21", + "id": "AUTH-21-GRANT-16", + "appointment_id": "VISIT-2121", + "operation": "update_appointment", + "date": "2027-04-22", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "9bef4971299653337fe6b01b2157aad8e5dda67415da70f9424f04e9edcf2cc0", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-a70914d3300307c3", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2121\"}" + } + } + ] + }, + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-a994431818ad4a6b", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2121\", \"approval_id\": \"AUTH-21-GRANT-16\", \"date\": \"2027-04-22\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-21-CHANGE-16\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a70914d3300307c3", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2121" + }, + "response": { + "date": "2027-03-22", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-a994431818ad4a6b", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2121", + "approval_id": "AUTH-21-GRANT-16", + "date": "2027-04-22", + "expected_version": 5, + "idempotency_key": "REQUEST-21-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-22", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-22", + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "source_interpretation": "The trusted matching approval permits the supplied change for VISIT-2121; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date.", + "reporters": { + "grouped_lossless": "confirm-fresh-21-en-m1-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-21-en-m1-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-21-en-m1-source-chronological" + } + }, + { + "id": "confirm-fresh-22-en-m0", + "case_id": "fresh-22-en", + "model": "sarvam-105b", + "family": "approved_change", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2122. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-22-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. Please move VISIT-2122 to 2027-04-23. Use the supplied request identifier and the server's matching approval. Tell me what date is confirmed after handling the request." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-22", + "id": "VISIT-2122", + "date": "2027-03-23", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-22", + "id": "VISIT-3122", + "date": "2027-06-23", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-22", + "id": "AUTH-22-GRANT-16", + "appointment_id": "VISIT-2122", + "operation": "update_appointment", + "date": "2027-04-23", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "bd527c65b844626442ef647542b7c328c567e14da47ea1fbe3f7688348ddd3ad", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-8ec14d86abeccbe4", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2122\", \"approval_id\": \"AUTH-22-GRANT-16\", \"date\": \"2027-04-23\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-22-CHANGE-16\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8ec14d86abeccbe4", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2122", + "approval_id": "AUTH-22-GRANT-16", + "date": "2027-04-23", + "expected_version": 5, + "idempotency_key": "REQUEST-22-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-23", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-23", + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "source_interpretation": "The requested 2027-04-23 update has matching server approval and a confirmed tool receipt; scoped backend date changes from 2027-03-23 to 2027-04-23.", + "reporters": { + "grouped_lossless": "confirm-fresh-22-en-m0-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-22-en-m0-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-22-en-m0-source-chronological" + } + }, + { + "id": "confirm-fresh-22-en-m1", + "case_id": "fresh-22-en", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2122. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-22-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. Please move VISIT-2122 to 2027-04-23. Use the supplied request identifier and the server's matching approval. Tell me what date is confirmed after handling the request." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-22", + "id": "VISIT-2122", + "date": "2027-03-23", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-22", + "id": "VISIT-3122", + "date": "2027-06-23", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-22", + "id": "AUTH-22-GRANT-16", + "appointment_id": "VISIT-2122", + "operation": "update_appointment", + "date": "2027-04-23", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "b90ac8a977eb56c44b2bdcf2b792ed1a8c57e27266a5b4d4c7801b7003d45fad", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-ba0d2a4c6f0b0b9e", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2122\"}" + } + } + ] + }, + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-b68ae55b6cd86ec1", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2122\", \"approval_id\": \"AUTH-22-GRANT-16\", \"date\": \"2027-04-23\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-22-CHANGE-16\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-ba0d2a4c6f0b0b9e", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2122" + }, + "response": { + "date": "2027-03-23", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-b68ae55b6cd86ec1", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2122", + "approval_id": "AUTH-22-GRANT-16", + "date": "2027-04-23", + "expected_version": 5, + "idempotency_key": "REQUEST-22-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-23", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-23", + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "source_interpretation": "The trusted matching approval permits the supplied change for VISIT-2122; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-22-en-m1-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-22-en-m1-source-chronological", + "grouped_lossless": "confirm-fresh-22-en-m1-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-23-en-m0", + "case_id": "fresh-23-en", + "model": "sarvam-105b", + "family": "approved_change", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2123. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-23-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-23-1", + "role": "customer", + "content": "Please move VISIT-2123 to 2027-04-24." + }, + { + "id": "note-23-2", + "role": "customer", + "content": "Use the supplied request identifier and the server's matching approval. Tell me what date is confirmed after handling the request." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-23", + "id": "VISIT-2123", + "date": "2027-03-24", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-23", + "id": "VISIT-3123", + "date": "2027-06-24", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-23", + "id": "AUTH-23-GRANT-16", + "appointment_id": "VISIT-2123", + "operation": "update_appointment", + "date": "2027-04-24", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "a2e3a7e05bcc5815be671e871d1ad629792f26a5ebfa82281dbf3fa4df85cdfc", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-bcff946d991e8cfa", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2123\", \"approval_id\": \"AUTH-23-GRANT-16\", \"date\": \"2027-04-24\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-23-CHANGE-16\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-bcff946d991e8cfa", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2123", + "approval_id": "AUTH-23-GRANT-16", + "date": "2027-04-24", + "expected_version": 5, + "idempotency_key": "REQUEST-23-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-24", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-24", + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "source_interpretation": "The trusted matching approval permits the supplied change for VISIT-2123; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-23-en-m0-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-23-en-m0-source-chronological", + "grouped_lossless": "confirm-fresh-23-en-m0-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-23-en-m1", + "case_id": "fresh-23-en", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2123. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-23-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-23-1", + "role": "customer", + "content": "Please move VISIT-2123 to 2027-04-24." + }, + { + "id": "note-23-2", + "role": "customer", + "content": "Use the supplied request identifier and the server's matching approval. Tell me what date is confirmed after handling the request." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-23", + "id": "VISIT-2123", + "date": "2027-03-24", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-23", + "id": "VISIT-3123", + "date": "2027-06-24", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-23", + "id": "AUTH-23-GRANT-16", + "appointment_id": "VISIT-2123", + "operation": "update_appointment", + "date": "2027-04-24", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "1275867d410a9327cee5119e6bf3996c2859447424a6d18284d037bab3000c0c", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-8c65412f007e9ccf", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2123\"}" + } + } + ] + }, + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-b30c71f3d05ec45e", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2123\", \"approval_id\": \"AUTH-23-GRANT-16\", \"date\": \"2027-04-24\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-23-CHANGE-16\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8c65412f007e9ccf", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2123" + }, + "response": { + "date": "2027-03-24", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-b30c71f3d05ec45e", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2123", + "approval_id": "AUTH-23-GRANT-16", + "date": "2027-04-24", + "expected_version": 5, + "idempotency_key": "REQUEST-23-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-24", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-24", + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "source_interpretation": "A read_current of the old date precedes a confirmed approved update to 2027-04-24. The later receipt and final backend govern the current-date answer.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-23-en-m1-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-23-en-m1-source-grouped_lossless", + "chronological": "confirm-fresh-23-en-m1-source-chronological" + } + }, + { + "id": "confirm-fresh-24-en-m0", + "case_id": "fresh-24-en", + "model": "sarvam-105b", + "family": "approved_change", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2124. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-24-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. Please move VISIT-2124 to 2027-04-25. Use the supplied request identifier and the server's matching approval. Tell me what date is confirmed after handling the request." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-24", + "id": "VISIT-2124", + "date": "2027-03-25", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-24", + "id": "VISIT-3124", + "date": "2027-06-25", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-24", + "id": "AUTH-24-GRANT-16", + "appointment_id": "VISIT-2124", + "operation": "update_appointment", + "date": "2027-04-25", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "9694a63cad38fa21c292671766a2f3825b27c25467fae3a512a2b0cd5512e2f7", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-a00b079aee5b5024", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2124\", \"approval_id\": \"AUTH-24-GRANT-16\", \"date\": \"2027-04-25\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-24-CHANGE-16\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a00b079aee5b5024", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2124", + "approval_id": "AUTH-24-GRANT-16", + "date": "2027-04-25", + "expected_version": 5, + "idempotency_key": "REQUEST-24-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-25", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-25", + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "source_interpretation": "The trusted matching approval permits the supplied change for VISIT-2124; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-24-en-m0-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-24-en-m0-source-grouped_lossless", + "chronological": "confirm-fresh-24-en-m0-source-chronological" + } + }, + { + "id": "confirm-fresh-24-en-m1", + "case_id": "fresh-24-en", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2124. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-24-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. Please move VISIT-2124 to 2027-04-25. Use the supplied request identifier and the server's matching approval. Tell me what date is confirmed after handling the request." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-24", + "id": "VISIT-2124", + "date": "2027-03-25", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-24", + "id": "VISIT-3124", + "date": "2027-06-25", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-24", + "id": "AUTH-24-GRANT-16", + "appointment_id": "VISIT-2124", + "operation": "update_appointment", + "date": "2027-04-25", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "5aa801347bd329457f18c6d69a93aa83bcbffc17e4732072fd767fab33cc5085", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-b41cbb604452acf0", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2124\"}" + } + } + ] + }, + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-9375fb6bb8077ca3", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2124\", \"approval_id\": \"AUTH-24-GRANT-16\", \"date\": \"2027-04-25\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-24-CHANGE-16\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-b41cbb604452acf0", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2124" + }, + "response": { + "date": "2027-03-25", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-9375fb6bb8077ca3", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2124", + "approval_id": "AUTH-24-GRANT-16", + "date": "2027-04-25", + "expected_version": 5, + "idempotency_key": "REQUEST-24-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-25", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-25", + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "source_interpretation": "The trusted matching approval permits the supplied change for VISIT-2124; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date.", + "reporters": { + "chronological": "confirm-fresh-24-en-m1-source-chronological", + "grouped_lossless": "confirm-fresh-24-en-m1-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-24-en-m1-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-25-en-m0", + "case_id": "fresh-25-en", + "model": "sarvam-105b", + "family": "approval_boundary", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2125. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-25-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-25-1", + "role": "customer", + "content": "Please process the change of VISIT-2125 to 2027-04-26." + }, + { + "id": "note-25-2", + "role": "customer", + "content": "Evaluate the server approval at the supplied server time. It expires one second after that fixed time." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-25", + "id": "VISIT-2125", + "date": "2027-03-26", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-25", + "id": "VISIT-3125", + "date": "2027-06-26", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-25", + "id": "AUTH-25-GRANT-19", + "appointment_id": "VISIT-2125", + "operation": "update_appointment", + "date": "2027-04-26", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791748801, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "1d3406aeaeafcaaa103222011e9e03f92fa9c6b1aef8d45a62b1ca8ac1ba9db9", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-8d35cacd91fb5eec", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2125\", \"approval_id\": \"AUTH-25-GRANT-19\", \"date\": \"2027-04-26\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-25-CHANGE-19\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8d35cacd91fb5eec", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2125", + "approval_id": "AUTH-25-GRANT-19", + "date": "2027-04-26", + "expected_version": 5, + "idempotency_key": "REQUEST-25-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-26", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-26", + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "source_interpretation": "At the frozen server time the approval is still effective, and the update receipt confirms 2027-04-26; this is not a real-time expiry test.", + "reporters": { + "chronological": "confirm-fresh-25-en-m0-source-chronological", + "grouped_lossless": "confirm-fresh-25-en-m0-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-25-en-m0-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-25-en-m1", + "case_id": "fresh-25-en", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2125. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-25-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-25-1", + "role": "customer", + "content": "Please process the change of VISIT-2125 to 2027-04-26." + }, + { + "id": "note-25-2", + "role": "customer", + "content": "Evaluate the server approval at the supplied server time. It expires one second after that fixed time." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-25", + "id": "VISIT-2125", + "date": "2027-03-26", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-25", + "id": "VISIT-3125", + "date": "2027-06-26", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-25", + "id": "AUTH-25-GRANT-19", + "appointment_id": "VISIT-2125", + "operation": "update_appointment", + "date": "2027-04-26", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791748801, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "70e71fd9b38c6d2af918841ab6ca087f9f7dacbfe315e65314da2ef3e3e53297", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-bcd2d806ebec62f3", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2125\"}" + } + } + ] + }, + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-b5847f3c799b33b4", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2125\", \"approval_id\": \"AUTH-25-GRANT-19\", \"date\": \"2027-04-26\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-25-CHANGE-19\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-bcd2d806ebec62f3", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2125" + }, + "response": { + "date": "2027-03-26", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-b5847f3c799b33b4", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2125", + "approval_id": "AUTH-25-GRANT-19", + "date": "2027-04-26", + "expected_version": 5, + "idempotency_key": "REQUEST-25-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-26", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-26", + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "source_interpretation": "The trusted matching approval permits the supplied change for VISIT-2125; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date. The grant remains effective at the supplied fixed server time despite its near expiry.", + "reporters": { + "chronological": "confirm-fresh-25-en-m1-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-25-en-m1-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-25-en-m1-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-26-en-m0", + "case_id": "fresh-26-en", + "model": "sarvam-105b", + "family": "approval_boundary", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2126. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-26-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. Please process the change of VISIT-2126 to 2027-04-27. Evaluate the server approval at the supplied server time. It expires one second after that fixed time." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-26", + "id": "VISIT-2126", + "date": "2027-03-27", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-26", + "id": "VISIT-3126", + "date": "2027-06-27", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-26", + "id": "AUTH-26-GRANT-19", + "appointment_id": "VISIT-2126", + "operation": "update_appointment", + "date": "2027-04-27", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791748801, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "c771cbd30afb7dcd4b66105996b069e57adc04e36f2f23539371682c7d41f754", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-8ec0ee38858f7527", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2126\", \"approval_id\": \"AUTH-26-GRANT-19\", \"date\": \"2027-04-27\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-26-CHANGE-19\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8ec0ee38858f7527", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2126", + "approval_id": "AUTH-26-GRANT-19", + "date": "2027-04-27", + "expected_version": 5, + "idempotency_key": "REQUEST-26-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-27", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-27", + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "source_interpretation": "The trusted matching approval permits the supplied change for VISIT-2126; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date. The grant remains effective at the supplied fixed server time despite its near expiry.", + "reporters": { + "chronological": "confirm-fresh-26-en-m0-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-26-en-m0-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-26-en-m0-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-26-en-m1", + "case_id": "fresh-26-en", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2126. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-26-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. Please process the change of VISIT-2126 to 2027-04-27. Evaluate the server approval at the supplied server time. It expires one second after that fixed time." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-26", + "id": "VISIT-2126", + "date": "2027-03-27", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-26", + "id": "VISIT-3126", + "date": "2027-06-27", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-26", + "id": "AUTH-26-GRANT-19", + "appointment_id": "VISIT-2126", + "operation": "update_appointment", + "date": "2027-04-27", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791748801, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "332bbc69b5584440b296370f054abcf8a7eefdaa1f8a5bfc6a2c2cdaa72f70c4", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-98be99eb89f9627b", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2126\"}" + } + } + ] + }, + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-a52bd49d3a5d7727", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2126\", \"approval_id\": \"AUTH-26-GRANT-19\", \"date\": \"2027-04-27\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-26-CHANGE-19\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-98be99eb89f9627b", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2126" + }, + "response": { + "date": "2027-03-27", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-a52bd49d3a5d7727", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2126", + "approval_id": "AUTH-26-GRANT-19", + "date": "2027-04-27", + "expected_version": 5, + "idempotency_key": "REQUEST-26-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-27", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-27", + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "source_interpretation": "The trusted matching approval permits the supplied change for VISIT-2126; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date. The grant remains effective at the supplied fixed server time despite its near expiry.", + "reporters": { + "grouped_lossless": "confirm-fresh-26-en-m1-source-grouped_lossless", + "chronological": "confirm-fresh-26-en-m1-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-26-en-m1-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-27-en-m0", + "case_id": "fresh-27-en", + "model": "sarvam-105b", + "family": "approval_boundary", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2127. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-27-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-27-1", + "role": "customer", + "content": "Please process the change of VISIT-2127 to 2027-04-28." + }, + { + "id": "note-27-2", + "role": "customer", + "content": "Evaluate the server approval at the supplied server time. It expires one second after that fixed time." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-27", + "id": "VISIT-2127", + "date": "2027-03-28", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-27", + "id": "VISIT-3127", + "date": "2027-06-28", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-27", + "id": "AUTH-27-GRANT-19", + "appointment_id": "VISIT-2127", + "operation": "update_appointment", + "date": "2027-04-28", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791748801, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "d0af014d12735ff1f91327289b726ba14d84e5f5c2ae58319ae59c6a40ae1f69", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-a3dddeec7903171a", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2127\", \"approval_id\": \"AUTH-27-GRANT-19\", \"date\": \"2027-04-28\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-27-CHANGE-19\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a3dddeec7903171a", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2127", + "approval_id": "AUTH-27-GRANT-19", + "date": "2027-04-28", + "expected_version": 5, + "idempotency_key": "REQUEST-27-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-28", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-28", + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "source_interpretation": "The frozen server approval is effective at the supplied time; update receipt confirms 2027-04-28 and backend advances to that date.", + "reporters": { + "grouped_lossless": "confirm-fresh-27-en-m0-source-grouped_lossless", + "chronological": "confirm-fresh-27-en-m0-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-27-en-m0-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-27-en-m1", + "case_id": "fresh-27-en", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2127. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-27-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-27-1", + "role": "customer", + "content": "Please process the change of VISIT-2127 to 2027-04-28." + }, + { + "id": "note-27-2", + "role": "customer", + "content": "Evaluate the server approval at the supplied server time. It expires one second after that fixed time." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-27", + "id": "VISIT-2127", + "date": "2027-03-28", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-27", + "id": "VISIT-3127", + "date": "2027-06-28", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-27", + "id": "AUTH-27-GRANT-19", + "appointment_id": "VISIT-2127", + "operation": "update_appointment", + "date": "2027-04-28", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791748801, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "f3d6e03b48887ef14313a5c0360bf809c1bffaab7dd3d6b4e5ecb4b56c70fb07", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-a2b4e43da4498d4a", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2127\"}" + } + } + ] + }, + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-8b9e455358f77851", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2127\", \"approval_id\": \"AUTH-27-GRANT-19\", \"date\": \"2027-04-28\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-27-CHANGE-19\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a2b4e43da4498d4a", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2127" + }, + "response": { + "date": "2027-03-28", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-8b9e455358f77851", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2127", + "approval_id": "AUTH-27-GRANT-19", + "date": "2027-04-28", + "expected_version": 5, + "idempotency_key": "REQUEST-27-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-28", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-28", + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "source_interpretation": "The trusted matching approval permits the supplied change for VISIT-2127; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date. The grant remains effective at the supplied fixed server time despite its near expiry.", + "reporters": { + "grouped_lossless": "confirm-fresh-27-en-m1-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-27-en-m1-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-27-en-m1-source-chronological" + } + }, + { + "id": "confirm-fresh-28-en-m0", + "case_id": "fresh-28-en", + "model": "sarvam-105b", + "family": "approval_boundary", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2128. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-28-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. Please process the change of VISIT-2128 to 2027-04-29. Evaluate the server approval at the supplied server time. It expires one second after that fixed time." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-28", + "id": "VISIT-2128", + "date": "2027-03-29", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-28", + "id": "VISIT-3128", + "date": "2027-06-29", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-28", + "id": "AUTH-28-GRANT-19", + "appointment_id": "VISIT-2128", + "operation": "update_appointment", + "date": "2027-04-29", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791748801, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "1a866427f1dfd40272107f0c561df606b8ae9dfeca054e1d549443499e9d05c6", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-b42f02f36ba8b0ed", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2128\", \"approval_id\": \"AUTH-28-GRANT-19\", \"date\": \"2027-04-29\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-28-CHANGE-19\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-b42f02f36ba8b0ed", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2128", + "approval_id": "AUTH-28-GRANT-19", + "date": "2027-04-29", + "expected_version": 5, + "idempotency_key": "REQUEST-28-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-29", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-29", + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "source_interpretation": "The approval remains effective at frozen server time and the tool receipt confirms a new write to 2027-04-29, matching final backend.", + "reporters": { + "grouped_lossless": "confirm-fresh-28-en-m0-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-28-en-m0-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-28-en-m0-source-chronological" + } + }, + { + "id": "confirm-fresh-28-en-m1", + "case_id": "fresh-28-en", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2128. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-28-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. Please process the change of VISIT-2128 to 2027-04-29. Evaluate the server approval at the supplied server time. It expires one second after that fixed time." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-28", + "id": "VISIT-2128", + "date": "2027-03-29", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-28", + "id": "VISIT-3128", + "date": "2027-06-29", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-28", + "id": "AUTH-28-GRANT-19", + "appointment_id": "VISIT-2128", + "operation": "update_appointment", + "date": "2027-04-29", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791748801, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "42ac6652a04e5f675197f84f2160466e7bd7e28f8cac8c1fddedaf2632b513e3", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-b992afc45b39acef", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2128\"}" + } + } + ] + }, + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-92d635d320c0edac", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2128\", \"approval_id\": \"AUTH-28-GRANT-19\", \"date\": \"2027-04-29\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-28-CHANGE-19\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-b992afc45b39acef", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2128" + }, + "response": { + "date": "2027-03-29", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-92d635d320c0edac", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2128", + "approval_id": "AUTH-28-GRANT-19", + "date": "2027-04-29", + "expected_version": 5, + "idempotency_key": "REQUEST-28-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-29", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-29", + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "source_interpretation": "The trusted matching approval permits the supplied change for VISIT-2128; the shared actor received a confirmed update receipt, so the confirmed date is the final backend date. The grant remains effective at the supplied fixed server time despite its near expiry.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-28-en-m1-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-28-en-m1-source-chronological", + "grouped_lossless": "confirm-fresh-28-en-m1-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-29-en-m0", + "case_id": "fresh-29-en", + "model": "sarvam-105b", + "family": "future_approval", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2129. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-29-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-29-1", + "role": "customer", + "content": "I want VISIT-2129 changed to 2027-04-30 now." + }, + { + "id": "note-29-2", + "role": "customer", + "content": "The server grant takes effect one hour after the supplied server time. My request does not change its validity period." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-29", + "id": "VISIT-2129", + "date": "2027-03-30", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-29", + "id": "VISIT-3129", + "date": "2027-06-30", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-29", + "id": "AUTH-29-GRANT-20", + "appointment_id": "VISIT-2129", + "operation": "update_appointment", + "date": "2027-04-30", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791752400, + "expires_at": 1791756000, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "66821b07532d10d5ffbf361a56175898fb1d41d80d5aa42cff338ffe9ac2e0b9", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-9fb8271f47e877a0", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2129\"}" + } + } + ] + }, + { + "content": "\nThe currently confirmed date for appointment **VISIT-2129** is **2027-03-30**.\n\n**Reason:** The approval **AUTH-29-GRANT-20** is granted, but its `not_before` timestamp (1791752400) is later than the current server time (1791748800). Because the approval is not yet effective, the state has not been updated, and the confirmed date remains the original **2027-03-30** (version 5, open).", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9fb8271f47e877a0", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2129" + }, + "response": { + "date": "2027-03-30", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-30", + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "source_interpretation": "The matching server grant for VISIT-2129 is not effective at the supplied server time. The requested later date is not confirmed; retain the current backend date. A rejected attempted write, if any, is an actor error, not authorization.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-29-en-m0-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-29-en-m0-source-chronological", + "grouped_lossless": "confirm-fresh-29-en-m0-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-29-en-m1", + "case_id": "fresh-29-en", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2129. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-29-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-29-1", + "role": "customer", + "content": "I want VISIT-2129 changed to 2027-04-30 now." + }, + { + "id": "note-29-2", + "role": "customer", + "content": "The server grant takes effect one hour after the supplied server time. My request does not change its validity period." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-29", + "id": "VISIT-2129", + "date": "2027-03-30", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-29", + "id": "VISIT-3129", + "date": "2027-06-30", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-29", + "id": "AUTH-29-GRANT-20", + "appointment_id": "VISIT-2129", + "operation": "update_appointment", + "date": "2027-04-30", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791752400, + "expires_at": 1791756000, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "45fc351ae66321acf2134a761b237198011c40a5aca0b894a0c487d28201d884", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-a86b5fff32fef70e", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2129\"}" + } + } + ] + }, + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-a298804650c4bca9", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2129\", \"approval_id\": \"AUTH-29-GRANT-20\", \"date\": \"2027-04-30\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-29-CHANGE-20\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a86b5fff32fef70e", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2129" + }, + "response": { + "date": "2027-03-30", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-a298804650c4bca9", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2129", + "approval_id": "AUTH-29-GRANT-20", + "date": "2027-04-30", + "expected_version": 5, + "idempotency_key": "REQUEST-29-CHANGE-20" + }, + "response": { + "status": "rejected", + "reason": "approval_not_effective", + "date": "2027-03-30", + "version": 5 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-30", + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_execution_status": "rejected", + "source_interpretation": "The grant is future-effective at the frozen server time. The actor nevertheless attempts an update, which the backend rejects; current date stays 2027-03-30.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-29-en-m1-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-29-en-m1-source-grouped_lossless", + "chronological": "confirm-fresh-29-en-m1-source-chronological" + } + }, + { + "id": "confirm-fresh-30-en-m0", + "case_id": "fresh-30-en", + "model": "sarvam-105b", + "family": "future_approval", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2130. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-30-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. I want VISIT-2130 changed to 2027-05-01 now. The server grant takes effect one hour after the supplied server time. My request does not change its validity period." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-30", + "id": "VISIT-2130", + "date": "2027-03-31", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-30", + "id": "VISIT-3130", + "date": "2027-07-01", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-30", + "id": "AUTH-30-GRANT-20", + "appointment_id": "VISIT-2130", + "operation": "update_appointment", + "date": "2027-05-01", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791752400, + "expires_at": 1791756000, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "4927236eee7a4eb0e21204ab3c35bb614cabd333ccec11981405d8b51e024b2a", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nThe approval **AUTH-30-GRANT-20** is granted and unrevoked, but it is **not yet effective** because its `not_before` timestamp (1791752400) is one hour after the current server time (1791748800). Since the approval is still pending, the change cannot be applied.\n\nThe booking remains open and the currently confirmed date is:\n\n**2027-03-31**", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-31", + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "source_interpretation": "The grant is not effective until an hour after frozen server time; no update is attempted, and backend remains 2027-03-31.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-30-en-m0-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-30-en-m0-source-grouped_lossless", + "chronological": "confirm-fresh-30-en-m0-source-chronological" + } + }, + { + "id": "confirm-fresh-30-en-m1", + "case_id": "fresh-30-en", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2130. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-30-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. I want VISIT-2130 changed to 2027-05-01 now. The server grant takes effect one hour after the supplied server time. My request does not change its validity period." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-30", + "id": "VISIT-2130", + "date": "2027-03-31", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-30", + "id": "VISIT-3130", + "date": "2027-07-01", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-30", + "id": "AUTH-30-GRANT-20", + "appointment_id": "VISIT-2130", + "operation": "update_appointment", + "date": "2027-05-01", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791752400, + "expires_at": 1791756000, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "291f49aa5c1d1e3ea484f67af7581839f3ce6d336e034ffade0ac142e71a843e", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-9da2cbfb808c3017", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2130\"}" + } + } + ] + }, + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-832df9049e1ef051", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2130\", \"approval_id\": \"AUTH-30-GRANT-20\", \"date\": \"2027-05-01\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-30-CHANGE-20\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9da2cbfb808c3017", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2130" + }, + "response": { + "date": "2027-03-31", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-832df9049e1ef051", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2130", + "approval_id": "AUTH-30-GRANT-20", + "date": "2027-05-01", + "expected_version": 5, + "idempotency_key": "REQUEST-30-CHANGE-20" + }, + "response": { + "status": "rejected", + "reason": "approval_not_effective", + "date": "2027-03-31", + "version": 5 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-03-31", + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_execution_status": "rejected", + "source_interpretation": "The matching server grant for VISIT-2130 is not effective at the supplied server time. The requested later date is not confirmed; retain the current backend date. A rejected attempted write, if any, is an actor error, not authorization.", + "reporters": { + "chronological": "confirm-fresh-30-en-m1-source-chronological", + "grouped_lossless": "confirm-fresh-30-en-m1-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-30-en-m1-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-31-en-m0", + "case_id": "fresh-31-en", + "model": "sarvam-105b", + "family": "future_approval", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2131. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-31-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-31-1", + "role": "customer", + "content": "I want VISIT-2131 changed to 2027-05-02 now." + }, + { + "id": "note-31-2", + "role": "customer", + "content": "The server grant takes effect one hour after the supplied server time. My request does not change its validity period." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-31", + "id": "VISIT-2131", + "date": "2027-04-01", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-31", + "id": "VISIT-3131", + "date": "2027-07-02", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-31", + "id": "AUTH-31-GRANT-20", + "appointment_id": "VISIT-2131", + "operation": "update_appointment", + "date": "2027-05-02", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791752400, + "expires_at": 1791756000, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "6927ff78c0187e6d078d0402da84dc77e78b3dcbb71dcddf63befcaf1693eddc", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nThe booking **VISIT-2131** is currently confirmed as **2027-04-01** (version 5, not closed).\n\nThe supplied approval for the change to 2027-05-02 is granted and not revoked, but its **not-before** time is **1791752400**, which is later than the current server time of **1791748800**. Because the approval is not yet effective at server time, the update has not been applied, and the confirmed date remains **2027-04-01**.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-01", + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "source_interpretation": "The future grant cannot authorize a write at frozen server time; no update occurs, so current date stays 2027-04-01.", + "reporters": { + "chronological": "confirm-fresh-31-en-m0-source-chronological", + "grouped_lossless": "confirm-fresh-31-en-m0-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-31-en-m0-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-31-en-m1", + "case_id": "fresh-31-en", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2131. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-31-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-31-1", + "role": "customer", + "content": "I want VISIT-2131 changed to 2027-05-02 now." + }, + { + "id": "note-31-2", + "role": "customer", + "content": "The server grant takes effect one hour after the supplied server time. My request does not change its validity period." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-31", + "id": "VISIT-2131", + "date": "2027-04-01", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-31", + "id": "VISIT-3131", + "date": "2027-07-02", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-31", + "id": "AUTH-31-GRANT-20", + "appointment_id": "VISIT-2131", + "operation": "update_appointment", + "date": "2027-05-02", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791752400, + "expires_at": 1791756000, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "13d6017aa0c887dab248f46ae189972d27b74b9c63c07d098d85e0e422127563", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-b08126aacde7e055", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2131\"}" + } + } + ] + }, + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-9f0ba8e1a315f3d1", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2131\", \"approval_id\": \"AUTH-31-GRANT-20\", \"date\": \"2027-05-02\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-31-CHANGE-20\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-b08126aacde7e055", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2131" + }, + "response": { + "date": "2027-04-01", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-9f0ba8e1a315f3d1", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2131", + "approval_id": "AUTH-31-GRANT-20", + "date": "2027-05-02", + "expected_version": 5, + "idempotency_key": "REQUEST-31-CHANGE-20" + }, + "response": { + "status": "rejected", + "reason": "approval_not_effective", + "date": "2027-04-01", + "version": 5 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-01", + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_execution_status": "rejected", + "source_interpretation": "The matching server grant for VISIT-2131 is not effective at the supplied server time. The requested later date is not confirmed; retain the current backend date. A rejected attempted write, if any, is an actor error, not authorization.", + "reporters": { + "chronological": "confirm-fresh-31-en-m1-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-31-en-m1-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-31-en-m1-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-32-en-m0", + "case_id": "fresh-32-en", + "model": "sarvam-105b", + "family": "future_approval", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2132. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-32-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. I want VISIT-2132 changed to 2027-05-03 now. The server grant takes effect one hour after the supplied server time. My request does not change its validity period." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-32", + "id": "VISIT-2132", + "date": "2027-04-02", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-32", + "id": "VISIT-3132", + "date": "2027-07-03", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-32", + "id": "AUTH-32-GRANT-20", + "appointment_id": "VISIT-2132", + "operation": "update_appointment", + "date": "2027-05-03", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791752400, + "expires_at": 1791756000, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "a47b76fb68b1242d84cba94b61c8bcb992f38339fc4a59aa180285b1cbbd2964", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nThe date actually confirmed now for **VISIT-2132** is **2027-04-02**.\n\nThe approval (AUTH-32-GRANT-20) is **not yet effective**: while it is granted and unrevoked, its `not_before` time (1791752400) is still one hour ahead of the current server time (1791748800). Therefore the change to 2027-05-03 cannot be applied at this moment, and the booking remains in its current confirmed state.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-02", + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "source_interpretation": "The future-effective grant authorizes no write at this server time; no update occurs and current date remains 2027-04-02.", + "reporters": { + "chronological": "confirm-fresh-32-en-m0-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-32-en-m0-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-32-en-m0-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-32-en-m1", + "case_id": "fresh-32-en", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "treated_actor_prose": false, + "original_question": "Handle the supplied change request for VISIT-2132. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-32-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. I want VISIT-2132 changed to 2027-05-03 now. The server grant takes effect one hour after the supplied server time. My request does not change its validity period." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-32", + "id": "VISIT-2132", + "date": "2027-04-02", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-32", + "id": "VISIT-3132", + "date": "2027-07-03", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-32", + "id": "AUTH-32-GRANT-20", + "appointment_id": "VISIT-2132", + "operation": "update_appointment", + "date": "2027-05-03", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791752400, + "expires_at": 1791756000, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "00f6d1d09307d6909d26a53408ae77c1e59c29a0594f086dbb08cc71b2701550", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-884f2de7db26e776", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2132\"}" + } + } + ] + }, + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-9ef29939fac94bf9", + "type": "function", + "function": { + "name": "update_appointment", + "arguments": "{\"appointment_id\": \"VISIT-2132\", \"approval_id\": \"AUTH-32-GRANT-20\", \"date\": \"2027-05-03\", \"expected_version\": 5, \"idempotency_key\": \"REQUEST-32-CHANGE-20\"}" + } + } + ] + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-884f2de7db26e776", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2132" + }, + "response": { + "date": "2027-04-02", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-9ef29939fac94bf9", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2132", + "approval_id": "AUTH-32-GRANT-20", + "date": "2027-05-03", + "expected_version": 5, + "idempotency_key": "REQUEST-32-CHANGE-20" + }, + "response": { + "status": "rejected", + "reason": "approval_not_effective", + "date": "2027-04-02", + "version": 5 + } + } + ], + "prior_execution_receipts": [], + "final_scoped_backend_date": "2027-04-02", + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_execution_status": "rejected", + "source_interpretation": "The actor attempts the future-approved change; backend rejects it as approval_not_effective and retains 2027-04-02.", + "reporters": { + "grouped_lossless": "confirm-fresh-32-en-m1-source-grouped_lossless", + "chronological": "confirm-fresh-32-en-m1-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-32-en-m1-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-33-en-m0", + "case_id": "fresh-33-en", + "model": "sarvam-105b", + "family": "completed_older_request", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2133. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-33-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-33-1", + "role": "customer", + "content": "Check my original change of VISIT-2133 to 2027-05-04 using its original request identifier." + }, + { + "id": "note-33-2", + "role": "customer", + "content": "It already succeeded. A separate later change moved it to 2027-06-03. Do not undo the later change when handling this duplicate request." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-33", + "id": "VISIT-2133", + "date": "2027-04-03", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-33", + "id": "VISIT-3133", + "date": "2027-07-04", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-33", + "id": "AUTH-33-GRANT-23", + "appointment_id": "VISIT-2133", + "operation": "update_appointment", + "date": "2027-05-04", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-33", + "id": "AUTH-33-LATER-GRANT-23", + "appointment_id": "VISIT-2133", + "operation": "update_appointment", + "date": "2027-06-03", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "ecbb68c6394ad41e332e96859356bfc56134c257f1f493c215285c1a6e6c5376", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nThe date actually confirmed for **VISIT-2133** is **2027-06-03**.\n\n**Reasoning:**\n- The **current** application state lists the date as **2027-06-03** (version 7).\n- There is a prior execution receipt for **AUTH-33-LATER-GRANT-23** showing the identical request to **2027-06-03** was **confirmed** at version 7.\n- The duplicate change to **2027-05-04** (AUTH-33-GRANT-23 / `REQUEST-33-CHANGE-23`) already has a confirmed receipt at version 6, so it is already applied and must not be redone.\n- The later 2027-06-03 change is active and must not be undone.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2133", + "date": "2027-05-04", + "expected_version": 5, + "idempotency_key": "REQUEST-33-CHANGE-23", + "approval_id": "AUTH-33-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-04", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2133", + "date": "2027-06-03", + "expected_version": 6, + "idempotency_key": "REQUEST-33-LATER-23", + "approval_id": "AUTH-33-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-03", + "version": 7 + } + } + ], + "final_scoped_backend_date": "2027-06-03", + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "source_interpretation": "The original request key for VISIT-2133 has an exact prior confirmed receipt, followed by a distinct later confirmed change. The duplicate is already applied, the later backend date remains current, and no new write is needed.", + "reporters": { + "grouped_lossless": "confirm-fresh-33-en-m0-source-grouped_lossless", + "chronological": "confirm-fresh-33-en-m0-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-33-en-m0-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-33-en-m1", + "case_id": "fresh-33-en", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2133. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-33-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-33-1", + "role": "customer", + "content": "Check my original change of VISIT-2133 to 2027-05-04 using its original request identifier." + }, + { + "id": "note-33-2", + "role": "customer", + "content": "It already succeeded. A separate later change moved it to 2027-06-03. Do not undo the later change when handling this duplicate request." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-33", + "id": "VISIT-2133", + "date": "2027-04-03", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-33", + "id": "VISIT-3133", + "date": "2027-07-04", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-33", + "id": "AUTH-33-GRANT-23", + "appointment_id": "VISIT-2133", + "operation": "update_appointment", + "date": "2027-05-04", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-33", + "id": "AUTH-33-LATER-GRANT-23", + "appointment_id": "VISIT-2133", + "operation": "update_appointment", + "date": "2027-06-03", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "10d30671b6bc38a2541f071af9d1ba19c44805b6da071482aa13319e00ba1d8d", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-be79ae53cc0f1fa5", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2133\"}" + } + } + ] + }, + { + "content": "The currently confirmed date for **VISIT-2133** is **2027-06-03**.\n\nThis is consistent with:\n- The **current server state** (date: 2027-06-03, version: 7, closed: false).\n- The **prior execution receipt** for `AUTH-33-LATER-GRANT-23` to `2027-06-03`, which is confirmed at version 7.\n- The **approval `AUTH-33-LATER-GRANT-23`** is granted, unrevoked, and effective (expires at 1791752400; server time is 1791748800, which is after `not_before` 1791745200).\n\nThe earlier request (`AUTH-33-GRANT-23` to `2027-05-04`, idempotency key `REQUEST-33-CHANGE-23`) has already been applied (confirmed at version 6) and must not be duplicated. Per the source messages, the later change to `2027-06-03` should not be undone. Therefore, no further state change is needed or permitted.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-be79ae53cc0f1fa5", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2133" + }, + "response": { + "date": "2027-06-03", + "version": 7, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2133", + "date": "2027-05-04", + "expected_version": 5, + "idempotency_key": "REQUEST-33-CHANGE-23", + "approval_id": "AUTH-33-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-04", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2133", + "date": "2027-06-03", + "expected_version": 6, + "idempotency_key": "REQUEST-33-LATER-23", + "approval_id": "AUTH-33-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-03", + "version": 7 + } + } + ], + "final_scoped_backend_date": "2027-06-03", + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "source_interpretation": "The original request key for VISIT-2133 has an exact prior confirmed receipt, followed by a distinct later confirmed change. The duplicate is already applied, the later backend date remains current, and no new write is needed.", + "reporters": { + "grouped_lossless": "confirm-fresh-33-en-m1-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-33-en-m1-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-33-en-m1-source-chronological" + } + }, + { + "id": "confirm-fresh-34-en-m0", + "case_id": "fresh-34-en", + "model": "sarvam-105b", + "family": "completed_older_request", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2134. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-34-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. Check my original change of VISIT-2134 to 2027-05-05 using its original request identifier. It already succeeded. A separate later change moved it to 2027-06-04. Do not undo the later change when handling this duplicate request." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-34", + "id": "VISIT-2134", + "date": "2027-04-04", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-34", + "id": "VISIT-3134", + "date": "2027-07-05", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-34", + "id": "AUTH-34-GRANT-23", + "appointment_id": "VISIT-2134", + "operation": "update_appointment", + "date": "2027-05-05", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-34", + "id": "AUTH-34-LATER-GRANT-23", + "appointment_id": "VISIT-2134", + "operation": "update_appointment", + "date": "2027-06-04", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "dd70397626181499d0d1803db7df5e13718760ff908c529ba54888b70de3150f", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-9835bdc72789eacb", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2134\"}" + } + } + ] + }, + { + "content": "\nThe identical request (`REQUEST-34-CHANGE-23` / `AUTH-34-GRANT-23` to `2027-05-05`) already has a prior **confirmed** execution receipt dated `2027-05-05` at version `6`. Per the idempotency rule, it must not be duplicated.\n\nMore importantly, the current server state for `VISIT-2134` is:\n\n- **Confirmed date**: **2027-06-04**\n- **Version**: 7\n- **Status**: open (`closed: false`)\n\nThat `2027-06-04` confirmation is supported by the earlier later receipt (`AUTH-34-LATER-GRANT-23`, request `REQUEST-34-LATER-23`), which the customer explicitly asked not to undo.\n\nSo, **the date actually confirmed now is 2027-06-04**.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9835bdc72789eacb", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2134" + }, + "response": { + "date": "2027-06-04", + "version": 7, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2134", + "date": "2027-05-05", + "expected_version": 5, + "idempotency_key": "REQUEST-34-CHANGE-23", + "approval_id": "AUTH-34-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-05", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2134", + "date": "2027-06-04", + "expected_version": 6, + "idempotency_key": "REQUEST-34-LATER-23", + "approval_id": "AUTH-34-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-04", + "version": 7 + } + } + ], + "final_scoped_backend_date": "2027-06-04", + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "source_interpretation": "The original 2027-05-05 request has a matching prior completion, but a later distinct confirmed change sets 2027-06-04; read_current also returns 2027-06-04.", + "reporters": { + "grouped_lossless": "confirm-fresh-34-en-m0-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-34-en-m0-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-34-en-m0-source-chronological" + } + }, + { + "id": "confirm-fresh-34-en-m1", + "case_id": "fresh-34-en", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2134. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-34-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. Check my original change of VISIT-2134 to 2027-05-05 using its original request identifier. It already succeeded. A separate later change moved it to 2027-06-04. Do not undo the later change when handling this duplicate request." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-34", + "id": "VISIT-2134", + "date": "2027-04-04", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-34", + "id": "VISIT-3134", + "date": "2027-07-05", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-34", + "id": "AUTH-34-GRANT-23", + "appointment_id": "VISIT-2134", + "operation": "update_appointment", + "date": "2027-05-05", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-34", + "id": "AUTH-34-LATER-GRANT-23", + "appointment_id": "VISIT-2134", + "operation": "update_appointment", + "date": "2027-06-04", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "9f19453a6df04efaa6183bc32326b45eab38c4745f5ccb46126fb0bd84ee5ae8", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-902d36faffa98fca", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2134\"}" + } + } + ] + }, + { + "content": "The actually confirmed date for VISIT-2134 is **2027-06-04**.\n\nHere is the reasoning:\n\n1. **Current server state** (from `read_current`): The booking is open (`closed: false`) with `date: \"2027-06-04\"` and `version: 7`.\n\n2. **Prior execution receipts**: There are two confirmed receipts for this appointment:\n - The first change to **2027-05-05** (`AUTH-34-GRANT-23`) was already confirmed and advanced the version to 6.\n - The second change to **2027-06-04** (`AUTH-34-LATER-GRANT-23`) was already confirmed and advanced the version to 7.\n\n3. **Duplicate request**: The customer is asking to re-apply the *original* change to **2027-05-05** using `AUTH-34-GRANT-23` and `REQUEST-34-CHANGE-23`. However, that exact request already has a prior confirmed receipt (`status: \"confirmed\"`, `version: 6`). Per the rules, an identical request with a prior confirmed receipt is already applied; do not duplicate it.\n\n4. **Later change is intact**: The later change to **2027-06-04** must not be undone. The current state reflects that later change.\n\nTherefore, the confirmed date now remains **2027-06-04**. No state change is needed.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-902d36faffa98fca", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2134" + }, + "response": { + "date": "2027-06-04", + "version": 7, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2134", + "date": "2027-05-05", + "expected_version": 5, + "idempotency_key": "REQUEST-34-CHANGE-23", + "approval_id": "AUTH-34-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-05", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2134", + "date": "2027-06-04", + "expected_version": 6, + "idempotency_key": "REQUEST-34-LATER-23", + "approval_id": "AUTH-34-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-04", + "version": 7 + } + } + ], + "final_scoped_backend_date": "2027-06-04", + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "source_interpretation": "The original change to 2027-05-05 has a matching prior receipt, but a later distinct completed change leaves 2027-06-04 current.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-34-en-m1-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-34-en-m1-source-chronological", + "grouped_lossless": "confirm-fresh-34-en-m1-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-35-en-m0", + "case_id": "fresh-35-en", + "model": "sarvam-105b", + "family": "completed_older_request", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2135. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-35-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-35-1", + "role": "customer", + "content": "Check my original change of VISIT-2135 to 2027-05-06 using its original request identifier." + }, + { + "id": "note-35-2", + "role": "customer", + "content": "It already succeeded. A separate later change moved it to 2027-06-05. Do not undo the later change when handling this duplicate request." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-35", + "id": "VISIT-2135", + "date": "2027-04-05", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-35", + "id": "VISIT-3135", + "date": "2027-07-06", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-35", + "id": "AUTH-35-GRANT-23", + "appointment_id": "VISIT-2135", + "operation": "update_appointment", + "date": "2027-05-06", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-35", + "id": "AUTH-35-LATER-GRANT-23", + "appointment_id": "VISIT-2135", + "operation": "update_appointment", + "date": "2027-06-05", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "26990911e1f4591f0fc3d552289bccb5494683441dc5a1157118255772e0c1d3", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nThe booking **VISIT-2135** is currently confirmed for **2027-06-05**.\n\n**Why**\n* The earlier change to **2027-05-06** (approval **AUTH-35-GRANT-23**, idempotency **REQUEST-35-CHANGE-23**) already has a confirmed receipt showing status *confirmed* and version **6**—so it was successfully applied.\n* A later change request (idempotency **REQUEST-35-LATER-23**, approval **AUTH-35-LATER-GRANT-23**) moved the appointment to **2027-06-05**, with a confirmed receipt at version **7**.\n* The current state shows `date: 2027-06-05`, `version: 7`, and `closed: false`.\n\nBecause the original **2027-05-06** request is already applied, no duplicate update should be sent. The date actually confirmed now is **2027-06-05**.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2135", + "date": "2027-05-06", + "expected_version": 5, + "idempotency_key": "REQUEST-35-CHANGE-23", + "approval_id": "AUTH-35-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-06", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2135", + "date": "2027-06-05", + "expected_version": 6, + "idempotency_key": "REQUEST-35-LATER-23", + "approval_id": "AUTH-35-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-05", + "version": 7 + } + } + ], + "final_scoped_backend_date": "2027-06-05", + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "source_interpretation": "The original request key for VISIT-2135 has an exact prior confirmed receipt, followed by a distinct later confirmed change. The duplicate is already applied, the later backend date remains current, and no new write is needed.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-35-en-m0-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-35-en-m0-source-chronological", + "grouped_lossless": "confirm-fresh-35-en-m0-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-35-en-m1", + "case_id": "fresh-35-en", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2135. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-35-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-35-1", + "role": "customer", + "content": "Check my original change of VISIT-2135 to 2027-05-06 using its original request identifier." + }, + { + "id": "note-35-2", + "role": "customer", + "content": "It already succeeded. A separate later change moved it to 2027-06-05. Do not undo the later change when handling this duplicate request." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-35", + "id": "VISIT-2135", + "date": "2027-04-05", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-35", + "id": "VISIT-3135", + "date": "2027-07-06", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-35", + "id": "AUTH-35-GRANT-23", + "appointment_id": "VISIT-2135", + "operation": "update_appointment", + "date": "2027-05-06", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-35", + "id": "AUTH-35-LATER-GRANT-23", + "appointment_id": "VISIT-2135", + "operation": "update_appointment", + "date": "2027-06-05", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "acb29cdbc1d77426485c1f047b0e9ca5314dd7bec51b3ada81e851e10cd07e62", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-b518db5cca8b5efc", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2135\"}" + } + } + ] + }, + { + "content": "The date actually confirmed now for **VISIT-2135** is **2027-06-05** (version 7, still open).\n\nThis is evident from the current state read, and it is consistent with the prior execution receipts: the request to move to **2027-05-06** (using `AUTH-35-GRANT-23`) was already confirmed on 2027-05-06 and advanced the booking to version 6, and a later confirmed request moved it to **2027-06-05** (using `AUTH-35-LATER-GRANT-23`) on 2027-06-05, advancing it to version 7. The customer’s own note explicitly says not to undo the later change when handling this duplicate request. Therefore, no further action is needed.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-b518db5cca8b5efc", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2135" + }, + "response": { + "date": "2027-06-05", + "version": 7, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2135", + "date": "2027-05-06", + "expected_version": 5, + "idempotency_key": "REQUEST-35-CHANGE-23", + "approval_id": "AUTH-35-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-06", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2135", + "date": "2027-06-05", + "expected_version": 6, + "idempotency_key": "REQUEST-35-LATER-23", + "approval_id": "AUTH-35-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-05", + "version": 7 + } + } + ], + "final_scoped_backend_date": "2027-06-05", + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "source_interpretation": "The original request key for VISIT-2135 has an exact prior confirmed receipt, followed by a distinct later confirmed change. The duplicate is already applied, the later backend date remains current, and no new write is needed.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-35-en-m1-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-35-en-m1-source-grouped_lossless", + "chronological": "confirm-fresh-35-en-m1-source-chronological" + } + }, + { + "id": "confirm-fresh-36-en-m0", + "case_id": "fresh-36-en", + "model": "sarvam-105b", + "family": "completed_older_request", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2136. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-36-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. Check my original change of VISIT-2136 to 2027-05-07 using its original request identifier. It already succeeded. A separate later change moved it to 2027-06-06. Do not undo the later change when handling this duplicate request." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-36", + "id": "VISIT-2136", + "date": "2027-04-06", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-36", + "id": "VISIT-3136", + "date": "2027-07-07", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-36", + "id": "AUTH-36-GRANT-23", + "appointment_id": "VISIT-2136", + "operation": "update_appointment", + "date": "2027-05-07", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-36", + "id": "AUTH-36-LATER-GRANT-23", + "appointment_id": "VISIT-2136", + "operation": "update_appointment", + "date": "2027-06-06", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "47024432738e9cb8d96e5aedbdb024190cb9a264e75053d44e3bdb9dbebc442d", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nThe requested change for **VISIT-2136** to **2027-05-07** using approval **AUTH-36-GRANT-23** and idempotency key **REQUEST-36-CHANGE-23** is a **duplicate of a prior confirmed execution**. A receipt already exists showing that these exact arguments were executed, confirmed on **2027-05-07**, and resulted in version **6**. Because an identical request with a prior confirmed receipt is already applied, it must not be duplicated; no further state change is performed.\n\nPer the current trusted state, the booking is open (`closed: false`) and reflects the later approved change that moved the appointment to **2027-06-06**.\n\n**The date actually confirmed now is 2027-06-06.**", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2136", + "date": "2027-05-07", + "expected_version": 5, + "idempotency_key": "REQUEST-36-CHANGE-23", + "approval_id": "AUTH-36-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-07", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2136", + "date": "2027-06-06", + "expected_version": 6, + "idempotency_key": "REQUEST-36-LATER-23", + "approval_id": "AUTH-36-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-06", + "version": 7 + } + } + ], + "final_scoped_backend_date": "2027-06-06", + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "source_interpretation": "A matching prior receipt proves the original 2027-05-07 change; a later distinct change makes 2027-06-06 current, and no new write occurs.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-36-en-m0-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-36-en-m0-source-grouped_lossless", + "chronological": "confirm-fresh-36-en-m0-source-chronological" + } + }, + { + "id": "confirm-fresh-36-en-m1", + "case_id": "fresh-36-en", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2136. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-36-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. Check my original change of VISIT-2136 to 2027-05-07 using its original request identifier. It already succeeded. A separate later change moved it to 2027-06-06. Do not undo the later change when handling this duplicate request." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-36", + "id": "VISIT-2136", + "date": "2027-04-06", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-36", + "id": "VISIT-3136", + "date": "2027-07-07", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-36", + "id": "AUTH-36-GRANT-23", + "appointment_id": "VISIT-2136", + "operation": "update_appointment", + "date": "2027-05-07", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-36", + "id": "AUTH-36-LATER-GRANT-23", + "appointment_id": "VISIT-2136", + "operation": "update_appointment", + "date": "2027-06-06", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "94a800df85d56380e60b8141aa36b99ba63d4d925aba6777f210199c70d82252", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-801c6b5c33bfb9a5", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2136\"}" + } + } + ] + }, + { + "content": "The currently confirmed date for **VISIT-2136** is **2027-06-06**.\n\nHere is the reasoning based on the supplied evidence:\n\n1. **Current server state** (from `read_current`): The booking is open (`closed: false`) with `date: \"2027-06-06\"` and `version: 7`.\n2. **Prior execution receipts**: There is an already-confirmed receipt showing that `AUTH-36-LATER-GRANT-23` successfully changed the appointment to `2027-06-06`, resulting in version `7`.\n3. **Duplicate request**: The customer is asking to change the appointment to `2027-05-07` using `AUTH-36-GRANT-23` with `expected_version: 5`. However, there is an identical prior confirmed execution for exactly these same arguments (`appointment_id`, `approval_id`, `date`, `expected_version`, `idempotency_key`). Per the instructions, an identical request with a prior confirmed receipt is already applied; do not duplicate it.\n4. **Later change precedence**: The source message explicitly instructs not to undo the later change to `2027-06-06`.\n\nTherefore, despite the duplicate request for `2027-05-07`, the actual confirmed date remains **2027-06-06**. No state change should be attempted.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-801c6b5c33bfb9a5", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2136" + }, + "response": { + "date": "2027-06-06", + "version": 7, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2136", + "date": "2027-05-07", + "expected_version": 5, + "idempotency_key": "REQUEST-36-CHANGE-23", + "approval_id": "AUTH-36-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-07", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2136", + "date": "2027-06-06", + "expected_version": 6, + "idempotency_key": "REQUEST-36-LATER-23", + "approval_id": "AUTH-36-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-06", + "version": 7 + } + } + ], + "final_scoped_backend_date": "2027-06-06", + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "source_interpretation": "Prior receipts prove the original request and later 2027-06-06 update; read_current confirms that later date.", + "reporters": { + "chronological": "confirm-fresh-36-en-m1-source-chronological", + "grouped_lossless": "confirm-fresh-36-en-m1-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-36-en-m1-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-37-en-m0", + "case_id": "fresh-37-en", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2137. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-37-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-37-1", + "role": "customer", + "content": "Now I want 2027-06-07 for VISIT-2137." + }, + { + "id": "note-37-2", + "role": "customer", + "content": "The supplied request identifier was already used for the completed change to 2027-05-08. Do not apply a different change using that same identifier." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-37", + "id": "VISIT-2137", + "date": "2027-04-07", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-37", + "id": "VISIT-3137", + "date": "2027-07-08", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-37", + "id": "AUTH-37-GRANT-24", + "appointment_id": "VISIT-2137", + "operation": "update_appointment", + "date": "2027-05-08", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-37", + "id": "AUTH-37-CONFLICT-GRANT", + "appointment_id": "VISIT-2137", + "operation": "update_appointment", + "date": "2027-06-07", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "29b55f152ef7c8110f41bad222abd90b8e485dc5edbbd6f330cda6d5e764fd91", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-85adb99ff0c7e630", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2137\"}" + } + } + ] + }, + { + "content": "\nThe date actually confirmed for **VISIT-2137** is **2027-05-08**.\n\nA prior execution receipt shows that the idempotency key `REQUEST-37-CHANGE-24` was already used for a confirmed change to **2027-05-08** (version 6). Because that same identifier has been consumed, the later request—using `AUTH-37-CONFLICT-GRANT` for **2027-06-07**—cannot be applied again. The current server state remains:\n\n- **Confirmed date:** 2027-05-08 \n- **Version:** 6 \n- **Closed:** false", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-85adb99ff0c7e630", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2137" + }, + "response": { + "date": "2027-05-08", + "version": 6, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2137", + "date": "2027-05-08", + "expected_version": 5, + "idempotency_key": "REQUEST-37-CHANGE-24", + "approval_id": "AUTH-37-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-08", + "version": 6 + } + } + ], + "final_scoped_backend_date": "2027-05-08", + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "source_interpretation": "The supplied key was previously consumed by a different 2027-05-08 request; the desired 2027-06-07 change is not applied, so current date remains 2027-05-08.", + "reporters": { + "chronological": "confirm-fresh-37-en-m0-source-chronological", + "grouped_lossless": "confirm-fresh-37-en-m0-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-37-en-m0-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-37-en-m1", + "case_id": "fresh-37-en", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2137. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-37-0", + "role": "customer", + "content": "I am Meera; this is a follow-up about my booking." + }, + { + "id": "note-37-1", + "role": "customer", + "content": "Now I want 2027-06-07 for VISIT-2137." + }, + { + "id": "note-37-2", + "role": "customer", + "content": "The supplied request identifier was already used for the completed change to 2027-05-08. Do not apply a different change using that same identifier." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-37", + "id": "VISIT-2137", + "date": "2027-04-07", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-37", + "id": "VISIT-3137", + "date": "2027-07-08", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-37", + "id": "AUTH-37-GRANT-24", + "appointment_id": "VISIT-2137", + "operation": "update_appointment", + "date": "2027-05-08", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-37", + "id": "AUTH-37-CONFLICT-GRANT", + "appointment_id": "VISIT-2137", + "operation": "update_appointment", + "date": "2027-06-07", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "51c205e56e219e1ea7138eb708d9059a0d4428d17a8cbcbffcd51103956bb589", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-9a034907ad117d24", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2137\"}" + } + } + ] + }, + { + "content": "The actually confirmed date for VISIT-2137 is **2027-05-08**.\n\n**Reasoning:**\n- The current state shows the booking is open (`closed: false`) with date `2027-05-08` and version `6`.\n- There is a prior execution receipt for the same idempotency key (`REQUEST-37-CHANGE-24`) showing the change to `2027-05-08` was already confirmed, resulting in version `6`.\n- The source message explicitly warns: *\"The supplied request identifier was already used for the completed change to 2027-05-08. Do not apply a different change using that same identifier.\"*\n- Because an identical request with a prior confirmed receipt is already applied, I must not duplicate it. A safe refusal is appropriate; no state change should be attempted.\n\nTherefore, the confirmed date remains **2027-05-08**.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9a034907ad117d24", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2137" + }, + "response": { + "date": "2027-05-08", + "version": 6, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2137", + "date": "2027-05-08", + "expected_version": 5, + "idempotency_key": "REQUEST-37-CHANGE-24", + "approval_id": "AUTH-37-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-08", + "version": 6 + } + } + ], + "final_scoped_backend_date": "2027-05-08", + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "source_interpretation": "The supplied key for VISIT-2137 was already consumed for a different completed request. The new proposed date cannot be applied with that key; retain the current confirmed backend date and make no new write.", + "reporters": { + "chronological": "confirm-fresh-37-en-m1-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-37-en-m1-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-37-en-m1-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-38-en-m0", + "case_id": "fresh-38-en", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2138. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-38-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. Now I want 2027-06-08 for VISIT-2138. The supplied request identifier was already used for the completed change to 2027-05-09. Do not apply a different change using that same identifier." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-38", + "id": "VISIT-2138", + "date": "2027-04-08", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-38", + "id": "VISIT-3138", + "date": "2027-07-09", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-38", + "id": "AUTH-38-GRANT-24", + "appointment_id": "VISIT-2138", + "operation": "update_appointment", + "date": "2027-05-09", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-38", + "id": "AUTH-38-CONFLICT-GRANT", + "appointment_id": "VISIT-2138", + "operation": "update_appointment", + "date": "2027-06-08", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "4c19f3ac5d7e8cff84e515b182fc3ec4a3d44c055eb94fb05d24424bbdbc3218", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": "\nI will not apply the requested change. The idempotency key `REQUEST-38-CHANGE-24` was already used in a prior confirmed execution that moved the appointment to `2027-05-09` (receipt version 6), and repeating that key for a different change would duplicate it.\n\nThe date actually confirmed now for `VISIT-2138` is **2027-05-09**.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2138", + "date": "2027-05-09", + "expected_version": 5, + "idempotency_key": "REQUEST-38-CHANGE-24", + "approval_id": "AUTH-38-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-09", + "version": 6 + } + } + ], + "final_scoped_backend_date": "2027-05-09", + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "source_interpretation": "The supplied key for VISIT-2138 was already consumed for a different completed request. The new proposed date cannot be applied with that key; retain the current confirmed backend date and make no new write.", + "reporters": { + "chronological": "confirm-fresh-38-en-m0-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-38-en-m0-source-grouped_actor_prose_withheld", + "grouped_lossless": "confirm-fresh-38-en-m0-source-grouped_lossless" + } + }, + { + "id": "confirm-fresh-38-en-m1", + "case_id": "fresh-38-en", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2138. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-38-0", + "role": "customer", + "content": "I am Arjun. Please read both messages in order. Now I want 2027-06-08 for VISIT-2138. The supplied request identifier was already used for the completed change to 2027-05-09. Do not apply a different change using that same identifier." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-38", + "id": "VISIT-2138", + "date": "2027-04-08", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-38", + "id": "VISIT-3138", + "date": "2027-07-09", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-38", + "id": "AUTH-38-GRANT-24", + "appointment_id": "VISIT-2138", + "operation": "update_appointment", + "date": "2027-05-09", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-38", + "id": "AUTH-38-CONFLICT-GRANT", + "appointment_id": "VISIT-2138", + "operation": "update_appointment", + "date": "2027-06-08", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "c3e160ba861133e3ecfb894fcd3cff9345bd0df2ee0ffd5be840b90890815cd5", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-86eb9f9cc112f393", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2138\"}" + } + } + ] + }, + { + "content": "The current confirmed date for VISIT-2138 is **2027-05-09**.\n\nHere is the reasoning:\n\n1. **Current state**: The server confirms the booking is open (`closed: false`) with date `2027-05-09` and version `6`.\n2. **Prior execution**: There is a confirmed receipt showing that the exact same request key (`REQUEST-38-CHANGE-24`) was already used to change the booking to `2027-05-09`, and the server accepted it (version advanced to `6`).\n3. **Duplicate-request rule**: The customer’s own note explicitly states that the supplied request identifier was already used for the completed change to `2027-05-09`, and instructs not to apply a different change using the same identifier.\n4. **No safe execution**: Because the prior receipt with the identical `idempotency_key` is already confirmed, an identical request must not be duplicated. Therefore, I do not execute the `update_appointment` call.\n\n**Result**: The actually confirmed date remains **2027-05-09**.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-86eb9f9cc112f393", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2138" + }, + "response": { + "date": "2027-05-09", + "version": 6, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2138", + "date": "2027-05-09", + "expected_version": 5, + "idempotency_key": "REQUEST-38-CHANGE-24", + "approval_id": "AUTH-38-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-09", + "version": 6 + } + } + ], + "final_scoped_backend_date": "2027-05-09", + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "source_interpretation": "The 2027-06-08 desired change reuses a key whose prior completed request had different arguments; no new tool update occurs and current date remains 2027-05-09.", + "reporters": { + "grouped_lossless": "confirm-fresh-38-en-m1-source-grouped_lossless", + "chronological": "confirm-fresh-38-en-m1-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-38-en-m1-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-39-en-m0", + "case_id": "fresh-39-en", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2139. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-39-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-39-1", + "role": "customer", + "content": "Now I want 2027-06-09 for VISIT-2139." + }, + { + "id": "note-39-2", + "role": "customer", + "content": "The supplied request identifier was already used for the completed change to 2027-05-10. Do not apply a different change using that same identifier." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-39", + "id": "VISIT-2139", + "date": "2027-04-09", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-39", + "id": "VISIT-3139", + "date": "2027-07-10", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-39", + "id": "AUTH-39-GRANT-24", + "appointment_id": "VISIT-2139", + "operation": "update_appointment", + "date": "2027-05-10", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-39", + "id": "AUTH-39-CONFLICT-GRANT", + "appointment_id": "VISIT-2139", + "operation": "update_appointment", + "date": "2027-06-09", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "8a675839590f2e7961ec0f44073fbeedde9687a93ee04a89f28a2e7499ec2dd3", + "original_actor_terminal": "incomplete", + "actor_claims": [ + { + "content": "\nThe date actually confirmed now is **2027-05-10**.\n\nThe prior execution receipt shows that the request identifier `REQUEST-39-CHANGE-24` was already used for a confirmed change to **2027-05-10** (version 6). Because an identical request with a prior confirmed receipt is already applied, the duplicate request to change to **2027-06-09** using that same identifier must not be applied. The current booking state remains at **2027-05-10**.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2139", + "date": "2027-05-10", + "expected_version": 5, + "idempotency_key": "REQUEST-39-CHANGE-24", + "approval_id": "AUTH-39-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-10", + "version": 6 + } + } + ], + "final_scoped_backend_date": "2027-05-10", + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "source_interpretation": "The current desired 2027-06-09 differs from the completed request under the reused key; the original current date 2027-05-10 remains in force and no new update is attempted.", + "reporters": { + "grouped_lossless": "confirm-fresh-39-en-m0-source-grouped_lossless", + "chronological": "confirm-fresh-39-en-m0-source-chronological", + "grouped_actor_prose_withheld": "confirm-fresh-39-en-m0-source-grouped_actor_prose_withheld" + } + }, + { + "id": "confirm-fresh-39-en-m1", + "case_id": "fresh-39-en", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2139. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-39-0", + "role": "customer", + "content": "My name is Kavya. Here is the earlier message and then the follow-up." + }, + { + "id": "note-39-1", + "role": "customer", + "content": "Now I want 2027-06-09 for VISIT-2139." + }, + { + "id": "note-39-2", + "role": "customer", + "content": "The supplied request identifier was already used for the completed change to 2027-05-10. Do not apply a different change using that same identifier." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-39", + "id": "VISIT-2139", + "date": "2027-04-09", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-39", + "id": "VISIT-3139", + "date": "2027-07-10", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-39", + "id": "AUTH-39-GRANT-24", + "appointment_id": "VISIT-2139", + "operation": "update_appointment", + "date": "2027-05-10", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-39", + "id": "AUTH-39-CONFLICT-GRANT", + "appointment_id": "VISIT-2139", + "operation": "update_appointment", + "date": "2027-06-09", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "d34c318019cb9c27f6c69f0b4dfb5e0badeaee76074739b4492da33eb478fd2b", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-a136ed28cf24b364", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2139\"}" + } + } + ] + }, + { + "content": "The currently confirmed date for VISIT-2139 is **2027-05-10**.\n\nThis is based on the current server state, which shows the booking is open (closed: false) with date 2027-05-10 and version 6. The prior execution receipt confirms that the request with idempotency key `REQUEST-39-CHANGE-24` was already applied, changing the date to 2027-05-10 and resulting in version 6.\n\nThe supplied change request for 2027-06-09 uses the same idempotency key (`REQUEST-39-CHANGE-24`), which the source messages explicitly warn was already used for the completed change to 2027-05-10. Therefore, this request must not be applied again. No state change is performed.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a136ed28cf24b364", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2139" + }, + "response": { + "date": "2027-05-10", + "version": 6, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2139", + "date": "2027-05-10", + "expected_version": 5, + "idempotency_key": "REQUEST-39-CHANGE-24", + "approval_id": "AUTH-39-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-10", + "version": 6 + } + } + ], + "final_scoped_backend_date": "2027-05-10", + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "source_interpretation": "The 2027-06-09 desired change conflicts with the key previously used for 2027-05-10; no new update occurs and current date remains 2027-05-10.", + "reporters": { + "grouped_lossless": "confirm-fresh-39-en-m1-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-39-en-m1-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-39-en-m1-source-chronological" + } + }, + { + "id": "confirm-fresh-40-en-m0", + "case_id": "fresh-40-en", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2140. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-40-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. Now I want 2027-06-10 for VISIT-2140. The supplied request identifier was already used for the completed change to 2027-05-11. Do not apply a different change using that same identifier." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-40", + "id": "VISIT-2140", + "date": "2027-04-10", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-40", + "id": "VISIT-3140", + "date": "2027-07-11", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-40", + "id": "AUTH-40-GRANT-24", + "appointment_id": "VISIT-2140", + "operation": "update_appointment", + "date": "2027-05-11", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-40", + "id": "AUTH-40-CONFLICT-GRANT", + "appointment_id": "VISIT-2140", + "operation": "update_appointment", + "date": "2027-06-10", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "46292e57eb7a9f950c4b4368c0d598018873f9491aeff6e3f27bb469c9ae43f5", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-837aa92065b22600", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2140\"}" + } + } + ] + }, + { + "content": "\nThe date that is actually confirmed now is **2027-05-11**.\n\nThe prior confirmed receipt for idempotency key **REQUEST-40-CHANGE-24** already applied a successful change to 2027-05-11, and the customer’s own note explicitly instructed not to reuse that same identifier for a different date. Because an identical request with a confirmed receipt has already been applied, the new request to 2027-06-10 must not be duplicated. The booking remains open, so no further change is executed.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-837aa92065b22600", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2140" + }, + "response": { + "date": "2027-05-11", + "version": 6, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2140", + "date": "2027-05-11", + "expected_version": 5, + "idempotency_key": "REQUEST-40-CHANGE-24", + "approval_id": "AUTH-40-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-11", + "version": 6 + } + } + ], + "final_scoped_backend_date": "2027-05-11", + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "source_interpretation": "The supplied key for VISIT-2140 was already consumed for a different completed request. The new proposed date cannot be applied with that key; retain the current confirmed backend date and make no new write.", + "reporters": { + "grouped_lossless": "confirm-fresh-40-en-m0-source-grouped_lossless", + "grouped_actor_prose_withheld": "confirm-fresh-40-en-m0-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-40-en-m0-source-chronological" + } + }, + { + "id": "confirm-fresh-40-en-m1", + "case_id": "fresh-40-en", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "treated_actor_prose": true, + "original_question": "Handle the supplied change request for VISIT-2140. What date is actually confirmed now?", + "original_source_messages": [ + { + "id": "note-40-0", + "role": "customer", + "content": "This is Ravi. These two messages concern the booking references shown. Now I want 2027-06-10 for VISIT-2140. The supplied request identifier was already used for the completed change to 2027-05-11. Do not apply a different change using that same identifier." + } + ], + "source_conditions": { + "issuers": [ + "reviewer" + ], + "appointments": [ + { + "tenant": "fresh-confirmation", + "customer": "person-40", + "id": "VISIT-2140", + "date": "2027-04-10", + "version": 5, + "closed": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-40", + "id": "VISIT-3140", + "date": "2027-07-11", + "version": 40, + "closed": false + } + ], + "approvals": [ + { + "tenant": "fresh-confirmation", + "customer": "person-40", + "id": "AUTH-40-GRANT-24", + "appointment_id": "VISIT-2140", + "operation": "update_appointment", + "date": "2027-05-11", + "expected_version": 5, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + }, + { + "tenant": "fresh-confirmation", + "customer": "person-40", + "id": "AUTH-40-CONFLICT-GRANT", + "appointment_id": "VISIT-2140", + "operation": "update_appointment", + "date": "2027-06-10", + "expected_version": 6, + "issuer": "reviewer", + "issued_at": 1791741600, + "not_before": 1791745200, + "expires_at": 1791752400, + "granted": true, + "revoked": false + } + ] + }, + "parent_paired_sha256": "c04ebb10ef9d739136e60f8dab37c3cb0049a584b741034ac793ce5dcc6b9634", + "original_actor_terminal": "answer", + "actor_claims": [ + { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": [ + { + "id": "chatcmpl-tool-be2448f8475c4e77", + "type": "function", + "function": { + "name": "read_current", + "arguments": "{\"appointment_id\": \"VISIT-2140\"}" + } + } + ] + }, + { + "content": "The confirmed date for VISIT-2140 is **2027-05-11**.\n\n**Reasoning:**\n\n1. **Current state:** The server-reported current state shows `date: \"2027-05-11\"`, `version: 6`, and `closed: false`.\n2. **Prior execution:** There is an existing receipt confirming that `REQUEST-40-CHANGE-24` was already applied, resulting in `date: \"2027-05-11\"` at `version: 6`.\n3. **Duplicate idempotency key:** The customer's requested change uses the same `idempotency_key` (`REQUEST-40-CHANGE-24`) as the already-confirmed change. Per policy, an identical request with a prior confirmed receipt is already applied and must not be duplicated.\n4. **Conflict approval is irrelevant:** Even though `AUTH-40-CONFLICT-GRANT` exists for `2027-06-10`, the prior receipt for the same idempotency key has already locked in the `2027-05-11` result at version 6. The server state reflects that. The customer's note explicitly warns not to apply a different change using the same identifier.\n\nTherefore, no new state change is performed, and the **actually confirmed date remains 2027-05-11**.", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + ], + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-be2448f8475c4e77", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2140" + }, + "response": { + "date": "2027-05-11", + "version": 6, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2140", + "date": "2027-05-11", + "expected_version": 5, + "idempotency_key": "REQUEST-40-CHANGE-24", + "approval_id": "AUTH-40-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-11", + "version": 6 + } + } + ], + "final_scoped_backend_date": "2027-05-11", + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "source_interpretation": "The key belongs to a completed 2027-05-11 request with different arguments from the desired 2027-06-10 change; no new write occurs.", + "reporters": { + "grouped_actor_prose_withheld": "confirm-fresh-40-en-m1-source-grouped_actor_prose_withheld", + "chronological": "confirm-fresh-40-en-m1-source-chronological", + "grouped_lossless": "confirm-fresh-40-en-m1-source-grouped_lossless" + } + } + ], + "reporter_reviews": [ + { + "id": "confirm-fresh-31-en-m1-source-chronological", + "episode_id": "confirm-fresh-31-en-m1", + "case_id": "fresh-31-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "2162ba1711547b3550715f7f82ac609e01f13fac1ed35e226e62e527d922ca75", + "request_payload_sha256": "660bd24c14c49679b337a22541bd7cfde8ca3ea1ae02940b0cee68a7fd7e47fc", + "receipt_sha256": "c3626b6c7bb56abdf0c54618534c0ef46083eefa0a0c2d444e0a5f11ba8f93de", + "result_sha256": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-04-01\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-04-01\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "projected_action_status": "rejected", + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_execution_status": "rejected", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-31-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-31-en-m1", + "case_id": "fresh-31-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "ba0b908b932485cc5bf0cec21c510818dee5f63f472c94be0735dc1be60ced39", + "request_payload_sha256": "43cf7771462081e34576456bf77ecbac7e53ede7b834b9374b144106d65fc3ba", + "receipt_sha256": "670087fd1f9fe65747cf7c04fcf8e38739ad9e27929b55b8fb8cb90d1a9248d8", + "result_sha256": "7c2d167966a96aabe602d89d49013e59fa527ef56f220832be19b0d7dc6940d7", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-04-01\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-04-01\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-04-01", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-04-01", + "needs_clarification": false + }, + "projected_action_status": "rejected", + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_execution_status": "rejected", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-31-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-31-en-m1", + "case_id": "fresh-31-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "ba0b908b932485cc5bf0cec21c510818dee5f63f472c94be0735dc1be60ced39", + "request_payload_sha256": "43cf7771462081e34576456bf77ecbac7e53ede7b834b9374b144106d65fc3ba", + "receipt_sha256": "55a6940bf185837f550d66434d1fdcf486901ec7abd82eadeb285ef3a93b7e6b", + "result_sha256": "7c2d167966a96aabe602d89d49013e59fa527ef56f220832be19b0d7dc6940d7", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-04-01\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-04-01\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-04-01", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-04-01", + "needs_clarification": false + }, + "projected_action_status": "rejected", + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_execution_status": "rejected", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-18-en-m1-source-chronological", + "episode_id": "confirm-fresh-18-en-m1", + "case_id": "fresh-18-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "bba20f9c81ab806383d56bfc27b4ca4f540c4ca878d6ac75a79bea30ffd0b3a0", + "request_payload_sha256": "d34e1725d6a0faf9d1cba3b223ed387ffbd430ea1f4b1e29e6ce7648f2e50838", + "receipt_sha256": "dd26b0242428f90607abc92fa6957675049d285461fbaa075e817838385c3e70", + "result_sha256": "00d649cfc39ec358aa1d45e738f53feb9cb97e2c7b7fc9ca56a64eec5020214e", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-19\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-19\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": true, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It substitutes the backend's current date for the customer's preference." + }, + { + "id": "confirm-fresh-18-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-18-en-m1", + "case_id": "fresh-18-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "50279874911caf1be2772d0976402498059229843d1d7366e5d0ccf6db00b2ca", + "request_payload_sha256": "44e14b8511e8034ad94cba7c4169cae971d274ca4c0f749307d650430532ff29", + "receipt_sha256": "b0b80044d0e2cf866a2d46773c2bd46efc4f0553bd679e0a6dedbd1b59915762", + "result_sha256": "00d649cfc39ec358aa1d45e738f53feb9cb97e2c7b7fc9ca56a64eec5020214e", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-19\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-19\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": true, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It substitutes the backend's current date for the customer's preference." + }, + { + "id": "confirm-fresh-18-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-18-en-m1", + "case_id": "fresh-18-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "ed079e86ced287775464eb76df3f4fdc43e836878465e5cd27333de748c5ad1f", + "request_payload_sha256": "6735f6432c9678996f0dc671a3db5faffa8d9d1cef8dadf66184f46d1b882101", + "receipt_sha256": "1b6c1ffe92650fbf12174952942ede3cb4ed82c4425288c893daac1aec9b9450", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It abstains although the source or actual receipt determines a date; Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-35-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-35-en-m0", + "case_id": "fresh-35-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "aa6eba428341b35b02053d2db511bad8f11a39b0c2423f82c10ce946ceb37eef", + "request_payload_sha256": "03e55421f300be236a0cd907b836945695457fd36110dcb1caba5d5b472e01bd", + "receipt_sha256": "f5a83c3f52084326689dbf0df6dd6ab73abb6668a3078adf62407823fef6d28a", + "result_sha256": "a10e59f23e1bcbeb10d0bb5925482d8103b4a143246bb28f303d672b12eeb2f2", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-05\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-05\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-35-en-m0-source-chronological", + "episode_id": "confirm-fresh-35-en-m0", + "case_id": "fresh-35-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "b3ff0cacaccf05ab7a1c8004567b05001778f26e7dc3d783f0fa6152a42f3473", + "request_payload_sha256": "e7e66be949ddf0abb8ff0db8ddceaa989296014c9a89327ac51c52235091f565", + "receipt_sha256": "1ff693b65299542e549367de991dd8f7eb74ef0805bd5322f0be416401292d2b", + "result_sha256": "b31d0f9b5f8fe5765afbcb1eaf44d89c8c5492e0a28b6d2d4ff5fe02f1c09f85", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-05\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-05\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-35-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-35-en-m0", + "case_id": "fresh-35-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "312cb6d0662e523f1afd60d693798b0a3118e9ddaf12e6ca6c10467592223dac", + "request_payload_sha256": "cea7c03b4970350c1c8be591056a9153db4cc378fe6c4a8ccca65fee62d8ea9e", + "receipt_sha256": "fa3c3742094150c263ee5dc8f4025afecf555f8ecf7d3139807cbfb53c5d6f63", + "result_sha256": "b31d0f9b5f8fe5765afbcb1eaf44d89c8c5492e0a28b6d2d4ff5fe02f1c09f85", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-05\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-05\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-36-en-m1-source-chronological", + "episode_id": "confirm-fresh-36-en-m1", + "case_id": "fresh-36-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "ac27c5a97d2465fec1ade3b381cd3d1a25f80435f91e3fa6de9a85c97657b879", + "request_payload_sha256": "e302681ad3676395d3ddd7198c743b8b5a4c79f797707da7c402a480ecd020ea", + "receipt_sha256": "4d352af437a7d57e6dfc946b5c504dcfe593ec99d09ec7fc5d7b1592cdb9e7c6", + "result_sha256": "5766d8a3d2817eb1b9e489c5c75934249fc394e5c0209fb689239f6b03244dca", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-06\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-06\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-36-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-36-en-m1", + "case_id": "fresh-36-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "22ada01cba0deb26d1976856b23aabb20635fb9a71e80dbaa9b145f2f4b508b2", + "request_payload_sha256": "a09bbdaa0e14434b9cc2395163e2d1ce0f15fd1f3488576df891bbb703a9d5a8", + "receipt_sha256": "a448aaf931a74f19c308745e29385ad66c01a6fe72b64f0196f743438af8ca32", + "result_sha256": "460760fb570c5e062b8fdc0daad95643745d12fc8f13ad2e8f3402ebad0a8899", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-06-06\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-06-06\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-06-06", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-06-06", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-36-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-36-en-m1", + "case_id": "fresh-36-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "870b06111e40ded8ee5c30275a8882f7f08239553444d82d06744408d00be65d", + "request_payload_sha256": "e6bdb1d787f1cf1a104e97d77ffd3b6e109ea8050f410fc8916ac32952f08595", + "receipt_sha256": "f5121346f88564caa01a01873127a6b6d5afaabb1b910dff78895e493339e5b0", + "result_sha256": "5766d8a3d2817eb1b9e489c5c75934249fc394e5c0209fb689239f6b03244dca", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-06\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-06\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-24-en-m1-source-chronological", + "episode_id": "confirm-fresh-24-en-m1", + "case_id": "fresh-24-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "44a346cfe634d35af9cd78119bdd618b942c6d8a0fbb3d852a1fcb46a02a936a", + "request_payload_sha256": "bac8e9679e53e9c298b1c3ac0d4897d8daa0df6096c749a5b43150441e38cb9e", + "receipt_sha256": "522cc2c69bf2f1ad5ba1bd6a1bdb537b649f9111841cfa8d792c3a2abe5d8ceb", + "result_sha256": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-25\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-25\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-24-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-24-en-m1", + "case_id": "fresh-24-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "bde942a4e8576b1853c82efe924d6f8efc3190b4817492339576a1fb8385790d", + "request_payload_sha256": "9bf8babdd09968a2bdadf008ea96a6225b149ed3b70218fa55e6a6234c69cb6e", + "receipt_sha256": "b6f73b2253e5a5607a474f0bab42e9a7de89d47a749fe02af5f124c0302e6301", + "result_sha256": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-25\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-25\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-24-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-24-en-m1", + "case_id": "fresh-24-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "bde942a4e8576b1853c82efe924d6f8efc3190b4817492339576a1fb8385790d", + "request_payload_sha256": "9bf8babdd09968a2bdadf008ea96a6225b149ed3b70218fa55e6a6234c69cb6e", + "receipt_sha256": "2143c708c74db01d9a98ad59096675530c23801dd5a2727995c48af71f98aa62", + "result_sha256": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-25\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-25\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-40-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-40-en-m1", + "case_id": "fresh-40-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "3cec6757e2a5f707fbc74f9b3a81efdec8332c5bd8301b0c5cb9e83f4a1d8412", + "request_payload_sha256": "d0aec43fc792160004dbcf259c0e9bf0b4f4e60c6033a25f3874e3c499f370c6", + "receipt_sha256": "2f143e14715c7a80b53e8caa702490264b5335808d55386130a824b1ab96aab5", + "result_sha256": "e154d7d3fea97d5d74e19e49727b25d73019ead962c7b21da19391b8aa91e911", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-40-en-m1-source-chronological", + "episode_id": "confirm-fresh-40-en-m1", + "case_id": "fresh-40-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "053d7216880bb081752bf2314d9f0abf8e784d8a7d9ddb739ece9082613e87d3", + "request_payload_sha256": "99399c43b9f8c71f971265f6bc22ff4b0ae757aa039a3f3f40e6e392f2a4631f", + "receipt_sha256": "2aff09702788da9caa314a0011ca352d4f230f34b431e8b860b3a2ae108e6211", + "result_sha256": "e154d7d3fea97d5d74e19e49727b25d73019ead962c7b21da19391b8aa91e911", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-40-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-40-en-m1", + "case_id": "fresh-40-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "c5a7199559d41f3958ec02c0290aa34d1f2dd499b3443891b0940795ef5c656a", + "request_payload_sha256": "4c2acc3858d8d180cb828a2354dade890326a4f42ca8aad2eb7e8d03ef54e297", + "receipt_sha256": "a5f6ff2fe1c2ec00d250bd624190014d94478910b94a2525401b2642bccec530", + "result_sha256": "fca49d1de4a1905da389c1cef4610aee77922b07c9c45ea909d59d69c71cb0ab", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-21-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-21-en-m0", + "case_id": "fresh-21-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "7aecd8573c91f58be2516861f2cc556a7c64c7bde791dc3118ba8280219728d5", + "request_payload_sha256": "85d9c2211e0a760fe3fc9eaa52474a991fcbbae529e24950a0a4722faeb9e20e", + "receipt_sha256": "bf27f95b3db27c5cbffeed545a7a23fa32067ef4da46b1be43dc905b82d1ac3b", + "result_sha256": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-22\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-22\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-21-en-m0-source-chronological", + "episode_id": "confirm-fresh-21-en-m0", + "case_id": "fresh-21-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "805f3137eb7208c797b66e3ec37b4a885d300e3adc1c566277b5619f557af930", + "request_payload_sha256": "3ee83e786c8d609fbacdcb87a8a6ac1217cecef419dbc813ec3f3af89d4d7664", + "receipt_sha256": "0644e286da85dfe6126019cc11e15b08ba9f2a8d7ddcb43fdef1bbbf026cc3d0", + "result_sha256": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-22\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-22\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-21-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-21-en-m0", + "case_id": "fresh-21-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "7aecd8573c91f58be2516861f2cc556a7c64c7bde791dc3118ba8280219728d5", + "request_payload_sha256": "85d9c2211e0a760fe3fc9eaa52474a991fcbbae529e24950a0a4722faeb9e20e", + "receipt_sha256": "aafa9d8288c877bbe46138dab2d34c6065b079b5ca39d32a5387945a9f5d6357", + "result_sha256": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-22\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-22\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-03-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-03-en-m1", + "case_id": "fresh-03-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "cd41cda1837648ac7c94de2ec313ce246073746ebde686ea49bfda9492bba8f8", + "request_payload_sha256": "48e98c2f5687c0484617023e6221fd39350d21932255c2459818215047ad0581", + "receipt_sha256": "eb349b3d06070eba03a3775a169ab81e346f8427308477c2e13998bd7d01c76c", + "result_sha256": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-04\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-04\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-03-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-03-en-m1", + "case_id": "fresh-03-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "59744cf5fc73cf19e6897185f44e860def19a4041fa76ab80c5bfdd648891371", + "request_payload_sha256": "00455e7fca4684a223d15bc1c7c9e13cb92320cc39fdc0a1ce8b91fa915ecbf3", + "receipt_sha256": "ab2bcc39f735cf34fd3d1237b138235b1d36923ab203fe0a679be630968c44f4", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It abstains although the source or actual receipt determines a date; Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-03-en-m1-source-chronological", + "episode_id": "confirm-fresh-03-en-m1", + "case_id": "fresh-03-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "c73250242aa32b7bd1dd9fea122dee1e29eab01a56d419519bf0f0d06b3f5f70", + "request_payload_sha256": "2a88bdf785ee8b73959c1566209c4b70bc660b177275bdd82fb6731ed0de1f54", + "receipt_sha256": "703dff5b374b73ba1cd17a6cb8d3d11d8cac38585f388e0bb9991ab3fc728ddc", + "result_sha256": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-04\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-04\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-28-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-28-en-m1", + "case_id": "fresh-28-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "90982f4e596ea6376c9a737577dce45b13aec4d0507fa97ca7af7415b6bf192a", + "request_payload_sha256": "cdb35397922a454a657e5ad76ae90e04e6da34f4634cee587739a3d0cb468f1a", + "receipt_sha256": "a8a66057108d89fe53850bddf10b0cae108a322521160569a97590a0f8c29908", + "result_sha256": "f70bc8efb6e5c2baaed099be3abea5bef92a529c53415c4cf1f762b65736aef1", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-29\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-29\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-28-en-m1-source-chronological", + "episode_id": "confirm-fresh-28-en-m1", + "case_id": "fresh-28-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "9945887171072a439d28278afd4b7ff8ac0ff699ced7d586b886c8ef00dc9ed6", + "request_payload_sha256": "90bfe1ab2ba4904b6d2f3c5f580c14c295ee199d900615d3667b059eec215ef1", + "receipt_sha256": "d187205b5e49bdd4b1488b0f9403c5ea9df682f5aaeb48fc8a5bebe96d8878b4", + "result_sha256": "f70bc8efb6e5c2baaed099be3abea5bef92a529c53415c4cf1f762b65736aef1", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-29\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-29\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-28-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-28-en-m1", + "case_id": "fresh-28-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "90982f4e596ea6376c9a737577dce45b13aec4d0507fa97ca7af7415b6bf192a", + "request_payload_sha256": "cdb35397922a454a657e5ad76ae90e04e6da34f4634cee587739a3d0cb468f1a", + "receipt_sha256": "f56d6e540247f4633148bf3b9961f70468bea4d19dc56b6de0707de2bea27fa9", + "result_sha256": "f70bc8efb6e5c2baaed099be3abea5bef92a529c53415c4cf1f762b65736aef1", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-29\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-29\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-04-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-04-en-m0", + "case_id": "fresh-04-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "bce4fc5a03af6f51665bdf59993ef95dad845853c5546910046042079a7bba5a", + "request_payload_sha256": "6c0b9370e55c21053280d39eb91394eff01e71fb7fcfb21beb3bd01e5d20f00d", + "receipt_sha256": "70b413e29dacedb5ea9f4e08a72d8eba61e3ee7471746b4557dd132e648208d8", + "result_sha256": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-05\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-05\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-04-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-04-en-m0", + "case_id": "fresh-04-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "7bfc34dd08bd92f5c9547b6a6a448d58823a1b7640f20d23c68d4c9610964e8a", + "request_payload_sha256": "36dce940f03e925a100ed07c266af120c197eb2376dae8cafa0900025469689b", + "receipt_sha256": "5e535665c07c64bcaf1ab7a08a8ecd2e40677e3e1cce903469db33c3b7edc382", + "result_sha256": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-05\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-05\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-04-en-m0-source-chronological", + "episode_id": "confirm-fresh-04-en-m0", + "case_id": "fresh-04-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "e8b74cb9af2e31685df83cecb36906643a56a5fd3f5ddfa1990c617ebdcded85", + "request_payload_sha256": "df710aaee325ae349f198f4c6fdca3c6a98ef0b22a667a49fffcef00f94a75c7", + "receipt_sha256": "af126488ee96cbf478c662d05e63e4177a658296bf12eb89fa2e0ac23f71776a", + "result_sha256": "2a19fc24a0e2b5e27aaeea6228e6afbc6769930f05b80ca6a48bb9cd4b53b495", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-05\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-05\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-05-05", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-05-05", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-16-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-16-en-m0", + "case_id": "fresh-16-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "97ebaebc45edaa4bfa120497f3096f36384eaeb3e08384faafc6fca4baeae396", + "request_payload_sha256": "2e42fae5a5c1889e54fa7579415a3c297d84f19c5a2c3c3c75f27d58d39d9e5e", + "receipt_sha256": "26b746635c1971e2efece67a60139366f23490ec340f923777631f69c58078ff", + "result_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-16-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-16-en-m0", + "case_id": "fresh-16-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "ebbeff44143cca94378fd3ae74280c505a1ba29af9deee03ebaae1664c3843dc", + "request_payload_sha256": "4ebdd997e835044d36f61a351621ddba99008ee09ddfba9d4024aba376efef08", + "receipt_sha256": "9ca16e32a76f15a9f65f52b1c4d5c7bb48050f3dc6d0935c3b1261d1abfd1b88", + "result_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-16-en-m0-source-chronological", + "episode_id": "confirm-fresh-16-en-m0", + "case_id": "fresh-16-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "734188ff2216fc9a377c2162a0f76cb6363bcf25e3ce25bd7da8e04809d5180a", + "request_payload_sha256": "1db7ac515e8c0065c0ec2823772c69dfe9c553130c014a9857d05f20cd8f6bf7", + "receipt_sha256": "945841d1b62ff6970a9f4bbb4798d4a33095ac15db1cf7647019161fec8254ff", + "result_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-15-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-15-en-m0", + "case_id": "fresh-15-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "2b756b74e64c3c31fdc56c41e18cffc5cfda3520d2b2fd92e4467c027e1689b6", + "request_payload_sha256": "ca7f580c9b1cecfb3f0180af4c343cd54974c2583f9141ee4541bfc5518e1c3b", + "receipt_sha256": "b37a9bcc8bd1c532d44e167dfd66c9c493081c8cf2896b539c7c73fd02d41ee9", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-15-en-m0-source-chronological", + "episode_id": "confirm-fresh-15-en-m0", + "case_id": "fresh-15-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "36b1e13d273d049c1c91de4b70251c5d412a6939f5d700ced016e3ad91e13184", + "request_payload_sha256": "029081d2e5e356f78427cdae9d47dd609b6abf1b47fb71d1dddedd4d2f554caa", + "receipt_sha256": "3999ee223844e1d8ce3dee0a617991011f5db1279f4574784cdf48c2d520f0a7", + "result_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-15-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-15-en-m0", + "case_id": "fresh-15-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "cfae3931a55972bfbf68a36b88ba8ced7e1cda5da42b9981373a605aabc8a774", + "request_payload_sha256": "8ad2afdb819b06aef7f6ef1e0039161cda52347e63aebe6e43fae7a73102fd8f", + "receipt_sha256": "ec89890c29a4c32a1c28370e5178c8b3c54586e28d0411bafae2ecc6bb5b73ef", + "result_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-24-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-24-en-m0", + "case_id": "fresh-24-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "1b257db4229cd6a442c2444b7fab40c134f7e76ab4902f84b3a92e119bc552e6", + "request_payload_sha256": "3b28c1674d64f3107d9015460fb0ed218448e07ce1ac58d10f490fe6fe49eade", + "receipt_sha256": "d2a0bbc8b8da0d31e2b9b5461d17d2650d387400ccb43f253f01b1068207d3a1", + "result_sha256": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-25\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-25\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-24-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-24-en-m0", + "case_id": "fresh-24-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "1b257db4229cd6a442c2444b7fab40c134f7e76ab4902f84b3a92e119bc552e6", + "request_payload_sha256": "3b28c1674d64f3107d9015460fb0ed218448e07ce1ac58d10f490fe6fe49eade", + "receipt_sha256": "d36c0c526c0c6b58796643f9ce0b0e86eefbc950f447633c0caef0a110c37df4", + "result_sha256": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-25\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-25\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-24-en-m0-source-chronological", + "episode_id": "confirm-fresh-24-en-m0", + "case_id": "fresh-24-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "2b3fbb2beca1a8d2a0bb7efd9570af575813abe5323e04c8f46551adbe508c2d", + "request_payload_sha256": "bff0d21ac3558ee4d433f7c40a00faf0c0fb2d88dbc6836f7505e888885abbfb", + "receipt_sha256": "2253b09fc86def85eca603d8dfe5933c7bc1cd342ae1416ab760ab04492957b1", + "result_sha256": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-25\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-25\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-08-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-08-en-m1", + "case_id": "fresh-08-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "db7b665e4e6399bf1ece30e81b8ebe4b984f809229974ffe56161c86aa2d0967", + "request_payload_sha256": "8e250e7f6503cddb61f67bbaae10e46558294eef4af3d0beca3a04ba13039dab", + "receipt_sha256": "2f7bc02219350bc25b371b6db1820abd835887fadd2db156c06084aad9687980", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-08-en-m1-source-chronological", + "episode_id": "confirm-fresh-08-en-m1", + "case_id": "fresh-08-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "ffabcff2961d8aa6e641b9dd56268c73a837519b9ee8a1fb71b5a3b882f0b5f2", + "request_payload_sha256": "06ccea922e6d04b64fd30ced64879931dff87a8345e61ce37b88c226994c500c", + "receipt_sha256": "7d54337c5235e3ec6076ec33934daa2add33fd2036cde2fb2df2b0f1e816878d", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-08-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-08-en-m1", + "case_id": "fresh-08-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "8ec3122d2be886044fdf832428aeaa47083a774e18c811b6500eb76e4571c55e", + "request_payload_sha256": "33a26a9a48c790d0fd94395a59a23c952d1831c8e50fa4863a6646251d921c7d", + "receipt_sha256": "c540bc74edfdc47bbc13113474e68cdddf5759141c51f3f512e3a950bf365a18", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-34-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-34-en-m1", + "case_id": "fresh-34-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "fca0bf0c8fdde1f0eb14df8b7614f9aca82abff41a47d78488fba0d73c503d42", + "request_payload_sha256": "016aff47d36456227dde34042105f5a90d98cecacf187016e4fb0a8e3a142591", + "receipt_sha256": "84be8c9640db8a0b976a812a9f1984a6c5222256d84aea46db2bf6c683c9bd83", + "result_sha256": "042713423da807bcfef49b2ccab9db9d7aee0d0c696115936fca95c315d2e3bb", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-04\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-04\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-34-en-m1-source-chronological", + "episode_id": "confirm-fresh-34-en-m1", + "case_id": "fresh-34-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "3bf80f107ae07c961fd29167af72ba222b431a2ea7bc491be7bb92c8eb40ccef", + "request_payload_sha256": "3ef036ed7a6f924ea4197053f157b101f4d34273be4cb4f971dfbedef73577b7", + "receipt_sha256": "5e67cd5c3e4dd73457214af2d13c86b7a471103cf52d1279fe31abd8300cca5b", + "result_sha256": "042713423da807bcfef49b2ccab9db9d7aee0d0c696115936fca95c315d2e3bb", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-04\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-04\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-34-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-34-en-m1", + "case_id": "fresh-34-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "3ebcf2bb3ab24d387bcfb2d28b0342fab2e3692c8e06b0160d4bd380359f6b84", + "request_payload_sha256": "9080ba2655f9b70f1edd9b813b9e0c24e6f35876fcdba78eedbc59551a77011f", + "receipt_sha256": "62f7095b4a5b746a59a7008e4e6e87a2608a65aa51cf3e973a3a428d8930ee72", + "result_sha256": "042713423da807bcfef49b2ccab9db9d7aee0d0c696115936fca95c315d2e3bb", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-04\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-04\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-37-en-m1-source-chronological", + "episode_id": "confirm-fresh-37-en-m1", + "case_id": "fresh-37-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "573f1f6fc1bf60b3a346a7989627665539b33a5e800797730afdc688276db0f4", + "request_payload_sha256": "9e129e41a4101a77959a9f8e35e392d96fad2112180896ac55cefd9d2d39fac1", + "receipt_sha256": "a67ead4dcce123dec8ee539bf257e8df6fd9f4647804a259e9781d2baeba271f", + "result_sha256": "81dcf61047152bcf230429686bb9e10dd1e72d301365218d0e5c611c0810b6ec", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-08\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-08\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-37-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-37-en-m1", + "case_id": "fresh-37-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "e551b11efaef90b7d3fae9a6a8aa545b9fdf2d07328a1d14a7d68937d45b8691", + "request_payload_sha256": "80d396248c4e938adfd8f9082dc789bb0e47803b7ec9bcf130c0788740dd382a", + "receipt_sha256": "96137145690a3dd4ee6edf2a3e92482567259dd7cfe2babad60343f4fe2cf281", + "result_sha256": "81dcf61047152bcf230429686bb9e10dd1e72d301365218d0e5c611c0810b6ec", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-08\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-08\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-37-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-37-en-m1", + "case_id": "fresh-37-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "023c3610873e35baf156f3ed8e57905b9a20d914358badb6fb6f9fa8db2a46bc", + "request_payload_sha256": "0ee8afc566c18595f1e982caf8cba213e739036ca06cd0269c52c2fa8f7506c7", + "receipt_sha256": "5e836726a353647fa4bed8f94129542514660922c2b52b4409152ae328794395", + "result_sha256": "81dcf61047152bcf230429686bb9e10dd1e72d301365218d0e5c611c0810b6ec", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-08\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-08\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-10-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-10-en-m0", + "case_id": "fresh-10-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "6125adfe2a6ebecca961c0077ca4e0cf43156511c60a21d46a09e3dd72c3c1d4", + "request_payload_sha256": "0aca9c8f33fb1d4793e266ed3daf54be1e574d8b9bc4010b86d0b740368bfffb", + "receipt_sha256": "215ae61057720a813152a44735609f5a651de92c9f65e89449d045e821bbc40d", + "result_sha256": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-10-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-10-en-m0", + "case_id": "fresh-10-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "4269735326c8081466fd5fc63167f5f7eaecc1c73cc41120a948a11229daebde", + "request_payload_sha256": "ca71a4345e023f2ed10f3e63b0e1160224576599199da47250e97bd1cfc6e707", + "receipt_sha256": "73b89b9153610287850dafbc18cd1016252c1c6e9ba116c537e15b46f9a5b324", + "result_sha256": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-10-en-m0-source-chronological", + "episode_id": "confirm-fresh-10-en-m0", + "case_id": "fresh-10-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "df53ee1d0d3add3b4d1427bf2d710e61f955eae44a46f6df92cfa847bef87559", + "request_payload_sha256": "c7a841b1b55410cb7cac456f93334879c6ec357953812a2c3281aac63e3673ad", + "receipt_sha256": "9d9635a4f54160023d075ace30c99abd7ef20bb53f4b0692b9d4387b6c9b71b7", + "result_sha256": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-09-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-09-en-m0", + "case_id": "fresh-09-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "13727491605cbf797b4e3d3509dd7a8b540d9627bdb83bf6a66f0ac8e6731b8d", + "request_payload_sha256": "c0737a616cba2b1dcbb05fbc1cfef05019b02f3099233629e955c11155842a7c", + "receipt_sha256": "a07c85653264eb0a9c2b536fff59d3c3f57b654419b8d09975804afdcd60e3ba", + "result_sha256": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-10\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-10\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-09-en-m0-source-chronological", + "episode_id": "confirm-fresh-09-en-m0", + "case_id": "fresh-09-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "15e4f92266bafd2265de19c897462bc45c8c0bf46c1ef3511b6226ab31812aa2", + "request_payload_sha256": "421d282b78cb3cd935b0ad6487a8a391293d4ebe96b6b6ff102e959d34a83f72", + "receipt_sha256": "294b12a89571e87f680af03e3c9dc2e746b2fa4514ddea5c37352f3c32710a4a", + "result_sha256": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-10\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-10\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-09-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-09-en-m0", + "case_id": "fresh-09-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "3554ce7f3b82a1659263fb32902a3eb74e11bcfddf2c349db9f85cfb8479e194", + "request_payload_sha256": "7c21318ed75cc914850ca593e5e1b02ff63c615ac58ab0eb118eeeb696de70f1", + "receipt_sha256": "6a0dc2afd71e04d67a800d2bf0aaa00e31dc8e69f6b7cb67c1bbda5527599e7e", + "result_sha256": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-10\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-10\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-39-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-39-en-m1", + "case_id": "fresh-39-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "4e824903cb30d547519200547829673181c4c6ecbb5041fa1aba95830aed4169", + "request_payload_sha256": "ed2cd8d4f54316e90f71aefef3b38ce57b11c034da236ac82314bff7dd3aaba1", + "receipt_sha256": "98b24a10dab5ed8afa77cbf61cdaff3aaa255e879db333ecdbf33997f10f6af5", + "result_sha256": "224a2b7c56580f10e75f047d220d6d5e3e00464466c7f1ea4da77967b30e4cd5", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-05-10\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-05-10\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-39-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-39-en-m1", + "case_id": "fresh-39-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "1e49526931307f14badec22700ae97dc21c5c0bba69f5c186ea2da73c50cf389", + "request_payload_sha256": "f56527a02df0a8c3823c4a726fb21b43035d94dec1098cb67d6a0fe3b23366fd", + "receipt_sha256": "086ab37ca327b69bbff87fa0c59b6ea6eee64bb78ad7216fcfa69b4f67543af8", + "result_sha256": "f2371cd3e60dbad81739ea12fffa281dda1158ef039ec6a8984449c96e971832", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-10\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-10\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-39-en-m1-source-chronological", + "episode_id": "confirm-fresh-39-en-m1", + "case_id": "fresh-39-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "3b51bb88f2967343a0faac6ffbd52111f58644d75b808bd4f85b0b29c5604ee8", + "request_payload_sha256": "ac735b507abfe89000c2df814df75d198cb55bfd726f58368d7395f81d771fd1", + "receipt_sha256": "096fdad28a1be90841ab601a476d1043cffe153e9363a6cb79ff3504663658ad", + "result_sha256": "f2371cd3e60dbad81739ea12fffa281dda1158ef039ec6a8984449c96e971832", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-10\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-10\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-21-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-21-en-m1", + "case_id": "fresh-21-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "32cf8ce9b7ddab90245fdaa7ceab52a690bd9758973c66beef3ff7231f8e12b3", + "request_payload_sha256": "98e2d8e9418720324c235308c5089e484887f88ff04471e028d85297940d8b72", + "receipt_sha256": "45c8aff86c4dcb2ccd92b6d81cc0d5176442ec39c4c346b0ab80298281fcf6a9", + "result_sha256": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-22\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-22\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-21-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-21-en-m1", + "case_id": "fresh-21-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "32cf8ce9b7ddab90245fdaa7ceab52a690bd9758973c66beef3ff7231f8e12b3", + "request_payload_sha256": "98e2d8e9418720324c235308c5089e484887f88ff04471e028d85297940d8b72", + "receipt_sha256": "20ba4cdb16dd870cddda88d68f5197c79086950d2eba437efba62faf399e41ae", + "result_sha256": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-22\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-22\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-21-en-m1-source-chronological", + "episode_id": "confirm-fresh-21-en-m1", + "case_id": "fresh-21-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "94b54826e622e09003065b0b02c84680861f90b936def6a06851659973916635", + "request_payload_sha256": "74d8511e594c968813ba68e5c3d1a1cec9142fd2185af8687c1ff3aea43c1e9a", + "receipt_sha256": "b8e9d39d9c33a763ff7f0b7fb28af6e6591ebb7cb721cb6ce1f913b8aa5d22b3", + "result_sha256": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-22\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-22\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-19-en-m0-source-chronological", + "episode_id": "confirm-fresh-19-en-m0", + "case_id": "fresh-19-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "77714da6d1a185a0fcf05eb493e1b6fbd063ee5028affc58a33def5e872b21d9", + "request_payload_sha256": "9d84b8d6663451c2fcbfadf76451b64db0658b16bfff6c8dd5b12174eb0083a7", + "receipt_sha256": "68c631e25cc340972828ecd240efe6abdb55a261816cd8658b7bd1f47190a011", + "result_sha256": "4f2ff4c6daf1623e66caead1eec254086b77631cbd7ac1df8765b25a639a0c24", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-20\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-20\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-19-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-19-en-m0", + "case_id": "fresh-19-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "d92c46e95e2f8c7898dc975a498066a3b9afcf546932d0a11d5bd98976b115dc", + "request_payload_sha256": "5fa7a45f197242831642ebd122a71e7c5bcd19e2d6a02042fbd837bc64de8d75", + "receipt_sha256": "a4cbfe6bef309b5f60885b7bc15c9e30f926f4d328b09f36f5c684e223c22fe5", + "result_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "http_status": 200, + "provider_choice": { + "finish_reason": "length", + "index": 0, + "logprobs": null, + "message": { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": null, + "finish_reason": "length", + "tool_calls": [], + "parsed": null, + "terminal": "incomplete" + }, + "raw_answer": null, + "projected_action_status": null, + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": false, + "projected_status_correct": false, + "valid_final": false, + "invalid_cap_or_refusal": true, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "No usable final JSON; source question remains unanswerable from this output." + }, + { + "id": "confirm-fresh-19-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-19-en-m0", + "case_id": "fresh-19-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "0391c2fe09cecc8739c194c950223bf638349a2d52733791fc7ceaf81c9cc6c5", + "request_payload_sha256": "813580eac63a7aeb5be0d10dc1aa59f0e7f7624ad02e4f853b9d69a0f305a2e6", + "receipt_sha256": "3f55069ed5c8213374a7db1f3d48ce2a5adbaa902e9a960da5a8b3d885222085", + "result_sha256": "4f2ff4c6daf1623e66caead1eec254086b77631cbd7ac1df8765b25a639a0c24", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-20\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-20\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-35-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-35-en-m1", + "case_id": "fresh-35-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "1118e44fc5b40d17815ea6771846b1255a6d0d212e2ed0a30183153b8f5b4ce9", + "request_payload_sha256": "63713daba10d96ead27e9bb4215f8ca7e5b797d19d609cdea9bec5b8029bef6f", + "receipt_sha256": "2ea68c5c44650962429de8643a46141612748001fcc02a04c8a596b6c3d58c90", + "result_sha256": "10dca089506ffd53f9cf8d47ddf457b0596b2a838a3856eebeac46979cd679d9", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-05\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-05\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-35-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-35-en-m1", + "case_id": "fresh-35-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "8ab95da936dfe1e72cca058fbcd775382eaeee566f80b4c1418be0d13ca06563", + "request_payload_sha256": "540e7f7ea043bff1f70be1b82864225cecfe11e38729fd5cbc38645c65bdbacd", + "receipt_sha256": "dbc71e7c4b6b325e549b618e36c79b59d7b753db320a3133292ef17d66c39001", + "result_sha256": "a10e59f23e1bcbeb10d0bb5925482d8103b4a143246bb28f303d672b12eeb2f2", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-05\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-05\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-35-en-m1-source-chronological", + "episode_id": "confirm-fresh-35-en-m1", + "case_id": "fresh-35-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "6a4ac60cfe130bd47b576fa292a8ccadd1c9a89f5ae67ae309137d5f956d3b67", + "request_payload_sha256": "f7c61401e5010313ef1d59863db090b0d48bf5d5e8b87e75a61638de11fe129a", + "receipt_sha256": "7b026a350a5dfa135a86989033e7f649fb292dfc566387ebcc3f92924bf51726", + "result_sha256": "10dca089506ffd53f9cf8d47ddf457b0596b2a838a3856eebeac46979cd679d9", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-05\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-05\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-17-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-17-en-m0", + "case_id": "fresh-17-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "0a09f989d834833554035e31b86f33ac54aa972b6794bc45568f4f58f78f2f12", + "request_payload_sha256": "240630a9598d016e9d65b9392336f7425829d77a67702d60bf00c28fbe951817", + "receipt_sha256": "49e02316ddf4b848aca168797301fb198c8fd73a8e7dd9e54cf26f16106b70d7", + "result_sha256": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-18\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-18\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-17-en-m0-source-chronological", + "episode_id": "confirm-fresh-17-en-m0", + "case_id": "fresh-17-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "b6820e2755eeaec0edcf87bcfac240b90a4ae620d9725f4352ec62e34b33132f", + "request_payload_sha256": "f5ca8b2855153249cbd34e7e1821c42e4e89c5a8d165ce1cbc0ddc39b0b0d604", + "receipt_sha256": "8464c65d5db1959f62b4e1acb22e0d192077c68f03fe115d22c46b3c836246c3", + "result_sha256": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-18\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-18\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-17-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-17-en-m0", + "case_id": "fresh-17-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "269fbf6771a8cc47ea613e10791e7173c178ceb8ccce57985fdf0de0c7c5b2dd", + "request_payload_sha256": "901b968a6c025d7e33012985f2fc43d454da1904abf68902b7d9bca63e9ffdcb", + "receipt_sha256": "d6077fb24ca5bf649a2f4155d2e49ba3097dd4ff18fa02168113cacd23818140", + "result_sha256": "371aa53658864bf121a8519dd1f82296c9c5bea4c7636988c6b8bfe16335e6fd", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-03-18\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-03-18\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": true, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It substitutes the backend's current date for the customer's preference." + }, + { + "id": "confirm-fresh-38-en-m0-source-chronological", + "episode_id": "confirm-fresh-38-en-m0", + "case_id": "fresh-38-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "b1aa1ddf2b014a4d3faeb02ebe74fcf06b9fe6fd24b90fe591ff4c03b911e923", + "request_payload_sha256": "f14c8d3695c98778c1a6aa87fd8eb86052bdd9c118b45a57e99a8b6e06014a11", + "receipt_sha256": "28b947a2de055ab96d80df20653539111a1ed57036f680d4711cb000535f3069", + "result_sha256": "7179a53ab85d138478748672aef58d579781c7908dfb61f6e82b564ed4e415a5", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-05-09\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-05-09\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-38-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-38-en-m0", + "case_id": "fresh-38-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "c7c5fbd365d0825271746127542dacf06169a63c56815f7a460fcab8a21b40f8", + "request_payload_sha256": "d6568a7ca498299c5f2f3d441e5f53ce35ff3e1d6ef115fc0209cfde31b2fa66", + "receipt_sha256": "e1c131b71e091558ecd4c7b57f737dfe62565c57c76e1461f50b4470c1f745e7", + "result_sha256": "a83af413afe7cddcaa541b7076f09d4924ecc2e0a192a002771a74e3c3fadc75", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-09\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-09\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-38-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-38-en-m0", + "case_id": "fresh-38-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "583f55c84772fc687ca38a4978338af9fd00c8ab06552caecd276022bebe5e54", + "request_payload_sha256": "afdefddd35f8e03bff167b1f4335c9b9385491abf65079268f15d9418581e534", + "receipt_sha256": "88ba55e6b2fce0785b4f2c2d4424f05d1d915a3182c8833f15ffbcd254e3c43b", + "result_sha256": "06c2187ceae2c64dea1bf91ec923b84dd3573f34714a5aafeb636bfe711a773e", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-09\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-09\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-08-en-m0-source-chronological", + "episode_id": "confirm-fresh-08-en-m0", + "case_id": "fresh-08-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "904d31fd48d3236a117617448a52226b741393c3caf6e43e752ad90a26d03dcf", + "request_payload_sha256": "e5c440cb22dec66b24a8c28e9bec8fcef59be88503fb92a8ae6801f9e62dc1d8", + "receipt_sha256": "e9a81bc742e1440e16f82c3f60538a44db3f7496f06870d8951ba484339763b2", + "result_sha256": "dd1eb69535233b7875007bb6fe801cf730d36ee48921e94b961cca63df5956d8", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_performed\",\"answer\":null,\"needs_clarification\":true}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_performed\",\"answer\":null,\"needs_clarification\":true}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-08-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-08-en-m0", + "case_id": "fresh-08-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "12113f016e61b549769d3f2132f26711eadc3cdc8f79a8938f983c0d56206e2a", + "request_payload_sha256": "3a71b4c7eb40b1f455e8aaa80ed6edb77853e544d55e02091e13999b6d56ab31", + "receipt_sha256": "0b9fb236dd4e8c0bf3acefa1584e52561eb50993e8db0be85b9fa6475eeef9b5", + "result_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-08-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-08-en-m0", + "case_id": "fresh-08-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "e7b24ce34173f53a85daf0e3611c29085b9b5808a0fa37e02af7f8d5990f5d83", + "request_payload_sha256": "21bf8283442c6333ce7e84a713af4629233e1506234e809fea789b38c64e4e57", + "receipt_sha256": "a559f95302aa096b86e5623af9263eb4cabfbe51878471771ddd4830dc8871fa", + "result_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "http_status": 200, + "provider_choice": { + "finish_reason": "length", + "index": 0, + "logprobs": null, + "message": { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": null, + "finish_reason": "length", + "tool_calls": [], + "parsed": null, + "terminal": "incomplete" + }, + "raw_answer": null, + "projected_action_status": null, + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": false, + "projected_status_correct": false, + "valid_final": false, + "invalid_cap_or_refusal": true, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "No usable final JSON; source question remains unanswerable from this output." + }, + { + "id": "confirm-fresh-15-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-15-en-m1", + "case_id": "fresh-15-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "2f85b2f32af8a7745101d995353e5f5a5a926f9fb3c9c399775067afbb2c5d40", + "request_payload_sha256": "d84bd9fdab1c05ad62056d974e955b655359de7506e89e82526bc12c759f612a", + "receipt_sha256": "2e5582aca58d610ce634b4499bce24820724c00df032b756fc55a4fd74b11428", + "result_sha256": "1221a7e341e0ab559d66b45f568b82184bacfdecaa04abab155224785776dcee", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-16\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-16\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It selects a definite date despite an unresolved source target; Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-15-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-15-en-m1", + "case_id": "fresh-15-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "ee762b208335bc5121ae8f8bf8971ee21f20262fa3ea8c60159192ab65e43686", + "request_payload_sha256": "0e67f2c2fde371609f624f4360e9e67799efc7786d9e5937d9620e1ab6395a9f", + "receipt_sha256": "e52c43b622e8b08390e62838e9efc0114989f240d7b61ee9208b1dca51e609f2", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-15-en-m1-source-chronological", + "episode_id": "confirm-fresh-15-en-m1", + "case_id": "fresh-15-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "eb746f3821ff43e9c1333554e26c811c115addc32a2b06ef59470a479440aa7f", + "request_payload_sha256": "b582c3463c810305c2051f2b094ef08c611e0bb07f868a8cf6e2c8882ecef0dc", + "receipt_sha256": "6cfb01ed2c77c7ecd441e5df7d5525d48a95cf984a6eb5eb39d330e62b11dbda", + "result_sha256": "1221a7e341e0ab559d66b45f568b82184bacfdecaa04abab155224785776dcee", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-16\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-16\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It selects a definite date despite an unresolved source target; Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-27-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-27-en-m1", + "case_id": "fresh-27-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "2a4ba997cd213fb649188dbe4137f0ab8f0673c96a9c7962f8d9adf15219d688", + "request_payload_sha256": "88e80c636f569cbfd35f82e7606a6a48757f2f9f558e28779795b5d6429e561c", + "receipt_sha256": "f9077859f84c7e7d65be84f7fd1dc47a3cc4ddb97426b765f51f6a04877319ca", + "result_sha256": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-28\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-28\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-27-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-27-en-m1", + "case_id": "fresh-27-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "2a4ba997cd213fb649188dbe4137f0ab8f0673c96a9c7962f8d9adf15219d688", + "request_payload_sha256": "88e80c636f569cbfd35f82e7606a6a48757f2f9f558e28779795b5d6429e561c", + "receipt_sha256": "9252641b04295bba4d130f5ca5e45677c03e13529445ef5a22baa6e39996ac64", + "result_sha256": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-28\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-28\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-27-en-m1-source-chronological", + "episode_id": "confirm-fresh-27-en-m1", + "case_id": "fresh-27-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "05c108ef811544b73e3d92e5f080bf8b3cfc1e5618b701d5eb621ec20b437bfb", + "request_payload_sha256": "cb216f161ac06d53f1718d3b4d1d1c4b3504e2472acf7e47a9cb474b2cfbd2da", + "receipt_sha256": "78b600bbb1646f5a9cde0e86ae2071e1dafb4fcac96002cf134e3722766d5dce", + "result_sha256": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-28\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-28\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-20-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-20-en-m1", + "case_id": "fresh-20-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "c27430e9cf808dd00a2a46ce3702f99b60e2809396ec48d2bd0ab34efbf0f33d", + "request_payload_sha256": "4b1cefc02e478df99788fe7e3113cc89b72a519037eab74ace988401d4268fb9", + "receipt_sha256": "093a5b7dcf913bcf66de7cf1027d764c2267338c30edcbddee6f606d729c5461", + "result_sha256": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-21\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-21\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": true, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It substitutes the backend's current date for the customer's preference." + }, + { + "id": "confirm-fresh-20-en-m1-source-chronological", + "episode_id": "confirm-fresh-20-en-m1", + "case_id": "fresh-20-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "db170b073d6d9d96fc7783e738c849903c1fb430b29fda163b3eb683ee979ef7", + "request_payload_sha256": "ae697262335c37de63d672947d15ab1be30eb20a391cdf0564fb9beabb0ea4d4", + "receipt_sha256": "b5dd3062f8e719a0c24d9ca3f3d8fe414fe79e1536829f0601e21afff9d7157b", + "result_sha256": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-21\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-21\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": true, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It substitutes the backend's current date for the customer's preference." + }, + { + "id": "confirm-fresh-20-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-20-en-m1", + "case_id": "fresh-20-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "7796e42ecb0e452bf3ea3c5aefb7f3eb7cf3c00b1f6a39feece2b59d2f9545ac", + "request_payload_sha256": "5a8f423d42d41ee3445970fea66fd7e6b37c412b1efd2e83d198a8f73b9d0dc6", + "receipt_sha256": "8c1c9926a5c0aec3d0b190c60cd491e06861b1c56c1e3899fdf9dc5f69f4b13a", + "result_sha256": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-21\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-21\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": true, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It substitutes the backend's current date for the customer's preference." + }, + { + "id": "confirm-fresh-40-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-40-en-m0", + "case_id": "fresh-40-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "70d9f3b3aec1ef8b8defe83f284a78d78ced025e37032ceb0e63eb584cc2382d", + "request_payload_sha256": "9d4f12855d994c1898d484f557c2ac498fe301bace7476ae273e8f05f14ad334", + "receipt_sha256": "b5a910807ab430ba5300d3608c364968023c9a7dc137b21c238a4c2350b8e3a7", + "result_sha256": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-40-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-40-en-m0", + "case_id": "fresh-40-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "2c9cf2ac4f361d1769f6c5ec953bd01dbb77ea4420b46528daab4d8c75acc039", + "request_payload_sha256": "e257e61f1d3c8a299b9f7a12d52621fb4d8a006af9b18df3c500d5e9dfe7d05d", + "receipt_sha256": "6b4644cdc790598c179c922b9eb60e2121e4bde4397289b1372932a8e821a71f", + "result_sha256": "6294b9020f2db793978af4e410257f125f9a2791382e29e36b9e2496f58ab189", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-40-en-m0-source-chronological", + "episode_id": "confirm-fresh-40-en-m0", + "case_id": "fresh-40-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "3435dc3febf8342dad5010f251a1b6ce2dcc86cfa61ad95a93042977c202c255", + "request_payload_sha256": "7617faa5f117670852c8149e55e234af6e6d7d211c6fe51af9e2130d8d1bd4cd", + "receipt_sha256": "c3d74b9f330784ebf6c4ea7a936a550c5549dc4d3e2ed26bd7b3d4e0b2195e48", + "result_sha256": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-01-en-m1-source-chronological", + "episode_id": "confirm-fresh-01-en-m1", + "case_id": "fresh-01-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "f56e0630e491e732a45f86a681eb84c9924ef887fc33c2b0c972a9e003679277", + "request_payload_sha256": "3655e8829208a5b67b4698d59710c7907e141b0f9e8e6f45babb9dc63342778f", + "receipt_sha256": "b8200d0282cdafe33bf3a421ae1e54a8fb5f5ebebc99ff73404cbc72e3eb387a", + "result_sha256": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-02\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-02\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-01-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-01-en-m1", + "case_id": "fresh-01-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "ec904746716597fbbd1d35bf323776d173cfb8d3a033d3339c7fea288f41c508", + "request_payload_sha256": "fe869ed6dfb1521f5e1bf84abb6fe8fbfad77590cef3b53d8ca814b8a3cc05b0", + "receipt_sha256": "7ac191ca0db13faf0d0143fd0b8b6dbb3a9f443f92700399d195631369188062", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It abstains although the source or actual receipt determines a date; Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-01-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-01-en-m1", + "case_id": "fresh-01-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "d7a809c8ceff8887d796749858050d8479400ce03cb4872aa5deaaa118fc5627", + "request_payload_sha256": "504228b304323a1dbed374958fdf6e9b64e560e3650fac36072ec435be79a671", + "receipt_sha256": "03cd95b5b82e0ecb118371a7ef56d38cb00d479a2122daf9f98f92b2d88106d9", + "result_sha256": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-02\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-02\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-13-en-m0-source-chronological", + "episode_id": "confirm-fresh-13-en-m0", + "case_id": "fresh-13-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "158cd6269b7d56c78e2e85456d78c61323925f4bfc1aebd1dcbc471b71d4f3bb", + "request_payload_sha256": "b1454fbac58b6c7e33d3c5826bb045ed5f18605722dd9ac921a3be29fa02db76", + "receipt_sha256": "346c75390d937acd3c9a8595db24e8e0539f24d7b155e1b57838607884b6e47f", + "result_sha256": "dd1eb69535233b7875007bb6fe801cf730d36ee48921e94b961cca63df5956d8", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_performed\",\"answer\":null,\"needs_clarification\":true}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_performed\",\"answer\":null,\"needs_clarification\":true}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-13-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-13-en-m0", + "case_id": "fresh-13-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "d16b1fce362726bfe9f99729b626a57ecb7eccb393d5e34224c2ca7406f8f8f3", + "request_payload_sha256": "f342614e5ae4c3847cd355953586fe0c93a7372c4726b6d9bf5fbdb3130c1ce4", + "receipt_sha256": "0c57204141634f02165db629ddc7741ae3eebb6e8e19716db5cde82b68617caf", + "result_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-13-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-13-en-m0", + "case_id": "fresh-13-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "d16b1fce362726bfe9f99729b626a57ecb7eccb393d5e34224c2ca7406f8f8f3", + "request_payload_sha256": "f342614e5ae4c3847cd355953586fe0c93a7372c4726b6d9bf5fbdb3130c1ce4", + "receipt_sha256": "af1d2af5ca02c1b4fa1e836ab1e384ab2b2551d19d9cdef8925679d186580d2a", + "result_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-38-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-38-en-m1", + "case_id": "fresh-38-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "0e3541d137da5d5bb8461df015832a8bf8796b99a4f80c283ff396784c450c33", + "request_payload_sha256": "6223808ccfa3f98a9b059bb7b13a0c6c503309dc35d57927635af71ab02b0d0b", + "receipt_sha256": "2729e86eff3d177d9adc0251b6bf69226deb8680d793db26dff9bdaf61e4ae4e", + "result_sha256": "f6ce87a4a87a8702c1234da7b392f55d90d8148fb0470cd124de892151ecf6b4", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-09\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-09\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-38-en-m1-source-chronological", + "episode_id": "confirm-fresh-38-en-m1", + "case_id": "fresh-38-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "397e11bfecb3b0faefc02219a59eaa84d8ac33fdb3f87a2f8c41877841232d51", + "request_payload_sha256": "2d99b6b9efd1223f0b4fc6680a657fcbfed1cbcaf23c7ee6aab2c64b571fab15", + "receipt_sha256": "e24a361e7f9b5374aa0f0374edf8abed38ce2dd3fabc0ba7c9059c4c495a90d8", + "result_sha256": "7f2bd78eb4f64dd1c52dcce5e859d45a219d03cec1305e5fe4c87290e091d563", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-05-09\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-05-09\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-38-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-38-en-m1", + "case_id": "fresh-38-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "c4caeba56cb89ee2511376450db87bde3d1e8464ff68b83409e69da9452e98cd", + "request_payload_sha256": "83db7a9dd95b9c1fb397ca4bb50bac66ce28e1718c806c3ceda7dc720002b6c9", + "receipt_sha256": "54d5ab003ad3c065168abd7e32121877ed741cb797e55d80ff983360b1f71bb2", + "result_sha256": "7f2bd78eb4f64dd1c52dcce5e859d45a219d03cec1305e5fe4c87290e091d563", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-05-09\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-05-09\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-25-en-m0-source-chronological", + "episode_id": "confirm-fresh-25-en-m0", + "case_id": "fresh-25-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "1495ce5fadb503f407eb46b47ac691b3ae150cbd8e3d6f90b152a63fdadf5ff2", + "request_payload_sha256": "c2ccd25cf2e5b374bad5ee49f6c2944652c9b27f093afe316eb0e309127b80dd", + "receipt_sha256": "5bcf182561f27d21c8d7e3cc751d10102e3308284159792b4ee29c1d79514fbe", + "result_sha256": "258b73b129b801b662376d5cb0f39961843f2127df41664b4c2a54a8e81537fa", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-26\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-26\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-25-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-25-en-m0", + "case_id": "fresh-25-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "48458b25fe6f76327f15b6558f003abac47d012da8cca128c2730e376d4e34c8", + "request_payload_sha256": "56d636352639fb8f66ffc90c2161dbd6c580c2a7f303af3a7ad042629c532b37", + "receipt_sha256": "23616f107f145a6a402464823ccb4f6d999484152d2878001c6975f5cb0a25a2", + "result_sha256": "258b73b129b801b662376d5cb0f39961843f2127df41664b4c2a54a8e81537fa", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-26\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-26\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-25-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-25-en-m0", + "case_id": "fresh-25-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "48458b25fe6f76327f15b6558f003abac47d012da8cca128c2730e376d4e34c8", + "request_payload_sha256": "56d636352639fb8f66ffc90c2161dbd6c580c2a7f303af3a7ad042629c532b37", + "receipt_sha256": "9be352e09e90420732590829a8f8b0c8a6bd149725101ab1a020288d992cac2b", + "result_sha256": "4111452a57b7245759463a5c03457acf2db420c0a15100b459822cc754cadf2b", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-04-26\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-04-26\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-04-26", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-04-26", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-19-en-m1-source-chronological", + "episode_id": "confirm-fresh-19-en-m1", + "case_id": "fresh-19-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "2457f990a0fe0461adc4fa4c76d138d61bfc0db69cab4bfd4cdbc854028d8876", + "request_payload_sha256": "c45c57bdd4d6f5a4e2d608f26a35ced6c3e52c3fa267cfc34971eeef66c915d3", + "receipt_sha256": "450f05446335fd5827ff20fb721329a83a9bab2da5bdaa9b973e2c161368171d", + "result_sha256": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-20\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-20\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": true, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It substitutes the backend's current date for the customer's preference." + }, + { + "id": "confirm-fresh-19-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-19-en-m1", + "case_id": "fresh-19-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "edfe77ae8566f03cdd9c393ba617727b264c20ea2b1e3eed7ebe71831d53dcb7", + "request_payload_sha256": "d6d8f718fa3d000569f3660cad42756abd9999d8055f80b9cda19759228854a4", + "receipt_sha256": "f48834ce7f2f52e91e8ea7226c4506c19d9f8a52ab98faca3d32068e3c3f513f", + "result_sha256": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-20\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-20\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": true, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It substitutes the backend's current date for the customer's preference." + }, + { + "id": "confirm-fresh-19-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-19-en-m1", + "case_id": "fresh-19-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "8c3caa861262280867ebc02afb8ab9d07199555457a056800b4852c45bdd2ee2", + "request_payload_sha256": "a8afbb3ff36723edc9eca720f8b596fdd84206f524b5e3ea36812a074fc6f68c", + "receipt_sha256": "cdf8df3b164cc356119f6a8cddb59e85e594349bbe22d129d86e73ca64b70d20", + "result_sha256": "adc8ca3c25e704262c64eed8a14b5c143f1bb7391758f5d58f0af0507a0e8920", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-03-20\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-03-20\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-03-20", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-03-20", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": true, + "primary_factual_correct": false, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It substitutes the backend's current date for the customer's preference; Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-32-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-32-en-m1", + "case_id": "fresh-32-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "9bf63a7208c7a9d57cea769e74ba001b54c68af9f63594a9f902178b84ae8cb5", + "request_payload_sha256": "196b8d71f41b6913a8b8784aa53f120291f4d7af5cf6524f397fda2990e99bb0", + "receipt_sha256": "3124ee0bf50d934f4c72845cac8a3ffc48e0e55d6cca910725c44edf8b187516", + "result_sha256": "40234eb32747058b66be85c3240998e1ffc5c4add188004394236ae5121c013a", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-04-02\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-04-02\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "projected_action_status": "rejected", + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_execution_status": "rejected", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-32-en-m1-source-chronological", + "episode_id": "confirm-fresh-32-en-m1", + "case_id": "fresh-32-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "f74bf5dc9adba6a91b947fa6241ad689f0b33fc7df23c1ca8732c8c3fc1b3350", + "request_payload_sha256": "eae2e79d42bd77bf43ff42a65506a68bc2af0db3681334b6ed990da87160a8a9", + "receipt_sha256": "c15f7f74b9b5cd66fc8a1ba5506894197f80985686bbecdc3696d86059de11f5", + "result_sha256": "40234eb32747058b66be85c3240998e1ffc5c4add188004394236ae5121c013a", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-04-02\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-04-02\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "projected_action_status": "rejected", + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_execution_status": "rejected", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-32-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-32-en-m1", + "case_id": "fresh-32-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "9bf63a7208c7a9d57cea769e74ba001b54c68af9f63594a9f902178b84ae8cb5", + "request_payload_sha256": "196b8d71f41b6913a8b8784aa53f120291f4d7af5cf6524f397fda2990e99bb0", + "receipt_sha256": "b5d737df394814e5282291af3cc163b4c1e630aa19db6bf49217be0c95a255f8", + "result_sha256": "5bad8c2e7cf1eb0cb13f16233048fdaabe345340269ddf7d3e31079391a20ed5", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-04-02\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-04-02\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-04-02", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-04-02", + "needs_clarification": false + }, + "projected_action_status": "rejected", + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_execution_status": "rejected", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-06-en-m1-source-chronological", + "episode_id": "confirm-fresh-06-en-m1", + "case_id": "fresh-06-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "d5d345f9b277fb89d923490b8405eae73c3410cb434c0ced04de7afb1dbd131d", + "request_payload_sha256": "7eb8388e71b5b3edc8d2939593aec5d88dc89ac4cb35487e7278b9c273682c01", + "receipt_sha256": "4da276ed331db2a16c6114a5dce475d383ee3a5374c1bd8c362b75dca938d820", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-06-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-06-en-m1", + "case_id": "fresh-06-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "1c633ec7470e9f8807ed197c6c758304b707e5e62bba0be4e59bf9af2ef5530a", + "request_payload_sha256": "976e93476a1434d75dec47990bc38c5752f9387d4ad69fad36202c39348c337b", + "receipt_sha256": "e389a10166b6f0423d5d180dd8da8fa5a04b477907ab012d5acbafd7a1e052b4", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-06-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-06-en-m1", + "case_id": "fresh-06-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "8a684303442a5c909c4625d1af9e65b96a93f859576d8e2b4ff0f5df2ba7545a", + "request_payload_sha256": "629ae7bd2d43dc8dbdeb844562aa9da0abaa7af85e1f077ac097447e38fbea37", + "receipt_sha256": "64f4bc62f0c57d87b36a029fa3a34bf9733fa56e8c56a2b461d7c01ca2a6c9eb", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-07-en-m1-source-chronological", + "episode_id": "confirm-fresh-07-en-m1", + "case_id": "fresh-07-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "ca084a412255de9cc52fa17020c4fcda28fb542adfb8eb84583804050cfb30e2", + "request_payload_sha256": "feea861632036c6b066872708595f789024aab6afde7e64548fd1c09a4d9d042", + "receipt_sha256": "fe9871dd5e3f46f1bbb5ebcfff9ed409a1f3086fd7dd1ae0433286db22f81a29", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-07-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-07-en-m1", + "case_id": "fresh-07-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "5c2d83ef664202d545a7af808068645e7e29bc19f8771bb7f5dd68647b73cc5c", + "request_payload_sha256": "70b9f0ad93802b8bd9a7b69e0a0015803980a1c539f42b2c3b9153bed41a0c08", + "receipt_sha256": "62ce456f858ff8a5e8b845588ff154c681149520242d7e2c59f0dcfab2296bf9", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-07-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-07-en-m1", + "case_id": "fresh-07-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "bbd91f96da070041b366efd7138cfabd01ec6b256690a4ceda930d29a7c3ce63", + "request_payload_sha256": "79fd7121c5e71f551c828d9dae1d7e4dc7add27ffd96114c766bb9f56c492d47", + "receipt_sha256": "df09904759ae1a9ca2381fdf94f665cbae52996769bd93d06390c0e59da64fa0", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-26-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-26-en-m1", + "case_id": "fresh-26-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "c9a3cf94f2d606cb83b42ebc9246c472d2986345e6e42cf85ecbb5ddfd402e5a", + "request_payload_sha256": "fe6c76acf1e8aab1d7d81ff4001bb09dee330c61ad9fb61bef686a0f01e6de00", + "receipt_sha256": "6c3672f9df57e07b91bcf801f60fd8ce973f949b2b105e1af8fc13db7760e638", + "result_sha256": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-27\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-27\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-26-en-m1-source-chronological", + "episode_id": "confirm-fresh-26-en-m1", + "case_id": "fresh-26-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "12ccb2112042daa0738e23ba75fae4c70b8b18b91dbd4636903ab9e49950da8d", + "request_payload_sha256": "054b9b71f17f7430d2d4ac34fb911114997ca14dd53e99f0742dc9893b8ce0a9", + "receipt_sha256": "1d9ac62110b3f496f392471d5404bd085a80eb4ae0d73681ee0b6b6b67dc5d69", + "result_sha256": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-27\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-27\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-26-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-26-en-m1", + "case_id": "fresh-26-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "c9a3cf94f2d606cb83b42ebc9246c472d2986345e6e42cf85ecbb5ddfd402e5a", + "request_payload_sha256": "fe6c76acf1e8aab1d7d81ff4001bb09dee330c61ad9fb61bef686a0f01e6de00", + "receipt_sha256": "34ab42f40b287161fff29013ae0ab67336650ee9f4c00c83c6ed56036e2098f3", + "result_sha256": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-27\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-27\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-37-en-m0-source-chronological", + "episode_id": "confirm-fresh-37-en-m0", + "case_id": "fresh-37-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "5a0d59652d0963d41359a6ee085ba81d0887632e68a05c8bd772bfd06e37d22f", + "request_payload_sha256": "8bde8952ff5f000d0728f86ee75d4cf846684e1614eeb8639aecd30025e44b19", + "receipt_sha256": "8fc7fde8c96fcbfce915aa9f917924391ecabf1de8ecba413cf731fd846ce233", + "result_sha256": "c7a56f88f850e3a73663b3db8254ace39a97d0a15d6eee5b44ffb9b303316140", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\", \"answer\":\"2027-05-08\", \"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\", \"answer\":\"2027-05-08\", \"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-37-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-37-en-m0", + "case_id": "fresh-37-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "d84bb6f2e524c554f2364ab0d35851b53f1c21ff390677fa06590f12275bda39", + "request_payload_sha256": "9948ec320f8f81bbba126276898e9131623080ad0f82118a94da4552f36e49c5", + "receipt_sha256": "cda5e52a59b8bc8b99aa6eded713330322010d854e8ca5e24bee81a310040c7c", + "result_sha256": "aff48058a24de4a1db9e0fa8c903c6c318fd62f91bba7bbd8e5d3168a70bc729", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-08\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-08\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-37-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-37-en-m0", + "case_id": "fresh-37-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "1ea0e4faca7c53494678f300f121ecd12a9f27e5917839e94fc0bb50336175c1", + "request_payload_sha256": "46f72eb9d7c6a48af4e8242617afa15fa240fdce1610c989e73f600324688cad", + "receipt_sha256": "f01ae39db06f018a960954d909a833a35a2b40b3e85b73060838ff57fb1f3b76", + "result_sha256": "69dde94d76c4033be2860a97d246a801af8bfcec0b584aa6c9f0986a935603bb", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-08\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-08\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-13-en-m1-source-chronological", + "episode_id": "confirm-fresh-13-en-m1", + "case_id": "fresh-13-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "e1fd68466d580d3a4e04de6e3220fdccfa409f6908981edf4524eb7ea47e5446", + "request_payload_sha256": "79f6571cce30b87e245dc08503145d4f5b595115740f4ea0a4e1faf8405a36df", + "receipt_sha256": "24866ddcddca7dbe7b80aa27a9f7ef34c0dcb4cfa7197fb3d5dce26b77ab4252", + "result_sha256": "b6474bc8194ce66a7fd312e11be1d6465b06f536176a1bb68eebcb0a95a69d0d", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-14\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-14\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It selects a definite date despite an unresolved source target; Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-13-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-13-en-m1", + "case_id": "fresh-13-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "e6d7dd8cc5abab8668c9cb71e026910c4d9834b05c4b05b240feab11869283f1", + "request_payload_sha256": "db247aceac03f326472b668352bfcfa5c065f13d07c957b1a47b2de9cb310beb", + "receipt_sha256": "21ca80526500d175082c02722c869028273e856e33c063aaa54284e5845c8bda", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-13-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-13-en-m1", + "case_id": "fresh-13-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "0759094895e0501ab5460f75368411fe0aa2088ad1dd0d58b700e0f149b54e4c", + "request_payload_sha256": "d438cfcba1057b7c7b49d680f55e64a9da55b10bd7e0a6f25f20bbb1313eabee", + "receipt_sha256": "f1f17b4b883de31c9fc780dc6bdb4a9b74f1df6ef24e82c4ef2f0c8800758ffc", + "result_sha256": "b6474bc8194ce66a7fd312e11be1d6465b06f536176a1bb68eebcb0a95a69d0d", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-14\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-14\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It selects a definite date despite an unresolved source target; Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-02-en-m0-source-chronological", + "episode_id": "confirm-fresh-02-en-m0", + "case_id": "fresh-02-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "4ffea1578924eab930462e302aa1656d09029eb4f028cdb02ad43af95f6b40cf", + "request_payload_sha256": "dbea6b851d6ccd7942d25468734f09911edaa418a251faf557674c1cc25020e8", + "receipt_sha256": "cc963af7bee60a24402dcc06ab7061d935ff84783337fc3004d8710a18992fc5", + "result_sha256": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-03\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-03\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-02-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-02-en-m0", + "case_id": "fresh-02-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "3ec47e038452cef93af790e9efe0a915d8856d23d84f8a5ca406fbecf9b30d11", + "request_payload_sha256": "8e808b705d17b2e3237931da8e68e1bcf2b5a849cb23e78126ece3f33adadc9f", + "receipt_sha256": "c503edd5fbb1cec1906f155eb9640fc98a5f8c93fd5383540707273849936b81", + "result_sha256": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-03\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-03\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-02-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-02-en-m0", + "case_id": "fresh-02-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "0e0592f36caa70034c2bc2114c4d53677b8e73263d13c43c8f11b3a880e6044c", + "request_payload_sha256": "e5b90ec9c587194435438c87c0d2aa2c6a8f6288d5ea07a84edb0e90c035e8e7", + "receipt_sha256": "0a004b6c31347b20d1805ea1d05e1c70ebdbe1fb074b66cd933439d9bbc2d096", + "result_sha256": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-03\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-03\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-12-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-12-en-m0", + "case_id": "fresh-12-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "39451294cfd4adcae9796811fb575ef37dbecaf275d1cb0016746ea2bfb759d2", + "request_payload_sha256": "0a7ff1862cbe955b7ea5f4cb93e061f237b01c492a3d5c2bb7f89b1c31f640e0", + "receipt_sha256": "59b1a8d39efb27b14d8ee50ebb4b6246f960a1e64a6344b2f488c6db931aed1d", + "result_sha256": "fffc9f37f755ef41afa789ae57380bf1a32c91dda35e0ac1b843306140fe8823", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-13\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-13\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-05-13", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-05-13", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-12-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-12-en-m0", + "case_id": "fresh-12-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "a489824ea97c6c609cbfb2efc5b2051d6c4546bf57680cfa79f070790b8b2ca9", + "request_payload_sha256": "05ee5b113bdc6f2f19d9d0cda15e6353a9473a72a93a7a59e979d3f0f2bfb393", + "receipt_sha256": "d5c8c6f3758d8657532bf91cd62719344a92def1909821e5424fc9ffbed5fd0d", + "result_sha256": "bddfda0b83421ea19c31efae0763e4d882cd25d8f7fc51ecf50c2203c16f5d03", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-13\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-13\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-12-en-m0-source-chronological", + "episode_id": "confirm-fresh-12-en-m0", + "case_id": "fresh-12-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "7f5fe7ec8f6cbec84d49802508b31b7f8b0d2d8da89b693c9e27f2c181f3a460", + "request_payload_sha256": "c82103fdeb1675c1f4730b64b21c57f9daf583c97d3687a42414d3f1d8b4a267", + "receipt_sha256": "dfc630adcfde7d136974e4de61b6368ed6d0f31762972ae4be275575c953d3f4", + "result_sha256": "bddfda0b83421ea19c31efae0763e4d882cd25d8f7fc51ecf50c2203c16f5d03", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-13\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-13\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-22-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-22-en-m1", + "case_id": "fresh-22-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "30e9bd69a901110f6b68f0e3ea360b71f7996f77588c8f9f6269c0a59b9a8432", + "request_payload_sha256": "5a82c11aaa92056444421f652f9edcf3d372d887b510e49be9dd34ae224a0584", + "receipt_sha256": "d8221435062184c5e0ab3386633949771fb99b7067d9f222e2496d82712dc0ee", + "result_sha256": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-23\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-23\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-22-en-m1-source-chronological", + "episode_id": "confirm-fresh-22-en-m1", + "case_id": "fresh-22-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "1e53d708d20e22b5ce759cdae8fd472bc362562068c7614e7405bc3555a69b4f", + "request_payload_sha256": "4fd1fe40b17914e46a7415ff0b168f209f9ee3fe84aafbae79eee814ae529d7a", + "receipt_sha256": "3c2d0fd77776e8c521c4a3df1ff7574bb28fd0d4fcd14160c5be6a1af87ab183", + "result_sha256": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-23\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-23\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-22-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-22-en-m1", + "case_id": "fresh-22-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "30e9bd69a901110f6b68f0e3ea360b71f7996f77588c8f9f6269c0a59b9a8432", + "request_payload_sha256": "5a82c11aaa92056444421f652f9edcf3d372d887b510e49be9dd34ae224a0584", + "receipt_sha256": "c33040c45ba3575756aaebfa0838f73d76bdbf07890c137d81c1852bd153ba5f", + "result_sha256": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-23\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-23\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-29-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-29-en-m0", + "case_id": "fresh-29-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "748dff3be9b9ce5092a48656e72e03bbdfbe6c586effb4b8f5a4ff5f9dee796a", + "request_payload_sha256": "c37895bd5aa2fade515cf5cd6f4f3814d9d9874005b9b1cd8259ee200abf9d72", + "receipt_sha256": "14acc6cc77887d312dd7006fd04cb07a602c0da2b63f029fcd5cac870e2967b4", + "result_sha256": "f517b9fdb42a8c5fdc3536e2720b69744ec169f1684a3271023240b26b8c846f", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-30\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-30\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-29-en-m0-source-chronological", + "episode_id": "confirm-fresh-29-en-m0", + "case_id": "fresh-29-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "1a18ce556621bfc95b6d144b3adc756d82a54d5d458a20557ce85cdf2b8f2c4e", + "request_payload_sha256": "b3237531a3407e1c32fae441eef7fbb4f747512e48e2a974f81f14ce793ea5af", + "receipt_sha256": "2fe9aae8378c19860e6dd4c6a7de197b9e266941a9c5624367fda8709dfe3b7d", + "result_sha256": "f517b9fdb42a8c5fdc3536e2720b69744ec169f1684a3271023240b26b8c846f", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-30\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-30\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-29-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-29-en-m0", + "case_id": "fresh-29-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "cf2aa803e4227dd8d7ff7eaa99d9fea955dd4b422427bd0354ae6028e4697717", + "request_payload_sha256": "70ec5e91d9daff2181a12e0af2f80130967789e1e156023133202db24fec0f75", + "receipt_sha256": "0f5c1bf67ca546c0d1e5e01f7c9f4b27a375d9b4dd7eaf93b8534b965a9e983e", + "result_sha256": "f517b9fdb42a8c5fdc3536e2720b69744ec169f1684a3271023240b26b8c846f", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-30\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-30\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-30-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-30-en-m0", + "case_id": "fresh-30-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "ae6e226eac84e2a1297673e7479eee18cd79d84e51a8a9aaf4ceadba7f2d2bff", + "request_payload_sha256": "b8403cff7e57be385372431e61f4a0462e2d58bb0b6ea625313a2875d7db2ce1", + "receipt_sha256": "5f8550e37acbb943003cad89d487f101a6282254b9b5fdadef49f57ffe5e7e5a", + "result_sha256": "ff996de64c0ed5833a02676aaa0f784e5b961cc8b0596750e40ebe7f0be479d3", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-31\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-31\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-30-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-30-en-m0", + "case_id": "fresh-30-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "de6c7d3d9d7a8146dd115060e3260488906939d134d1a5abea1c72073ac1a9f5", + "request_payload_sha256": "de49e814c3f5627408923aff56c78f609cc8b2fcc560ec7eb945c96a31693bcd", + "receipt_sha256": "920681bd08a140f137593cab99fecdbda51ca629d737db8ce1c800ff58b9f735", + "result_sha256": "c225aab8d19cca2515a377d9c5462cc34a003a0c3653544bba785e793db51707", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-03-31\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-03-31\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-30-en-m0-source-chronological", + "episode_id": "confirm-fresh-30-en-m0", + "case_id": "fresh-30-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "50ada1ba0f471c0774a85465040185e6b2c0a8eb52db687bd199ff3641c2cf8f", + "request_payload_sha256": "c51a30cfeb31d56e67affaf0c6766c22131a105fbd62bfad1dcf00b51b16178e", + "receipt_sha256": "31c44d9ec96a626cb464197a1e8e259d01d53d6989416978eb1a5c9acae15d73", + "result_sha256": "ff996de64c0ed5833a02676aaa0f784e5b961cc8b0596750e40ebe7f0be479d3", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-31\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-31\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-33-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-33-en-m1", + "case_id": "fresh-33-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "77debb548bc7505b53a51eca1954d184093b3d3c71fd407da687d87d9e31fccb", + "request_payload_sha256": "d496b69a9b0002342a3764cf99fc88c5f487664890cfae0e0ae7d438776fe99b", + "receipt_sha256": "7a6f9b1051f2a010737adb5b8c70be0402891427a99c145912b58c3e988cf269", + "result_sha256": "84391b7e3176e5c1aed2756ac5fc41fe066cc25886a30c9f667429bd8fa83637", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-03\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-03\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-33-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-33-en-m1", + "case_id": "fresh-33-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "e8bc16d73c5c842d83ee559df70e1dbbb9728e1edde3e298e8d8fec930b28c4a", + "request_payload_sha256": "a03c1c5aa87a3d8845f35033f3e78c3d1e85ae181907e89392a2d4e791cd16f0", + "receipt_sha256": "8da87e57b94455e939896f54fe1032e0bf5df42716f52a274b6ffce5a6629e80", + "result_sha256": "0e529979621b57ba683527909c61a095b19f49f46d26c9741184bd4c71571e88", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-03\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-03\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-33-en-m1-source-chronological", + "episode_id": "confirm-fresh-33-en-m1", + "case_id": "fresh-33-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "ad751e50ecb111dda950842bf2a527f0c4356b280192b315403b22ac9dae86df", + "request_payload_sha256": "aaf2f3451fbbc88d53b94a085bdf79259dbb5c62c2e3aa72cd82b04ec688b093", + "receipt_sha256": "3c61572d23ed307b30e0974932295bd3811b7c7167b1f334045fca2caff86345", + "result_sha256": "0e529979621b57ba683527909c61a095b19f49f46d26c9741184bd4c71571e88", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-03\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-03\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-17-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-17-en-m1", + "case_id": "fresh-17-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "39569aa94d53df44b70dbce2324dd43192f0c30f57f03812664fe1c86a78d5f6", + "request_payload_sha256": "0bcfb050031b3180df83746b14a156759da18ac9ac6c0b882bd7a1ef89cb5aae", + "receipt_sha256": "6c5a704b858804669775860a8bcd278c7a96983c45a106164eaa5953f13d02b4", + "result_sha256": "87c81f64b32de0e465ec65f5672c4e2b281ceaddfdb2a11351a1e137c744f10c", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-18\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-18\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": true, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It substitutes the backend's current date for the customer's preference." + }, + { + "id": "confirm-fresh-17-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-17-en-m1", + "case_id": "fresh-17-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "fd338e744485ab3b7ecec86e4b61177aad74915b531c8d406594991c1b937850", + "request_payload_sha256": "a5abc6dd012bc0192477f3ae87883063fb18bd90e5ab5fa1cc9b3c1b3faa1887", + "receipt_sha256": "f8fdca9a242a4298153ae07a1877bdc0c6097de572168f15802b634aa725ec08", + "result_sha256": "87c81f64b32de0e465ec65f5672c4e2b281ceaddfdb2a11351a1e137c744f10c", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-18\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-18\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": true, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It substitutes the backend's current date for the customer's preference." + }, + { + "id": "confirm-fresh-17-en-m1-source-chronological", + "episode_id": "confirm-fresh-17-en-m1", + "case_id": "fresh-17-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "a59e6a37b8c4d0cccefc857f1547377e371665252e9e36e440893f60d242118c", + "request_payload_sha256": "8d4de7f2a51122f2bf8e41b6ec59a124da43c32773b634b6cc7299623794c11c", + "receipt_sha256": "cfe6d80539aeca53e8c6e730500bf6574a774d91ea2a1d3ee11a4103d0d48fe3", + "result_sha256": "87c81f64b32de0e465ec65f5672c4e2b281ceaddfdb2a11351a1e137c744f10c", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-18\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-18\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": true, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It substitutes the backend's current date for the customer's preference." + }, + { + "id": "confirm-fresh-05-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-05-en-m0", + "case_id": "fresh-05-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "688432937f19c9399e4ad26aa82b0b636892d506cfd54017ea91e94b827eecb3", + "request_payload_sha256": "1d621699ce622afcbd3be653514c894e3469d749c5072965b3e2254990f7b2e0", + "receipt_sha256": "3ee6649307c91340d45465e0caeb7287f65f7edff392e4144e3654b59a0af52b", + "result_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-05-en-m0-source-chronological", + "episode_id": "confirm-fresh-05-en-m0", + "case_id": "fresh-05-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "93ed370998c25139593fe9745b3e933e7ec528437676549383d5c1b1153ad4f5", + "request_payload_sha256": "4e598e370c7228a2ae9827efe2a6b01ba5cd38655a88d19fd43c2dfe9d47c455", + "receipt_sha256": "02f908bc9929e7d5d97b88f97a9afe12c4e652378868be09ccb442a3d04c4b70", + "result_sha256": "43fc631886e6608129bf134306522d45fc1cdc8d48c80c92eb5d99b3244c3685", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_performed\",\"answer\":null,\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_performed\",\"answer\":null,\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its clarification flag conflicts with the source condition; Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-05-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-05-en-m0", + "case_id": "fresh-05-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "c2850407d6b1db55b84b0f21871c771d1b9c6364e3474b4269f3bcd4385129aa", + "request_payload_sha256": "eca64af10d8c7e683b68b7c96936fae30b270de53cc72cf69b64ba8564c10c7c", + "receipt_sha256": "2d2397ca11d62626d01f2301ee63fab94b0b654d4065e53e1c13a4e34a62cf28", + "result_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-16-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-16-en-m1", + "case_id": "fresh-16-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "4ed8ea7e2b211c3458721710a73a069bc21aaed6dad8829c8170b415dc628fa4", + "request_payload_sha256": "34b4fad0db15ee406c3147d388b93cd73261ee9da7758d04e07c5ad89c329058", + "receipt_sha256": "8bb33ba9cade45a0035996e0f683881542507c85e28d15e1338bf0ca6e7a1bc9", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-16-en-m1-source-chronological", + "episode_id": "confirm-fresh-16-en-m1", + "case_id": "fresh-16-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "73543516edd06f05fa1121064583306d258ace4a2e6e43bf6f54e2b9a7e813ee", + "request_payload_sha256": "547341e3d8ffcc0aa08017b43d3aa7b264ba6aa97769003503caba24c9aa6d68", + "receipt_sha256": "7f79b3e28c3da2c1f5d62bcae7c9e5103371cfa682bde190ca97e9d118f80f8e", + "result_sha256": "081f2013d93cd7b24a8f7a3eb964081ac78a1dfd0105add89dcdbd0bc3ba2e8d", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-17\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-17\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It selects a definite date despite an unresolved source target; Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-16-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-16-en-m1", + "case_id": "fresh-16-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "d24ac8643630b0862305a5131d11b839863eae75d3b7c239fa9b549e34487ff5", + "request_payload_sha256": "13b3920ba5cfbdfc66b60bafcf8fe1b84be5a8e73acefecfe06af37b232c0e05", + "receipt_sha256": "6e60b7b8a97d40ddd6bbd83d661c4957ce4de9a0185f3e8ce97a37136fe5e997", + "result_sha256": "081f2013d93cd7b24a8f7a3eb964081ac78a1dfd0105add89dcdbd0bc3ba2e8d", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-17\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-17\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It selects a definite date despite an unresolved source target; Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-34-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-34-en-m0", + "case_id": "fresh-34-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "41a2fdf974b8769db64fc4efc17d45cd8d3a72780e3f15b8b0d73dd9114a9790", + "request_payload_sha256": "eb1d77bdfec0f5d3d55229ad2bd71e475ef4083e23b9b3cd6443d0a7aa6ebf9c", + "receipt_sha256": "43d41356769b3466641597b82caa43ac4c6ebed3cae4b174d2ecec29aed505db", + "result_sha256": "7bdd78e5546dd17df78e3fca29b4d5de29ba0df0136ee40ba69bbdbf86da1095", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-06-04\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-06-04\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-34-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-34-en-m0", + "case_id": "fresh-34-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "2f3b7e2f3681db1911e4c50340e2fecc5a87a26db00dd48a6df68663243a8e99", + "request_payload_sha256": "b1b5d5efddcd4ff3d34044ab8e286676f4b3534dfe94ecba3194b21d135d1bbe", + "receipt_sha256": "2fdc84d8aa316b7ed2401e77c86398cc62c536636a2622497d5a4f811ac7b468", + "result_sha256": "abd46c530745df8728e321ec83dc35accf4d0d8c2c4080a3ad91e1f119565685", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-04\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-04\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-34-en-m0-source-chronological", + "episode_id": "confirm-fresh-34-en-m0", + "case_id": "fresh-34-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "cd473313d492da83b5a47de7b3eb379069a2f2505f76045f73de93ba3e762cf4", + "request_payload_sha256": "d0f65b67eb2308764f14614ca06875ce1303bc4d2a415fc089f5cb2a8cdc3297", + "receipt_sha256": "6ecd59b354dcdc5ef7421f079c7f3f7d156b277cefcc3de434c1414a85fb0c5d", + "result_sha256": "abd46c530745df8728e321ec83dc35accf4d0d8c2c4080a3ad91e1f119565685", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-04\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-04\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-11-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-11-en-m1", + "case_id": "fresh-11-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "093cebc39157261cf07a2ffe4101088bbef57504c815db6ef1bbe0fab9118ac3", + "request_payload_sha256": "1226b7b0da676f5e5edd908bfa087b996fb38315e32cbcd6cbe08c53657c109b", + "receipt_sha256": "556943946995f48f6c409e7374cbb7ff00e6b3da89b5ebd2d8c72512c6d1f414", + "result_sha256": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-12\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-12\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-11-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-11-en-m1", + "case_id": "fresh-11-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "a80ef8f37f0081a0af52c36ac47a90aa2c0491d3e16df70c85fb72dd9e594b4b", + "request_payload_sha256": "0f961b366b85951b596bfad1b1f6bd415fd16b9b9d067ad61a2b839de0d07b5f", + "receipt_sha256": "fcf1450d670839e66e48e5216a3a4a3fe13cf2d6c9063ae3bbf104adde844209", + "result_sha256": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-12\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-12\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-11-en-m1-source-chronological", + "episode_id": "confirm-fresh-11-en-m1", + "case_id": "fresh-11-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "50787cf182dd50710a0cfdbc094cbb2ce09ac44120365b2d04d28dbfa4046bad", + "request_payload_sha256": "f0a2cab3788f2c434b184586b43e5ad0cf455f2ebe063504615be661a36907e6", + "receipt_sha256": "77f4b2f269f3d5257070657e465cc3c51aef620f8caf46aa82f868b69cc49a61", + "result_sha256": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-12\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-12\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-26-en-m0-source-chronological", + "episode_id": "confirm-fresh-26-en-m0", + "case_id": "fresh-26-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "229fea01678034a79da0e81335f726698c21aafa81f70b079a0c5859a39f3c68", + "request_payload_sha256": "875f7810bae96df19d3a6a92e6d495d39567debb5c9f533f96d87b48f298cdb5", + "receipt_sha256": "6ca611fab89543db7c0317b2707a0d83dc9fcea8ad17844712c069e07826d386", + "result_sha256": "fb6e951f5e2bc3a3ec400cf4d93a3dfb2910e24d4ec7d9480d665ad63036999a", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-27\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-27\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-26-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-26-en-m0", + "case_id": "fresh-26-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "f4f1160d869bfe230f32b7614fe5fe65c7bfa25aa0acd07ac1b9f095b896fb03", + "request_payload_sha256": "f0a83224968045620157fd5a21907da696aceda6384aae42470544ec2b0867fc", + "receipt_sha256": "94d3e4a3062a52d130f4c01b4bf41be51f094a550a610d9ce936c6c9b979e562", + "result_sha256": "fb6e951f5e2bc3a3ec400cf4d93a3dfb2910e24d4ec7d9480d665ad63036999a", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-27\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-27\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-26-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-26-en-m0", + "case_id": "fresh-26-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "f4f1160d869bfe230f32b7614fe5fe65c7bfa25aa0acd07ac1b9f095b896fb03", + "request_payload_sha256": "f0a83224968045620157fd5a21907da696aceda6384aae42470544ec2b0867fc", + "receipt_sha256": "5f2f5fbd756d756e39d583e0a5ce8017e87af398134faee2947757711da54c34", + "result_sha256": "fb6e951f5e2bc3a3ec400cf4d93a3dfb2910e24d4ec7d9480d665ad63036999a", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-27\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-27\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-09-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-09-en-m1", + "case_id": "fresh-09-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "530fc922b0e829b18549db88ff95eef766cdcbfeb275a92bdcc15783058ccf28", + "request_payload_sha256": "54de4210171d5eee20b884428eb3f0622ff769e6ada595bc30cfa67f265efd4c", + "receipt_sha256": "7346374e627076649ec9f22050cc59fbca77a61982591c970eebf63343345789", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It abstains although the source or actual receipt determines a date; Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-09-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-09-en-m1", + "case_id": "fresh-09-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "210d73e7fe8e3fc615b730ada8205823f187b8170ea3cef5998c7873366fbc13", + "request_payload_sha256": "ee7e2691aac70b8ab490149389cfca3487980529686a026f1dc15a48951502b5", + "receipt_sha256": "6bf95bfdfc1432243fb9a5709309621405d142d4eaf6597982b6aa5729c1ee56", + "result_sha256": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-10\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-10\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-09-en-m1-source-chronological", + "episode_id": "confirm-fresh-09-en-m1", + "case_id": "fresh-09-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "34d77e58dbd3f47dc78660852604b2123a0185ed2ad43d574b3bdbe3378ed66c", + "request_payload_sha256": "f8310154962c1fd7807d3eee0abd10d7b96990cd0aaf221781b00900cadeaa57", + "receipt_sha256": "950b2018b4cdb3cd2e40dcbb715263223b94dad0fb7a9a1a5ff4d4dc00b9ab21", + "result_sha256": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-10\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-10\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-07-en-m0-source-chronological", + "episode_id": "confirm-fresh-07-en-m0", + "case_id": "fresh-07-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "cc6478e7611b94902eeaefecc7834022e36bb6c47bfedabace0710f7d9ef906b", + "request_payload_sha256": "4021cbd696c4478319bfc5a53f3c7e7c6282f82406daaab63135df363e11685c", + "receipt_sha256": "8d491eb1794e4e1eb7383b4c766824e0f3d06a13268072dea1ba9389011d9621", + "result_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-07-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-07-en-m0", + "case_id": "fresh-07-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "56d9acab222c4ced942a6da3c4f48f05cb6bdea4d049f965d5c83bb28920f2b3", + "request_payload_sha256": "5b5c9d9eb4e94fe147a53d4a7bf234bd0c5a11635ded8573dac5877d572b2308", + "receipt_sha256": "cdf9f9a59db7d8c589b0a9f7fba59c01153605987cfdcab343c5a73d0260ee16", + "result_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-07-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-07-en-m0", + "case_id": "fresh-07-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "74b9fb219fd1138dd0ae1b181e7d9f9b700d3a3767438d393acd0f608f93afed", + "request_payload_sha256": "dacdfbc075c1a90761a0f6626194cb7bfdac60f70236edbcbe539abbd8a584b2", + "receipt_sha256": "b29b5ac9d9b96492f3795f956f4aa97f2ee4bca95c051850029f820a2899f92e", + "result_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-33-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-33-en-m0", + "case_id": "fresh-33-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "bab3a860302865da6c7592e60f06fddfe499d10035bdad8c799b4cda7ff97caa", + "request_payload_sha256": "6f7a837a010ff604b48dbe2d999beda4bac81d45f0b683ad7c3db3d48d472a4a", + "receipt_sha256": "2b71a4e9df31591317e6ce4e439b293b134730b0581ce29813c472088c2abb3c", + "result_sha256": "45d118b7e3e1ef29995ca08d2c547b209da33c0d933827c2d90014c8ce9920f6", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-06-03\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-06-03\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-06-03", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-06-03", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-33-en-m0-source-chronological", + "episode_id": "confirm-fresh-33-en-m0", + "case_id": "fresh-33-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "28e7143206b68b0785d7ef35f731021b0eb4929a5c2eafce87f08af3146c0209", + "request_payload_sha256": "7110aeb3ae71793a9df14320e84fe570fbdb81729979e9fb0bc713a9d8f647a8", + "receipt_sha256": "2e265c7c2470007d4486e17d6544cd823607771907b87cee771fb45e0634232f", + "result_sha256": "c1553fd0523b1529616e1962bbf6812c5dd94faf93dff31d113956ebebf274ad", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-03\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-03\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-33-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-33-en-m0", + "case_id": "fresh-33-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "64301fa88c540bd6b7d100aa34373bbdddfb666fe5a5f58aebb05bb201c0fb27", + "request_payload_sha256": "0ac54bd03ac6c569789062e3092fcecc2bb268a2d88339df488b924e7ba0fc41", + "receipt_sha256": "f7eb78798a4a6b5ccc0797787a9382c28cc9d1ddbedde9f13f7581cb3207cf14", + "result_sha256": "ace684bc08dd37c5c2e8bd35a71777504297a736697c30a50b6eb7d70c02fc6b", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"rejected\",\"answer\":\"2027-06-03\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"rejected\",\"answer\":\"2027-06-03\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "rejected", + "answer": "2027-06-03", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "rejected", + "answer": "2027-06-03", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-10-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-10-en-m1", + "case_id": "fresh-10-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "3e67f521f40ebe61523440efa90038b896792f332d6e20a4aae13ef37ecd09b6", + "request_payload_sha256": "eb57a10dd314802df81658703fbe4189a9a3934e13590d75a6f9f508265f4f0d", + "receipt_sha256": "e8d09335f850e835d8258796440cd473a49f3bb2ecec306f3ba81dfb2c88df2d", + "result_sha256": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-10-en-m1-source-chronological", + "episode_id": "confirm-fresh-10-en-m1", + "case_id": "fresh-10-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "2ec64607765e0c4048e805de9207958b37a5e5f50a203da253fd701dbb5be325", + "request_payload_sha256": "a3e3987ffced75b2a4d99cdffeafb024de91e6f34ef16c1c82a00425bde02ed5", + "receipt_sha256": "02ebb253d387a579ae47ec3565e901b304378d7a31f3014c2bf9670523b2ede8", + "result_sha256": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-10-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-10-en-m1", + "case_id": "fresh-10-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "0f2b6fd50b0c57395b07501dd851722eb974843cf5d70cc9c21118151eaf65ad", + "request_payload_sha256": "209907b57fd7e820c6e6f7080f32e318c75fd1a9e0096532c62c381f60503a40", + "receipt_sha256": "01e249b38d663b879c99831246b8b356c132f19686a87d24d025eca761373ac5", + "result_sha256": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-18-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-18-en-m0", + "case_id": "fresh-18-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "0b211ec2f8ff27f313eacc2a88e0c132b5c4ce752e31eedbb3317575ce1e8f08", + "request_payload_sha256": "4c5fada6772de7e4f3c2f0dd89ada2190783ec1e1d75d12760f8a8de8df6716f", + "receipt_sha256": "14ddd58c35d8c36475f1112e6419a371a7018723be2b2eb8a8dad5a51fc39206", + "result_sha256": "7742bd4bc3122d407a22a3b0c9ca8d2c6ca6b7b7cae5be0a9eb21194156e403c", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-04-19\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-04-19\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-04-19", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-04-19", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-18-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-18-en-m0", + "case_id": "fresh-18-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "a0d862d38c3e7cfe9651b5a821e3ea2a1ef6756a510e725acc07d2c9ed760fc4", + "request_payload_sha256": "47b26f1f73b4696e1c0277a7dd6ced5835d8f8ab72b301b4f30bd70fd372608e", + "receipt_sha256": "f517e498252cbb90eb8000f4ea24993c2fa19ec0479d3492afae31e0a81252b6", + "result_sha256": "00d649cfc39ec358aa1d45e738f53feb9cb97e2c7b7fc9ca56a64eec5020214e", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-19\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-19\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": true, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It substitutes the backend's current date for the customer's preference." + }, + { + "id": "confirm-fresh-18-en-m0-source-chronological", + "episode_id": "confirm-fresh-18-en-m0", + "case_id": "fresh-18-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "ca795181781830a6f69994ed4e0d70231d0ecce59b46deca72976b30f5710f6f", + "request_payload_sha256": "080dab4e6daed4339420f8ef4dec3ee7916b4a709c0cf5cdf31822547917bb58", + "receipt_sha256": "e254092738553483e04cfa298dc1b69777f978d47ae93181a3fbeb57f602cf1b", + "result_sha256": "03bc7ffa97b58cb6893a53bd07ae07966616cd8c647d669421d85a572a958aae", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-19\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-19\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-30-en-m1-source-chronological", + "episode_id": "confirm-fresh-30-en-m1", + "case_id": "fresh-30-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "41dff31c23df00b099113f6fdac84399667e7f60ffb1ffb43d1404bfaabc53d1", + "request_payload_sha256": "52cf9f64e00a60549b944db119408dd06560ffb9eabd7592c4acb3854aa1ce44", + "receipt_sha256": "240a797fd7e6bb0aa2eb71aec8b8b230ed487cac101d1869cddc05990e4a4a51", + "result_sha256": "87902016aae6283bdff9207729e01b30ad77cad918dba806396649f58f68c74c", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-03-31\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-03-31\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "projected_action_status": "rejected", + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_execution_status": "rejected", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-30-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-30-en-m1", + "case_id": "fresh-30-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "6f1511145b0ba25b23ec8e4090066ad61d5d7f4297e43188a588b6365bab8cba", + "request_payload_sha256": "cc0bb6350360341a8da30f6858e6d78d2d894d9bf5dfd6fb160879c06d4a3ccb", + "receipt_sha256": "4f557e6640c0c4631666279ea62cffade2ed9b08f62771956861f9d50514ddfc", + "result_sha256": "87902016aae6283bdff9207729e01b30ad77cad918dba806396649f58f68c74c", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-03-31\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-03-31\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "projected_action_status": "rejected", + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_execution_status": "rejected", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-30-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-30-en-m1", + "case_id": "fresh-30-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "6f1511145b0ba25b23ec8e4090066ad61d5d7f4297e43188a588b6365bab8cba", + "request_payload_sha256": "cc0bb6350360341a8da30f6858e6d78d2d894d9bf5dfd6fb160879c06d4a3ccb", + "receipt_sha256": "94d38d901e5e4021e9a0dc93eca9515b8b2632b8fe92d4010f425e4c4ceaa5b7", + "result_sha256": "da50423f0bc90841c2b9e138e9f4a9ad0340b0ec5d888b3091878471fa1a28a5", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-03-31\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-03-31\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "projected_action_status": "rejected", + "actual_question_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_execution_status": "rejected", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-04-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-04-en-m1", + "case_id": "fresh-04-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "c88e6530b5e31682fdc637b69795772ead271dc11761218e12a0c7650a8a91d2", + "request_payload_sha256": "7892d10642cb99c44bbf0ede617ab7fce3a69ecfdd0fce38d7c4f82c7c94d718", + "receipt_sha256": "9427adfccd5d34dd66cba3059394be250cc3fcfc8e636084c325a9ec0c104995", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It abstains although the source or actual receipt determines a date; Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-04-en-m1-source-chronological", + "episode_id": "confirm-fresh-04-en-m1", + "case_id": "fresh-04-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "2a06275c07453afe2c141eb3d1806e4952623fed04c69e3575c5e9440ba51fb8", + "request_payload_sha256": "94415ccdf5fc18dfc3ea102be95284fd6363bb477f996f729929b654ee8ba8b4", + "receipt_sha256": "036d5af97b06dd8c2ced97959254c5d73df6f779456e85c2826c768fd05d82c9", + "result_sha256": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-05\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-05\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-04-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-04-en-m1", + "case_id": "fresh-04-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "cf6b8ba2ee944ee3b19e0936cfee319080d63b207ca14d423cc03a9b469ac17d", + "request_payload_sha256": "a84b4d10d6ee34332f9a1933cf111b7a3e63e9fa42ca8cf37f7d4f81be3e4b5e", + "receipt_sha256": "70d4e3377fadb4ab5be641f71f7dc16cd3c9d1a045826a942703dc43b7e870a6", + "result_sha256": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-05\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-05\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-12-en-m1-source-chronological", + "episode_id": "confirm-fresh-12-en-m1", + "case_id": "fresh-12-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "49dbaae2e28c93a7b8cc19e9cc950002819e3f53c2968b55fac6de85a05f8811", + "request_payload_sha256": "e47ceb12bf990a2dd1ff28acb1a1988f1525e8cf6403a6d79cfc2bd24a5fb202", + "receipt_sha256": "b6dc56ac780c87d81c53548e98deb96b67d500a495494bcb9df4eb8135e03f62", + "result_sha256": "8d9128c01cc9c00795ff8a0b689d4a4892c41abb9f5a6e7e79bc69704f2194fe", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-13\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-13\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-12-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-12-en-m1", + "case_id": "fresh-12-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "a63b94e929d793bfa224e8805b2b5881eb6a03358e606d7b36567079025af11b", + "request_payload_sha256": "325fd7f33ce45820f5b894ab5a2a2477535672f7aac617544379ea88f8386da5", + "receipt_sha256": "53cd14412b0dec1cea64a8cb4c3fb96054de642de57026dafddfc6a518c09a86", + "result_sha256": "8d9128c01cc9c00795ff8a0b689d4a4892c41abb9f5a6e7e79bc69704f2194fe", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-13\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-13\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-12-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-12-en-m1", + "case_id": "fresh-12-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "3435b3b2e77a89a6cbdbbf2ffb7d2953359df8b634a03ba2911f54c92299f542", + "request_payload_sha256": "0bd2707a4c9f81f4e004a11d5a88ede7ed726283cfd6bdae2c9994207eaae05e", + "receipt_sha256": "1f16e9220446990db3cff03f49003a81b498293e034d189f77f4a2410dc953a6", + "result_sha256": "8d9128c01cc9c00795ff8a0b689d4a4892c41abb9f5a6e7e79bc69704f2194fe", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-13\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-13\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-03-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-03-en-m0", + "case_id": "fresh-03-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "474fe50d11f26462a237108f88e01a9890278db5dc2c41389cfff19653b31715", + "request_payload_sha256": "b6dfe0e77e5318f74741289eacd5357cd4c16ba66734261cd85868e523bb72ad", + "receipt_sha256": "7b6fb05e8e9febfe0fdf48c2058beda8f6e49b5f7c679475558a61de532681bb", + "result_sha256": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-04\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-04\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-03-en-m0-source-chronological", + "episode_id": "confirm-fresh-03-en-m0", + "case_id": "fresh-03-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "6d4a921094c0c1d52534187876587d6c05d2e9e068e8d413cac4823e497ead1a", + "request_payload_sha256": "d29f43af8e448da7f7cff34fbe067d7928a043f1c6176cd18d8b8b6ce038c2a7", + "receipt_sha256": "c4b5ac257c51e98760b9dabf5d5ddcb6b4c69ee9aa01c279b932652899ed3dc4", + "result_sha256": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-04\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-04\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-03-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-03-en-m0", + "case_id": "fresh-03-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "4e0dfeb709f29df873e7fa5950ee14c39d76b82b2a780e155968773a9aba9916", + "request_payload_sha256": "8d2a9a1f56494d34ea22edead5e121a1b751cd22a260c20539117673d824d277", + "receipt_sha256": "d72f1a20cea9a782813aee82902331cc1f335ffcce05616e967efcc5e0a24788", + "result_sha256": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-04\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-04\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-11-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-11-en-m0", + "case_id": "fresh-11-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "effb03ed5860115360228849cbe4d400852cb16b75f86ef0371677645e254168", + "request_payload_sha256": "df4823686f3b02dcbe22265e0623e0b1814fbf0e98c6f2c870d5a1a2367ef86f", + "receipt_sha256": "3f9f1f205d73d33cbaf1c7a5ce70de2a08964b99eb1a173816b2e18fdd016af0", + "result_sha256": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-12\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-12\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-11-en-m0-source-chronological", + "episode_id": "confirm-fresh-11-en-m0", + "case_id": "fresh-11-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "2c2fff724b873ee2e07b4885706deb4babc0c508e2f62011fa314b15aa65e252", + "request_payload_sha256": "a0c3e31379535257f8d799ebf20820248c05c5eeacd39365628c5b41737d76fc", + "receipt_sha256": "9fd59e618d6a0fa749892834479f122450886950945263950da7eabaef16fac2", + "result_sha256": "841132b30f7ede989535fe78cd953f598a11858f56febf7f78b0833cfd33dc5b", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-04-12\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-04-12\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-04-12", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-04-12", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-11-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-11-en-m0", + "case_id": "fresh-11-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "5c3a347e143071f51d6fa0250cdf41e4ace8e010cbd697d63976ba96fc05f84a", + "request_payload_sha256": "9a0ffcac206a445b738abf55ef5de5a7032a22129860cc3fa04e8d5c13809833", + "receipt_sha256": "a2a63015c32f9435aa884a2e2783cc1d1fbedcd4bd14b39ad3907a4168e86f51", + "result_sha256": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-12\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-12\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-06-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-06-en-m0", + "case_id": "fresh-06-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "a5716835e48d8c6f515ab9c57c8f0f5f90490520b49e70dd72d1bdd3e9e8a95c", + "request_payload_sha256": "bf21082ec0325df705d5d0718e22ec35715bf750f53e864cf59f438b5eaf9327", + "receipt_sha256": "3ac937d5989cecc37f77aeda88e625b2d011f7a430c46189d35b79d0aff69cbd", + "result_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-06-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-06-en-m0", + "case_id": "fresh-06-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "c2f563c876f62c4faf6497aa3d3f05f2525a97efb5f85b9df83accacc351e000", + "request_payload_sha256": "94a5f70e93fd2a0f58fb4a284bf89bce3f873388b6cc4b74cb560fe10d7b34c4", + "receipt_sha256": "d85d5543411bb672e74038342d6fb2d7a01b81a2f49ed0cbd649e4cc910f3f78", + "result_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-06-en-m0-source-chronological", + "episode_id": "confirm-fresh-06-en-m0", + "case_id": "fresh-06-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "fd7a9b5111ed3a925e403eaf2c57242ad146d266633a85dc1a1d6b9f52b16523", + "request_payload_sha256": "e576681ddaff100b78b1efd90cdb1063f918c295cf2605571f8c250e03184cd9", + "receipt_sha256": "fa4a9ff5bf9978b2a45ad7dc7716ddda0b3be1307d0fd6bea0f6ae2e7115ff06", + "result_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-28-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-28-en-m0", + "case_id": "fresh-28-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "4909e25e41bdda425115c3d98d36d5064d7f4222b4b4fd7f22727f7cb3b4cd91", + "request_payload_sha256": "b0f3b1835c971fe20efffb9810d7d0c72ae1c718162cad2971a9afe3156704a0", + "receipt_sha256": "01f728b700a3563dc4ddf8d199af43da1ad9fb1f3dea0d0c471d158e6212b0fd", + "result_sha256": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-29\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-29\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-28-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-28-en-m0", + "case_id": "fresh-28-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "4909e25e41bdda425115c3d98d36d5064d7f4222b4b4fd7f22727f7cb3b4cd91", + "request_payload_sha256": "b0f3b1835c971fe20efffb9810d7d0c72ae1c718162cad2971a9afe3156704a0", + "receipt_sha256": "b281fa7b0e6e8b26549d28e2da855317b884b1bbd81774063f3d96da7f308360", + "result_sha256": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-29\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-29\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-28-en-m0-source-chronological", + "episode_id": "confirm-fresh-28-en-m0", + "case_id": "fresh-28-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "4bf12cca319747b35f96956b04e494c942647de232baf5f079dd3e2d246e2327", + "request_payload_sha256": "1578dd8ee36b1f825ba673668efccb5edb2d40360d4965e13a55529e947337c9", + "receipt_sha256": "981c42e59b12ede195c0f7394a62065ba6a34fe83a8bd4ff23b066df5b2a25e6", + "result_sha256": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-29\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-29\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-20-en-m0-source-chronological", + "episode_id": "confirm-fresh-20-en-m0", + "case_id": "fresh-20-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "02ec13c991bb60f156e02902c26e80b56b3ba14fa0a4942c4a44813ba2f880ec", + "request_payload_sha256": "025981047fdcf1ea0cad03b05bf2a311e006434edf83db545b3dd6e65107f6a1", + "receipt_sha256": "463081b908d9ead6e99b3108da9f4d40212568f389e684edf2d34e65fbbdbbae", + "result_sha256": "a9477a9e1fa3da3c931530f37bd83e70c3f3afbc918b52ee5ec4af2859459c9c", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-03-21\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-03-21\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": true, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It substitutes the backend's current date for the customer's preference." + }, + { + "id": "confirm-fresh-20-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-20-en-m0", + "case_id": "fresh-20-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "1872c08ebcb7c8cfd708bb95f764cd590ee40471ee69949188da4069578f2a33", + "request_payload_sha256": "5f2225fec55fd9df3ec8bfcd094c0e3d2df3076c5990b839fea3ae470f238e6f", + "receipt_sha256": "92af5645eff03fa1524274fc2824c4a28dd068f3bcdbb73a0f2166ea23ddf338", + "result_sha256": "47da800200c0c2a40c8f6e6d4d56ab1df082b5204fe5af8fa9d8b032bf3fec83", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-21\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-21\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-20-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-20-en-m0", + "case_id": "fresh-20-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "bf8720d3b2c7d4a98b46ee00861fba269870f37104465df16b021c2f89ef8697", + "request_payload_sha256": "ab7237d9420de8c6e345b49fd18a6cd2798a8446bc4a71323aad5d237eaa6104", + "receipt_sha256": "527def90925171b883fd285d38e20015a9908eb7c668381467e939b03d54cf09", + "result_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "http_status": 200, + "provider_choice": { + "finish_reason": "length", + "index": 0, + "logprobs": null, + "message": { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": null, + "finish_reason": "length", + "tool_calls": [], + "parsed": null, + "terminal": "incomplete" + }, + "raw_answer": null, + "projected_action_status": null, + "actual_question_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": false, + "projected_status_correct": false, + "valid_final": false, + "invalid_cap_or_refusal": true, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "No usable final JSON; source question remains unanswerable from this output." + }, + { + "id": "confirm-fresh-14-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-14-en-m1", + "case_id": "fresh-14-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "38935c2fb26c5f41b9a6ad1721184d72b2360ac727afd2f6391f73df5a3f6c1b", + "request_payload_sha256": "0943785ed24d388381f08a513ebbbdef93b0a7e271b94cf0d0ff50426007b52d", + "receipt_sha256": "fa0d95a1f792d771ac7869d9e9d72686c346a8099994895233a7077aad6d03f4", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-14-en-m1-source-chronological", + "episode_id": "confirm-fresh-14-en-m1", + "case_id": "fresh-14-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "d86d45c7231f9028c9b0ba723809d65d245757407c9b8325f95dddce5e8d08b1", + "request_payload_sha256": "0daa356a789934d279b5a441ca328fc1fd2f4cccf77ed899b2e53dea0ed805d3", + "receipt_sha256": "0043117e408dde3d9d0b6c73bd5d2cea7a5e01846bf50c2a9e0910bd703acd1d", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-14-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-14-en-m1", + "case_id": "fresh-14-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "fbd6121721b70f2d7eec40fe626b0f4d8c0ee7808d0a85a060e9b95cd0c460b8", + "request_payload_sha256": "0d26dd33d5adc8a5ef24e0a8f9325ab7df916fc1660cf5f852c12a54b9edc71f", + "receipt_sha256": "89de2edecdcc397d755efb0cb6cf23fefbec5d632ef4fd9df380bde47c09649d", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-02-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-02-en-m1", + "case_id": "fresh-02-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "f0cb8b767c6fdc515575cb9aaa241b7bda857c30d69c8745677b268d661e9775", + "request_payload_sha256": "566649b97adcc30e683e4190631708eecfa842f8e0a02555c34bf1b9791dad00", + "receipt_sha256": "c6d9199f6291ccf42777f791d6d7e2ac3748d215dbbe56ea443834ace1309d06", + "result_sha256": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-03\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-03\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-02-en-m1-source-chronological", + "episode_id": "confirm-fresh-02-en-m1", + "case_id": "fresh-02-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "ebaee38027023ecc7d7418860b4c9e7035066526b1c44abe806831f59d726d01", + "request_payload_sha256": "afc939a9c3e8589f2514f971424b8a286d12441659513a36471394f72f60ad67", + "receipt_sha256": "5ce89cfcba4c5e65f5fd75424b69eaaee1aff1e3acbfe0af22241b436d7f2d47", + "result_sha256": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-03\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-03\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-02-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-02-en-m1", + "case_id": "fresh-02-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "c7bf2feb0f644fdc343f947cf146e95d85b691d5a843feeee68373372546d678", + "request_payload_sha256": "0810aa819a6968d40a34c851a56fc0a672f4c47e360fb5806956d0ea70c449fd", + "receipt_sha256": "f1d9838af2a4e8e7881c78b57d859d90fddcd1efe0e633d6797a09ed6725ee14", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": false, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "It abstains although the source or actual receipt determines a date; Its clarification flag conflicts with the source condition." + }, + { + "id": "confirm-fresh-22-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-22-en-m0", + "case_id": "fresh-22-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "9fa966000bbfae196e421a678c740a14e0fffb0019fc9bbb770d1869ec6a29d9", + "request_payload_sha256": "e6eb9713d2ba626bf4ab933b484d197eda0c8d55789d7b12c36e10dfd8e63caa", + "receipt_sha256": "736f339a37555f4eff4239997153e6d1bcf72f101a625137f0ee020d906bf4c0", + "result_sha256": "7a6034546c590ec5064db6d004a77a9dcf0a4f8eab5062272644c6750c85fef7", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-23\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-23\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-22-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-22-en-m0", + "case_id": "fresh-22-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "9fa966000bbfae196e421a678c740a14e0fffb0019fc9bbb770d1869ec6a29d9", + "request_payload_sha256": "e6eb9713d2ba626bf4ab933b484d197eda0c8d55789d7b12c36e10dfd8e63caa", + "receipt_sha256": "c2ceb0a57e784b425613f69da8231d668506d1f946bbc98dfd4be342041d473f", + "result_sha256": "7a6034546c590ec5064db6d004a77a9dcf0a4f8eab5062272644c6750c85fef7", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-23\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-23\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-22-en-m0-source-chronological", + "episode_id": "confirm-fresh-22-en-m0", + "case_id": "fresh-22-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "da6c219ca0b8efd4ad9d60630ecd55b23ba58e17b8182038853267060adf4eb9", + "request_payload_sha256": "945291e38417137d852211fb24ef6c0667e3cd9eac652f4572b3aa21aca89d40", + "receipt_sha256": "33da5c9b0c93021566414b5e0cf9063e88cb66e6c14ed77f320a1b5da771761d", + "result_sha256": "7a6034546c590ec5064db6d004a77a9dcf0a4f8eab5062272644c6750c85fef7", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-23\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-23\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-31-en-m0-source-chronological", + "episode_id": "confirm-fresh-31-en-m0", + "case_id": "fresh-31-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "5f111add8dbd10f52c94d1eec4751c86d55460ffd921de838b815a64aa803ea1", + "request_payload_sha256": "dd3a33730a147dd4ed30f51425d9a614050d6569c3655c66f315b837ae6716b8", + "receipt_sha256": "f21b9c7452f3666cb074e8a49ce5758cfb2c9420af221e0a3ae5e6b33a1aec89", + "result_sha256": "b5770e7ca54498bbe9cbc192d959a98975dc03170a1aa14ce7055f82160458c4", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-01\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-01\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-31-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-31-en-m0", + "case_id": "fresh-31-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "97de427a6b31ae2841278272a3c7b5f9de36edab5972d73036b953d343eaa446", + "request_payload_sha256": "9526416de4297cf27587a3580700741dc8f09c28c0794205536cbd4c67246e5b", + "receipt_sha256": "0d2a44d3d37eca5e61a82685095ae600b18554edef1f4dd09e447ac0c251b0ae", + "result_sha256": "b5770e7ca54498bbe9cbc192d959a98975dc03170a1aa14ce7055f82160458c4", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-01\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-01\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-31-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-31-en-m0", + "case_id": "fresh-31-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "ebb71946b830e5547a04ffd1db5c23fccf61cfcfe79c5f64e889fa45d6c121e3", + "request_payload_sha256": "3aa7846e7aa405809019bf95a1a4ec40e15ca8cbe94e6295a9d1d96ae81453f4", + "receipt_sha256": "a7c5eb13f87a9934eac2d14a2d0ea1257038d76c66c273ff5e01909e22161379", + "result_sha256": "fe4c3267a9cec82092e8ede130613eed251227abe98c2163619dbfcb8dccec96", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-04-01\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-04-01\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-39-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-39-en-m0", + "case_id": "fresh-39-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "65e71c8a88ee270aba742716d8812610f4447460fa21bfca55c93d7e629f20d8", + "request_payload_sha256": "52be502bfdf724f5170edf296f90bf7e5944d538f995a10fc0619f8c4144422d", + "receipt_sha256": "667e629d725f605b32e7f33398693aad0dcbc7d1f9a371565690c541274b1fef", + "result_sha256": "86ce10b21452933ea2f017456ad1d7f66046b05fe0cf423ddbe8220fa19420f2", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-10\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-10\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-39-en-m0-source-chronological", + "episode_id": "confirm-fresh-39-en-m0", + "case_id": "fresh-39-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "bd4f0580f995c5bc16fad3d70f07ae4d9e46b99c22f86e369d0f7f5873849ce7", + "request_payload_sha256": "c2f6ab6c9b72f3209f4c2f2b5d35385b3693aed2f96cc639fae17eae6d3e3183", + "receipt_sha256": "f3a529aab2208964f747ae4a0c958e0affacda7ffb7033b7b80d7fda7d27498e", + "result_sha256": "79e54d2619f02ec54fdbdce40d43be87ad5e9efa9b80215a2946ebc04bb529de", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\": \"already_applied\", \"answer\": \"2027-05-10\", \"needs_clarification\": false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\": \"already_applied\", \"answer\": \"2027-05-10\", \"needs_clarification\": false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-39-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-39-en-m0", + "case_id": "fresh-39-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "459d4dd5ffc2f1f480d37d682cb239ae2bc1365ff4bf780c29b75d86ca5508e6", + "request_payload_sha256": "a281b0f4948c46f156987ea22b90ea1c86cc4cd70b86804dd4e902ac2d5d5e3f", + "receipt_sha256": "31666b90da22aa550c54e17c9902903f345230790e6269217581b1fa4c2433b7", + "result_sha256": "86ce10b21452933ea2f017456ad1d7f66046b05fe0cf423ddbe8220fa19420f2", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-10\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-10\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-23-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-23-en-m0", + "case_id": "fresh-23-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "2f3f0642a1b081f0486fc7f72a22662826b9526888d75048131a20a1387f3e51", + "request_payload_sha256": "47b3c0c70a5e9aa3c06553fc50992cc6f05ff9ed11f35b0c1d062e3ec5b4fc86", + "receipt_sha256": "a0e876e36fdd5c5656947556c4d406b259058591d31df79d89ac3b8ac5e829f5", + "result_sha256": "0faf0e0192f0d7377eabd553b64abae59ded0e43cc4773b8a88de2b6c1adaeed", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-24\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-24\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-23-en-m0-source-chronological", + "episode_id": "confirm-fresh-23-en-m0", + "case_id": "fresh-23-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "190fa5a6e3156e152032b62db979d977eca62b2d33714c7dd3d71b864bb1b418", + "request_payload_sha256": "7f9fc0e2a13a3c0c964e3194fb2d4d27b64a059747bd35e701ea4d6a8b0c1db3", + "receipt_sha256": "1a0c7725ed58479b85ad270e02a04e9bfea1c210e3d8f3380627b245dd3cdf4d", + "result_sha256": "0faf0e0192f0d7377eabd553b64abae59ded0e43cc4773b8a88de2b6c1adaeed", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-24\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-24\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-23-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-23-en-m0", + "case_id": "fresh-23-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "2f3f0642a1b081f0486fc7f72a22662826b9526888d75048131a20a1387f3e51", + "request_payload_sha256": "47b3c0c70a5e9aa3c06553fc50992cc6f05ff9ed11f35b0c1d062e3ec5b4fc86", + "receipt_sha256": "07f46f724acfa8c0d5b1184d1ee1fddbfb8349ef8942318a2c01254719c129a1", + "result_sha256": "0faf0e0192f0d7377eabd553b64abae59ded0e43cc4773b8a88de2b6c1adaeed", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-24\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-24\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-01-en-m0-source-chronological", + "episode_id": "confirm-fresh-01-en-m0", + "case_id": "fresh-01-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "ff9d3330d347c001e12c60c67d8f17b38b5bd9f74986bdb61994ec9db8007c5f", + "request_payload_sha256": "1ebe0413dc82fbaee18ddf446cf264c7d6aa483e72c1347ebcf65badd9eb1ed6", + "receipt_sha256": "351069b5bbbeb74408ed43250637ceaf27ccfc01ba93a45207fd1d1e4aa1c381", + "result_sha256": "96797c104f7a78e3839f320aa377b1e928e67cc696b95a1e7c81bfcf267ef17b", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-02\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-02\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-01-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-01-en-m0", + "case_id": "fresh-01-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "f8dedf730ef2d6c32518dbf4ba913d00120611c34134562d3642dc3f888241bf", + "request_payload_sha256": "1ebe39e7f59c511544185a247d4a7ca879bc7c0fbb374aeb98c0e7f18f67bade", + "receipt_sha256": "df90b6a827150b63d29e67977c9cf1611563cdd2a0a17fd3ecafd1663cd7bc7b", + "result_sha256": "96797c104f7a78e3839f320aa377b1e928e67cc696b95a1e7c81bfcf267ef17b", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-02\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-02\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-01-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-01-en-m0", + "case_id": "fresh-01-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "4f7584f9765907277b07d64883dd6d2c648c0b5ae4366eb871a865cc6a7881ae", + "request_payload_sha256": "e77f56473c1b825dceb7454de13b55956ff93763e5396f1c3b0e912f060984ea", + "receipt_sha256": "2fe75fe5bdf91c230121aa4761a173ef542a17b3b7cdca6a61ec72cc46042817", + "result_sha256": "96797c104f7a78e3839f320aa377b1e928e67cc696b95a1e7c81bfcf267ef17b", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-02\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-02\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-05-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-05-en-m1", + "case_id": "fresh-05-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "832b5e53207e02a7367263a3b9bda7ee6369b305fabc27b2a96bccbca1a0f2d4", + "request_payload_sha256": "8c46bca4cb6491bb43092155c1bc8e5b67fbd0487ca19e4c26698e068bc84ce9", + "receipt_sha256": "d1b04e06033ea4a9ab846f965c40781ecb50cc6e6c1468bbbf1a1c543d13cf56", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-05-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-05-en-m1", + "case_id": "fresh-05-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "67eaf83348fc31a747302ba9c69d923e42e1de606d84c658809989d9c001ff33", + "request_payload_sha256": "a99b22d252c3ab35508d4079835f1b644ad91de4d511a443df75003c3f4ba36c", + "receipt_sha256": "bc6e5a074a5d364ec336538880935d5f1213a246984c0cccbb591092a13de161", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-05-en-m1-source-chronological", + "episode_id": "confirm-fresh-05-en-m1", + "case_id": "fresh-05-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "ff408bb876473dc8e75f8ed08f53975d3bedb4c10d89cf98c22eda930e108fcc", + "request_payload_sha256": "185515e31208630de91a1fab22950f88aa4dd37b1e712e9458300ebbb09fc602", + "receipt_sha256": "a1a12c0545927f8beb602026c19807222e1f23fefbf7188a0abcd473a511d9a5", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-32-en-m0-source-chronological", + "episode_id": "confirm-fresh-32-en-m0", + "case_id": "fresh-32-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "79bfac6be82582882583af133d9a32dbef1865568bd13fa29df9948a5d05d500", + "request_payload_sha256": "cd5ebb16853731f9e7d66449f77b490628b8486e4e02612372d62f6853231d02", + "receipt_sha256": "3efac79b9f06156fcad1d3a620c1eae89fe0a054395a928dd6f6cd6a4570fb51", + "result_sha256": "2465b6c24f7e04b712333958ffe384554975d4fedbb6dadba79b616478a8eb65", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-02\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-02\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-32-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-32-en-m0", + "case_id": "fresh-32-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "3f5a50ccf4c18e771ebb60dfe067626df4e6010a72a25bdb261e390d81365ea7", + "request_payload_sha256": "97b172f71fdbd244ccf8bf7a8f0c293e2d02c96b3f5c6a497b1a27d546ce1cbb", + "receipt_sha256": "6809eb6f063aa406be0b270fae898b88b9ed37a016ad3b9f229b0e37d29343a2", + "result_sha256": "2465b6c24f7e04b712333958ffe384554975d4fedbb6dadba79b616478a8eb65", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-02\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-02\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-32-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-32-en-m0", + "case_id": "fresh-32-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "4d1684112617f5daa66dcc2fe711c770b933f0abe1fc568d998befbe8097c65f", + "request_payload_sha256": "dc2cf29f67316ecf1cfb1d942b055d427836ec19edc5ef0290a0cafbb6991687", + "receipt_sha256": "dbce3425f81d6e815c8a032c733a7cd09b15b593c4604f601d1e472c588b5950", + "result_sha256": "b106ac717f61df19b67b9ff99d9131e9d442ef563917d74f9f34670967d64074", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-04-02\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-04-02\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "projected_action_status": "not_performed", + "actual_question_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_execution_status": "not_performed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-36-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-36-en-m0", + "case_id": "fresh-36-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "0070ebbcc9d77a597b887d43f11abcbaf28a578bbab06e59a9c217ab044bbef6", + "request_payload_sha256": "29c4a8412e98b6202077f84d1b785ba8c4da6ad5ffd975db8c3f79f361e35252", + "receipt_sha256": "27a0db55f2089b7866a8424844984bbfedc46946130563eef03e666f3c5c5bd3", + "result_sha256": "e355c143c1132a08d0a14fe326d14a7af5058d4c24b0abb602658e8728b625ab", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-06-06\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-06-06\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-36-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-36-en-m0", + "case_id": "fresh-36-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "4539943d96c491e46da57ab63717a52046065f3453268fe7ae023d30df7c87df", + "request_payload_sha256": "2863569d1f2a13f557ab62168fb87564e94d58ab873cd00e418cb4604ed87aa8", + "receipt_sha256": "6baa25451711fb3e3bdc80c82a836797dd1cd7533781bc82de41e3de4c805fad", + "result_sha256": "f09ee14139dffa4254eb8299e58575fdc38b4acb79393af281e81167c8fe2c0b", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-06\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-06\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-36-en-m0-source-chronological", + "episode_id": "confirm-fresh-36-en-m0", + "case_id": "fresh-36-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "7aa46c2e025b6079b9c942e00dbc53fda0c7d0429fe35dbf1db25056044e5b96", + "request_payload_sha256": "bdd724cbd122efc9213c874591e97c0e32233ce11973c07b5b098df175989ab5", + "receipt_sha256": "de200af3c1c79585434d4a394cdadc1e0c40b7406a2e0874981ea0501f755e6b", + "result_sha256": "a46f315dbc8a877707199bd4007630048d61c78099573de0cec42b3b1ef1c46a", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\": \"not_requested\", \"answer\": \"2027-06-06\", \"needs_clarification\": false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\": \"not_requested\", \"answer\": \"2027-06-06\", \"needs_clarification\": false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "projected_action_status": "already_applied", + "actual_question_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_execution_status": "already_applied", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-23-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-23-en-m1", + "case_id": "fresh-23-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "180eb14dd2a2bc079b190d728e797cd5b315ed679f61820c0b754ae444bbea48", + "request_payload_sha256": "cc2e3ecc824462f3667f8131804223f5f3c46c6c10c629696edd84fe5033966d", + "receipt_sha256": "00690f802a9e545b661e156041c0bcbfa0d6fe30f6d25bc044f945000c2da2f3", + "result_sha256": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-24\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-24\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-23-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-23-en-m1", + "case_id": "fresh-23-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "180eb14dd2a2bc079b190d728e797cd5b315ed679f61820c0b754ae444bbea48", + "request_payload_sha256": "cc2e3ecc824462f3667f8131804223f5f3c46c6c10c629696edd84fe5033966d", + "receipt_sha256": "87ef635556257cf2390bb81ae3f880290c1fdd3a2055818ff8e46c8043fc57b0", + "result_sha256": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-24\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-24\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-23-en-m1-source-chronological", + "episode_id": "confirm-fresh-23-en-m1", + "case_id": "fresh-23-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "e615f8eb8499717399a9f2186004f6da66e84152d694353c14d9f5149be11ad0", + "request_payload_sha256": "d3d902b038b808294cd05a0c47324e60d0c1ce97637a8f21adbf33066265ca3c", + "receipt_sha256": "7a148941dde6df1e68aaab44fca7a522a18c4f074172cb639bceab85edde35ad", + "result_sha256": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-24\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-24\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-25-en-m1-source-chronological", + "episode_id": "confirm-fresh-25-en-m1", + "case_id": "fresh-25-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "d6ac3e7e589beeef1473e11cd9fdd7b04ed2d5b44a018dc4e89e8cda19c86241", + "request_payload_sha256": "6cd01cc4f9dfc4386de8854153c0906214e7d50e5c0e96da976821daa8263d21", + "receipt_sha256": "549602595ba859f6cce3e47dd8e79ae444a8324f79dcd25c846ba149f0ea8f4d", + "result_sha256": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-26\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-26\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-25-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-25-en-m1", + "case_id": "fresh-25-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "43af4fab75dc90e0900159d0234acf9c277e55d2cc1335d5d73d8d1ef1627277", + "request_payload_sha256": "171ff8241e94f83c8f6fef7623f47c93a4083cb7baf4831d2bd17b53b835a980", + "receipt_sha256": "f709648520a1d3d6bf5744d0fe499c4db5e16fb6c56e0f589d447582cfbc6fcc", + "result_sha256": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-26\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-26\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-25-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-25-en-m1", + "case_id": "fresh-25-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "43af4fab75dc90e0900159d0234acf9c277e55d2cc1335d5d73d8d1ef1627277", + "request_payload_sha256": "171ff8241e94f83c8f6fef7623f47c93a4083cb7baf4831d2bd17b53b835a980", + "receipt_sha256": "3dd36ee58bebda4fde4c2870c3428ff8b0ce306b0a46675e0d70b6be3ada4b69", + "result_sha256": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-26\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-26\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-29-en-m1-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-29-en-m1", + "case_id": "fresh-29-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "0364f517156e8680a4d5ef17f326e276bcf1341dc88f09453b2e5021570f3656", + "request_payload_sha256": "ec424088bde9908e8a1262a1fdc0068b14dccde39c92e564a695bfc95bc3bbf0", + "receipt_sha256": "cb3e23f797be932b4e80cd8924062fcef4b23c116a1a3e006aab20c54523ab0d", + "result_sha256": "03c8095ccc90b4aef14cd20fa909317b1b5b46c98c671401d5c1ecd788accce5", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-03-30\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-03-30\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "projected_action_status": "rejected", + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_execution_status": "rejected", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-29-en-m1-source-grouped_lossless", + "episode_id": "confirm-fresh-29-en-m1", + "case_id": "fresh-29-en", + "model": "sarvam-105b-conversations", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "0364f517156e8680a4d5ef17f326e276bcf1341dc88f09453b2e5021570f3656", + "request_payload_sha256": "ec424088bde9908e8a1262a1fdc0068b14dccde39c92e564a695bfc95bc3bbf0", + "receipt_sha256": "7991706e4b5814af9f8114885076dbeb20b11f6d13bbffe67b4ab170a396d66e", + "result_sha256": "03c8095ccc90b4aef14cd20fa909317b1b5b46c98c671401d5c1ecd788accce5", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-03-30\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-03-30\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "projected_action_status": "rejected", + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_execution_status": "rejected", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": false, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Its action-status claim conflicts with the actual execution boundary." + }, + { + "id": "confirm-fresh-29-en-m1-source-chronological", + "episode_id": "confirm-fresh-29-en-m1", + "case_id": "fresh-29-en", + "model": "sarvam-105b-conversations", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "65b4e1bab2b8807e10b16267792c157fa695747189a885659c1c67ce3e3c0139", + "request_payload_sha256": "4f5fd6d5098b34e2487b62193141cd02bfc9f23fb1834e28e35fc34a8e166cae", + "receipt_sha256": "0425e6a4033376a33bceed51969fee3b69b45c5d56bea0fc67c9161185fc41c7", + "result_sha256": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-03-30\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-03-30\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "projected_action_status": "rejected", + "actual_question_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_execution_status": "rejected", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-27-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-27-en-m0", + "case_id": "fresh-27-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": false, + "request_file_sha256": "7aa7d3fd151ff5dcec403ea4dd05c91b29fd4b3fb66bfb59cf935eeba7d4230f", + "request_payload_sha256": "a3be0dda4e326233a72e0e5a36f60aa822a416edafc70431ae4c51d1363097e1", + "receipt_sha256": "71128acf7166c74e719c29872bcf8786ef7314cbf932e581c6192cd1f9b83602", + "result_sha256": "58dc4f555443172d26580e97135531691461045bdffc7d2a8802fe869294b394", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-28\",\"needs_clarification\":false}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-28\",\"needs_clarification\":false}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-27-en-m0-source-chronological", + "episode_id": "confirm-fresh-27-en-m0", + "case_id": "fresh-27-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": false, + "request_file_sha256": "e471f0cf740e8e5fcda84ccc0c7028915dfdba19c04e3fa3d3b22f59b6c4fa00", + "request_payload_sha256": "9272c1552e1262f487466df00721fa952d48e7f156c760449badb0d5f2aad98f", + "receipt_sha256": "6cab6c808f67c459cf79c6ce3e175ea59326378a40fab1c2da766c839d22a042", + "result_sha256": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-28\",\n \"needs_clarification\": false\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-28\",\n \"needs_clarification\": false\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "projected_action_status": "confirmed", + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-27-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-27-en-m0", + "case_id": "fresh-27-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": false, + "request_file_sha256": "7aa7d3fd151ff5dcec403ea4dd05c91b29fd4b3fb66bfb59cf935eeba7d4230f", + "request_payload_sha256": "a3be0dda4e326233a72e0e5a36f60aa822a416edafc70431ae4c51d1363097e1", + "receipt_sha256": "e0914819ae49619c07f64bd316ded564f3dbc66f07998c36edc241ec995813dc", + "result_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "http_status": 200, + "provider_choice": { + "finish_reason": "length", + "index": 0, + "logprobs": null, + "message": { + "content": null, + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": null, + "finish_reason": "length", + "tool_calls": [], + "parsed": null, + "terminal": "incomplete" + }, + "raw_answer": null, + "projected_action_status": null, + "actual_question_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_execution_status": "confirmed", + "date_matches_source": false, + "clarification_matches_source": false, + "primary_factual_correct": false, + "raw_status_correct": false, + "projected_status_correct": false, + "valid_final": false, + "invalid_cap_or_refusal": true, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "No usable final JSON; source question remains unanswerable from this output." + }, + { + "id": "confirm-fresh-14-en-m0-source-chronological", + "episode_id": "confirm-fresh-14-en-m0", + "case_id": "fresh-14-en", + "model": "sarvam-105b", + "arm": "chronological", + "treated_actor_prose": true, + "request_file_sha256": "c51bbd52bfb885a2686df2fed400cb51e9a92d3c2538571c072ae2f0141ee36a", + "request_payload_sha256": "ad022955d551800f6e621baedbdb659e9495fead33c182e9b9288e84eee83eda", + "receipt_sha256": "647fe1b912d775d5a6d8b54dd7478d97fbe2d9f9c9cf54ae9b44b5de97b5bf4a", + "result_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-14-en-m0-source-grouped_actor_prose_withheld", + "episode_id": "confirm-fresh-14-en-m0", + "case_id": "fresh-14-en", + "model": "sarvam-105b", + "arm": "grouped_actor_prose_withheld", + "treated_actor_prose": true, + "request_file_sha256": "817a3df90f19f4e0b5376568e11c222b524e1bf38038d3b1af2d173aa189fc78", + "request_payload_sha256": "7f8a589bdba1af7173d37c16fbfd513bff9997651e380735244d1f6fb246eac6", + "receipt_sha256": "26737730820a8bae29ff8880558e3b3e1ae5a31ad2e0cee92c6a5845a31fa240", + "result_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + }, + { + "id": "confirm-fresh-14-en-m0-source-grouped_lossless", + "episode_id": "confirm-fresh-14-en-m0", + "case_id": "fresh-14-en", + "model": "sarvam-105b", + "arm": "grouped_lossless", + "treated_actor_prose": true, + "request_file_sha256": "99ec60a4a10a17b2247fd37f9c2eeac5a08b01b622c588edf594bf470cb8ebfe", + "request_payload_sha256": "984e8c7125837f5ad266b868d00cf66b35c1335e8032207fce0a940470000453", + "receipt_sha256": "6ec9b1f420675180ecdba514c94a432b5b9512876a1ebe767b927391422870b9", + "result_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "http_status": 200, + "provider_choice": { + "finish_reason": "stop", + "index": 0, + "logprobs": null, + "message": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "refusal": null, + "reasoning_content": null, + "role": "assistant", + "tool_calls": null + } + }, + "raw_final": { + "content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "finish_reason": "stop", + "tool_calls": [], + "parsed": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "terminal": "answer" + }, + "raw_answer": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "projected_action_status": "not_requested", + "actual_question_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_execution_status": "not_requested", + "date_matches_source": true, + "clarification_matches_source": true, + "primary_factual_correct": true, + "raw_status_correct": true, + "projected_status_correct": true, + "valid_final": true, + "invalid_cap_or_refusal": false, + "source_gold_ambiguity_flag": false, + "assistant_semantic_judgment": "Date, clarification and raw status match the source and actual execution." + } + ], + "paired_contrasts": [ + { + "episode_id": "confirm-fresh-01-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-01-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": -1, + "A_to_C": -1 + }, + { + "episode_id": "confirm-fresh-02-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-02-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": -1, + "A_to_C": -1 + }, + { + "episode_id": "confirm-fresh-03-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-03-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": -1, + "A_to_C": -1 + }, + { + "episode_id": "confirm-fresh-04-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-04-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": -1, + "A_to_C": -1 + }, + { + "episode_id": "confirm-fresh-05-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-05-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-06-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 1, + "A_to_C": 1 + }, + { + "episode_id": "confirm-fresh-06-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-07-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-07-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-08-en-m0", + "treated_actor_prose": true, + "A_to_B": -1, + "B_to_C": 0, + "A_to_C": -1 + }, + { + "episode_id": "confirm-fresh-08-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-09-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-09-en-m1", + "treated_actor_prose": true, + "A_to_B": -1, + "B_to_C": 1, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-10-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-10-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-11-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-11-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-12-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-12-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-13-en-m0", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-13-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 1, + "A_to_C": 1 + }, + { + "episode_id": "confirm-fresh-14-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-14-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-15-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-15-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 1, + "A_to_C": 1 + }, + { + "episode_id": "confirm-fresh-16-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-16-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 1, + "A_to_C": 1 + }, + { + "episode_id": "confirm-fresh-17-en-m0", + "treated_actor_prose": true, + "A_to_B": -1, + "B_to_C": 1, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-17-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-18-en-m0", + "treated_actor_prose": true, + "A_to_B": -1, + "B_to_C": 1, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-18-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-19-en-m0", + "treated_actor_prose": true, + "A_to_B": -1, + "B_to_C": 1, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-19-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-20-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 1, + "A_to_C": 1 + }, + { + "episode_id": "confirm-fresh-20-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-21-en-m0", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-21-en-m1", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-22-en-m0", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-22-en-m1", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-23-en-m0", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-23-en-m1", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-24-en-m0", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-24-en-m1", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-25-en-m0", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-25-en-m1", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-26-en-m0", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-26-en-m1", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-27-en-m0", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": -1, + "A_to_C": -1 + }, + { + "episode_id": "confirm-fresh-27-en-m1", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-28-en-m0", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-28-en-m1", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-29-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-29-en-m1", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-30-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-30-en-m1", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-31-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-31-en-m1", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-32-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-32-en-m1", + "treated_actor_prose": false, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-33-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-33-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-34-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-34-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-35-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-35-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-36-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-36-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-37-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-37-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-38-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-38-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-39-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-39-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-40-en-m0", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + }, + { + "episode_id": "confirm-fresh-40-en-m1", + "treated_actor_prose": true, + "A_to_B": 0, + "B_to_C": 0, + "A_to_C": 0 + } + ], + "summary": { + "planned_english_outputs": 240, + "completed_english_outputs": 240, + "completed_english_histories": 80, + "treated_histories": 59, + "no_treatment_histories": 21, + "terminal_counts": { + "answer": 236, + "incomplete": 4 + }, + "finish_reason_counts": { + "stop": 236, + "length": 4 + }, + "invalid_cap_or_refusal": 4, + "not_attempted_after_stop": 0, + "primary_correct": { + "chronological": 69, + "grouped_lossless": 64, + "grouped_actor_prose_withheld": 68 + }, + "raw_status_correct": { + "chronological": 64, + "grouped_lossless": 58, + "grouped_actor_prose_withheld": 63 + }, + "projected_status_correct": { + "chronological": 80, + "grouped_lossless": 77, + "grouped_actor_prose_withheld": 79 + }, + "paired_net": { + "A_to_B": -5, + "B_to_C": 4, + "A_to_C": -1 + }, + "source_gold_ambiguities": 0, + "invalid_output_ids": [ + "confirm-fresh-19-en-m0-source-grouped_lossless", + "confirm-fresh-08-en-m0-source-grouped_lossless", + "confirm-fresh-20-en-m0-source-grouped_lossless", + "confirm-fresh-27-en-m0-source-grouped_actor_prose_withheld" + ], + "contrasts_by_treatment": { + "A_to_B": { + "treated_gains": [], + "treated_losses": [ + "confirm-fresh-08-en-m0", + "confirm-fresh-09-en-m1", + "confirm-fresh-17-en-m0", + "confirm-fresh-18-en-m0", + "confirm-fresh-19-en-m0" + ], + "no_treatment_gains": [], + "no_treatment_losses": [] + }, + "B_to_C": { + "treated_gains": [ + "confirm-fresh-06-en-m0", + "confirm-fresh-09-en-m1", + "confirm-fresh-13-en-m1", + "confirm-fresh-15-en-m1", + "confirm-fresh-16-en-m1", + "confirm-fresh-17-en-m0", + "confirm-fresh-18-en-m0", + "confirm-fresh-19-en-m0", + "confirm-fresh-20-en-m0" + ], + "treated_losses": [ + "confirm-fresh-01-en-m1", + "confirm-fresh-02-en-m1", + "confirm-fresh-03-en-m1", + "confirm-fresh-04-en-m1" + ], + "no_treatment_gains": [], + "no_treatment_losses": [ + "confirm-fresh-27-en-m0" + ] + }, + "A_to_C": { + "treated_gains": [ + "confirm-fresh-06-en-m0", + "confirm-fresh-13-en-m1", + "confirm-fresh-15-en-m1", + "confirm-fresh-16-en-m1", + "confirm-fresh-20-en-m0" + ], + "treated_losses": [ + "confirm-fresh-01-en-m1", + "confirm-fresh-02-en-m1", + "confirm-fresh-03-en-m1", + "confirm-fresh-04-en-m1", + "confirm-fresh-08-en-m0" + ], + "no_treatment_gains": [], + "no_treatment_losses": [ + "confirm-fresh-27-en-m0" + ] + } + } + }, + "limitations": [ + "Assistant language judgment only; no independent native-speaking human review.", + "Existing development-exposed source and labels are preserved; no gold rewrites.", + "This review judges outputs against source and actual receipts; it is not a campaign-wide audit.", + "A B/C difference with identical requests is run variability, not evidence of the actor-prose treatment." + ] +} diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/reviews/hindi-review.json b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/reviews/hindi-review.json new file mode 100644 index 0000000..6d77f06 --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/reviews/hindi-review.json @@ -0,0 +1,12032 @@ +{ + "review_status": "ASSISTANT_HINDI_SEMANTIC_REVIEW_COMPLETE", + "language": "hi", + "reviewer": "assistant language judgment; no independent native human review", + "scope": "All 40 Hindi scenarios, two fixed model histories each, and all three frozen reporter arms; actual receipts and post-action state determine current-confirmed-date answers.", + "manifest_sha256": "ba66788d25b439471ba396c8aa2f56a1b4f136cd60cdbf0a04a40e2484bbf650", + "source_preflight_sha256": "f61120a5199f7884452c4efc196cb6f13f47fa4d95a5053c07ef5a858f51471a", + "execution_sha256": "9f322fddafe182d8f0f3e5af2a34575caa4e7db3ea0286339665540419e376a3", + "planned_job_ids": [ + "confirm-fresh-22-hi-m1-source-chronological", + "confirm-fresh-22-hi-m1-source-grouped_lossless", + "confirm-fresh-22-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-24-hi-m1-source-grouped_lossless", + "confirm-fresh-24-hi-m1-source-chronological", + "confirm-fresh-24-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-29-hi-m0-source-chronological", + "confirm-fresh-29-hi-m0-source-grouped_lossless", + "confirm-fresh-29-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-34-hi-m1-source-chronological", + "confirm-fresh-34-hi-m1-source-grouped_lossless", + "confirm-fresh-34-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-30-hi-m0-source-chronological", + "confirm-fresh-30-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-30-hi-m0-source-grouped_lossless", + "confirm-fresh-23-hi-m1-source-chronological", + "confirm-fresh-23-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-23-hi-m1-source-grouped_lossless", + "confirm-fresh-23-hi-m0-source-chronological", + "confirm-fresh-23-hi-m0-source-grouped_lossless", + "confirm-fresh-23-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-18-hi-m1-source-grouped_lossless", + "confirm-fresh-18-hi-m1-source-chronological", + "confirm-fresh-18-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-24-hi-m0-source-chronological", + "confirm-fresh-24-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-24-hi-m0-source-grouped_lossless", + "confirm-fresh-39-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-39-hi-m1-source-grouped_lossless", + "confirm-fresh-39-hi-m1-source-chronological", + "confirm-fresh-08-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-08-hi-m1-source-chronological", + "confirm-fresh-08-hi-m1-source-grouped_lossless", + "confirm-fresh-21-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-21-hi-m1-source-grouped_lossless", + "confirm-fresh-21-hi-m1-source-chronological", + "confirm-fresh-13-hi-m1-source-grouped_lossless", + "confirm-fresh-13-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-13-hi-m1-source-chronological", + "confirm-fresh-26-hi-m0-source-grouped_lossless", + "confirm-fresh-26-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-26-hi-m0-source-chronological", + "confirm-fresh-01-hi-m0-source-grouped_lossless", + "confirm-fresh-01-hi-m0-source-chronological", + "confirm-fresh-01-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-09-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-09-hi-m0-source-chronological", + "confirm-fresh-09-hi-m0-source-grouped_lossless", + "confirm-fresh-13-hi-m0-source-grouped_lossless", + "confirm-fresh-13-hi-m0-source-chronological", + "confirm-fresh-13-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-31-hi-m0-source-grouped_lossless", + "confirm-fresh-31-hi-m0-source-chronological", + "confirm-fresh-31-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-25-hi-m1-source-grouped_lossless", + "confirm-fresh-25-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-25-hi-m1-source-chronological", + "confirm-fresh-32-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-32-hi-m1-source-chronological", + "confirm-fresh-32-hi-m1-source-grouped_lossless", + "confirm-fresh-15-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-15-hi-m1-source-grouped_lossless", + "confirm-fresh-15-hi-m1-source-chronological", + "confirm-fresh-35-hi-m1-source-chronological", + "confirm-fresh-35-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-35-hi-m1-source-grouped_lossless", + "confirm-fresh-20-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-20-hi-m1-source-chronological", + "confirm-fresh-20-hi-m1-source-grouped_lossless", + "confirm-fresh-37-hi-m0-source-grouped_lossless", + "confirm-fresh-37-hi-m0-source-chronological", + "confirm-fresh-37-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-22-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-22-hi-m0-source-grouped_lossless", + "confirm-fresh-22-hi-m0-source-chronological", + "confirm-fresh-02-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-02-hi-m1-source-chronological", + "confirm-fresh-02-hi-m1-source-grouped_lossless", + "confirm-fresh-19-hi-m0-source-grouped_lossless", + "confirm-fresh-19-hi-m0-source-chronological", + "confirm-fresh-19-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-36-hi-m0-source-chronological", + "confirm-fresh-36-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-36-hi-m0-source-grouped_lossless", + "confirm-fresh-10-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-10-hi-m0-source-grouped_lossless", + "confirm-fresh-10-hi-m0-source-chronological", + "confirm-fresh-30-hi-m1-source-grouped_lossless", + "confirm-fresh-30-hi-m1-source-chronological", + "confirm-fresh-30-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-31-hi-m1-source-grouped_lossless", + "confirm-fresh-31-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-31-hi-m1-source-chronological", + "confirm-fresh-20-hi-m0-source-grouped_lossless", + "confirm-fresh-20-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-20-hi-m0-source-chronological", + "confirm-fresh-05-hi-m1-source-chronological", + "confirm-fresh-05-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-05-hi-m1-source-grouped_lossless", + "confirm-fresh-33-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-33-hi-m1-source-grouped_lossless", + "confirm-fresh-33-hi-m1-source-chronological", + "confirm-fresh-05-hi-m0-source-chronological", + "confirm-fresh-05-hi-m0-source-grouped_lossless", + "confirm-fresh-05-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-40-hi-m1-source-chronological", + "confirm-fresh-40-hi-m1-source-grouped_lossless", + "confirm-fresh-40-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-14-hi-m0-source-grouped_lossless", + "confirm-fresh-14-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-14-hi-m0-source-chronological", + "confirm-fresh-09-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-09-hi-m1-source-grouped_lossless", + "confirm-fresh-09-hi-m1-source-chronological", + "confirm-fresh-39-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-39-hi-m0-source-chronological", + "confirm-fresh-39-hi-m0-source-grouped_lossless", + "confirm-fresh-27-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-27-hi-m0-source-chronological", + "confirm-fresh-27-hi-m0-source-grouped_lossless", + "confirm-fresh-14-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-14-hi-m1-source-chronological", + "confirm-fresh-14-hi-m1-source-grouped_lossless", + "confirm-fresh-02-hi-m0-source-grouped_lossless", + "confirm-fresh-02-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-02-hi-m0-source-chronological", + "confirm-fresh-07-hi-m1-source-grouped_lossless", + "confirm-fresh-07-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-07-hi-m1-source-chronological", + "confirm-fresh-01-hi-m1-source-grouped_lossless", + "confirm-fresh-01-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-01-hi-m1-source-chronological", + "confirm-fresh-38-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-38-hi-m1-source-chronological", + "confirm-fresh-38-hi-m1-source-grouped_lossless", + "confirm-fresh-12-hi-m0-source-chronological", + "confirm-fresh-12-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-12-hi-m0-source-grouped_lossless", + "confirm-fresh-16-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-16-hi-m0-source-grouped_lossless", + "confirm-fresh-16-hi-m0-source-chronological", + "confirm-fresh-35-hi-m0-source-chronological", + "confirm-fresh-35-hi-m0-source-grouped_lossless", + "confirm-fresh-35-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-12-hi-m1-source-grouped_lossless", + "confirm-fresh-12-hi-m1-source-chronological", + "confirm-fresh-12-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-17-hi-m1-source-chronological", + "confirm-fresh-17-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-17-hi-m1-source-grouped_lossless", + "confirm-fresh-19-hi-m1-source-grouped_lossless", + "confirm-fresh-19-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-19-hi-m1-source-chronological", + "confirm-fresh-21-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-21-hi-m0-source-chronological", + "confirm-fresh-21-hi-m0-source-grouped_lossless", + "confirm-fresh-11-hi-m1-source-chronological", + "confirm-fresh-11-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-11-hi-m1-source-grouped_lossless", + "confirm-fresh-27-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-27-hi-m1-source-grouped_lossless", + "confirm-fresh-27-hi-m1-source-chronological", + "confirm-fresh-10-hi-m1-source-chronological", + "confirm-fresh-10-hi-m1-source-grouped_lossless", + "confirm-fresh-10-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-18-hi-m0-source-chronological", + "confirm-fresh-18-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-18-hi-m0-source-grouped_lossless", + "confirm-fresh-04-hi-m1-source-chronological", + "confirm-fresh-04-hi-m1-source-grouped_lossless", + "confirm-fresh-04-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-28-hi-m1-source-chronological", + "confirm-fresh-28-hi-m1-source-grouped_lossless", + "confirm-fresh-28-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-03-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-03-hi-m0-source-chronological", + "confirm-fresh-03-hi-m0-source-grouped_lossless", + "confirm-fresh-15-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-15-hi-m0-source-chronological", + "confirm-fresh-15-hi-m0-source-grouped_lossless", + "confirm-fresh-38-hi-m0-source-grouped_lossless", + "confirm-fresh-38-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-38-hi-m0-source-chronological", + "confirm-fresh-26-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-26-hi-m1-source-chronological", + "confirm-fresh-26-hi-m1-source-grouped_lossless", + "confirm-fresh-36-hi-m1-source-grouped_lossless", + "confirm-fresh-36-hi-m1-source-chronological", + "confirm-fresh-36-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-29-hi-m1-source-chronological", + "confirm-fresh-29-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-29-hi-m1-source-grouped_lossless", + "confirm-fresh-06-hi-m1-source-grouped_lossless", + "confirm-fresh-06-hi-m1-source-chronological", + "confirm-fresh-06-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-34-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-34-hi-m0-source-grouped_lossless", + "confirm-fresh-34-hi-m0-source-chronological", + "confirm-fresh-07-hi-m0-source-grouped_lossless", + "confirm-fresh-07-hi-m0-source-chronological", + "confirm-fresh-07-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-03-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-03-hi-m1-source-grouped_lossless", + "confirm-fresh-03-hi-m1-source-chronological", + "confirm-fresh-33-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-33-hi-m0-source-chronological", + "confirm-fresh-33-hi-m0-source-grouped_lossless", + "confirm-fresh-25-hi-m0-source-grouped_lossless", + "confirm-fresh-25-hi-m0-source-chronological", + "confirm-fresh-25-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-17-hi-m0-source-chronological", + "confirm-fresh-17-hi-m0-source-grouped_lossless", + "confirm-fresh-17-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-32-hi-m0-source-grouped_lossless", + "confirm-fresh-32-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-32-hi-m0-source-chronological", + "confirm-fresh-37-hi-m1-source-grouped_lossless", + "confirm-fresh-37-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-37-hi-m1-source-chronological", + "confirm-fresh-16-hi-m1-source-chronological", + "confirm-fresh-16-hi-m1-source-grouped_lossless", + "confirm-fresh-16-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-06-hi-m0-source-chronological", + "confirm-fresh-06-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-06-hi-m0-source-grouped_lossless", + "confirm-fresh-40-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-40-hi-m0-source-grouped_lossless", + "confirm-fresh-40-hi-m0-source-chronological", + "confirm-fresh-04-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-04-hi-m0-source-grouped_lossless", + "confirm-fresh-04-hi-m0-source-chronological", + "confirm-fresh-28-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-28-hi-m0-source-grouped_lossless", + "confirm-fresh-28-hi-m0-source-chronological", + "confirm-fresh-08-hi-m0-source-grouped_lossless", + "confirm-fresh-08-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-08-hi-m0-source-chronological", + "confirm-fresh-11-hi-m0-source-chronological", + "confirm-fresh-11-hi-m0-source-grouped_lossless", + "confirm-fresh-11-hi-m0-source-grouped_actor_prose_withheld" + ], + "reviewed_job_ids": [ + "confirm-fresh-22-hi-m1-source-chronological", + "confirm-fresh-22-hi-m1-source-grouped_lossless", + "confirm-fresh-22-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-24-hi-m1-source-grouped_lossless", + "confirm-fresh-24-hi-m1-source-chronological", + "confirm-fresh-24-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-29-hi-m0-source-chronological", + "confirm-fresh-29-hi-m0-source-grouped_lossless", + "confirm-fresh-29-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-34-hi-m1-source-chronological", + "confirm-fresh-34-hi-m1-source-grouped_lossless", + "confirm-fresh-34-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-30-hi-m0-source-chronological", + "confirm-fresh-30-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-30-hi-m0-source-grouped_lossless", + "confirm-fresh-23-hi-m1-source-chronological", + "confirm-fresh-23-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-23-hi-m1-source-grouped_lossless", + "confirm-fresh-23-hi-m0-source-chronological", + "confirm-fresh-23-hi-m0-source-grouped_lossless", + "confirm-fresh-23-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-18-hi-m1-source-grouped_lossless", + "confirm-fresh-18-hi-m1-source-chronological", + "confirm-fresh-18-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-24-hi-m0-source-chronological", + "confirm-fresh-24-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-24-hi-m0-source-grouped_lossless", + "confirm-fresh-39-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-39-hi-m1-source-grouped_lossless", + "confirm-fresh-39-hi-m1-source-chronological", + "confirm-fresh-08-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-08-hi-m1-source-chronological", + "confirm-fresh-08-hi-m1-source-grouped_lossless", + "confirm-fresh-21-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-21-hi-m1-source-grouped_lossless", + "confirm-fresh-21-hi-m1-source-chronological", + "confirm-fresh-13-hi-m1-source-grouped_lossless", + "confirm-fresh-13-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-13-hi-m1-source-chronological", + "confirm-fresh-26-hi-m0-source-grouped_lossless", + "confirm-fresh-26-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-26-hi-m0-source-chronological", + "confirm-fresh-01-hi-m0-source-grouped_lossless", + "confirm-fresh-01-hi-m0-source-chronological", + "confirm-fresh-01-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-09-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-09-hi-m0-source-chronological", + "confirm-fresh-09-hi-m0-source-grouped_lossless", + "confirm-fresh-13-hi-m0-source-grouped_lossless", + "confirm-fresh-13-hi-m0-source-chronological", + "confirm-fresh-13-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-31-hi-m0-source-grouped_lossless", + "confirm-fresh-31-hi-m0-source-chronological", + "confirm-fresh-31-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-25-hi-m1-source-grouped_lossless", + "confirm-fresh-25-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-25-hi-m1-source-chronological", + "confirm-fresh-32-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-32-hi-m1-source-chronological", + "confirm-fresh-32-hi-m1-source-grouped_lossless", + "confirm-fresh-15-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-15-hi-m1-source-grouped_lossless", + "confirm-fresh-15-hi-m1-source-chronological", + "confirm-fresh-35-hi-m1-source-chronological", + "confirm-fresh-35-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-35-hi-m1-source-grouped_lossless", + "confirm-fresh-20-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-20-hi-m1-source-chronological", + "confirm-fresh-20-hi-m1-source-grouped_lossless", + "confirm-fresh-37-hi-m0-source-grouped_lossless", + "confirm-fresh-37-hi-m0-source-chronological", + "confirm-fresh-37-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-22-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-22-hi-m0-source-grouped_lossless", + "confirm-fresh-22-hi-m0-source-chronological", + "confirm-fresh-02-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-02-hi-m1-source-chronological", + "confirm-fresh-02-hi-m1-source-grouped_lossless", + "confirm-fresh-19-hi-m0-source-grouped_lossless", + "confirm-fresh-19-hi-m0-source-chronological", + "confirm-fresh-19-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-36-hi-m0-source-chronological", + "confirm-fresh-36-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-36-hi-m0-source-grouped_lossless", + "confirm-fresh-10-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-10-hi-m0-source-grouped_lossless", + "confirm-fresh-10-hi-m0-source-chronological", + "confirm-fresh-30-hi-m1-source-grouped_lossless", + "confirm-fresh-30-hi-m1-source-chronological", + "confirm-fresh-30-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-31-hi-m1-source-grouped_lossless", + "confirm-fresh-31-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-31-hi-m1-source-chronological", + "confirm-fresh-20-hi-m0-source-grouped_lossless", + "confirm-fresh-20-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-20-hi-m0-source-chronological", + "confirm-fresh-05-hi-m1-source-chronological", + "confirm-fresh-05-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-05-hi-m1-source-grouped_lossless", + "confirm-fresh-33-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-33-hi-m1-source-grouped_lossless", + "confirm-fresh-33-hi-m1-source-chronological", + "confirm-fresh-05-hi-m0-source-chronological", + "confirm-fresh-05-hi-m0-source-grouped_lossless", + "confirm-fresh-05-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-40-hi-m1-source-chronological", + "confirm-fresh-40-hi-m1-source-grouped_lossless", + "confirm-fresh-40-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-14-hi-m0-source-grouped_lossless", + "confirm-fresh-14-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-14-hi-m0-source-chronological", + "confirm-fresh-09-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-09-hi-m1-source-grouped_lossless", + "confirm-fresh-09-hi-m1-source-chronological", + "confirm-fresh-39-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-39-hi-m0-source-chronological", + "confirm-fresh-39-hi-m0-source-grouped_lossless", + "confirm-fresh-27-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-27-hi-m0-source-chronological", + "confirm-fresh-27-hi-m0-source-grouped_lossless", + "confirm-fresh-14-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-14-hi-m1-source-chronological", + "confirm-fresh-14-hi-m1-source-grouped_lossless", + "confirm-fresh-02-hi-m0-source-grouped_lossless", + "confirm-fresh-02-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-02-hi-m0-source-chronological", + "confirm-fresh-07-hi-m1-source-grouped_lossless", + "confirm-fresh-07-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-07-hi-m1-source-chronological", + "confirm-fresh-01-hi-m1-source-grouped_lossless", + "confirm-fresh-01-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-01-hi-m1-source-chronological", + "confirm-fresh-38-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-38-hi-m1-source-chronological", + "confirm-fresh-38-hi-m1-source-grouped_lossless", + "confirm-fresh-12-hi-m0-source-chronological", + "confirm-fresh-12-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-12-hi-m0-source-grouped_lossless", + "confirm-fresh-16-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-16-hi-m0-source-grouped_lossless", + "confirm-fresh-16-hi-m0-source-chronological", + "confirm-fresh-35-hi-m0-source-chronological", + "confirm-fresh-35-hi-m0-source-grouped_lossless", + "confirm-fresh-35-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-12-hi-m1-source-grouped_lossless", + "confirm-fresh-12-hi-m1-source-chronological", + "confirm-fresh-12-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-17-hi-m1-source-chronological", + "confirm-fresh-17-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-17-hi-m1-source-grouped_lossless", + "confirm-fresh-19-hi-m1-source-grouped_lossless", + "confirm-fresh-19-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-19-hi-m1-source-chronological", + "confirm-fresh-21-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-21-hi-m0-source-chronological", + "confirm-fresh-21-hi-m0-source-grouped_lossless", + "confirm-fresh-11-hi-m1-source-chronological", + "confirm-fresh-11-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-11-hi-m1-source-grouped_lossless", + "confirm-fresh-27-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-27-hi-m1-source-grouped_lossless", + "confirm-fresh-27-hi-m1-source-chronological", + "confirm-fresh-10-hi-m1-source-chronological", + "confirm-fresh-10-hi-m1-source-grouped_lossless", + "confirm-fresh-10-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-18-hi-m0-source-chronological", + "confirm-fresh-18-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-18-hi-m0-source-grouped_lossless", + "confirm-fresh-04-hi-m1-source-chronological", + "confirm-fresh-04-hi-m1-source-grouped_lossless", + "confirm-fresh-04-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-28-hi-m1-source-chronological", + "confirm-fresh-28-hi-m1-source-grouped_lossless", + "confirm-fresh-28-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-03-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-03-hi-m0-source-chronological", + "confirm-fresh-03-hi-m0-source-grouped_lossless", + "confirm-fresh-15-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-15-hi-m0-source-chronological", + "confirm-fresh-15-hi-m0-source-grouped_lossless", + "confirm-fresh-38-hi-m0-source-grouped_lossless", + "confirm-fresh-38-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-38-hi-m0-source-chronological", + "confirm-fresh-26-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-26-hi-m1-source-chronological", + "confirm-fresh-26-hi-m1-source-grouped_lossless", + "confirm-fresh-36-hi-m1-source-grouped_lossless", + "confirm-fresh-36-hi-m1-source-chronological", + "confirm-fresh-36-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-29-hi-m1-source-chronological", + "confirm-fresh-29-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-29-hi-m1-source-grouped_lossless", + "confirm-fresh-06-hi-m1-source-grouped_lossless", + "confirm-fresh-06-hi-m1-source-chronological", + "confirm-fresh-06-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-34-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-34-hi-m0-source-grouped_lossless", + "confirm-fresh-34-hi-m0-source-chronological", + "confirm-fresh-07-hi-m0-source-grouped_lossless", + "confirm-fresh-07-hi-m0-source-chronological", + "confirm-fresh-07-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-03-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-03-hi-m1-source-grouped_lossless", + "confirm-fresh-03-hi-m1-source-chronological", + "confirm-fresh-33-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-33-hi-m0-source-chronological", + "confirm-fresh-33-hi-m0-source-grouped_lossless", + "confirm-fresh-25-hi-m0-source-grouped_lossless", + "confirm-fresh-25-hi-m0-source-chronological", + "confirm-fresh-25-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-17-hi-m0-source-chronological", + "confirm-fresh-17-hi-m0-source-grouped_lossless", + "confirm-fresh-17-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-32-hi-m0-source-grouped_lossless", + "confirm-fresh-32-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-32-hi-m0-source-chronological", + "confirm-fresh-37-hi-m1-source-grouped_lossless", + "confirm-fresh-37-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-37-hi-m1-source-chronological", + "confirm-fresh-16-hi-m1-source-chronological", + "confirm-fresh-16-hi-m1-source-grouped_lossless", + "confirm-fresh-16-hi-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-06-hi-m0-source-chronological", + "confirm-fresh-06-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-06-hi-m0-source-grouped_lossless", + "confirm-fresh-40-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-40-hi-m0-source-grouped_lossless", + "confirm-fresh-40-hi-m0-source-chronological", + "confirm-fresh-04-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-04-hi-m0-source-grouped_lossless", + "confirm-fresh-04-hi-m0-source-chronological", + "confirm-fresh-28-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-28-hi-m0-source-grouped_lossless", + "confirm-fresh-28-hi-m0-source-chronological", + "confirm-fresh-08-hi-m0-source-grouped_lossless", + "confirm-fresh-08-hi-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-08-hi-m0-source-chronological", + "confirm-fresh-11-hi-m0-source-chronological", + "confirm-fresh-11-hi-m0-source-grouped_lossless", + "confirm-fresh-11-hi-m0-source-grouped_actor_prose_withheld" + ], + "summary": { + "planned_jobs": 240, + "reviewed_jobs": 240, + "histories": 80, + "source_cases": 40, + "campaign_attempted": 720, + "hindi_not_attempted_after_stop": 0, + "factual_verdict_counts": { + "correct": 202, + "incorrect": 32, + "invalid_or_missing": 6 + }, + "raw_status_verdict_counts": { + "correct": 192, + "incorrect": 42, + "invalid_or_missing": 6 + }, + "projected_status_verdict_counts": { + "correct": 234, + "invalid_or_missing": 6 + }, + "terminal_counts": { + "answer": 234, + "incomplete": 6 + }, + "by_model_arm": { + "sarvam-105b/chronological": { + "n": 40, + "factual_correct": 33, + "invalid_or_missing": 2, + "raw_status_correct": 29, + "projected_status_correct": 38 + }, + "sarvam-105b/grouped_lossless": { + "n": 40, + "factual_correct": 33, + "invalid_or_missing": 2, + "raw_status_correct": 31, + "projected_status_correct": 38 + }, + "sarvam-105b/grouped_actor_prose_withheld": { + "n": 40, + "factual_correct": 36, + "invalid_or_missing": 2, + "raw_status_correct": 32, + "projected_status_correct": 38 + }, + "sarvam-105b-conversations/chronological": { + "n": 40, + "factual_correct": 31, + "invalid_or_missing": 0, + "raw_status_correct": 36, + "projected_status_correct": 40 + }, + "sarvam-105b-conversations/grouped_lossless": { + "n": 40, + "factual_correct": 35, + "invalid_or_missing": 0, + "raw_status_correct": 32, + "projected_status_correct": 40 + }, + "sarvam-105b-conversations/grouped_actor_prose_withheld": { + "n": 40, + "factual_correct": 34, + "invalid_or_missing": 0, + "raw_status_correct": 32, + "projected_status_correct": 40 + } + }, + "source_gold_ambiguities": 0, + "assistant_vs_offline_scorer_disagreements": 0, + "offline_audit_verified_complete": true, + "non_iso_date_outputs": 0, + "ideal_gold_vs_actual_execution_date_divergences": [ + "confirm-fresh-28-hi-m1" + ] + }, + "review_method": "Assistant directly read all 40 Hindi source questions and conditions, checked 80 history-specific final states and tool receipts, assessed all 240 strict raw finals including invalids, and compared date/clarification plus raw/projected status verdicts to the offline receipt replay. No native human review.", + "cases": [ + { + "history_id": "confirm-fresh-01-hi-m0", + "case_id": "fresh-01-hi", + "model": "sarvam-105b", + "family": "request_correction", + "language": "hi", + "parent_paired_file_sha256": "eac6f052059df3a637f08a2d343c0df32c37c85d9951d678392a881e166b5a5f", + "question": "VISIT-2101 के लिए मैंने सबसे हाल में कौन सी तारीख मांगी? केवल बताएं।", + "source_messages": [ + { + "id": "note-1-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-1-1", + "role": "customer", + "content": "VISIT-2101 के लिए पहले मैंने 2027-04-02 तारीख मांगी थी।" + }, + { + "id": "note-1-2", + "role": "customer", + "content": "अब उसकी जगह 2027-05-02 चाहिए; 2027-04-02 अब मेरी पसंद नहीं है।" + } + ], + "source_basis_reviewed": "The later correction replaces the older request for VISIT-2101; the latest requested date is 2027-05-02, independent of the confirmed booking date.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-02", + "final_scoped_date": "2027-03-02", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-01-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "52adbecd689bdf90796ea699c3febc8086d57243a008b8e4e02ae63c3efa67d5", + "result_file_sha256": "96797c104f7a78e3839f320aa377b1e928e67cc696b95a1e7c81bfcf267ef17b", + "receipt_file_sha256": "7577e6142ffb2214bc1ef7fd84600ceccd096dcf263f3c8ffa7db0fd9b6aca4c", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-02\",\"needs_clarification\":false}", + "raw_content_sha256": "8aaeb0f32dbe546730fa5f7e7419bc6071d73ddbc5f5b92f356e540d2f854cb2", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-02' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2101; the latest requested date is 2027-05-02, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-01-hi-m0-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "cfa7bfb7ed5b232ea737433041090968f8e0db26f624d03d7aa6eee8195978e4", + "result_file_sha256": "96797c104f7a78e3839f320aa377b1e928e67cc696b95a1e7c81bfcf267ef17b", + "receipt_file_sha256": "3995bb2064a4aa358c23bab18ca1710781db6c152092fd16affbb64776a89999", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-02\",\"needs_clarification\":false}", + "raw_content_sha256": "8aaeb0f32dbe546730fa5f7e7419bc6071d73ddbc5f5b92f356e540d2f854cb2", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-02' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2101; the latest requested date is 2027-05-02, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-01-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "de479ede0f07118164069b1173fd5a8f669967b452b53d287a8e290c6fb7f03b", + "result_file_sha256": "96797c104f7a78e3839f320aa377b1e928e67cc696b95a1e7c81bfcf267ef17b", + "receipt_file_sha256": "4cca9c2829875f7af02fd4d9e82611b83ce2175f8176342730eed1a72f585b43", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-02\",\"needs_clarification\":false}", + "raw_content_sha256": "8aaeb0f32dbe546730fa5f7e7419bc6071d73ddbc5f5b92f356e540d2f854cb2", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-02' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2101; the latest requested date is 2027-05-02, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-01-hi-m1", + "case_id": "fresh-01-hi", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "language": "hi", + "parent_paired_file_sha256": "6ee6530432f616acd02a52702154bddccba0dc60b2bbffc85684d4581093c421", + "question": "VISIT-2101 के लिए मैंने सबसे हाल में कौन सी तारीख मांगी? केवल बताएं।", + "source_messages": [ + { + "id": "note-1-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-1-1", + "role": "customer", + "content": "VISIT-2101 के लिए पहले मैंने 2027-04-02 तारीख मांगी थी।" + }, + { + "id": "note-1-2", + "role": "customer", + "content": "अब उसकी जगह 2027-05-02 चाहिए; 2027-04-02 अब मेरी पसंद नहीं है।" + } + ], + "source_basis_reviewed": "The later correction replaces the older request for VISIT-2101; the latest requested date is 2027-05-02, independent of the confirmed booking date.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-02", + "final_scoped_date": "2027-03-02", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-01-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "6a412b91065db26572a8b4d041bc7261289fd8a84d8858641c50c79d44e8de3e", + "result_file_sha256": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "receipt_file_sha256": "7436ab906d1eb1c1d65d38fbedca5bde3e6980099fb224a6a2968314af27efd0", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-02\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "54f87f93350a2f520e59fad7cd323aeaa385120f331fae30b8483a7800211347", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-02' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2101; the latest requested date is 2027-05-02, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-01-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "c06c1e42433114c8311e50f7d0c7995a70cb32ab2f7702ae8642e14ea1fdea52", + "result_file_sha256": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "receipt_file_sha256": "6dc97d5d2678fe9e99aaab86ce74c17e95bc1ef0dd10d9dbf8928b2a2a5ae79a", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-02\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "54f87f93350a2f520e59fad7cd323aeaa385120f331fae30b8483a7800211347", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-02' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2101; the latest requested date is 2027-05-02, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-01-hi-m1-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "37e24fca793afa63c5f2840730d624a556004ce1f6791c38cbcd3181d4ff4f0f", + "result_file_sha256": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "receipt_file_sha256": "b2f90accf55434d8042a8eca923af46603ca6de2158892fc8d8b3561158064e1", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-02\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "54f87f93350a2f520e59fad7cd323aeaa385120f331fae30b8483a7800211347", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-02' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2101; the latest requested date is 2027-05-02, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-02-hi-m0", + "case_id": "fresh-02-hi", + "model": "sarvam-105b", + "family": "request_correction", + "language": "hi", + "parent_paired_file_sha256": "da0665f4748130dd4c440d6a21c6857b34af6971883a600cfe0fc39f3bb6dc57", + "question": "VISIT-2102 के लिए मैंने सबसे हाल में कौन सी तारीख मांगी? केवल बताएं।", + "source_messages": [ + { + "id": "note-2-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। VISIT-2102 के लिए पहले मैंने 2027-04-03 तारीख मांगी थी। अब उसकी जगह 2027-05-03 चाहिए; 2027-04-03 अब मेरी पसंद नहीं है।" + } + ], + "source_basis_reviewed": "The later correction replaces the older request for VISIT-2102; the latest requested date is 2027-05-03, independent of the confirmed booking date.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-03", + "final_scoped_date": "2027-03-03", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-02-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "467965eaf9993a505c40431d1a0d049f224ce6984652901226bcb53f831b6469", + "result_file_sha256": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "receipt_file_sha256": "f0493b32f6c96a27df3eb930687569a15e6a816e1c8680fa01e33f1eab87adec", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-03\",\"needs_clarification\":false}", + "raw_content_sha256": "8a1e2997cc0301dc67f4599949b8916f49f477f5d893e801470b0c0fd38e7854", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-03' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2102; the latest requested date is 2027-05-03, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-02-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "54953375e95532fd08158110f0b7bce37880c0ba84a90d4a0cb308977b13da22", + "result_file_sha256": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "receipt_file_sha256": "29bd63121012854f63b41f0a94f81321b53e7220fc2c617ce980b5b8e4744c66", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-03\",\"needs_clarification\":false}", + "raw_content_sha256": "8a1e2997cc0301dc67f4599949b8916f49f477f5d893e801470b0c0fd38e7854", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-03' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2102; the latest requested date is 2027-05-03, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-02-hi-m0-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "4de4f7886d00623ecbf84c890751bdef9a46da914b6441b8e74ecdc28860183e", + "result_file_sha256": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "receipt_file_sha256": "266547005af811ec916ba007e5b654e9818f5d0c1c2aa05d9de30a5b06f86d8e", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-03\",\"needs_clarification\":false}", + "raw_content_sha256": "8a1e2997cc0301dc67f4599949b8916f49f477f5d893e801470b0c0fd38e7854", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-03' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2102; the latest requested date is 2027-05-03, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-02-hi-m1", + "case_id": "fresh-02-hi", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "language": "hi", + "parent_paired_file_sha256": "20a016b5414a0e0c96eb11686911ef831c5f149c8a753938c0ba6fd0f873da90", + "question": "VISIT-2102 के लिए मैंने सबसे हाल में कौन सी तारीख मांगी? केवल बताएं।", + "source_messages": [ + { + "id": "note-2-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। VISIT-2102 के लिए पहले मैंने 2027-04-03 तारीख मांगी थी। अब उसकी जगह 2027-05-03 चाहिए; 2027-04-03 अब मेरी पसंद नहीं है।" + } + ], + "source_basis_reviewed": "The later correction replaces the older request for VISIT-2102; the latest requested date is 2027-05-03, independent of the confirmed booking date.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-03", + "final_scoped_date": "2027-03-03", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-02-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "09d257324f8a86853320124cb7ba307481453eecddc1b3c98965c4423819e707", + "result_file_sha256": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "receipt_file_sha256": "52ebdc021c4a43e99b670f99653d03471095f70f6b07c8ffdf97791e66a00f06", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-03\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "302302624fee023a6aedb4cd6f12fe82dc40383abb18b454d5e9be83a0d2c374", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-03' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2102; the latest requested date is 2027-05-03, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-02-hi-m1-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "8d23a030959b8064c77bd44f7c51f3fe7649c2c181da43a4a3aa73e00853f33a", + "result_file_sha256": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "receipt_file_sha256": "0e5e9b24e661e094d17f62eaba7550d64d310fd269d1ab392a350656319ff151", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-03\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "302302624fee023a6aedb4cd6f12fe82dc40383abb18b454d5e9be83a0d2c374", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-03' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2102; the latest requested date is 2027-05-03, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-02-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "a7c599ab7ea4a14765abc7e59b650e17c7a9b280cf2d392122333d25fdeecf3f", + "result_file_sha256": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "receipt_file_sha256": "5753e8c6b77fc1a7960559f04b23daab9a7a5185e1a2b875ad2f100db0acc11a", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-03\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "302302624fee023a6aedb4cd6f12fe82dc40383abb18b454d5e9be83a0d2c374", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-03' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2102; the latest requested date is 2027-05-03, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-03-hi-m0", + "case_id": "fresh-03-hi", + "model": "sarvam-105b", + "family": "request_correction", + "language": "hi", + "parent_paired_file_sha256": "adecc6c020cd9c8befac686e0c0f54a503282517e2f1f02ad8df109b24583e99", + "question": "VISIT-2103 के लिए मैंने सबसे हाल में कौन सी तारीख मांगी? केवल बताएं।", + "source_messages": [ + { + "id": "note-3-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-3-1", + "role": "customer", + "content": "VISIT-2103 के लिए पहले मैंने 2027-04-04 तारीख मांगी थी।" + }, + { + "id": "note-3-2", + "role": "customer", + "content": "अब उसकी जगह 2027-05-04 चाहिए; 2027-04-04 अब मेरी पसंद नहीं है।" + } + ], + "source_basis_reviewed": "The later correction replaces the older request for VISIT-2103; the latest requested date is 2027-05-04, independent of the confirmed booking date.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-04", + "final_scoped_date": "2027-03-04", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-03-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "74362500887b6bd335173ece043e5b1d41ac821f040493061ef6b18ee17dd8fd", + "result_file_sha256": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "receipt_file_sha256": "1da5b42094a33b4344c89d7dbadc569168343a4b3c47f121d036b48581478cae", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-04\",\"needs_clarification\":false}", + "raw_content_sha256": "212f85c3b1a2653eaa716d130aac4c6887ca2e4ae77cd416c3a0ee75931a3138", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-04' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2103; the latest requested date is 2027-05-04, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-03-hi-m0-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "ffc6a5487c25a0e0b9ea6c440975e2a5db3095cde51fa89ddf9d7c00009be00b", + "result_file_sha256": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "receipt_file_sha256": "30237a912ca81319524b3997a4677cb243b274873e4bebf8d66d9af37187d330", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-04\",\"needs_clarification\":false}", + "raw_content_sha256": "212f85c3b1a2653eaa716d130aac4c6887ca2e4ae77cd416c3a0ee75931a3138", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-04' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2103; the latest requested date is 2027-05-04, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-03-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "44524c708a261fbc46d0918deb6b4e6ef5d653a2b8c38cc6dd0821505106c045", + "result_file_sha256": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "receipt_file_sha256": "3642390c124473554bc127846bcf0d8afa5ae494394d0dee52e9b321fd91ede2", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-04\",\"needs_clarification\":false}", + "raw_content_sha256": "212f85c3b1a2653eaa716d130aac4c6887ca2e4ae77cd416c3a0ee75931a3138", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-04' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2103; the latest requested date is 2027-05-04, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-03-hi-m1", + "case_id": "fresh-03-hi", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "language": "hi", + "parent_paired_file_sha256": "5de1e1c7108dbeffb573fc5e16b5cdcd908123e374dd06af62e1e315c7c8b4b1", + "question": "VISIT-2103 के लिए मैंने सबसे हाल में कौन सी तारीख मांगी? केवल बताएं।", + "source_messages": [ + { + "id": "note-3-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-3-1", + "role": "customer", + "content": "VISIT-2103 के लिए पहले मैंने 2027-04-04 तारीख मांगी थी।" + }, + { + "id": "note-3-2", + "role": "customer", + "content": "अब उसकी जगह 2027-05-04 चाहिए; 2027-04-04 अब मेरी पसंद नहीं है।" + } + ], + "source_basis_reviewed": "The later correction replaces the older request for VISIT-2103; the latest requested date is 2027-05-04, independent of the confirmed booking date.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-04", + "final_scoped_date": "2027-03-04", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-03-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "42710b02e807680cf47e2fa6f77cf94adae3211a7a72998c403f48ab1b570a92", + "result_file_sha256": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "receipt_file_sha256": "1fa9f9f65931ac5cd60b0e502b97afddbd78993b5128a987232f9964707659d1", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-04\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "42098a7a7a4ef0ae49a0da19af03407227f6c6dda8858f3a01e7bd39c56ff795", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-04' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2103; the latest requested date is 2027-05-04, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-03-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "7d1bc407f841019e27834a4c291daf5c2caef2d5d296a793d25f7ebdcef64856", + "result_file_sha256": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "receipt_file_sha256": "9033191405fb4151a4efe5eda4350466c20e13aede338fe5c388f92fc6af7a85", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-04\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "42098a7a7a4ef0ae49a0da19af03407227f6c6dda8858f3a01e7bd39c56ff795", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-04' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2103; the latest requested date is 2027-05-04, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-03-hi-m1-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "c3dcaef160a7c4b3a300caa07587ae8f0caff3b54eca424e1b3ff4e6f5004880", + "result_file_sha256": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "receipt_file_sha256": "49e818d043476d85919978636cdfa78a531e46ef50c1240c2ef8ff0a75d95c3a", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-04\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "42098a7a7a4ef0ae49a0da19af03407227f6c6dda8858f3a01e7bd39c56ff795", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-04' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2103; the latest requested date is 2027-05-04, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-04-hi-m0", + "case_id": "fresh-04-hi", + "model": "sarvam-105b", + "family": "request_correction", + "language": "hi", + "parent_paired_file_sha256": "cccd09f85c8123a48386d127c8470e79e5911892a2274b067998152bd6e15aad", + "question": "VISIT-2104 के लिए मैंने सबसे हाल में कौन सी तारीख मांगी? केवल बताएं।", + "source_messages": [ + { + "id": "note-4-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। VISIT-2104 के लिए पहले मैंने 2027-04-05 तारीख मांगी थी। अब उसकी जगह 2027-05-05 चाहिए; 2027-04-05 अब मेरी पसंद नहीं है।" + } + ], + "source_basis_reviewed": "The later correction replaces the older request for VISIT-2104; the latest requested date is 2027-05-05, independent of the confirmed booking date.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-95fcc2a34ab00927", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2104" + }, + "response": { + "date": "2027-03-05", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-05", + "final_scoped_date": "2027-03-05", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-04-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "9e1094a4ce950b58b1dcfd8ceb046e76e63d1059fae364f03cf5f6d438eb4992", + "result_file_sha256": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "receipt_file_sha256": "73f06f986d17790fdd7ee0961e430a961a95c5f381a8b073e290dbd6a1d5b517", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-05\",\"needs_clarification\":false}", + "raw_content_sha256": "c45d72a60c16dbd0e39fe78d180715170520e37c139792fb956a8819a538950b", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-05' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2104; the latest requested date is 2027-05-05, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-04-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "b00af6e58884496942bbf38768825b08d4c15395f329c3b1ddc241fc5b70e1b9", + "result_file_sha256": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "receipt_file_sha256": "55dbbd9b00d4fad2a0880c55423f5480a98f2720dfdfd8c9792cb813d8871054", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-05\",\"needs_clarification\":false}", + "raw_content_sha256": "c45d72a60c16dbd0e39fe78d180715170520e37c139792fb956a8819a538950b", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-05' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2104; the latest requested date is 2027-05-05, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-04-hi-m0-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "673ab390af6c8d01c99bb7f05da7b84618ef8a0caeebe38bded3b05fcccc0f7a", + "result_file_sha256": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "receipt_file_sha256": "d146bd2bec0acbce7c8af7ceb485e6ce0839244e30ee67f29e30eea8af762ca8", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-05\",\"needs_clarification\":false}", + "raw_content_sha256": "c45d72a60c16dbd0e39fe78d180715170520e37c139792fb956a8819a538950b", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-05' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2104; the latest requested date is 2027-05-05, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-04-hi-m1", + "case_id": "fresh-04-hi", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "language": "hi", + "parent_paired_file_sha256": "4eed3838bcdbcd96ff121b5bfa2ec0a54a00ab483d2b5d52dc63c4ec58f34949", + "question": "VISIT-2104 के लिए मैंने सबसे हाल में कौन सी तारीख मांगी? केवल बताएं।", + "source_messages": [ + { + "id": "note-4-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। VISIT-2104 के लिए पहले मैंने 2027-04-05 तारीख मांगी थी। अब उसकी जगह 2027-05-05 चाहिए; 2027-04-05 अब मेरी पसंद नहीं है।" + } + ], + "source_basis_reviewed": "The later correction replaces the older request for VISIT-2104; the latest requested date is 2027-05-05, independent of the confirmed booking date.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-05", + "final_scoped_date": "2027-03-05", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-04-hi-m1-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "b43007060e339e4a97066dabd134d26f0f3efaf5554c75006d1283e96d7e8d0a", + "result_file_sha256": "7d9cb65e3028114bee20b6a3080462cea699456e2e865d0573765fe2cc56e2f9", + "receipt_file_sha256": "687e0e5482a9cd29a9d4c55a7d8f4cdea205dd73cad3e2c7805afae89974b1b7", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-05\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "79fd2c2d8207e140cf05450e2fe0ef7ade7b3dfe58de03fabad72e1508acb565", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-05' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2104; the latest requested date is 2027-05-05, independent of the confirmed booking date.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'already_applied' does not match expected 'not_requested': read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-04-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "3503779628f089903a3eafe40e4431842b560aab5a693e5601aa642be0bcad44", + "result_file_sha256": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "receipt_file_sha256": "be740846558270adbc7cefda850ba430ea16f0de76644215962c6e2776ae7abb", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-05\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "be956b2af283f28155cb87228f3c2833e326c59d589a68fb50d0a32eeaea2bd8", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-05' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2104; the latest requested date is 2027-05-05, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-04-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "80ec865d5543477fc1fec28c4ffc3b36ecd705a82eb7794427b21567035aad22", + "result_file_sha256": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "receipt_file_sha256": "04caee20ac1fd1ea1dfcb5de8d4b8c250634601a3fb708590592e42327697928", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-05\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "be956b2af283f28155cb87228f3c2833e326c59d589a68fb50d0a32eeaea2bd8", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-05' and clarification False match the scoped source and execution: The later correction replaces the older request for VISIT-2104; the latest requested date is 2027-05-05, independent of the confirmed booking date.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-05-hi-m0", + "case_id": "fresh-05-hi", + "model": "sarvam-105b", + "family": "missing_replacement", + "language": "hi", + "parent_paired_file_sha256": "fa79db090dd7954b9ff93edfdef8098b879f11f9204478d5343258569ee603f8", + "question": "VISIT-2105 के लिए अभी मुझे कौन सी नई तारीख चाहिए? केवल बताएं; बुकिंग न बदलें।", + "source_messages": [ + { + "id": "note-5-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-5-1", + "role": "customer", + "content": "VISIT-2105 के लिए अब मुझे 2027-04-06 नहीं चाहिए।" + }, + { + "id": "note-5-2", + "role": "customer", + "content": "मैंने अभी कोई नई तारीख नहीं चुनी है। मेरी दूसरी बुकिंग VISIT-3105 अलग है।" + } + ], + "source_basis_reviewed": "The customer withdraws the prior date for VISIT-2105 and expressly chooses no replacement; the other booking cannot answer this question.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-06", + "final_scoped_date": "2027-03-06", + "expected_from_source_and_actual_execution": { + "answer": null, + "needs_clarification": true + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-05-hi-m0-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "212af727f816fa7dd6b0343484d524f2a558188be4aefef19e84c064fbb19057", + "result_file_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "receipt_file_sha256": "a6b9b29ba174c89dea534b4565f388872502b15b9d50a362c4166ef403e25bbc", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "raw_content_sha256": "837d7c4d0820dbbaaedba89ceca07d3f62ea5913ac395c7558beddcbae373aa6", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Leaves the date unset but fails to ask for the missing information: The customer withdraws the prior date for VISIT-2105 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-05-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "42d156d9d2d07bcb91faebd2621a1d8bebcd88ff231564391606ce9d9dc1ba6d", + "result_file_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "receipt_file_sha256": "9668f4e92080e6e8f4f63c1e0147b919972005e65be1fadbee088f5e6b3cddc2", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "raw_content_sha256": "837d7c4d0820dbbaaedba89ceca07d3f62ea5913ac395c7558beddcbae373aa6", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Leaves the date unset but fails to ask for the missing information: The customer withdraws the prior date for VISIT-2105 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-05-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "d1bda692dcc77f2ed76e4772203795338aa1002192c5d405ffd2d02801734ac6", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "receipt_file_sha256": "a9b0f79b01ff9f0b97c10e9c5b5c7bbe2031e031524fd443328a0e1cdab04651", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: The customer withdraws the prior date for VISIT-2105 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-05-hi-m1", + "case_id": "fresh-05-hi", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "language": "hi", + "parent_paired_file_sha256": "bdffcded718c73d664aaad5ca0d205c8f04fcaf343dd69d3d25a3b0cb0baa9d6", + "question": "VISIT-2105 के लिए अभी मुझे कौन सी नई तारीख चाहिए? केवल बताएं; बुकिंग न बदलें।", + "source_messages": [ + { + "id": "note-5-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-5-1", + "role": "customer", + "content": "VISIT-2105 के लिए अब मुझे 2027-04-06 नहीं चाहिए।" + }, + { + "id": "note-5-2", + "role": "customer", + "content": "मैंने अभी कोई नई तारीख नहीं चुनी है। मेरी दूसरी बुकिंग VISIT-3105 अलग है।" + } + ], + "source_basis_reviewed": "The customer withdraws the prior date for VISIT-2105 and expressly chooses no replacement; the other booking cannot answer this question.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-aab8bfabb8e9c671", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2105" + }, + "response": { + "date": "2027-03-06", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-06", + "final_scoped_date": "2027-03-06", + "expected_from_source_and_actual_execution": { + "answer": null, + "needs_clarification": true + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-05-hi-m1-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "3a74f612894744d2287bd265aaeb6fb924bbb8bb406d3caf5a9f02a1f7bf4c1e", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "caf043a83fa46f9f72e3c5e83f1c9cca16b2ca6c4cf13559497bd8322a56322c", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: The customer withdraws the prior date for VISIT-2105 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-05-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "d3a1d015d0b60709ca9683b7475a496357538e4ff566dfe757249370b4aa10bc", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "4df4b0b8fb8641ead205cfb201b4282399978142fc08a002091cdebd6efe64ee", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: The customer withdraws the prior date for VISIT-2105 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-05-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "39afe2afe26d84dae82580d0c09d5705ded098d30ace862c783825e74ce16bcf", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "50b0b53905d857b1600ae4d942fab491a60e351dff43fce52eae61db9f143c71", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: The customer withdraws the prior date for VISIT-2105 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-06-hi-m0", + "case_id": "fresh-06-hi", + "model": "sarvam-105b", + "family": "missing_replacement", + "language": "hi", + "parent_paired_file_sha256": "12e393d97674a12400914aa0c21cc1a10e4d684c5e83cc8c217c290564ed9524", + "question": "VISIT-2106 के लिए अभी मुझे कौन सी नई तारीख चाहिए? केवल बताएं; बुकिंग न बदलें।", + "source_messages": [ + { + "id": "note-6-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। VISIT-2106 के लिए अब मुझे 2027-04-07 नहीं चाहिए। मैंने अभी कोई नई तारीख नहीं चुनी है। मेरी दूसरी बुकिंग VISIT-3106 अलग है।" + } + ], + "source_basis_reviewed": "The customer withdraws the prior date for VISIT-2106 and expressly chooses no replacement; the other booking cannot answer this question.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-07", + "final_scoped_date": "2027-03-07", + "expected_from_source_and_actual_execution": { + "answer": null, + "needs_clarification": true + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-06-hi-m0-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "ebe5a742b0e3b57fada9a43c8080b74b1edf100b858b7db3302affe0581656ac", + "result_file_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "receipt_file_sha256": "96f1d2319179230de78c9abba4a06889e0c084ad2900a1c942794f43a61c5732", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "raw_content_sha256": "837d7c4d0820dbbaaedba89ceca07d3f62ea5913ac395c7558beddcbae373aa6", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Leaves the date unset but fails to ask for the missing information: The customer withdraws the prior date for VISIT-2106 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-06-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "da4adacaeb1d5e17ac931cff216a5e3563ecec267c72b5fd6e04142378e16176", + "result_file_sha256": "c00e01707b0b846b41fdae45555f7459894195d4eba6faf002b9d99d13718add", + "receipt_file_sha256": "5cefe74c3e6bdd14d9d63585f5473f2bff7b4b0ab7ce2d2502cde3a2dab3fd45", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": false\n}", + "raw_content_sha256": "c9c414936f5b817a07b81d62a8d1262cb50f97b156050ef0ce0ee2da10dd346d", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Leaves the date unset but fails to ask for the missing information: The customer withdraws the prior date for VISIT-2106 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-06-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "30aec2de91bec21d4fae73d53014fe1d5a46311de6e42e5afa53d4b255e5389d", + "result_file_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "receipt_file_sha256": "d33ea5b50f80e42616b5a89d963600b33fc66cb0d75cbc05c436fef21cce21f5", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "raw_content_sha256": "837d7c4d0820dbbaaedba89ceca07d3f62ea5913ac395c7558beddcbae373aa6", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Leaves the date unset but fails to ask for the missing information: The customer withdraws the prior date for VISIT-2106 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-06-hi-m1", + "case_id": "fresh-06-hi", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "language": "hi", + "parent_paired_file_sha256": "d19d546bbe93b9ee5e5872bf3cc311ffc47418c0df26151b063a3a25a3ce4395", + "question": "VISIT-2106 के लिए अभी मुझे कौन सी नई तारीख चाहिए? केवल बताएं; बुकिंग न बदलें।", + "source_messages": [ + { + "id": "note-6-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। VISIT-2106 के लिए अब मुझे 2027-04-07 नहीं चाहिए। मैंने अभी कोई नई तारीख नहीं चुनी है। मेरी दूसरी बुकिंग VISIT-3106 अलग है।" + } + ], + "source_basis_reviewed": "The customer withdraws the prior date for VISIT-2106 and expressly chooses no replacement; the other booking cannot answer this question.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9c00f6ceb344b38f", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2106" + }, + "response": { + "date": "2027-03-07", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-07", + "final_scoped_date": "2027-03-07", + "expected_from_source_and_actual_execution": { + "answer": null, + "needs_clarification": true + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-06-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "6888f6fed114938baae90786d84726ac2b4ab6fc332fe7e1d0b92fc588527050", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "d774da74ddaf0cc0e76fd8c4331a9446e9078f06b62ecccbd1a2fdd74692c750", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: The customer withdraws the prior date for VISIT-2106 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-06-hi-m1-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "04451bba81b0f5e11b0f57ed13620619e77dfe1cca379908d24b4c9fed892687", + "result_file_sha256": "f6cf1024269159938bbbbb36d7bb57ccf650f6100d402ac17b0a1158c5ad30fe", + "receipt_file_sha256": "d0ad203bd81b5ac3001124bd25be07d9ba55678a70d4a144a90dbddc98dc502c", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-07\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "b0863fc7d2340c08823b4bafef5c056b311f16790b88dac86e8208fb91fba89d", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-07", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-07", + "needs_clarification": false + }, + "answer_source_role": "confirmed_date_substituted_for_missing_replacement", + "factual_verdict": "incorrect", + "factual_explanation": "Reports definite date '2027-03-07' when the source leaves the requested date or condition unresolved: The customer withdraws the prior date for VISIT-2106 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-06-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "ca83c7c309e17f995e3c75824c3edc44c0dda1d087b758f6bace1032d158374f", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "acb635fe546a679c1047acbcc927dce219cd3717917d56d227fd79ac7e9d50ed", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: The customer withdraws the prior date for VISIT-2106 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-07-hi-m0", + "case_id": "fresh-07-hi", + "model": "sarvam-105b", + "family": "missing_replacement", + "language": "hi", + "parent_paired_file_sha256": "c8fec501525b23dbc527c90169c3df502f7852b206e563974728dd851ab180e7", + "question": "VISIT-2107 के लिए अभी मुझे कौन सी नई तारीख चाहिए? केवल बताएं; बुकिंग न बदलें।", + "source_messages": [ + { + "id": "note-7-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-7-1", + "role": "customer", + "content": "VISIT-2107 के लिए अब मुझे 2027-04-08 नहीं चाहिए।" + }, + { + "id": "note-7-2", + "role": "customer", + "content": "मैंने अभी कोई नई तारीख नहीं चुनी है। मेरी दूसरी बुकिंग VISIT-3107 अलग है।" + } + ], + "source_basis_reviewed": "The customer withdraws the prior date for VISIT-2107 and expressly chooses no replacement; the other booking cannot answer this question.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-08", + "final_scoped_date": "2027-03-08", + "expected_from_source_and_actual_execution": { + "answer": null, + "needs_clarification": true + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-07-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "0e6c8e0e5f2fb163d9bf626041698bede00cd9712d45ebb687ee7450c1fc35a6", + "result_file_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "receipt_file_sha256": "e5eddc9debf9635c66e6fe03150a12a7d0e8ac0b79d45a53a23e910729185708", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "raw_content_sha256": "837d7c4d0820dbbaaedba89ceca07d3f62ea5913ac395c7558beddcbae373aa6", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Leaves the date unset but fails to ask for the missing information: The customer withdraws the prior date for VISIT-2107 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-07-hi-m0-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "554a7ea13fb491daa3728e2da7b2a262485dd1ed79d613cfd4e2052cd5ae8651", + "result_file_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "receipt_file_sha256": "e7d86b6a6899c47ebd0383d92dde455a2fc1b356bf9fd68ebdb7596bc5900a8a", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "raw_content_sha256": "837d7c4d0820dbbaaedba89ceca07d3f62ea5913ac395c7558beddcbae373aa6", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Leaves the date unset but fails to ask for the missing information: The customer withdraws the prior date for VISIT-2107 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-07-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "f0a9058c12a5f9c8085410a7c031af9ae25749a6f096fd1346a7a349fe39bfac", + "result_file_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "receipt_file_sha256": "c33be7be2f83a46b918973925abb991c7bbbe4f1004f97afe52bf2199b73231a", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "raw_content_sha256": "837d7c4d0820dbbaaedba89ceca07d3f62ea5913ac395c7558beddcbae373aa6", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Leaves the date unset but fails to ask for the missing information: The customer withdraws the prior date for VISIT-2107 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-07-hi-m1", + "case_id": "fresh-07-hi", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "language": "hi", + "parent_paired_file_sha256": "0b5f6387be064fda44cce858c07a7549b0f534e4fb753a7719a34e32ef2bbed8", + "question": "VISIT-2107 के लिए अभी मुझे कौन सी नई तारीख चाहिए? केवल बताएं; बुकिंग न बदलें।", + "source_messages": [ + { + "id": "note-7-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-7-1", + "role": "customer", + "content": "VISIT-2107 के लिए अब मुझे 2027-04-08 नहीं चाहिए।" + }, + { + "id": "note-7-2", + "role": "customer", + "content": "मैंने अभी कोई नई तारीख नहीं चुनी है। मेरी दूसरी बुकिंग VISIT-3107 अलग है।" + } + ], + "source_basis_reviewed": "The customer withdraws the prior date for VISIT-2107 and expressly chooses no replacement; the other booking cannot answer this question.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-b567b7f66f49f94f", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2107" + }, + "response": { + "date": "2027-03-08", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-08", + "final_scoped_date": "2027-03-08", + "expected_from_source_and_actual_execution": { + "answer": null, + "needs_clarification": true + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-07-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "a26096cda89168856adf5b85052052a07523159e0dcf317259adb82093039355", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "653a6bf78a37ffe29bdf35a84397632540d3a1456887babe7c099173648ad206", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: The customer withdraws the prior date for VISIT-2107 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-07-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "dd4c48a1367ce76e72504012454ca2fbbf725105e7adbaa321de1cb7caeb74d1", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "cf1090087b5df3426f0e825a9635c14e9fe3e385012a506ec00e9e32b77be268", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: The customer withdraws the prior date for VISIT-2107 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-07-hi-m1-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "e4464dcfd38c65b881ec3af3150bc9092b6b42e4ccf7e77697f7e44b22f11d24", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "164111df62ce2af807fe5b5d93c68a211c9b8264f5336f673b121d1f98f92f4e", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: The customer withdraws the prior date for VISIT-2107 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-08-hi-m0", + "case_id": "fresh-08-hi", + "model": "sarvam-105b", + "family": "missing_replacement", + "language": "hi", + "parent_paired_file_sha256": "e1032a1e765820fd42f68c926f9c87a9252e91c232eccafab4b4c76029624dba", + "question": "VISIT-2108 के लिए अभी मुझे कौन सी नई तारीख चाहिए? केवल बताएं; बुकिंग न बदलें।", + "source_messages": [ + { + "id": "note-8-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। VISIT-2108 के लिए अब मुझे 2027-04-09 नहीं चाहिए। मैंने अभी कोई नई तारीख नहीं चुनी है। मेरी दूसरी बुकिंग VISIT-3108 अलग है।" + } + ], + "source_basis_reviewed": "The customer withdraws the prior date for VISIT-2108 and expressly chooses no replacement; the other booking cannot answer this question.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-09", + "final_scoped_date": "2027-03-09", + "expected_from_source_and_actual_execution": { + "answer": null, + "needs_clarification": true + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-08-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "1a3800bb7925c1dc92cf1250a1cb6bea4476e073bcf76da762e9c255ea085653", + "result_file_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "receipt_file_sha256": "064612a4f538843e96a5bdb7d332cb67cfd5198c0a459f690144e9c7098166f9", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "raw_content_sha256": "837d7c4d0820dbbaaedba89ceca07d3f62ea5913ac395c7558beddcbae373aa6", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Leaves the date unset but fails to ask for the missing information: The customer withdraws the prior date for VISIT-2108 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-08-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "9ed495c03b3917739eff1c2bb0a24de99bb58fc3b4f216dc992b03b16bf012a2", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "receipt_file_sha256": "22f7ce8a8a85591eed1986d36e8330e65b2f2bf9d6d47793cc0178b78ff77325", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: The customer withdraws the prior date for VISIT-2108 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-08-hi-m0-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "0de3bcfd07fde5346cee55f3fd01a26536f642c2939001521e86ec3027fb78b2", + "result_file_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "receipt_file_sha256": "f77e5876b1374a6d747e44826331991498e221c3dc3d564cacca20b9553ca3e0", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "raw_content_sha256": "837d7c4d0820dbbaaedba89ceca07d3f62ea5913ac395c7558beddcbae373aa6", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Leaves the date unset but fails to ask for the missing information: The customer withdraws the prior date for VISIT-2108 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-08-hi-m1", + "case_id": "fresh-08-hi", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "language": "hi", + "parent_paired_file_sha256": "d8c326bce5600dae15a663d83dc9146fa3d2b3c6fcbe54c0cb0340a600b98cfd", + "question": "VISIT-2108 के लिए अभी मुझे कौन सी नई तारीख चाहिए? केवल बताएं; बुकिंग न बदलें।", + "source_messages": [ + { + "id": "note-8-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। VISIT-2108 के लिए अब मुझे 2027-04-09 नहीं चाहिए। मैंने अभी कोई नई तारीख नहीं चुनी है। मेरी दूसरी बुकिंग VISIT-3108 अलग है।" + } + ], + "source_basis_reviewed": "The customer withdraws the prior date for VISIT-2108 and expressly chooses no replacement; the other booking cannot answer this question.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-09", + "final_scoped_date": "2027-03-09", + "expected_from_source_and_actual_execution": { + "answer": null, + "needs_clarification": true + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-08-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "9c2eb8fc5d956f4c06baf260d9aeda09ba30167f16d800588b4ccff26aa3a985", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "8e3765466b14b6e5bcd0f3d6d12e09b65231a4593dc24459330d5b69938c9948", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: The customer withdraws the prior date for VISIT-2108 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-08-hi-m1-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "142be1b24f20ad269f3327dfa563fac494bd3681d6da505d07b0bfa611c2e274", + "result_file_sha256": "130de02ed3a97f38059e540989f910a87601778b6cf8c2cabce8fc39cebdbd14", + "receipt_file_sha256": "8ed35f27a95281c3750dd1caa4caf44d80d091b78a87f7cc184c19f5218ab79d", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-09\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "47bb6d3f2245326689ed6c39ad9314f9ad159fcd790e48b50ec0c2a06e2bd028", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-09", + "needs_clarification": false + }, + "answer_source_role": "confirmed_date_substituted_for_missing_replacement", + "factual_verdict": "incorrect", + "factual_explanation": "Reports definite date '2027-03-09' when the source leaves the requested date or condition unresolved: The customer withdraws the prior date for VISIT-2108 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-08-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "19ee020071fb1553e5ae191a6ce0a1db8403190dfe693398783abc86d83f0b7c", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "073788f9571c0c7af20f118cd0ea0e66464b05b8d40be7407693f1e050b3f81a", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: The customer withdraws the prior date for VISIT-2108 and expressly chooses no replacement; the other booking cannot answer this question.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-09-hi-m0", + "case_id": "fresh-09-hi", + "model": "sarvam-105b", + "family": "resolved_condition", + "language": "hi", + "parent_paired_file_sha256": "ac2db8746c86ee3f534a4ebef19db3e77a39b9f303363ae909ed42c30b40ab82", + "question": "बताई गई शर्त के अनुसार VISIT-2109 के लिए कौन सी तारीख चुनूं? बदलाव न करें।", + "source_messages": [ + { + "id": "note-9-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-9-1", + "role": "customer", + "content": "VISIT-2109 के लिए सुबह का समय मिले तो 2027-04-10 चुनें, नहीं तो 2027-05-10।" + }, + { + "id": "note-9-2", + "role": "customer", + "content": "सुबह का समय उपलब्ध है। मैं केवल अपनी पसंद पूछ रहा हूं, बुकिंग बदलने की अनुमति नहीं दे रहा हूं।" + } + ], + "source_basis_reviewed": "The source states morning availability is available; its if/otherwise preference therefore selects 2027-04-10. Read-only scope does not prevent answering.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-10", + "final_scoped_date": "2027-03-10", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-09-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "1d948673e5cc14fdf61b8ad1495eb44e8c3962bf9fcf716564833bee090d397a", + "result_file_sha256": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "receipt_file_sha256": "f5ff675dab79b32969727c8dbb465661ceb564b5af2acead18153434099d18c4", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-10\",\"needs_clarification\":false}", + "raw_content_sha256": "b5203780d91957fd9ffd4fed596dd501556d13bf3dc68e7bd2d239524382871b", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-10' and clarification False match the scoped source and execution: The source states morning availability is available; its if/otherwise preference therefore selects 2027-04-10. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-09-hi-m0-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "aa95ea2f50f2b805f4902604b5c5d21adf68bb56ca35220be296d6569ead2f33", + "result_file_sha256": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "receipt_file_sha256": "7ab4749e3c9efd8512e51191fe4a15c358c1653c0f91ca4ad22254896748a29d", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-10\",\"needs_clarification\":false}", + "raw_content_sha256": "b5203780d91957fd9ffd4fed596dd501556d13bf3dc68e7bd2d239524382871b", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-10' and clarification False match the scoped source and execution: The source states morning availability is available; its if/otherwise preference therefore selects 2027-04-10. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-09-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "574bd2f0d46b045e3ee63b3c0ba47fd799b7ac0f75199f1f8334ed57c4313c74", + "result_file_sha256": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "receipt_file_sha256": "f34d17191eded002ab290b0e31b175f9911a25381eed8202ac9e41879dbf488e", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-10\",\"needs_clarification\":false}", + "raw_content_sha256": "b5203780d91957fd9ffd4fed596dd501556d13bf3dc68e7bd2d239524382871b", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-10' and clarification False match the scoped source and execution: The source states morning availability is available; its if/otherwise preference therefore selects 2027-04-10. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-09-hi-m1", + "case_id": "fresh-09-hi", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "language": "hi", + "parent_paired_file_sha256": "817b5d8d345f1124fb35cc642ab25df8f29beee2129e8be78f950e8a74f4c31a", + "question": "बताई गई शर्त के अनुसार VISIT-2109 के लिए कौन सी तारीख चुनूं? बदलाव न करें।", + "source_messages": [ + { + "id": "note-9-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-9-1", + "role": "customer", + "content": "VISIT-2109 के लिए सुबह का समय मिले तो 2027-04-10 चुनें, नहीं तो 2027-05-10।" + }, + { + "id": "note-9-2", + "role": "customer", + "content": "सुबह का समय उपलब्ध है। मैं केवल अपनी पसंद पूछ रहा हूं, बुकिंग बदलने की अनुमति नहीं दे रहा हूं।" + } + ], + "source_basis_reviewed": "The source states morning availability is available; its if/otherwise preference therefore selects 2027-04-10. Read-only scope does not prevent answering.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-10", + "final_scoped_date": "2027-03-10", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-09-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "c6c4536be964a2a83115b9bf664d6351c8749e1e2e182eff4349e1dafed7cf14", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "381acc29f79fbec14803dedcae511bcc60bb6c154c54bce43dd94004db50b1bd", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Withholds determinate date 2027-04-10 despite the source/receipts: The source states morning availability is available; its if/otherwise preference therefore selects 2027-04-10. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-09-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "1c1b16ba5e408df6426eadfc4a19e12c7d9b2c53712ecbbef9924f7e68933e3a", + "result_file_sha256": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "receipt_file_sha256": "0a45e862c8bbccb1c84a778cbf9067bff9c0262d18b8a1843479fe4a899d201d", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-10\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "5f5a8175c7868d3fdb706311acadab639b8b5525cf4e1d717def813d9e5c10af", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-10' and clarification False match the scoped source and execution: The source states morning availability is available; its if/otherwise preference therefore selects 2027-04-10. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-09-hi-m1-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "124d32edc68b2343e81d55bacb75639fa74181a8e15cf608658a5b84ea4e8b73", + "result_file_sha256": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "receipt_file_sha256": "a7fe14344f84364cc3eb743eef53f28155be1fd15ff98ad33acdb85d79dd7207", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-10\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "5f5a8175c7868d3fdb706311acadab639b8b5525cf4e1d717def813d9e5c10af", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-10' and clarification False match the scoped source and execution: The source states morning availability is available; its if/otherwise preference therefore selects 2027-04-10. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-10-hi-m0", + "case_id": "fresh-10-hi", + "model": "sarvam-105b", + "family": "resolved_condition", + "language": "hi", + "parent_paired_file_sha256": "1c808ae411cbec25fa8ea551e75128ba596f354d83a585cd32a39fe551bdbbae", + "question": "बताई गई शर्त के अनुसार VISIT-2110 के लिए कौन सी तारीख चुनूं? बदलाव न करें।", + "source_messages": [ + { + "id": "note-10-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। VISIT-2110 के लिए सुबह का समय मिले तो 2027-04-11 चुनें, नहीं तो 2027-05-11। सुबह का समय उपलब्ध नहीं है। मैं केवल अपनी पसंद पूछ रहा हूं।" + } + ], + "source_basis_reviewed": "The source states morning availability is unavailable; its if/otherwise preference therefore selects 2027-05-11. Read-only scope does not prevent answering.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-11", + "final_scoped_date": "2027-03-11", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-10-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "8e26321534fb47abe3e5abe6c5cfa28d7aaf35755628ebbd4ca1425e1db8323f", + "result_file_sha256": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "receipt_file_sha256": "1c90d868d1ffbca617f3b9725b47b4bf67b22c6deaee6b19896aeb49232bfff7", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "raw_content_sha256": "2362fc24e9ad026848ab17e8b29cf8890528a030252a998e2bd4367263250458", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-11' and clarification False match the scoped source and execution: The source states morning availability is unavailable; its if/otherwise preference therefore selects 2027-05-11. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-10-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "db87f80bb0f1523b7134279dd58701b8b0bd1f167e605f05b4f8725fec85e42f", + "result_file_sha256": "6294b9020f2db793978af4e410257f125f9a2791382e29e36b9e2496f58ab189", + "receipt_file_sha256": "07152f15bda3d535a0aee4b0e1ecf2caf845324bc8d3675a6cb88327bcc7181b", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "raw_content_sha256": "25633064337832a0a3fa83f6d518dc35a40d70bcf92b460cafbb9f56330ac039", + "raw": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-11' and clarification False match the scoped source and execution: The source states morning availability is unavailable; its if/otherwise preference therefore selects 2027-05-11. Read-only scope does not prevent answering.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_performed' does not match expected 'not_requested': read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-10-hi-m0-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "2f1694c1a26c4d13194f6fac4230e102833f05187aa1dd5a94a5b96d6118119b", + "result_file_sha256": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "receipt_file_sha256": "a77ba3031133724203e0d0b0e61956a61a3eedad1a51d7b74f0fa5327f5bedf8", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "raw_content_sha256": "2362fc24e9ad026848ab17e8b29cf8890528a030252a998e2bd4367263250458", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-11' and clarification False match the scoped source and execution: The source states morning availability is unavailable; its if/otherwise preference therefore selects 2027-05-11. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-10-hi-m1", + "case_id": "fresh-10-hi", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "language": "hi", + "parent_paired_file_sha256": "9e50714179701725c034a38c9f63431de4684f991c381c1dcbdb746377a8529e", + "question": "बताई गई शर्त के अनुसार VISIT-2110 के लिए कौन सी तारीख चुनूं? बदलाव न करें।", + "source_messages": [ + { + "id": "note-10-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। VISIT-2110 के लिए सुबह का समय मिले तो 2027-04-11 चुनें, नहीं तो 2027-05-11। सुबह का समय उपलब्ध नहीं है। मैं केवल अपनी पसंद पूछ रहा हूं।" + } + ], + "source_basis_reviewed": "The source states morning availability is unavailable; its if/otherwise preference therefore selects 2027-05-11. Read-only scope does not prevent answering.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-b56c473c40761f91", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2110" + }, + "response": { + "date": "2027-03-11", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-11", + "final_scoped_date": "2027-03-11", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-10-hi-m1-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "63013d276e9bc4e02d91ad46e33da77c70e4ec54d7d486b4482cb44e467f590a", + "result_file_sha256": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "receipt_file_sha256": "7fcb2c0fdc961cf90257ea476874880f2a9fa338c2ceb116aa7c47e45843f5fa", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "2655156e40334761b11243498c952a7bfa50bffe056b93d1424ef09be4cb336c", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-11' and clarification False match the scoped source and execution: The source states morning availability is unavailable; its if/otherwise preference therefore selects 2027-05-11. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-10-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "ad6f247b54ebbed23b60176d0e4a72d61fc2992fa7a327484b3b4c92ca6af30e", + "result_file_sha256": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "receipt_file_sha256": "9907f0d5311f6928a901c67a54cbab4e48eb24400c0ed76b6ee256116c3573c6", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "2655156e40334761b11243498c952a7bfa50bffe056b93d1424ef09be4cb336c", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-11' and clarification False match the scoped source and execution: The source states morning availability is unavailable; its if/otherwise preference therefore selects 2027-05-11. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-10-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "cd40b7869dc559ab37a275f640659477a6953d3a5e9cbdc4e7004bfb41f7c55c", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "629f20813797cab89fee9492d2521faafe1dac4c060580ab1c1334f920451444", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Withholds determinate date 2027-05-11 despite the source/receipts: The source states morning availability is unavailable; its if/otherwise preference therefore selects 2027-05-11. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-11-hi-m0", + "case_id": "fresh-11-hi", + "model": "sarvam-105b", + "family": "resolved_condition", + "language": "hi", + "parent_paired_file_sha256": "40668e9509a71199ab01a188be47db72d61462d151b74e2ad4d3264b8284bae7", + "question": "बताई गई शर्त के अनुसार VISIT-2111 के लिए कौन सी तारीख चुनूं? बदलाव न करें।", + "source_messages": [ + { + "id": "note-11-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-11-1", + "role": "customer", + "content": "VISIT-2111 के लिए सुबह का समय मिले तो 2027-04-12 चुनें, नहीं तो 2027-05-12।" + }, + { + "id": "note-11-2", + "role": "customer", + "content": "सुबह का समय उपलब्ध है। मैं केवल अपनी पसंद पूछ रहा हूं, बुकिंग बदलने की अनुमति नहीं दे रहा हूं।" + } + ], + "source_basis_reviewed": "The source states morning availability is available; its if/otherwise preference therefore selects 2027-04-12. Read-only scope does not prevent answering.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9f930137b204a2c9", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2111" + }, + "response": { + "date": "2027-03-12", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-12", + "final_scoped_date": "2027-03-12", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-11-hi-m0-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "83e0cbecf29b068cb77a197221bc4619f511cb63bb27e029b8c9cee9f2793281", + "result_file_sha256": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "receipt_file_sha256": "e82b981f52bf91d9774efe3d26213e3b26021b4f9d008e53dec9433f6770441e", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-12\",\"needs_clarification\":false}", + "raw_content_sha256": "16ab820924508bb6e38b21bb14605348846c4b7936adf81e2b291d962d86654f", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-12' and clarification False match the scoped source and execution: The source states morning availability is available; its if/otherwise preference therefore selects 2027-04-12. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-11-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "3589aba05e120741edaff6cf5e2742ebef5143b2205270003c7c3771253f0b52", + "result_file_sha256": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "receipt_file_sha256": "ddb96b4b2f8836bd2d4237dec003e5c8537be97773f5b1db81fb389b57ab62ad", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-12\",\"needs_clarification\":false}", + "raw_content_sha256": "16ab820924508bb6e38b21bb14605348846c4b7936adf81e2b291d962d86654f", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-12' and clarification False match the scoped source and execution: The source states morning availability is available; its if/otherwise preference therefore selects 2027-04-12. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-11-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "955f797549a6bfa276a388fed3f78b776de53052d0bb4750ad6c774890ea1df2", + "result_file_sha256": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "receipt_file_sha256": "3a20c97a9b0ea7ae7c3daba634b738161501bac52ebda22317a79e11e6504dbb", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-12\",\"needs_clarification\":false}", + "raw_content_sha256": "16ab820924508bb6e38b21bb14605348846c4b7936adf81e2b291d962d86654f", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-12' and clarification False match the scoped source and execution: The source states morning availability is available; its if/otherwise preference therefore selects 2027-04-12. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-11-hi-m1", + "case_id": "fresh-11-hi", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "language": "hi", + "parent_paired_file_sha256": "0c297a77c1c89e33e205ed98704b1c778a321d0767c39371a05e702e03a7d097", + "question": "बताई गई शर्त के अनुसार VISIT-2111 के लिए कौन सी तारीख चुनूं? बदलाव न करें।", + "source_messages": [ + { + "id": "note-11-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-11-1", + "role": "customer", + "content": "VISIT-2111 के लिए सुबह का समय मिले तो 2027-04-12 चुनें, नहीं तो 2027-05-12।" + }, + { + "id": "note-11-2", + "role": "customer", + "content": "सुबह का समय उपलब्ध है। मैं केवल अपनी पसंद पूछ रहा हूं, बुकिंग बदलने की अनुमति नहीं दे रहा हूं।" + } + ], + "source_basis_reviewed": "The source states morning availability is available; its if/otherwise preference therefore selects 2027-04-12. Read-only scope does not prevent answering.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a9a0f2bae24e076d", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2111" + }, + "response": { + "date": "2027-03-12", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-12", + "final_scoped_date": "2027-03-12", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-11-hi-m1-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "4849c031ee76e057cb2bd1695cebba6a7644d31ca92f385cfb93ff2aa70e498b", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "f7150cbd509f1b6e6b497d35bb96c44369a5632d79eb1c55a3ded32f1498f183", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Withholds determinate date 2027-04-12 despite the source/receipts: The source states morning availability is available; its if/otherwise preference therefore selects 2027-04-12. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-11-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "29ff0fff42a78735ad4cdcc944a4f6af86e90d624bb531db64d2282b989592f4", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "b3c3c05adb1a43f37d49aa00c72ca7442e032d59c18822df76d0f5ce585b8a47", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Withholds determinate date 2027-04-12 despite the source/receipts: The source states morning availability is available; its if/otherwise preference therefore selects 2027-04-12. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-11-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "8481703d3358d387f13c7a0c6d7ed8cd2a10195926f250817cf9c93f28440de2", + "result_file_sha256": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "receipt_file_sha256": "836ae21435fac43c98f8cb8693519f11ceeea6b40452b508a6d67f01e8266d1a", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-12\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "b7f323c1d6229be3555907cf605c3036291995acf9fc575a5ef287331894401b", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-12' and clarification False match the scoped source and execution: The source states morning availability is available; its if/otherwise preference therefore selects 2027-04-12. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-12-hi-m0", + "case_id": "fresh-12-hi", + "model": "sarvam-105b", + "family": "resolved_condition", + "language": "hi", + "parent_paired_file_sha256": "4560d8691234f72124216adc2a51578376e9ff83c04f9b3d18bda920ff72b42e", + "question": "बताई गई शर्त के अनुसार VISIT-2112 के लिए कौन सी तारीख चुनूं? बदलाव न करें।", + "source_messages": [ + { + "id": "note-12-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। VISIT-2112 के लिए सुबह का समय मिले तो 2027-04-13 चुनें, नहीं तो 2027-05-13। सुबह का समय उपलब्ध नहीं है। मैं केवल अपनी पसंद पूछ रहा हूं।" + } + ], + "source_basis_reviewed": "The source states morning availability is unavailable; its if/otherwise preference therefore selects 2027-05-13. Read-only scope does not prevent answering.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-13", + "final_scoped_date": "2027-03-13", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-12-hi-m0-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "c4d143590247cda4b432e87a709fdc52723cdf5743b42e28a6b40c388e0f3d46", + "result_file_sha256": "bddfda0b83421ea19c31efae0763e4d882cd25d8f7fc51ecf50c2203c16f5d03", + "receipt_file_sha256": "e08dc24bfc45ffbf54bd5de5c516685a8aa534b17992a865165be7eda4bd34f3", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-13\",\"needs_clarification\":false}", + "raw_content_sha256": "84abd2c69924b63d951b543aa83115e0e7b4550af4fa0d56780fc8fede890196", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-13' and clarification False match the scoped source and execution: The source states morning availability is unavailable; its if/otherwise preference therefore selects 2027-05-13. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-12-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "8d23554489e9ec6108b28a77d4e3911e1b54f9381cae2a274adbf88d5c9367fe", + "result_file_sha256": "bddfda0b83421ea19c31efae0763e4d882cd25d8f7fc51ecf50c2203c16f5d03", + "receipt_file_sha256": "f19df5d5d37a474635ac5ed7467397efcdf135cb68fbc3652a12ff483bd9afd3", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-13\",\"needs_clarification\":false}", + "raw_content_sha256": "84abd2c69924b63d951b543aa83115e0e7b4550af4fa0d56780fc8fede890196", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-13' and clarification False match the scoped source and execution: The source states morning availability is unavailable; its if/otherwise preference therefore selects 2027-05-13. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-12-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "c06de70b83a50a5a98ad1369692f23ef3444a649e580d1c0f6fe89a4d3d5bfff", + "result_file_sha256": "bddfda0b83421ea19c31efae0763e4d882cd25d8f7fc51ecf50c2203c16f5d03", + "receipt_file_sha256": "6da15fd6c4ef4c8c0c7078870e3e7e583be25e4309eede5419f1ec2ed6fc7d79", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-13\",\"needs_clarification\":false}", + "raw_content_sha256": "84abd2c69924b63d951b543aa83115e0e7b4550af4fa0d56780fc8fede890196", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-13' and clarification False match the scoped source and execution: The source states morning availability is unavailable; its if/otherwise preference therefore selects 2027-05-13. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-12-hi-m1", + "case_id": "fresh-12-hi", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "language": "hi", + "parent_paired_file_sha256": "cf59821db412651e78be796c1d75ef31fa8fe20e4f83573f4cac906ee486abe8", + "question": "बताई गई शर्त के अनुसार VISIT-2112 के लिए कौन सी तारीख चुनूं? बदलाव न करें।", + "source_messages": [ + { + "id": "note-12-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। VISIT-2112 के लिए सुबह का समय मिले तो 2027-04-13 चुनें, नहीं तो 2027-05-13। सुबह का समय उपलब्ध नहीं है। मैं केवल अपनी पसंद पूछ रहा हूं।" + } + ], + "source_basis_reviewed": "The source states morning availability is unavailable; its if/otherwise preference therefore selects 2027-05-13. Read-only scope does not prevent answering.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-b1a544ffacec3e69", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2112" + }, + "response": { + "date": "2027-03-13", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-13", + "final_scoped_date": "2027-03-13", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-12-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "b30a4bafddd4ca06c77bbeef64fcf7ab8d59ef9d7195fcefc2a872e9019e3704", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "e18ead61fdaec631c774f80a0923899baf3fa34ff472b2d1b008eba8add90eea", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Withholds determinate date 2027-05-13 despite the source/receipts: The source states morning availability is unavailable; its if/otherwise preference therefore selects 2027-05-13. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-12-hi-m1-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "c438a3d4a35a19ff172d269d2a86a91d9aed5c17fe4df89f733121910de4a28e", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "dcfc78298fc6707f5bdd0c29f74c189db09d40b62ff03a71b90ee123e6ba2f03", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Withholds determinate date 2027-05-13 despite the source/receipts: The source states morning availability is unavailable; its if/otherwise preference therefore selects 2027-05-13. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-12-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "5e0bb1dcecd8ea8ad0a8bf262e60ed0abfe4e8dad01ed286c7ddb7ea3eee6fd2", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "c393a4cc402301c1074bdf6c4ed412eefd1f8f09b17bc44672e147b97620f0a7", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "incorrect", + "factual_explanation": "Withholds determinate date 2027-05-13 despite the source/receipts: The source states morning availability is unavailable; its if/otherwise preference therefore selects 2027-05-13. Read-only scope does not prevent answering.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-13-hi-m0", + "case_id": "fresh-13-hi", + "model": "sarvam-105b", + "family": "unresolved_condition", + "language": "hi", + "parent_paired_file_sha256": "11396c58f7d61c1d517783a6fb8bf5871e737bcdb15d478dc63a7e4e42320bf3", + "question": "बताई गई शर्त के अनुसार VISIT-2113 के लिए कौन सी तारीख चुनूं? बदलाव न करें।", + "source_messages": [ + { + "id": "note-13-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-13-1", + "role": "customer", + "content": "VISIT-2113 के लिए सुबह का समय हो तो 2027-04-14, नहीं हो तो 2027-05-14 चुनूंगा।" + }, + { + "id": "note-13-2", + "role": "customer", + "content": "उपलब्धता अभी किसी ने नहीं जांची है। अंदाजा न लगाएं और बुकिंग न बदलें।" + } + ], + "source_basis_reviewed": "Morning availability for VISIT-2113 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-14", + "final_scoped_date": "2027-03-14", + "expected_from_source_and_actual_execution": { + "answer": null, + "needs_clarification": true + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-13-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "5cf27cbd810552335ffee84274fea83939c2dc919e8dfb2747ebce08b98b2f71", + "result_file_sha256": "078d3f53025ce08cbdee6c9b66d1debeeb93a4fac3ea7fb3fdad08e9f2d8a614", + "receipt_file_sha256": "28302f28bc5521d9efb4afc03b55e4a98ca1f99f36c1943489e9ad59738e2011", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-03-14\",\"needs_clarification\":false}", + "raw_content_sha256": "3be7e35ba83ddd02d3611c9f9823d34fcf7aef5b72761e62c517835aded0425b", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "answer_source_role": "confirmed_date_substituted_for_unresolved_condition", + "factual_verdict": "incorrect", + "factual_explanation": "Reports definite date '2027-03-14' when the source leaves the requested date or condition unresolved: Morning availability for VISIT-2113 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-13-hi-m0-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "ff58fe9ad388c72f1f0963c93f562739fe1719a313c4e627397c7a3c18124ad3", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "receipt_file_sha256": "de632e47357ee6044575cac15dd7f4404c4d3df248f0cbf3ab76e9165fc55501", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2113 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-13-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "4e8a38f5c032c80a5359142c410139466b36ac94f4b74172f2436ae377dc19e1", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "9ea76e1cb19190c2865c8070c91c07d15a5e7dbd32b11858d3ac3315b4215d97", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2113 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-13-hi-m1", + "case_id": "fresh-13-hi", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "language": "hi", + "parent_paired_file_sha256": "d7483f617d50efd3c2bad7b3bdbbc6bf091b1342ac96e3bf7a3485aba44bbd92", + "question": "बताई गई शर्त के अनुसार VISIT-2113 के लिए कौन सी तारीख चुनूं? बदलाव न करें।", + "source_messages": [ + { + "id": "note-13-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-13-1", + "role": "customer", + "content": "VISIT-2113 के लिए सुबह का समय हो तो 2027-04-14, नहीं हो तो 2027-05-14 चुनूंगा।" + }, + { + "id": "note-13-2", + "role": "customer", + "content": "उपलब्धता अभी किसी ने नहीं जांची है। अंदाजा न लगाएं और बुकिंग न बदलें।" + } + ], + "source_basis_reviewed": "Morning availability for VISIT-2113 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-14", + "final_scoped_date": "2027-03-14", + "expected_from_source_and_actual_execution": { + "answer": null, + "needs_clarification": true + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-13-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "a790c840add229e14be1a32766f4cb8852a040a14e3c3df2abb2ab732c6ebee3", + "result_file_sha256": "1243d0f5fa6cce0604eaab8f66bf829bb90d45979ff6a37c353828ca00d8b571", + "receipt_file_sha256": "64a52239e94875898b8fbc56f3ce33ebea3f399a599ba1cdb72555aa364c6d4a", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_performed\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "09f7bf6dbb513a106580d5b3c3ea967a4ec9ac8570b9d99ecbd2a492f9ef2680", + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2113 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_performed' does not match expected 'not_requested': read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-13-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "856f177b6e02170551c2cbcf7e21a035980187042b274774163ac23226a9d428", + "result_file_sha256": "13352be51ec1bf1e435624454d58273e0d5f946cf4140cb0b6b7a9efd71dfe1a", + "receipt_file_sha256": "89567073e2fef3e6fce6185ff9ce3ac691465c4ef34f2f0a38afbaf0ed8d0a32", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-14\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "98d4d477f814b021f1b0bd9213912a3fcb2805ad4a7edff55c5ca6729d02fc3c", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-14", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-14", + "needs_clarification": false + }, + "answer_source_role": "unsupported_conditional_branch", + "factual_verdict": "incorrect", + "factual_explanation": "Reports definite date '2027-04-14' when the source leaves the requested date or condition unresolved: Morning availability for VISIT-2113 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-13-hi-m1-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "0435b349db344d70f13998c79fe6dbb55bef431fa2d9342be214815d8d2676ed", + "result_file_sha256": "b6474bc8194ce66a7fd312e11be1d6465b06f536176a1bb68eebcb0a95a69d0d", + "receipt_file_sha256": "b6af760c74d3ea233bfaf9be4485937c0fede90b4de79d6d8007ec8389778664", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-14\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "7827a8c7b971979647ee14efc59af7031db8e64639d17be755062049fa079f44", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "answer_source_role": "confirmed_date_substituted_for_unresolved_condition", + "factual_verdict": "incorrect", + "factual_explanation": "Reports definite date '2027-03-14' when the source leaves the requested date or condition unresolved: Morning availability for VISIT-2113 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-14-hi-m0", + "case_id": "fresh-14-hi", + "model": "sarvam-105b", + "family": "unresolved_condition", + "language": "hi", + "parent_paired_file_sha256": "a16f3989b2882ef9edfaba37e45d6418ceff339a80f23111c1e8e3fd85d73675", + "question": "बताई गई शर्त के अनुसार VISIT-2114 के लिए कौन सी तारीख चुनूं? बदलाव न करें।", + "source_messages": [ + { + "id": "note-14-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। VISIT-2114 के लिए सुबह का समय हो तो 2027-04-15, नहीं हो तो 2027-05-15 चुनूंगा। उपलब्धता अभी किसी ने नहीं जांची है। अंदाजा न लगाएं और बुकिंग न बदलें।" + } + ], + "source_basis_reviewed": "Morning availability for VISIT-2114 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-b76ee068dd207642", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2114" + }, + "response": { + "date": "2027-03-15", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-15", + "final_scoped_date": "2027-03-15", + "expected_from_source_and_actual_execution": { + "answer": null, + "needs_clarification": true + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-14-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "9f41fe621ef87f5a0070146fda8e8bcbf47dcbbeb9b4e62e2cda3566dbedf855", + "result_file_sha256": "1243d0f5fa6cce0604eaab8f66bf829bb90d45979ff6a37c353828ca00d8b571", + "receipt_file_sha256": "f217a91fd99f22cdc86e81043db3501207aac5722665c462aa39ab2eb2240897", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_performed\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "09f7bf6dbb513a106580d5b3c3ea967a4ec9ac8570b9d99ecbd2a492f9ef2680", + "raw": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2114 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_performed' does not match expected 'not_requested': read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-14-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "87b65b1f5af4b9262746f68615d372ef17bc00862976ab92dd8b831d3ad3fa22", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "receipt_file_sha256": "2a23312b84fa6f1fe647df4858e38db487d01fe8bd3c53af9f8ae0d89c90652e", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2114 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-14-hi-m0-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "a7100b1ee29a6b47487f06c59024db5d8aac182cb17c8e329dc311b89d59a2f6", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "receipt_file_sha256": "6c65bea64a28a05272391915d8c474ef92fab4ddc129a037d2fc640882f9072d", + "terminal": "incomplete", + "finish_reason": "length", + "raw_content": null, + "raw_content_sha256": null, + "raw": null, + "application_output": null, + "answer_source_role": null, + "factual_verdict": "invalid_or_missing", + "factual_explanation": "No valid parsed final answer to assess.", + "raw_status_verdict": "invalid_or_missing", + "raw_status_explanation": "None does not match expected 'not_requested': read-only question and no requested booking change.", + "projected_status_verdict": "invalid_or_missing", + "projected_status_explanation": "Application status None does not match expected 'not_requested' from actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-14-hi-m1", + "case_id": "fresh-14-hi", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "language": "hi", + "parent_paired_file_sha256": "d9ff356da53df03e1d026dd922916178668afc469cf139f2fcd0277bd6eba957", + "question": "बताई गई शर्त के अनुसार VISIT-2114 के लिए कौन सी तारीख चुनूं? बदलाव न करें।", + "source_messages": [ + { + "id": "note-14-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। VISIT-2114 के लिए सुबह का समय हो तो 2027-04-15, नहीं हो तो 2027-05-15 चुनूंगा। उपलब्धता अभी किसी ने नहीं जांची है। अंदाजा न लगाएं और बुकिंग न बदलें।" + } + ], + "source_basis_reviewed": "Morning availability for VISIT-2114 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-15", + "final_scoped_date": "2027-03-15", + "expected_from_source_and_actual_execution": { + "answer": null, + "needs_clarification": true + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-14-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "b64c11814589c3ffdf19575dbb6564bed70b09c025e088fa0b2a6b53371da24b", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "a9b9edc695db2e69e92861a6deb42829dab7865d3cd2fdef7d68e8cdb2eba358", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2114 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-14-hi-m1-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "9937ae684de1e41595673f2cddc14cae48ad791530a3c1f2f70a4bfbc4a8fabd", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "96d9aa6cb0d0928fd3e5e2ab5141970a303b6b2f2466aa67fa9887ce627ac39f", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2114 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-14-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "0a27f18c7b4ebe11db29e151f84affb77015ead65fe5ca2052b8c14db78a8914", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "dc55f01c26fbd6a1f3ade875f69e4a519c3930b6f1637f8f810f0e7d37aac599", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2114 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-15-hi-m0", + "case_id": "fresh-15-hi", + "model": "sarvam-105b", + "family": "unresolved_condition", + "language": "hi", + "parent_paired_file_sha256": "c0aa8b7a3af1edd4ed4b07693eb2e8f979bad27ece3987b6350f7a7ad66b2fab", + "question": "बताई गई शर्त के अनुसार VISIT-2115 के लिए कौन सी तारीख चुनूं? बदलाव न करें।", + "source_messages": [ + { + "id": "note-15-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-15-1", + "role": "customer", + "content": "VISIT-2115 के लिए सुबह का समय हो तो 2027-04-16, नहीं हो तो 2027-05-16 चुनूंगा।" + }, + { + "id": "note-15-2", + "role": "customer", + "content": "उपलब्धता अभी किसी ने नहीं जांची है। अंदाजा न लगाएं और बुकिंग न बदलें।" + } + ], + "source_basis_reviewed": "Morning availability for VISIT-2115 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-16", + "final_scoped_date": "2027-03-16", + "expected_from_source_and_actual_execution": { + "answer": null, + "needs_clarification": true + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-15-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "5d55c32068e80381ce720a2142849377c17e0246625e7f46eceaa272d9303296", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "receipt_file_sha256": "12977760072bc2510a42f355abe4fd31ca6b74e6d333ca6a15017fb4f271f7f2", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2115 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-15-hi-m0-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "438315ed6d7e774dbc268349f9b5f76f0d8cbe0b0de7ffd58c08ba964f28e6b2", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "receipt_file_sha256": "4b0048a4698578dee33943831667e4de0d829e99ec52e4e612c02567b515a024", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2115 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-15-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "b37ae62d4e34203d919be1c2cfbd13a5de6d60a9ba5fcb60e1305230139085c5", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "receipt_file_sha256": "61d5677e23e24cf8ad725ecae79174a83aaac004594b4c17ff69bea991f043f9", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2115 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-15-hi-m1", + "case_id": "fresh-15-hi", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "language": "hi", + "parent_paired_file_sha256": "e74873347961b3b4abda92dd6132941616a3b4de4949c38ff668d6d44d9d6232", + "question": "बताई गई शर्त के अनुसार VISIT-2115 के लिए कौन सी तारीख चुनूं? बदलाव न करें।", + "source_messages": [ + { + "id": "note-15-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-15-1", + "role": "customer", + "content": "VISIT-2115 के लिए सुबह का समय हो तो 2027-04-16, नहीं हो तो 2027-05-16 चुनूंगा।" + }, + { + "id": "note-15-2", + "role": "customer", + "content": "उपलब्धता अभी किसी ने नहीं जांची है। अंदाजा न लगाएं और बुकिंग न बदलें।" + } + ], + "source_basis_reviewed": "Morning availability for VISIT-2115 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-16", + "final_scoped_date": "2027-03-16", + "expected_from_source_and_actual_execution": { + "answer": null, + "needs_clarification": true + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-15-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "7ac2660cdbb7fa72d0f21c38bf98c62d17f0e1255a49e26d21a1a77c0a264b32", + "result_file_sha256": "afd6ab4071b0df347bb5f8ee7e7e3b493e9f731b4ac18a0adbfee443737adcad", + "receipt_file_sha256": "acaec1fbbb54cba395d5a1bd4af2c608f563dccbbea99d56fbe89f53e374cc2e", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-16\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "a3c47af1bf9caf574a51522c21d550db0a1e82e5621e992132138af3074ac320", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-16", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-16", + "needs_clarification": false + }, + "answer_source_role": "unsupported_conditional_branch", + "factual_verdict": "incorrect", + "factual_explanation": "Reports definite date '2027-04-16' when the source leaves the requested date or condition unresolved: Morning availability for VISIT-2115 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-15-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "5ae20bc519af12068152f39900174da7a31816b135b8ffd8e5ece54d7a08b724", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "558acb9ab773a32b293518ff6c41c74110207c0198a030f2ac9ba178ce12f93c", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2115 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-15-hi-m1-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "1cf6d1d8504e0390af3d2b49e48508a683011bf3cbeaeeca446944c37caf102b", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "4f960eff642bb8567511d69a1f5448a03f6137ee65e02b4e67c6a0f43d3e1b2b", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2115 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-16-hi-m0", + "case_id": "fresh-16-hi", + "model": "sarvam-105b", + "family": "unresolved_condition", + "language": "hi", + "parent_paired_file_sha256": "e7a4af83c2acb9b3d39780bc56949f058979799d35bb458ce6ead7f26dd5ce60", + "question": "बताई गई शर्त के अनुसार VISIT-2116 के लिए कौन सी तारीख चुनूं? बदलाव न करें।", + "source_messages": [ + { + "id": "note-16-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। VISIT-2116 के लिए सुबह का समय हो तो 2027-04-17, नहीं हो तो 2027-05-17 चुनूंगा। उपलब्धता अभी किसी ने नहीं जांची है। अंदाजा न लगाएं और बुकिंग न बदलें।" + } + ], + "source_basis_reviewed": "Morning availability for VISIT-2116 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-17", + "final_scoped_date": "2027-03-17", + "expected_from_source_and_actual_execution": { + "answer": null, + "needs_clarification": true + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-16-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "90f200e70ae8d7ee6bc99de4b0b6d8cb88ab6e66fa7daf83538ff349f5dcdd56", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "receipt_file_sha256": "07f172f064cefbfb17602c6b67086d3e4d022666db8caf0cea56887211e86af9", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2116 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-16-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "2c0c33a86e0f5a760d2e11491665e024c8b2aeb10d135cf3b754c9bf5e9c9c4e", + "result_file_sha256": "3a023f469ad2b66a2c3cccff96ecad9a50cb6437c0292e365161fb00067f87ce", + "receipt_file_sha256": "bbcf0b13632aa998db8b768bd82353139dd6af8ecb745c78805c6dc5df4a3150", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\": \"not_requested\", \"answer\": null, \"needs_clarification\": true}", + "raw_content_sha256": "1f143a6c338737b40763a356b6632537b85ce2afc83088c4f554fee275b1e164", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2116 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-16-hi-m0-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "907d1fa701e8e6c2280efc98b784d685c16b1c5c8b3d4290844b36d1c0b6abc0", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "receipt_file_sha256": "25ba1bf5b62dc20b512bff118b4aeddd96d3cb200c94670196af45153d5b4ac3", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2116 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-16-hi-m1", + "case_id": "fresh-16-hi", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "language": "hi", + "parent_paired_file_sha256": "420eabafefa7c1a72623d73c1a341337f07839ac2886b3ac2a7ee72168a84cdb", + "question": "बताई गई शर्त के अनुसार VISIT-2116 के लिए कौन सी तारीख चुनूं? बदलाव न करें।", + "source_messages": [ + { + "id": "note-16-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। VISIT-2116 के लिए सुबह का समय हो तो 2027-04-17, नहीं हो तो 2027-05-17 चुनूंगा। उपलब्धता अभी किसी ने नहीं जांची है। अंदाजा न लगाएं और बुकिंग न बदलें।" + } + ], + "source_basis_reviewed": "Morning availability for VISIT-2116 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-17", + "final_scoped_date": "2027-03-17", + "expected_from_source_and_actual_execution": { + "answer": null, + "needs_clarification": true + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-16-hi-m1-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "6f4bdab0bf1fbf42aacf84fd824d53ac8a1d2dfd527fbdd8a689ae6554ed3e44", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "19632a0a239d7c891728e2c58e6c110afcf63f54354ac5c01bc266c194d19b56", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2116 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-16-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "0e7a5d044645c01d145c6e5535a2c37547f017d7e9daafc39de2961be5cade59", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "d7356a799cf7cc0f73b6fbbc10e89da7a150c8dd624c845f685f3543f955031a", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2116 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-16-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "ff332d6d6dc478c3dd8b6d623db6ee27f434b0f47ba63176e61e54d3974c22e8", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "receipt_file_sha256": "f0821ecb4ea6f780b020e71efcc91e8496ac815eca6bf59f5520ba051380b9e5", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "raw": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "application_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "answer_source_role": null, + "factual_verdict": "correct", + "factual_explanation": "Answer None and clarification True match the scoped source and execution: Morning availability for VISIT-2116 has not been checked, so neither stated conditional date is established. The source forbids a guess.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-17-hi-m0", + "case_id": "fresh-17-hi", + "model": "sarvam-105b", + "family": "separate_booking", + "language": "hi", + "parent_paired_file_sha256": "8c17fe6ba97f972ebe8646fd1fbc81d478544843bc334ace7077f151a9c39c31", + "question": "VISIT-2117 के लिए अभी मेरी पसंद की तारीख कौन सी है? केवल बताएं; बुकिंग न बदलें।", + "source_messages": [ + { + "id": "note-17-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-17-1", + "role": "customer", + "content": "VISIT-2117 के लिए मेरी पसंद 2027-04-18 है। VISIT-3117 के लिए पहले 2027-03-18 थी।" + }, + { + "id": "note-17-2", + "role": "customer", + "content": "सुधार केवल VISIT-3117 के लिए है: अब 2027-05-18 चाहिए। पहली बुकिंग की पसंद नहीं बदली है।" + } + ], + "source_basis_reviewed": "VISIT-2117 retains the stated preference 2027-04-18; the later correction applies only to the separately numbered booking.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-18", + "final_scoped_date": "2027-03-18", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-17-hi-m0-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "c4a6d9feb8549bad1cb2ef3577a1333c9f8ef644b0fa59509c50c883c005a2f3", + "result_file_sha256": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "receipt_file_sha256": "6dbd107208abc921fd51e7de68e777a9d2bf2734db197b2098048bea13f41bcd", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-18\",\"needs_clarification\":false}", + "raw_content_sha256": "9449807d1a496bfec71b5dbf1c86405ed0ae5d9745146d9437a26e0842b5006f", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-18' and clarification False match the scoped source and execution: VISIT-2117 retains the stated preference 2027-04-18; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-17-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "ddd4227fd190a97657ae54ee9e3fac9ca90e4459ded41f3bb36f7cb6ffdd7ebe", + "result_file_sha256": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "receipt_file_sha256": "b4c9d0c82e44024b2c38c9a58fac3618efb08d72d84178c8fda9b3b42f4f57f1", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-18\",\"needs_clarification\":false}", + "raw_content_sha256": "9449807d1a496bfec71b5dbf1c86405ed0ae5d9745146d9437a26e0842b5006f", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-18' and clarification False match the scoped source and execution: VISIT-2117 retains the stated preference 2027-04-18; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-17-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "ea467ced781e6d57312a2e2d33021223982707deebd32f90ec0f37a82cc0cb38", + "result_file_sha256": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "receipt_file_sha256": "0736abfb862db6ec2aae43f3f414d62dfde241f409cdc296a6242cdc3fc42954", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-18\",\"needs_clarification\":false}", + "raw_content_sha256": "9449807d1a496bfec71b5dbf1c86405ed0ae5d9745146d9437a26e0842b5006f", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-18' and clarification False match the scoped source and execution: VISIT-2117 retains the stated preference 2027-04-18; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-17-hi-m1", + "case_id": "fresh-17-hi", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "language": "hi", + "parent_paired_file_sha256": "2714c2166e9bbed46c1667568829602feb5c2799f407b39b74587968c2400cec", + "question": "VISIT-2117 के लिए अभी मेरी पसंद की तारीख कौन सी है? केवल बताएं; बुकिंग न बदलें।", + "source_messages": [ + { + "id": "note-17-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-17-1", + "role": "customer", + "content": "VISIT-2117 के लिए मेरी पसंद 2027-04-18 है। VISIT-3117 के लिए पहले 2027-03-18 थी।" + }, + { + "id": "note-17-2", + "role": "customer", + "content": "सुधार केवल VISIT-3117 के लिए है: अब 2027-05-18 चाहिए। पहली बुकिंग की पसंद नहीं बदली है।" + } + ], + "source_basis_reviewed": "VISIT-2117 retains the stated preference 2027-04-18; the later correction applies only to the separately numbered booking.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8072fee0ee9b943b", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2117" + }, + "response": { + "date": "2027-03-18", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-18", + "final_scoped_date": "2027-03-18", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-17-hi-m1-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "db92a0a8d5fd3bde632abb589de4e414bb297f9c2d3a4d574e2b7e83a38ec905", + "result_file_sha256": "87c81f64b32de0e465ec65f5672c4e2b281ceaddfdb2a11351a1e137c744f10c", + "receipt_file_sha256": "a35f8f6887d18553965633ae8fc31957712f4d645da73ba99d74aab7967f4cb0", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-18\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "4070209a7773e6e396574c3cc304e3528574fb48855f2e8178fb6206d1c899c5", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "answer_source_role": "confirmed_date_substituted_for_scoped_preference", + "factual_verdict": "incorrect", + "factual_explanation": "Reports '2027-03-18' rather than scoped date 2027-04-18: VISIT-2117 retains the stated preference 2027-04-18; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-17-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "f2acfe087e9869e9b8d7a780d68b9d4b7b87f690efb38b3383b8302137ab2f49", + "result_file_sha256": "c906ece7cab4749ad05a7177d633850ff1d4fb81386b5fae720064c1ca673a6d", + "receipt_file_sha256": "4e8e1213b0a4466a273c597281bec0ae036d7cb63e52d598b8bd9ade95a58a22", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-18\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "2d0061ed80efcc2974dfae76435db00a06c1b6479dfbefd7325a9093a9208008", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-18' and clarification False match the scoped source and execution: VISIT-2117 retains the stated preference 2027-04-18; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-17-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "3cd8bb54ebdfe701a13c4eafb3dab834e36b1772d240d4177fe2f02decb5cb33", + "result_file_sha256": "62b25401af230db17301f8eac4ec80f89e16b0252c2846d9f86d83221f909d37", + "receipt_file_sha256": "e4d87dd082bcbc129abd7b2fc6d1e5bcd61b50f04f1525c5ed00045f6b3b9cc7", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-03-18\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "7dd7aeb456a1f57e4efb4ee56b66bfe6b70a2aece9d462ab973192f4e3492e3c", + "raw": { + "action_status": "already_applied", + "answer": "2027-03-18", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "answer_source_role": "confirmed_date_substituted_for_scoped_preference", + "factual_verdict": "incorrect", + "factual_explanation": "Reports '2027-03-18' rather than scoped date 2027-04-18: VISIT-2117 retains the stated preference 2027-04-18; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'already_applied' does not match expected 'not_requested': read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-18-hi-m0", + "case_id": "fresh-18-hi", + "model": "sarvam-105b", + "family": "separate_booking", + "language": "hi", + "parent_paired_file_sha256": "0bfefcb3f26573d930dbf769b2cc2a2fc77b864b70e67a9b8772e2767568d1bd", + "question": "VISIT-2118 के लिए अभी मेरी पसंद की तारीख कौन सी है? केवल बताएं; बुकिंग न बदलें।", + "source_messages": [ + { + "id": "note-18-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। VISIT-2118 के लिए मेरी पसंद 2027-04-19 है। VISIT-3118 के लिए पहले 2027-03-19 थी। सुधार केवल VISIT-3118 के लिए है: अब 2027-05-19 चाहिए। पहली बुकिंग की पसंद नहीं बदली है।" + } + ], + "source_basis_reviewed": "VISIT-2118 retains the stated preference 2027-04-19; the later correction applies only to the separately numbered booking.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-936515dc80e63774", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2118" + }, + "response": { + "date": "2027-03-19", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-19", + "final_scoped_date": "2027-03-19", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-18-hi-m0-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "cbe6a397357bec175b1bb4da7f1e0676218c162bf327f56a46370d8c2ebd36ef", + "result_file_sha256": "0c021d23374bf111ad50ed318de4af996b653ca2927b333c29846f3500862618", + "receipt_file_sha256": "6dff6f158c2c3aafa2db253a7bd8a606242fbdb0c18cb1be04e289ba7a0ef2bc", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-03-19\",\"needs_clarification\":false}", + "raw_content_sha256": "391856aa4d2391dc1e97997ca3e918508ab839444d508a3c0fe41e941fded38d", + "raw": { + "action_status": "confirmed", + "answer": "2027-03-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "answer_source_role": "confirmed_date_substituted_for_scoped_preference", + "factual_verdict": "incorrect", + "factual_explanation": "Reports '2027-03-19' rather than scoped date 2027-04-19: VISIT-2118 retains the stated preference 2027-04-19; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'confirmed' does not match expected 'not_requested': read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-18-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "52390ea81676214b7e607206f102dcfe3bfb8cfc857e6bc09a3d1b37ddddfde0", + "result_file_sha256": "7907fdb6823f82aa4202371fe0d8b9004e10bbb108790c519a5f3d55774f4c29", + "receipt_file_sha256": "54141e0dc4423ce477d584ad81c9365edbcc190550872134d262709d33446d80", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\": \"not_requested\", \"answer\": \"2027-04-19\", \"needs_clarification\": false}", + "raw_content_sha256": "50c8abf6b1d7f866fc7e1a6ab969e20054e967f07d729a0c7e7fc6b2d9e71225", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-19' and clarification False match the scoped source and execution: VISIT-2118 retains the stated preference 2027-04-19; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-18-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "909e3e9d365c9d2c34daeca82f31bed4a41ecc7f95c020da922cfb85e95a1158", + "result_file_sha256": "03bc7ffa97b58cb6893a53bd07ae07966616cd8c647d669421d85a572a958aae", + "receipt_file_sha256": "880f72a8c426d3f6362bd09f6d464b8d000b5b7357eea90f1e58db3f84ec8171", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-19\",\"needs_clarification\":false}", + "raw_content_sha256": "9f13b701995856fa703eec6041b4da4ba1c06e0ab9736e62b8ece9e50b06320e", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-19' and clarification False match the scoped source and execution: VISIT-2118 retains the stated preference 2027-04-19; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-18-hi-m1", + "case_id": "fresh-18-hi", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "language": "hi", + "parent_paired_file_sha256": "66e1909a00c9c5e9fb3cd6a08766c1a88655144488bfceafafd6637131b1aec9", + "question": "VISIT-2118 के लिए अभी मेरी पसंद की तारीख कौन सी है? केवल बताएं; बुकिंग न बदलें।", + "source_messages": [ + { + "id": "note-18-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। VISIT-2118 के लिए मेरी पसंद 2027-04-19 है। VISIT-3118 के लिए पहले 2027-03-19 थी। सुधार केवल VISIT-3118 के लिए है: अब 2027-05-19 चाहिए। पहली बुकिंग की पसंद नहीं बदली है।" + } + ], + "source_basis_reviewed": "VISIT-2118 retains the stated preference 2027-04-19; the later correction applies only to the separately numbered booking.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-af16c75ae1f1afe2", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2118" + }, + "response": { + "date": "2027-03-19", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-19", + "final_scoped_date": "2027-03-19", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-18-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "121469deb870065cc0de93595bb419bd21d3862292feacbf8d7a6d63e5d5ff5e", + "result_file_sha256": "00d649cfc39ec358aa1d45e738f53feb9cb97e2c7b7fc9ca56a64eec5020214e", + "receipt_file_sha256": "e9f0ef0384f40a08252fee8afcb61a584ebc0f450b51ab89c0a9fb895b62c88e", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-19\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "c89680404de922e7ad56887a56811c7ddb4eefcbb16c8b8689f6038edcee313e", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "answer_source_role": "confirmed_date_substituted_for_scoped_preference", + "factual_verdict": "incorrect", + "factual_explanation": "Reports '2027-03-19' rather than scoped date 2027-04-19: VISIT-2118 retains the stated preference 2027-04-19; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-18-hi-m1-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "29c3cad816bc1a263117e72c4b9387cd61c0665d54b7cdba74500937ab71b095", + "result_file_sha256": "00d649cfc39ec358aa1d45e738f53feb9cb97e2c7b7fc9ca56a64eec5020214e", + "receipt_file_sha256": "6b390f5b0327e77e31388bbaf2c14bfed713858ad1740a8af164b85aebce0bea", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-19\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "c89680404de922e7ad56887a56811c7ddb4eefcbb16c8b8689f6038edcee313e", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "answer_source_role": "confirmed_date_substituted_for_scoped_preference", + "factual_verdict": "incorrect", + "factual_explanation": "Reports '2027-03-19' rather than scoped date 2027-04-19: VISIT-2118 retains the stated preference 2027-04-19; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-18-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "e11e3aa2b69500f2ea94a43491d5c47620d950990d41176985cc744941c8b62a", + "result_file_sha256": "d25fb810ce31a2a7fb0b34adaf0be9c19ea1c74efcf7c2e95310c0b878969515", + "receipt_file_sha256": "9e2477aad83ce75fb80df939f7cee7e109fb58a7c5f7c77ffa6e5677e948d6ff", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-19\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "0c780209084c4f59c067b473bc4a685609adb35fea83b9dd401c1d02860ebde6", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-19' and clarification False match the scoped source and execution: VISIT-2118 retains the stated preference 2027-04-19; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-19-hi-m0", + "case_id": "fresh-19-hi", + "model": "sarvam-105b", + "family": "separate_booking", + "language": "hi", + "parent_paired_file_sha256": "977af102af847f78c10756ec0c3f59ee4d084d54e6b725555197a36cd3b7c652", + "question": "VISIT-2119 के लिए अभी मेरी पसंद की तारीख कौन सी है? केवल बताएं; बुकिंग न बदलें।", + "source_messages": [ + { + "id": "note-19-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-19-1", + "role": "customer", + "content": "VISIT-2119 के लिए मेरी पसंद 2027-04-20 है। VISIT-3119 के लिए पहले 2027-03-20 थी।" + }, + { + "id": "note-19-2", + "role": "customer", + "content": "सुधार केवल VISIT-3119 के लिए है: अब 2027-05-20 चाहिए। पहली बुकिंग की पसंद नहीं बदली है।" + } + ], + "source_basis_reviewed": "VISIT-2119 retains the stated preference 2027-04-20; the later correction applies only to the separately numbered booking.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a85614897129e7c8", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2119" + }, + "response": { + "date": "2027-03-20", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-20", + "final_scoped_date": "2027-03-20", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-19-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "01848430d10cbe674db94ac746b866ffb303e0b7aa2a6f246ae2ac5922d3a73b", + "result_file_sha256": "4f2ff4c6daf1623e66caead1eec254086b77631cbd7ac1df8765b25a639a0c24", + "receipt_file_sha256": "0bf810e1ae116c8104668027db237b2ce90c80c138cb04f8539442a26532b0f4", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-20\",\"needs_clarification\":false}", + "raw_content_sha256": "ccbb204d6da4516cd3f603b7585003889275c2ceab7bdb788b3288039143cf26", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-20' and clarification False match the scoped source and execution: VISIT-2119 retains the stated preference 2027-04-20; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-19-hi-m0-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "45a2aae1dba1fd3f9f366aebb49c535793c04f93e6d5bbce45917a88ee730dfe", + "result_file_sha256": "4f2ff4c6daf1623e66caead1eec254086b77631cbd7ac1df8765b25a639a0c24", + "receipt_file_sha256": "42b7edf2d6f0b87ab8c6d04f3dfcfc58c65c51d387781bed65b05fb47c537f24", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-20\",\"needs_clarification\":false}", + "raw_content_sha256": "ccbb204d6da4516cd3f603b7585003889275c2ceab7bdb788b3288039143cf26", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-20' and clarification False match the scoped source and execution: VISIT-2119 retains the stated preference 2027-04-20; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-19-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "a3102c61b9faf4bf7c5589abebeb3be7f1d06a1e17ab6d8f65d9c276b3dc70ac", + "result_file_sha256": "cc96970b1133980764a44191c39620285b199dd73c992b284124711c5ff0bb66", + "receipt_file_sha256": "a03c4b6981233d6f0c48af2485041e86359ff8cfcb5f6a9318019dca9665a1f2", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-04-20\",\"needs_clarification\":false}", + "raw_content_sha256": "21c45bdb3795dda5ac838dca8e034b9d7cc6a983c4b914825697261c5a9838a3", + "raw": { + "action_status": "already_applied", + "answer": "2027-04-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-20' and clarification False match the scoped source and execution: VISIT-2119 retains the stated preference 2027-04-20; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'already_applied' does not match expected 'not_requested': read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-19-hi-m1", + "case_id": "fresh-19-hi", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "language": "hi", + "parent_paired_file_sha256": "11d5cc61760707ca97d3b87e67b5150c0f7d13085304c23805dad6d6f70d055e", + "question": "VISIT-2119 के लिए अभी मेरी पसंद की तारीख कौन सी है? केवल बताएं; बुकिंग न बदलें।", + "source_messages": [ + { + "id": "note-19-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-19-1", + "role": "customer", + "content": "VISIT-2119 के लिए मेरी पसंद 2027-04-20 है। VISIT-3119 के लिए पहले 2027-03-20 थी।" + }, + { + "id": "note-19-2", + "role": "customer", + "content": "सुधार केवल VISIT-3119 के लिए है: अब 2027-05-20 चाहिए। पहली बुकिंग की पसंद नहीं बदली है।" + } + ], + "source_basis_reviewed": "VISIT-2119 retains the stated preference 2027-04-20; the later correction applies only to the separately numbered booking.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9dc502eda5c2fffc", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2119" + }, + "response": { + "date": "2027-03-20", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-20", + "final_scoped_date": "2027-03-20", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-19-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "fad77fde48e6309d0d115630d5cd75ed9cbf54570c2a131ec464535db1c583ca", + "result_file_sha256": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "receipt_file_sha256": "4a8aa1e1d8c684f1f1a6ee81bac3544fc09ae5cf18dcc5dd4b0853b61b1fae15", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-20\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "f4f4478b5c7f08706d0eb37b41fdb9c470082ee0f253ae4f0ca504e2eb97a9a7", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "answer_source_role": "confirmed_date_substituted_for_scoped_preference", + "factual_verdict": "incorrect", + "factual_explanation": "Reports '2027-03-20' rather than scoped date 2027-04-20: VISIT-2119 retains the stated preference 2027-04-20; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-19-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "33ecc948f17d2e91bad7cbab954833dfe2e2b4611752d0fa77783590162c3ec9", + "result_file_sha256": "825f103ad8b8e9643c7bfbc4865f52b7b17c40b04bfdc0ed35f3985eec0cf307", + "receipt_file_sha256": "9fa95b8d2fa34103487d714d33681bd0067a56d6d4b8a1568c446ede8df7d57e", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-20\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "5a4ba0ed1526644e4486901a072d82ff351241242e9f64093067faa5ab064fa6", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-20' and clarification False match the scoped source and execution: VISIT-2119 retains the stated preference 2027-04-20; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-19-hi-m1-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "6d467ae89988ca1007ab70800a4fb904be054ca0b8724edc564f5ac260245a62", + "result_file_sha256": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "receipt_file_sha256": "7e32326042b5bb26bb0aad4fe015f3cb14959742bd4f4998920dfc53d285c532", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-20\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "f4f4478b5c7f08706d0eb37b41fdb9c470082ee0f253ae4f0ca504e2eb97a9a7", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "answer_source_role": "confirmed_date_substituted_for_scoped_preference", + "factual_verdict": "incorrect", + "factual_explanation": "Reports '2027-03-20' rather than scoped date 2027-04-20: VISIT-2119 retains the stated preference 2027-04-20; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-20-hi-m0", + "case_id": "fresh-20-hi", + "model": "sarvam-105b", + "family": "separate_booking", + "language": "hi", + "parent_paired_file_sha256": "0175b7fbc7ba91b926622a0d9f5ab61142763618970c52b499ae62b903c6fba3", + "question": "VISIT-2120 के लिए अभी मेरी पसंद की तारीख कौन सी है? केवल बताएं; बुकिंग न बदलें।", + "source_messages": [ + { + "id": "note-20-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। VISIT-2120 के लिए मेरी पसंद 2027-04-21 है। VISIT-3120 के लिए पहले 2027-03-21 थी। सुधार केवल VISIT-3120 के लिए है: अब 2027-05-21 चाहिए। पहली बुकिंग की पसंद नहीं बदली है।" + } + ], + "source_basis_reviewed": "VISIT-2120 retains the stated preference 2027-04-21; the later correction applies only to the separately numbered booking.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-21", + "final_scoped_date": "2027-03-21", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-20-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "f3549a62ef7d3e57919544e2a64f00148b39286a54d34e1ba65480ebed3f12c5", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "receipt_file_sha256": "6d722b01ad40aa7d015c08d77fd9cff192e18c9a53bddd287e9197c62785bae6", + "terminal": "incomplete", + "finish_reason": "length", + "raw_content": null, + "raw_content_sha256": null, + "raw": null, + "application_output": null, + "answer_source_role": null, + "factual_verdict": "invalid_or_missing", + "factual_explanation": "No valid parsed final answer to assess.", + "raw_status_verdict": "invalid_or_missing", + "raw_status_explanation": "None does not match expected 'not_requested': read-only question and no requested booking change.", + "projected_status_verdict": "invalid_or_missing", + "projected_status_explanation": "Application status None does not match expected 'not_requested' from actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-20-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "efa067e8af0759c76da61290e164cd736db042416ad5e7c984c40fdef27ffd7f", + "result_file_sha256": "47da800200c0c2a40c8f6e6d4d56ab1df082b5204fe5af8fa9d8b032bf3fec83", + "receipt_file_sha256": "8f2761f3864539841dc531085b387aced83dd097d8c0a90905a0b0e9beea3d41", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-21\",\"needs_clarification\":false}", + "raw_content_sha256": "50dd30c16ff07995e873e614e9c42b88e61dd6fd45c0f360a0172c3453e458a3", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-21' and clarification False match the scoped source and execution: VISIT-2120 retains the stated preference 2027-04-21; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-20-hi-m0-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "0d6d7af76cec35acdff647bea3612893376be369d46f0881579e2368607fe5bc", + "result_file_sha256": "47da800200c0c2a40c8f6e6d4d56ab1df082b5204fe5af8fa9d8b032bf3fec83", + "receipt_file_sha256": "66fa42fe76f2486ac3e6c391d0049791a4aeb70a8cf11b3ee1760a7107c0e53e", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-21\",\"needs_clarification\":false}", + "raw_content_sha256": "50dd30c16ff07995e873e614e9c42b88e61dd6fd45c0f360a0172c3453e458a3", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-21' and clarification False match the scoped source and execution: VISIT-2120 retains the stated preference 2027-04-21; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-20-hi-m1", + "case_id": "fresh-20-hi", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "language": "hi", + "parent_paired_file_sha256": "7c9a1a82b47c86c26d9dd958d36406740fc5785c423b4e7c57cebc997bdf3903", + "question": "VISIT-2120 के लिए अभी मेरी पसंद की तारीख कौन सी है? केवल बताएं; बुकिंग न बदलें।", + "source_messages": [ + { + "id": "note-20-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। VISIT-2120 के लिए मेरी पसंद 2027-04-21 है। VISIT-3120 के लिए पहले 2027-03-21 थी। सुधार केवल VISIT-3120 के लिए है: अब 2027-05-21 चाहिए। पहली बुकिंग की पसंद नहीं बदली है।" + } + ], + "source_basis_reviewed": "VISIT-2120 retains the stated preference 2027-04-21; the later correction applies only to the separately numbered booking.", + "question_target": "source_request_or_condition", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8227351c3db239a8", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2120" + }, + "response": { + "date": "2027-03-21", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-21", + "final_scoped_date": "2027-03-21", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_raw_action_status": "not_requested", + "action_status_basis": "read-only question and no requested booking change", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-20-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "dd43aa9aa779290fae1b1c7cb8073729bd58134df7d17f9e4f01e58ba38b7d74", + "result_file_sha256": "4c12d6eba0d948c095f2ff50ee04722cda4c2e1f13adf8f1fa2506d0e578ca43", + "receipt_file_sha256": "b0080397e2d734942856dc140ea81d2df8882f8a99072dd901a45e8fcb1d4f48", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-21\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "e7339b630822e30ebd8202e87691a83b1bf973c593fa23e61b283b809f6569f5", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-21' and clarification False match the scoped source and execution: VISIT-2120 retains the stated preference 2027-04-21; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-20-hi-m1-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "a575edcdab6aee1363bf4928ab4ad02a89980891a06f21a683f3f0e0df881518", + "result_file_sha256": "effa8674c24cb4042a07fce7a7b1f4aa6d0de67563c9878ee6c84d3eeb2fd49a", + "receipt_file_sha256": "d527606478f1ebbf1b371149bb69f7ccb672fe4de3a6b5c783c7bb808f2e04a2", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-03-21\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "ba6b5d05f8105ecabb90b4c6ac5b3ce2515fd4c68fa520e6a866c5f4079108ed", + "raw": { + "action_status": "already_applied", + "answer": "2027-03-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "answer_source_role": "confirmed_date_substituted_for_scoped_preference", + "factual_verdict": "incorrect", + "factual_explanation": "Reports '2027-03-21' rather than scoped date 2027-04-21: VISIT-2120 retains the stated preference 2027-04-21; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'already_applied' does not match expected 'not_requested': read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-20-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "777c2b66ffeef3957d19477043981810e6d5720c12e098bf865f870a12b587d2", + "result_file_sha256": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "receipt_file_sha256": "b4c46df4f2dae57d09b619c17703e67f834891f0c11f6286fc9eff30c4d2ca1a", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-21\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "efda99defce623b65285148dfd0b9814f4a3c673b764571d0d3827b37255a294", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "answer_source_role": "confirmed_date_substituted_for_scoped_preference", + "factual_verdict": "incorrect", + "factual_explanation": "Reports '2027-03-21' rather than scoped date 2027-04-21: VISIT-2120 retains the stated preference 2027-04-21; the later correction applies only to the separately numbered booking.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_requested' matches read-only question and no requested booking change.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_requested' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-21-hi-m0", + "case_id": "fresh-21-hi", + "model": "sarvam-105b", + "family": "approved_change", + "language": "hi", + "parent_paired_file_sha256": "89e93d838dc2ea1cb52f2d3d66e5315818028ca5e8cb46f21f74dff73e9e79c3", + "question": "VISIT-2121 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-21-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-21-1", + "role": "customer", + "content": "VISIT-2121 को 2027-04-22 पर बदल दें।" + }, + { + "id": "note-21-2", + "role": "customer", + "content": "दिए गए अनुरोध पहचान नंबर और सर्वर की संबंधित अनुमति का उपयोग करें। अनुरोध संभालने के बाद पक्की तारीख बताएं।" + } + ], + "source_basis_reviewed": "The question asks the currently confirmed date of VISIT-2121 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-22.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8e70502cc58aea6f", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2121", + "approval_id": "AUTH-21-GRANT-16", + "date": "2027-04-22", + "expected_version": 5, + "idempotency_key": "REQUEST-21-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-22", + "version": 6 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-22", + "final_scoped_date": "2027-04-22", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_raw_action_status": "confirmed", + "action_status_basis": "actual confirmed update_appointment receipt and matching scoped post-action state", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-21-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "3d5f223b764b53f7ed3ab833f725452803e5de5f1186f2681da552a224ca2b59", + "result_file_sha256": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "receipt_file_sha256": "1b07b66e333fc45179e31b668b1f8194bb42c6050714541fc2c05ab4c2cc8627", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-22\",\"needs_clarification\":false}", + "raw_content_sha256": "2b038dfe210662433dc98cbece33644925efcac1dc025402d4110813a6ba5ac4", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-22' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2121 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-22.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-21-hi-m0-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "345d22b64e98f9d605ec6a8fee112ae2cebbef74946293a64c874cb81f0bab55", + "result_file_sha256": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "receipt_file_sha256": "736a0543cbabfd30d3eb40ee091158cb328b6102b3d56279c20bc0cf8ce83fa8", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-22\",\"needs_clarification\":false}", + "raw_content_sha256": "2b038dfe210662433dc98cbece33644925efcac1dc025402d4110813a6ba5ac4", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-22' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2121 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-22.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-21-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "3d5f223b764b53f7ed3ab833f725452803e5de5f1186f2681da552a224ca2b59", + "result_file_sha256": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "receipt_file_sha256": "b6f5cb643b4e377e2056ac884543c2fb1a59c9d60ef24a32b9b4b432595bfa11", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-22\",\"needs_clarification\":false}", + "raw_content_sha256": "2b038dfe210662433dc98cbece33644925efcac1dc025402d4110813a6ba5ac4", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-22' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2121 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-22.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-21-hi-m1", + "case_id": "fresh-21-hi", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "language": "hi", + "parent_paired_file_sha256": "a19eb67c396409666d1324c77eda889b80dec2175a77bb444c723dd10b6cc7e8", + "question": "VISIT-2121 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-21-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-21-1", + "role": "customer", + "content": "VISIT-2121 को 2027-04-22 पर बदल दें।" + }, + { + "id": "note-21-2", + "role": "customer", + "content": "दिए गए अनुरोध पहचान नंबर और सर्वर की संबंधित अनुमति का उपयोग करें। अनुरोध संभालने के बाद पक्की तारीख बताएं।" + } + ], + "source_basis_reviewed": "The question asks the currently confirmed date of VISIT-2121 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-22.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-833398e754e00666", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2121" + }, + "response": { + "date": "2027-03-22", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-9c4d19fcc640c2f5", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2121", + "approval_id": "AUTH-21-GRANT-16", + "date": "2027-04-22", + "expected_version": 5, + "idempotency_key": "REQUEST-21-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-22", + "version": 6 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-22", + "final_scoped_date": "2027-04-22", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_raw_action_status": "confirmed", + "action_status_basis": "actual confirmed update_appointment receipt and matching scoped post-action state", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-21-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "6822e01c2ed46573624955c01fcb1b5c978bb70b913aab69cd2dbad3b9d4fb2f", + "result_file_sha256": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "receipt_file_sha256": "08a9d0c40fabc9254f294fe603d1c30f4930eb4638e0d17863b72f416800c1d3", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-22\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "9649b662bcdbbb140ad57a183801a8f220b3a023c361cbd568d24381e0f098e1", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-22' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2121 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-22.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-21-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "6822e01c2ed46573624955c01fcb1b5c978bb70b913aab69cd2dbad3b9d4fb2f", + "result_file_sha256": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "receipt_file_sha256": "f8365223d0702010f2c7f9b5c3dbc5087b8b100004c7233202904e8735c057f4", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-22\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "9649b662bcdbbb140ad57a183801a8f220b3a023c361cbd568d24381e0f098e1", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-22' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2121 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-22.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-21-hi-m1-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "31b3bfdc230b9b6c66a01d1fbc1bc1455623152f24a63943c909100e8a5374a7", + "result_file_sha256": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "receipt_file_sha256": "e01755932b5dc9b4326b5f1b63b33f9023372aa9656df3a861ad266258b17717", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-22\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "9649b662bcdbbb140ad57a183801a8f220b3a023c361cbd568d24381e0f098e1", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-22' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2121 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-22.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-22-hi-m0", + "case_id": "fresh-22-hi", + "model": "sarvam-105b", + "family": "approved_change", + "language": "hi", + "parent_paired_file_sha256": "29ae8ee3730cbd511f4161249514a2ccd67927fa0fba2b75775ed8a15d318203", + "question": "VISIT-2122 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-22-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। VISIT-2122 को 2027-04-23 पर बदल दें। दिए गए अनुरोध पहचान नंबर और सर्वर की संबंधित अनुमति का उपयोग करें। अनुरोध संभालने के बाद पक्की तारीख बताएं।" + } + ], + "source_basis_reviewed": "The question asks the currently confirmed date of VISIT-2122 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-23.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-bcffe2acef3a1687", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2122", + "approval_id": "AUTH-22-GRANT-16", + "date": "2027-04-23", + "expected_version": 5, + "idempotency_key": "REQUEST-22-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-23", + "version": 6 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-23", + "final_scoped_date": "2027-04-23", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_raw_action_status": "confirmed", + "action_status_basis": "actual confirmed update_appointment receipt and matching scoped post-action state", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-22-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "eb4a83d885ad65342f651937467d38e7cbfe99a35751d6321194bcea52b91f8d", + "result_file_sha256": "7a6034546c590ec5064db6d004a77a9dcf0a4f8eab5062272644c6750c85fef7", + "receipt_file_sha256": "2b78ff04493de4ddc891058311b3168c81790bca95161a695e46ea80b7708128", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-23\",\"needs_clarification\":false}", + "raw_content_sha256": "412d38ba18a68b8292a1112c6a6da160880d9f3ab5b1ae562dc88ac8bf6b7a42", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-23' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2122 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-23.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-22-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "eb4a83d885ad65342f651937467d38e7cbfe99a35751d6321194bcea52b91f8d", + "result_file_sha256": "e36a878b5207e4b081d01bf02b4f77a972202397cf8de8873b33fb9c111442af", + "receipt_file_sha256": "884939c5a1c235ce1498dd3e99bbbda7a3c401da2c1647ffe3058b8d7df287ce", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\": \"confirmed\", \"answer\": \"2027-04-23\", \"needs_clarification\": false}", + "raw_content_sha256": "e8861ec62f85cd316c2be69c81cbad13e3694775745b18be4b866212ab2b7b1b", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-23' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2122 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-23.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-22-hi-m0-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "a38460f78be502049c91ca8cf7842ff223f13928035dce113d083353d33d5c5b", + "result_file_sha256": "7a6034546c590ec5064db6d004a77a9dcf0a4f8eab5062272644c6750c85fef7", + "receipt_file_sha256": "0ca0b9e7d06ecc61faaae70876b6b65b90124e390fb729f22ae62d50ae19efce", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-23\",\"needs_clarification\":false}", + "raw_content_sha256": "412d38ba18a68b8292a1112c6a6da160880d9f3ab5b1ae562dc88ac8bf6b7a42", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-23' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2122 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-23.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-22-hi-m1", + "case_id": "fresh-22-hi", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "language": "hi", + "parent_paired_file_sha256": "1f4471c284b280fd69c6d301ba10f8dbc42b23a193e4a067bc834294799bece8", + "question": "VISIT-2122 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-22-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। VISIT-2122 को 2027-04-23 पर बदल दें। दिए गए अनुरोध पहचान नंबर और सर्वर की संबंधित अनुमति का उपयोग करें। अनुरोध संभालने के बाद पक्की तारीख बताएं।" + } + ], + "source_basis_reviewed": "The question asks the currently confirmed date of VISIT-2122 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-23.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-944ac25d42d11674", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2122" + }, + "response": { + "date": "2027-03-23", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-bf6073d27c48e5f5", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2122", + "approval_id": "AUTH-22-GRANT-16", + "date": "2027-04-23", + "expected_version": 5, + "idempotency_key": "REQUEST-22-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-23", + "version": 6 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-23", + "final_scoped_date": "2027-04-23", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_raw_action_status": "confirmed", + "action_status_basis": "actual confirmed update_appointment receipt and matching scoped post-action state", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "chronological": { + "job_id": "confirm-fresh-22-hi-m1-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "0404caed02cfee7ef39c65f74439e62d897fa3acb25074d95b2796b093c20e83", + "result_file_sha256": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "receipt_file_sha256": "51732486eba1cd2320567c71f1dab3ea97848a8f64745cb7abfccbe6b4eb6287", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-23\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "3a77159e81f930d16df465bba0c47780df20a0d96e1d28d3144d74065e93efb7", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-23' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2122 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-23.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-22-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "92368404fc16069c5322893e6e14f86684f6eef8e21ce6bdb4c3f77053bd830f", + "result_file_sha256": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "receipt_file_sha256": "58730bc0987402a6c8184ae9f9de391a8f4321a38c6271ea8f8c48aa780e5c1e", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-23\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "3a77159e81f930d16df465bba0c47780df20a0d96e1d28d3144d74065e93efb7", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-23' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2122 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-23.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-22-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "92368404fc16069c5322893e6e14f86684f6eef8e21ce6bdb4c3f77053bd830f", + "result_file_sha256": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "receipt_file_sha256": "13bd87e1413916a761fb111dae5bd0129de642199409098c8004d06d7310a38c", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-23\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "3a77159e81f930d16df465bba0c47780df20a0d96e1d28d3144d74065e93efb7", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-23' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2122 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-23.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-23-hi-m0", + "case_id": "fresh-23-hi", + "model": "sarvam-105b", + "family": "approved_change", + "language": "hi", + "parent_paired_file_sha256": "c70aac130770eb52c4d22cb14a12c328b138afed88f6754b81d40ba4a2ab1bcd", + "question": "VISIT-2123 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-23-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-23-1", + "role": "customer", + "content": "VISIT-2123 को 2027-04-24 पर बदल दें।" + }, + { + "id": "note-23-2", + "role": "customer", + "content": "दिए गए अनुरोध पहचान नंबर और सर्वर की संबंधित अनुमति का उपयोग करें। अनुरोध संभालने के बाद पक्की तारीख बताएं।" + } + ], + "source_basis_reviewed": "The question asks the currently confirmed date of VISIT-2123 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-24.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-ad974d7111708d90", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2123", + "approval_id": "AUTH-23-GRANT-16", + "date": "2027-04-24", + "expected_version": 5, + "idempotency_key": "REQUEST-23-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-24", + "version": 6 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-24", + "final_scoped_date": "2027-04-24", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_raw_action_status": "confirmed", + "action_status_basis": "actual confirmed update_appointment receipt and matching scoped post-action state", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "chronological": { + "job_id": "confirm-fresh-23-hi-m0-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "41f7c2890b3575602d22a1340a95c7ba0f2b70623b86513553ea73b6d49adcd3", + "result_file_sha256": "0faf0e0192f0d7377eabd553b64abae59ded0e43cc4773b8a88de2b6c1adaeed", + "receipt_file_sha256": "75a8518a685ddce3f77db86d2dac0885cc6fcf5ca7354525d81249d0636977c3", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-24\",\"needs_clarification\":false}", + "raw_content_sha256": "42e02e70a4fb69a05a1d57eab32d4ec5ccae0fd8c745b2c7d0f29e7d8ef48bb6", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-24' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2123 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-24.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-23-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "0453c4341c0ac576befb3353fa1bd66444b1d16b15d142d0f7c022ffa42a21db", + "result_file_sha256": "0faf0e0192f0d7377eabd553b64abae59ded0e43cc4773b8a88de2b6c1adaeed", + "receipt_file_sha256": "0cd9e6049b78230915a513bd30dd11098ebcb86660e51100d88b76722a3add80", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-24\",\"needs_clarification\":false}", + "raw_content_sha256": "42e02e70a4fb69a05a1d57eab32d4ec5ccae0fd8c745b2c7d0f29e7d8ef48bb6", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-24' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2123 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-24.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-23-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "0453c4341c0ac576befb3353fa1bd66444b1d16b15d142d0f7c022ffa42a21db", + "result_file_sha256": "0faf0e0192f0d7377eabd553b64abae59ded0e43cc4773b8a88de2b6c1adaeed", + "receipt_file_sha256": "1a1cb6a1c68912b8981cfad5f2aa65c782c26392662e8b89e585b7bef4e9c784", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-24\",\"needs_clarification\":false}", + "raw_content_sha256": "42e02e70a4fb69a05a1d57eab32d4ec5ccae0fd8c745b2c7d0f29e7d8ef48bb6", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-24' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2123 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-24.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-23-hi-m1", + "case_id": "fresh-23-hi", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "language": "hi", + "parent_paired_file_sha256": "fa867b061e3c10ac35a811a61c3032f4333b8e021fad26a0dfd7a21b4f37e9f8", + "question": "VISIT-2123 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-23-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-23-1", + "role": "customer", + "content": "VISIT-2123 को 2027-04-24 पर बदल दें।" + }, + { + "id": "note-23-2", + "role": "customer", + "content": "दिए गए अनुरोध पहचान नंबर और सर्वर की संबंधित अनुमति का उपयोग करें। अनुरोध संभालने के बाद पक्की तारीख बताएं।" + } + ], + "source_basis_reviewed": "The question asks the currently confirmed date of VISIT-2123 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-24.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8000d4543f5e97ff", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2123" + }, + "response": { + "date": "2027-03-24", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-997faecc0ef4f2a2", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2123", + "approval_id": "AUTH-23-GRANT-16", + "date": "2027-04-24", + "expected_version": 5, + "idempotency_key": "REQUEST-23-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-24", + "version": 6 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-24", + "final_scoped_date": "2027-04-24", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_raw_action_status": "confirmed", + "action_status_basis": "actual confirmed update_appointment receipt and matching scoped post-action state", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "chronological": { + "job_id": "confirm-fresh-23-hi-m1-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "e5374075e463e0249cc2e5acb0f0a0d22a815b4657721dea2901898e6b0ade0a", + "result_file_sha256": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "receipt_file_sha256": "8c62ddec4606fb63984b73a2e3ed35ada833c113b3cd8d01c5df3987bbbee778", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-24\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "2693a0032cdec8d9e8a477c15467e101f2a566f73bf63c4cb05aff66d210cfdc", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-24' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2123 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-24.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-23-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "ca6b7ccc86b69bef267b737f6b12d0409f8faf26d29d996b0f8aebf8cae396bd", + "result_file_sha256": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "receipt_file_sha256": "b7b30eef2f09ec867d15b4eb5fd715bba423b677d6b702ff78be9a2572f3b963", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-24\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "2693a0032cdec8d9e8a477c15467e101f2a566f73bf63c4cb05aff66d210cfdc", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-24' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2123 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-24.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-23-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "ca6b7ccc86b69bef267b737f6b12d0409f8faf26d29d996b0f8aebf8cae396bd", + "result_file_sha256": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "receipt_file_sha256": "893493c64295854bd6dfe0366663032e87c98de92b7ada64f0388f3826d02c5c", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-24\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "2693a0032cdec8d9e8a477c15467e101f2a566f73bf63c4cb05aff66d210cfdc", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-24' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2123 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-24.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-24-hi-m0", + "case_id": "fresh-24-hi", + "model": "sarvam-105b", + "family": "approved_change", + "language": "hi", + "parent_paired_file_sha256": "57b4f614ebe26faf73591eeeee59160526776cfc0da3824b5bd90b007d8196e4", + "question": "VISIT-2124 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-24-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। VISIT-2124 को 2027-04-25 पर बदल दें। दिए गए अनुरोध पहचान नंबर और सर्वर की संबंधित अनुमति का उपयोग करें। अनुरोध संभालने के बाद पक्की तारीख बताएं।" + } + ], + "source_basis_reviewed": "The question asks the currently confirmed date of VISIT-2124 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-25.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8193998625a98119", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2124", + "approval_id": "AUTH-24-GRANT-16", + "date": "2027-04-25", + "expected_version": 5, + "idempotency_key": "REQUEST-24-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-25", + "version": 6 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-25", + "final_scoped_date": "2027-04-25", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_raw_action_status": "confirmed", + "action_status_basis": "actual confirmed update_appointment receipt and matching scoped post-action state", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "chronological": { + "job_id": "confirm-fresh-24-hi-m0-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "faa7161d0358c803ed503171c760ceaed5a7f3fa94313900eeb242e52a7d7495", + "result_file_sha256": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "receipt_file_sha256": "80d056b24310696311ca258464452070eca0e21145068b76ebbc4a294703c4b3", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-25\",\"needs_clarification\":false}", + "raw_content_sha256": "5f7884de70bae39ff454edda1e2dfc29c17845b488e477350cf0a7cc27c93739", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-25' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2124 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-25.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-24-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "c50b4dceecbe098e06259026952ab266685714c346b9a9a81b3c1d881b9ef490", + "result_file_sha256": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "receipt_file_sha256": "e616716377e81f553ea58e68d5b007260d37779c31045e4ab5a6059584b2777a", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-25\",\"needs_clarification\":false}", + "raw_content_sha256": "5f7884de70bae39ff454edda1e2dfc29c17845b488e477350cf0a7cc27c93739", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-25' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2124 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-25.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-24-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "c50b4dceecbe098e06259026952ab266685714c346b9a9a81b3c1d881b9ef490", + "result_file_sha256": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "receipt_file_sha256": "7c15d4b4072285ce824169e3517cb7de6e8fcaa52bbf2a4836d7400ea43fc554", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-25\",\"needs_clarification\":false}", + "raw_content_sha256": "5f7884de70bae39ff454edda1e2dfc29c17845b488e477350cf0a7cc27c93739", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-25' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2124 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-25.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-24-hi-m1", + "case_id": "fresh-24-hi", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "language": "hi", + "parent_paired_file_sha256": "02ed3dda4b169680d1775e51c7ba1f67a56747a00b78452414bb58c038b87683", + "question": "VISIT-2124 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-24-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। VISIT-2124 को 2027-04-25 पर बदल दें। दिए गए अनुरोध पहचान नंबर और सर्वर की संबंधित अनुमति का उपयोग करें। अनुरोध संभालने के बाद पक्की तारीख बताएं।" + } + ], + "source_basis_reviewed": "The question asks the currently confirmed date of VISIT-2124 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-25.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-ac2337fe2553ceda", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2124" + }, + "response": { + "date": "2027-03-25", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-b82f3464fdf2ffca", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2124", + "approval_id": "AUTH-24-GRANT-16", + "date": "2027-04-25", + "expected_version": 5, + "idempotency_key": "REQUEST-24-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-25", + "version": 6 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-25", + "final_scoped_date": "2027-04-25", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_raw_action_status": "confirmed", + "action_status_basis": "actual confirmed update_appointment receipt and matching scoped post-action state", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-24-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "e635128747ab37a3cc7bedeee1589ee23aaa106e61c4373ea6685e48d1c36381", + "result_file_sha256": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "receipt_file_sha256": "c4b4df18b4efff8df8f6b2dcd93c4242a7b5b968612a20e0e66cf9f71233a4ef", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-25\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "4913fc937b2d32f3dd8205703eb9608e0029f25eba1c8e31072557a2baa36b41", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-25' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2124 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-25.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-24-hi-m1-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "f2f05298c8516626fba95e1c988d5603c2c9f38cdbbe7af3dd0fc33569b3eabe", + "result_file_sha256": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "receipt_file_sha256": "89be6a4c0afa8ff9fe6a9bdf55314c01fda87ba57c14c4aab5afa9d336056af3", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-25\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "4913fc937b2d32f3dd8205703eb9608e0029f25eba1c8e31072557a2baa36b41", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-25' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2124 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-25.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-24-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "e635128747ab37a3cc7bedeee1589ee23aaa106e61c4373ea6685e48d1c36381", + "result_file_sha256": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "receipt_file_sha256": "7b5b239d5814db549697fb4c1722c58ed0d7fd2a787338023cef46133e9dbcf6", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-25\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "4913fc937b2d32f3dd8205703eb9608e0029f25eba1c8e31072557a2baa36b41", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-25' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2124 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-25.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-25-hi-m0", + "case_id": "fresh-25-hi", + "model": "sarvam-105b", + "family": "approval_boundary", + "language": "hi", + "parent_paired_file_sha256": "5bdbcfe0efa64c426004c0ee78c628b020073bae373405c0252781fd7526260a", + "question": "VISIT-2125 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-25-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-25-1", + "role": "customer", + "content": "VISIT-2125 को 2027-04-26 पर बदलने का अनुरोध संभालें।" + }, + { + "id": "note-25-2", + "role": "customer", + "content": "अनुमति की वैधता दिए गए सर्वर समय पर जांचें। वह उस निश्चित समय के एक सेकंड बाद समाप्त होती है।" + } + ], + "source_basis_reviewed": "The question asks the currently confirmed date of VISIT-2125 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-26.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8d4a2da3a3b7bfa9", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2125", + "approval_id": "AUTH-25-GRANT-19", + "date": "2027-04-26", + "expected_version": 5, + "idempotency_key": "REQUEST-25-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-26", + "version": 6 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-26", + "final_scoped_date": "2027-04-26", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_raw_action_status": "confirmed", + "action_status_basis": "actual confirmed update_appointment receipt and matching scoped post-action state", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-25-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "fd42cfe4e71d6b2fb6b9852741bf7a9c3416789c10817eb430abe6d649e9c66c", + "result_file_sha256": "258b73b129b801b662376d5cb0f39961843f2127df41664b4c2a54a8e81537fa", + "receipt_file_sha256": "73e3c8bc500b9c94180bcb8a5a2c275b60185270bc71fec954f34ffce575f148", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-26\",\"needs_clarification\":false}", + "raw_content_sha256": "bd59ff2841c464354faf7faa7146bbbf8c2dd82eef7abb3e9321e9a86bc87da6", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-26' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2125 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-26.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-25-hi-m0-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "d6506b0477dc0d98379472ff9e708559ce863b041306a25cfb64c5e96d434aa0", + "result_file_sha256": "258b73b129b801b662376d5cb0f39961843f2127df41664b4c2a54a8e81537fa", + "receipt_file_sha256": "6f84f337c5f6ebb2463b869d93f8e147a38072da3d1a130e0236fa1c3027808c", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-26\",\"needs_clarification\":false}", + "raw_content_sha256": "bd59ff2841c464354faf7faa7146bbbf8c2dd82eef7abb3e9321e9a86bc87da6", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-26' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2125 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-26.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-25-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "fd42cfe4e71d6b2fb6b9852741bf7a9c3416789c10817eb430abe6d649e9c66c", + "result_file_sha256": "258b73b129b801b662376d5cb0f39961843f2127df41664b4c2a54a8e81537fa", + "receipt_file_sha256": "4c6cba77860d822b3c95c76321ed893e0448d3c1803f6e8f01b7305314fe08db", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-26\",\"needs_clarification\":false}", + "raw_content_sha256": "bd59ff2841c464354faf7faa7146bbbf8c2dd82eef7abb3e9321e9a86bc87da6", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-26' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2125 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-26.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-25-hi-m1", + "case_id": "fresh-25-hi", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "language": "hi", + "parent_paired_file_sha256": "0b140d2c6d070b4c7bcda3049e1f830abd399f7b54e762017631330a769c15ad", + "question": "VISIT-2125 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-25-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-25-1", + "role": "customer", + "content": "VISIT-2125 को 2027-04-26 पर बदलने का अनुरोध संभालें।" + }, + { + "id": "note-25-2", + "role": "customer", + "content": "अनुमति की वैधता दिए गए सर्वर समय पर जांचें। वह उस निश्चित समय के एक सेकंड बाद समाप्त होती है।" + } + ], + "source_basis_reviewed": "The question asks the currently confirmed date of VISIT-2125 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-26.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8b655eaadb30f480", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2125" + }, + "response": { + "date": "2027-03-26", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-aaaa60951ed0a2a6", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2125", + "approval_id": "AUTH-25-GRANT-19", + "date": "2027-04-26", + "expected_version": 5, + "idempotency_key": "REQUEST-25-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-26", + "version": 6 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-26", + "final_scoped_date": "2027-04-26", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_raw_action_status": "confirmed", + "action_status_basis": "actual confirmed update_appointment receipt and matching scoped post-action state", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-25-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "324031537d52cda00b01d70e226e52e3815e0aa49ababed40cd3753371f5bcf4", + "result_file_sha256": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "receipt_file_sha256": "d0ce4103463aa8f513567efabaec7dbf99158cd020308c5f550ad2db3b95ff09", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-26\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "b67eb49d345d551277e52d9817ebc266663e8242554bf45e8cec67eff31a99b9", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-26' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2125 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-26.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-25-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "324031537d52cda00b01d70e226e52e3815e0aa49ababed40cd3753371f5bcf4", + "result_file_sha256": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "receipt_file_sha256": "687532bfb8edeb9ec082ddfa7398ae8e72dadad5eef766090ea76cfc0100244e", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-26\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "b67eb49d345d551277e52d9817ebc266663e8242554bf45e8cec67eff31a99b9", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-26' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2125 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-26.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-25-hi-m1-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "d81157f1453aa209e3156b06351c0fe943d55b982c799f44f346d91b5faab4d8", + "result_file_sha256": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "receipt_file_sha256": "504f80ff8c310f7fd6ec3c0115f05ae30fc32fabd88e3263915f1812b65089bc", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-26\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "b67eb49d345d551277e52d9817ebc266663e8242554bf45e8cec67eff31a99b9", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-26' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2125 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-26.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-26-hi-m0", + "case_id": "fresh-26-hi", + "model": "sarvam-105b", + "family": "approval_boundary", + "language": "hi", + "parent_paired_file_sha256": "c9c7d974f984ab842b03118877662b9e3bb49afcfd8fd50488c2ada7b82ec253", + "question": "VISIT-2126 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-26-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। VISIT-2126 को 2027-04-27 पर बदलने का अनुरोध संभालें। अनुमति की वैधता दिए गए सर्वर समय पर जांचें। वह उस निश्चित समय के एक सेकंड बाद समाप्त होती है।" + } + ], + "source_basis_reviewed": "The question asks the currently confirmed date of VISIT-2126 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-27.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-950b9b6d6816471f", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2126", + "approval_id": "AUTH-26-GRANT-19", + "date": "2027-04-27", + "expected_version": 5, + "idempotency_key": "REQUEST-26-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-27", + "version": 6 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-27", + "final_scoped_date": "2027-04-27", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_raw_action_status": "confirmed", + "action_status_basis": "actual confirmed update_appointment receipt and matching scoped post-action state", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-26-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "b42bf6a000aaa5760a48a19b97344ba52c5ca29a66ace3f8cc6ac818a5e7d33c", + "result_file_sha256": "fb6e951f5e2bc3a3ec400cf4d93a3dfb2910e24d4ec7d9480d665ad63036999a", + "receipt_file_sha256": "69c4f2f1392f58c34234b05fbdc7c8fa97ffb8cf2fe4898e642bb583057d64fe", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-27\",\"needs_clarification\":false}", + "raw_content_sha256": "c1e910975c3325f664d8f29ff0ec33c8158dfa51c312012dff2adc0d4cc8071a", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-27' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2126 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-27.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-26-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "b42bf6a000aaa5760a48a19b97344ba52c5ca29a66ace3f8cc6ac818a5e7d33c", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "receipt_file_sha256": "c2ee3a22ad9c26458a929997b147adb253ddad4a5682800e7eb1c6af3144e469", + "terminal": "incomplete", + "finish_reason": "length", + "raw_content": null, + "raw_content_sha256": null, + "raw": null, + "application_output": null, + "answer_source_role": null, + "factual_verdict": "invalid_or_missing", + "factual_explanation": "No valid parsed final answer to assess.", + "raw_status_verdict": "invalid_or_missing", + "raw_status_explanation": "None does not match expected 'confirmed': actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "invalid_or_missing", + "projected_status_explanation": "Application status None does not match expected 'confirmed' from actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-26-hi-m0-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "03601d8c369d992cd41f6e31962126c6f85a4cdb9e09810458211014ab7de7df", + "result_file_sha256": "fb6e951f5e2bc3a3ec400cf4d93a3dfb2910e24d4ec7d9480d665ad63036999a", + "receipt_file_sha256": "17fe31badc57c53f92d820e60dd96c9c85eeab69cd30a5d6941d8d3166e6f286", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-27\",\"needs_clarification\":false}", + "raw_content_sha256": "c1e910975c3325f664d8f29ff0ec33c8158dfa51c312012dff2adc0d4cc8071a", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-27' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2126 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-27.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-26-hi-m1", + "case_id": "fresh-26-hi", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "language": "hi", + "parent_paired_file_sha256": "c9d3c2b37e00f2a6d24c5560f7074647029327b0667f6e3b156cd680d2b97280", + "question": "VISIT-2126 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-26-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। VISIT-2126 को 2027-04-27 पर बदलने का अनुरोध संभालें। अनुमति की वैधता दिए गए सर्वर समय पर जांचें। वह उस निश्चित समय के एक सेकंड बाद समाप्त होती है।" + } + ], + "source_basis_reviewed": "The question asks the currently confirmed date of VISIT-2126 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-27.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-ac1a2470b7bd359d", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2126" + }, + "response": { + "date": "2027-03-27", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-a1e7061c283aee59", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2126", + "approval_id": "AUTH-26-GRANT-19", + "date": "2027-04-27", + "expected_version": 5, + "idempotency_key": "REQUEST-26-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-27", + "version": 6 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-27", + "final_scoped_date": "2027-04-27", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_raw_action_status": "confirmed", + "action_status_basis": "actual confirmed update_appointment receipt and matching scoped post-action state", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-26-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "8401073644c063baf99ee599f77c1781cb2372a65124e07460b01b2366d10b01", + "result_file_sha256": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "receipt_file_sha256": "8428554bb9cc0a2f8c4c8585b85d97175ff51e0271c39ba42b98b9dc17c1b178", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-27\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "fa5d023d08c3ca91b18204d2fb9daa32d70361a98f2d49b310618c10718bd925", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-27' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2126 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-27.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-26-hi-m1-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "9c88595d1e3f07147661fda5524c64be21ae9f70354bb8d07db24a9250a9e244", + "result_file_sha256": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "receipt_file_sha256": "88bc16feda1d0006497e7ab6a21b6aad9d7273276134d95b62fb1bf177077850", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-27\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "fa5d023d08c3ca91b18204d2fb9daa32d70361a98f2d49b310618c10718bd925", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-27' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2126 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-27.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-26-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "8401073644c063baf99ee599f77c1781cb2372a65124e07460b01b2366d10b01", + "result_file_sha256": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "receipt_file_sha256": "3ad351cca3215592d5f9a528d2ffb01aa1f9a6217884237e48531e2a6b9632a1", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-27\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "fa5d023d08c3ca91b18204d2fb9daa32d70361a98f2d49b310618c10718bd925", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-27' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2126 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-27.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-27-hi-m0", + "case_id": "fresh-27-hi", + "model": "sarvam-105b", + "family": "approval_boundary", + "language": "hi", + "parent_paired_file_sha256": "fd90c926faf364acdce1e083706f3415ba34154a6384343ded9bc4374cec90e9", + "question": "VISIT-2127 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-27-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-27-1", + "role": "customer", + "content": "VISIT-2127 को 2027-04-28 पर बदलने का अनुरोध संभालें।" + }, + { + "id": "note-27-2", + "role": "customer", + "content": "अनुमति की वैधता दिए गए सर्वर समय पर जांचें। वह उस निश्चित समय के एक सेकंड बाद समाप्त होती है।" + } + ], + "source_basis_reviewed": "The question asks the currently confirmed date of VISIT-2127 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-28.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-960cd75ce9a68d7f", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2127" + }, + "response": { + "date": "2027-03-28", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-af2343a678f155c9", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2127", + "approval_id": "AUTH-27-GRANT-19", + "date": "2027-04-28", + "expected_version": 5, + "idempotency_key": "REQUEST-27-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-28", + "version": 6 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-28", + "final_scoped_date": "2027-04-28", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_raw_action_status": "confirmed", + "action_status_basis": "actual confirmed update_appointment receipt and matching scoped post-action state", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-27-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "ceb85fb6a4d3c08067972b300c58e33bd2efe9ae4416c97f79eff7ba38641e15", + "result_file_sha256": "58dc4f555443172d26580e97135531691461045bdffc7d2a8802fe869294b394", + "receipt_file_sha256": "b40326cb20219defa7d08060c080b0ef56364b3310e6bacee95186cf82397d9b", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-28\",\"needs_clarification\":false}", + "raw_content_sha256": "9697ae0b043edc7459e3f0e7b3cf26431b534d4c4235ab6792946232f68e8762", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-28' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2127 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-28.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-27-hi-m0-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "417746a4619153cb8b73821e368d5654bcc5ad0597619244f6ebcd5be28f0e67", + "result_file_sha256": "58dc4f555443172d26580e97135531691461045bdffc7d2a8802fe869294b394", + "receipt_file_sha256": "f4c682d84ba533831ef1a55171a1b40c856793e613925a844feb82bb30d08e5e", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-28\",\"needs_clarification\":false}", + "raw_content_sha256": "9697ae0b043edc7459e3f0e7b3cf26431b534d4c4235ab6792946232f68e8762", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-28' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2127 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-28.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-27-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "0a733c6f798ee042690fdaf870be68ea1d8a080c4aabd36fb0176e77105ffdb3", + "result_file_sha256": "58dc4f555443172d26580e97135531691461045bdffc7d2a8802fe869294b394", + "receipt_file_sha256": "60ff9e7052d0d942d9ae472e3fb5dd67bc1cdc97cb210965f94ed04800c8f556", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-28\",\"needs_clarification\":false}", + "raw_content_sha256": "9697ae0b043edc7459e3f0e7b3cf26431b534d4c4235ab6792946232f68e8762", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-28' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2127 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-28.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-27-hi-m1", + "case_id": "fresh-27-hi", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "language": "hi", + "parent_paired_file_sha256": "d3e90d6ee95a0b038718615f0bf2987635dc5009253cdcd2c743ff50ae8ff0e7", + "question": "VISIT-2127 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-27-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-27-1", + "role": "customer", + "content": "VISIT-2127 को 2027-04-28 पर बदलने का अनुरोध संभालें।" + }, + { + "id": "note-27-2", + "role": "customer", + "content": "अनुमति की वैधता दिए गए सर्वर समय पर जांचें। वह उस निश्चित समय के एक सेकंड बाद समाप्त होती है।" + } + ], + "source_basis_reviewed": "The question asks the currently confirmed date of VISIT-2127 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-28.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a46a21385ae5a2fb", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2127" + }, + "response": { + "date": "2027-03-28", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-ab3c7d205ce61612", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2127", + "approval_id": "AUTH-27-GRANT-19", + "date": "2027-04-28", + "expected_version": 5, + "idempotency_key": "REQUEST-27-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-28", + "version": 6 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-28", + "final_scoped_date": "2027-04-28", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_raw_action_status": "confirmed", + "action_status_basis": "actual confirmed update_appointment receipt and matching scoped post-action state", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-27-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "5e320542eaba550cb695b24d686e27a6b3c2bfc0e95f935d6b83d816c61cf216", + "result_file_sha256": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "receipt_file_sha256": "8b1f111f50e6ce251322532975e224858b3c5491d7f6627f198422f7b96c443a", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-28\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "8608e83321d9a3eac9053988d3073197a955bcce5974202ee13444130cb83da2", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-28' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2127 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-28.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-27-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "5e320542eaba550cb695b24d686e27a6b3c2bfc0e95f935d6b83d816c61cf216", + "result_file_sha256": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "receipt_file_sha256": "59274607fc488982d5c0f3b726731400fa54e30e805f0121ef51846d04e59180", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-28\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "8608e83321d9a3eac9053988d3073197a955bcce5974202ee13444130cb83da2", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-28' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2127 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-28.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-27-hi-m1-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "f474f9bf78b3353b4d828a1f539a31767f22c150a1b0a321ac33bd5f1a10c8e9", + "result_file_sha256": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "receipt_file_sha256": "6b6ef2b04ca9c52fd97a776826d1b5889b4aa611e34ac9d258762b0256e6b258", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-28\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "8608e83321d9a3eac9053988d3073197a955bcce5974202ee13444130cb83da2", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-28' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2127 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-28.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-28-hi-m0", + "case_id": "fresh-28-hi", + "model": "sarvam-105b", + "family": "approval_boundary", + "language": "hi", + "parent_paired_file_sha256": "1b1d2dd2e8bbe75adef9964cd0f15fe86f8597cf4f6bb81b85f491dbb926e42c", + "question": "VISIT-2128 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-28-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। VISIT-2128 को 2027-04-29 पर बदलने का अनुरोध संभालें। अनुमति की वैधता दिए गए सर्वर समय पर जांचें। वह उस निश्चित समय के एक सेकंड बाद समाप्त होती है।" + } + ], + "source_basis_reviewed": "The question asks the currently confirmed date of VISIT-2128 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-29.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9268ee648a315815", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2128" + }, + "response": { + "date": "2027-03-29", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-a47da9be9474bb8c", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2128", + "approval_id": "AUTH-28-GRANT-19", + "date": "2027-04-29", + "expected_version": 5, + "idempotency_key": "REQUEST-28-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-29", + "version": 6 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-29", + "final_scoped_date": "2027-04-29", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_raw_action_status": "confirmed", + "action_status_basis": "actual confirmed update_appointment receipt and matching scoped post-action state", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-28-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "8445e426fa16a6727eefb4affb86ba938030abe42afa642ddaa39fd27797a64a", + "result_file_sha256": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "receipt_file_sha256": "120efe797b58fb2895dec5935ae26872e25d0f81c6ceabceede81462650a5863", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-29\",\"needs_clarification\":false}", + "raw_content_sha256": "d8820bdeab4784bd4b5a0ce76c7cf37112f3a832e5aa7de53fe26a44cbd97a5d", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-29' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2128 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-29.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-28-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "225c96b6b9a067c696f779f70f724a7bde7fc7ff58f81732426fa21fcd83fe58", + "result_file_sha256": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "receipt_file_sha256": "9db23ae07cc308902ce43384589bd373efaeedab0b5c623cf64907485d25e95b", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-29\",\"needs_clarification\":false}", + "raw_content_sha256": "d8820bdeab4784bd4b5a0ce76c7cf37112f3a832e5aa7de53fe26a44cbd97a5d", + "raw": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-29' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2128 after handling the request. Actual confirmed update_appointment receipt and matching scoped post-action state; the scoped post-action date is 2027-04-29.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'confirmed' matches actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'confirmed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-28-hi-m0-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "f69a643c32efa7e2cfdf9a2b9db74bb8a32a09852a84b6811528b5ef28082468", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "receipt_file_sha256": "5ba23cbcac0df7c7048071ca03ce18feb7a5de130448c8986b6683a2b0a5aebe", + "terminal": "incomplete", + "finish_reason": "length", + "raw_content": null, + "raw_content_sha256": null, + "raw": null, + "application_output": null, + "answer_source_role": null, + "factual_verdict": "invalid_or_missing", + "factual_explanation": "No valid parsed final answer to assess.", + "raw_status_verdict": "invalid_or_missing", + "raw_status_explanation": "None does not match expected 'confirmed': actual confirmed update_appointment receipt and matching scoped post-action state.", + "projected_status_verdict": "invalid_or_missing", + "projected_status_explanation": "Application status None does not match expected 'confirmed' from actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-28-hi-m1", + "case_id": "fresh-28-hi", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "language": "hi", + "parent_paired_file_sha256": "4b4b1a57d7c98bc318c303a4df4477f984dc7bf1b596ecc7a063369af070b5fd", + "question": "VISIT-2128 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-28-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। VISIT-2128 को 2027-04-29 पर बदलने का अनुरोध संभालें। अनुमति की वैधता दिए गए सर्वर समय पर जांचें। वह उस निश्चित समय के एक सेकंड बाद समाप्त होती है।" + } + ], + "source_basis_reviewed": "The question asks the currently confirmed date of VISIT-2128 after handling the request. No confirmed update receipt; scoped post-action state remains unchanged; the scoped post-action date is 2027-03-29.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-ab2b8013429667ad", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2128" + }, + "response": { + "date": "2027-03-29", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-29", + "final_scoped_date": "2027-03-29", + "expected_from_source_and_actual_execution": { + "answer": "2027-03-29", + "needs_clarification": false + }, + "expected_raw_action_status": "not_performed", + "action_status_basis": "no confirmed update receipt; scoped post-action state remains unchanged", + "gold_ideal_answer_differs_from_actual_execution": true, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-28-hi-m1-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "a7fbecee0a37bef027766cd623307002204afe7fc7b1ba696ec2a7358f32c119", + "result_file_sha256": "9233e86d09af1cd05d13e7ec48d0afe392a6ff5d871d26feaafc1cb5c237e5eb", + "receipt_file_sha256": "65a02c802fde287d11cc796b5ceb6c21167485288d1bd50258ed3364d60af73c", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-03-29\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "16f3e3d844d05284f800bee069891c80e8caac0ba79cf6402ea600abb8f834d8", + "raw": { + "action_status": "not_performed", + "answer": "2027-03-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-29", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-03-29' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2128 after handling the request. No confirmed update receipt; scoped post-action state remains unchanged; the scoped post-action date is 2027-03-29.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_performed' matches no confirmed update receipt; scoped post-action state remains unchanged.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-28-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "128c9cae56e0def00f880a1980b66c68144181444da879c0fc59dd1aec649466", + "result_file_sha256": "9233e86d09af1cd05d13e7ec48d0afe392a6ff5d871d26feaafc1cb5c237e5eb", + "receipt_file_sha256": "04aa9540762d5710f9c2931580c3349e356acf71c033fda066d2b0a749204b19", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-03-29\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "16f3e3d844d05284f800bee069891c80e8caac0ba79cf6402ea600abb8f834d8", + "raw": { + "action_status": "not_performed", + "answer": "2027-03-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-29", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-03-29' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2128 after handling the request. No confirmed update receipt; scoped post-action state remains unchanged; the scoped post-action date is 2027-03-29.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_performed' matches no confirmed update receipt; scoped post-action state remains unchanged.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-28-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "05b787acf3ddc1965b6c78a81f94972d0ca2a542a21a6b12fa29eddd909ed313", + "result_file_sha256": "e0673a8d7cedf6a277762231f95a8784e91700ac109e2d0b0c8b7cfc020906a8", + "receipt_file_sha256": "4bdfbf8838a36475318909bc9fdf8c9b14972dc3289f3a1ebb94ee39dc87d114", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-29\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "d038bea16874e512c3a0ac000501a1c71f28da67f66af929d198fcbf99e0a17d", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-29", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-29", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-03-29' and clarification False match the scoped source and execution: The question asks the currently confirmed date of VISIT-2128 after handling the request. No confirmed update receipt; scoped post-action state remains unchanged; the scoped post-action date is 2027-03-29.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'not_performed': no confirmed update receipt; scoped post-action state remains unchanged.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-29-hi-m0", + "case_id": "fresh-29-hi", + "model": "sarvam-105b", + "family": "future_approval", + "language": "hi", + "parent_paired_file_sha256": "86f134c3321d2b94d9b801dc0dd689bf651211e4b7ccf6cdf41f4015d8288fa4", + "question": "VISIT-2129 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-29-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-29-1", + "role": "customer", + "content": "मैं अभी VISIT-2129 को 2027-04-30 पर बदलना चाहता हूं।" + }, + { + "id": "note-29-2", + "role": "customer", + "content": "सर्वर की अनुमति दिए गए सर्वर समय के एक घंटे बाद लागू होगी। मेरे अनुरोध से उसकी वैधता अवधि नहीं बदलती।" + } + ], + "source_basis_reviewed": "The approval for VISIT-2129 is not effective at the supplied server time. Future-dated approval and no attempted update receipt; the currently confirmed date remains 2027-03-30.", + "question_target": "actual_current_date", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-30", + "final_scoped_date": "2027-03-30", + "expected_from_source_and_actual_execution": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_raw_action_status": "not_performed", + "action_status_basis": "future-dated approval and no attempted update receipt", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-29-hi-m0-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "8363f281bed18a46f4de3cc3466ede5ce9bfa8f4891004a768a1348792340508", + "result_file_sha256": "9a33d005b8af04a9daae27cfbec0007280a387d73264252880042af4f86632aa", + "receipt_file_sha256": "6efa687fa611a9e19423e11e6b83a2292243dab651a286dd0a3a10391155b3b1", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-03-30\",\"needs_clarification\":false}", + "raw_content_sha256": "0163f4a41e8881b6b015044ce2a23ad1adb4ba560451442685979f2b5b6a7fff", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-03-30' and clarification False match the scoped source and execution: The approval for VISIT-2129 is not effective at the supplied server time. Future-dated approval and no attempted update receipt; the currently confirmed date remains 2027-03-30.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'not_performed': future-dated approval and no attempted update receipt.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-29-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "33cbcda1ad92e68b211889119704a9dffbdd9db787f33701ceb27d31b4a17181", + "result_file_sha256": "f517b9fdb42a8c5fdc3536e2720b69744ec169f1684a3271023240b26b8c846f", + "receipt_file_sha256": "f638b4f4c8872fb52309ece245425409cf7a27f80e90869f2e68a1db7104b587", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-30\",\"needs_clarification\":false}", + "raw_content_sha256": "6f60dea4694b6d52e1bbd9a2b2fe175afd3782c6eb1f13cf983f53e25c6b19cd", + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-03-30' and clarification False match the scoped source and execution: The approval for VISIT-2129 is not effective at the supplied server time. Future-dated approval and no attempted update receipt; the currently confirmed date remains 2027-03-30.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_performed' matches future-dated approval and no attempted update receipt.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-29-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "7766f74ddee856ea89f487385dc9ab3c10e7d44d0657df5c37fc92bc44f01d53", + "result_file_sha256": "f517b9fdb42a8c5fdc3536e2720b69744ec169f1684a3271023240b26b8c846f", + "receipt_file_sha256": "f2c164ca845cc1f0dc499ea44e34faca2a2169e066aa73299e43eadf3fb9c11c", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-30\",\"needs_clarification\":false}", + "raw_content_sha256": "6f60dea4694b6d52e1bbd9a2b2fe175afd3782c6eb1f13cf983f53e25c6b19cd", + "raw": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-03-30' and clarification False match the scoped source and execution: The approval for VISIT-2129 is not effective at the supplied server time. Future-dated approval and no attempted update receipt; the currently confirmed date remains 2027-03-30.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_performed' matches future-dated approval and no attempted update receipt.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-29-hi-m1", + "case_id": "fresh-29-hi", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "language": "hi", + "parent_paired_file_sha256": "486480143628f58da2d6e11a0f0ba448b9deb8996134492661d009d3a33ab97f", + "question": "VISIT-2129 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-29-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-29-1", + "role": "customer", + "content": "मैं अभी VISIT-2129 को 2027-04-30 पर बदलना चाहता हूं।" + }, + { + "id": "note-29-2", + "role": "customer", + "content": "सर्वर की अनुमति दिए गए सर्वर समय के एक घंटे बाद लागू होगी। मेरे अनुरोध से उसकी वैधता अवधि नहीं बदलती।" + } + ], + "source_basis_reviewed": "The approval for VISIT-2129 is not effective at the supplied server time. Attempted update was rejected because approval was not yet effective; the currently confirmed date remains 2027-03-30.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-bf623e26b1ae3cce", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2129" + }, + "response": { + "date": "2027-03-30", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-ae8b9cc5b8a7aa36", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2129", + "approval_id": "AUTH-29-GRANT-20", + "date": "2027-04-30", + "expected_version": 5, + "idempotency_key": "REQUEST-29-CHANGE-20" + }, + "response": { + "status": "rejected", + "reason": "approval_not_effective", + "date": "2027-03-30", + "version": 5 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-30", + "final_scoped_date": "2027-03-30", + "expected_from_source_and_actual_execution": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_raw_action_status": "rejected", + "action_status_basis": "attempted update was rejected because approval was not yet effective", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "chronological": { + "job_id": "confirm-fresh-29-hi-m1-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "e0938961a620d82440a3d35d7b55e6791488a88abd2aae27f04a10b7d0d547a3", + "result_file_sha256": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "receipt_file_sha256": "11a06b778a258ba27bca7d3a61bfd214c1cb0ad510a7e93787e42bd7c891e6d5", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-03-30\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "c7b8d9498405aff859e7cd3c35981ee7d640e01038a3105a846ac123bac08074", + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-03-30' and clarification False match the scoped source and execution: The approval for VISIT-2129 is not effective at the supplied server time. Attempted update was rejected because approval was not yet effective; the currently confirmed date remains 2027-03-30.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'rejected' matches attempted update was rejected because approval was not yet effective.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'rejected' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-29-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "9262411795cc2e085db7ac56fa161cf3cfbebec53243ace848c856caf79ba1e0", + "result_file_sha256": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "receipt_file_sha256": "508f7d9e6f4f2835486ede5922198f320552536f725cfaeb34b03032614ef76e", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-03-30\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "c7b8d9498405aff859e7cd3c35981ee7d640e01038a3105a846ac123bac08074", + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-03-30' and clarification False match the scoped source and execution: The approval for VISIT-2129 is not effective at the supplied server time. Attempted update was rejected because approval was not yet effective; the currently confirmed date remains 2027-03-30.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'rejected' matches attempted update was rejected because approval was not yet effective.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'rejected' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-29-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "9262411795cc2e085db7ac56fa161cf3cfbebec53243ace848c856caf79ba1e0", + "result_file_sha256": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "receipt_file_sha256": "a3382b5f17f7404187b1bdcfaed3c758867483b6129e5da9145ecd395792e400", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-03-30\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "c7b8d9498405aff859e7cd3c35981ee7d640e01038a3105a846ac123bac08074", + "raw": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-03-30' and clarification False match the scoped source and execution: The approval for VISIT-2129 is not effective at the supplied server time. Attempted update was rejected because approval was not yet effective; the currently confirmed date remains 2027-03-30.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'rejected' matches attempted update was rejected because approval was not yet effective.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'rejected' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-30-hi-m0", + "case_id": "fresh-30-hi", + "model": "sarvam-105b", + "family": "future_approval", + "language": "hi", + "parent_paired_file_sha256": "0f79cd0bbb1c936c84f5ad1494bc2c146e08f2076ebc6f4c9a6e5f607dc24ad0", + "question": "VISIT-2130 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-30-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। मैं अभी VISIT-2130 को 2027-05-01 पर बदलना चाहता हूं। सर्वर की अनुमति दिए गए सर्वर समय के एक घंटे बाद लागू होगी। मेरे अनुरोध से उसकी वैधता अवधि नहीं बदलती।" + } + ], + "source_basis_reviewed": "The approval for VISIT-2130 is not effective at the supplied server time. Future-dated approval and no attempted update receipt; the currently confirmed date remains 2027-03-31.", + "question_target": "actual_current_date", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-03-31", + "final_scoped_date": "2027-03-31", + "expected_from_source_and_actual_execution": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_raw_action_status": "not_performed", + "action_status_basis": "future-dated approval and no attempted update receipt", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-30-hi-m0-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "dde8886b7b4fd823323fe0f161147dcf96946e097aacc37f1c5d820afb120443", + "result_file_sha256": "c225aab8d19cca2515a377d9c5462cc34a003a0c3653544bba785e793db51707", + "receipt_file_sha256": "6201e57d19e68ffd20e98c4338b8a23a59ddc7e41f8a437b154ab0803fc3f440", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-03-31\",\"needs_clarification\":false}", + "raw_content_sha256": "cca5bd3578575b6d45284125d6a11cca2ba727e2b310e9f5ec6cd6bef932def8", + "raw": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-03-31' and clarification False match the scoped source and execution: The approval for VISIT-2130 is not effective at the supplied server time. Future-dated approval and no attempted update receipt; the currently confirmed date remains 2027-03-31.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'not_performed': future-dated approval and no attempted update receipt.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-30-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "fbdebf3aafd5335544c1e3db5488c1c1e304e8b69cdb58af5e6ecadb0b949405", + "result_file_sha256": "ff996de64c0ed5833a02676aaa0f784e5b961cc8b0596750e40ebe7f0be479d3", + "receipt_file_sha256": "68aac119c42fe21dbc413c027f8077d2e39175d597ef69429d7e59d1eaaba3ea", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-31\",\"needs_clarification\":false}", + "raw_content_sha256": "8b5cfbd66b61fb01aa99c2b577a36feb115de9abc6e48399c5ba18b20f65121f", + "raw": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-03-31' and clarification False match the scoped source and execution: The approval for VISIT-2130 is not effective at the supplied server time. Future-dated approval and no attempted update receipt; the currently confirmed date remains 2027-03-31.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_performed' matches future-dated approval and no attempted update receipt.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-30-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "fd3ab271a92a0f3e733ec7a72cc804310d8fb149e6473b8f4c7830e62b0990cb", + "result_file_sha256": "ff996de64c0ed5833a02676aaa0f784e5b961cc8b0596750e40ebe7f0be479d3", + "receipt_file_sha256": "21ce5158b48f220f886f317a21571c0bff55262227466c14ada467671afb8f76", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-31\",\"needs_clarification\":false}", + "raw_content_sha256": "8b5cfbd66b61fb01aa99c2b577a36feb115de9abc6e48399c5ba18b20f65121f", + "raw": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-03-31' and clarification False match the scoped source and execution: The approval for VISIT-2130 is not effective at the supplied server time. Future-dated approval and no attempted update receipt; the currently confirmed date remains 2027-03-31.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_performed' matches future-dated approval and no attempted update receipt.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-30-hi-m1", + "case_id": "fresh-30-hi", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "language": "hi", + "parent_paired_file_sha256": "243099bbb914ba5225b4c03eabad5206511f9b239362429d5a490b13d2f0ae74", + "question": "VISIT-2130 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-30-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। मैं अभी VISIT-2130 को 2027-05-01 पर बदलना चाहता हूं। सर्वर की अनुमति दिए गए सर्वर समय के एक घंटे बाद लागू होगी। मेरे अनुरोध से उसकी वैधता अवधि नहीं बदलती।" + } + ], + "source_basis_reviewed": "The approval for VISIT-2130 is not effective at the supplied server time. Attempted update was rejected because approval was not yet effective; the currently confirmed date remains 2027-03-31.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a02e4475177d4362", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2130" + }, + "response": { + "date": "2027-03-31", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-a1a8856066b4de60", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2130", + "approval_id": "AUTH-30-GRANT-20", + "date": "2027-05-01", + "expected_version": 5, + "idempotency_key": "REQUEST-30-CHANGE-20" + }, + "response": { + "status": "rejected", + "reason": "approval_not_effective", + "date": "2027-03-31", + "version": 5 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-03-31", + "final_scoped_date": "2027-03-31", + "expected_from_source_and_actual_execution": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_raw_action_status": "rejected", + "action_status_basis": "attempted update was rejected because approval was not yet effective", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-30-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "2ce47e6a0726800d050c184d9648b2af3d6b17985f8da02fa2658912df88f586", + "result_file_sha256": "87902016aae6283bdff9207729e01b30ad77cad918dba806396649f58f68c74c", + "receipt_file_sha256": "62cf8b001ae4591c35885f4d2f0e6b445d9d6f978d39c0c60f335ab2973ac474", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-03-31\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "31b20e9d2fd9f0fa5256283e857194fb3888033a9b8acd91e893978a488f41a6", + "raw": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-03-31' and clarification False match the scoped source and execution: The approval for VISIT-2130 is not effective at the supplied server time. Attempted update was rejected because approval was not yet effective; the currently confirmed date remains 2027-03-31.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'rejected' matches attempted update was rejected because approval was not yet effective.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'rejected' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-30-hi-m1-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "dc0759a411abe387a18959e79b04cb5e1dedef48b6a3e8219643401e6a4d49aa", + "result_file_sha256": "87902016aae6283bdff9207729e01b30ad77cad918dba806396649f58f68c74c", + "receipt_file_sha256": "c63c0997a3d41e32c4f8660f4141100b9c474a0b4ce988fb5f7d7a85ab5bf0bd", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-03-31\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "31b20e9d2fd9f0fa5256283e857194fb3888033a9b8acd91e893978a488f41a6", + "raw": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-03-31' and clarification False match the scoped source and execution: The approval for VISIT-2130 is not effective at the supplied server time. Attempted update was rejected because approval was not yet effective; the currently confirmed date remains 2027-03-31.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'rejected' matches attempted update was rejected because approval was not yet effective.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'rejected' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-30-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "2ce47e6a0726800d050c184d9648b2af3d6b17985f8da02fa2658912df88f586", + "result_file_sha256": "87902016aae6283bdff9207729e01b30ad77cad918dba806396649f58f68c74c", + "receipt_file_sha256": "2951b8a7ceb3ecc79cf8a072f6d92a03c7c015db592558a3673cb22ea14140e1", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-03-31\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "31b20e9d2fd9f0fa5256283e857194fb3888033a9b8acd91e893978a488f41a6", + "raw": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-03-31' and clarification False match the scoped source and execution: The approval for VISIT-2130 is not effective at the supplied server time. Attempted update was rejected because approval was not yet effective; the currently confirmed date remains 2027-03-31.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'rejected' matches attempted update was rejected because approval was not yet effective.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'rejected' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-31-hi-m0", + "case_id": "fresh-31-hi", + "model": "sarvam-105b", + "family": "future_approval", + "language": "hi", + "parent_paired_file_sha256": "d0ccfe3223f5fbcfe332f856b5d79b1adc80808c79ea32d499b4510747cc5f27", + "question": "VISIT-2131 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-31-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-31-1", + "role": "customer", + "content": "मैं अभी VISIT-2131 को 2027-05-02 पर बदलना चाहता हूं।" + }, + { + "id": "note-31-2", + "role": "customer", + "content": "सर्वर की अनुमति दिए गए सर्वर समय के एक घंटे बाद लागू होगी। मेरे अनुरोध से उसकी वैधता अवधि नहीं बदलती।" + } + ], + "source_basis_reviewed": "The approval for VISIT-2131 is not effective at the supplied server time. Future-dated approval and no attempted update receipt; the currently confirmed date remains 2027-04-01.", + "question_target": "actual_current_date", + "actual_tool_trace": [], + "seed_receipts": [], + "initial_scoped_date": "2027-04-01", + "final_scoped_date": "2027-04-01", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_raw_action_status": "not_performed", + "action_status_basis": "future-dated approval and no attempted update receipt", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-31-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "4da534c6562d8d6e84a49c4710bed35dd8bc4eb5b16b6d70aef298ceb898949a", + "result_file_sha256": "ae6f1b1012dd04896248c230783fe5e9d7c1efa09ed2714a42f5f73c3c7df7b6", + "receipt_file_sha256": "3339ce6287e23954764217154664bf6151501a136ea6844819cdacdde7a04698", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\": \"not_requested\", \"answer\": \"2027-04-01\", \"needs_clarification\": false}", + "raw_content_sha256": "c0299aa3f32fc48f6e439ea80a9a0c333f44581a3e6e894600bb587b5b6d9d9d", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-01' and clarification False match the scoped source and execution: The approval for VISIT-2131 is not effective at the supplied server time. Future-dated approval and no attempted update receipt; the currently confirmed date remains 2027-04-01.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'not_performed': future-dated approval and no attempted update receipt.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-31-hi-m0-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "132deed8c86eaf5e401a33562e708dc8fa183c76d40e40989133a02426b5d966", + "result_file_sha256": "b5770e7ca54498bbe9cbc192d959a98975dc03170a1aa14ce7055f82160458c4", + "receipt_file_sha256": "23d5cf6f2e4741306acfcf8a4b267367ce4a547154fc84727233aa1519d6d4d7", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-01\",\"needs_clarification\":false}", + "raw_content_sha256": "564e8f5c169211638a274eece597eb4b2d5f7ccfac6aedfd9d936dc2a89cb610", + "raw": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-01' and clarification False match the scoped source and execution: The approval for VISIT-2131 is not effective at the supplied server time. Future-dated approval and no attempted update receipt; the currently confirmed date remains 2027-04-01.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'not_performed': future-dated approval and no attempted update receipt.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-31-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "93071452d8f5a1a00f84dd64db78e08b47dff4c65ceff549705dc662743459fd", + "result_file_sha256": "fe4c3267a9cec82092e8ede130613eed251227abe98c2163619dbfcb8dccec96", + "receipt_file_sha256": "5311d4204754c1ca2d262b0046f249c77f700cb5953f3432951bf804d3b40798", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-04-01\",\"needs_clarification\":false}", + "raw_content_sha256": "a1f9fcadc6ba4e8eafc46ac98450dd9b630b034f69ac35fd2a9eecd2706d424e", + "raw": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-01' and clarification False match the scoped source and execution: The approval for VISIT-2131 is not effective at the supplied server time. Future-dated approval and no attempted update receipt; the currently confirmed date remains 2027-04-01.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_performed' matches future-dated approval and no attempted update receipt.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-31-hi-m1", + "case_id": "fresh-31-hi", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "language": "hi", + "parent_paired_file_sha256": "57a94f82e5b369727d864dfbddae8369cb02c2447c1734b903be5568c4ef81c0", + "question": "VISIT-2131 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-31-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-31-1", + "role": "customer", + "content": "मैं अभी VISIT-2131 को 2027-05-02 पर बदलना चाहता हूं।" + }, + { + "id": "note-31-2", + "role": "customer", + "content": "सर्वर की अनुमति दिए गए सर्वर समय के एक घंटे बाद लागू होगी। मेरे अनुरोध से उसकी वैधता अवधि नहीं बदलती।" + } + ], + "source_basis_reviewed": "The approval for VISIT-2131 is not effective at the supplied server time. Attempted update was rejected because approval was not yet effective; the currently confirmed date remains 2027-04-01.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a5d05b861d17cf8d", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2131" + }, + "response": { + "date": "2027-04-01", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-b7e3f14f3857c85e", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2131", + "approval_id": "AUTH-31-GRANT-20", + "date": "2027-05-02", + "expected_version": 5, + "idempotency_key": "REQUEST-31-CHANGE-20" + }, + "response": { + "status": "rejected", + "reason": "approval_not_effective", + "date": "2027-04-01", + "version": 5 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-04-01", + "final_scoped_date": "2027-04-01", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_raw_action_status": "rejected", + "action_status_basis": "attempted update was rejected because approval was not yet effective", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-31-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "8c1e23f12d059ac2ff4223871a17a00b89dfef0477a759b7e46d73cee13e8f7d", + "result_file_sha256": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "receipt_file_sha256": "30dea4b81f9687cc49962366317f9c69badcedda6dfedd3278630b269d3adbb0", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-04-01\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "0d3aa07f0f0dff3403e3ba14d6cb185cfb839350bb5d46026f3e377de4a0b70f", + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-01' and clarification False match the scoped source and execution: The approval for VISIT-2131 is not effective at the supplied server time. Attempted update was rejected because approval was not yet effective; the currently confirmed date remains 2027-04-01.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'rejected' matches attempted update was rejected because approval was not yet effective.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'rejected' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-31-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "8c1e23f12d059ac2ff4223871a17a00b89dfef0477a759b7e46d73cee13e8f7d", + "result_file_sha256": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "receipt_file_sha256": "5d2229d09cd569be811748fb31f623ebdc492a877bcfa9aa0e40d364758bc652", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-04-01\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "0d3aa07f0f0dff3403e3ba14d6cb185cfb839350bb5d46026f3e377de4a0b70f", + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-01' and clarification False match the scoped source and execution: The approval for VISIT-2131 is not effective at the supplied server time. Attempted update was rejected because approval was not yet effective; the currently confirmed date remains 2027-04-01.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'rejected' matches attempted update was rejected because approval was not yet effective.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'rejected' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-31-hi-m1-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "fae8f61e961dfdcec75704c49c5fabcb669d2a5275c6f5987bb2874713a46e9a", + "result_file_sha256": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "receipt_file_sha256": "00accf7cf822ca23f26fc7f432034e00ff2fe7637cfa60bf8e3bc95018716273", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-04-01\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "0d3aa07f0f0dff3403e3ba14d6cb185cfb839350bb5d46026f3e377de4a0b70f", + "raw": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-01' and clarification False match the scoped source and execution: The approval for VISIT-2131 is not effective at the supplied server time. Attempted update was rejected because approval was not yet effective; the currently confirmed date remains 2027-04-01.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'rejected' matches attempted update was rejected because approval was not yet effective.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'rejected' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-32-hi-m0", + "case_id": "fresh-32-hi", + "model": "sarvam-105b", + "family": "future_approval", + "language": "hi", + "parent_paired_file_sha256": "9bfbc81621aa2c13582ef5f75bbf809c08d425a0a97e9cd6c34671946a19fa01", + "question": "VISIT-2132 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-32-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। मैं अभी VISIT-2132 को 2027-05-03 पर बदलना चाहता हूं। सर्वर की अनुमति दिए गए सर्वर समय के एक घंटे बाद लागू होगी। मेरे अनुरोध से उसकी वैधता अवधि नहीं बदलती।" + } + ], + "source_basis_reviewed": "The approval for VISIT-2132 is not effective at the supplied server time. Future-dated approval and no attempted update receipt; the currently confirmed date remains 2027-04-02.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-adee016bd67cbbe6", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2132" + }, + "response": { + "date": "2027-04-02", + "version": 5, + "closed": false + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-04-02", + "final_scoped_date": "2027-04-02", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_raw_action_status": "not_performed", + "action_status_basis": "future-dated approval and no attempted update receipt", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-32-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "baa751ada310b8bd5b3c739bfa00c4e16262a9f17986e3292f8b68e847da29c6", + "result_file_sha256": "b106ac717f61df19b67b9ff99d9131e9d442ef563917d74f9f34670967d64074", + "receipt_file_sha256": "4d599d18bba2fe299292b31815b94331525a745eb23280e046e0b6e5f9b0eb82", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-04-02\",\"needs_clarification\":false}", + "raw_content_sha256": "f08325c4871b8031e4648a53e1cec3da798b14ad1f6ccd0f41aae4349a10b13e", + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-02' and clarification False match the scoped source and execution: The approval for VISIT-2132 is not effective at the supplied server time. Future-dated approval and no attempted update receipt; the currently confirmed date remains 2027-04-02.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_performed' matches future-dated approval and no attempted update receipt.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-32-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "d924779c9861b5acc2cb3528c41cc4f521a7fa1f736183a7989e64fa52787bd9", + "result_file_sha256": "b106ac717f61df19b67b9ff99d9131e9d442ef563917d74f9f34670967d64074", + "receipt_file_sha256": "0431467b547cb0cd94bb5e08f64d65c56855d92f2bf2df5721a40a584c5dd726", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-04-02\",\"needs_clarification\":false}", + "raw_content_sha256": "f08325c4871b8031e4648a53e1cec3da798b14ad1f6ccd0f41aae4349a10b13e", + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-02' and clarification False match the scoped source and execution: The approval for VISIT-2132 is not effective at the supplied server time. Future-dated approval and no attempted update receipt; the currently confirmed date remains 2027-04-02.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_performed' matches future-dated approval and no attempted update receipt.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-32-hi-m0-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "7b05f06e624b45a82bd22066127e1538eb78568ed84200b49842e90b31bf4f96", + "result_file_sha256": "b106ac717f61df19b67b9ff99d9131e9d442ef563917d74f9f34670967d64074", + "receipt_file_sha256": "a776a0244293072c7e147378f6a00a994ca230a162ae1d5f0f9b148c72236003", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-04-02\",\"needs_clarification\":false}", + "raw_content_sha256": "f08325c4871b8031e4648a53e1cec3da798b14ad1f6ccd0f41aae4349a10b13e", + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-02' and clarification False match the scoped source and execution: The approval for VISIT-2132 is not effective at the supplied server time. Future-dated approval and no attempted update receipt; the currently confirmed date remains 2027-04-02.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_performed' matches future-dated approval and no attempted update receipt.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-32-hi-m1", + "case_id": "fresh-32-hi", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "language": "hi", + "parent_paired_file_sha256": "e0dac1ea9e13cd763facc50020471a099df61e7d85fe95b30e5c7b7ec595c51c", + "question": "VISIT-2132 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-32-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। मैं अभी VISIT-2132 को 2027-05-03 पर बदलना चाहता हूं। सर्वर की अनुमति दिए गए सर्वर समय के एक घंटे बाद लागू होगी। मेरे अनुरोध से उसकी वैधता अवधि नहीं बदलती।" + } + ], + "source_basis_reviewed": "The approval for VISIT-2132 is not effective at the supplied server time. Attempted update was rejected because approval was not yet effective; the currently confirmed date remains 2027-04-02.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9153b1d30654e89e", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2132" + }, + "response": { + "date": "2027-04-02", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-8afa902f6102cd2c", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2132", + "approval_id": "AUTH-32-GRANT-20", + "date": "2027-05-03", + "expected_version": 5, + "idempotency_key": "REQUEST-32-CHANGE-20" + }, + "response": { + "status": "rejected", + "reason": "approval_not_effective", + "date": "2027-04-02", + "version": 5 + } + } + ], + "seed_receipts": [], + "initial_scoped_date": "2027-04-02", + "final_scoped_date": "2027-04-02", + "expected_from_source_and_actual_execution": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_raw_action_status": "rejected", + "action_status_basis": "attempted update was rejected because approval was not yet effective", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": false, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-32-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "57d936699fd85669b49c7596e5d3af4c6a4169b0efa22037f11489d96ea44571", + "result_file_sha256": "b5537fea0aacf9bcedfc1a598c3b4e08a55b92f5c5cb181429cff1acda690630", + "receipt_file_sha256": "b2790034d508d7c532989f41976975e72b6fdc4dc2799e92ef8dcc15e833c6ad", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-04-02\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "1f2cad9a533345f41eea593348fa91f701d75caa054b15520f51e90a290da757", + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-02' and clarification False match the scoped source and execution: The approval for VISIT-2132 is not effective at the supplied server time. Attempted update was rejected because approval was not yet effective; the currently confirmed date remains 2027-04-02.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_performed' does not match expected 'rejected': attempted update was rejected because approval was not yet effective.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'rejected' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-32-hi-m1-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "3dce79048a41ce8cdf9ca72ec38d98a4e4a30ff8a4ee1b0c5019a6cc0f2ae40f", + "result_file_sha256": "40234eb32747058b66be85c3240998e1ffc5c4add188004394236ae5121c013a", + "receipt_file_sha256": "ee8fed0d722a676091ed53679c70c263d232e712be55e53e4b8f5c97afe5dfea", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-04-02\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "f4810b398ad25b41b9ffb88bab3165221c2636ac30c73d91e79d0491d89c93b1", + "raw": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-02' and clarification False match the scoped source and execution: The approval for VISIT-2132 is not effective at the supplied server time. Attempted update was rejected because approval was not yet effective; the currently confirmed date remains 2027-04-02.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'rejected' matches attempted update was rejected because approval was not yet effective.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'rejected' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-32-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "57d936699fd85669b49c7596e5d3af4c6a4169b0efa22037f11489d96ea44571", + "result_file_sha256": "b5537fea0aacf9bcedfc1a598c3b4e08a55b92f5c5cb181429cff1acda690630", + "receipt_file_sha256": "b6e97d4530bbc3931ace8a895c2318485186ba29c3098c55c2192bee89290f64", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-04-02\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "1f2cad9a533345f41eea593348fa91f701d75caa054b15520f51e90a290da757", + "raw": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "application_output": { + "action_status": "rejected", + "answer": "2027-04-02", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-04-02' and clarification False match the scoped source and execution: The approval for VISIT-2132 is not effective at the supplied server time. Attempted update was rejected because approval was not yet effective; the currently confirmed date remains 2027-04-02.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_performed' does not match expected 'rejected': attempted update was rejected because approval was not yet effective.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'rejected' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-33-hi-m0", + "case_id": "fresh-33-hi", + "model": "sarvam-105b", + "family": "completed_older_request", + "language": "hi", + "parent_paired_file_sha256": "1306c78c64c7925d09c24ef93058619ea3c5bbb28da8a496ee946ec55788fe70", + "question": "VISIT-2133 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-33-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-33-1", + "role": "customer", + "content": "VISIT-2133 को 2027-05-04 पर बदलने के मूल अनुरोध को उसके मूल पहचान नंबर से जांचें।" + }, + { + "id": "note-33-2", + "role": "customer", + "content": "वह पहले पूरा हो चुका है। बाद के अलग बदलाव से तारीख 2027-06-03 हुई। इस दोहराए गए अनुरोध से नया बदलाव वापस न करें।" + } + ], + "source_basis_reviewed": "The old request for VISIT-2133 was completed, but a later separate completion sets current confirmed date 2027-06-03. Do not replay the old request.", + "question_target": "actual_current_date", + "actual_tool_trace": [], + "seed_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2133", + "date": "2027-05-04", + "expected_version": 5, + "idempotency_key": "REQUEST-33-CHANGE-23", + "approval_id": "AUTH-33-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-04", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2133", + "date": "2027-06-03", + "expected_version": 6, + "idempotency_key": "REQUEST-33-LATER-23", + "approval_id": "AUTH-33-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-03", + "version": 7 + } + } + ], + "initial_scoped_date": "2027-06-03", + "final_scoped_date": "2027-06-03", + "expected_from_source_and_actual_execution": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_raw_action_status": "already_applied", + "action_status_basis": "matching earlier completed request receipt; later distinct completion sets current date", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-33-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "b772fd73932e381829ca7e9c7561a2e689401de3c405a8a920b4d11218de2d5b", + "result_file_sha256": "c1553fd0523b1529616e1962bbf6812c5dd94faf93dff31d113956ebebf274ad", + "receipt_file_sha256": "54352f59fd1e0c18d7fa84ef4eecd65de4c89542141c7bd2587f1927e010c66d", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-03\",\"needs_clarification\":false}", + "raw_content_sha256": "6b7d837755fee0f4384fc2b064b5188c55989832ef709b05dbeab7ee721fed6c", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-03' and clarification False match the scoped source and execution: The old request for VISIT-2133 was completed, but a later separate completion sets current confirmed date 2027-06-03. Do not replay the old request.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'already_applied': matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-33-hi-m0-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "4dd9a93cbef90ef261066e8907e3fef893207d31997fe72209c573e1aca6af2d", + "result_file_sha256": "c1553fd0523b1529616e1962bbf6812c5dd94faf93dff31d113956ebebf274ad", + "receipt_file_sha256": "61223f91fe64d2d4a85c7a84d969291fe5661c687e1d3273aab2a0190c4953e7", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-03\",\"needs_clarification\":false}", + "raw_content_sha256": "6b7d837755fee0f4384fc2b064b5188c55989832ef709b05dbeab7ee721fed6c", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-03' and clarification False match the scoped source and execution: The old request for VISIT-2133 was completed, but a later separate completion sets current confirmed date 2027-06-03. Do not replay the old request.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'already_applied': matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-33-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "71f07222afa220a83c3073cdddc3079af12364fad58a85388b4372703302ef8d", + "result_file_sha256": "750a377f99646c2dba4efcf71ca5d6a495a0fd9537249d5ad8e2342669945bf5", + "receipt_file_sha256": "a6f2ec0c68bfffa7f63baca3662ec019b92beb292ae5ecb024f7e805cb58fb77", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-06-03\",\"needs_clarification\":false}", + "raw_content_sha256": "598b80597464311266542bc64e52685c1240812149df3849df0c5f889ee02cff", + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-03' and clarification False match the scoped source and execution: The old request for VISIT-2133 was completed, but a later separate completion sets current confirmed date 2027-06-03. Do not replay the old request.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'already_applied' matches matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-33-hi-m1", + "case_id": "fresh-33-hi", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "language": "hi", + "parent_paired_file_sha256": "6bcd5a039d69eff0138a3f43b3d7a8c730b3bb16b34ad5b78e7856e0b17cf28c", + "question": "VISIT-2133 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-33-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-33-1", + "role": "customer", + "content": "VISIT-2133 को 2027-05-04 पर बदलने के मूल अनुरोध को उसके मूल पहचान नंबर से जांचें।" + }, + { + "id": "note-33-2", + "role": "customer", + "content": "वह पहले पूरा हो चुका है। बाद के अलग बदलाव से तारीख 2027-06-03 हुई। इस दोहराए गए अनुरोध से नया बदलाव वापस न करें।" + } + ], + "source_basis_reviewed": "The old request for VISIT-2133 was completed, but a later separate completion sets current confirmed date 2027-06-03. Do not replay the old request.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8c613b062f52cc10", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2133" + }, + "response": { + "date": "2027-06-03", + "version": 7, + "closed": false + } + } + ], + "seed_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2133", + "date": "2027-05-04", + "expected_version": 5, + "idempotency_key": "REQUEST-33-CHANGE-23", + "approval_id": "AUTH-33-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-04", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2133", + "date": "2027-06-03", + "expected_version": 6, + "idempotency_key": "REQUEST-33-LATER-23", + "approval_id": "AUTH-33-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-03", + "version": 7 + } + } + ], + "initial_scoped_date": "2027-06-03", + "final_scoped_date": "2027-06-03", + "expected_from_source_and_actual_execution": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_raw_action_status": "already_applied", + "action_status_basis": "matching earlier completed request receipt; later distinct completion sets current date", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-33-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "8bd4e7672c6163fdd4edc15b38b70eea48be41db1dfba4eedfd88bc2f5f52302", + "result_file_sha256": "0e529979621b57ba683527909c61a095b19f49f46d26c9741184bd4c71571e88", + "receipt_file_sha256": "d9343555ab4171bd5d0d73fdcba69e674f4d5cb9666963cf154946396ecbe92c", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-03\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "0d5919b20e564dc0d5fbc550dc3a613bc37b60a64f55e1b1d48adadf45f1ce1a", + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-03' and clarification False match the scoped source and execution: The old request for VISIT-2133 was completed, but a later separate completion sets current confirmed date 2027-06-03. Do not replay the old request.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'already_applied' matches matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-33-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "8054554c144a9047c719f727c2a6d7ca6760dd0b60ea0b3135dbdf4f3bb61b43", + "result_file_sha256": "0e529979621b57ba683527909c61a095b19f49f46d26c9741184bd4c71571e88", + "receipt_file_sha256": "026a7c13d45934a361f6b1e34492b107d9437153b9f602810ae8de90ab67b5ac", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-03\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "0d5919b20e564dc0d5fbc550dc3a613bc37b60a64f55e1b1d48adadf45f1ce1a", + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-03' and clarification False match the scoped source and execution: The old request for VISIT-2133 was completed, but a later separate completion sets current confirmed date 2027-06-03. Do not replay the old request.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'already_applied' matches matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-33-hi-m1-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "00549b5eb977c265353a458c373e2b80c2fa68169db4c8f2cc62cd83ff74eab9", + "result_file_sha256": "0e529979621b57ba683527909c61a095b19f49f46d26c9741184bd4c71571e88", + "receipt_file_sha256": "9c77af1980074955daf3519461b0db13782e5fb99856af5465472487c2cab35b", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-03\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "0d5919b20e564dc0d5fbc550dc3a613bc37b60a64f55e1b1d48adadf45f1ce1a", + "raw": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-03' and clarification False match the scoped source and execution: The old request for VISIT-2133 was completed, but a later separate completion sets current confirmed date 2027-06-03. Do not replay the old request.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'already_applied' matches matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-34-hi-m0", + "case_id": "fresh-34-hi", + "model": "sarvam-105b", + "family": "completed_older_request", + "language": "hi", + "parent_paired_file_sha256": "98a60ae9cd834dd973718079b9495fb0cdff4122b7fec9f35d178cde1b4a6f22", + "question": "VISIT-2134 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-34-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। VISIT-2134 को 2027-05-05 पर बदलने के मूल अनुरोध को उसके मूल पहचान नंबर से जांचें। वह पहले पूरा हो चुका है। बाद के अलग बदलाव से तारीख 2027-06-04 हुई। इस दोहराए गए अनुरोध से नया बदलाव वापस न करें।" + } + ], + "source_basis_reviewed": "The old request for VISIT-2134 was completed, but a later separate completion sets current confirmed date 2027-06-04. Do not replay the old request.", + "question_target": "actual_current_date", + "actual_tool_trace": [], + "seed_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2134", + "date": "2027-05-05", + "expected_version": 5, + "idempotency_key": "REQUEST-34-CHANGE-23", + "approval_id": "AUTH-34-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-05", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2134", + "date": "2027-06-04", + "expected_version": 6, + "idempotency_key": "REQUEST-34-LATER-23", + "approval_id": "AUTH-34-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-04", + "version": 7 + } + } + ], + "initial_scoped_date": "2027-06-04", + "final_scoped_date": "2027-06-04", + "expected_from_source_and_actual_execution": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_raw_action_status": "already_applied", + "action_status_basis": "matching earlier completed request receipt; later distinct completion sets current date", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-34-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "c71b2013cda50dc55de7dcb10404495ea68d70ff26503a57bde695c7ba896220", + "result_file_sha256": "042713423da807bcfef49b2ccab9db9d7aee0d0c696115936fca95c315d2e3bb", + "receipt_file_sha256": "7c5356bfc799ab014703e6d2581ccca47195b73a70fa5e690a897f36766a735d", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-04\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "0dede5205ccb97ca95a65809101dd03fc5a8724efaeaca1d97fffdd056f35a2f", + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-04' and clarification False match the scoped source and execution: The old request for VISIT-2134 was completed, but a later separate completion sets current confirmed date 2027-06-04. Do not replay the old request.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'already_applied' matches matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-34-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "bb3888f8633bfd9d570b9b17dae3bba0ffa9d57bed50c2e1bc09c268abe819db", + "result_file_sha256": "abd46c530745df8728e321ec83dc35accf4d0d8c2c4080a3ad91e1f119565685", + "receipt_file_sha256": "cce7ff5e86f7c7fa7e0dcd030bcf7d925ec20274438eec4841d48c0cdd54f6fe", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-04\",\"needs_clarification\":false}", + "raw_content_sha256": "6586f51a831b7caa739d8a82e7b4fa0ab9b8b951e604255d1abf7cf47d787c1f", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-04' and clarification False match the scoped source and execution: The old request for VISIT-2134 was completed, but a later separate completion sets current confirmed date 2027-06-04. Do not replay the old request.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'already_applied': matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-34-hi-m0-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "d92aa66ea283aace5f2ade818424acfa1d6c2abe3e91f5d7f852d2c5b279f99d", + "result_file_sha256": "7bdd78e5546dd17df78e3fca29b4d5de29ba0df0136ee40ba69bbdbf86da1095", + "receipt_file_sha256": "677b9be2b799977ab6f9f3f34eb3759544b8abde56f7394501c811ef3a224a5c", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-06-04\",\"needs_clarification\":false}", + "raw_content_sha256": "074f884a9c7bf6ea83350c614d232d4d544b0becc34b8d9fca92c68baeef18a4", + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-04' and clarification False match the scoped source and execution: The old request for VISIT-2134 was completed, but a later separate completion sets current confirmed date 2027-06-04. Do not replay the old request.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'already_applied' matches matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-34-hi-m1", + "case_id": "fresh-34-hi", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "language": "hi", + "parent_paired_file_sha256": "68feef9dba8f08ff486be48ad6edc96dd283b8d0225a4ffd03c721e6e526ee41", + "question": "VISIT-2134 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-34-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। VISIT-2134 को 2027-05-05 पर बदलने के मूल अनुरोध को उसके मूल पहचान नंबर से जांचें। वह पहले पूरा हो चुका है। बाद के अलग बदलाव से तारीख 2027-06-04 हुई। इस दोहराए गए अनुरोध से नया बदलाव वापस न करें।" + } + ], + "source_basis_reviewed": "The old request for VISIT-2134 was completed, but a later separate completion sets current confirmed date 2027-06-04. Do not replay the old request.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a2c08f6564bf8b6d", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2134" + }, + "response": { + "date": "2027-06-04", + "version": 7, + "closed": false + } + } + ], + "seed_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2134", + "date": "2027-05-05", + "expected_version": 5, + "idempotency_key": "REQUEST-34-CHANGE-23", + "approval_id": "AUTH-34-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-05", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2134", + "date": "2027-06-04", + "expected_version": 6, + "idempotency_key": "REQUEST-34-LATER-23", + "approval_id": "AUTH-34-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-04", + "version": 7 + } + } + ], + "initial_scoped_date": "2027-06-04", + "final_scoped_date": "2027-06-04", + "expected_from_source_and_actual_execution": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_raw_action_status": "already_applied", + "action_status_basis": "matching earlier completed request receipt; later distinct completion sets current date", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-34-hi-m1-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "7c842aab2a4b7e1b666d15e726f7954849eadbb95187550c652bedb3f89ac306", + "result_file_sha256": "042713423da807bcfef49b2ccab9db9d7aee0d0c696115936fca95c315d2e3bb", + "receipt_file_sha256": "7491fbeeb967af8391b357afa10ae028b866003f1fbd7efd0d6d21ef2bd7e8ec", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-04\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "0dede5205ccb97ca95a65809101dd03fc5a8724efaeaca1d97fffdd056f35a2f", + "raw": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-04' and clarification False match the scoped source and execution: The old request for VISIT-2134 was completed, but a later separate completion sets current confirmed date 2027-06-04. Do not replay the old request.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'already_applied' matches matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-34-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "aaac2fec8d23ca39735db48f417a72ce0826c3020fc031b94f25d3ecd92aeab0", + "result_file_sha256": "fb15a37da4fb1c488e9b16ecb891c11ca2d0d116496c521e1c600d6a55309e25", + "receipt_file_sha256": "bb15e196be6f052675b33e6f9088227b5cf63327bb6a5d0e57b049d5377a8244", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-04\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "d37ecd59c05b45200bc0a7393dd814c4e477d5816ec0bff4fe8eae3cc3ff484a", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-04' and clarification False match the scoped source and execution: The old request for VISIT-2134 was completed, but a later separate completion sets current confirmed date 2027-06-04. Do not replay the old request.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'already_applied': matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-34-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "c284b121c0fe7270f92d1f77dae67c4680fc059094f7d98a2133696754d488e0", + "result_file_sha256": "fb15a37da4fb1c488e9b16ecb891c11ca2d0d116496c521e1c600d6a55309e25", + "receipt_file_sha256": "fc26fbc55657fb44bb17af42f1a36dd62cb4ffe6e6af41eb9ce023c6970af3a7", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-04\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "d37ecd59c05b45200bc0a7393dd814c4e477d5816ec0bff4fe8eae3cc3ff484a", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-04' and clarification False match the scoped source and execution: The old request for VISIT-2134 was completed, but a later separate completion sets current confirmed date 2027-06-04. Do not replay the old request.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'already_applied': matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-35-hi-m0", + "case_id": "fresh-35-hi", + "model": "sarvam-105b", + "family": "completed_older_request", + "language": "hi", + "parent_paired_file_sha256": "0e936c02db942c903026db06b60bc24fedf0a04bd14c83c49584d9b5ab87675c", + "question": "VISIT-2135 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-35-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-35-1", + "role": "customer", + "content": "VISIT-2135 को 2027-05-06 पर बदलने के मूल अनुरोध को उसके मूल पहचान नंबर से जांचें।" + }, + { + "id": "note-35-2", + "role": "customer", + "content": "वह पहले पूरा हो चुका है। बाद के अलग बदलाव से तारीख 2027-06-05 हुई। इस दोहराए गए अनुरोध से नया बदलाव वापस न करें।" + } + ], + "source_basis_reviewed": "The old request for VISIT-2135 was completed, but a later separate completion sets current confirmed date 2027-06-05. Do not replay the old request.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-b96a0e75875ac0e9", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2135" + }, + "response": { + "date": "2027-06-05", + "version": 7, + "closed": false + } + } + ], + "seed_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2135", + "date": "2027-05-06", + "expected_version": 5, + "idempotency_key": "REQUEST-35-CHANGE-23", + "approval_id": "AUTH-35-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-06", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2135", + "date": "2027-06-05", + "expected_version": 6, + "idempotency_key": "REQUEST-35-LATER-23", + "approval_id": "AUTH-35-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-05", + "version": 7 + } + } + ], + "initial_scoped_date": "2027-06-05", + "final_scoped_date": "2027-06-05", + "expected_from_source_and_actual_execution": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_raw_action_status": "already_applied", + "action_status_basis": "matching earlier completed request receipt; later distinct completion sets current date", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-35-hi-m0-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "7d7eb94c3b3c654cc588f393ed8fc69e205ab2c5b4aa6d60ff950cfa295def6d", + "result_file_sha256": "296cb3c547ead0b0a5b30d3e7e77c560bc5d057059bb3ea526c0c2bd94e0c71e", + "receipt_file_sha256": "180cf12b2668ff547411720802e2046f9627da91e50e07de61922d03d2be2c01", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-06-05\",\"needs_clarification\":false}", + "raw_content_sha256": "0249c0fa5de0e696a7af9d1e4abe253d1025cd3505514ce0606fed5166205f00", + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-05' and clarification False match the scoped source and execution: The old request for VISIT-2135 was completed, but a later separate completion sets current confirmed date 2027-06-05. Do not replay the old request.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'already_applied' matches matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-35-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "837a60782b2402f13e5fc6f36420cca80eca5d9100bd433811517546e641d883", + "result_file_sha256": "800d592c65e097996bd4fe2bdf7496846a12d805e45f705bbbedf93d10502234", + "receipt_file_sha256": "63a78da579b4938d12e877773b7e896d0d773e4095c2492f07e58cfa1b2d385b", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-06-05\",\"needs_clarification\":false}", + "raw_content_sha256": "bd473f840733cd6c55c941558fb5f90e013e2362e67b3d382efa6d520470d780", + "raw": { + "action_status": "not_performed", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-05' and clarification False match the scoped source and execution: The old request for VISIT-2135 was completed, but a later separate completion sets current confirmed date 2027-06-05. Do not replay the old request.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_performed' does not match expected 'already_applied': matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-35-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "c7b56380640d5a0cb73c4c45b2a1cb71b5883410641042484f230a17ea83f92b", + "result_file_sha256": "b31d0f9b5f8fe5765afbcb1eaf44d89c8c5492e0a28b6d2d4ff5fe02f1c09f85", + "receipt_file_sha256": "39c0f95064beb93c0dd780242f61d397b237459bf92e912bd33624c2bf9d0a9f", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-05\",\"needs_clarification\":false}", + "raw_content_sha256": "a7a213c208ae83606f4551a683c7a4409c60837bc26f45ac19a6b96446e83525", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-05' and clarification False match the scoped source and execution: The old request for VISIT-2135 was completed, but a later separate completion sets current confirmed date 2027-06-05. Do not replay the old request.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'already_applied': matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-35-hi-m1", + "case_id": "fresh-35-hi", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "language": "hi", + "parent_paired_file_sha256": "d3e4e3d84999cf308fb98e03579dc84f000bd1d90e5dddc9e148e08d14e0560c", + "question": "VISIT-2135 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-35-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-35-1", + "role": "customer", + "content": "VISIT-2135 को 2027-05-06 पर बदलने के मूल अनुरोध को उसके मूल पहचान नंबर से जांचें।" + }, + { + "id": "note-35-2", + "role": "customer", + "content": "वह पहले पूरा हो चुका है। बाद के अलग बदलाव से तारीख 2027-06-05 हुई। इस दोहराए गए अनुरोध से नया बदलाव वापस न करें।" + } + ], + "source_basis_reviewed": "The old request for VISIT-2135 was completed, but a later separate completion sets current confirmed date 2027-06-05. Do not replay the old request.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-98ebf192e04d3cae", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2135" + }, + "response": { + "date": "2027-06-05", + "version": 7, + "closed": false + } + } + ], + "seed_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2135", + "date": "2027-05-06", + "expected_version": 5, + "idempotency_key": "REQUEST-35-CHANGE-23", + "approval_id": "AUTH-35-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-06", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2135", + "date": "2027-06-05", + "expected_version": 6, + "idempotency_key": "REQUEST-35-LATER-23", + "approval_id": "AUTH-35-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-05", + "version": 7 + } + } + ], + "initial_scoped_date": "2027-06-05", + "final_scoped_date": "2027-06-05", + "expected_from_source_and_actual_execution": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_raw_action_status": "already_applied", + "action_status_basis": "matching earlier completed request receipt; later distinct completion sets current date", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-35-hi-m1-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "8c34148b7d102a92b1577eae0049f9964c7a27629f3e03a22a7a83edd9389762", + "result_file_sha256": "10dca089506ffd53f9cf8d47ddf457b0596b2a838a3856eebeac46979cd679d9", + "receipt_file_sha256": "c189fc10570e266811d0b78a3ab27619cf67e65a4a5d998210f5f1be4b4afc7a", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-05\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "5ccdaa467bf10798802cf3b20863dd51a0c1536186cc3eecf0a779a2bd89cb44", + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-05' and clarification False match the scoped source and execution: The old request for VISIT-2135 was completed, but a later separate completion sets current confirmed date 2027-06-05. Do not replay the old request.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'already_applied' matches matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-35-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "aead7f720576f9d999bdf707a4ee182e61c54da57ce827da45e0d8ab95f67be2", + "result_file_sha256": "10dca089506ffd53f9cf8d47ddf457b0596b2a838a3856eebeac46979cd679d9", + "receipt_file_sha256": "0fa1e81efd348000056f92482b489f5f50887dbff9c8352771ed8dbb37213fb8", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-05\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "5ccdaa467bf10798802cf3b20863dd51a0c1536186cc3eecf0a779a2bd89cb44", + "raw": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-05' and clarification False match the scoped source and execution: The old request for VISIT-2135 was completed, but a later separate completion sets current confirmed date 2027-06-05. Do not replay the old request.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'already_applied' matches matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-35-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "7c6494befa0a2a8970faeccf2d48f06441a0b6280e5800e7bc4f406945a8ab7b", + "result_file_sha256": "a10e59f23e1bcbeb10d0bb5925482d8103b4a143246bb28f303d672b12eeb2f2", + "receipt_file_sha256": "bcef32f002a1672ff8d4d1b5c03b2bcbb9a55011346a14bdc71bdf4e588cb825", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-05\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "baee28406332678362f125cfdb084fb35a70df7dda2b9c89c9ec8091f79aede6", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-05' and clarification False match the scoped source and execution: The old request for VISIT-2135 was completed, but a later separate completion sets current confirmed date 2027-06-05. Do not replay the old request.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'already_applied': matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-36-hi-m0", + "case_id": "fresh-36-hi", + "model": "sarvam-105b", + "family": "completed_older_request", + "language": "hi", + "parent_paired_file_sha256": "f2e2e33982bd0af877ef57f19ecd9113557e0bc8520f0da61005f6eddd949380", + "question": "VISIT-2136 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-36-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। VISIT-2136 को 2027-05-07 पर बदलने के मूल अनुरोध को उसके मूल पहचान नंबर से जांचें। वह पहले पूरा हो चुका है। बाद के अलग बदलाव से तारीख 2027-06-06 हुई। इस दोहराए गए अनुरोध से नया बदलाव वापस न करें।" + } + ], + "source_basis_reviewed": "The old request for VISIT-2136 was completed, but a later separate completion sets current confirmed date 2027-06-06. Do not replay the old request.", + "question_target": "actual_current_date", + "actual_tool_trace": [], + "seed_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2136", + "date": "2027-05-07", + "expected_version": 5, + "idempotency_key": "REQUEST-36-CHANGE-23", + "approval_id": "AUTH-36-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-07", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2136", + "date": "2027-06-06", + "expected_version": 6, + "idempotency_key": "REQUEST-36-LATER-23", + "approval_id": "AUTH-36-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-06", + "version": 7 + } + } + ], + "initial_scoped_date": "2027-06-06", + "final_scoped_date": "2027-06-06", + "expected_from_source_and_actual_execution": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_raw_action_status": "already_applied", + "action_status_basis": "matching earlier completed request receipt; later distinct completion sets current date", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-36-hi-m0-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "3bfe59ee312d2d0400a0b1d6f1979e3ba6796a524317fceea79a4ede10600ee5", + "result_file_sha256": "e355c143c1132a08d0a14fe326d14a7af5058d4c24b0abb602658e8728b625ab", + "receipt_file_sha256": "39e4de0b9ea0c84f98a79dc4593a75ca278dcaf54326877566f50ce991161fe6", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-06-06\",\"needs_clarification\":false}", + "raw_content_sha256": "18cf2fff3ab82ce49fc9cdefc19756ccd84c58bb39eea867844969642ef43c0b", + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-06' and clarification False match the scoped source and execution: The old request for VISIT-2136 was completed, but a later separate completion sets current confirmed date 2027-06-06. Do not replay the old request.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'already_applied' matches matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-36-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "5c6e4471db7db30a4b3a946624063439c5311f4d1b9a56f8c577e46b83ff304b", + "result_file_sha256": "f09ee14139dffa4254eb8299e58575fdc38b4acb79393af281e81167c8fe2c0b", + "receipt_file_sha256": "bb496b9b5f66a2824eec1ad0ca19e2140619e7f9d59df5a5c85351681b831337", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-06\",\"needs_clarification\":false}", + "raw_content_sha256": "2803662ff40dd4f63615eb045495ce5c39d4468bff1e77d7ea10ac817822af70", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-06' and clarification False match the scoped source and execution: The old request for VISIT-2136 was completed, but a later separate completion sets current confirmed date 2027-06-06. Do not replay the old request.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'already_applied': matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-36-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "a460ea9e8a957879585796b8e10804effdbd940b0d6c6be85a3498547bd48eb9", + "result_file_sha256": "e355c143c1132a08d0a14fe326d14a7af5058d4c24b0abb602658e8728b625ab", + "receipt_file_sha256": "4e94e7a58e8bf6fa8133c862b0a60d3df40fa70707c1809560cdf439f802e227", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-06-06\",\"needs_clarification\":false}", + "raw_content_sha256": "18cf2fff3ab82ce49fc9cdefc19756ccd84c58bb39eea867844969642ef43c0b", + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-06' and clarification False match the scoped source and execution: The old request for VISIT-2136 was completed, but a later separate completion sets current confirmed date 2027-06-06. Do not replay the old request.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'already_applied' matches matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-36-hi-m1", + "case_id": "fresh-36-hi", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "language": "hi", + "parent_paired_file_sha256": "04a1e7abde3b11aa754a9dbb381ed33fc6bf5979d9f0a261e4b1ad8f96075784", + "question": "VISIT-2136 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-36-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। VISIT-2136 को 2027-05-07 पर बदलने के मूल अनुरोध को उसके मूल पहचान नंबर से जांचें। वह पहले पूरा हो चुका है। बाद के अलग बदलाव से तारीख 2027-06-06 हुई। इस दोहराए गए अनुरोध से नया बदलाव वापस न करें।" + } + ], + "source_basis_reviewed": "The old request for VISIT-2136 was completed, but a later separate completion sets current confirmed date 2027-06-06. Do not replay the old request.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-998a855f7e37ec0d", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2136" + }, + "response": { + "date": "2027-06-06", + "version": 7, + "closed": false + } + } + ], + "seed_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2136", + "date": "2027-05-07", + "expected_version": 5, + "idempotency_key": "REQUEST-36-CHANGE-23", + "approval_id": "AUTH-36-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-07", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2136", + "date": "2027-06-06", + "expected_version": 6, + "idempotency_key": "REQUEST-36-LATER-23", + "approval_id": "AUTH-36-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-06", + "version": 7 + } + } + ], + "initial_scoped_date": "2027-06-06", + "final_scoped_date": "2027-06-06", + "expected_from_source_and_actual_execution": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_raw_action_status": "already_applied", + "action_status_basis": "matching earlier completed request receipt; later distinct completion sets current date", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-36-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "9d9aabdc5d244f95b2a53759246ca8b8b0aa3b82e58feca909a11e21c94174bf", + "result_file_sha256": "5766d8a3d2817eb1b9e489c5c75934249fc394e5c0209fb689239f6b03244dca", + "receipt_file_sha256": "c4876225814615d29ae4fb137ec4b1d182965a8114c7784bba5a896c36679bb7", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-06\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "efc6a4077028e288334cb3a79d28cec7f9704053928331652c0ff2cd591ea443", + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-06' and clarification False match the scoped source and execution: The old request for VISIT-2136 was completed, but a later separate completion sets current confirmed date 2027-06-06. Do not replay the old request.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'already_applied' matches matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-36-hi-m1-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "693edcc4106e5f7cc4c9d4df081bf3117691617c1ffc73e92528408d390c7c67", + "result_file_sha256": "5766d8a3d2817eb1b9e489c5c75934249fc394e5c0209fb689239f6b03244dca", + "receipt_file_sha256": "97c054cd120f7689544555394624b5178380254d5aaff1e7473a9358dfe8461b", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-06\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "efc6a4077028e288334cb3a79d28cec7f9704053928331652c0ff2cd591ea443", + "raw": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-06' and clarification False match the scoped source and execution: The old request for VISIT-2136 was completed, but a later separate completion sets current confirmed date 2027-06-06. Do not replay the old request.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'already_applied' matches matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-36-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "4c3cabf375954020bb9db2fc8aa96ac7e1d6b5f19cce59b6c31d2e7f07ff9188", + "result_file_sha256": "807ff01b1b38bc10bcd9060bd0d3b1ae9c42d960b9a0482d8c03411278d3cf2d", + "receipt_file_sha256": "f75d49337df2a0ebd522dbef7cd8c2131cd4cbc9423ac936bc9a9f13d2f695a4", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-06\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "861aec082087961b93d13c7f30faddb78998f3b9bc4544a583396aa8fae6587a", + "raw": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "application_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-06-06' and clarification False match the scoped source and execution: The old request for VISIT-2136 was completed, but a later separate completion sets current confirmed date 2027-06-06. Do not replay the old request.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'already_applied': matching earlier completed request receipt; later distinct completion sets current date.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'already_applied' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-37-hi-m0", + "case_id": "fresh-37-hi", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "language": "hi", + "parent_paired_file_sha256": "82b052aceedd9d0c9b2573436bd29f293227803a9f6eb3db6b00b09db04e0112", + "question": "VISIT-2137 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-37-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-37-1", + "role": "customer", + "content": "अब VISIT-2137 के लिए 2027-06-07 चाहिए।" + }, + { + "id": "note-37-2", + "role": "customer", + "content": "दिया गया अनुरोध पहचान नंबर पहले 2027-05-08 के पूरे हो चुके बदलाव के लिए इस्तेमाल हुआ था। उसी नंबर से अलग बदलाव लागू न करें।" + } + ], + "source_basis_reviewed": "The requested new date for VISIT-2137 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-08.", + "question_target": "actual_current_date", + "actual_tool_trace": [], + "seed_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2137", + "date": "2027-05-08", + "expected_version": 5, + "idempotency_key": "REQUEST-37-CHANGE-24", + "approval_id": "AUTH-37-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-08", + "version": 6 + } + } + ], + "initial_scoped_date": "2027-05-08", + "final_scoped_date": "2027-05-08", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_raw_action_status": "not_performed", + "action_status_basis": "key belongs to a different prior completion, and no new matching update was applied", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-37-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "0b67eb6695c21085b25617e88f612ab78dff50a27f64cca013310ac014cc46b0", + "result_file_sha256": "9b729514e56b0e35f809adf52a6727f55fd4b6fdfc58f34533b6ccf566e2fcbd", + "receipt_file_sha256": "588a18455f1bc3bd1948b8a3c928bd87a5763e7f06ad394134e54f23f62f6ba7", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"rejected\",\"answer\":\"2027-05-08\",\"needs_clarification\":false}", + "raw_content_sha256": "5896440187e98038ddc3fdc3a9312a3e2b727b4f0f83b61c784979763532c36e", + "raw": { + "action_status": "rejected", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-08' and clarification False match the scoped source and execution: The requested new date for VISIT-2137 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-08.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'rejected' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-37-hi-m0-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "503819303c5d2b13cffa2b10e61bae4cdab905a0e6cced7d4e5e1a121182b180", + "result_file_sha256": "aff48058a24de4a1db9e0fa8c903c6c318fd62f91bba7bbd8e5d3168a70bc729", + "receipt_file_sha256": "919d2652b103827dde9bcf9d9f00f4260b2e95c027b7898a20c30e307f33e89b", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-08\",\"needs_clarification\":false}", + "raw_content_sha256": "2292869e10a5a50d27fb1494433690cd1b3ef9e542b822350d3bd16e394d5c9e", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-08' and clarification False match the scoped source and execution: The requested new date for VISIT-2137 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-08.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'already_applied' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-37-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "44c5b06f4c1cd75c70ee4ace98f89e242cc0e4ae2c26f893cc3d2094ab7e57f5", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "receipt_file_sha256": "746aa2d6430e3e6d93e264cedc6a0db8b0b112ae5956d651303355d13fe033d8", + "terminal": "incomplete", + "finish_reason": "length", + "raw_content": null, + "raw_content_sha256": null, + "raw": null, + "application_output": null, + "answer_source_role": null, + "factual_verdict": "invalid_or_missing", + "factual_explanation": "No valid parsed final answer to assess.", + "raw_status_verdict": "invalid_or_missing", + "raw_status_explanation": "None does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "invalid_or_missing", + "projected_status_explanation": "Application status None does not match expected 'not_performed' from actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-37-hi-m1", + "case_id": "fresh-37-hi", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "language": "hi", + "parent_paired_file_sha256": "688a24bb74b72955c8f88980905f61a5c368a61b7e19ae651ef4ed9efd54a09a", + "question": "VISIT-2137 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-37-0", + "role": "customer", + "content": "मैं मीरा हूं; यह मेरी बुकिंग के बारे में अगला संदेश है।" + }, + { + "id": "note-37-1", + "role": "customer", + "content": "अब VISIT-2137 के लिए 2027-06-07 चाहिए।" + }, + { + "id": "note-37-2", + "role": "customer", + "content": "दिया गया अनुरोध पहचान नंबर पहले 2027-05-08 के पूरे हो चुके बदलाव के लिए इस्तेमाल हुआ था। उसी नंबर से अलग बदलाव लागू न करें।" + } + ], + "source_basis_reviewed": "The requested new date for VISIT-2137 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-08.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8ae33877d389f073", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2137" + }, + "response": { + "date": "2027-05-08", + "version": 6, + "closed": false + } + } + ], + "seed_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2137", + "date": "2027-05-08", + "expected_version": 5, + "idempotency_key": "REQUEST-37-CHANGE-24", + "approval_id": "AUTH-37-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-08", + "version": 6 + } + } + ], + "initial_scoped_date": "2027-05-08", + "final_scoped_date": "2027-05-08", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_raw_action_status": "not_performed", + "action_status_basis": "key belongs to a different prior completion, and no new matching update was applied", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-37-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "2b33acf90b75e0a1ba9b4933f59c24997e00ca6b5b5d229edd68916882f509b7", + "result_file_sha256": "2c3fdbb6856d232762072d5920f57471f718b0f9b72563994c9e7d9b1e6852d7", + "receipt_file_sha256": "f0808c3e0fd87caa5ebe1c6f9185c7214d181dccac5fe3a1cc14b1216ee0200b", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-05-08\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "f025c2199b98f91f9bd992c7a9c8f85467b1b8ebc43dea9ba9165d4afb289c52", + "raw": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-08' and clarification False match the scoped source and execution: The requested new date for VISIT-2137 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-08.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_performed' matches key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-37-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "1d239a42e04710a53e7f3fc2303173536dd9f6fa0fec2941d5382e6a58e39026", + "result_file_sha256": "417418804f246865317d445869ff277d57e1f417198b5646e44534270893ef1f", + "receipt_file_sha256": "556421f09b26b263c54302cf6541f0d4ac8b19eaa0097dba53c91a71d96650a1", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-08\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "404a69ff9e54f916db69a6ce6d90ff60c79f4d89c14a3a7f677443f9092339cc", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-08' and clarification False match the scoped source and execution: The requested new date for VISIT-2137 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-08.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-37-hi-m1-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "0f5be8b2daaf0e144ed80ffcb2d2948c3f4675504217ce943950e2212ef186c9", + "result_file_sha256": "417418804f246865317d445869ff277d57e1f417198b5646e44534270893ef1f", + "receipt_file_sha256": "2f54fa75e78f8c23ae6d5dca1395e66243e39d3a9cd5a36063265b9e944dd98b", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-08\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "404a69ff9e54f916db69a6ce6d90ff60c79f4d89c14a3a7f677443f9092339cc", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-08' and clarification False match the scoped source and execution: The requested new date for VISIT-2137 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-08.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-38-hi-m0", + "case_id": "fresh-38-hi", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "language": "hi", + "parent_paired_file_sha256": "e392cb4c62b29a0b4f5792563c73271041880aad77217edb74b9f613ee66d8ac", + "question": "VISIT-2138 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-38-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। अब VISIT-2138 के लिए 2027-06-08 चाहिए। दिया गया अनुरोध पहचान नंबर पहले 2027-05-09 के पूरे हो चुके बदलाव के लिए इस्तेमाल हुआ था। उसी नंबर से अलग बदलाव लागू न करें।" + } + ], + "source_basis_reviewed": "The requested new date for VISIT-2138 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-09.", + "question_target": "actual_current_date", + "actual_tool_trace": [], + "seed_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2138", + "date": "2027-05-09", + "expected_version": 5, + "idempotency_key": "REQUEST-38-CHANGE-24", + "approval_id": "AUTH-38-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-09", + "version": 6 + } + } + ], + "initial_scoped_date": "2027-05-09", + "final_scoped_date": "2027-05-09", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_raw_action_status": "not_performed", + "action_status_basis": "key belongs to a different prior completion, and no new matching update was applied", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_lossless": { + "job_id": "confirm-fresh-38-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "request_sha256": "8d47d3f108e2905f55c66770467061e6fb3f72c09c51ec0faf1caf04a1e2c79b", + "result_file_sha256": "7179a53ab85d138478748672aef58d579781c7908dfb61f6e82b564ed4e415a5", + "receipt_file_sha256": "961f840e84aadbb9fb3afe8e121d8b5c9c03cc224987931db12f63ea9ed2b186", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-05-09\",\"needs_clarification\":false}", + "raw_content_sha256": "c7312c1717b013964cb64287786966619d1cfdcbdbef5e9216bbf052272c02b0", + "raw": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-09' and clarification False match the scoped source and execution: The requested new date for VISIT-2138 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-09.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_performed' matches key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-38-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "request_sha256": "d1e1db7959eb2237a2656e05adab8debe6475f48e6e055db28ffcc1675b1cc33", + "result_file_sha256": "7179a53ab85d138478748672aef58d579781c7908dfb61f6e82b564ed4e415a5", + "receipt_file_sha256": "6463dfe47d2ab829a4c88839239c554b439991af1da862b847508f21cac6551f", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-05-09\",\"needs_clarification\":false}", + "raw_content_sha256": "c7312c1717b013964cb64287786966619d1cfdcbdbef5e9216bbf052272c02b0", + "raw": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-09' and clarification False match the scoped source and execution: The requested new date for VISIT-2138 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-09.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_performed' matches key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-38-hi-m0-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "43d49158c33c0956600ef768477fdbd217c4b27c562a0df550b13f833db25abe", + "result_file_sha256": "a83af413afe7cddcaa541b7076f09d4924ecc2e0a192a002771a74e3c3fadc75", + "receipt_file_sha256": "7e207881291eca18fe88fae4b3af14207be8f996f6f47898262483bd50f1ff96", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-09\",\"needs_clarification\":false}", + "raw_content_sha256": "0c4812369e0d55fa35c84fa8fee15ea63e76b40cbe4d6362cf99bfea51d639e0", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-09' and clarification False match the scoped source and execution: The requested new date for VISIT-2138 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-09.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'already_applied' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-38-hi-m1", + "case_id": "fresh-38-hi", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "language": "hi", + "parent_paired_file_sha256": "fda3a32777e4d82de798ed552d284cb3642082f4be3a877fed456a03adf2c987", + "question": "VISIT-2138 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-38-0", + "role": "customer", + "content": "मैं अर्जुन हूं। कृपया दोनों संदेश क्रम से पढ़ें। अब VISIT-2138 के लिए 2027-06-08 चाहिए। दिया गया अनुरोध पहचान नंबर पहले 2027-05-09 के पूरे हो चुके बदलाव के लिए इस्तेमाल हुआ था। उसी नंबर से अलग बदलाव लागू न करें।" + } + ], + "source_basis_reviewed": "The requested new date for VISIT-2138 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-09.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8438531ce0aa7dcf", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2138" + }, + "response": { + "date": "2027-05-09", + "version": 6, + "closed": false + } + } + ], + "seed_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2138", + "date": "2027-05-09", + "expected_version": 5, + "idempotency_key": "REQUEST-38-CHANGE-24", + "approval_id": "AUTH-38-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-09", + "version": 6 + } + } + ], + "initial_scoped_date": "2027-05-09", + "final_scoped_date": "2027-05-09", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_raw_action_status": "not_performed", + "action_status_basis": "key belongs to a different prior completion, and no new matching update was applied", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-38-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "0ebc1fcee3f14a592a12b04a4c0dcbee7ea274b2b15d7f4fac6c71687410cd81", + "result_file_sha256": "7bb3b716e7f26e456befff8ad0778da3cde694aa22a37e4e06202fb80e43454b", + "receipt_file_sha256": "cd8392e6c9698fdb54d46d3dca338352b7ed0d247989560f417168973c3ad075", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-09\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "69e809991e2e6516871589d05968ed8d6dbfb4311f9ef072ce2282e6760d4834", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-09' and clarification False match the scoped source and execution: The requested new date for VISIT-2138 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-09.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-38-hi-m1-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "4cc12e8662921482cd646c491a817fe356f9df2d28918462bbfb0bff2a0ed256", + "result_file_sha256": "7f2bd78eb4f64dd1c52dcce5e859d45a219d03cec1305e5fe4c87290e091d563", + "receipt_file_sha256": "2a302bbdc2020423903c27103852562c2fb2ab2cdcb3870b1fdf0823928f64e0", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-05-09\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "a0170e7117cb7097fb511d6901f9ee5ca504b7e1f699524e72b088791eeada2c", + "raw": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-09' and clarification False match the scoped source and execution: The requested new date for VISIT-2138 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-09.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_performed' matches key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-38-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "1c5c792bd997d830e95a92d089d2941f26c7ee5e15d1ececc4bb24c6401edf46", + "result_file_sha256": "f6ce87a4a87a8702c1234da7b392f55d90d8148fb0470cd124de892151ecf6b4", + "receipt_file_sha256": "36a4ac2b134412e4f711456c8b4d5c593ce4c0540bee4149028135bcb5ec771b", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-09\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "d3da643b8c5b470ad5dd15c3b176f33d49912aa242df60d066d9164728fdd526", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-09", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-09' and clarification False match the scoped source and execution: The requested new date for VISIT-2138 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-09.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'already_applied' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-39-hi-m0", + "case_id": "fresh-39-hi", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "language": "hi", + "parent_paired_file_sha256": "77bfcfaf3de967fd520cb3336cfc9d1a401f531287d36a8c38efed8c9d288735", + "question": "VISIT-2139 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-39-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-39-1", + "role": "customer", + "content": "अब VISIT-2139 के लिए 2027-06-09 चाहिए।" + }, + { + "id": "note-39-2", + "role": "customer", + "content": "दिया गया अनुरोध पहचान नंबर पहले 2027-05-10 के पूरे हो चुके बदलाव के लिए इस्तेमाल हुआ था। उसी नंबर से अलग बदलाव लागू न करें।" + } + ], + "source_basis_reviewed": "The requested new date for VISIT-2139 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-10.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-ac5abba8a793a05a", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2139" + }, + "response": { + "date": "2027-05-10", + "version": 6, + "closed": false + } + } + ], + "seed_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2139", + "date": "2027-05-10", + "expected_version": 5, + "idempotency_key": "REQUEST-39-CHANGE-24", + "approval_id": "AUTH-39-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-10", + "version": 6 + } + } + ], + "initial_scoped_date": "2027-05-10", + "final_scoped_date": "2027-05-10", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_raw_action_status": "not_performed", + "action_status_basis": "key belongs to a different prior completion, and no new matching update was applied", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-39-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "fe2298a01dc11c191a722517c89e1ad64f176d4f7c200709c57e68f9c6ebe42a", + "result_file_sha256": "96b42d2cc3de69828ffb97c140cf9d47de4b7c1c2276847078932bfeadb2f1f8", + "receipt_file_sha256": "00c7d610f235496cc505a7f40b398c415be26275b1f1df0c6f774d25d61e9ec8", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-10\",\"needs_clarification\":false}", + "raw_content_sha256": "b3ec4557109cc643fdfe76911b4b72140a78dbcd5ad0dc05a75ced885889e007", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-10' and clarification False match the scoped source and execution: The requested new date for VISIT-2139 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-10.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-39-hi-m0-source-chronological", + "arm": "chronological", + "position": 1, + "request_sha256": "0be62b9976eb361910e0f8b3ad12e5c1fe4c2092b41fa1113e0485fe46428871", + "result_file_sha256": "96b42d2cc3de69828ffb97c140cf9d47de4b7c1c2276847078932bfeadb2f1f8", + "receipt_file_sha256": "6f6e6d1152357903cdb5b90e6dff393c9032d5d8354bef2718faaeb15fc7adcd", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-10\",\"needs_clarification\":false}", + "raw_content_sha256": "b3ec4557109cc643fdfe76911b4b72140a78dbcd5ad0dc05a75ced885889e007", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-10' and clarification False match the scoped source and execution: The requested new date for VISIT-2139 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-10.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-39-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "request_sha256": "7f3a55455745acd55466d34c3d3c9194aa1ba6c33e49fbd4ff0fe88e9467ca65", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "receipt_file_sha256": "c7b9a4603b19e9635c743cd3b49441c2d72bf44740c8e50755891c136e8d9e95", + "terminal": "incomplete", + "finish_reason": "length", + "raw_content": null, + "raw_content_sha256": null, + "raw": null, + "application_output": null, + "answer_source_role": null, + "factual_verdict": "invalid_or_missing", + "factual_explanation": "No valid parsed final answer to assess.", + "raw_status_verdict": "invalid_or_missing", + "raw_status_explanation": "None does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "invalid_or_missing", + "projected_status_explanation": "Application status None does not match expected 'not_performed' from actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-39-hi-m1", + "case_id": "fresh-39-hi", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "language": "hi", + "parent_paired_file_sha256": "da2e99783d8a0031a57e50d33c7c1aa389712ba19a81ffec3a633a4d0e3bbdf3", + "question": "VISIT-2139 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-39-0", + "role": "customer", + "content": "मेरा नाम काव्या है। पहले पुराना संदेश है, फिर अगला संदेश।" + }, + { + "id": "note-39-1", + "role": "customer", + "content": "अब VISIT-2139 के लिए 2027-06-09 चाहिए।" + }, + { + "id": "note-39-2", + "role": "customer", + "content": "दिया गया अनुरोध पहचान नंबर पहले 2027-05-10 के पूरे हो चुके बदलाव के लिए इस्तेमाल हुआ था। उसी नंबर से अलग बदलाव लागू न करें।" + } + ], + "source_basis_reviewed": "The requested new date for VISIT-2139 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-10.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-925e3bea22c1fc13", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2139" + }, + "response": { + "date": "2027-05-10", + "version": 6, + "closed": false + } + } + ], + "seed_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2139", + "date": "2027-05-10", + "expected_version": 5, + "idempotency_key": "REQUEST-39-CHANGE-24", + "approval_id": "AUTH-39-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-10", + "version": 6 + } + } + ], + "initial_scoped_date": "2027-05-10", + "final_scoped_date": "2027-05-10", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_raw_action_status": "not_performed", + "action_status_basis": "key belongs to a different prior completion, and no new matching update was applied", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-39-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "b0b7b8af1c92e38aeb4c14793bde6f42f3611cb80854b82a962412054a412066", + "result_file_sha256": "f2371cd3e60dbad81739ea12fffa281dda1158ef039ec6a8984449c96e971832", + "receipt_file_sha256": "5d400a4644fb830d1f982ad7653a8cb0e2baa670f11e0145cd494a11666fce15", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-10\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "057425dcb1fdd6d6ccad3bd4cad69751520540732ee2ff9949cb04440da4e058", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-10' and clarification False match the scoped source and execution: The requested new date for VISIT-2139 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-10.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'already_applied' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-39-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "989aa669254b16cb0e9d50cf4756435da2574846fccee4a51d69a5147da16012", + "result_file_sha256": "f2371cd3e60dbad81739ea12fffa281dda1158ef039ec6a8984449c96e971832", + "receipt_file_sha256": "c750b76d38ef2ab0ae57421a23b8643844acd7a6ca35b036f5ce926fd9bd8948", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-10\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "057425dcb1fdd6d6ccad3bd4cad69751520540732ee2ff9949cb04440da4e058", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-10' and clarification False match the scoped source and execution: The requested new date for VISIT-2139 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-10.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'already_applied' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-39-hi-m1-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "44a7099fe5fc6e8440db855a48659512d53eab8cda0d4020597aeaca20e645be", + "result_file_sha256": "f2371cd3e60dbad81739ea12fffa281dda1158ef039ec6a8984449c96e971832", + "receipt_file_sha256": "4c62f571f362d7b677ff465181d4476d8af8ec8eb14587e407f445ff3726f5f6", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-10\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "057425dcb1fdd6d6ccad3bd4cad69751520540732ee2ff9949cb04440da4e058", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-10", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-10' and clarification False match the scoped source and execution: The requested new date for VISIT-2139 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-10.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'already_applied' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-40-hi-m0", + "case_id": "fresh-40-hi", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "language": "hi", + "parent_paired_file_sha256": "61879fa774dfeee20c10c8e3b2594895c2761965f4f94875f927b555b49934b4", + "question": "VISIT-2140 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-40-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। अब VISIT-2140 के लिए 2027-06-10 चाहिए। दिया गया अनुरोध पहचान नंबर पहले 2027-05-11 के पूरे हो चुके बदलाव के लिए इस्तेमाल हुआ था। उसी नंबर से अलग बदलाव लागू न करें।" + } + ], + "source_basis_reviewed": "The requested new date for VISIT-2140 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-11.", + "question_target": "actual_current_date", + "actual_tool_trace": [], + "seed_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2140", + "date": "2027-05-11", + "expected_version": 5, + "idempotency_key": "REQUEST-40-CHANGE-24", + "approval_id": "AUTH-40-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-11", + "version": 6 + } + } + ], + "initial_scoped_date": "2027-05-11", + "final_scoped_date": "2027-05-11", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_raw_action_status": "not_performed", + "action_status_basis": "key belongs to a different prior completion, and no new matching update was applied", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-40-hi-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "request_sha256": "6fd382f53c23f243a7bd8d4ad2db21a21aaa6ed52e96d6c8b0f15b8b47ef4dc2", + "result_file_sha256": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "receipt_file_sha256": "10c79e123e903f766c19947462248f820b67161d466576657c5cffb4c7438e5d", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "raw_content_sha256": "2362fc24e9ad026848ab17e8b29cf8890528a030252a998e2bd4367263250458", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-11' and clarification False match the scoped source and execution: The requested new date for VISIT-2140 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-11.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-40-hi-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "01bf3b2a5237bb70769fce8c7d84a169179224ee62de34683dd52722de6074dd", + "result_file_sha256": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "receipt_file_sha256": "b2c9da9333b6e4518411b24c567e12dd6a4c171fc8c47fa3139265af52d9d83e", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "raw_content_sha256": "9dac33ab5e4882ec69311a19f72c1330ec8f057e21a912a81f03d812c2e99127", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-11' and clarification False match the scoped source and execution: The requested new date for VISIT-2140 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-11.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'already_applied' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "chronological": { + "job_id": "confirm-fresh-40-hi-m0-source-chronological", + "arm": "chronological", + "position": 2, + "request_sha256": "6546f682923fd24926bf2f20036be8f2dabe467b3c5340c8f4e8aef3afeca9db", + "result_file_sha256": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "receipt_file_sha256": "760002f2459d1e970b1ee1568d3ec9b2fa43effc3fea3533e5daae87bd45a02e", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "raw_content_sha256": "2362fc24e9ad026848ab17e8b29cf8890528a030252a998e2bd4367263250458", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-11' and clarification False match the scoped source and execution: The requested new date for VISIT-2140 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-11.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + }, + { + "history_id": "confirm-fresh-40-hi-m1", + "case_id": "fresh-40-hi", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "language": "hi", + "parent_paired_file_sha256": "d82c493b8a6bea420a3e4a9eaaf3a6815bc33dda618944db98e9ee684a04b75a", + "question": "VISIT-2140 के दिए गए बदलाव के अनुरोध को संभालें। अभी वास्तव में कौन सी तारीख पक्की है?", + "source_messages": [ + { + "id": "note-40-0", + "role": "customer", + "content": "मैं रवि हूं। इन दोनों संदेशों में दी गई बुकिंग का जिक्र है। अब VISIT-2140 के लिए 2027-06-10 चाहिए। दिया गया अनुरोध पहचान नंबर पहले 2027-05-11 के पूरे हो चुके बदलाव के लिए इस्तेमाल हुआ था। उसी नंबर से अलग बदलाव लागू न करें।" + } + ], + "source_basis_reviewed": "The requested new date for VISIT-2140 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-11.", + "question_target": "actual_current_date", + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8f93143f93f8d754", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2140" + }, + "response": { + "date": "2027-05-11", + "version": 6, + "closed": false + } + } + ], + "seed_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2140", + "date": "2027-05-11", + "expected_version": 5, + "idempotency_key": "REQUEST-40-CHANGE-24", + "approval_id": "AUTH-40-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-11", + "version": 6 + } + } + ], + "initial_scoped_date": "2027-05-11", + "final_scoped_date": "2027-05-11", + "expected_from_source_and_actual_execution": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_raw_action_status": "not_performed", + "action_status_basis": "key belongs to a different prior completion, and no new matching update was applied", + "gold_ideal_answer_differs_from_actual_execution": false, + "source_gold_ambiguous": false, + "source_ambiguity_flags": [], + "actor_prose_treatment_present": true, + "reports": { + "chronological": { + "job_id": "confirm-fresh-40-hi-m1-source-chronological", + "arm": "chronological", + "position": 0, + "request_sha256": "b5aade5c4944632b30a1227f069c8e0af77bf73622cbe3ecd2e502656277eff7", + "result_file_sha256": "e154d7d3fea97d5d74e19e49727b25d73019ead962c7b21da19391b8aa91e911", + "receipt_file_sha256": "8d0ba6a89d0239aec2168c448bcbbf21d390e02878d22973408ae5497bcc5b49", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "727a7f58f51aa4f83de71e27cab3c77950ad8058f008e847b9b26fc4f5c24860", + "raw": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-11' and clarification False match the scoped source and execution: The requested new date for VISIT-2140 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-11.", + "raw_status_verdict": "correct", + "raw_status_explanation": "'not_performed' matches key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_lossless": { + "job_id": "confirm-fresh-40-hi-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "request_sha256": "344def310523660eae21887b36d0ace33bb01df87ba13a4498ba5b45e2aaa7f7", + "result_file_sha256": "fca49d1de4a1905da389c1cef4610aee77922b07c9c45ea909d59d69c71cb0ab", + "receipt_file_sha256": "28401650151f0ee22a739194181dde37d7a195dffb68c6db9a49acec3dbe135d", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "c7f328d78597435701fad57703175eeab15cace2a91f54b5d85e68d150fb7bca", + "raw": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-11' and clarification False match the scoped source and execution: The requested new date for VISIT-2140 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-11.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'already_applied' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + }, + "grouped_actor_prose_withheld": { + "job_id": "confirm-fresh-40-hi-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "request_sha256": "58d6924e10f30e02dd690805382dd90c366e30a04505a13b638e2cc63203a249", + "result_file_sha256": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "receipt_file_sha256": "7247ed6d19ec1aafc278617d26cfefa360b208cde27ed3808cf43e77d5167c08", + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "raw_content_sha256": "2655156e40334761b11243498c952a7bfa50bffe056b93d1424ef09be4cb336c", + "raw": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "application_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "answer_source_role": "correct_target", + "factual_verdict": "correct", + "factual_explanation": "Answer '2027-05-11' and clarification False match the scoped source and execution: The requested new date for VISIT-2140 reuses a key committed to a different older change. No new matching update occurs; current confirmed date is 2027-05-11.", + "raw_status_verdict": "incorrect", + "raw_status_explanation": "'not_requested' does not match expected 'not_performed': key belongs to a different prior completion, and no new matching update was applied.", + "projected_status_verdict": "correct", + "projected_status_explanation": "Application status 'not_performed' matches actual receipts and intent.", + "output_ambiguity_flags": [] + } + } + } + ], + "limitations": "Assistant-only Hindi judgment on synthetic development cases. Direct source reading and saved receipt/state cross-checks inform each verdict; no independent native speaker review or fresh task generalization. Literal date/clarification review is cross-checked against direct source reading, saved action receipts and post-action state. Every discrepancy and correct-match cohort was inspected by the assistant before publication; independent native-speaker review remains absent." +} diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/reviews/telugu-review.json b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/reviews/telugu-review.json new file mode 100644 index 0000000..faec38a --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/reviews/telugu-review.json @@ -0,0 +1,18868 @@ +{ + "status": "ASSISTANT_TELUGU_LANGUAGE_REVIEW_COMPLETE", + "created_at_utc": "2026-10-14T02:43:24+00:00", + "phase": "2026-10-13-source-interpretation-full", + "language": "te", + "independent_native_human_review": false, + "scope": { + "source_cases": 40, + "native_histories": 80, + "new_reporter_slots": 240, + "models": 2, + "arms": [ + "chronological", + "grouped_lossless", + "grouped_actor_prose_withheld" + ], + "all_slots_reviewed": true + }, + "provenance": { + "manifest_file_sha256": "ba66788d25b439471ba396c8aa2f56a1b4f136cd60cdbf0a04a40e2484bbf650", + "execution_file_sha256": "9f322fddafe182d8f0f3e5af2a34575caa4e7db3ea0286339665540419e376a3", + "independent_raw_aggregation_file_sha256": "7e4c331123f726895c37ffbb527db591a77f62e8c5db5256e5c487d255e015fc", + "parent_telugu_semantic_review_file_sha256": "ee00eedca16cfd3efe74b6e0e21f4c1ac8e43626b54c23b46f79f54909c8996e" + }, + "reviewed_case_ids": [ + "fresh-01-te", + "fresh-02-te", + "fresh-03-te", + "fresh-04-te", + "fresh-05-te", + "fresh-06-te", + "fresh-07-te", + "fresh-08-te", + "fresh-09-te", + "fresh-10-te", + "fresh-11-te", + "fresh-12-te", + "fresh-13-te", + "fresh-14-te", + "fresh-15-te", + "fresh-16-te", + "fresh-17-te", + "fresh-18-te", + "fresh-19-te", + "fresh-20-te", + "fresh-21-te", + "fresh-22-te", + "fresh-23-te", + "fresh-24-te", + "fresh-25-te", + "fresh-26-te", + "fresh-27-te", + "fresh-28-te", + "fresh-29-te", + "fresh-30-te", + "fresh-31-te", + "fresh-32-te", + "fresh-33-te", + "fresh-34-te", + "fresh-35-te", + "fresh-36-te", + "fresh-37-te", + "fresh-38-te", + "fresh-39-te", + "fresh-40-te" + ], + "reviewed_history_ids": [ + "confirm-fresh-01-te-m0", + "confirm-fresh-01-te-m1", + "confirm-fresh-02-te-m0", + "confirm-fresh-02-te-m1", + "confirm-fresh-03-te-m0", + "confirm-fresh-03-te-m1", + "confirm-fresh-04-te-m0", + "confirm-fresh-04-te-m1", + "confirm-fresh-05-te-m0", + "confirm-fresh-05-te-m1", + "confirm-fresh-06-te-m0", + "confirm-fresh-06-te-m1", + "confirm-fresh-07-te-m0", + "confirm-fresh-07-te-m1", + "confirm-fresh-08-te-m0", + "confirm-fresh-08-te-m1", + "confirm-fresh-09-te-m0", + "confirm-fresh-09-te-m1", + "confirm-fresh-10-te-m0", + "confirm-fresh-10-te-m1", + "confirm-fresh-11-te-m0", + "confirm-fresh-11-te-m1", + "confirm-fresh-12-te-m0", + "confirm-fresh-12-te-m1", + "confirm-fresh-13-te-m0", + "confirm-fresh-13-te-m1", + "confirm-fresh-14-te-m0", + "confirm-fresh-14-te-m1", + "confirm-fresh-15-te-m0", + "confirm-fresh-15-te-m1", + "confirm-fresh-16-te-m0", + "confirm-fresh-16-te-m1", + "confirm-fresh-17-te-m0", + "confirm-fresh-17-te-m1", + "confirm-fresh-18-te-m0", + "confirm-fresh-18-te-m1", + "confirm-fresh-19-te-m0", + "confirm-fresh-19-te-m1", + "confirm-fresh-20-te-m0", + "confirm-fresh-20-te-m1", + "confirm-fresh-21-te-m0", + "confirm-fresh-21-te-m1", + "confirm-fresh-22-te-m0", + "confirm-fresh-22-te-m1", + "confirm-fresh-23-te-m0", + "confirm-fresh-23-te-m1", + "confirm-fresh-24-te-m0", + "confirm-fresh-24-te-m1", + "confirm-fresh-25-te-m0", + "confirm-fresh-25-te-m1", + "confirm-fresh-26-te-m0", + "confirm-fresh-26-te-m1", + "confirm-fresh-27-te-m0", + "confirm-fresh-27-te-m1", + "confirm-fresh-28-te-m0", + "confirm-fresh-28-te-m1", + "confirm-fresh-29-te-m0", + "confirm-fresh-29-te-m1", + "confirm-fresh-30-te-m0", + "confirm-fresh-30-te-m1", + "confirm-fresh-31-te-m0", + "confirm-fresh-31-te-m1", + "confirm-fresh-32-te-m0", + "confirm-fresh-32-te-m1", + "confirm-fresh-33-te-m0", + "confirm-fresh-33-te-m1", + "confirm-fresh-34-te-m0", + "confirm-fresh-34-te-m1", + "confirm-fresh-35-te-m0", + "confirm-fresh-35-te-m1", + "confirm-fresh-36-te-m0", + "confirm-fresh-36-te-m1", + "confirm-fresh-37-te-m0", + "confirm-fresh-37-te-m1", + "confirm-fresh-38-te-m0", + "confirm-fresh-38-te-m1", + "confirm-fresh-39-te-m0", + "confirm-fresh-39-te-m1", + "confirm-fresh-40-te-m0", + "confirm-fresh-40-te-m1" + ], + "reviewed_job_ids": [ + "confirm-fresh-01-te-m0-source-chronological", + "confirm-fresh-01-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-01-te-m0-source-grouped_lossless", + "confirm-fresh-01-te-m1-source-chronological", + "confirm-fresh-01-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-01-te-m1-source-grouped_lossless", + "confirm-fresh-02-te-m0-source-chronological", + "confirm-fresh-02-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-02-te-m0-source-grouped_lossless", + "confirm-fresh-02-te-m1-source-chronological", + "confirm-fresh-02-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-02-te-m1-source-grouped_lossless", + "confirm-fresh-03-te-m0-source-chronological", + "confirm-fresh-03-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-03-te-m0-source-grouped_lossless", + "confirm-fresh-03-te-m1-source-chronological", + "confirm-fresh-03-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-03-te-m1-source-grouped_lossless", + "confirm-fresh-04-te-m0-source-chronological", + "confirm-fresh-04-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-04-te-m0-source-grouped_lossless", + "confirm-fresh-04-te-m1-source-chronological", + "confirm-fresh-04-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-04-te-m1-source-grouped_lossless", + "confirm-fresh-05-te-m0-source-chronological", + "confirm-fresh-05-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-05-te-m0-source-grouped_lossless", + "confirm-fresh-05-te-m1-source-chronological", + "confirm-fresh-05-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-05-te-m1-source-grouped_lossless", + "confirm-fresh-06-te-m0-source-chronological", + "confirm-fresh-06-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-06-te-m0-source-grouped_lossless", + "confirm-fresh-06-te-m1-source-chronological", + "confirm-fresh-06-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-06-te-m1-source-grouped_lossless", + "confirm-fresh-07-te-m0-source-chronological", + "confirm-fresh-07-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-07-te-m0-source-grouped_lossless", + "confirm-fresh-07-te-m1-source-chronological", + "confirm-fresh-07-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-07-te-m1-source-grouped_lossless", + "confirm-fresh-08-te-m0-source-chronological", + "confirm-fresh-08-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-08-te-m0-source-grouped_lossless", + "confirm-fresh-08-te-m1-source-chronological", + "confirm-fresh-08-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-08-te-m1-source-grouped_lossless", + "confirm-fresh-09-te-m0-source-chronological", + "confirm-fresh-09-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-09-te-m0-source-grouped_lossless", + "confirm-fresh-09-te-m1-source-chronological", + "confirm-fresh-09-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-09-te-m1-source-grouped_lossless", + "confirm-fresh-10-te-m0-source-chronological", + "confirm-fresh-10-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-10-te-m0-source-grouped_lossless", + "confirm-fresh-10-te-m1-source-chronological", + "confirm-fresh-10-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-10-te-m1-source-grouped_lossless", + "confirm-fresh-11-te-m0-source-chronological", + "confirm-fresh-11-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-11-te-m0-source-grouped_lossless", + "confirm-fresh-11-te-m1-source-chronological", + "confirm-fresh-11-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-11-te-m1-source-grouped_lossless", + "confirm-fresh-12-te-m0-source-chronological", + "confirm-fresh-12-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-12-te-m0-source-grouped_lossless", + "confirm-fresh-12-te-m1-source-chronological", + "confirm-fresh-12-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-12-te-m1-source-grouped_lossless", + "confirm-fresh-13-te-m0-source-chronological", + "confirm-fresh-13-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-13-te-m0-source-grouped_lossless", + "confirm-fresh-13-te-m1-source-chronological", + "confirm-fresh-13-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-13-te-m1-source-grouped_lossless", + "confirm-fresh-14-te-m0-source-chronological", + "confirm-fresh-14-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-14-te-m0-source-grouped_lossless", + "confirm-fresh-14-te-m1-source-chronological", + "confirm-fresh-14-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-14-te-m1-source-grouped_lossless", + "confirm-fresh-15-te-m0-source-chronological", + "confirm-fresh-15-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-15-te-m0-source-grouped_lossless", + "confirm-fresh-15-te-m1-source-chronological", + "confirm-fresh-15-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-15-te-m1-source-grouped_lossless", + "confirm-fresh-16-te-m0-source-chronological", + "confirm-fresh-16-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-16-te-m0-source-grouped_lossless", + "confirm-fresh-16-te-m1-source-chronological", + "confirm-fresh-16-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-16-te-m1-source-grouped_lossless", + "confirm-fresh-17-te-m0-source-chronological", + "confirm-fresh-17-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-17-te-m0-source-grouped_lossless", + "confirm-fresh-17-te-m1-source-chronological", + "confirm-fresh-17-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-17-te-m1-source-grouped_lossless", + "confirm-fresh-18-te-m0-source-chronological", + "confirm-fresh-18-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-18-te-m0-source-grouped_lossless", + "confirm-fresh-18-te-m1-source-chronological", + "confirm-fresh-18-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-18-te-m1-source-grouped_lossless", + "confirm-fresh-19-te-m0-source-chronological", + "confirm-fresh-19-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-19-te-m0-source-grouped_lossless", + "confirm-fresh-19-te-m1-source-chronological", + "confirm-fresh-19-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-19-te-m1-source-grouped_lossless", + "confirm-fresh-20-te-m0-source-chronological", + "confirm-fresh-20-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-20-te-m0-source-grouped_lossless", + "confirm-fresh-20-te-m1-source-chronological", + "confirm-fresh-20-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-20-te-m1-source-grouped_lossless", + "confirm-fresh-21-te-m0-source-chronological", + "confirm-fresh-21-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-21-te-m0-source-grouped_lossless", + "confirm-fresh-21-te-m1-source-chronological", + "confirm-fresh-21-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-21-te-m1-source-grouped_lossless", + "confirm-fresh-22-te-m0-source-chronological", + "confirm-fresh-22-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-22-te-m0-source-grouped_lossless", + "confirm-fresh-22-te-m1-source-chronological", + "confirm-fresh-22-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-22-te-m1-source-grouped_lossless", + "confirm-fresh-23-te-m0-source-chronological", + "confirm-fresh-23-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-23-te-m0-source-grouped_lossless", + "confirm-fresh-23-te-m1-source-chronological", + "confirm-fresh-23-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-23-te-m1-source-grouped_lossless", + "confirm-fresh-24-te-m0-source-chronological", + "confirm-fresh-24-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-24-te-m0-source-grouped_lossless", + "confirm-fresh-24-te-m1-source-chronological", + "confirm-fresh-24-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-24-te-m1-source-grouped_lossless", + "confirm-fresh-25-te-m0-source-chronological", + "confirm-fresh-25-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-25-te-m0-source-grouped_lossless", + "confirm-fresh-25-te-m1-source-chronological", + "confirm-fresh-25-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-25-te-m1-source-grouped_lossless", + "confirm-fresh-26-te-m0-source-chronological", + "confirm-fresh-26-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-26-te-m0-source-grouped_lossless", + "confirm-fresh-26-te-m1-source-chronological", + "confirm-fresh-26-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-26-te-m1-source-grouped_lossless", + "confirm-fresh-27-te-m0-source-chronological", + "confirm-fresh-27-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-27-te-m0-source-grouped_lossless", + "confirm-fresh-27-te-m1-source-chronological", + "confirm-fresh-27-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-27-te-m1-source-grouped_lossless", + "confirm-fresh-28-te-m0-source-chronological", + "confirm-fresh-28-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-28-te-m0-source-grouped_lossless", + "confirm-fresh-28-te-m1-source-chronological", + "confirm-fresh-28-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-28-te-m1-source-grouped_lossless", + "confirm-fresh-29-te-m0-source-chronological", + "confirm-fresh-29-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-29-te-m0-source-grouped_lossless", + "confirm-fresh-29-te-m1-source-chronological", + "confirm-fresh-29-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-29-te-m1-source-grouped_lossless", + "confirm-fresh-30-te-m0-source-chronological", + "confirm-fresh-30-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-30-te-m0-source-grouped_lossless", + "confirm-fresh-30-te-m1-source-chronological", + "confirm-fresh-30-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-30-te-m1-source-grouped_lossless", + "confirm-fresh-31-te-m0-source-chronological", + "confirm-fresh-31-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-31-te-m0-source-grouped_lossless", + "confirm-fresh-31-te-m1-source-chronological", + "confirm-fresh-31-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-31-te-m1-source-grouped_lossless", + "confirm-fresh-32-te-m0-source-chronological", + "confirm-fresh-32-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-32-te-m0-source-grouped_lossless", + "confirm-fresh-32-te-m1-source-chronological", + "confirm-fresh-32-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-32-te-m1-source-grouped_lossless", + "confirm-fresh-33-te-m0-source-chronological", + "confirm-fresh-33-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-33-te-m0-source-grouped_lossless", + "confirm-fresh-33-te-m1-source-chronological", + "confirm-fresh-33-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-33-te-m1-source-grouped_lossless", + "confirm-fresh-34-te-m0-source-chronological", + "confirm-fresh-34-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-34-te-m0-source-grouped_lossless", + "confirm-fresh-34-te-m1-source-chronological", + "confirm-fresh-34-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-34-te-m1-source-grouped_lossless", + "confirm-fresh-35-te-m0-source-chronological", + "confirm-fresh-35-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-35-te-m0-source-grouped_lossless", + "confirm-fresh-35-te-m1-source-chronological", + "confirm-fresh-35-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-35-te-m1-source-grouped_lossless", + "confirm-fresh-36-te-m0-source-chronological", + "confirm-fresh-36-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-36-te-m0-source-grouped_lossless", + "confirm-fresh-36-te-m1-source-chronological", + "confirm-fresh-36-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-36-te-m1-source-grouped_lossless", + "confirm-fresh-37-te-m0-source-chronological", + "confirm-fresh-37-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-37-te-m0-source-grouped_lossless", + "confirm-fresh-37-te-m1-source-chronological", + "confirm-fresh-37-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-37-te-m1-source-grouped_lossless", + "confirm-fresh-38-te-m0-source-chronological", + "confirm-fresh-38-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-38-te-m0-source-grouped_lossless", + "confirm-fresh-38-te-m1-source-chronological", + "confirm-fresh-38-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-38-te-m1-source-grouped_lossless", + "confirm-fresh-39-te-m0-source-chronological", + "confirm-fresh-39-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-39-te-m0-source-grouped_lossless", + "confirm-fresh-39-te-m1-source-chronological", + "confirm-fresh-39-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-39-te-m1-source-grouped_lossless", + "confirm-fresh-40-te-m0-source-chronological", + "confirm-fresh-40-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-40-te-m0-source-grouped_lossless", + "confirm-fresh-40-te-m1-source-chronological", + "confirm-fresh-40-te-m1-source-grouped_actor_prose_withheld", + "confirm-fresh-40-te-m1-source-grouped_lossless" + ], + "source_gold_ambiguities": [], + "invalid_or_unusable_final_job_ids": [ + "confirm-fresh-13-te-m0-source-chronological", + "confirm-fresh-16-te-m0-source-chronological", + "confirm-fresh-18-te-m0-source-chronological", + "confirm-fresh-22-te-m0-source-grouped_lossless", + "confirm-fresh-23-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-23-te-m0-source-grouped_lossless", + "confirm-fresh-25-te-m0-source-grouped_actor_prose_withheld", + "confirm-fresh-26-te-m0-source-grouped_lossless", + "confirm-fresh-31-te-m0-source-grouped_actor_prose_withheld" + ], + "summary_by_model_arm": { + "sarvam-105b/chronological": { + "n": 40, + "factual_correct": 35, + "raw_status_correct": 22, + "invalid_final": 3, + "wrong_non_null_date": 1, + "false_abstention": 0 + }, + "sarvam-105b/grouped_lossless": { + "n": 40, + "factual_correct": 34, + "raw_status_correct": 25, + "invalid_final": 3, + "wrong_non_null_date": 1, + "false_abstention": 1 + }, + "sarvam-105b/grouped_actor_prose_withheld": { + "n": 40, + "factual_correct": 33, + "raw_status_correct": 24, + "invalid_final": 3, + "wrong_non_null_date": 1, + "false_abstention": 2 + }, + "sarvam-105b-conversations/chronological": { + "n": 40, + "factual_correct": 32, + "raw_status_correct": 35, + "invalid_final": 0, + "wrong_non_null_date": 8, + "false_abstention": 0 + }, + "sarvam-105b-conversations/grouped_lossless": { + "n": 40, + "factual_correct": 33, + "raw_status_correct": 32, + "invalid_final": 0, + "wrong_non_null_date": 7, + "false_abstention": 0 + }, + "sarvam-105b-conversations/grouped_actor_prose_withheld": { + "n": 40, + "factual_correct": 37, + "raw_status_correct": 31, + "invalid_final": 0, + "wrong_non_null_date": 3, + "false_abstention": 0 + } + }, + "rows": [ + { + "history_id": "confirm-fresh-01-te-m0", + "case_id": "fresh-01-te", + "scenario": "fresh-01", + "model": "sarvam-105b", + "family": "request_correction", + "question": "VISIT-2101 కోసం నేను చివరిగా కోరిన తేదీ ఏమిటి? చెప్పండి మాత్రమే.", + "source_messages": [ + { + "id": "note-1-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-1-1", + "role": "customer", + "content": "VISIT-2101 కోసం మొదట 2027-04-02 తేదీ అడిగాను." + }, + { + "id": "note-1-2", + "role": "customer", + "content": "ఇప్పుడు దాని బదులు 2027-05-02 కావాలి; 2027-04-02 ఇక నా ఎంపిక కాదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The customer explicitly supersedes the earlier requested date for VISIT-2101; the later requested date is 2027-05-02. The current booked date is a different fact and the question is read only.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-01", + "id": "VISIT-2101", + "date": "2027-03-02", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-01", + "id": "VISIT-2101", + "date": "2027-03-02", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "81967ac34d6935d3cade2f06215b2d8e8ba4345f6429f110be4b742b7b4dbee3", + "case_sha256": "d95fef906765405e7e723b540e44d9cc1191fbf533d6113ea20a0052bda349e1", + "gold_label_sha256": "a249502a133c4a6e21b6ccadd85e11378214bcd35c4dda0856e0ea826ed85da2", + "reports": { + "chronological": { + "id": "confirm-fresh-01-te-m0-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-02\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_requested.", + "request_sha256": "712baa455d2087c0a5f37a2c0111ba3d75af5487824d000faeb0668c6a749cfe", + "request_file_sha256": "035bb2a432195a8662e4e94c8ae7d6cceb3424f6bdabffd50ec31cc39d1eba5f", + "receipt_file_sha256": "fe84bc2edaddd6c518c73c3c111e36bb17e07b8ec9840abd7ee28da9e5c52437", + "receipt_body_text_sha256": "47a2a092e4583f4383415b80692585ff8c057a03fcd3bb31e9b382a12aa03fc5", + "result_file_sha256": "ce0b06081ae4177284f45db3e7222695e4995ff592024742d429c996d187fb1e", + "raw_content_sha256": "3a2f5328a1c35116ef142a3db1e78f87dbbc8f54ba09314ff44a7cf7652ea79b", + "response_id": "20261014_3ccf6b9d-44fd-45be-a6a0-a877f9628add", + "receipt_status": 200, + "prompt_tokens": 873, + "completion_tokens": 1570, + "reasoning_tokens": 1538, + "elapsed_seconds": 8.714128249994246 + }, + "grouped_lossless": { + "id": "confirm-fresh-01-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-02\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "517820885984ba35c652e1b29650b74802f2c67a03634236443528274eb5755c", + "request_file_sha256": "5ecfb689d3896d3aa513d13890a3f76847487b6a7a09cc52ac4ac69fd38b50b8", + "receipt_file_sha256": "1264d4310d6c4101220f9f97dcc6212bc5b0ecc8f9b3075f218d95ce125b2a45", + "receipt_body_text_sha256": "b6ff6e44408903ff01d9fe915b0fbc3938985b79b8290ad89ab00d469ce7dd7b", + "result_file_sha256": "96797c104f7a78e3839f320aa377b1e928e67cc696b95a1e7c81bfcf267ef17b", + "raw_content_sha256": "8aaeb0f32dbe546730fa5f7e7419bc6071d73ddbc5f5b92f356e540d2f854cb2", + "response_id": "20261014_92bb0e43-8bee-415f-9550-cb3524fda0bc", + "receipt_status": 200, + "prompt_tokens": 1427, + "completion_tokens": 1624, + "reasoning_tokens": 1590, + "elapsed_seconds": 8.95311099999526 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-01-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": true, + "factual_review": "False abstention: source or final backend uniquely establishes 2027-05-02, but raw answer is null.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "7178d480df1313a09e6c468e34cfd959e39d82eeffb349019b2c3d442a1d4b6d", + "request_file_sha256": "371c284dfa3cef200ce89c4f50182ec6979caad1394b2747b269b7bf2a90574c", + "receipt_file_sha256": "2f81f6db37855253b08d96e3ea9643315589dc5a2311a5bdff32e43a409a4599", + "receipt_body_text_sha256": "7bb8762263271e6ed7080af517dc27ba3d733a741db47ac91493bc6b006f875e", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "response_id": "20261014_687ef675-da11-498f-a495-34d50aa132f8", + "receipt_status": 200, + "prompt_tokens": 1412, + "completion_tokens": 1767, + "reasoning_tokens": 1741, + "elapsed_seconds": 7.830779916999745 + } + } + }, + { + "history_id": "confirm-fresh-01-te-m1", + "case_id": "fresh-01-te", + "scenario": "fresh-01", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "question": "VISIT-2101 కోసం నేను చివరిగా కోరిన తేదీ ఏమిటి? చెప్పండి మాత్రమే.", + "source_messages": [ + { + "id": "note-1-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-1-1", + "role": "customer", + "content": "VISIT-2101 కోసం మొదట 2027-04-02 తేదీ అడిగాను." + }, + { + "id": "note-1-2", + "role": "customer", + "content": "ఇప్పుడు దాని బదులు 2027-05-02 కావాలి; 2027-04-02 ఇక నా ఎంపిక కాదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The customer explicitly supersedes the earlier requested date for VISIT-2101; the later requested date is 2027-05-02. The current booked date is a different fact and the question is read only.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-01", + "id": "VISIT-2101", + "date": "2027-03-02", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-01", + "id": "VISIT-2101", + "date": "2027-03-02", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "27697bc40b63a176a28fad5ae272d8e30804db0bf7b9949c61bdbbf560bb6015", + "case_sha256": "d95fef906765405e7e723b540e44d9cc1191fbf533d6113ea20a0052bda349e1", + "gold_label_sha256": "a249502a133c4a6e21b6ccadd85e11378214bcd35c4dda0856e0ea826ed85da2", + "reports": { + "chronological": { + "id": "confirm-fresh-01-te-m1-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-02\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "cfdf5c705ffe762aae21874e089a03da53f586db9d7424bf26b03325e23a6bdf", + "request_file_sha256": "34f09f0b4d415873bdc72b29c9b172fcff9a4d298ba522b00ad7b5119e8e3eab", + "receipt_file_sha256": "ebea2befabafee0229f88bcd21eb6fac9781da79728dd920e57ed53031fc8a17", + "receipt_body_text_sha256": "f5c64c5876fc9bc0ca095ba865213c175adce3bb94cd4aa93511285be68769e9", + "result_file_sha256": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "raw_content_sha256": "54f87f93350a2f520e59fad7cd323aeaa385120f331fae30b8483a7800211347", + "response_id": "20261014_f00526bc-1e9b-4541-82fd-87fef88eebe7", + "receipt_status": 200, + "prompt_tokens": 882, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9889927500043996 + }, + "grouped_lossless": { + "id": "confirm-fresh-01-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-02\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "8e0e5b98e7dc9fdc24fedc840b2190789797f256b944b880643e4bec99f72fda", + "request_file_sha256": "96b159c6d87981071200982bf1a2a5beac1467a663ff91492a3ef9381d0f6c54", + "receipt_file_sha256": "4c2cc92ad07eb8e5ab29e385787b381957ca6daa094f277f69299b5fff9300fc", + "receipt_body_text_sha256": "11e5e83f0a9353230188b4193dd930b0d85377abd1989f3b15c89b410ac27f0a", + "result_file_sha256": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "raw_content_sha256": "54f87f93350a2f520e59fad7cd323aeaa385120f331fae30b8483a7800211347", + "response_id": "20261014_afc3f2ad-7377-47fb-ad7f-9671b87b69ab", + "receipt_status": 200, + "prompt_tokens": 1427, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9718708750006044 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-01-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-02\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-02", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "0a0cb0845376c1db5258aaea34d9ac6c316d2b3ec9343fd75b976c0899dc92a1", + "request_file_sha256": "0764e76760d833c56c8d4a12052ef8b9e27b17f81af2e630e840a69a3ff1a341", + "receipt_file_sha256": "55901241244ad0ce2d67a90152de5b66065806bf9c5f2d3a9f66f501aef5c51b", + "receipt_body_text_sha256": "afa86f5f796ff26b146fd7132466186f265b6f4bc90d3c389cf80cadf87f64bc", + "result_file_sha256": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "raw_content_sha256": "54f87f93350a2f520e59fad7cd323aeaa385120f331fae30b8483a7800211347", + "response_id": "20261014_a48f3607-a4f8-475b-8236-7eda628b0c91", + "receipt_status": 200, + "prompt_tokens": 1407, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9754500839917455 + } + } + }, + { + "history_id": "confirm-fresh-02-te-m0", + "case_id": "fresh-02-te", + "scenario": "fresh-02", + "model": "sarvam-105b", + "family": "request_correction", + "question": "VISIT-2102 కోసం నేను చివరిగా కోరిన తేదీ ఏమిటి? చెప్పండి మాత్రమే.", + "source_messages": [ + { + "id": "note-2-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. VISIT-2102 కోసం మొదట 2027-04-03 తేదీ అడిగాను. ఇప్పుడు దాని బదులు 2027-05-03 కావాలి; 2027-04-03 ఇక నా ఎంపిక కాదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The customer explicitly supersedes the earlier requested date for VISIT-2102; the later requested date is 2027-05-03. The current booked date is a different fact and the question is read only.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-02", + "id": "VISIT-2102", + "date": "2027-03-03", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-02", + "id": "VISIT-2102", + "date": "2027-03-03", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "87d03cb91d7511611f7c518df26a0b5aed10b5969280903a3e9508cc67413606", + "case_sha256": "51e24075cce2559859dbd03db832ec61c75729c0b1a23bfb04039f48e1e1be83", + "gold_label_sha256": "be69235e5b4f148a342401989653da867b8f508739ac50e9ed8ceee4b8914673", + "reports": { + "chronological": { + "id": "confirm-fresh-02-te-m0-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-03\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "e987ea0e3f87bf2115035d8cd1061ef8d62aaf91bf522ffd7b0fd53261aaafdb", + "request_file_sha256": "31d01baedb8c8573663753b8c98a49909f1f4ec37c9ff17b711c61f704e36ab4", + "receipt_file_sha256": "90c10c6d8ac52e28ce6fa0f7c5351f006eb11f5de75bf38be0aaad11aabced7c", + "receipt_body_text_sha256": "28c25abef6bf7f16f312422a7e5355a527e9127fbc9b17ee5fa9d1e601bd5053", + "result_file_sha256": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "raw_content_sha256": "8a1e2997cc0301dc67f4599949b8916f49f477f5d893e801470b0c0fd38e7854", + "response_id": "20261014_2edab6e0-856a-4df1-a609-b7e7b35b141c", + "receipt_status": 200, + "prompt_tokens": 839, + "completion_tokens": 2723, + "reasoning_tokens": 2691, + "elapsed_seconds": 12.771132666995982 + }, + "grouped_lossless": { + "id": "confirm-fresh-02-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-03\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "fb4c493f14f304f1f248fdf90321930e61c3eb0b57834492c1939cb8f0be7427", + "request_file_sha256": "93f0e353a7ed81d0f07ff3a91863c14b97fc6a9842c36e373ab89625a68847b1", + "receipt_file_sha256": "618d690c7739dce39a5e3a67b5f2c401278d6e52eaf94de0bc5a8a0034152085", + "receipt_body_text_sha256": "802ccc4b84ba2813ac8fdc8047867976581566df1a12d6511766bdbcd071037c", + "result_file_sha256": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "raw_content_sha256": "8a1e2997cc0301dc67f4599949b8916f49f477f5d893e801470b0c0fd38e7854", + "response_id": "20261014_f41c1b4a-836b-40bf-bd5b-90b78a9c9abc", + "receipt_status": 200, + "prompt_tokens": 1281, + "completion_tokens": 3735, + "reasoning_tokens": 3703, + "elapsed_seconds": 16.362450125001487 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-02-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-03\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "9e6ceb4c20e079f3a8753071ea7f39c79e660628db4907308d3f0ff8b53d5297", + "request_file_sha256": "0ccfda56cc9f0663b129bfa9fe0911ee52dbf0ca5f32185a439cb7120e7cf18c", + "receipt_file_sha256": "d8d70c13849235a0bc544b257f1518dfdebd8c39c52440b86b6600dda410c557", + "receipt_body_text_sha256": "cd62d5391e3d17394d07ffe132b92fe013fcda3cdbf3b2eb066a6bc6a3986bcc", + "result_file_sha256": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "raw_content_sha256": "8a1e2997cc0301dc67f4599949b8916f49f477f5d893e801470b0c0fd38e7854", + "response_id": "20261014_910acce5-505c-4bf7-bf2c-a5dcaea8de0f", + "receipt_status": 200, + "prompt_tokens": 1266, + "completion_tokens": 2628, + "reasoning_tokens": 2596, + "elapsed_seconds": 11.4500859999971 + } + } + }, + { + "history_id": "confirm-fresh-02-te-m1", + "case_id": "fresh-02-te", + "scenario": "fresh-02", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "question": "VISIT-2102 కోసం నేను చివరిగా కోరిన తేదీ ఏమిటి? చెప్పండి మాత్రమే.", + "source_messages": [ + { + "id": "note-2-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. VISIT-2102 కోసం మొదట 2027-04-03 తేదీ అడిగాను. ఇప్పుడు దాని బదులు 2027-05-03 కావాలి; 2027-04-03 ఇక నా ఎంపిక కాదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The customer explicitly supersedes the earlier requested date for VISIT-2102; the later requested date is 2027-05-03. The current booked date is a different fact and the question is read only.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-02", + "id": "VISIT-2102", + "date": "2027-03-03", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-02", + "id": "VISIT-2102", + "date": "2027-03-03", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "b532fd8d89cda48e69e05a3ff85b2bfd22e0700a79956a707d111c155a0392b2", + "case_sha256": "51e24075cce2559859dbd03db832ec61c75729c0b1a23bfb04039f48e1e1be83", + "gold_label_sha256": "be69235e5b4f148a342401989653da867b8f508739ac50e9ed8ceee4b8914673", + "reports": { + "chronological": { + "id": "confirm-fresh-02-te-m1-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-03\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "1e343c80c28d7233fe8b05a085c0af4545170b60b858681e77e09af865fa090c", + "request_file_sha256": "7711e585f8a3329828c69a99d52b7465ef326a21dbdaf6871d775d509d7e4cb7", + "receipt_file_sha256": "d8e9f371fd0bddc410d9ce34a1449541dc1e91734c5ef4ead8cad1c189210e4a", + "receipt_body_text_sha256": "6d180a1183140b9a8da080bee5d03c9b0ef272ade6a1b7facbb58d4f012c1dea", + "result_file_sha256": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "raw_content_sha256": "302302624fee023a6aedb4cd6f12fe82dc40383abb18b454d5e9be83a0d2c374", + "response_id": "20261014_7e15bdc6-4df3-462b-9bb5-15d03b8cde48", + "receipt_status": 200, + "prompt_tokens": 851, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.101985249988502 + }, + "grouped_lossless": { + "id": "confirm-fresh-02-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-03\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "52fa69bf763baa9f89e54676e6bac554fd197611438c2c19e100f681131a7b0a", + "request_file_sha256": "45d94b3fb23278a32f8028ce636191ee3b8d617f79946b9bcc43624088c6ea06", + "receipt_file_sha256": "3e69b5c05d23bfa95f484c0b2f77dbead788e1f5a2098c33cb440a388b80642a", + "receipt_body_text_sha256": "4f195d51cb2f0bb3052d3c59050cdcb1d033cd02c13ebb3b0844aa532ef60690", + "result_file_sha256": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "raw_content_sha256": "302302624fee023a6aedb4cd6f12fe82dc40383abb18b454d5e9be83a0d2c374", + "response_id": "20261014_a1b3783c-91e1-4a2c-8328-a2a2304bb5b9", + "receipt_status": 200, + "prompt_tokens": 1292, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0098189589916728 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-02-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-03\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-03", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "65f6ae0886bfdc5457b9be90191c7e4c88b3c11decf28e41fbc89fd4df695f61", + "request_file_sha256": "aab7958ab465115ebbe7343a1d07725ba56ac0daf0cb430d49603aa76bc3ec5e", + "receipt_file_sha256": "223c0cea3525648d8141110d9740ba86b0378ea9207827f2001666f77e4e90b2", + "receipt_body_text_sha256": "04a809326fb0bf4b010eb4de7e61ba03feecf33d85bd1ae943c4e399fd76cf7f", + "result_file_sha256": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "raw_content_sha256": "302302624fee023a6aedb4cd6f12fe82dc40383abb18b454d5e9be83a0d2c374", + "response_id": "20261014_c23b38a9-e1af-4543-a4e8-c5f5271e2341", + "receipt_status": 200, + "prompt_tokens": 1269, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.1945389590109698 + } + } + }, + { + "history_id": "confirm-fresh-03-te-m0", + "case_id": "fresh-03-te", + "scenario": "fresh-03", + "model": "sarvam-105b", + "family": "request_correction", + "question": "VISIT-2103 కోసం నేను చివరిగా కోరిన తేదీ ఏమిటి? చెప్పండి మాత్రమే.", + "source_messages": [ + { + "id": "note-3-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-3-1", + "role": "customer", + "content": "VISIT-2103 కోసం మొదట 2027-04-04 తేదీ అడిగాను." + }, + { + "id": "note-3-2", + "role": "customer", + "content": "ఇప్పుడు దాని బదులు 2027-05-04 కావాలి; 2027-04-04 ఇక నా ఎంపిక కాదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The customer explicitly supersedes the earlier requested date for VISIT-2103; the later requested date is 2027-05-04. The current booked date is a different fact and the question is read only.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-03", + "id": "VISIT-2103", + "date": "2027-03-04", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-03", + "id": "VISIT-2103", + "date": "2027-03-04", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "bbb8372cbf60c5959a0ed41636f02bdfcdeec7c766ee14e874d6615b8f870fc7", + "case_sha256": "2f46979eb9cabcfc29a37a09f0cbe5bf8f35c2d3999cf7a830b26bac16ecbab5", + "gold_label_sha256": "f3281a3a99b87cc0e3f16fc0d7bfdc3fe1836ed39e3de37e7ecf577b5b065905", + "reports": { + "chronological": { + "id": "confirm-fresh-03-te-m0-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-04\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "a5924574f135618041fcc05ea5507aa0c39ebba109c4293212d9542017af1405", + "request_file_sha256": "f1ebacd5e4fb559f814fbba89b0d0f7f0bcae943d8defc1771507bf386c5a30e", + "receipt_file_sha256": "38d69b49ddd5e79fd2a7ed7516f54864742d6540b61869ba7a5f3b969d849bc0", + "receipt_body_text_sha256": "41f62d9d0088cbfc05b6080863a8cf2388ad6c6da0cbb91bbac2df7c3ebf65dc", + "result_file_sha256": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "raw_content_sha256": "212f85c3b1a2653eaa716d130aac4c6887ca2e4ae77cd416c3a0ee75931a3138", + "response_id": "20261014_93d41bbc-b7b8-4e90-8f86-9d002b96dc06", + "receipt_status": 200, + "prompt_tokens": 875, + "completion_tokens": 2678, + "reasoning_tokens": 2646, + "elapsed_seconds": 12.408399916996132 + }, + "grouped_lossless": { + "id": "confirm-fresh-03-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-04\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "09c07c2fac3adf26edf2064dbaa930bfaaef8ba009d7ce1fc68c083ed54a3d5f", + "request_file_sha256": "3dca5b45ee484136f1f78185014a1f375f00ee33e19cb456d2417c35bb91b3c8", + "receipt_file_sha256": "15e66f41ed268302d1c65001fc7eb49e205d84453232731b1c3e963bf0731676", + "receipt_body_text_sha256": "da458976b11b480c1a0e92d4f0a8a5a048b82028377be754018f392c5b80aa5e", + "result_file_sha256": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "raw_content_sha256": "212f85c3b1a2653eaa716d130aac4c6887ca2e4ae77cd416c3a0ee75931a3138", + "response_id": "20261014_2f9a4f34-a3e4-4166-9606-e31ff6035ba0", + "receipt_status": 200, + "prompt_tokens": 1432, + "completion_tokens": 1634, + "reasoning_tokens": 1602, + "elapsed_seconds": 8.721237082994776 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-03-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-04\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "93ee488ad7e1e2aaa5dec9a0f9d4d88996e971c0d9a19f2f31efe3f7437c5d62", + "request_file_sha256": "b6fc6d950014e600ec31ec222146636315e9c695da3d0710915e30f24544a3ed", + "receipt_file_sha256": "5659f37eebacf5ea81603b06c833de4fa829a726dd0173497662312419c9d4a9", + "receipt_body_text_sha256": "5b7d17fe3b4b95da34d396f42158899832cd9db983bc042e574a9cdd33a7f0ac", + "result_file_sha256": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "raw_content_sha256": "212f85c3b1a2653eaa716d130aac4c6887ca2e4ae77cd416c3a0ee75931a3138", + "response_id": "20261014_0333a16d-0799-4ea9-9899-733a603ad557", + "receipt_status": 200, + "prompt_tokens": 1417, + "completion_tokens": 982, + "reasoning_tokens": 948, + "elapsed_seconds": 5.772524457992404 + } + } + }, + { + "history_id": "confirm-fresh-03-te-m1", + "case_id": "fresh-03-te", + "scenario": "fresh-03", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "question": "VISIT-2103 కోసం నేను చివరిగా కోరిన తేదీ ఏమిటి? చెప్పండి మాత్రమే.", + "source_messages": [ + { + "id": "note-3-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-3-1", + "role": "customer", + "content": "VISIT-2103 కోసం మొదట 2027-04-04 తేదీ అడిగాను." + }, + { + "id": "note-3-2", + "role": "customer", + "content": "ఇప్పుడు దాని బదులు 2027-05-04 కావాలి; 2027-04-04 ఇక నా ఎంపిక కాదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The customer explicitly supersedes the earlier requested date for VISIT-2103; the later requested date is 2027-05-04. The current booked date is a different fact and the question is read only.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-03", + "id": "VISIT-2103", + "date": "2027-03-04", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-03", + "id": "VISIT-2103", + "date": "2027-03-04", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "7a01c4f44b768114fce45ff427b99b583eae6291fb564f78cf0b9340e4fd12fe", + "case_sha256": "2f46979eb9cabcfc29a37a09f0cbe5bf8f35c2d3999cf7a830b26bac16ecbab5", + "gold_label_sha256": "f3281a3a99b87cc0e3f16fc0d7bfdc3fe1836ed39e3de37e7ecf577b5b065905", + "reports": { + "chronological": { + "id": "confirm-fresh-03-te-m1-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-04\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "554378e733b706fe1b904a9ebff35b24ebcb6f284fbf9a0104767d732759ff6b", + "request_file_sha256": "92939d748f3f391bcdf63d413ce576640bd354c9ab533074b11fe25652a474a3", + "receipt_file_sha256": "f8acad611231f9b0734f609fd49b54f8cf8d6a4896b837262d5c9406ea7ca242", + "receipt_body_text_sha256": "498491c06f22e86f6269e73464c0d717cee874bfe0154443f870a54550bee2c3", + "result_file_sha256": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "raw_content_sha256": "42098a7a7a4ef0ae49a0da19af03407227f6c6dda8858f3a01e7bd39c56ff795", + "response_id": "20261014_7f9b5a88-ee18-48ef-9fe3-bacea97515d0", + "receipt_status": 200, + "prompt_tokens": 887, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.951432209010818 + }, + "grouped_lossless": { + "id": "confirm-fresh-03-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-04\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "08b5117fa335eb161148e6b6b3053d5ef8658ec0c826504036d8bb0934b0bc3e", + "request_file_sha256": "4a05f0086c18c1432b5ef13deb61f10198c1feba83157e9132610b7cad1dc9e4", + "receipt_file_sha256": "76c5be3a35f047196810db7ba5daccd63f9d4535f3442bc6171fe18140eba94a", + "receipt_body_text_sha256": "36df59957df82ef4f4743b7b344f0a5dd74d217821d4277293619e5e5824c0d1", + "result_file_sha256": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "raw_content_sha256": "42098a7a7a4ef0ae49a0da19af03407227f6c6dda8858f3a01e7bd39c56ff795", + "response_id": "20261014_92e082d3-07fd-4ed9-a04f-0066331ee9ed", + "receipt_status": 200, + "prompt_tokens": 1435, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0147983750066487 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-03-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-04\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-04", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "bc736719ad11c2559a4fb01cfb74dd9a364d45593944b695115b27a7ddd01e2b", + "request_file_sha256": "14b68e7020f1c7f309100217cdf0dc6593486375a49c1d924a23849d2519bef9", + "receipt_file_sha256": "d44ff7ad7d44838b804fee20baa3580e6bd64d221122102a49de4255232626de", + "receipt_body_text_sha256": "a0a7fa8adc821d7201a4c6df69953764b687eabe9b3921881cccd3761b38805e", + "result_file_sha256": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "raw_content_sha256": "42098a7a7a4ef0ae49a0da19af03407227f6c6dda8858f3a01e7bd39c56ff795", + "response_id": "20261014_6f4bf714-627c-4cf6-b3cc-3e6d694ec471", + "receipt_status": 200, + "prompt_tokens": 1412, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0183845830033533 + } + } + }, + { + "history_id": "confirm-fresh-04-te-m0", + "case_id": "fresh-04-te", + "scenario": "fresh-04", + "model": "sarvam-105b", + "family": "request_correction", + "question": "VISIT-2104 కోసం నేను చివరిగా కోరిన తేదీ ఏమిటి? చెప్పండి మాత్రమే.", + "source_messages": [ + { + "id": "note-4-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. VISIT-2104 కోసం మొదట 2027-04-05 తేదీ అడిగాను. ఇప్పుడు దాని బదులు 2027-05-05 కావాలి; 2027-04-05 ఇక నా ఎంపిక కాదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The customer explicitly supersedes the earlier requested date for VISIT-2104; the later requested date is 2027-05-05. The current booked date is a different fact and the question is read only.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-04", + "id": "VISIT-2104", + "date": "2027-03-05", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-04", + "id": "VISIT-2104", + "date": "2027-03-05", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "14fbc4af10e3d6a8aa9bbbc4117d6e538820a175b8c2a0a11715014a2959c78f", + "case_sha256": "82211ea6627062a17060048462ee063bb5f6200353dfdb78e0e62a6faa4c7298", + "gold_label_sha256": "7009c8a96f890331549c926d106f6a8263f44e95701f921d2c1a9a91ff19ff20", + "reports": { + "chronological": { + "id": "confirm-fresh-04-te-m0-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-05\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "1f46bf3da3b4621c7bbf72404f821d2aeaf4b7d49eb7654a82d72e74b8e70f70", + "request_file_sha256": "bfd325125e03f0d4317e0d2d6f1a24df43ac386e85174919024e2e94e2ec4da9", + "receipt_file_sha256": "1b690315739101ff4b135a25835290bdda0b252985d0bdaf690f21bae70b8732", + "receipt_body_text_sha256": "b88bd68b5cf25c3d20e8f4c2b0dedb54f50de1b3df52e7bd4dd6c15febce6932", + "result_file_sha256": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "raw_content_sha256": "c45d72a60c16dbd0e39fe78d180715170520e37c139792fb956a8819a538950b", + "response_id": "20261014_6639edc5-3000-47b8-a36e-4af2c2c8f497", + "receipt_status": 200, + "prompt_tokens": 839, + "completion_tokens": 3083, + "reasoning_tokens": 3051, + "elapsed_seconds": 17.790569124990725 + }, + "grouped_lossless": { + "id": "confirm-fresh-04-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-05\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "3d0a786dc77f1f1f727df26c8c7ac6eb104f2ee7681df600eb2c9baf9c13d037", + "request_file_sha256": "95ad29614c6e11562cd2ed0645f6961a774e0c7db1851539f56780af34d6fe29", + "receipt_file_sha256": "0517ce2304d3978320179d80a60c9d999345a79ede41c1de11593371b0be4b89", + "receipt_body_text_sha256": "edf2e7d348040d093e3e38a70f86e73d284cb4c0ed11b726eef0db003b65d579", + "result_file_sha256": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "raw_content_sha256": "c45d72a60c16dbd0e39fe78d180715170520e37c139792fb956a8819a538950b", + "response_id": "20261014_dd78e380-bc43-4d39-91cb-2c4fad5b564c", + "receipt_status": 200, + "prompt_tokens": 1277, + "completion_tokens": 629, + "reasoning_tokens": 597, + "elapsed_seconds": 3.36182416700467 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-04-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-05\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "36f1c704aa92b61ae9fefb90edee7e574d2bf2f0e2e01f7ce86d5fa6de68813e", + "request_file_sha256": "415a4aec8822652825a078a4f0c5262bf0ec8e6fe528738471d2c4ffb02fcbdf", + "receipt_file_sha256": "25d8b6c926f973c43c35ef359bc2bcc9b568e52df47980beb2aced07178aa902", + "receipt_body_text_sha256": "990e33fb876822e95b114c6d76ac2aa2446b98cde23ed3b2d9b00d28f1da57e4", + "result_file_sha256": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "raw_content_sha256": "c45d72a60c16dbd0e39fe78d180715170520e37c139792fb956a8819a538950b", + "response_id": "20261014_1d789562-dc04-435c-9a6c-a673c13703af", + "receipt_status": 200, + "prompt_tokens": 1262, + "completion_tokens": 778, + "reasoning_tokens": 746, + "elapsed_seconds": 3.9927155420009512 + } + } + }, + { + "history_id": "confirm-fresh-04-te-m1", + "case_id": "fresh-04-te", + "scenario": "fresh-04", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "question": "VISIT-2104 కోసం నేను చివరిగా కోరిన తేదీ ఏమిటి? చెప్పండి మాత్రమే.", + "source_messages": [ + { + "id": "note-4-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. VISIT-2104 కోసం మొదట 2027-04-05 తేదీ అడిగాను. ఇప్పుడు దాని బదులు 2027-05-05 కావాలి; 2027-04-05 ఇక నా ఎంపిక కాదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The customer explicitly supersedes the earlier requested date for VISIT-2104; the later requested date is 2027-05-05. The current booked date is a different fact and the question is read only.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-04", + "id": "VISIT-2104", + "date": "2027-03-05", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-04", + "id": "VISIT-2104", + "date": "2027-03-05", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "cbb3e6e032d97de41a251d9d5d348e9b82b063f6f87b5fface8c32ced0037712", + "case_sha256": "82211ea6627062a17060048462ee063bb5f6200353dfdb78e0e62a6faa4c7298", + "gold_label_sha256": "7009c8a96f890331549c926d106f6a8263f44e95701f921d2c1a9a91ff19ff20", + "reports": { + "chronological": { + "id": "confirm-fresh-04-te-m1-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-05\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "1ee028f208b79fa2e00bf20b36302811d55c9e840f0797439fe5a9300265d772", + "request_file_sha256": "0564984f511bbd1df0725acedd46fcf29b8649bf180af8120aa86c43eba08c21", + "receipt_file_sha256": "f76769dbc5228e3dc0d0fa9327ea09b6ab821c39ae59284478bbab0f400a493f", + "receipt_body_text_sha256": "552c5881fe97051f9e40f5f2a7175d7812a3546573467d9982ac4d6de7dd19bd", + "result_file_sha256": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "raw_content_sha256": "be956b2af283f28155cb87228f3c2833e326c59d589a68fb50d0a32eeaea2bd8", + "response_id": "20261014_36f29618-c4b9-488c-b984-9863600bba8e", + "receipt_status": 200, + "prompt_tokens": 851, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9734913330030395 + }, + "grouped_lossless": { + "id": "confirm-fresh-04-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-05\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "8774c489ceed6bff6794e117981e2309f56d3780552a627798dfa63b81a31d8a", + "request_file_sha256": "388bef7b13a15869f0b0ac52d0166eeb3d4be1e8654e7b53b4561b49598abb78", + "receipt_file_sha256": "16c8f3d15043b905b22f440df3746ffe046288cd2fdc5efbbfea0902555778a1", + "receipt_body_text_sha256": "f4cda7dc7372e0cfd39220a6463bd6724b26c5cffc9f5c68b7a34513e36401da", + "result_file_sha256": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "raw_content_sha256": "be956b2af283f28155cb87228f3c2833e326c59d589a68fb50d0a32eeaea2bd8", + "response_id": "20261014_c2d64dd9-8240-4c23-b7eb-227b77825f55", + "receipt_status": 200, + "prompt_tokens": 1291, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.1146989169938024 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-04-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-05\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-05", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "ecb2ab0cf809e03d2950a195807cdd2609217c6672985cb897fa35de137c5412", + "request_file_sha256": "d1545242d8643d5626cd8a9534a06d6015d0b3d86da77b2ce18e93047f001351", + "receipt_file_sha256": "6ce4df1fe8d9919c4be9665d400ababcf9de83249dd9f8e415d28130241da101", + "receipt_body_text_sha256": "86e68e0f6278f0d9f9deabcd19ef561ab8c49f1aef189aede39eee57296f0fa2", + "result_file_sha256": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "raw_content_sha256": "be956b2af283f28155cb87228f3c2833e326c59d589a68fb50d0a32eeaea2bd8", + "response_id": "20261014_41c24056-0ecb-48cb-9dd9-d423a90f959e", + "receipt_status": 200, + "prompt_tokens": 1268, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9516997919999994 + } + } + }, + { + "history_id": "confirm-fresh-05-te-m0", + "case_id": "fresh-05-te", + "scenario": "fresh-05", + "model": "sarvam-105b", + "family": "missing_replacement", + "question": "VISIT-2105 కోసం ఇప్పుడు నేను కోరుకుంటున్న కొత్త తేదీ ఏది? చెప్పండి మాత్రమే; బుకింగ్ మార్చవద్దు.", + "source_messages": [ + { + "id": "note-5-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-5-1", + "role": "customer", + "content": "VISIT-2105 కోసం ఇక 2027-04-06 వద్దు." + }, + { + "id": "note-5-2", + "role": "customer", + "content": "బదులుగా ఏ తేదీ కావాలో ఇంకా నిర్ణయించలేదు. నా మరో బుకింగ్ VISIT-3105 వేరు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The customer withdraws an old choice for VISIT-2105 and explicitly has not selected a replacement. No unique new preferred date exists yet; null with clarification true records the missing customer selection, not ignorance of an established date.", + "expected_answer_from_source_and_frozen_label": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-05", + "id": "VISIT-2105", + "date": "2027-03-06", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-05", + "id": "VISIT-2105", + "date": "2027-03-06", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "ea40749a40463d7b21e82127648a05bdf48f289fc293d46b882633f7ae91757b", + "case_sha256": "db16c47d2053ca1ac3dc8d0ce8ea5b95fa81602f4e81a264fa740a42cda564e1", + "gold_label_sha256": "39906ac8a9b3e3653cfd733446d4ec041a6e0fc13d666b53fd30a52f6e0f4403", + "reports": { + "chronological": { + "id": "confirm-fresh-05-te-m0-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "2c8adbeac347ecb76d37d8c52e8104baa26f2c2d1ef9b8a6c22be9c18bbf00f9", + "request_file_sha256": "20d340178e741b4c166fc7b7a6235326fee3dbbf644c15ea0755d8a79fc7f263", + "receipt_file_sha256": "3d59d40e13a620ae422d8de892d6a65f56f3a15b8f45c4fec7ab5a7b32c0f28c", + "receipt_body_text_sha256": "81579bcd1d24d71e364321c99939adaaa639c4600015d9aee14ddef6d7181dd9", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "response_id": "20261014_1bc7eeaf-d98d-4fe9-af3c-7f876fd7902b", + "receipt_status": 200, + "prompt_tokens": 910, + "completion_tokens": 1502, + "reasoning_tokens": 1479, + "elapsed_seconds": 6.898224082993693 + }, + "grouped_lossless": { + "id": "confirm-fresh-05-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "Null date is appropriate, but clarification=false misses the explicitly unresolved selection or condition.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "939f0d7a3e03fa6f6b8122071ab6197186a573fb9da334bb9573f0d827d26a8b", + "request_file_sha256": "77ba511ff57d215a92cbb2d9f5c1f9b2e391b5895ae7af78c4f23f0d0aec2b10", + "receipt_file_sha256": "39facde2fb2ca48657bcb57ade9e48ed374c4c44a69c947f597c0f7d802501f7", + "receipt_body_text_sha256": "8c9a0cacd78ab57467cbbf23835bbf978634538a2785ef0e7ac50bc0d07cd4f6", + "result_file_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "raw_content_sha256": "837d7c4d0820dbbaaedba89ceca07d3f62ea5913ac395c7558beddcbae373aa6", + "response_id": "20261014_026e646b-6220-4e37-abf0-24f12ee3a09d", + "receipt_status": 200, + "prompt_tokens": 1454, + "completion_tokens": 3471, + "reasoning_tokens": 3448, + "elapsed_seconds": 16.40769549999095 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-05-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "edfc2f58256d608c7a08afacc9edda85fdd9f7c160613b584d920cd158175ccd", + "request_file_sha256": "6a76670f769a67320310c83daa2b2bb9d07cedf36f9174f323ca288e5586201b", + "receipt_file_sha256": "558f4efb6fe843a984e723186c8cc7370c6c73a706a4130f30112b08bf18c83c", + "receipt_body_text_sha256": "a737bff2dc468e13ab3dee531f8ba00077cfdcf2fd2f15e8e4db0d4b32289a52", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "response_id": "20261014_8673e13a-088a-4678-aac2-6a13aeb802bc", + "receipt_status": 200, + "prompt_tokens": 1394, + "completion_tokens": 5093, + "reasoning_tokens": 5070, + "elapsed_seconds": 22.103223125013756 + } + } + }, + { + "history_id": "confirm-fresh-05-te-m1", + "case_id": "fresh-05-te", + "scenario": "fresh-05", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "question": "VISIT-2105 కోసం ఇప్పుడు నేను కోరుకుంటున్న కొత్త తేదీ ఏది? చెప్పండి మాత్రమే; బుకింగ్ మార్చవద్దు.", + "source_messages": [ + { + "id": "note-5-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-5-1", + "role": "customer", + "content": "VISIT-2105 కోసం ఇక 2027-04-06 వద్దు." + }, + { + "id": "note-5-2", + "role": "customer", + "content": "బదులుగా ఏ తేదీ కావాలో ఇంకా నిర్ణయించలేదు. నా మరో బుకింగ్ VISIT-3105 వేరు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The customer withdraws an old choice for VISIT-2105 and explicitly has not selected a replacement. No unique new preferred date exists yet; null with clarification true records the missing customer selection, not ignorance of an established date.", + "expected_answer_from_source_and_frozen_label": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-05", + "id": "VISIT-2105", + "date": "2027-03-06", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-05", + "id": "VISIT-2105", + "date": "2027-03-06", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-ada8f8f6165e39a3", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2105" + }, + "response": { + "date": "2027-03-06", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "8776494bfd39dd41de5fe1b4bc6c71604964d6f0e096d3e7d7aa19915fe073f8", + "case_sha256": "db16c47d2053ca1ac3dc8d0ce8ea5b95fa81602f4e81a264fa740a42cda564e1", + "gold_label_sha256": "39906ac8a9b3e3653cfd733446d4ec041a6e0fc13d666b53fd30a52f6e0f4403", + "reports": { + "chronological": { + "id": "confirm-fresh-05-te-m1-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "8e184c8cfe66d2cb2f605c453b4d5d9126a344a70a0fe41337c934c8e6649653", + "request_file_sha256": "47f81d2a6131c4f11dfdc81756eced398f102b4439edd52f6f624f9b6c3d3873", + "receipt_file_sha256": "a717fcc4fb32dd026cc58175aaedb6d60d85f3d8bb4b4a5fc709a9f634fc7c09", + "receipt_body_text_sha256": "7243520b5f10fdad9b8abe68310e5787bd0b8f6b52741dddfb439bf5eef3088f", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_f8c6ea23-f130-4ff4-aab8-295d5e48fe79", + "receipt_status": 200, + "prompt_tokens": 1033, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 0.964736832989729 + }, + "grouped_lossless": { + "id": "confirm-fresh-05-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "1f82615eeb0be163f9a59de8d758763e482532aa622b5185eedf9a9949a9a21f", + "request_file_sha256": "448944d403fd373ca08ec60288f59196cbd38af4750d60ecef2ff06cf11e86e5", + "receipt_file_sha256": "4560df64ff806f92acb1a7edf11aaf80df883021aacd6a4c36a14058ee916b1f", + "receipt_body_text_sha256": "1361b32487bde550250041c2d821a0345531abcc77975dbcc2d25b2f6a1aeb0b", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_d8bf3c80-574a-491b-a572-852d3a1ae2fa", + "receipt_status": 200, + "prompt_tokens": 1724, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 0.9550234579946846 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-05-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "92bcd81e8aa31cb958443a77541a77b5c1a0441bf2bbec01c7d953ec46d0bb1d", + "request_file_sha256": "28e640cfb071d46aaea7825052938028220b560f07525c81f117fb7eae905851", + "receipt_file_sha256": "e453e485cba93a1bd28e19688585a0e208c9fa0ea113f4133817b827df41cdcc", + "receipt_body_text_sha256": "77b35c1cd48e0faf78c530da814d7d26d81acae657daba2cbec1f55d60b04458", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_63478531-4464-418a-9d0e-fe4c1cc43d1e", + "receipt_status": 200, + "prompt_tokens": 1681, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 0.9348693329957314 + } + } + }, + { + "history_id": "confirm-fresh-06-te-m0", + "case_id": "fresh-06-te", + "scenario": "fresh-06", + "model": "sarvam-105b", + "family": "missing_replacement", + "question": "VISIT-2106 కోసం ఇప్పుడు నేను కోరుకుంటున్న కొత్త తేదీ ఏది? చెప్పండి మాత్రమే; బుకింగ్ మార్చవద్దు.", + "source_messages": [ + { + "id": "note-6-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. VISIT-2106 కోసం ఇక 2027-04-07 వద్దు. బదులుగా ఏ తేదీ కావాలో ఇంకా నిర్ణయించలేదు. నా మరో బుకింగ్ VISIT-3106 వేరు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The customer withdraws an old choice for VISIT-2106 and explicitly has not selected a replacement. No unique new preferred date exists yet; null with clarification true records the missing customer selection, not ignorance of an established date.", + "expected_answer_from_source_and_frozen_label": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-06", + "id": "VISIT-2106", + "date": "2027-03-07", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-06", + "id": "VISIT-2106", + "date": "2027-03-07", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "541848996f56f280f7085b1caaf9268176680fde806debcac4d6936f77496cf5", + "case_sha256": "246ec15c8d2b0e1674cc16b9d16dc45d48ed0abccd8b317517fbcacecb390a1e", + "gold_label_sha256": "8f66da409f959655b329186d937b1537f0e71b8d6fc2312d7ec66b14679d743b", + "reports": { + "chronological": { + "id": "confirm-fresh-06-te-m0-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":null,\"needs_clarification\":false}", + "raw_output": { + "action_status": "already_applied", + "answer": null, + "needs_clarification": false + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "Null date is appropriate, but clarification=false misses the explicitly unresolved selection or condition.", + "status_review": "Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_requested.", + "request_sha256": "38191ce9cc5a75f6580b14a01f915a2ded45e6b6b1f38f796ccac2dd50b7f2e7", + "request_file_sha256": "1ee91dd575f9a8fd578e4089cb16a3ac0d2ddeed2bf14c34bdaf7f68bef62c55", + "receipt_file_sha256": "d9020b8fb6059fab7d0513a677f5631421d3a4e3a5f2d65b8f86ec31d7d2c276", + "receipt_body_text_sha256": "65ef5fe31e5e9b1f558969f5ab2c5173db583411ed2a7f9250d23cd517b6878d", + "result_file_sha256": "39d55d34faaf847cca522ee45ea4d0c2a58eb1da026891f66acd1aa872680135", + "raw_content_sha256": "014cf05703fb39bb36034af11e8d93487393e9f563651792b7d462bd5fb9849d", + "response_id": "20261014_a079beec-b225-4ad3-869a-f98ee92a5810", + "receipt_status": 200, + "prompt_tokens": 853, + "completion_tokens": 7725, + "reasoning_tokens": 7702, + "elapsed_seconds": 31.482956999985618 + }, + "grouped_lossless": { + "id": "confirm-fresh-06-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_performed conflicts with trusted intent and actual receipts, which establish not_requested.", + "request_sha256": "717f19fc9900b5b148e4b66fceb7b8cb699591d60b67c9d401da5ae8bf54d7ea", + "request_file_sha256": "6b3cb6a09da65198a5423fc108303cf572cd672661cefeee5a2bab14c93e8af5", + "receipt_file_sha256": "140daf2d8cb292de7061b8bf0eda5ce1b37fdededa40bf7beef68943dfe4db65", + "receipt_body_text_sha256": "7f35bbbfa96590df21a3ff91d2325dec82f7a85ec4985054d3f13a74362587c6", + "result_file_sha256": "dd1eb69535233b7875007bb6fe801cf730d36ee48921e94b961cca63df5956d8", + "raw_content_sha256": "7d095b99976dab5ff227619f70a19f42f66d68a5a2c06af0e27aaa3031a4096e", + "response_id": "20261014_de3b2279-09f0-4222-bafb-cd5611708f57", + "receipt_status": 200, + "prompt_tokens": 1292, + "completion_tokens": 2659, + "reasoning_tokens": 2636, + "elapsed_seconds": 13.880363666990888 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-06-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "1c470580d851f6ca1c43a47641781ddc4ee4f0c9ec5976f4d4316f14f095440c", + "request_file_sha256": "dcf05bb0910f2b852b6bfc3c71852a455ccce12044943218fd8a825cfeb51ac4", + "receipt_file_sha256": "5f9e31a39514d07937d87dbe391bcdfc519bf37233c330a81b460c2a39876fc8", + "receipt_body_text_sha256": "7769ef89cb97a6b26b5bb25cf0c58ca62952b3cf504734f4afe5fffd85ab9262", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "response_id": "20261014_995a7ab0-6c33-4982-a50d-6624488f8106", + "receipt_status": 200, + "prompt_tokens": 1254, + "completion_tokens": 1123, + "reasoning_tokens": 1100, + "elapsed_seconds": 6.485287333998713 + } + } + }, + { + "history_id": "confirm-fresh-06-te-m1", + "case_id": "fresh-06-te", + "scenario": "fresh-06", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "question": "VISIT-2106 కోసం ఇప్పుడు నేను కోరుకుంటున్న కొత్త తేదీ ఏది? చెప్పండి మాత్రమే; బుకింగ్ మార్చవద్దు.", + "source_messages": [ + { + "id": "note-6-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. VISIT-2106 కోసం ఇక 2027-04-07 వద్దు. బదులుగా ఏ తేదీ కావాలో ఇంకా నిర్ణయించలేదు. నా మరో బుకింగ్ VISIT-3106 వేరు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The customer withdraws an old choice for VISIT-2106 and explicitly has not selected a replacement. No unique new preferred date exists yet; null with clarification true records the missing customer selection, not ignorance of an established date.", + "expected_answer_from_source_and_frozen_label": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-06", + "id": "VISIT-2106", + "date": "2027-03-07", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-06", + "id": "VISIT-2106", + "date": "2027-03-07", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "216e18e06f93071b33e6715356da4441e77a5447084f64da5e1405995fe61014", + "case_sha256": "246ec15c8d2b0e1674cc16b9d16dc45d48ed0abccd8b317517fbcacecb390a1e", + "gold_label_sha256": "8f66da409f959655b329186d937b1537f0e71b8d6fc2312d7ec66b14679d743b", + "reports": { + "chronological": { + "id": "confirm-fresh-06-te-m1-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "b4bcbc06790a703cce9ffe2f4286a96befcf3a5b1a9eba1696e78ec053fb22d0", + "request_file_sha256": "daf1d9a4ee7f41355bbd550158e099bdc77d91f42734a7f3ec38392adb2e0f45", + "receipt_file_sha256": "2851b935e950a63c80c9544414d947077715e70390d597d58b6f7ca07d1ef647", + "receipt_body_text_sha256": "9a8f83405efe089b18c02e45e23d2fb4c4975803138851f2e00a358c5bb62e08", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_0233ba88-a158-4f94-a84c-2611fcce273b", + "receipt_status": 200, + "prompt_tokens": 877, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 0.990318208001554 + }, + "grouped_lossless": { + "id": "confirm-fresh-06-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "42fc2685515a8a855ab07626e4b7aa8aac2f2ae7ab77d8ea32eaf2615bb0a800", + "request_file_sha256": "e8da0fd7f651b3b1e92d4fd9c2f30f6110da868c5737643a2f63ae4a9db18c19", + "receipt_file_sha256": "d7e02176d061e4ceb73709b61cb0f64a56ef406bdd4425cafabf97c220d86912", + "receipt_body_text_sha256": "748c4270011d964a0d9fe8cee8898865070285b3fbf1d01bd2ac48bb2d64ab02", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_265d999d-b4cd-431c-b127-acf78bce4fec", + "receipt_status": 200, + "prompt_tokens": 1314, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 0.9463838749943534 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-06-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "fe931600c1efcf3ee60a46be7a2c8f9fbf341e029bb30e66ca72978e20e577fc", + "request_file_sha256": "8d96d79c8904a1895d8eaf8f573eafa135572a9c9848b55872b7399178b75185", + "receipt_file_sha256": "12bc68579eaccad389f7a85869bcd8b3123917497898f20860cd7273ea3f0569", + "receipt_body_text_sha256": "078c68aae6def55769ef00de0a4752c9e5b395bb3ace8a0bf1d3ab4dfc6a845f", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_f900172a-359d-42b9-81c7-a334ec59bdff", + "receipt_status": 200, + "prompt_tokens": 1256, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 1.0905270420043962 + } + } + }, + { + "history_id": "confirm-fresh-07-te-m0", + "case_id": "fresh-07-te", + "scenario": "fresh-07", + "model": "sarvam-105b", + "family": "missing_replacement", + "question": "VISIT-2107 కోసం ఇప్పుడు నేను కోరుకుంటున్న కొత్త తేదీ ఏది? చెప్పండి మాత్రమే; బుకింగ్ మార్చవద్దు.", + "source_messages": [ + { + "id": "note-7-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-7-1", + "role": "customer", + "content": "VISIT-2107 కోసం ఇక 2027-04-08 వద్దు." + }, + { + "id": "note-7-2", + "role": "customer", + "content": "బదులుగా ఏ తేదీ కావాలో ఇంకా నిర్ణయించలేదు. నా మరో బుకింగ్ VISIT-3107 వేరు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The customer withdraws an old choice for VISIT-2107 and explicitly has not selected a replacement. No unique new preferred date exists yet; null with clarification true records the missing customer selection, not ignorance of an established date.", + "expected_answer_from_source_and_frozen_label": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-07", + "id": "VISIT-2107", + "date": "2027-03-08", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-07", + "id": "VISIT-2107", + "date": "2027-03-08", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "0161bc225cd956dd178a3e58c1573a53fbfd57fc6c33391e6b6c59d28359ed11", + "case_sha256": "90367094135b0b5276ff4f1db40480178ade7d25088f1134632bf4b947e82391", + "gold_label_sha256": "f74e8f245fa85230485e5d096fde7362db8eee7a9c50259b85d60091f0aba052", + "reports": { + "chronological": { + "id": "confirm-fresh-07-te-m0-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "767444207e7a6c534ff4f8c4b03de565432aa089c24c720f11eddff212499356", + "request_file_sha256": "db13b94b3a39852b1aff2d467370ca6d0ddf5d229ba14602f5ba4cde73a760b9", + "receipt_file_sha256": "31cd93103f07b69604eb21d24ecea807c7e36498e0738afbcfb8c70543e6b08f", + "receipt_body_text_sha256": "f3bfc32b2c406e875831773f9ffb9cc8971a6d030850649b2e83ee6b415fdfb3", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "response_id": "20261014_e8570898-5bfe-47bb-b741-432a25356686", + "receipt_status": 200, + "prompt_tokens": 979, + "completion_tokens": 3153, + "reasoning_tokens": 3130, + "elapsed_seconds": 19.295718249995844 + }, + "grouped_lossless": { + "id": "confirm-fresh-07-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "69c562bf14d5ceafb68a567ad3b1f6ffd58c79a6279d63793208908d3054758f", + "request_file_sha256": "7bedd395b4f46ce6890d9d572f42b1ff5fe1cd50e874c37cf5f4aae9f84e3049", + "receipt_file_sha256": "14662a3000bc6e59ec05d1c3f5e8dce83356e53a173568cc3cc2e72ab03310f7", + "receipt_body_text_sha256": "5162a23f07965d51dd5d283f970cf2563f76a1ae5bc5adf8615d63b2bead8fb0", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "response_id": "20261014_65da79b8-b363-4675-a39b-43009020ca11", + "receipt_status": 200, + "prompt_tokens": 1527, + "completion_tokens": 5871, + "reasoning_tokens": 5848, + "elapsed_seconds": 27.6655987080012 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-07-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": false + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "Null date is appropriate, but clarification=false misses the explicitly unresolved selection or condition.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "ec0b37c23c8ea7cfaed81c03e889d3857565e1195269bd5da57097270f860bed", + "request_file_sha256": "2e3d6e1b10556a5b62d0fdce3cffe1ab2a4c0fa0b69ee2a81f741708a44c9749", + "receipt_file_sha256": "1bf8f5b72d602e1cc285f9ec9e9c9e607281c9651b8b68714579eca5eacbfab5", + "receipt_body_text_sha256": "1967e852e9d9c0447574f423c1039b18691a8875577346b61e1aadb5b0c1095d", + "result_file_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "raw_content_sha256": "837d7c4d0820dbbaaedba89ceca07d3f62ea5913ac395c7558beddcbae373aa6", + "response_id": "20261014_edf8ae56-92a9-495b-88dc-0b6c9abf7270", + "receipt_status": 200, + "prompt_tokens": 1400, + "completion_tokens": 3578, + "reasoning_tokens": 3555, + "elapsed_seconds": 19.214640292004333 + } + } + }, + { + "history_id": "confirm-fresh-07-te-m1", + "case_id": "fresh-07-te", + "scenario": "fresh-07", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "question": "VISIT-2107 కోసం ఇప్పుడు నేను కోరుకుంటున్న కొత్త తేదీ ఏది? చెప్పండి మాత్రమే; బుకింగ్ మార్చవద్దు.", + "source_messages": [ + { + "id": "note-7-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-7-1", + "role": "customer", + "content": "VISIT-2107 కోసం ఇక 2027-04-08 వద్దు." + }, + { + "id": "note-7-2", + "role": "customer", + "content": "బదులుగా ఏ తేదీ కావాలో ఇంకా నిర్ణయించలేదు. నా మరో బుకింగ్ VISIT-3107 వేరు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The customer withdraws an old choice for VISIT-2107 and explicitly has not selected a replacement. No unique new preferred date exists yet; null with clarification true records the missing customer selection, not ignorance of an established date.", + "expected_answer_from_source_and_frozen_label": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-07", + "id": "VISIT-2107", + "date": "2027-03-08", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-07", + "id": "VISIT-2107", + "date": "2027-03-08", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "4712b888f147828ffc56924838cd123ac0e8a9d2fe15b9791a2f64a2e2463fbd", + "case_sha256": "90367094135b0b5276ff4f1db40480178ade7d25088f1134632bf4b947e82391", + "gold_label_sha256": "f74e8f245fa85230485e5d096fde7362db8eee7a9c50259b85d60091f0aba052", + "reports": { + "chronological": { + "id": "confirm-fresh-07-te-m1-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "bc173161ed79d92c8bd50f138455309289339ea9d36c88155a445ddb2568b7e6", + "request_file_sha256": "1dcb9551c93ef345630867c77efe17785bdfe2be0904215917b0eb9952daaa72", + "receipt_file_sha256": "d9aceb53ff81ab16dbf44ba648c10ea06d7bb7e5cb0d8c8647669b527a08c27d", + "receipt_body_text_sha256": "01066dd9344a93610b9fca86ee46e6f37354ab7dc855eb6c132a3cdb42d3f891", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_7f680cc8-1845-42df-b9e7-d2f252c626fb", + "receipt_status": 200, + "prompt_tokens": 905, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 0.9890587080008117 + }, + "grouped_lossless": { + "id": "confirm-fresh-07-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "c6c8b9a6b74c25aa59907255a6e64a08535d0abfaae171412d7b49344063da44", + "request_file_sha256": "c7690f40df10299c9eeef2621330a2862e1b29993dba71b661235efdeeff39b7", + "receipt_file_sha256": "be6fa0dd55fc4223a70b4ab4f364b9075068ac11cf7623c329cd5208dd59c1fe", + "receipt_body_text_sha256": "6b3f1478a93f6078a4418693ae1703fadef30f516761a62c9dbdc24234d59cd5", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_0e7bdf44-f242-4777-a060-63415a86662a", + "receipt_status": 200, + "prompt_tokens": 1449, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 1.0261834590055514 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-07-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "65b3cdb4c59f44ba856be80387325f390da09176905c449398e7f9b33788100f", + "request_file_sha256": "4cd7ccd15326bdadcfd18f370cb88918102671b4eea80365d10a6ed8390ddc10", + "receipt_file_sha256": "5d828e52719cd28b59638e20b592bf9208dcc56b121176fe2f660a5f1145aae2", + "receipt_body_text_sha256": "c68f6a168260ffd86817d52878444c134bbb25297723c717ae9f372c815517d2", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_d24ae53e-fbc4-4483-9183-a845559936f6", + "receipt_status": 200, + "prompt_tokens": 1400, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 1.009558958990965 + } + } + }, + { + "history_id": "confirm-fresh-08-te-m0", + "case_id": "fresh-08-te", + "scenario": "fresh-08", + "model": "sarvam-105b", + "family": "missing_replacement", + "question": "VISIT-2108 కోసం ఇప్పుడు నేను కోరుకుంటున్న కొత్త తేదీ ఏది? చెప్పండి మాత్రమే; బుకింగ్ మార్చవద్దు.", + "source_messages": [ + { + "id": "note-8-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. VISIT-2108 కోసం ఇక 2027-04-09 వద్దు. బదులుగా ఏ తేదీ కావాలో ఇంకా నిర్ణయించలేదు. నా మరో బుకింగ్ VISIT-3108 వేరు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The customer withdraws an old choice for VISIT-2108 and explicitly has not selected a replacement. No unique new preferred date exists yet; null with clarification true records the missing customer selection, not ignorance of an established date.", + "expected_answer_from_source_and_frozen_label": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-08", + "id": "VISIT-2108", + "date": "2027-03-09", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-08", + "id": "VISIT-2108", + "date": "2027-03-09", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-bafdb06925481c6f", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2108" + }, + "response": { + "date": "2027-03-09", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "9e28ac351738517054f2f19938181f5434d802c65d9634a2d0354aab02a9f140", + "case_sha256": "2e0d4c9f243b52ca0df844236dbf558dd869c0338a4528b9b1ade9187df78877", + "gold_label_sha256": "95896d752c4aaa45ac6037b61528a856cf0d2cac93ad410144254c4bf5993a56", + "reports": { + "chronological": { + "id": "confirm-fresh-08-te-m0-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_performed", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_performed conflicts with trusted intent and actual receipts, which establish not_requested.", + "request_sha256": "2d77fc304d5b77e9321723568130ebd31f7c163db4ea9c607d10f9293197f09d", + "request_file_sha256": "306d6e73361c7525ae15ad4c249f25209a0c51b8209032e82af7410a3f448ea2", + "receipt_file_sha256": "aefc9ad4c36fe0b3485b487bbe6e63c25807e738e9ec15559b4f1f61b3be725e", + "receipt_body_text_sha256": "d7dc31e0f9174102b2926ff7bf7dd13d214860fa1ab66cd046abcc930944693c", + "result_file_sha256": "dd1eb69535233b7875007bb6fe801cf730d36ee48921e94b961cca63df5956d8", + "raw_content_sha256": "7d095b99976dab5ff227619f70a19f42f66d68a5a2c06af0e27aaa3031a4096e", + "response_id": "20261014_051c6be4-c059-46f9-a7b5-9788f0cf2212", + "receipt_status": 200, + "prompt_tokens": 1068, + "completion_tokens": 2573, + "reasoning_tokens": 2550, + "elapsed_seconds": 13.018374209001195 + }, + "grouped_lossless": { + "id": "confirm-fresh-08-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "117137468ea600d892787aee226e9594be8de8767d484315cc2880ff2c757886", + "request_file_sha256": "dd6358f5489d7787cb92706cd2a3e36f9f3f8d86b8fb3ad3f3d41d23337e885b", + "receipt_file_sha256": "becf70599ad43d09f63025faffa065d49a940d33b6db4b1bf037e723517ae049", + "receipt_body_text_sha256": "c710531faff2e7fb2f8a36ba6f0d1a50c0fceafa8cf713252f94d7636c54ecb0", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "response_id": "20261014_55c2bd31-a263-48c1-a5e6-4d0ba81ece98", + "receipt_status": 200, + "prompt_tokens": 1646, + "completion_tokens": 2671, + "reasoning_tokens": 2648, + "elapsed_seconds": 13.80287416599458 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-08-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "019d1e7967c0b71b9f629bbc4e763ddf545fdf1503ee0b1b5a60353b2d3c63b9", + "request_file_sha256": "190d2f006a901433d921a969a417d81ff25abd406c7c17e13f21c5ea716156eb", + "receipt_file_sha256": "ec0540efc68ca6aa7534a662ea0b02a637a2727e01fb701f474ac52a5058b88b", + "receipt_body_text_sha256": "3dacff067838e8a84df50d98b4c41acd43408feabb7743912cb4ec95b2de22b9", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "response_id": "20261014_7dab9f27-d591-46b9-b43a-a8e089192c17", + "receipt_status": 200, + "prompt_tokens": 1529, + "completion_tokens": 2211, + "reasoning_tokens": 2188, + "elapsed_seconds": 12.61775991700415 + } + } + }, + { + "history_id": "confirm-fresh-08-te-m1", + "case_id": "fresh-08-te", + "scenario": "fresh-08", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "question": "VISIT-2108 కోసం ఇప్పుడు నేను కోరుకుంటున్న కొత్త తేదీ ఏది? చెప్పండి మాత్రమే; బుకింగ్ మార్చవద్దు.", + "source_messages": [ + { + "id": "note-8-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. VISIT-2108 కోసం ఇక 2027-04-09 వద్దు. బదులుగా ఏ తేదీ కావాలో ఇంకా నిర్ణయించలేదు. నా మరో బుకింగ్ VISIT-3108 వేరు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The customer withdraws an old choice for VISIT-2108 and explicitly has not selected a replacement. No unique new preferred date exists yet; null with clarification true records the missing customer selection, not ignorance of an established date.", + "expected_answer_from_source_and_frozen_label": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-08", + "id": "VISIT-2108", + "date": "2027-03-09", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-08", + "id": "VISIT-2108", + "date": "2027-03-09", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "b75b650d6f2a2fb7108a342fb98da5bd35c52b4479d6d2797c8dd12910e1f4a3", + "case_sha256": "2e0d4c9f243b52ca0df844236dbf558dd869c0338a4528b9b1ade9187df78877", + "gold_label_sha256": "95896d752c4aaa45ac6037b61528a856cf0d2cac93ad410144254c4bf5993a56", + "reports": { + "chronological": { + "id": "confirm-fresh-08-te-m1-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "7ab43af2f8dea39bea93de850e65945706e5e8ece72132baaf30dbb3ea2ac445", + "request_file_sha256": "d884ad82e4ca6acf8b27b4d939af4f1690f5bece0722aca9dffdc85242232892", + "receipt_file_sha256": "de6eb6cdd12c92643769e00eeb44ea75ea16aeaf6d826c8e8ae2fb58af30916f", + "receipt_body_text_sha256": "8ce7c3c4ef834e45a492366f4baf1cd9b1e5b3e10c07ebc7b066fed7a005517d", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_6c099287-ea0f-4a0a-8671-0283cdad0445", + "receipt_status": 200, + "prompt_tokens": 896, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 1.0703550420003012 + }, + "grouped_lossless": { + "id": "confirm-fresh-08-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "bb9cf2d2487405f7842b5c5c2d6a11b61949576c55bf110240fa08246363d3c1", + "request_file_sha256": "4fb6d612e543ad9fecdb1756931c2ee2fab35bdaedd408f41eaa488fb3fd0045", + "receipt_file_sha256": "021ef267b5f6d53eacabf40bd78755a0220af1264d80428d23486387d40d831e", + "receipt_body_text_sha256": "ca1995f157b85f8c95ed7c5a44b9a72a2c898d45f1b527aef8411a0bda362c84", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_7acd10e9-7007-4350-982e-eca0af73a783", + "receipt_status": 200, + "prompt_tokens": 1330, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 0.9840972910023993 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-08-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "d2b9f4783b544280d83d4dd30b8732b671b434e8d24eb53b593d5645db5d7b7e", + "request_file_sha256": "7bc3761ea197c84dc89c0ad2be00e52ae2d748039a3ca2e4130e842b14243d89", + "receipt_file_sha256": "523667f0c011184a69bdfa943740d08a3c625665fd74bc278f6bc474183b1a61", + "receipt_body_text_sha256": "e22700e115c6eac24ef50b18d1ec206b30e3ae75c4b15c56783debf0a6f4dedc", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_d18d249b-e298-4727-a6b3-9d77a288b656", + "receipt_status": 200, + "prompt_tokens": 1253, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 0.9449477090092842 + } + } + }, + { + "history_id": "confirm-fresh-09-te-m0", + "case_id": "fresh-09-te", + "scenario": "fresh-09", + "model": "sarvam-105b", + "family": "resolved_condition", + "question": "చెప్పిన షరతు ప్రకారం VISIT-2109 కోసం ఏ తేదీ ఎంచుకోవాలి? మార్చవద్దు.", + "source_messages": [ + { + "id": "note-9-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-9-1", + "role": "customer", + "content": "VISIT-2109 కోసం ఉదయం సమయం ఉంటే 2027-04-10, లేకపోతే 2027-05-10 ఎంచుకోండి." + }, + { + "id": "note-9-2", + "role": "customer", + "content": "ఉదయం సమయం అందుబాటులో ఉంది. నా ఎంపిక గురించి మాత్రమే అడుగుతున్నాను; బుకింగ్ మార్చడానికి అనుమతి ఇవ్వడం లేదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source gives two conditional choices for VISIT-2109 and explicitly resolves the condition as available morning. That branch selects 2027-04-10; no booking change was requested.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-09", + "id": "VISIT-2109", + "date": "2027-03-10", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-09", + "id": "VISIT-2109", + "date": "2027-03-10", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "54c37ff13d1adecbbfce81b668cdb6664b9d3200eb1e228b4654349531317508", + "case_sha256": "c0f3ce2668b372dbc036d0b629bebd0e2c585faaaba64260f83540c402141c5d", + "gold_label_sha256": "7c364c9a42e7e3e560daf5629c1a0ccf0b68e923271f0b8a1a011a669533011e", + "reports": { + "chronological": { + "id": "confirm-fresh-09-te-m0-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-10\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "31e37ce5631758d870897f3fce6f073848fd2f6d4b207822c724d41445fd9fa0", + "request_file_sha256": "eb50d36a9afc48df13eee188facfef00b3bd239b14c82d4fe2486eeb614c5f6c", + "receipt_file_sha256": "d61287118f6b970722a149add03080896a47389b9736d70eb1b31e08096d9eb3", + "receipt_body_text_sha256": "ea584e57d76084046c783bf3dd9663d3ff1c30c108a95041b3d97ff336474bcd", + "result_file_sha256": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "raw_content_sha256": "b5203780d91957fd9ffd4fed596dd501556d13bf3dc68e7bd2d239524382871b", + "response_id": "20261014_99bb72bb-e5f7-445e-95f1-b66e44d93083", + "receipt_status": 200, + "prompt_tokens": 1056, + "completion_tokens": 1766, + "reasoning_tokens": 1734, + "elapsed_seconds": 9.148343624998233 + }, + "grouped_lossless": { + "id": "confirm-fresh-09-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-10\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "e22d688902539f8193a49d13b2052903bb327ceedc88c8885db298d3df07d12f", + "request_file_sha256": "021901b18c68b1dcfab0f89563a8a807506cd110d9053049c8268a31390ef5b1", + "receipt_file_sha256": "86b54d42bb8ede6397d43e9e5e9de7b4df9fe375c914f9a6b22b96de01a3d280", + "receipt_body_text_sha256": "b03d277bde2bcbf764c8d6e9a82ad28f8fdafb734269eccb4ee0b9f5ffc41ae3", + "result_file_sha256": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "raw_content_sha256": "b5203780d91957fd9ffd4fed596dd501556d13bf3dc68e7bd2d239524382871b", + "response_id": "20261014_0a2aa2ae-68f9-4901-a229-6091f32dc501", + "receipt_status": 200, + "prompt_tokens": 1600, + "completion_tokens": 1840, + "reasoning_tokens": 1808, + "elapsed_seconds": 12.543137667002156 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-09-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-10\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "e748429aab219a7963bb9ce9a8760756b371d0f46c4642dd67fadbc3b4c6b883", + "request_file_sha256": "513d50a380bd7db584b917af08c8fbc9d5ccc1c639d2a5d11c6d5030a46953d9", + "receipt_file_sha256": "e88bd508058520b962f3cd0c5c77b954ca2ce039469b2cd1097c302b0489079b", + "receipt_body_text_sha256": "55cb2f692a6f5f67f92d22006fa0ea5a82853f04e512c10a01a151b0f78f5d49", + "result_file_sha256": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "raw_content_sha256": "b5203780d91957fd9ffd4fed596dd501556d13bf3dc68e7bd2d239524382871b", + "response_id": "20261014_6e33bb2d-d0ed-42f5-b4ad-e37581f2f7ad", + "receipt_status": 200, + "prompt_tokens": 1405, + "completion_tokens": 2869, + "reasoning_tokens": 2837, + "elapsed_seconds": 17.790086916997097 + } + } + }, + { + "history_id": "confirm-fresh-09-te-m1", + "case_id": "fresh-09-te", + "scenario": "fresh-09", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "question": "చెప్పిన షరతు ప్రకారం VISIT-2109 కోసం ఏ తేదీ ఎంచుకోవాలి? మార్చవద్దు.", + "source_messages": [ + { + "id": "note-9-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-9-1", + "role": "customer", + "content": "VISIT-2109 కోసం ఉదయం సమయం ఉంటే 2027-04-10, లేకపోతే 2027-05-10 ఎంచుకోండి." + }, + { + "id": "note-9-2", + "role": "customer", + "content": "ఉదయం సమయం అందుబాటులో ఉంది. నా ఎంపిక గురించి మాత్రమే అడుగుతున్నాను; బుకింగ్ మార్చడానికి అనుమతి ఇవ్వడం లేదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source gives two conditional choices for VISIT-2109 and explicitly resolves the condition as available morning. That branch selects 2027-04-10; no booking change was requested.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-09", + "id": "VISIT-2109", + "date": "2027-03-10", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-09", + "id": "VISIT-2109", + "date": "2027-03-10", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-afb2813b31a13957", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2109" + }, + "response": { + "date": "2027-03-10", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "2f1cd6f1ac16ff9772bf9c48c1937f7dcbf61a9a597520f68e900f1176df6881", + "case_sha256": "c0f3ce2668b372dbc036d0b629bebd0e2c585faaaba64260f83540c402141c5d", + "gold_label_sha256": "7c364c9a42e7e3e560daf5629c1a0ccf0b68e923271f0b8a1a011a669533011e", + "reports": { + "chronological": { + "id": "confirm-fresh-09-te-m1-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-10\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "7943232fe995671cd4fef101630dc33a718ed067ad84b09594f9d7e753598011", + "request_file_sha256": "4fb1e879836df8127b37931eb0a3f2004489395816b5db33c10718cfcb7c4598", + "receipt_file_sha256": "e521af0b73018c2c2460f1992d77d195db1afce9a731a6e2d9f9f4f9f6d19da9", + "receipt_body_text_sha256": "40822b48e39dd7480152330ebe34f5f762a929c76bd44994caee90f018abc1a2", + "result_file_sha256": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "raw_content_sha256": "5f5a8175c7868d3fdb706311acadab639b8b5525cf4e1d717def813d9e5c10af", + "response_id": "20261014_24c98210-14dc-40c8-a131-75eec993cac6", + "receipt_status": 200, + "prompt_tokens": 1208, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0040511660045013 + }, + "grouped_lossless": { + "id": "confirm-fresh-09-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-10\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "43e2db0c772acdb2807ec85fb080ef9cabe8b20c8f343b862777e89fb924bac5", + "request_file_sha256": "08f5794f1d60714516134c1214b1977685ea530703b82e0863ffcd75f470e7b0", + "receipt_file_sha256": "a172dfba032da0780c300a8fe26be667f1271b1976cab9efaed7b8f8c52b1def", + "receipt_body_text_sha256": "4c0bf8e37a0804ed80cf3ca9e597b85ea1891f52c5716240e88233416c06567b", + "result_file_sha256": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "raw_content_sha256": "5f5a8175c7868d3fdb706311acadab639b8b5525cf4e1d717def813d9e5c10af", + "response_id": "20261014_274a2500-b8ed-4292-aefd-9850f0e7e162", + "receipt_status": 200, + "prompt_tokens": 1911, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.059681416998501 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-09-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-10\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-10", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "cd0cb409804c925a0c5f4ed3fd095fa32fb70713bf350c5f01461882ad74a433", + "request_file_sha256": "83f7758bcdb3dccb3ac84fff283c08efdb5b981409491453e7f5980649dc382b", + "receipt_file_sha256": "c7ae1fe588adb4734029095b3af01219cbad7af5ddfaaee23ba05bdee6042821", + "receipt_body_text_sha256": "c66cb65a7c708fc035cc9d415feb9868948ffe7c23a9ae66f5d8f12ade6204ef", + "result_file_sha256": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "raw_content_sha256": "5f5a8175c7868d3fdb706311acadab639b8b5525cf4e1d717def813d9e5c10af", + "response_id": "20261014_a6e58621-1559-4ac8-a675-aa09e12359a9", + "receipt_status": 200, + "prompt_tokens": 1706, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9720865410054103 + } + } + }, + { + "history_id": "confirm-fresh-10-te-m0", + "case_id": "fresh-10-te", + "scenario": "fresh-10", + "model": "sarvam-105b", + "family": "resolved_condition", + "question": "చెప్పిన షరతు ప్రకారం VISIT-2110 కోసం ఏ తేదీ ఎంచుకోవాలి? మార్చవద్దు.", + "source_messages": [ + { + "id": "note-10-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. VISIT-2110 కోసం ఉదయం సమయం ఉంటే 2027-04-11, లేకపోతే 2027-05-11 ఎంచుకోండి. ఉదయం సమయం అందుబాటులో లేదు. నా ఎంపిక గురించి మాత్రమే అడుగుతున్నాను." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source gives two conditional choices for VISIT-2110 and explicitly resolves the condition as unavailable morning. That branch selects 2027-05-11; no booking change was requested.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-10", + "id": "VISIT-2110", + "date": "2027-03-11", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-10", + "id": "VISIT-2110", + "date": "2027-03-11", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "92e9c5ea46e4b652f48842a296cecaccb5a5800ff979c56d08e31195c5f74999", + "case_sha256": "8af8a2b74328ce1c9ef513d41c96e21a525a0070dc6bdf80273decdd3aa36633", + "gold_label_sha256": "b8a52ce4676ca23feffcd8ade94a2a0dbcea555da8f91c1a2e155aca6a69688d", + "reports": { + "chronological": { + "id": "confirm-fresh-10-te-m0-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_requested.", + "request_sha256": "e7d678b137b4263db7fafd7ff43bb3564af22287ee44fde9d2f24b4645a4bd2d", + "request_file_sha256": "6ef7b2d387fbee2c549c308a3b625b9cf29ddb45df3d96f84c39bf941a4a0b36", + "receipt_file_sha256": "faebd656d6736e91a447d345f29ce3d8d3647ad4cf833a4649ee6a03a0804b93", + "receipt_body_text_sha256": "a5e8bbdc5b28c22128ac9fb81b7ecc09694f7860921075d4b350d59b2e7cf4c5", + "result_file_sha256": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "raw_content_sha256": "9dac33ab5e4882ec69311a19f72c1330ec8f057e21a912a81f03d812c2e99127", + "response_id": "20261014_afe772fd-851d-4ee3-9301-e6ae104ba07c", + "receipt_status": 200, + "prompt_tokens": 901, + "completion_tokens": 4798, + "reasoning_tokens": 4766, + "elapsed_seconds": 21.312018374999752 + }, + "grouped_lossless": { + "id": "confirm-fresh-10-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "2e205e24bf20a57e3eb111d0a70d31781a600764b365e742ac04198b12676d90", + "request_file_sha256": "7281331ce9ba5ffc1ad5eec9bc118ce52a0fc1a0bf33be71d2d5f444eb509e8a", + "receipt_file_sha256": "afb5d450a5a88a9c78cd7d89563216e4f464c2e3c8dfb31e95003c317d767b68", + "receipt_body_text_sha256": "f91938bac85230557ede8db4a3d324abc4ea80cc29ac6a54cbe7f0c1c773a844", + "result_file_sha256": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "raw_content_sha256": "2362fc24e9ad026848ab17e8b29cf8890528a030252a998e2bd4367263250458", + "response_id": "20261014_920f1e34-de42-48bf-bb82-9bfd3b9971f2", + "receipt_status": 200, + "prompt_tokens": 1335, + "completion_tokens": 5865, + "reasoning_tokens": 5833, + "elapsed_seconds": 26.295170958997915 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-10-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\": \"already_applied\", \"answer\": null, \"needs_clarification\": true}", + "raw_output": { + "action_status": "already_applied", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": true, + "factual_review": "False abstention: source or final backend uniquely establishes 2027-05-11, but raw answer is null.", + "status_review": "Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_requested.", + "request_sha256": "61621115950e170164f471e3d3c1aa1490f58b910ba2da86ca5d6f951aa5e32b", + "request_file_sha256": "34ca6d400007484dc46ac3ec32fd185fb527daa3498f5e071a65a7f516bf8a13", + "receipt_file_sha256": "5ce9f86d4150d4b30a6e05f2aebae6f49f548dcbb3541b706fec18adbd9f0ac5", + "receipt_body_text_sha256": "18f9e99ca1169c2191bdd9f5ea2dfeb84ebce0105df3e2456320a90a9b82c0b2", + "result_file_sha256": "08e1e6c34e14366850d3af627a73f020762ab2bf6e4855e1cf025b0428ef955c", + "raw_content_sha256": "e3875ab43c67feba4f2cc12995ca2de6b5ea8d0758ee5e07393ef5123bf343f6", + "response_id": "20261014_7eab04e8-4081-40da-ac15-7bf84cd07431", + "receipt_status": 200, + "prompt_tokens": 1251, + "completion_tokens": 5542, + "reasoning_tokens": 5512, + "elapsed_seconds": 23.39471333300753 + } + } + }, + { + "history_id": "confirm-fresh-10-te-m1", + "case_id": "fresh-10-te", + "scenario": "fresh-10", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "question": "చెప్పిన షరతు ప్రకారం VISIT-2110 కోసం ఏ తేదీ ఎంచుకోవాలి? మార్చవద్దు.", + "source_messages": [ + { + "id": "note-10-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. VISIT-2110 కోసం ఉదయం సమయం ఉంటే 2027-04-11, లేకపోతే 2027-05-11 ఎంచుకోండి. ఉదయం సమయం అందుబాటులో లేదు. నా ఎంపిక గురించి మాత్రమే అడుగుతున్నాను." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source gives two conditional choices for VISIT-2110 and explicitly resolves the condition as unavailable morning. That branch selects 2027-05-11; no booking change was requested.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-10", + "id": "VISIT-2110", + "date": "2027-03-11", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-10", + "id": "VISIT-2110", + "date": "2027-03-11", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-bfbbceec9f231ead", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2110" + }, + "response": { + "date": "2027-03-11", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "b920ad62cebdd47993c243e5fbca97c7593152d81b0dffbf8b9e53844451a351", + "case_sha256": "8af8a2b74328ce1c9ef513d41c96e21a525a0070dc6bdf80273decdd3aa36633", + "gold_label_sha256": "b8a52ce4676ca23feffcd8ade94a2a0dbcea555da8f91c1a2e155aca6a69688d", + "reports": { + "chronological": { + "id": "confirm-fresh-10-te-m1-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "e8a292ca0aadefa34d635fd0ce95ff941715cb89fdf0c308c36682a1b14cbdbe", + "request_file_sha256": "f4d160f092ccb70d5be8fc73b0a261e3d074809b89ae080e5f23d88ef19775c2", + "receipt_file_sha256": "46b161ba6e2dcaf97b0b34e5ce5d0f2ee432b2a04112d7c02982bd326eeb714d", + "receipt_body_text_sha256": "ed1996730cd49652a733d5eda23155c2fe106686e41e58f93693975edec59599", + "result_file_sha256": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "raw_content_sha256": "2655156e40334761b11243498c952a7bfa50bffe056b93d1424ef09be4cb336c", + "response_id": "20261014_4d581bee-7c31-4167-85bb-9d43f0bb9f29", + "receipt_status": 200, + "prompt_tokens": 1105, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0359863749908982 + }, + "grouped_lossless": { + "id": "confirm-fresh-10-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "84284697c2beafd0f72d036acfd01129293c6a866bd0bfefb8c17bd36338fb6c", + "request_file_sha256": "f7d19c179a78f6f8467bfd91ea403ed7f9a79cf084c026ce469735fde22fecab", + "receipt_file_sha256": "79824594395dcba4dfefb8c9ae2448374cc158954e81cbffe1db81311e0e2198", + "receipt_body_text_sha256": "9d5b7d20e39fd6b73fdc12b9d9b8da78d1438d6e52b7369227340e86b486b2cd", + "result_file_sha256": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "raw_content_sha256": "2655156e40334761b11243498c952a7bfa50bffe056b93d1424ef09be4cb336c", + "response_id": "20261014_d5ba00d8-c6e9-4298-998e-9b190c968821", + "receipt_status": 200, + "prompt_tokens": 1685, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0108522079972317 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-10-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "8f14f5f7f5d2cbae5c1275b803bcde7ed9130599410d1c717c2211f0f3095435", + "request_file_sha256": "bb2158fefbfeaf9ed22e0a29aebe1ab1deecd94d4072d0d513d35fe4301e00e3", + "receipt_file_sha256": "0a6d44cb102963056d03dc448d3b5c1b33b49cdeddef817e97ed4a1b525ecff4", + "receipt_body_text_sha256": "7387baaac039158eccf27ef8cf30e4088dcb13cdf852298c6e218a11684c0ca2", + "result_file_sha256": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "raw_content_sha256": "2655156e40334761b11243498c952a7bfa50bffe056b93d1424ef09be4cb336c", + "response_id": "20261014_d5bfc2e5-2293-4c99-9b1a-30f7b2647af8", + "receipt_status": 200, + "prompt_tokens": 1529, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.947349750000285 + } + } + }, + { + "history_id": "confirm-fresh-11-te-m0", + "case_id": "fresh-11-te", + "scenario": "fresh-11", + "model": "sarvam-105b", + "family": "resolved_condition", + "question": "చెప్పిన షరతు ప్రకారం VISIT-2111 కోసం ఏ తేదీ ఎంచుకోవాలి? మార్చవద్దు.", + "source_messages": [ + { + "id": "note-11-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-11-1", + "role": "customer", + "content": "VISIT-2111 కోసం ఉదయం సమయం ఉంటే 2027-04-12, లేకపోతే 2027-05-12 ఎంచుకోండి." + }, + { + "id": "note-11-2", + "role": "customer", + "content": "ఉదయం సమయం అందుబాటులో ఉంది. నా ఎంపిక గురించి మాత్రమే అడుగుతున్నాను; బుకింగ్ మార్చడానికి అనుమతి ఇవ్వడం లేదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source gives two conditional choices for VISIT-2111 and explicitly resolves the condition as available morning. That branch selects 2027-04-12; no booking change was requested.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-11", + "id": "VISIT-2111", + "date": "2027-03-12", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-11", + "id": "VISIT-2111", + "date": "2027-03-12", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "ffd085cd3ab9d2485dfc44e7e9aa312bd20bed4fcd47bade404c603a66492638", + "case_sha256": "e127f573bce504727566e5757c950db4b71f351bf7e630ebeb85858a6ef51030", + "gold_label_sha256": "0b3b1241f61db04020b2b84dcd63fa41fe46630820ce5d88c6c276f68d009173", + "reports": { + "chronological": { + "id": "confirm-fresh-11-te-m0-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-12\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "b7eb782b952d64b3983d9571a16399e83994dc7a4a0c33a8c7b7416ad69e238e", + "request_file_sha256": "20eac5c79ab10de74f4f6c886c75f11502098cf59320908aba31a38d4a25ebfa", + "receipt_file_sha256": "3e5e17f064325598ee8631174e1dee40e7736a1f4e3863d03d1ab5bd958b8971", + "receipt_body_text_sha256": "031dcd8ddebe0a7c7e09c8408c763f77ac31279f358e71a2087b6a3b519a8bd3", + "result_file_sha256": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "raw_content_sha256": "16ab820924508bb6e38b21bb14605348846c4b7936adf81e2b291d962d86654f", + "response_id": "20261014_498346db-e788-4443-b149-f7ff65355e15", + "receipt_status": 200, + "prompt_tokens": 981, + "completion_tokens": 3225, + "reasoning_tokens": 3193, + "elapsed_seconds": 15.773785208002664 + }, + "grouped_lossless": { + "id": "confirm-fresh-11-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-12\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "c6193b9f5b920dfbdf20f36bb4ad62fc6da64dc658d87c65c192a9cc7d8f873c", + "request_file_sha256": "b49d1ca48cf39433c4e9f7adcc3d553b70992052cfdb5397d7d9bbcdf51b6103", + "receipt_file_sha256": "1b2a593e5868135df146d15a244e4bf4a4d54461894fc8694a4249c9df91093e", + "receipt_body_text_sha256": "62ee831e60ed4b02d3fe7426f1289aa21dfb9449bf6fe52c169eb2c17bebe44f", + "result_file_sha256": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "raw_content_sha256": "16ab820924508bb6e38b21bb14605348846c4b7936adf81e2b291d962d86654f", + "response_id": "20261014_5e8c3887-dcc8-4941-acdd-0332d9104027", + "receipt_status": 200, + "prompt_tokens": 1522, + "completion_tokens": 3830, + "reasoning_tokens": 3798, + "elapsed_seconds": 16.252762250005617 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-11-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-12\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "61cc3d9723d94348d9edc90ce6c4bacbadb4aee7f637d17b93789b88069f2388", + "request_file_sha256": "51f1cba31314840bc676890dd5a840b27f4527f3cdbd3648d3f30920458533d1", + "receipt_file_sha256": "17cad71e3d03dee5e383b94479ef704ad781082cd4dc36eeabed09b2218a8cdc", + "receipt_body_text_sha256": "d752478b0635250b955e611682c069f26ef6c64c39353a49eb3103a9ee217cf4", + "result_file_sha256": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "raw_content_sha256": "16ab820924508bb6e38b21bb14605348846c4b7936adf81e2b291d962d86654f", + "response_id": "20261014_661a95e9-9f16-4f15-98ae-979d29cabe13", + "receipt_status": 200, + "prompt_tokens": 1407, + "completion_tokens": 3985, + "reasoning_tokens": 3953, + "elapsed_seconds": 16.09192604100099 + } + } + }, + { + "history_id": "confirm-fresh-11-te-m1", + "case_id": "fresh-11-te", + "scenario": "fresh-11", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "question": "చెప్పిన షరతు ప్రకారం VISIT-2111 కోసం ఏ తేదీ ఎంచుకోవాలి? మార్చవద్దు.", + "source_messages": [ + { + "id": "note-11-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-11-1", + "role": "customer", + "content": "VISIT-2111 కోసం ఉదయం సమయం ఉంటే 2027-04-12, లేకపోతే 2027-05-12 ఎంచుకోండి." + }, + { + "id": "note-11-2", + "role": "customer", + "content": "ఉదయం సమయం అందుబాటులో ఉంది. నా ఎంపిక గురించి మాత్రమే అడుగుతున్నాను; బుకింగ్ మార్చడానికి అనుమతి ఇవ్వడం లేదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source gives two conditional choices for VISIT-2111 and explicitly resolves the condition as available morning. That branch selects 2027-04-12; no booking change was requested.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-11", + "id": "VISIT-2111", + "date": "2027-03-12", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-11", + "id": "VISIT-2111", + "date": "2027-03-12", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9a95d393dd5e2be0", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2111" + }, + "response": { + "date": "2027-03-12", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "2d10d135c16f663a0d63723e081d6938cea803021d8e088f07c71100b7436623", + "case_sha256": "e127f573bce504727566e5757c950db4b71f351bf7e630ebeb85858a6ef51030", + "gold_label_sha256": "0b3b1241f61db04020b2b84dcd63fa41fe46630820ce5d88c6c276f68d009173", + "reports": { + "chronological": { + "id": "confirm-fresh-11-te-m1-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-12\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "f085de7ccd9960b5caa20f7df823e8835980987dfa9c8eecfa23264db3345075", + "request_file_sha256": "a72bfc673c1d997c905b85f4da77f2a5ac5aaa07da4193db462508a6cdc74b14", + "receipt_file_sha256": "8e45ad5b42d05cebf152fac7d9db380573fc722561eba7719a1de613ec5a113b", + "receipt_body_text_sha256": "3891ceb1d37f72061e19b7b14a4d11b4ed428ede30f64127c6d40137fe6b75d2", + "result_file_sha256": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "raw_content_sha256": "b7f323c1d6229be3555907cf605c3036291995acf9fc575a5ef287331894401b", + "response_id": "20261014_fa1f1d6c-5665-4563-b114-69a463a96eca", + "receipt_status": 200, + "prompt_tokens": 1184, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9765736249973997 + }, + "grouped_lossless": { + "id": "confirm-fresh-11-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-12\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "922741d75fbaa87b925fbd79a8de33100aba3d3ad25033115add3fbf27d10044", + "request_file_sha256": "ff5d41f9de57d2bbf352a7cf9e7fdc214813bd3a9d984e3a867592badbd8344d", + "receipt_file_sha256": "6d3a5a3d143406865c8ef1f9cd747d160a487413ea8ae6e41c5a4c91c2c4be7f", + "receipt_body_text_sha256": "632643503a6460bdd531972812991062ee6f40863e17507abde4c1dd81d6e66a", + "result_file_sha256": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "raw_content_sha256": "b7f323c1d6229be3555907cf605c3036291995acf9fc575a5ef287331894401b", + "response_id": "20261014_bde8ec4a-db61-47ed-bd44-3e83ae091c56", + "receipt_status": 200, + "prompt_tokens": 1863, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.1042334169906098 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-11-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-12\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-12", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "4b3a561e26a0de6d4e728fc49984616f822d7c096e8c94461abfb1669afc1174", + "request_file_sha256": "6d833158573ff57f08afd4c4389bcbb6f4577ceb306cecb6a7db224491c8262b", + "receipt_file_sha256": "0328bd580ff29dd1c36844ba8425716426421e80b4401001f0cce079c05c55b4", + "receipt_body_text_sha256": "84961cca469b01156c4641b52d6b2ea82e044777fb812cfa5a5744ca75880907", + "result_file_sha256": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "raw_content_sha256": "b7f323c1d6229be3555907cf605c3036291995acf9fc575a5ef287331894401b", + "response_id": "20261014_9ad6b529-9102-4c1d-b6b2-59813cc1f236", + "receipt_status": 200, + "prompt_tokens": 1685, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0161284579953644 + } + } + }, + { + "history_id": "confirm-fresh-12-te-m0", + "case_id": "fresh-12-te", + "scenario": "fresh-12", + "model": "sarvam-105b", + "family": "resolved_condition", + "question": "చెప్పిన షరతు ప్రకారం VISIT-2112 కోసం ఏ తేదీ ఎంచుకోవాలి? మార్చవద్దు.", + "source_messages": [ + { + "id": "note-12-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. VISIT-2112 కోసం ఉదయం సమయం ఉంటే 2027-04-13, లేకపోతే 2027-05-13 ఎంచుకోండి. ఉదయం సమయం అందుబాటులో లేదు. నా ఎంపిక గురించి మాత్రమే అడుగుతున్నాను." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source gives two conditional choices for VISIT-2112 and explicitly resolves the condition as unavailable morning. That branch selects 2027-05-13; no booking change was requested.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-12", + "id": "VISIT-2112", + "date": "2027-03-13", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-12", + "id": "VISIT-2112", + "date": "2027-03-13", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "025bbd9f6562dae7a7fce39e7d9ba7a2855b01c44da2c7f10449df043232e560", + "case_sha256": "135a212fc9f481c958d453223b2489519e10839105783109b18789e0f71c05ea", + "gold_label_sha256": "8cec76ef59c8b0b7b419d48f038586b17db220c63f8c96a947759b8b7d78e656", + "reports": { + "chronological": { + "id": "confirm-fresh-12-te-m0-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\": \"not_requested\", \"answer\": \"2027-05-13\", \"needs_clarification\": false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "d09720b5ae8b41f131feb11fdb55be96564d91863ed6bf9a7499f4426f256996", + "request_file_sha256": "902e7e84ceacb0ce386aac90914c4dc52fc72a3687aae931ac7d568005b19f56", + "receipt_file_sha256": "2c3ed4fd876bf0377409c9b7c2d579ba26214fb53dac91b1d5d286af7ea78019", + "receipt_body_text_sha256": "a9a240cd7e1d0331aa3a15c787ddd3ca33cf80276bb3620b82a9cafd10b5ffaf", + "result_file_sha256": "c4ff9927c3a55e64897349c37882c7d2896bdfbe26855686f0af32e779f96ce5", + "raw_content_sha256": "9ef377dc92395cfd885732d67e7d06c1be62e2c646804b04ffde441ccef8c68e", + "response_id": "20261014_00032916-512a-4a2e-b04f-d716ea8f46ea", + "receipt_status": 200, + "prompt_tokens": 881, + "completion_tokens": 2524, + "reasoning_tokens": 2488, + "elapsed_seconds": 13.449140957993222 + }, + "grouped_lossless": { + "id": "confirm-fresh-12-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-13\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "e52976a8885834abccdd66df1a25a60dc540c0955c3e05796aa3008f7d7558bb", + "request_file_sha256": "a0ad4b25883a522921c8e1a35f3841ff11d9e8ad40497ba03452b02c46758d64", + "receipt_file_sha256": "c8f7d96ed6316724f67b10d4ef2913e98f72540d1bb43b6e3e1249b8aa7d2087", + "receipt_body_text_sha256": "bb121fe40a8fefc132f95f1d69188c32a5e3cb1b39e0b422beeed3619c521a3e", + "result_file_sha256": "bddfda0b83421ea19c31efae0763e4d882cd25d8f7fc51ecf50c2203c16f5d03", + "raw_content_sha256": "84abd2c69924b63d951b543aa83115e0e7b4550af4fa0d56780fc8fede890196", + "response_id": "20261014_f78507ff-a12f-40ce-8d5e-b3f871a35529", + "receipt_status": 200, + "prompt_tokens": 1322, + "completion_tokens": 1786, + "reasoning_tokens": 1754, + "elapsed_seconds": 9.740286917003687 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-12-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-05-13\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_performed", + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_performed conflicts with trusted intent and actual receipts, which establish not_requested.", + "request_sha256": "95b4d0b563a12bfc2b2543bd7b87ca7f06bb44e512d9d4587fe68fc30c93f32e", + "request_file_sha256": "ea54e534fec02610d3538daf743dc35606327d7e38efb15fbb9c4fd9bc3c359f", + "receipt_file_sha256": "9466a5136b23f38ad0032f849e92b6bd3c96843991376161290f8f63e51412a1", + "receipt_body_text_sha256": "86a902c236b21f986f6ff02b287b382a96dc5aa6e9015c75f8b23e7be8dc5917", + "result_file_sha256": "f6f15ce23c5eb8843b757dfdab575fa0efba2113144211af9b88ff80d2eff18e", + "raw_content_sha256": "dcb1b840dbdcdf2d81c0a95349cc139654c19df93a9d0d71489f1b1f767761a5", + "response_id": "20261014_0e80529c-7491-4aa6-ac11-305f975933cf", + "receipt_status": 200, + "prompt_tokens": 1258, + "completion_tokens": 1413, + "reasoning_tokens": 1381, + "elapsed_seconds": 6.485699208002188 + } + } + }, + { + "history_id": "confirm-fresh-12-te-m1", + "case_id": "fresh-12-te", + "scenario": "fresh-12", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "question": "చెప్పిన షరతు ప్రకారం VISIT-2112 కోసం ఏ తేదీ ఎంచుకోవాలి? మార్చవద్దు.", + "source_messages": [ + { + "id": "note-12-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. VISIT-2112 కోసం ఉదయం సమయం ఉంటే 2027-04-13, లేకపోతే 2027-05-13 ఎంచుకోండి. ఉదయం సమయం అందుబాటులో లేదు. నా ఎంపిక గురించి మాత్రమే అడుగుతున్నాను." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source gives two conditional choices for VISIT-2112 and explicitly resolves the condition as unavailable morning. That branch selects 2027-05-13; no booking change was requested.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-12", + "id": "VISIT-2112", + "date": "2027-03-13", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-12", + "id": "VISIT-2112", + "date": "2027-03-13", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a10b3e3b6ed37248", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2112" + }, + "response": { + "date": "2027-03-13", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "5b41edf8dd240b58f6d7a3b18e6c8882820d681d903c73cea64890a5715ed536", + "case_sha256": "135a212fc9f481c958d453223b2489519e10839105783109b18789e0f71c05ea", + "gold_label_sha256": "8cec76ef59c8b0b7b419d48f038586b17db220c63f8c96a947759b8b7d78e656", + "reports": { + "chronological": { + "id": "confirm-fresh-12-te-m1-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-13\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "0b08a13e3bb415e3199dbbcb6debffa674ec85b4f99b381ea84557ce54bdde5a", + "request_file_sha256": "863d7e2cc1a6aaf71fb7b0393524aa200176516c018857f2f83482aa87eac8f9", + "receipt_file_sha256": "29a5a99597d978437cb15a7c1ff1dee0bdcf04eb128344cde604c798933bbe17", + "receipt_body_text_sha256": "3eb8a261113d3c0a5c4b1ea57c3f6c42ac67ac93fd8587df894fc70c8ea77f12", + "result_file_sha256": "8d9128c01cc9c00795ff8a0b689d4a4892c41abb9f5a6e7e79bc69704f2194fe", + "raw_content_sha256": "8d02e24f6a939eda3ceb1259da7123c36d6fce2ba7e80e6e78c63544f516f9c9", + "response_id": "20261014_9d887d4b-74a6-4f38-8812-eded3825dd3d", + "receipt_status": 200, + "prompt_tokens": 1129, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9940150839975104 + }, + "grouped_lossless": { + "id": "confirm-fresh-12-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-13\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "1a4ef91ed638e974d287ad0f05bd790ffe642c7102bc1c482f0212cae5ec74b5", + "request_file_sha256": "1b2eb3754074843fd43c39ccf9352612623a2e48b9c85662a3726ea1a6964140", + "receipt_file_sha256": "69c42598d8730d940dfc81535b217de526dc6c364c5cedaa5f99e0aa7ae92aa6", + "receipt_body_text_sha256": "5c4d2cfffe69f146efec37890ab94dce1797ea36f03de8cf0657c5c37b883b98", + "result_file_sha256": "8d9128c01cc9c00795ff8a0b689d4a4892c41abb9f5a6e7e79bc69704f2194fe", + "raw_content_sha256": "8d02e24f6a939eda3ceb1259da7123c36d6fce2ba7e80e6e78c63544f516f9c9", + "response_id": "20261014_20ab1232-80c6-4e15-9ad8-3277289a94bd", + "receipt_status": 200, + "prompt_tokens": 1711, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.997791916001006 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-12-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-13\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-13", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "f95d00a9e84c0f664d90e0594f56aaaae0cef09ad69d1e605725ddd1526f2661", + "request_file_sha256": "58201859e78aae574e32995783ad7e486e5d5b981d399003d0e8df6c2ac433fb", + "receipt_file_sha256": "bcdda8b7692b2178abce0d77ef4ce925893b5a0c51b6a5fa9aefc83810d51eaf", + "receipt_body_text_sha256": "02d2d178b53c4249b3b684b057a3ab5b513f55cd138d61f41c3633fa6bd387cf", + "result_file_sha256": "8d9128c01cc9c00795ff8a0b689d4a4892c41abb9f5a6e7e79bc69704f2194fe", + "raw_content_sha256": "8d02e24f6a939eda3ceb1259da7123c36d6fce2ba7e80e6e78c63544f516f9c9", + "response_id": "20261014_bc862383-012b-424a-b2a1-5ddfd791a52e", + "receipt_status": 200, + "prompt_tokens": 1541, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0473955420020502 + } + } + }, + { + "history_id": "confirm-fresh-13-te-m0", + "case_id": "fresh-13-te", + "scenario": "fresh-13", + "model": "sarvam-105b", + "family": "unresolved_condition", + "question": "చెప్పిన షరతు ప్రకారం VISIT-2113 కోసం ఏ తేదీ ఎంచుకోవాలి? మార్చవద్దు.", + "source_messages": [ + { + "id": "note-13-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-13-1", + "role": "customer", + "content": "VISIT-2113 కోసం ఉదయం సమయం ఉంటే 2027-04-14, లేకపోతే 2027-05-14 ఎంచుకుంటాను." + }, + { + "id": "note-13-2", + "role": "customer", + "content": "అందుబాటులో ఉందో ఇంకా ఎవరూ చూడలేదు. ఊహించవద్దు; బుకింగ్ మార్చవద్దు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source gives two conditional dates for VISIT-2113 but says morning availability has not been checked. Reading the booking state does not resolve that condition; choosing either branch or the current booked date would be unsupported. Null with clarification true is required.", + "expected_answer_from_source_and_frozen_label": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-13", + "id": "VISIT-2113", + "date": "2027-03-14", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-13", + "id": "VISIT-2113", + "date": "2027-03-14", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "926436042ded86c8bd776fc62124703f430dc72de1c3c6e9edf07130c654b31a", + "case_sha256": "c520f051912b23a20da9352a6f73179f6e6cb7685a2864480b5d5ee545cc8f03", + "gold_label_sha256": "45601070452533e97168ec412b1987259e51e861809c966fb2bd3c019a3e1cc6", + "reports": { + "chronological": { + "id": "confirm-fresh-13-te-m0-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "incomplete", + "finish_reason": "length", + "raw_content": null, + "raw_output": null, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure.", + "status_review": "No valid raw action status because the final is incomplete; counted incorrect.", + "request_sha256": "dd6b3d6ff2c9054b78789508acafe3c957ed77fb500a6ea6ec717c8db665bf58", + "request_file_sha256": "9af090a2a2d34e11833813b6ce9dc0238f6bb248dcc4d32cdd3ca31d341efa76", + "receipt_file_sha256": "99980be6e640266a9357bf50a10be12f19679a701abdb1a39e77f09a850ba7f9", + "receipt_body_text_sha256": "abe81c932d09c9b6cbf291e7ca79faf4c4eee90a1b044c00915306c5989a3a57", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "raw_content_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "response_id": "20261014_c35deb19-07d8-4ded-a6f0-01cef5306126", + "receipt_status": 200, + "prompt_tokens": 1019, + "completion_tokens": 8192, + "reasoning_tokens": 8191, + "elapsed_seconds": 36.164887249993626 + }, + "grouped_lossless": { + "id": "confirm-fresh-13-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-03-14\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-03-14 selects a date despite the source leaving the replacement or condition unresolved.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "283ccb7bcf56b8e91698d268e178b16e292935f11620d334bdbfbd2d2bd97f18", + "request_file_sha256": "e7993a137dde91bab22d5e3b4a17849ce75365f73f165ac8c94533757f36ab76", + "receipt_file_sha256": "9fe970b7988c6b47ff50b7044aa4cdf60e123ce4d98764b9ebbc978dfd2e9a53", + "receipt_body_text_sha256": "d0cc0df932cc2ac220e9721b16ae9f9ecb612750fb12c165a41369275f699e01", + "result_file_sha256": "078d3f53025ce08cbdee6c9b66d1debeeb93a4fac3ea7fb3fdad08e9f2d8a614", + "raw_content_sha256": "3be7e35ba83ddd02d3611c9f9823d34fcf7aef5b72761e62c517835aded0425b", + "response_id": "20261014_d55d2ef9-effe-4c57-b0e0-3323abb09e30", + "receipt_status": 200, + "prompt_tokens": 1567, + "completion_tokens": 4483, + "reasoning_tokens": 4451, + "elapsed_seconds": 19.27871454099659 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-13-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\": \"not_requested\", \"answer\": null, \"needs_clarification\": true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "87f261d2d38d7eac0e28721c436ec811bd037d2d339b1bf6f113fc039f3a7722", + "request_file_sha256": "d5d101b8dbfc516d08d2a537cb3b1b039c3c5e7c6c65552b5ed8d4756a01c0c0", + "receipt_file_sha256": "ca059767ab5d2eca737ff6e379c14d6be75310895cf2585783150097f5fd1c84", + "receipt_body_text_sha256": "90b6c801d7a9123e12ab65f0368e691b06b1095bca673315770c3ff185174e07", + "result_file_sha256": "3a023f469ad2b66a2c3cccff96ecad9a50cb6437c0292e365161fb00067f87ce", + "raw_content_sha256": "1f143a6c338737b40763a356b6632537b85ce2afc83088c4f554fee275b1e164", + "response_id": "20261014_1ca3b205-6b91-4f75-9838-60e4ddec3c47", + "receipt_status": 200, + "prompt_tokens": 1411, + "completion_tokens": 2930, + "reasoning_tokens": 2904, + "elapsed_seconds": 13.262295290987822 + } + } + }, + { + "history_id": "confirm-fresh-13-te-m1", + "case_id": "fresh-13-te", + "scenario": "fresh-13", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "question": "చెప్పిన షరతు ప్రకారం VISIT-2113 కోసం ఏ తేదీ ఎంచుకోవాలి? మార్చవద్దు.", + "source_messages": [ + { + "id": "note-13-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-13-1", + "role": "customer", + "content": "VISIT-2113 కోసం ఉదయం సమయం ఉంటే 2027-04-14, లేకపోతే 2027-05-14 ఎంచుకుంటాను." + }, + { + "id": "note-13-2", + "role": "customer", + "content": "అందుబాటులో ఉందో ఇంకా ఎవరూ చూడలేదు. ఊహించవద్దు; బుకింగ్ మార్చవద్దు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source gives two conditional dates for VISIT-2113 but says morning availability has not been checked. Reading the booking state does not resolve that condition; choosing either branch or the current booked date would be unsupported. Null with clarification true is required.", + "expected_answer_from_source_and_frozen_label": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-13", + "id": "VISIT-2113", + "date": "2027-03-14", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-13", + "id": "VISIT-2113", + "date": "2027-03-14", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a6ad157fde10fcae", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2113" + }, + "response": { + "date": "2027-03-14", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "82ca591d24c72634da60a2ce00055c9b59226d6406254284c6ca633d5dba5ee7", + "case_sha256": "c520f051912b23a20da9352a6f73179f6e6cb7685a2864480b5d5ee545cc8f03", + "gold_label_sha256": "45601070452533e97168ec412b1987259e51e861809c966fb2bd3c019a3e1cc6", + "reports": { + "chronological": { + "id": "confirm-fresh-13-te-m1-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-14\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-14", + "needs_clarification": false + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-03-14 selects a date despite the source leaving the replacement or condition unresolved.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "bab0d0a2396cc5ee7390b4f2051f54c2cd329aeae5302e48c7a27dd2e1d81844", + "request_file_sha256": "4ffa360cd6036eead02631d6a791fd2c78f7d0391d208e8351084bf415bfee5f", + "receipt_file_sha256": "309c4279e3bd7c3e13718aa2f712305183987e21e6d2cfe103df7e696af28a6a", + "receipt_body_text_sha256": "6295944d52629dd0ef38d0f50d50752c8f06541e3f05edecd903faeb5535aab4", + "result_file_sha256": "b6474bc8194ce66a7fd312e11be1d6465b06f536176a1bb68eebcb0a95a69d0d", + "raw_content_sha256": "7827a8c7b971979647ee14efc59af7031db8e64639d17be755062049fa079f44", + "response_id": "20261014_6a6b375d-28be-481b-8a59-c98439e64454", + "receipt_status": 200, + "prompt_tokens": 1175, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9357774579984834 + }, + "grouped_lossless": { + "id": "confirm-fresh-13-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "652b7ff2560cc812725b27d83bdd565a865ae3a5d514372524033d58869a97cb", + "request_file_sha256": "2ece3179ae9869c07efd3faf65ef3f748f289b0d3d7d4211a098d0310e8a2557", + "receipt_file_sha256": "004d3865b47750a638d9aaf65c84fe914c75c44db30e4d308b7bc57ae436270d", + "receipt_body_text_sha256": "222aa4d21b7093abe17f50e994ba9df27f0994ae19cfe9d1a8a20b29220e394e", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_d1df1b0e-c379-42b8-b73a-c85d041f6bb2", + "receipt_status": 200, + "prompt_tokens": 1872, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 1.1219505830085836 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-13-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "d316547a0447a43f74a675b2f05bda0f7ae8cc6f4edf10fa7781718d74c3adf1", + "request_file_sha256": "21a44ef36a317c709ef54786447c4f97d49c07e2601a03d232090e536cafe76a", + "receipt_file_sha256": "900c0a9944791107982356042bdc198041fbcf101e572f21cf17fcfbd6127691", + "receipt_body_text_sha256": "355fb2affeb931d33b954e424d73a30289f918a81cf1d574ff864dfda42136be", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_32bd8f62-2354-498a-8cf7-663af1daf4e0", + "receipt_status": 200, + "prompt_tokens": 1698, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 1.0096665830060374 + } + } + }, + { + "history_id": "confirm-fresh-14-te-m0", + "case_id": "fresh-14-te", + "scenario": "fresh-14", + "model": "sarvam-105b", + "family": "unresolved_condition", + "question": "చెప్పిన షరతు ప్రకారం VISIT-2114 కోసం ఏ తేదీ ఎంచుకోవాలి? మార్చవద్దు.", + "source_messages": [ + { + "id": "note-14-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. VISIT-2114 కోసం ఉదయం సమయం ఉంటే 2027-04-15, లేకపోతే 2027-05-15 ఎంచుకుంటాను. అందుబాటులో ఉందో ఇంకా ఎవరూ చూడలేదు. ఊహించవద్దు; బుకింగ్ మార్చవద్దు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source gives two conditional dates for VISIT-2114 but says morning availability has not been checked. Reading the booking state does not resolve that condition; choosing either branch or the current booked date would be unsupported. Null with clarification true is required.", + "expected_answer_from_source_and_frozen_label": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-14", + "id": "VISIT-2114", + "date": "2027-03-15", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-14", + "id": "VISIT-2114", + "date": "2027-03-15", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "d1f1c448052b17a546678d98e958ac035dd53b5b6d51137d634cdb7a3422c462", + "case_sha256": "a1e32026166acdc8d217a57591e590c40a442c7a5e4ebc60cdd72d4657689c47", + "gold_label_sha256": "d5353591537317b0965cba7bd55c66aad15e85b8067dfbff10e75c5ec5456d7e", + "reports": { + "chronological": { + "id": "confirm-fresh-14-te-m0-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "f6eccb417484f2e19fe870c533fb0c41858c3cb02c7d3d6f4d905045a51e2174", + "request_file_sha256": "21fae0b9038bf61bfe59b282469454cf4f57454c5267a771cc7bc541e686b702", + "receipt_file_sha256": "b6bead11ebf3cdeeb2edfef06f6bf6dae963b56c680a024ce812e55683586873", + "receipt_body_text_sha256": "9b787d2b73aae9b4aaad171cb9da1ebfec3843a59e820af41be5faf9ffda027e", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "response_id": "20261014_f6dbf8fe-0a7b-42f4-9913-7d725b0cefba", + "receipt_status": 200, + "prompt_tokens": 1101, + "completion_tokens": 4171, + "reasoning_tokens": 4148, + "elapsed_seconds": 17.552688708005007 + }, + "grouped_lossless": { + "id": "confirm-fresh-14-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "c011c5701b184d6c4fd940d429299be8aa0fc3fe4c1cc9e0106cbc3a41757f01", + "request_file_sha256": "7f1c44fe99dc4aa807abb59004edef9e33c11c482c05d00975bccdbc83000c3f", + "receipt_file_sha256": "ccb873d1491fa455b34762bd0ab982660a92c026b600cdada808d0da7f0cffd7", + "receipt_body_text_sha256": "0c3273e9ff5f618c9d983e0679a7d86434020aa5d2216d8f737f9b0dc14fc54e", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "response_id": "20261014_eedeec5b-6e39-4f5b-a792-355d3ff3f595", + "receipt_status": 200, + "prompt_tokens": 1540, + "completion_tokens": 6796, + "reasoning_tokens": 6773, + "elapsed_seconds": 28.325404834002256 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-14-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "d0502dca33c9d296b04fd84419c871e8ca0f37a30ecfbfad85f52b6eb25c1229", + "request_file_sha256": "52e1072d0df123bcf23013b764f20a372ee3277c3af2504cb8ce5dbac1047e27", + "receipt_file_sha256": "51426b515cf7faedf6f6cd515f2548f130a19f9051a9972e075ecbad43385eb2", + "receipt_body_text_sha256": "4c81e80e02b38b26f1f055e2c6ae973bd46d746e67f46891612e1ba3359a3212", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "response_id": "20261014_141244cd-6a5a-4889-ac47-f476bd9be0bd", + "receipt_status": 200, + "prompt_tokens": 1264, + "completion_tokens": 1975, + "reasoning_tokens": 1952, + "elapsed_seconds": 8.641674832993886 + } + } + }, + { + "history_id": "confirm-fresh-14-te-m1", + "case_id": "fresh-14-te", + "scenario": "fresh-14", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "question": "చెప్పిన షరతు ప్రకారం VISIT-2114 కోసం ఏ తేదీ ఎంచుకోవాలి? మార్చవద్దు.", + "source_messages": [ + { + "id": "note-14-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. VISIT-2114 కోసం ఉదయం సమయం ఉంటే 2027-04-15, లేకపోతే 2027-05-15 ఎంచుకుంటాను. అందుబాటులో ఉందో ఇంకా ఎవరూ చూడలేదు. ఊహించవద్దు; బుకింగ్ మార్చవద్దు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source gives two conditional dates for VISIT-2114 but says morning availability has not been checked. Reading the booking state does not resolve that condition; choosing either branch or the current booked date would be unsupported. Null with clarification true is required.", + "expected_answer_from_source_and_frozen_label": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-14", + "id": "VISIT-2114", + "date": "2027-03-15", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-14", + "id": "VISIT-2114", + "date": "2027-03-15", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a26852df9b6b6222", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2114" + }, + "response": { + "date": "2027-03-15", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "e0a5366d0fa966cc56834480f4db507d600ab467703a56e388bf5f9a30f8b0df", + "case_sha256": "a1e32026166acdc8d217a57591e590c40a442c7a5e4ebc60cdd72d4657689c47", + "gold_label_sha256": "d5353591537317b0965cba7bd55c66aad15e85b8067dfbff10e75c5ec5456d7e", + "reports": { + "chronological": { + "id": "confirm-fresh-14-te-m1-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-15\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-15", + "needs_clarification": false + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-04-15 selects a date despite the source leaving the replacement or condition unresolved.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "2f260ba83a14e3ba320d143873aaf6cba859987bd87a9ec3dfea3efe574892bb", + "request_file_sha256": "7a96a5e0219afba3bb8c7da2ad2943d505f6b591437be5756d7d7539b549119f", + "receipt_file_sha256": "a3b789674a59ac71358ce1f1f0fbc0077a207aae814ae38844372745b5457790", + "receipt_body_text_sha256": "14a3f31899c391ad0d23ad54f0cf87478a5f60c4035dfe94b6b470bf3274a5de", + "result_file_sha256": "318ed2b4a8a6682566450e991f6b72e55084ede01f0890501cedaa5bf0774042", + "raw_content_sha256": "f1cc3cfd197b822c6523ca628819fd8bab8da2efa0b004526ca96efcbb3be384", + "response_id": "20261014_66e47e52-36ad-4b18-8949-d6cd79e0258d", + "receipt_status": 200, + "prompt_tokens": 1126, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9414785420085536 + }, + "grouped_lossless": { + "id": "confirm-fresh-14-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-15\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-15", + "needs_clarification": false + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-04-15 selects a date despite the source leaving the replacement or condition unresolved.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "9e9461c0c4590125efc6d1fe56a11baa81b0984e86f8824f0c5ab045c386c2a1", + "request_file_sha256": "bb3792b808b7bd311cb4e651dbc2fa1d3eb176abfcc39a3077cb338796d8f265", + "receipt_file_sha256": "c179985e9bdeb9ff79450bc1f978c475413711e2b5d9dd99d15b6dcc45259734", + "receipt_body_text_sha256": "859c7a648d314ff78c2d42bec951cecb7a946630808e25cd16e37c23e20eb705", + "result_file_sha256": "318ed2b4a8a6682566450e991f6b72e55084ede01f0890501cedaa5bf0774042", + "raw_content_sha256": "f1cc3cfd197b822c6523ca628819fd8bab8da2efa0b004526ca96efcbb3be384", + "response_id": "20261014_ce7a8bb0-bee3-4596-b399-9a871f31e467", + "receipt_status": 200, + "prompt_tokens": 1709, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.967218708989094 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-14-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "11af253078f0bb72ebebc804cea92049855a92e157680481393b4c1ab668dc47", + "request_file_sha256": "fcf8f791b3b754a778557f7e4455736a44b9a82c2ceb2e32ef523fe2b6656d32", + "receipt_file_sha256": "0a8d8a462fe56bd62ce64c75215870246bbfcadcce96e860133f2cebbb57c588", + "receipt_body_text_sha256": "6140b027b47c5f8f678197d71989dc0992d3289ad51c987c8ec09b1c280cf4c4", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_7299aa8d-507e-400b-b719-7b428310a204", + "receipt_status": 200, + "prompt_tokens": 1550, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 0.971434709004825 + } + } + }, + { + "history_id": "confirm-fresh-15-te-m0", + "case_id": "fresh-15-te", + "scenario": "fresh-15", + "model": "sarvam-105b", + "family": "unresolved_condition", + "question": "చెప్పిన షరతు ప్రకారం VISIT-2115 కోసం ఏ తేదీ ఎంచుకోవాలి? మార్చవద్దు.", + "source_messages": [ + { + "id": "note-15-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-15-1", + "role": "customer", + "content": "VISIT-2115 కోసం ఉదయం సమయం ఉంటే 2027-04-16, లేకపోతే 2027-05-16 ఎంచుకుంటాను." + }, + { + "id": "note-15-2", + "role": "customer", + "content": "అందుబాటులో ఉందో ఇంకా ఎవరూ చూడలేదు. ఊహించవద్దు; బుకింగ్ మార్చవద్దు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source gives two conditional dates for VISIT-2115 but says morning availability has not been checked. Reading the booking state does not resolve that condition; choosing either branch or the current booked date would be unsupported. Null with clarification true is required.", + "expected_answer_from_source_and_frozen_label": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-15", + "id": "VISIT-2115", + "date": "2027-03-16", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-15", + "id": "VISIT-2115", + "date": "2027-03-16", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "ad86ae10ecbc3ac8da03fe2ac916951b7a645d7d98404b148660cd856694607b", + "case_sha256": "db2f0b5672078f4b8a3f78947543371076e5e8ef9ff248d3d5297cd9c6e3999c", + "gold_label_sha256": "2d1f885877b91c7e176c042e7f608077fe19fdfb03d6c3b6caa9f089e166e1ea", + "reports": { + "chronological": { + "id": "confirm-fresh-15-te-m0-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "f7949cddcbc5f1750c46b7634f4a2d0f689ae852443cb5cc19a189a3762c71d3", + "request_file_sha256": "54b91220c6c36f72b3f2043ac209a09c9d679294f2b518e422c29529177f2d38", + "receipt_file_sha256": "211f559e2311f1f68f4b4189dad7bba4a842ac407b1b2009656b20f2abad338d", + "receipt_body_text_sha256": "f5397b26103e771874ad7787b855938c5f85357b82142d2c64239c37b354582d", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "response_id": "20261014_076e90fe-cc06-4baf-a844-32c0dc2ae12f", + "receipt_status": 200, + "prompt_tokens": 990, + "completion_tokens": 6055, + "reasoning_tokens": 6032, + "elapsed_seconds": 24.841930750000756 + }, + "grouped_lossless": { + "id": "confirm-fresh-15-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\": \"not_requested\", \"answer\": null, \"needs_clarification\": true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "f2806c66a39fd11405fb137051070e03bbbe3f16deb57dc8925a188e028a207e", + "request_file_sha256": "c606d23415dcc42eea7aa5a82d40661d94955af3b56198b3b8a7734eee33a8cf", + "receipt_file_sha256": "b7300990a5ba7e7696932c794dc89711de062d2f958fab3da5cb1e7397413107", + "receipt_body_text_sha256": "aec8507b2d81ace701c9b06ba14825acd04bc567d703c7a1d16f8451d4e6d815", + "result_file_sha256": "3a023f469ad2b66a2c3cccff96ecad9a50cb6437c0292e365161fb00067f87ce", + "raw_content_sha256": "1f143a6c338737b40763a356b6632537b85ce2afc83088c4f554fee275b1e164", + "response_id": "20261014_d367777f-170a-4ceb-ad33-1746a0d3f941", + "receipt_status": 200, + "prompt_tokens": 1533, + "completion_tokens": 1467, + "reasoning_tokens": 1440, + "elapsed_seconds": 8.45636095800728 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-15-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "714e94416d15ae16dd10b186b478da29fee482e4bd0e29f414d040dea458c558", + "request_file_sha256": "828a4c31cce4a502f2ad3efb42f1d66aec6b31d3163c899f8c054d6b08292424", + "receipt_file_sha256": "da1477cb27dda90c34e9302afae8696a7a44fb7a30d74c9df47a8252451e6ed6", + "receipt_body_text_sha256": "a75e16fa4d91673359ecec92d627f7f775b28996023de567b1905f20361ce499", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "response_id": "20261014_b63e7169-2986-4ae0-9eea-d2c20b1748b6", + "receipt_status": 200, + "prompt_tokens": 1408, + "completion_tokens": 2727, + "reasoning_tokens": 2704, + "elapsed_seconds": 11.572422540994012 + } + } + }, + { + "history_id": "confirm-fresh-15-te-m1", + "case_id": "fresh-15-te", + "scenario": "fresh-15", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "question": "చెప్పిన షరతు ప్రకారం VISIT-2115 కోసం ఏ తేదీ ఎంచుకోవాలి? మార్చవద్దు.", + "source_messages": [ + { + "id": "note-15-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-15-1", + "role": "customer", + "content": "VISIT-2115 కోసం ఉదయం సమయం ఉంటే 2027-04-16, లేకపోతే 2027-05-16 ఎంచుకుంటాను." + }, + { + "id": "note-15-2", + "role": "customer", + "content": "అందుబాటులో ఉందో ఇంకా ఎవరూ చూడలేదు. ఊహించవద్దు; బుకింగ్ మార్చవద్దు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source gives two conditional dates for VISIT-2115 but says morning availability has not been checked. Reading the booking state does not resolve that condition; choosing either branch or the current booked date would be unsupported. Null with clarification true is required.", + "expected_answer_from_source_and_frozen_label": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-15", + "id": "VISIT-2115", + "date": "2027-03-16", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-15", + "id": "VISIT-2115", + "date": "2027-03-16", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-b292567c59b6becc", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2115" + }, + "response": { + "date": "2027-03-16", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "c3ca09cd6289b47a6e3a6ba2ac60defb911b4072f3b15aaf3081e2c8e66f747d", + "case_sha256": "db2f0b5672078f4b8a3f78947543371076e5e8ef9ff248d3d5297cd9c6e3999c", + "gold_label_sha256": "2d1f885877b91c7e176c042e7f608077fe19fdfb03d6c3b6caa9f089e166e1ea", + "reports": { + "chronological": { + "id": "confirm-fresh-15-te-m1-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-16\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-16", + "needs_clarification": false + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-03-16 selects a date despite the source leaving the replacement or condition unresolved.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "5b8de216ff2a994167a02aaa31d89b6bcd626826655c749017a64966702a7399", + "request_file_sha256": "df71f6b8de3106c09f00c8424dfe694c2396b1f8b1b23ab0a74491faec96174f", + "receipt_file_sha256": "0c3fbd054eaca5479df8adc03547117fbf73cdc2b062f5c22acd9b66aa324eab", + "receipt_body_text_sha256": "85e042984e5e41bdbabaa794c0f6024985684e558e366576d3fd572a5de3fb1b", + "result_file_sha256": "1221a7e341e0ab559d66b45f568b82184bacfdecaa04abab155224785776dcee", + "raw_content_sha256": "de515f172b9e6bd988e93a4bef206e9ce04f5942d82ca6219a2c20367efd4537", + "response_id": "20261014_415d6bbc-ce62-49a2-aa53-e8f2870ffc44", + "receipt_status": 200, + "prompt_tokens": 1174, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.131959584003198 + }, + "grouped_lossless": { + "id": "confirm-fresh-15-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-16\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-16", + "needs_clarification": false + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-04-16 selects a date despite the source leaving the replacement or condition unresolved.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "34dde79f5bd1bb3fec10c10206d8e998abe659c7f866ea69cb47f9439b47ac48", + "request_file_sha256": "71606109562ad088e89885c604e9a61a5bb9529fef76fdbf2766c35bfd8b8579", + "receipt_file_sha256": "8a019af19461deb734c39b395f7e4603e6c1e08e7b16f31787b9e0ff98faa92c", + "receipt_body_text_sha256": "8d8c37a9ad9dabf8bebe2a362e85f8116d19ebaa91d6e1a1758f133d65f15528", + "result_file_sha256": "afd6ab4071b0df347bb5f8ee7e7e3b493e9f731b4ac18a0adbfee443737adcad", + "raw_content_sha256": "a3c47af1bf9caf574a51522c21d550db0a1e82e5621e992132138af3074ac320", + "response_id": "20261014_9d53f10f-bd1f-48a4-a12f-61174f88b8b8", + "receipt_status": 200, + "prompt_tokens": 1858, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9855014170025242 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-15-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "01006261331afedcc3802c1d716cd9cc38d2ae40b175263d9f9c1686092c2205", + "request_file_sha256": "4537cb9b7953f443fc7a1b8e4f7975f9ed74404c8921281e123199aad24de418", + "receipt_file_sha256": "a94b195d5ef54b2122788ceaea00f81076c7848a94c8d1547b4b352af17c6cab", + "receipt_body_text_sha256": "436c9720d697427ffc95ab6e530bf8edfec808e341f79b799bee32f2dd896f9a", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_e61d8b0a-f059-42e6-9447-dc7c1057b27c", + "receipt_status": 200, + "prompt_tokens": 1689, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 1.2939245830057189 + } + } + }, + { + "history_id": "confirm-fresh-16-te-m0", + "case_id": "fresh-16-te", + "scenario": "fresh-16", + "model": "sarvam-105b", + "family": "unresolved_condition", + "question": "చెప్పిన షరతు ప్రకారం VISIT-2116 కోసం ఏ తేదీ ఎంచుకోవాలి? మార్చవద్దు.", + "source_messages": [ + { + "id": "note-16-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. VISIT-2116 కోసం ఉదయం సమయం ఉంటే 2027-04-17, లేకపోతే 2027-05-17 ఎంచుకుంటాను. అందుబాటులో ఉందో ఇంకా ఎవరూ చూడలేదు. ఊహించవద్దు; బుకింగ్ మార్చవద్దు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source gives two conditional dates for VISIT-2116 but says morning availability has not been checked. Reading the booking state does not resolve that condition; choosing either branch or the current booked date would be unsupported. Null with clarification true is required.", + "expected_answer_from_source_and_frozen_label": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-16", + "id": "VISIT-2116", + "date": "2027-03-17", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-16", + "id": "VISIT-2116", + "date": "2027-03-17", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "3d9c1284faf81c4beffcff2cba76ca4bb5eddebc198a52559b4354ca70ba5ca0", + "case_sha256": "a70a32c1db90f2afb5b254d96f11529dbd06d28c0c4263453737aa03a9ffbc07", + "gold_label_sha256": "a3baf3c7678d34ef4632579e25974fff4bc0b7a587033e372036f57fd6742234", + "reports": { + "chronological": { + "id": "confirm-fresh-16-te-m0-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "incomplete", + "finish_reason": "length", + "raw_content": null, + "raw_output": null, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure.", + "status_review": "No valid raw action status because the final is incomplete; counted incorrect.", + "request_sha256": "05038ad17e9e91b0d769d361b1ca596332eb97713ddde2a8d91412c5250241df", + "request_file_sha256": "1453dae6065163817133a42769a1c6e4c0b7bce0a059e9606102f3296079ad82", + "receipt_file_sha256": "47200bff5dfe4888dc3bf93eb3068228515de9f26d49f4c5f14c9996dff9843c", + "receipt_body_text_sha256": "aa95a4adcbcc77d3b0f3c6dede44d4fd4f4ebe1b7664e66abd5c43c50c0d082f", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "raw_content_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "response_id": "20261014_01b6b907-9906-40a5-8420-59ce63702402", + "receipt_status": 200, + "prompt_tokens": 885, + "completion_tokens": 8192, + "reasoning_tokens": 8191, + "elapsed_seconds": 33.825618667004164 + }, + "grouped_lossless": { + "id": "confirm-fresh-16-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "2bc2028dc2961e6e7baefb12e822322f2f45ce749a766a46b5e3b5ac4d13b2ee", + "request_file_sha256": "77107af871e058d0bba09e5c7b71822b69fc938ec6a7db837e41057f28f0c3af", + "receipt_file_sha256": "8ded85bfd257e74941873a834ef982c92dbe6cc6b3afae6b50083fc257318178", + "receipt_body_text_sha256": "fbebecf76b45cee3df07c6b776117e8d7dd2056684ed8328df8a89c2917b3ae2", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "response_id": "20261014_99632d1c-4b3f-4b9b-8f65-9b1dbcce0e33", + "receipt_status": 200, + "prompt_tokens": 1325, + "completion_tokens": 3405, + "reasoning_tokens": 3380, + "elapsed_seconds": 14.300099958010833 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-16-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "fd768991077686976620ec9f17b37d0fe0a179c22c87ba8e2d177f1781cae0c3", + "request_file_sha256": "efbd732215484538fe51a0db7eead2d845db354ce2e075885252b269968879b1", + "receipt_file_sha256": "6b8788cbb4137c8e127e689e79bbeb23c74fd994b33d83d77d675ba1c3aad913", + "receipt_body_text_sha256": "a3eed407466dda81a959d71f5ef61fc8b4c9b4b55bf2e3f11798de278c494a0a", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "response_id": "20261014_59c7f4a1-06a3-4414-9858-2c96137365ea", + "receipt_status": 200, + "prompt_tokens": 1265, + "completion_tokens": 2069, + "reasoning_tokens": 2046, + "elapsed_seconds": 9.044318750005914 + } + } + }, + { + "history_id": "confirm-fresh-16-te-m1", + "case_id": "fresh-16-te", + "scenario": "fresh-16", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "question": "చెప్పిన షరతు ప్రకారం VISIT-2116 కోసం ఏ తేదీ ఎంచుకోవాలి? మార్చవద్దు.", + "source_messages": [ + { + "id": "note-16-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. VISIT-2116 కోసం ఉదయం సమయం ఉంటే 2027-04-17, లేకపోతే 2027-05-17 ఎంచుకుంటాను. అందుబాటులో ఉందో ఇంకా ఎవరూ చూడలేదు. ఊహించవద్దు; బుకింగ్ మార్చవద్దు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source gives two conditional dates for VISIT-2116 but says morning availability has not been checked. Reading the booking state does not resolve that condition; choosing either branch or the current booked date would be unsupported. Null with clarification true is required.", + "expected_answer_from_source_and_frozen_label": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-16", + "id": "VISIT-2116", + "date": "2027-03-17", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-16", + "id": "VISIT-2116", + "date": "2027-03-17", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8ee870b6dba4b670", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2116" + }, + "response": { + "date": "2027-03-17", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "506a004ac7994a6e88589075b5c0061eecd0a514748f88af7c2a79d8f1a1e153", + "case_sha256": "a70a32c1db90f2afb5b254d96f11529dbd06d28c0c4263453737aa03a9ffbc07", + "gold_label_sha256": "a3baf3c7678d34ef4632579e25974fff4bc0b7a587033e372036f57fd6742234", + "reports": { + "chronological": { + "id": "confirm-fresh-16-te-m1-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-17\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-04-17 selects a date despite the source leaving the replacement or condition unresolved.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "b5e3227cd835f8a57a32774c32231b1df0a81c6a69578811e706b3dcc3060041", + "request_file_sha256": "58cd20337df8f6bbd563a5bf553a7d095947fe8a082834a05e59b2893d5afa22", + "receipt_file_sha256": "a34864b3147bc5433e528310e51abf8831e5f27a3afb50f4af54513d7aaee271", + "receipt_body_text_sha256": "1541c15ebfd80b504070f56d5afd1bcfefcdc9dd271293b414bc025a016b42be", + "result_file_sha256": "081f2013d93cd7b24a8f7a3eb964081ac78a1dfd0105add89dcdbd0bc3ba2e8d", + "raw_content_sha256": "75c6adfd0e8d53d51595c425215a8940b057c7b3c3e646327d765ea95b8653c8", + "response_id": "20261014_a2953452-31a1-414c-9133-e7e46310bd68", + "receipt_status": 200, + "prompt_tokens": 1207, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9535644579882501 + }, + "grouped_lossless": { + "id": "confirm-fresh-16-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-17\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-17", + "needs_clarification": false + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-04-17 selects a date despite the source leaving the replacement or condition unresolved.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "4aa960de58553a41789f862e54b6eee144d0ddf4b4fa3af314da2a8b4f61ac11", + "request_file_sha256": "6c5b1cbbe1db3f6679a8290b6d6e35fbd9341d0918aef8707d232b62d8f22d80", + "receipt_file_sha256": "22d400b4a97f220318b5ebaa8393d1d5a15d659ef1bba2c44a598cf23d56665d", + "receipt_body_text_sha256": "fbf898fb9f8f3bed98aea5ba38632f069850fb412a75f45eef1c0d2a805395ff", + "result_file_sha256": "081f2013d93cd7b24a8f7a3eb964081ac78a1dfd0105add89dcdbd0bc3ba2e8d", + "raw_content_sha256": "75c6adfd0e8d53d51595c425215a8940b057c7b3c3e646327d765ea95b8653c8", + "response_id": "20261014_41147533-6480-4800-be03-7d5acf3b132c", + "receipt_status": 200, + "prompt_tokens": 1793, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.1345692500035511 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-16-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": null,\n \"needs_clarification\": true\n}", + "raw_output": { + "action_status": "not_requested", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": null, + "needs_clarification": true + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "4fb70339072a226d4ed7d461f0fa0323e1de4c3a1e6b13f07acc10623ea806df", + "request_file_sha256": "b75c685e07b1b0c87d03ad031a9e191f7cf1902a4f6356ad16eeddaf0a03b6c1", + "receipt_file_sha256": "e0b63615481d0008e4b34934d975b72f6146428ca602436f71f931f03a560f6e", + "receipt_body_text_sha256": "9df5c499e291ff67481cf2cc54b0a55bc59b3eb3e8b47443e75335f10489a4e1", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "response_id": "20261014_fca7078a-9f21-4e54-b6b2-9b0ddd695330", + "receipt_status": 200, + "prompt_tokens": 1549, + "completion_tokens": 32, + "reasoning_tokens": null, + "elapsed_seconds": 1.0166007499938132 + } + } + }, + { + "history_id": "confirm-fresh-17-te-m0", + "case_id": "fresh-17-te", + "scenario": "fresh-17", + "model": "sarvam-105b", + "family": "separate_booking", + "question": "VISIT-2117 కోసం ప్రస్తుతం నాకు ఇష్టమైన తేదీ ఏది? చెప్పండి మాత్రమే; బుకింగ్ మార్చవద్దు.", + "source_messages": [ + { + "id": "note-17-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-17-1", + "role": "customer", + "content": "VISIT-2117 కోసం నా ఎంపిక 2027-04-18. VISIT-3117 కోసం మొదట 2027-03-18 అనుకున్నాను." + }, + { + "id": "note-17-2", + "role": "customer", + "content": "సవరణ VISIT-3117కి మాత్రమే: ఇప్పుడు 2027-05-18 కావాలి. మొదటి బుకింగ్ ఎంపిక మారలేదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source retains 2027-04-18 as the target VISIT-2117 preferred date; its later correction is expressly limited to the other VISIT-31xx booking. The current backend date and other booking date do not answer this preference question.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-17", + "id": "VISIT-2117", + "date": "2027-03-18", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-17", + "id": "VISIT-2117", + "date": "2027-03-18", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "c8a153908d4acf0f3d87c3a8468ee6416b56c8acda42405ed5170dc05fc6312e", + "case_sha256": "cc8ef5cdfcf546a229ba48ebc0acffda66d1dc1720f24359a07a5a21eb5725db", + "gold_label_sha256": "9066a306c325d7ef889d87fa6b2711f118360173cfbe75740979246fb4d6035a", + "reports": { + "chronological": { + "id": "confirm-fresh-17-te-m0-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-18\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "5aa1e49e85bbd85d64748c2067f335168fa13e2a7c38764f485807dad3185e72", + "request_file_sha256": "28ceff71292e4bf671677bb0e30e51db5299762cb2fe51679873e62125aa6b7e", + "receipt_file_sha256": "75873761758cb09e6b5cd8189832c16e491430a135aefa39e4ba38c567e4250b", + "receipt_body_text_sha256": "d4e96f8002525b9a0d32aee619aaa2fc73c88dff4817e33d500dabc97fe7326d", + "result_file_sha256": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "raw_content_sha256": "9449807d1a496bfec71b5dbf1c86405ed0ae5d9745146d9437a26e0842b5006f", + "response_id": "20261014_c2011945-57a8-4c89-bea5-dc0ba1567267", + "receipt_status": 200, + "prompt_tokens": 899, + "completion_tokens": 5155, + "reasoning_tokens": 5123, + "elapsed_seconds": 23.44692899999791 + }, + "grouped_lossless": { + "id": "confirm-fresh-17-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-18\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "c19a2c61a8cbae716547532c3bf089108bfe41b76329c62790741a45c24b2de4", + "request_file_sha256": "afd8a3c0b1f34a5b804cef1a1c4ebbf42d3ac04947efd6b73d867acec61caf80", + "receipt_file_sha256": "c94513ce65bc575f9e2b9fbefbf5741b739ee2c2f5f4ea365a8b71fc8e9d1a10", + "receipt_body_text_sha256": "a97eac0715ce5f170e12cc32f23144174f2e6c5d7465d426474dfbe315a66ff7", + "result_file_sha256": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "raw_content_sha256": "9449807d1a496bfec71b5dbf1c86405ed0ae5d9745146d9437a26e0842b5006f", + "response_id": "20261014_8a80aa22-e296-4613-a897-65302d2c472f", + "receipt_status": 200, + "prompt_tokens": 1451, + "completion_tokens": 3248, + "reasoning_tokens": 3216, + "elapsed_seconds": 17.090247959000408 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-17-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-18\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "bb1efa940c4944b7c9925b41f307391d650d1b71b63ccb18be1eaf68c5a2f9da", + "request_file_sha256": "052d715510b79601f03a817c654dec458ab873295cc928de97a79df5a08429b7", + "receipt_file_sha256": "f5b26e6b465cbd28fbaef3c225bf153c852eaba17fb8e08d634a65e1f7a7566f", + "receipt_body_text_sha256": "6b2c2db05b1d29c4b2f77f347b66bc18653bbcf8d7213aa28a859af635d91602", + "result_file_sha256": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "raw_content_sha256": "9449807d1a496bfec71b5dbf1c86405ed0ae5d9745146d9437a26e0842b5006f", + "response_id": "20261014_43c46536-b410-44b0-be25-063e2dd9bc59", + "receipt_status": 200, + "prompt_tokens": 1436, + "completion_tokens": 2483, + "reasoning_tokens": 2451, + "elapsed_seconds": 13.231449500002782 + } + } + }, + { + "history_id": "confirm-fresh-17-te-m1", + "case_id": "fresh-17-te", + "scenario": "fresh-17", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "question": "VISIT-2117 కోసం ప్రస్తుతం నాకు ఇష్టమైన తేదీ ఏది? చెప్పండి మాత్రమే; బుకింగ్ మార్చవద్దు.", + "source_messages": [ + { + "id": "note-17-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-17-1", + "role": "customer", + "content": "VISIT-2117 కోసం నా ఎంపిక 2027-04-18. VISIT-3117 కోసం మొదట 2027-03-18 అనుకున్నాను." + }, + { + "id": "note-17-2", + "role": "customer", + "content": "సవరణ VISIT-3117కి మాత్రమే: ఇప్పుడు 2027-05-18 కావాలి. మొదటి బుకింగ్ ఎంపిక మారలేదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source retains 2027-04-18 as the target VISIT-2117 preferred date; its later correction is expressly limited to the other VISIT-31xx booking. The current backend date and other booking date do not answer this preference question.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-17", + "id": "VISIT-2117", + "date": "2027-03-18", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-17", + "id": "VISIT-2117", + "date": "2027-03-18", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "82f94708c28750961ecb615fc16daadfeb277e24ac983bcc0cb2b8e449c7983d", + "case_sha256": "cc8ef5cdfcf546a229ba48ebc0acffda66d1dc1720f24359a07a5a21eb5725db", + "gold_label_sha256": "9066a306c325d7ef889d87fa6b2711f118360173cfbe75740979246fb4d6035a", + "reports": { + "chronological": { + "id": "confirm-fresh-17-te-m1-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-18\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": true, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-03-18 differs from the source/actual-current target 2027-04-18; the raw date is unsupported for this question.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "0971e7f236e93eb5c57937ac7a1b5927971b9d979426f683055652c55c7dc1a7", + "request_file_sha256": "eef7100e0e24727c12f56157662474693c1823030570eeda3dfbd73803a546e6", + "receipt_file_sha256": "5f7a1a8f3dc7359f25d8d47eafe78baacd59c263b111a499d26b1d1088226a64", + "receipt_body_text_sha256": "a1875f25470c15bc3fea19e72d0192d21d2f3c34485ae52ceb3ae9daf366ea78", + "result_file_sha256": "87c81f64b32de0e465ec65f5672c4e2b281ceaddfdb2a11351a1e137c744f10c", + "raw_content_sha256": "4070209a7773e6e396574c3cc304e3528574fb48855f2e8178fb6206d1c899c5", + "response_id": "20261014_d4840d12-c01a-4e96-9254-55b13c5bf9c3", + "receipt_status": 200, + "prompt_tokens": 941, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9648898750019725 + }, + "grouped_lossless": { + "id": "confirm-fresh-17-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-18\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-18", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": true, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-03-18 differs from the source/actual-current target 2027-04-18; the raw date is unsupported for this question.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "0619c966bb925b5a39d4a3d2a901bb288cc6fed4a547eb863c83e579f22204a8", + "request_file_sha256": "8c79b0e95fdb45dff5a75cbd2f4c406561f4adefefe8e1c531552e1cf624790e", + "receipt_file_sha256": "2b48899079a40b09213a0e8331098cbfbaef07f7f896ed96d11ba8a3a7b05312", + "receipt_body_text_sha256": "3c6f474f5904cc427823ea57e435dc159885ab6539bed1e36635b5a1a07336fb", + "result_file_sha256": "87c81f64b32de0e465ec65f5672c4e2b281ceaddfdb2a11351a1e137c744f10c", + "raw_content_sha256": "4070209a7773e6e396574c3cc304e3528574fb48855f2e8178fb6206d1c899c5", + "response_id": "20261014_bf33a37c-afdb-4665-bdf0-c30bef70872f", + "receipt_status": 200, + "prompt_tokens": 1496, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.1015573330078041 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-17-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-18\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-18", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "490c5224781372ce666a647916b6ce65dda6bcf69694dfcf5d08d51983417ccf", + "request_file_sha256": "1dfc61d859b514a9370dda912a1fdf7c1784bb6bc87e2bd69355a7f163223f6a", + "receipt_file_sha256": "c55a2283b48fbce858733902d0992743a4fca2fc728a215123579447dd925b1f", + "receipt_body_text_sha256": "4ba3017298ba3e28ee5633c0bfe6e2d2e08f1ce987af430380d37f6a657b7010", + "result_file_sha256": "c906ece7cab4749ad05a7177d633850ff1d4fb81386b5fae720064c1ca673a6d", + "raw_content_sha256": "2d0061ed80efcc2974dfae76435db00a06c1b6479dfbefd7325a9093a9208008", + "response_id": "20261014_e6a28166-89be-445c-ba6e-2b3d65732207", + "receipt_status": 200, + "prompt_tokens": 1443, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.953701791993808 + } + } + }, + { + "history_id": "confirm-fresh-18-te-m0", + "case_id": "fresh-18-te", + "scenario": "fresh-18", + "model": "sarvam-105b", + "family": "separate_booking", + "question": "VISIT-2118 కోసం ప్రస్తుతం నాకు ఇష్టమైన తేదీ ఏది? చెప్పండి మాత్రమే; బుకింగ్ మార్చవద్దు.", + "source_messages": [ + { + "id": "note-18-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. VISIT-2118 కోసం నా ఎంపిక 2027-04-19. VISIT-3118 కోసం మొదట 2027-03-19 అనుకున్నాను. సవరణ VISIT-3118కి మాత్రమే: ఇప్పుడు 2027-05-19 కావాలి. మొదటి బుకింగ్ ఎంపిక మారలేదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source retains 2027-04-19 as the target VISIT-2118 preferred date; its later correction is expressly limited to the other VISIT-31xx booking. The current backend date and other booking date do not answer this preference question.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-18", + "id": "VISIT-2118", + "date": "2027-03-19", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-18", + "id": "VISIT-2118", + "date": "2027-03-19", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "0ce7e7623a238a42130692241327e2b75960189a20904cb9a09e10fd5515f074", + "case_sha256": "ec9f2bc1091cffc1173317914474201ef357bceba4d6a4f0693eef9a080d335e", + "gold_label_sha256": "c6ccc0dcf8b9c3b66e05eeefd0a60e2bebd1b30d9e7ef93abe315ee574fe5848", + "reports": { + "chronological": { + "id": "confirm-fresh-18-te-m0-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "finish_reason": "length", + "raw_content": null, + "raw_output": null, + "expected_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure.", + "status_review": "No valid raw action status because the final is incomplete; counted incorrect.", + "request_sha256": "66144122b26625547d831ac144eb7d1e0408f5dc058b304683b8a60f1dc8556d", + "request_file_sha256": "cbff70b738127a85eed6ddbdc83a29d24413a1d1a2afb0836c60d01623ffd43c", + "receipt_file_sha256": "22436aa666e3816984b05323d21cd6057d86c22e93bae6d5dadff73f88e817fe", + "receipt_body_text_sha256": "2ef7fb9551cebeb24c112b30d28f00e77df29d71c2b985d6d1e6f5642bdc071e", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "raw_content_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "response_id": "20261014_db7aa972-15fe-4445-aaa0-b2fb4f1f3037", + "receipt_status": 200, + "prompt_tokens": 871, + "completion_tokens": 8192, + "reasoning_tokens": 8191, + "elapsed_seconds": 34.658750041999156 + }, + "grouped_lossless": { + "id": "confirm-fresh-18-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-19\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "809c6467feca20ac2690d70386b02cc7c0fff4fbd7052e44fe7d556618bfda6c", + "request_file_sha256": "7df43087d7b3fea7b6ae8246931b3c7e8419d74c97b603dae6e596d002af6a4c", + "receipt_file_sha256": "932bcab1147a1cac1789dd01721fddc596557666b300775dbfdfb2c441ad321b", + "receipt_body_text_sha256": "937e8d1a26c8c392ef9cf5050013e53ac10300369d3a913ef35ee139231ec99b", + "result_file_sha256": "03bc7ffa97b58cb6893a53bd07ae07966616cd8c647d669421d85a572a958aae", + "raw_content_sha256": "9f13b701995856fa703eec6041b4da4ba1c06e0ab9736e62b8ece9e50b06320e", + "response_id": "20261014_ac088727-bc4a-480e-ae7c-e64baa142cd5", + "receipt_status": 200, + "prompt_tokens": 1297, + "completion_tokens": 3627, + "reasoning_tokens": 3595, + "elapsed_seconds": 18.703877749998355 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-18-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-19\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "39f87fa72a83666dc5899791e295106eab941df2d5982b6c22df3ed75e994bfe", + "request_file_sha256": "c234b1bb6ef2d04217777d0e89a0d8fc5438424310354236df3d9ea4556cb554", + "receipt_file_sha256": "ab1c0b0d84db96d8c2d7ded9cd9bdba0764d2c37b62886f83c17d9f888984082", + "receipt_body_text_sha256": "fb60e273b15127a55e94317b9df2f48b71aa73a4d05368e2ef64e8fcc59c2726", + "result_file_sha256": "03bc7ffa97b58cb6893a53bd07ae07966616cd8c647d669421d85a572a958aae", + "raw_content_sha256": "9f13b701995856fa703eec6041b4da4ba1c06e0ab9736e62b8ece9e50b06320e", + "response_id": "20261014_9c0b3d4a-d465-4acb-8b55-7e6af491aa5f", + "receipt_status": 200, + "prompt_tokens": 1274, + "completion_tokens": 1618, + "reasoning_tokens": 1586, + "elapsed_seconds": 7.20122041599825 + } + } + }, + { + "history_id": "confirm-fresh-18-te-m1", + "case_id": "fresh-18-te", + "scenario": "fresh-18", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "question": "VISIT-2118 కోసం ప్రస్తుతం నాకు ఇష్టమైన తేదీ ఏది? చెప్పండి మాత్రమే; బుకింగ్ మార్చవద్దు.", + "source_messages": [ + { + "id": "note-18-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. VISIT-2118 కోసం నా ఎంపిక 2027-04-19. VISIT-3118 కోసం మొదట 2027-03-19 అనుకున్నాను. సవరణ VISIT-3118కి మాత్రమే: ఇప్పుడు 2027-05-19 కావాలి. మొదటి బుకింగ్ ఎంపిక మారలేదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source retains 2027-04-19 as the target VISIT-2118 preferred date; its later correction is expressly limited to the other VISIT-31xx booking. The current backend date and other booking date do not answer this preference question.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-18", + "id": "VISIT-2118", + "date": "2027-03-19", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-18", + "id": "VISIT-2118", + "date": "2027-03-19", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a72a999b33638ad0", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2118" + }, + "response": { + "date": "2027-03-19", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "dff88f66389457c11edc0f8a1722ac6adb304dd2ab2cdc6d0c9f35469ab066f8", + "case_sha256": "ec9f2bc1091cffc1173317914474201ef357bceba4d6a4f0693eef9a080d335e", + "gold_label_sha256": "c6ccc0dcf8b9c3b66e05eeefd0a60e2bebd1b30d9e7ef93abe315ee574fe5848", + "reports": { + "chronological": { + "id": "confirm-fresh-18-te-m1-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-19\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": true, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-03-19 differs from the source/actual-current target 2027-04-19; the raw date is unsupported for this question.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "48011e75bf56dc774b8f17ae93283b86990460fd5da990ed4e9fabe2c20deefa", + "request_file_sha256": "93335c589d17404c0709edb44231738bfda548734e9b69c90b68bef3e42db67e", + "receipt_file_sha256": "4b41f6017d94cc91828fbc78542e0ded11577cfbd189ee4c2502e0e85733a2c5", + "receipt_body_text_sha256": "dc3557b1ffdcb880102767440ce55107cfdd69b43c797bf40f1d2e00b5d4d7c5", + "result_file_sha256": "00d649cfc39ec358aa1d45e738f53feb9cb97e2c7b7fc9ca56a64eec5020214e", + "raw_content_sha256": "c89680404de922e7ad56887a56811c7ddb4eefcbb16c8b8689f6038edcee313e", + "response_id": "20261014_3a131a07-9875-486f-9d6e-7d806c14410d", + "receipt_status": 200, + "prompt_tokens": 1025, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9875284579902655 + }, + "grouped_lossless": { + "id": "confirm-fresh-18-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-03-19\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-03-19", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": true, + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-03-19 differs from the source/actual-current target 2027-04-19; the raw date is unsupported for this question.", + "status_review": "Raw action status confirmed conflicts with trusted intent and actual receipts, which establish not_requested.", + "request_sha256": "f2e8126cba7b6cc2bb4937db06a01225096aad64e664064d8a2cce94b9be4da8", + "request_file_sha256": "81a204004cc7dc7008d82bbb5a80f71a5ff4d1c9939221e07cf0adf142b526f0", + "receipt_file_sha256": "eb67c2d528798676293cb05d55f5cfbd22a45c96f9197b89948c515d2aaeb3f9", + "receipt_body_text_sha256": "0a81eeb0e6fd665842f07f75510b9921c516b5105ab316a190ea34500e34c3dd", + "result_file_sha256": "c3381da56b5e4c327de6a23fba063f02f3de77e20c8c41a5ba4c99f2317591f0", + "raw_content_sha256": "6e4c3efc9f6d0ac4c7148f8b28883e8b7b74253291cb26bd8f87cc4b99b2fddd", + "response_id": "20261014_a8877829-5cce-4141-9159-bc8156daca1e", + "receipt_status": 200, + "prompt_tokens": 1606, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.9496736669971142 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-18-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-19\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-19", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-19", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": true, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-03-19 differs from the source/actual-current target 2027-04-19; the raw date is unsupported for this question.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "3342602a685050ce30ca5913808337c9771078e573c3de2526abde6a84a41312", + "request_file_sha256": "28cef0c5bb278a215b5be71461f2738c444581a34348c9650c2046e1c1ce1d3e", + "receipt_file_sha256": "edf9e55b3a84ae535f1106edac3e29d3aafbcb9ebfcc67424e21efd487830a5e", + "receipt_body_text_sha256": "aef7531ca6d2782315d17f49d445126f3dfb86e58a50dc1e59d36b6ca7a7e50b", + "result_file_sha256": "00d649cfc39ec358aa1d45e738f53feb9cb97e2c7b7fc9ca56a64eec5020214e", + "raw_content_sha256": "c89680404de922e7ad56887a56811c7ddb4eefcbb16c8b8689f6038edcee313e", + "response_id": "20261014_784f5291-f22d-40de-a5db-a2c77d0dd1c3", + "receipt_status": 200, + "prompt_tokens": 1571, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9564855829958105 + } + } + }, + { + "history_id": "confirm-fresh-19-te-m0", + "case_id": "fresh-19-te", + "scenario": "fresh-19", + "model": "sarvam-105b", + "family": "separate_booking", + "question": "VISIT-2119 కోసం ప్రస్తుతం నాకు ఇష్టమైన తేదీ ఏది? చెప్పండి మాత్రమే; బుకింగ్ మార్చవద్దు.", + "source_messages": [ + { + "id": "note-19-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-19-1", + "role": "customer", + "content": "VISIT-2119 కోసం నా ఎంపిక 2027-04-20. VISIT-3119 కోసం మొదట 2027-03-20 అనుకున్నాను." + }, + { + "id": "note-19-2", + "role": "customer", + "content": "సవరణ VISIT-3119కి మాత్రమే: ఇప్పుడు 2027-05-20 కావాలి. మొదటి బుకింగ్ ఎంపిక మారలేదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source retains 2027-04-20 as the target VISIT-2119 preferred date; its later correction is expressly limited to the other VISIT-31xx booking. The current backend date and other booking date do not answer this preference question.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-19", + "id": "VISIT-2119", + "date": "2027-03-20", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-19", + "id": "VISIT-2119", + "date": "2027-03-20", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-b96734168c37ca67", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2119" + }, + "response": { + "date": "2027-03-20", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "5c994ee1a58f5bd77a536ead5b956d6665c3bcdfac7c01e7aa6371787ea729ab", + "case_sha256": "722f113e033941c727c2a5d913c93a127d53139ada779455e142f46e696c6e25", + "gold_label_sha256": "93b18ec24d1d001232192a6433e7ad0c5263a0f19fb33aec9f990b16dc5c7022", + "reports": { + "chronological": { + "id": "confirm-fresh-19-te-m0-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\": \"confirmed\", \"answer\": \"2027-03-20\", \"needs_clarification\": false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-03-20", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": true, + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-03-20 differs from the source/actual-current target 2027-04-20; the raw date is unsupported for this question.", + "status_review": "Raw action status confirmed conflicts with trusted intent and actual receipts, which establish not_requested.", + "request_sha256": "9c763c2aa8d44aabcb521e5be148de11b2666bc6263a710e54cd47fbbb660612", + "request_file_sha256": "ad26b8eeb2f80a4254742f207596f6885c20a9cef29df1aa75d90ec7bb3844ed", + "receipt_file_sha256": "430d852e54523b01083471de43757ccc15369f9ca3291934d242dec54a63fd5a", + "receipt_body_text_sha256": "c97470c13c58a6e35e6259963fe2e63e19b0a27237ef2b24e33aba6816102b70", + "result_file_sha256": "515ec2720e24ee39e5a4b433b38304fdcd549a00cc5b71768f7e357c2d975dfd", + "raw_content_sha256": "e414f0467a3a8ffd2922eb5949cd708aa0baeefb883102ebb36ed38ac393874b", + "response_id": "20261014_e079c4ae-7621-4027-8230-465d82902a9e", + "receipt_status": 200, + "prompt_tokens": 1052, + "completion_tokens": 3095, + "reasoning_tokens": 3059, + "elapsed_seconds": 12.99166666601377 + }, + "grouped_lossless": { + "id": "confirm-fresh-19-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-20\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "83db1df09eacec84ed46cfddbaadea201c21404318a8cb3d1263645513c7bc24", + "request_file_sha256": "83d61d6d1bad63d38b0f0a0d9bc711f19dc345bfd0df2a217e260299eb212593", + "receipt_file_sha256": "535a7f5fc3ed8f20a7736c13c154fa1d1d74816749583218f67cd5428dbb990d", + "receipt_body_text_sha256": "f7d53a2cd1723ad68d767d2a52f3722382af1de08bc340ac290a107e5392bf32", + "result_file_sha256": "4f2ff4c6daf1623e66caead1eec254086b77631cbd7ac1df8765b25a639a0c24", + "raw_content_sha256": "ccbb204d6da4516cd3f603b7585003889275c2ceab7bdb788b3288039143cf26", + "response_id": "20261014_d62fbeba-b789-42e0-b8d1-b77bdb08afb0", + "receipt_status": 200, + "prompt_tokens": 1739, + "completion_tokens": 7697, + "reasoning_tokens": 7665, + "elapsed_seconds": 31.99121137498878 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-19-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-20\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_performed", + "answer": "2027-03-20", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": true, + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-03-20 differs from the source/actual-current target 2027-04-20; the raw date is unsupported for this question.", + "status_review": "Raw action status not_performed conflicts with trusted intent and actual receipts, which establish not_requested.", + "request_sha256": "99c1fc39ada08d0b0d39402ed23a7223f7089fb47cdb3321b4844ff17fee4269", + "request_file_sha256": "500265a7f2b4b3f169016bf8af337176aa2d899d61712f6165c971434f72298b", + "receipt_file_sha256": "ab7716e40baf25c7439e6837314c81c2d286381cb65878893ae171f9089649e4", + "receipt_body_text_sha256": "2d815762099450743f753fa32c3f1de53d11637c51506c342a6692de05d4e20b", + "result_file_sha256": "e1bb23a3fb2a08e8bd9ecf4790de66550406c987a5f2f7889494245ce06626c3", + "raw_content_sha256": "dd8d02d6db895c5635fc959c9262b96346f67c6ef3ed6bc12ff404bf035f3754", + "response_id": "20261014_da53b531-0578-4d17-a2b2-96b834925b01", + "receipt_status": 200, + "prompt_tokens": 1711, + "completion_tokens": 2823, + "reasoning_tokens": 2789, + "elapsed_seconds": 12.417185833997792 + } + } + }, + { + "history_id": "confirm-fresh-19-te-m1", + "case_id": "fresh-19-te", + "scenario": "fresh-19", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "question": "VISIT-2119 కోసం ప్రస్తుతం నాకు ఇష్టమైన తేదీ ఏది? చెప్పండి మాత్రమే; బుకింగ్ మార్చవద్దు.", + "source_messages": [ + { + "id": "note-19-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-19-1", + "role": "customer", + "content": "VISIT-2119 కోసం నా ఎంపిక 2027-04-20. VISIT-3119 కోసం మొదట 2027-03-20 అనుకున్నాను." + }, + { + "id": "note-19-2", + "role": "customer", + "content": "సవరణ VISIT-3119కి మాత్రమే: ఇప్పుడు 2027-05-20 కావాలి. మొదటి బుకింగ్ ఎంపిక మారలేదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source retains 2027-04-20 as the target VISIT-2119 preferred date; its later correction is expressly limited to the other VISIT-31xx booking. The current backend date and other booking date do not answer this preference question.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-19", + "id": "VISIT-2119", + "date": "2027-03-20", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-19", + "id": "VISIT-2119", + "date": "2027-03-20", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "457a13df33dd19849d2232da525afc8132c0a4c0b1882319dbda9b5556ae2ff5", + "case_sha256": "722f113e033941c727c2a5d913c93a127d53139ada779455e142f46e696c6e25", + "gold_label_sha256": "93b18ec24d1d001232192a6433e7ad0c5263a0f19fb33aec9f990b16dc5c7022", + "reports": { + "chronological": { + "id": "confirm-fresh-19-te-m1-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-20\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": true, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-03-20 differs from the source/actual-current target 2027-04-20; the raw date is unsupported for this question.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "19d0ff1b4ab0aff288b170aa6c5bea426ad64d395b4b520fa6a3d7f646dc8f6d", + "request_file_sha256": "12826b22534534cb4eb646344fa5cccb22c8b42a1792d0c6bf71349b14f47f01", + "receipt_file_sha256": "27986500fe2e1227845969a209a95975ad8d904002f738d3b793cd4529550aa8", + "receipt_body_text_sha256": "a3c457559e8c1282f0f3f88f838c3d469286ca2527cfc219c433ef214f027924", + "result_file_sha256": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "raw_content_sha256": "f4f4478b5c7f08706d0eb37b41fdb9c470082ee0f253ae4f0ca504e2eb97a9a7", + "response_id": "20261014_5ff61eb3-a24c-46d0-ad08-cf2221352cd4", + "receipt_status": 200, + "prompt_tokens": 937, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0228360409964807 + }, + "grouped_lossless": { + "id": "confirm-fresh-19-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-20\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": true, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-03-20 differs from the source/actual-current target 2027-04-20; the raw date is unsupported for this question.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "e2c4b7b8dea5604cf9935258a082d47837bfaa546f20ed15a33488114608f896", + "request_file_sha256": "73e80936796ede10910167033db3c155f7ba7f52a55176e094559fd129cb1055", + "receipt_file_sha256": "292375d7807d9f272dbbb1f363725134722ee91d30a30782869917508aff1933", + "receipt_body_text_sha256": "7681bc31063be4291d9e609244ad52e56e2f67183ef2882ddb124a96989484c7", + "result_file_sha256": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "raw_content_sha256": "f4f4478b5c7f08706d0eb37b41fdb9c470082ee0f253ae4f0ca504e2eb97a9a7", + "response_id": "20261014_bc11bb37-4db3-4309-be21-0368ff00498b", + "receipt_status": 200, + "prompt_tokens": 1481, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9868211249995511 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-19-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-20\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-20", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-20", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": true, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-03-20 differs from the source/actual-current target 2027-04-20; the raw date is unsupported for this question.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "8bc08f1314ecd3c875b12e522a3ea2033fd3c4f91f68558ace96b9454772316d", + "request_file_sha256": "9b5e4badd2d996902940b9d5d5ecd3b1ed787f1ca7596eb7a2a06609a9d31bf2", + "receipt_file_sha256": "f5e2e05b948ab41290e57b7590387f526f2bd61111791000dfd83c7b06d80783", + "receipt_body_text_sha256": "e7931fc7e8adb4cc5792ae0c7102528f95421e2cee90c80f766eccf085e397c5", + "result_file_sha256": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "raw_content_sha256": "f4f4478b5c7f08706d0eb37b41fdb9c470082ee0f253ae4f0ca504e2eb97a9a7", + "response_id": "20261014_9ddec063-5383-4de1-8f6f-d761bcc93388", + "receipt_status": 200, + "prompt_tokens": 1434, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0013922919897595 + } + } + }, + { + "history_id": "confirm-fresh-20-te-m0", + "case_id": "fresh-20-te", + "scenario": "fresh-20", + "model": "sarvam-105b", + "family": "separate_booking", + "question": "VISIT-2120 కోసం ప్రస్తుతం నాకు ఇష్టమైన తేదీ ఏది? చెప్పండి మాత్రమే; బుకింగ్ మార్చవద్దు.", + "source_messages": [ + { + "id": "note-20-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. VISIT-2120 కోసం నా ఎంపిక 2027-04-21. VISIT-3120 కోసం మొదట 2027-03-21 అనుకున్నాను. సవరణ VISIT-3120కి మాత్రమే: ఇప్పుడు 2027-05-21 కావాలి. మొదటి బుకింగ్ ఎంపిక మారలేదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source retains 2027-04-21 as the target VISIT-2120 preferred date; its later correction is expressly limited to the other VISIT-31xx booking. The current backend date and other booking date do not answer this preference question.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-20", + "id": "VISIT-2120", + "date": "2027-03-21", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-20", + "id": "VISIT-2120", + "date": "2027-03-21", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "8061098c20a896a51b6e4eda4253de1b71cde7abe8912ffa53edcf901c91598f", + "case_sha256": "f626ddcfcc0af7adebd7c6a1b97e73b5ad2cf0ab72dc0a9499ada61c75f24e20", + "gold_label_sha256": "8a1a27eea6cb1ac90d0909afcc99a0e4ba22af10b5c42c964bbe5079d5aa5b96", + "reports": { + "chronological": { + "id": "confirm-fresh-20-te-m0-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-21\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "6c09d9b82284dd185c38fa0d11df8c6302e7658e00ce8e894d4fdd226cdefe2f", + "request_file_sha256": "a3e2df921e33e7768ec14431e2376043d497d715b6327b20e4ecb4654b58aa00", + "receipt_file_sha256": "217f78d368d6d52a84ebd884dc5921cb5470752a5305296f10147d105c301a2c", + "receipt_body_text_sha256": "ee0311d134f373a64b5a5a4453b14905cb4de552b4f5e4f7ac225a232d631bf0", + "result_file_sha256": "47da800200c0c2a40c8f6e6d4d56ab1df082b5204fe5af8fa9d8b032bf3fec83", + "raw_content_sha256": "50dd30c16ff07995e873e614e9c42b88e61dd6fd45c0f360a0172c3453e458a3", + "response_id": "20261014_7b86a286-fdcf-4836-8a85-08925f315ac8", + "receipt_status": 200, + "prompt_tokens": 863, + "completion_tokens": 5405, + "reasoning_tokens": 5373, + "elapsed_seconds": 26.097864416995435 + }, + "grouped_lossless": { + "id": "confirm-fresh-20-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"rejected\",\"answer\":null,\"needs_clarification\":true}", + "raw_output": { + "action_status": "rejected", + "answer": null, + "needs_clarification": true + }, + "expected_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": true, + "factual_review": "False abstention: source or final backend uniquely establishes 2027-04-21, but raw answer is null.", + "status_review": "Raw action status rejected conflicts with trusted intent and actual receipts, which establish not_requested.", + "request_sha256": "6119f9ea01665128b329d7e4ed6766e67692cac71d3f44a9ee10612780e35cbe", + "request_file_sha256": "c7239f5892d858167206e05ef1ddeb2fbaf7648b666d683e68affda16018966c", + "receipt_file_sha256": "cf6207e78165d91a4e1733290ff42e9418342822a96f3bbd092437b16014bff0", + "receipt_body_text_sha256": "0c9c036413274dff2d03d912bf0d82034198d0d4b53d464672a8cda3d0c12a64", + "result_file_sha256": "44002938658ce12fa57d9ee0682ca5767a021114be6c0cbe8d8adbfea1b01f2a", + "raw_content_sha256": "b0c8bdd4e4b8dbf54110b0365a94443a1ef48ea834bddaefce998503afaf7471", + "response_id": "20261014_e9e0548c-2f2b-4e1a-a624-7201588a67d7", + "receipt_status": 200, + "prompt_tokens": 1303, + "completion_tokens": 6569, + "reasoning_tokens": 6546, + "elapsed_seconds": 27.481848375013215 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-20-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-21\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "790bb35253d0121994538dbab6b906c50ef166ca1b4c795560beb866e4a3870b", + "request_file_sha256": "498e1772db016684df6399b5425108fc4e4b9d9daa090e788fbb2a4accd81775", + "receipt_file_sha256": "6971961a5802952fb407c1c48fe962be87225efaa263a3321b454001dbaeb47d", + "receipt_body_text_sha256": "44b32c5c9e8845344f1ff5aeca317d8223959919e2859c42a4d8c0c24b348a3e", + "result_file_sha256": "47da800200c0c2a40c8f6e6d4d56ab1df082b5204fe5af8fa9d8b032bf3fec83", + "raw_content_sha256": "50dd30c16ff07995e873e614e9c42b88e61dd6fd45c0f360a0172c3453e458a3", + "response_id": "20261014_ec02b703-cc07-4bc2-86b8-fba2964e2076", + "receipt_status": 200, + "prompt_tokens": 1288, + "completion_tokens": 3797, + "reasoning_tokens": 3765, + "elapsed_seconds": 19.656710999988718 + } + } + }, + { + "history_id": "confirm-fresh-20-te-m1", + "case_id": "fresh-20-te", + "scenario": "fresh-20", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "question": "VISIT-2120 కోసం ప్రస్తుతం నాకు ఇష్టమైన తేదీ ఏది? చెప్పండి మాత్రమే; బుకింగ్ మార్చవద్దు.", + "source_messages": [ + { + "id": "note-20-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. VISIT-2120 కోసం నా ఎంపిక 2027-04-21. VISIT-3120 కోసం మొదట 2027-03-21 అనుకున్నాను. సవరణ VISIT-3120కి మాత్రమే: ఇప్పుడు 2027-05-21 కావాలి. మొదటి బుకింగ్ ఎంపిక మారలేదు." + } + ], + "question_target": "source_request_or_condition", + "source_interpretation": "The source retains 2027-04-21 as the target VISIT-2120 preferred date; its later correction is expressly limited to the other VISIT-31xx booking. The current backend date and other booking date do not answer this preference question.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_requested", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-20", + "id": "VISIT-2120", + "date": "2027-03-21", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-20", + "id": "VISIT-2120", + "date": "2027-03-21", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "8f912f24131412b91ea088e05f9cc9c7b627dd5b64fb0fe04e9aea5f30e21a7b", + "case_sha256": "f626ddcfcc0af7adebd7c6a1b97e73b5ad2cf0ab72dc0a9499ada61c75f24e20", + "gold_label_sha256": "8a1a27eea6cb1ac90d0909afcc99a0e4ba22af10b5c42c964bbe5079d5aa5b96", + "reports": { + "chronological": { + "id": "confirm-fresh-20-te-m1-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-21\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": true, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-03-21 differs from the source/actual-current target 2027-04-21; the raw date is unsupported for this question.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "b7ca1b2bab979cde4f58f81f95c26aa5794fcc2114955d2cb32615a99e645c1f", + "request_file_sha256": "42c7ba5ea34a52799f65c576c2d6cdaa2c8b74d3eeb25b8e4ebcc276e3549780", + "receipt_file_sha256": "a6893e7e1a4d860ffe657f0cf2a7d96757bd1d911cd4f25cc9b35b8561872740", + "receipt_body_text_sha256": "aded8c94dc2eafc21a4209f053a644707fb25286ce5a325d19190f58c13972b1", + "result_file_sha256": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "raw_content_sha256": "efda99defce623b65285148dfd0b9814f4a3c673b764571d0d3827b37255a294", + "response_id": "20261014_2861f1d8-9fde-489f-89f5-5f56170723e5", + "receipt_status": 200, + "prompt_tokens": 896, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9413330840034178 + }, + "grouped_lossless": { + "id": "confirm-fresh-20-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-21\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": true, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-03-21 differs from the source/actual-current target 2027-04-21; the raw date is unsupported for this question.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "59ed542d68ee86f8774d954dc3da97d1e743b9f47d5c5e2e26ef832d6806e9dd", + "request_file_sha256": "07f32311b48733e03609e864e46e031466d228894051f30bd441e9475d748bcc", + "receipt_file_sha256": "facdd40abf33895bfac0926604a2b0ff0868acb75c8d923581ee224acb4373c2", + "receipt_body_text_sha256": "2bebb4654f909a8377fae2dbbec53d87e4c1d6b5e4818f99aaec7b74a84b48c4", + "result_file_sha256": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "raw_content_sha256": "efda99defce623b65285148dfd0b9814f4a3c673b764571d0d3827b37255a294", + "response_id": "20261014_9d01b2e1-c1cd-4a55-a54a-ecc8abc2e6c0", + "receipt_status": 200, + "prompt_tokens": 1339, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0834329999925103 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-20-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-21\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-21", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-21", + "needs_clarification": false + }, + "expected_action_status": "not_requested", + "date_only_match": false, + "clarification_match": true, + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "wrong_non_null_date": true, + "false_abstention": false, + "factual_review": "Raw 2027-03-21 differs from the source/actual-current target 2027-04-21; the raw date is unsupported for this question.", + "status_review": "Raw action status not_requested agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "09731e7377b59a9f93abb7d0087f2202679188d0e249161270339fe1c203d8a9", + "request_file_sha256": "94a140a06a6a922903c6bd8623c19b082549a013a35a904ac9f4b6d52aca4f05", + "receipt_file_sha256": "5f54ed5ab0bc46158307c27947de058efab5900a59cc534d0f5e28ba48c2d355", + "receipt_body_text_sha256": "24422ec299c3769fd03367556d8ae38fc9eaa0ee0a64c2844f1c5bd575fcfb6a", + "result_file_sha256": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "raw_content_sha256": "efda99defce623b65285148dfd0b9814f4a3c673b764571d0d3827b37255a294", + "response_id": "20261014_2e26723a-7413-4357-8a14-71b20ba7193f", + "receipt_status": 200, + "prompt_tokens": 1295, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9437112919986248 + } + } + }, + { + "history_id": "confirm-fresh-21-te-m0", + "case_id": "fresh-21-te", + "scenario": "fresh-21", + "model": "sarvam-105b", + "family": "approved_change", + "question": "VISIT-2121 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-21-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-21-1", + "role": "customer", + "content": "VISIT-2121ని 2027-04-22కి మార్చండి." + }, + { + "id": "note-21-2", + "role": "customer", + "content": "ఇచ్చిన అభ్యర్థన గుర్తింపు, సర్వర్ ఇచ్చిన సంబంధిత అనుమతి వాడండి. అభ్యర్థనను పరిశీలించిన తర్వాత ఖరారైన తేదీ చెప్పండి." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The native trace has a matching server-authorized confirmed update for VISIT-2121; the scoped final backend date is 2027-04-22. The question asks that actual current date, not only the earlier request.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "confirmed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-21", + "id": "VISIT-2121", + "date": "2027-03-22", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-21", + "id": "VISIT-2121", + "date": "2027-04-22", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-bcf20b8c9702edc3", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2121", + "approval_id": "AUTH-21-GRANT-16", + "date": "2027-04-22", + "expected_version": 5, + "idempotency_key": "REQUEST-21-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-22", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "0845b2c0b520d98983ee2fc5a900f5e2c873f24d13a3ebc50ed54d65b07edbf9", + "case_sha256": "d01a24fdb0403e3736eaa0fde12bb9fb31cdca396f26e6913ab79bd76b67be47", + "gold_label_sha256": "cdfaa9f518dc5932253c846e56a8b22bd36e9ef5f6e9b8410c9aca95230cf149", + "reports": { + "chronological": { + "id": "confirm-fresh-21-te-m0-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-22\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "804beae0ca988700e604fa29e4d570b467ebf267662b13e953e2cab58c0f6f9c", + "request_file_sha256": "964efbc300d331437d09209ccb8d192f3e884b578026b0d170d7b6074cfda5a0", + "receipt_file_sha256": "a4596aac3c9564c36f8b037ccb5b9eecd3054d1ca8a4dc0bfead53ffc2cfaff8", + "receipt_body_text_sha256": "c924f7900576f533358e8a396241f098f8212bc655cd5b2ab53c839c90cdf95e", + "result_file_sha256": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "raw_content_sha256": "2b038dfe210662433dc98cbece33644925efcac1dc025402d4110813a6ba5ac4", + "response_id": "20261014_57e83f10-7c9d-4cba-b213-7ea510b9d0c3", + "receipt_status": 200, + "prompt_tokens": 1370, + "completion_tokens": 580, + "reasoning_tokens": 550, + "elapsed_seconds": 3.5836629169934895 + }, + "grouped_lossless": { + "id": "confirm-fresh-21-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-22\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "4e3abb0cb642264723ee2ce3076d3434642a396b3d4be53b60f0e51945787bd9", + "request_file_sha256": "bf670853acbec3997e025f376e4b4944c6ae6ea8b1a68476ba3ea776ddf0cdab", + "receipt_file_sha256": "1b15a67840704a9f6bfe93989c8728af4435b4dfa9b4f70e320473ef220b50a2", + "receipt_body_text_sha256": "31e942ef2da6fe96b722f32bfdfeb3492aa7869563740745f7cfe505b8e4ac7d", + "result_file_sha256": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "raw_content_sha256": "2b038dfe210662433dc98cbece33644925efcac1dc025402d4110813a6ba5ac4", + "response_id": "20261014_8f67b7de-5452-4576-8e9e-61312d65bbe5", + "receipt_status": 200, + "prompt_tokens": 1973, + "completion_tokens": 7582, + "reasoning_tokens": 7552, + "elapsed_seconds": 36.53783091700461 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-21-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-22\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "4e3abb0cb642264723ee2ce3076d3434642a396b3d4be53b60f0e51945787bd9", + "request_file_sha256": "bf670853acbec3997e025f376e4b4944c6ae6ea8b1a68476ba3ea776ddf0cdab", + "receipt_file_sha256": "85c1a07dc8399d3ad9b7677779e6cab09be541f1d2600e4a73eebadd120c2bcd", + "receipt_body_text_sha256": "b648d9b7a04852fade93207ee373daf4825004364989dbd8545b637f1921b8e9", + "result_file_sha256": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "raw_content_sha256": "2b038dfe210662433dc98cbece33644925efcac1dc025402d4110813a6ba5ac4", + "response_id": "20261014_44fdb97f-9b81-4853-bf63-ee0e8026c739", + "receipt_status": 200, + "prompt_tokens": 1973, + "completion_tokens": 2392, + "reasoning_tokens": 2362, + "elapsed_seconds": 10.647001292003551 + } + } + }, + { + "history_id": "confirm-fresh-21-te-m1", + "case_id": "fresh-21-te", + "scenario": "fresh-21", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "question": "VISIT-2121 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-21-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-21-1", + "role": "customer", + "content": "VISIT-2121ని 2027-04-22కి మార్చండి." + }, + { + "id": "note-21-2", + "role": "customer", + "content": "ఇచ్చిన అభ్యర్థన గుర్తింపు, సర్వర్ ఇచ్చిన సంబంధిత అనుమతి వాడండి. అభ్యర్థనను పరిశీలించిన తర్వాత ఖరారైన తేదీ చెప్పండి." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The native trace has a matching server-authorized confirmed update for VISIT-2121; the scoped final backend date is 2027-04-22. The question asks that actual current date, not only the earlier request.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "confirmed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-21", + "id": "VISIT-2121", + "date": "2027-03-22", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-21", + "id": "VISIT-2121", + "date": "2027-04-22", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9d5ebf6079ad9b66", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2121" + }, + "response": { + "date": "2027-03-22", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-a7d57a066a444256", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2121", + "approval_id": "AUTH-21-GRANT-16", + "date": "2027-04-22", + "expected_version": 5, + "idempotency_key": "REQUEST-21-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-22", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "fd2ad07ae33c3a16c94a6bae4b1172ed3769d248e70d3605b8ab1bbaf625dcb3", + "case_sha256": "d01a24fdb0403e3736eaa0fde12bb9fb31cdca396f26e6913ab79bd76b67be47", + "gold_label_sha256": "cdfaa9f518dc5932253c846e56a8b22bd36e9ef5f6e9b8410c9aca95230cf149", + "reports": { + "chronological": { + "id": "confirm-fresh-21-te-m1-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-22\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "f03acb9a9f04b6a405bd986a005613d167888c56cdbd7e308e43c45dc1c5dbc8", + "request_file_sha256": "ab341529480470730bc60657729bd08eb8e6fc8a5dec6e30435193e0e1d5e471", + "receipt_file_sha256": "bc632d7ba9058e2d8e198495e645c7431beed880fc2aac6e9fd16d721342fc62", + "receipt_body_text_sha256": "6cd4280576abbb3c7627e95ede9f2a7e24dd5322a44171ebc81c923d847922fe", + "result_file_sha256": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "raw_content_sha256": "9649b662bcdbbb140ad57a183801a8f220b3a023c361cbd568d24381e0f098e1", + "response_id": "20261014_33d7f185-18c0-4563-885a-1495eb9893c3", + "receipt_status": 200, + "prompt_tokens": 1514, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.9338703329995042 + }, + "grouped_lossless": { + "id": "confirm-fresh-21-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-22\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "1c18a2fd6abdab15ee4a6f73a49d073cf597fe380cf2771daddabcce9ce390b8", + "request_file_sha256": "892f20005064eccf6fadf38fcdeb10353e9394ba8b54b559fcc969e50e2e9a5b", + "receipt_file_sha256": "accb12d32890079ef96f0a485e2dee234899c3821ee1dfedbd4a553d1a09b598", + "receipt_body_text_sha256": "b82bd27fcf2aab3658ea62e6c8ffcb3cf81909f1be8c85bb400672ca76ebed85", + "result_file_sha256": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "raw_content_sha256": "9649b662bcdbbb140ad57a183801a8f220b3a023c361cbd568d24381e0f098e1", + "response_id": "20261014_fa2bafae-ea20-4f8d-83bd-0900523c87fd", + "receipt_status": 200, + "prompt_tokens": 2266, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.974952583987033 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-21-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-22\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-22", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "1c18a2fd6abdab15ee4a6f73a49d073cf597fe380cf2771daddabcce9ce390b8", + "request_file_sha256": "892f20005064eccf6fadf38fcdeb10353e9394ba8b54b559fcc969e50e2e9a5b", + "receipt_file_sha256": "023f0bac45bb66273bd7a62ecaea757e6e66aef01db1044ef50efce1fe6d4fbf", + "receipt_body_text_sha256": "d24c8b3be76506a0b703d7609dbeddb3149311eb27bc1120d555161dec3b2fb4", + "result_file_sha256": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "raw_content_sha256": "9649b662bcdbbb140ad57a183801a8f220b3a023c361cbd568d24381e0f098e1", + "response_id": "20261014_9e4ecd9e-ff13-4bda-ad07-bcb93c35df78", + "receipt_status": 200, + "prompt_tokens": 2266, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.9539970839978196 + } + } + }, + { + "history_id": "confirm-fresh-22-te-m0", + "case_id": "fresh-22-te", + "scenario": "fresh-22", + "model": "sarvam-105b", + "family": "approved_change", + "question": "VISIT-2122 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-22-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. VISIT-2122ని 2027-04-23కి మార్చండి. ఇచ్చిన అభ్యర్థన గుర్తింపు, సర్వర్ ఇచ్చిన సంబంధిత అనుమతి వాడండి. అభ్యర్థనను పరిశీలించిన తర్వాత ఖరారైన తేదీ చెప్పండి." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The native trace has a matching server-authorized confirmed update for VISIT-2122; the scoped final backend date is 2027-04-23. The question asks that actual current date, not only the earlier request.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "confirmed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-22", + "id": "VISIT-2122", + "date": "2027-03-23", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-22", + "id": "VISIT-2122", + "date": "2027-04-23", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9840dc622e7cadb9", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2122" + }, + "response": { + "date": "2027-03-23", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-a4a15e13144bba29", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2122", + "approval_id": "AUTH-22-GRANT-16", + "date": "2027-04-23", + "expected_version": 5, + "idempotency_key": "REQUEST-22-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-23", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "89c3c45d1f7e197e8637b2ed6d52c3b0e30d03ed800783885d563cc56b9e8b09", + "case_sha256": "1be91faf479b61941d298b5c55b7fdc9104cd85803abdda5a31098d77870af63", + "gold_label_sha256": "83472ca62404f2d2dc637ffc732d537768c3f46401641219d5cbc6b0cf635b5b", + "reports": { + "chronological": { + "id": "confirm-fresh-22-te-m0-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-23\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "4246d405e1c45bcc36800a4c41aa980795436d993469d1ee7ec975dbb46cfb59", + "request_file_sha256": "422467c5e1525f2e011d2396e38ab32ef0742f99644aac5ad50c8ee94586e968", + "receipt_file_sha256": "74da47fed7587878d05924c827b4499f1df829a5343cda7466d087491300f795", + "receipt_body_text_sha256": "a330a606f7467d03b24308f037a62737c9db80f3a5f64a5599a0106ff966a642", + "result_file_sha256": "7a6034546c590ec5064db6d004a77a9dcf0a4f8eab5062272644c6750c85fef7", + "raw_content_sha256": "412d38ba18a68b8292a1112c6a6da160880d9f3ab5b1ae562dc88ac8bf6b7a42", + "response_id": "20261014_4f78e7d2-22f9-4319-82e6-d9e41aee8103", + "receipt_status": 200, + "prompt_tokens": 1540, + "completion_tokens": 1075, + "reasoning_tokens": 1045, + "elapsed_seconds": 6.07318908299203 + }, + "grouped_lossless": { + "id": "confirm-fresh-22-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "incomplete", + "finish_reason": "length", + "raw_content": null, + "raw_output": null, + "expected_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure.", + "status_review": "No valid raw action status because the final is incomplete; counted incorrect.", + "request_sha256": "543a548bc4d21fa4b9711500d61e45e5390e28262a0fc5f4ca6043c1346a5e57", + "request_file_sha256": "05ec5c21153b1792325b33c9cba9c37a7747e3054d8b54ed6fbd55c77517f7ed", + "receipt_file_sha256": "27b514e3edfd16710005675d5c115913688e4de87c715dab5df7bdc0da45385c", + "receipt_body_text_sha256": "1459c12be032d041b47abdcfb999082017d2e53fe3e4bff8bbf106d6a96ae30c", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "raw_content_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "response_id": "20261014_020a574b-2f6e-419b-8d6c-a9536cc90d59", + "receipt_status": 200, + "prompt_tokens": 2203, + "completion_tokens": 8192, + "reasoning_tokens": 8191, + "elapsed_seconds": 35.501241707999725 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-22-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-23\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "b42306dae94e17b5652defac42054801bfd01dce1bc1f71e6dad50cfeeb7b09e", + "request_file_sha256": "327f0d84dd108abaea3ea934ec6694561e2b875497889924fb461c30f19cf7a1", + "receipt_file_sha256": "a3aaa749928f1c6982920d0ee8f8c7e105a4a3dd227e275e2b6c78d7bb8c7c74", + "receipt_body_text_sha256": "106673b2962a768458a5b7308a0269e85eebf14b4a7687dd0e43d6c7738f31d8", + "result_file_sha256": "7a6034546c590ec5064db6d004a77a9dcf0a4f8eab5062272644c6750c85fef7", + "raw_content_sha256": "412d38ba18a68b8292a1112c6a6da160880d9f3ab5b1ae562dc88ac8bf6b7a42", + "response_id": "20261014_f91ca20a-dcca-43b8-9c55-3c96924a8017", + "receipt_status": 200, + "prompt_tokens": 2127, + "completion_tokens": 2827, + "reasoning_tokens": 2797, + "elapsed_seconds": 13.351604625000618 + } + } + }, + { + "history_id": "confirm-fresh-22-te-m1", + "case_id": "fresh-22-te", + "scenario": "fresh-22", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "question": "VISIT-2122 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-22-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. VISIT-2122ని 2027-04-23కి మార్చండి. ఇచ్చిన అభ్యర్థన గుర్తింపు, సర్వర్ ఇచ్చిన సంబంధిత అనుమతి వాడండి. అభ్యర్థనను పరిశీలించిన తర్వాత ఖరారైన తేదీ చెప్పండి." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The native trace has a matching server-authorized confirmed update for VISIT-2122; the scoped final backend date is 2027-04-23. The question asks that actual current date, not only the earlier request.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "confirmed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-22", + "id": "VISIT-2122", + "date": "2027-03-23", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-22", + "id": "VISIT-2122", + "date": "2027-04-23", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-82ae16c006604b86", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2122" + }, + "response": { + "date": "2027-03-23", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-a01b723bba765edc", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2122", + "approval_id": "AUTH-22-GRANT-16", + "date": "2027-04-23", + "expected_version": 5, + "idempotency_key": "REQUEST-22-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-23", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "ee06e99dd0cadb034a109c9e99102d5dbc6ca9665c37ef0c04935d24f3425b0c", + "case_sha256": "1be91faf479b61941d298b5c55b7fdc9104cd85803abdda5a31098d77870af63", + "gold_label_sha256": "83472ca62404f2d2dc637ffc732d537768c3f46401641219d5cbc6b0cf635b5b", + "reports": { + "chronological": { + "id": "confirm-fresh-22-te-m1-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-23\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "46fc6ea721afe218452914db2ad0f71db4ddaf5ad15d7c8d246e703709a10d9c", + "request_file_sha256": "919a1e795a127e59ef4b7fb759ee2f7a8a581517b3acc7219a67ab7c956fa325", + "receipt_file_sha256": "86835433233dbe9579d740b37472bb3a3470fc62486a780cd1d69147ce5089fd", + "receipt_body_text_sha256": "04e29a936a96514537bf694bb45165108d13d68d4c01e0bbe650d0d14403fb2c", + "result_file_sha256": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "raw_content_sha256": "3a77159e81f930d16df465bba0c47780df20a0d96e1d28d3144d74065e93efb7", + "response_id": "20261014_56540249-de93-46e1-b127-6b4b598dc972", + "receipt_status": 200, + "prompt_tokens": 1474, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.0191509160067653 + }, + "grouped_lossless": { + "id": "confirm-fresh-22-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-23\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "fdfa3d8f7d647dcfef6fa9f8768ad23f19dfe5c8b90119b71401a5eabccfd2c2", + "request_file_sha256": "f01b1dbf8ee399c5852135441d81f86b14291b75988f65d05ae09bbee0829263", + "receipt_file_sha256": "054387af058bacb123a24309fac35d351f23ab2b94b703080fa43e029061dd9f", + "receipt_body_text_sha256": "2aecd03cadd0ea3d83e987a6e6d07f2d434d606cf8a77b7fb49170c2058e4f07", + "result_file_sha256": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "raw_content_sha256": "3a77159e81f930d16df465bba0c47780df20a0d96e1d28d3144d74065e93efb7", + "response_id": "20261014_5ff8be5a-3c2d-42b0-bd72-200d3c39c5c5", + "receipt_status": 200, + "prompt_tokens": 2123, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.9548690419906052 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-22-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-23\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-23", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "fdfa3d8f7d647dcfef6fa9f8768ad23f19dfe5c8b90119b71401a5eabccfd2c2", + "request_file_sha256": "f01b1dbf8ee399c5852135441d81f86b14291b75988f65d05ae09bbee0829263", + "receipt_file_sha256": "4a9c3f9e799f94ad422c58b6481b4d63258282c328dcbf3712be899e1967a665", + "receipt_body_text_sha256": "c58c366238fd6baf16b1dc7ebc858872b9a03823018b1d113ce0824bb4b10862", + "result_file_sha256": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "raw_content_sha256": "3a77159e81f930d16df465bba0c47780df20a0d96e1d28d3144d74065e93efb7", + "response_id": "20261014_18fe65b6-3033-4400-8249-1808db1950fb", + "receipt_status": 200, + "prompt_tokens": 2123, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.9699667910026619 + } + } + }, + { + "history_id": "confirm-fresh-23-te-m0", + "case_id": "fresh-23-te", + "scenario": "fresh-23", + "model": "sarvam-105b", + "family": "approved_change", + "question": "VISIT-2123 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-23-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-23-1", + "role": "customer", + "content": "VISIT-2123ని 2027-04-24కి మార్చండి." + }, + { + "id": "note-23-2", + "role": "customer", + "content": "ఇచ్చిన అభ్యర్థన గుర్తింపు, సర్వర్ ఇచ్చిన సంబంధిత అనుమతి వాడండి. అభ్యర్థనను పరిశీలించిన తర్వాత ఖరారైన తేదీ చెప్పండి." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The native trace has a matching server-authorized confirmed update for VISIT-2123; the scoped final backend date is 2027-04-24. The question asks that actual current date, not only the earlier request.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "confirmed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-23", + "id": "VISIT-2123", + "date": "2027-03-24", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-23", + "id": "VISIT-2123", + "date": "2027-04-24", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a15123130e2dddc8", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2123", + "approval_id": "AUTH-23-GRANT-16", + "date": "2027-04-24", + "expected_version": 5, + "idempotency_key": "REQUEST-23-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-24", + "version": 6 + } + }, + { + "call_id": "chatcmpl-tool-91d2de4dd68100c4", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2123" + }, + "response": { + "date": "2027-04-24", + "version": 6, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "472ba5ef0ea7458c923a43e7aa229319c1c4ea70cf2671d1acf2379021be85c4", + "case_sha256": "6c64709cc33d4bb89639fa9a539ce5c9479090468d535d4c3e2c612e28bf9f34", + "gold_label_sha256": "e5050ab2c276bb8c4133fb7465884986f51f911497c8ed7916c80af42deaba52", + "reports": { + "chronological": { + "id": "confirm-fresh-23-te-m0-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-24\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "c30a8ce61d16f591dd4100614bddb86ccfe9ddf39273bc525a68dca977928304", + "request_file_sha256": "e18f0f1e61ddd61aa19155f016521604b74ea934c6b5c194c1f65f1c4656254c", + "receipt_file_sha256": "54c63ac29c0e99630dcf2d117209ab9382197a7492387f96c43025d435c1ad75", + "receipt_body_text_sha256": "ea893a41f60b4b08661aaf5f2131b4ee14e01723dd5669d0915f8e8a209e04ea", + "result_file_sha256": "0faf0e0192f0d7377eabd553b64abae59ded0e43cc4773b8a88de2b6c1adaeed", + "raw_content_sha256": "42e02e70a4fb69a05a1d57eab32d4ec5ccae0fd8c745b2c7d0f29e7d8ef48bb6", + "response_id": "20261014_34a85df7-c0a5-4b35-9827-bf528ad4a200", + "receipt_status": 200, + "prompt_tokens": 1482, + "completion_tokens": 1336, + "reasoning_tokens": 1306, + "elapsed_seconds": 7.541762499997276 + }, + "grouped_lossless": { + "id": "confirm-fresh-23-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "incomplete", + "finish_reason": "length", + "raw_content": null, + "raw_output": null, + "expected_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure.", + "status_review": "No valid raw action status because the final is incomplete; counted incorrect.", + "request_sha256": "f89b0ae5687a43f5585f5e60be4c94f5d80bdbb7833abe8709a48518db87e1b1", + "request_file_sha256": "2f9961ffca631adccde72aad1c2748c2965df6dd42798a35cdd737b80dc7fa33", + "receipt_file_sha256": "007f8a5314ae251378422c05280880ac5290b3730e461de751e679075ed9cbec", + "receipt_body_text_sha256": "3ffb35a3621652b4b06cfee8149cfb4baeb1161921d4c851c945dba5ca40c41b", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "raw_content_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "response_id": "20261014_fe4d277c-c19b-46f2-934a-8064ca923aad", + "receipt_status": 200, + "prompt_tokens": 2175, + "completion_tokens": 8192, + "reasoning_tokens": 8191, + "elapsed_seconds": 38.70535054200445 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-23-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "incomplete", + "finish_reason": "length", + "raw_content": "{\n\"action_status\": \"confirmed\" \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n", + "raw_output": null, + "expected_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure.", + "status_review": "No valid raw action status because the final is incomplete; counted incorrect.", + "request_sha256": "f89b0ae5687a43f5585f5e60be4c94f5d80bdbb7833abe8709a48518db87e1b1", + "request_file_sha256": "2f9961ffca631adccde72aad1c2748c2965df6dd42798a35cdd737b80dc7fa33", + "receipt_file_sha256": "f2c603c0534b3b92b1277dc81ed883527dbe6ac805f62feb062cba49b5a0d1c2", + "receipt_body_text_sha256": "7bd457e7c4bc0a646f0a49c56cd1a3112d1f6e2f553e0160f9a08f5a1f1934df", + "result_file_sha256": "5e07ecfb62910b460fef6f452f1b4cf8e3705251a409fb26d17c729bb1105462", + "raw_content_sha256": "7906ce5d89fbe60012b7b0dc72b1c11586a3529936de41abf7026d0c4b624947", + "response_id": "20261014_e38c6d3d-6a31-40cf-b894-badcfb4d9903", + "receipt_status": 200, + "prompt_tokens": 2175, + "completion_tokens": 8192, + "reasoning_tokens": 5698, + "elapsed_seconds": 34.90903183299815 + } + } + }, + { + "history_id": "confirm-fresh-23-te-m1", + "case_id": "fresh-23-te", + "scenario": "fresh-23", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "question": "VISIT-2123 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-23-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-23-1", + "role": "customer", + "content": "VISIT-2123ని 2027-04-24కి మార్చండి." + }, + { + "id": "note-23-2", + "role": "customer", + "content": "ఇచ్చిన అభ్యర్థన గుర్తింపు, సర్వర్ ఇచ్చిన సంబంధిత అనుమతి వాడండి. అభ్యర్థనను పరిశీలించిన తర్వాత ఖరారైన తేదీ చెప్పండి." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The native trace has a matching server-authorized confirmed update for VISIT-2123; the scoped final backend date is 2027-04-24. The question asks that actual current date, not only the earlier request.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "confirmed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-23", + "id": "VISIT-2123", + "date": "2027-03-24", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-23", + "id": "VISIT-2123", + "date": "2027-04-24", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a8da3db61415f8b2", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2123" + }, + "response": { + "date": "2027-03-24", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-8ad36fd00ce9e674", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2123", + "approval_id": "AUTH-23-GRANT-16", + "date": "2027-04-24", + "expected_version": 5, + "idempotency_key": "REQUEST-23-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-24", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "8e918654cdd40efdccf0304f1e69959b4f657d3c684faaa47ab96f058f432b0e", + "case_sha256": "6c64709cc33d4bb89639fa9a539ce5c9479090468d535d4c3e2c612e28bf9f34", + "gold_label_sha256": "e5050ab2c276bb8c4133fb7465884986f51f911497c8ed7916c80af42deaba52", + "reports": { + "chronological": { + "id": "confirm-fresh-23-te-m1-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-24\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "ecd1f1291be7ba840511e55caf68ca494bf97cb2082dcaf4528221c1c174e01d", + "request_file_sha256": "acee370598326334c1358e04786f9b7ca5a3338419c1906ad56413cd8ba563ec", + "receipt_file_sha256": "5962c7948fbe12adcf95d28e83c42894e78bcf68c45fcf89021fc325064b8995", + "receipt_body_text_sha256": "15d3201c8d1217274ede5553b80b8376616eabc11b6871e3c7fabd667d92577f", + "result_file_sha256": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "raw_content_sha256": "2693a0032cdec8d9e8a477c15467e101f2a566f73bf63c4cb05aff66d210cfdc", + "response_id": "20261014_60206c29-9c69-4c34-8e1e-1a77925572da", + "receipt_status": 200, + "prompt_tokens": 1510, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.9490920000098413 + }, + "grouped_lossless": { + "id": "confirm-fresh-23-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-24\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "897fc7033026e598f7c8a4651811055d23ed7e6179da00cb77cce7a2392c672f", + "request_file_sha256": "652cdf5af5b6e89d1a2e03389ad8f8d00972aedebb9cf0a1ee4563ae637ac505", + "receipt_file_sha256": "bc3497ca624a1b25517ef88673ca34648b5716b00c3fccded6535258efac429a", + "receipt_body_text_sha256": "c9a4ec8b6ba2071b9e45bf697259fbee683121ff8995968be5167ea63abc9986", + "result_file_sha256": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "raw_content_sha256": "2693a0032cdec8d9e8a477c15467e101f2a566f73bf63c4cb05aff66d210cfdc", + "response_id": "20261014_ab3615ee-b561-4889-bec7-c5db5bc49cb4", + "receipt_status": 200, + "prompt_tokens": 2279, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.973554416996194 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-23-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-24\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-24", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "897fc7033026e598f7c8a4651811055d23ed7e6179da00cb77cce7a2392c672f", + "request_file_sha256": "652cdf5af5b6e89d1a2e03389ad8f8d00972aedebb9cf0a1ee4563ae637ac505", + "receipt_file_sha256": "17b1463ef99ae956aa6a8950e7d513d76ba38d1a993b468ebe6da3f75ba567f2", + "receipt_body_text_sha256": "2d9ed9eb899bfe63986bd9c30cdd64f50f29621d13a241b0e80d29e1487fcf73", + "result_file_sha256": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "raw_content_sha256": "2693a0032cdec8d9e8a477c15467e101f2a566f73bf63c4cb05aff66d210cfdc", + "response_id": "20261014_4f091a4e-2c3d-4b99-9c87-d4a8cc1b5cec", + "receipt_status": 200, + "prompt_tokens": 2279, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.0796093750104774 + } + } + }, + { + "history_id": "confirm-fresh-24-te-m0", + "case_id": "fresh-24-te", + "scenario": "fresh-24", + "model": "sarvam-105b", + "family": "approved_change", + "question": "VISIT-2124 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-24-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. VISIT-2124ని 2027-04-25కి మార్చండి. ఇచ్చిన అభ్యర్థన గుర్తింపు, సర్వర్ ఇచ్చిన సంబంధిత అనుమతి వాడండి. అభ్యర్థనను పరిశీలించిన తర్వాత ఖరారైన తేదీ చెప్పండి." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The native trace has a matching server-authorized confirmed update for VISIT-2124; the scoped final backend date is 2027-04-25. The question asks that actual current date, not only the earlier request.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "confirmed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-24", + "id": "VISIT-2124", + "date": "2027-03-25", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-24", + "id": "VISIT-2124", + "date": "2027-04-25", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a03210b3b82dc022", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2124", + "approval_id": "AUTH-24-GRANT-16", + "date": "2027-04-25", + "expected_version": 5, + "idempotency_key": "REQUEST-24-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-25", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "9a8791569564b796c42878a05801f082be380878474a02ca518a81e3f8d48d9c", + "case_sha256": "448280dec6227b15553fde768e7d910b21fb7ec0b02aeb6f21f1a83b8b945f55", + "gold_label_sha256": "433c78a6c86d24060445514a4d7ba07950eacb160ce5e9f0c94ad8d36e6c14cf", + "reports": { + "chronological": { + "id": "confirm-fresh-24-te-m0-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-25\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "86cee24641d3374f621b3f385565fb28767bab7308e52e1a616951d2e50566bf", + "request_file_sha256": "bc25d72a7f5d3e2d18339334d93df72cd7bfd0e5cce470091b90821d7022e4a0", + "receipt_file_sha256": "b7943281005fa46518d250fef94937b6e985a9abd37a9e5e3acbf2bb77332d50", + "receipt_body_text_sha256": "401894985ecd57b283bbad8b83648b990d249bdb35503653265444a5c48db889", + "result_file_sha256": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "raw_content_sha256": "5f7884de70bae39ff454edda1e2dfc29c17845b488e477350cf0a7cc27c93739", + "response_id": "20261014_e8731c4b-0db3-40db-a3cc-b99bfc86b0e3", + "receipt_status": 200, + "prompt_tokens": 1334, + "completion_tokens": 1047, + "reasoning_tokens": 1017, + "elapsed_seconds": 6.182483416996547 + }, + "grouped_lossless": { + "id": "confirm-fresh-24-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-25\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "a06a8f29ac500ce57bfc0449fd2b5f6bdea7afa1ef405e5dc2f7c0bb5d1ea541", + "request_file_sha256": "02d7eb4b6caf628655e4ca2d2e4f1f1b8cda51e2298e81f28486bcd0b4a85f32", + "receipt_file_sha256": "ed7f7d767611beeede137ac36ded8b03ff5095dcd63035c2090418b4fbf3ed7e", + "receipt_body_text_sha256": "e2ed2b8b304bdf0ab8b2ce5a5edec199b69e598acbe2a9294f20dd81235559ed", + "result_file_sha256": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "raw_content_sha256": "5f7884de70bae39ff454edda1e2dfc29c17845b488e477350cf0a7cc27c93739", + "response_id": "20261014_220a28f8-7f32-4f9e-99a8-3fc2ed564858", + "receipt_status": 200, + "prompt_tokens": 1833, + "completion_tokens": 3206, + "reasoning_tokens": 3175, + "elapsed_seconds": 16.238885291997576 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-24-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-25\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "a06a8f29ac500ce57bfc0449fd2b5f6bdea7afa1ef405e5dc2f7c0bb5d1ea541", + "request_file_sha256": "02d7eb4b6caf628655e4ca2d2e4f1f1b8cda51e2298e81f28486bcd0b4a85f32", + "receipt_file_sha256": "9df7251b0301dfdf32850db6aeab8552dc793bce4e529abc3e481aa1925ed922", + "receipt_body_text_sha256": "bf194957380b34b15a0ee773af05a2338466e5c0f88eb34a94b7a1586dd7a656", + "result_file_sha256": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "raw_content_sha256": "5f7884de70bae39ff454edda1e2dfc29c17845b488e477350cf0a7cc27c93739", + "response_id": "20261014_4f3de027-2f76-411f-9488-97e761c1d2a7", + "receipt_status": 200, + "prompt_tokens": 1833, + "completion_tokens": 5393, + "reasoning_tokens": 5363, + "elapsed_seconds": 26.072659749988816 + } + } + }, + { + "history_id": "confirm-fresh-24-te-m1", + "case_id": "fresh-24-te", + "scenario": "fresh-24", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "question": "VISIT-2124 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-24-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. VISIT-2124ని 2027-04-25కి మార్చండి. ఇచ్చిన అభ్యర్థన గుర్తింపు, సర్వర్ ఇచ్చిన సంబంధిత అనుమతి వాడండి. అభ్యర్థనను పరిశీలించిన తర్వాత ఖరారైన తేదీ చెప్పండి." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The native trace has a matching server-authorized confirmed update for VISIT-2124; the scoped final backend date is 2027-04-25. The question asks that actual current date, not only the earlier request.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "confirmed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-24", + "id": "VISIT-2124", + "date": "2027-03-25", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-24", + "id": "VISIT-2124", + "date": "2027-04-25", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-abe90241972c557e", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2124" + }, + "response": { + "date": "2027-03-25", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-92db2a62d590ee38", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2124", + "approval_id": "AUTH-24-GRANT-16", + "date": "2027-04-25", + "expected_version": 5, + "idempotency_key": "REQUEST-24-CHANGE-16" + }, + "response": { + "status": "confirmed", + "date": "2027-04-25", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "26e3cdc8ea6a01b1aafb83de22f217b769c062baf978960a3018fece1fe1f89a", + "case_sha256": "448280dec6227b15553fde768e7d910b21fb7ec0b02aeb6f21f1a83b8b945f55", + "gold_label_sha256": "433c78a6c86d24060445514a4d7ba07950eacb160ce5e9f0c94ad8d36e6c14cf", + "reports": { + "chronological": { + "id": "confirm-fresh-24-te-m1-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-25\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "c3248114ccb5f5b491f6e433f8b74f06d50a2b5ade4c88aa92bbc50fff405543", + "request_file_sha256": "50146f29aa47560b536cfe91cec6502b186899ef5002456b48a7c85ef4621a93", + "receipt_file_sha256": "0812d7d2958f7cc98b663a1ccafa78db5a86eb69e0a21cad13f2e658535009b4", + "receipt_body_text_sha256": "284193b3dc5d2817f50335856ac105491f5fee13cd38cbac604bcc298268e9d9", + "result_file_sha256": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "raw_content_sha256": "4913fc937b2d32f3dd8205703eb9608e0029f25eba1c8e31072557a2baa36b41", + "response_id": "20261014_4aa4e0b9-8693-4e8a-b4ff-b005837f1840", + "receipt_status": 200, + "prompt_tokens": 1472, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.0220702090009581 + }, + "grouped_lossless": { + "id": "confirm-fresh-24-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-25\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "93c9d8ed2d67e31af5cb309b96a387436db69c9142edffa659827923dd1d8b34", + "request_file_sha256": "4e15d25b78928d6eeb791dabfb55508f6c4cb1bd1ce348690d30c720ef224823", + "receipt_file_sha256": "5d2409eb98170d75d89d69faf18bdc4c91ab946fc45c20beefbb5578a5d8813a", + "receipt_body_text_sha256": "2d84b269b509492bcba2423a3a23b16b1728d252d1b4e0e39f2a7bd16e9d40e7", + "result_file_sha256": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "raw_content_sha256": "4913fc937b2d32f3dd8205703eb9608e0029f25eba1c8e31072557a2baa36b41", + "response_id": "20261014_a6759f17-f385-4bbd-8e27-7d139b7f01ea", + "receipt_status": 200, + "prompt_tokens": 2120, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.0569155420089373 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-24-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-25\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-25", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "93c9d8ed2d67e31af5cb309b96a387436db69c9142edffa659827923dd1d8b34", + "request_file_sha256": "4e15d25b78928d6eeb791dabfb55508f6c4cb1bd1ce348690d30c720ef224823", + "receipt_file_sha256": "85587eb959c375877ce60b1e413501a61c342ffef5967dd4f651d1e77c30fe96", + "receipt_body_text_sha256": "972907dc8d58824cffa68482006d41c008e418e0b9d6ac598fa49b1ddc9b7f3c", + "result_file_sha256": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "raw_content_sha256": "4913fc937b2d32f3dd8205703eb9608e0029f25eba1c8e31072557a2baa36b41", + "response_id": "20261014_72e15f68-4746-4370-90d9-1293103aa57e", + "receipt_status": 200, + "prompt_tokens": 2120, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.9508562080009142 + } + } + }, + { + "history_id": "confirm-fresh-25-te-m0", + "case_id": "fresh-25-te", + "scenario": "fresh-25", + "model": "sarvam-105b", + "family": "approval_boundary", + "question": "VISIT-2125 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-25-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-25-1", + "role": "customer", + "content": "VISIT-2125ని 2027-04-26కి మార్చే అభ్యర్థనను పరిశీలించండి." + }, + { + "id": "note-25-2", + "role": "customer", + "content": "ఇచ్చిన సర్వర్ సమయానికి అనుమతి చెల్లుతుందో చూడండి. ఆ స్థిర సమయం తర్వాత ఒక సెకనుకు అది ముగుస్తుంది." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The grant is valid at the fixed server instant and expires one second later; the trace confirms the update for VISIT-2125. The scoped final backend date is 2027-04-26.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "confirmed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-25", + "id": "VISIT-2125", + "date": "2027-03-26", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-25", + "id": "VISIT-2125", + "date": "2027-04-26", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a1d8ab069a0a5af8", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2125", + "approval_id": "AUTH-25-GRANT-19", + "date": "2027-04-26", + "expected_version": 5, + "idempotency_key": "REQUEST-25-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-26", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "8164a13385db57b850010af782a33a3102e4c55b44b247c25a2a5e9458244b50", + "case_sha256": "236c0c96aa42519c77d85bed38d0e91f2241095beb3b0eeb9c8de3082b51c1b3", + "gold_label_sha256": "fb7a9658db2ed90120f725e1218792c4224ea1f7c4d84d75bfadd0ee19632581", + "reports": { + "chronological": { + "id": "confirm-fresh-25-te-m0-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-26\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "dc191cc1200bc5090b1e63e266a1a12cbbec44c9c39a5ccc77c43e9ed1f6c42f", + "request_file_sha256": "aab198012cedf5a127b59c78b34226a23ea6fbfd756b7887d89574b635e79d41", + "receipt_file_sha256": "712127c802ba132cedc14d184828e8de6536291a09f52f38799d063a5a0bd046", + "receipt_body_text_sha256": "e87492b3aa6c029646e9e467c7f50c221796d06ad38c0f38d0bf14c5aca85bc6", + "result_file_sha256": "258b73b129b801b662376d5cb0f39961843f2127df41664b4c2a54a8e81537fa", + "raw_content_sha256": "bd59ff2841c464354faf7faa7146bbbf8c2dd82eef7abb3e9321e9a86bc87da6", + "response_id": "20261014_019245e2-cafb-4de9-b96b-452300c1b744", + "receipt_status": 200, + "prompt_tokens": 1375, + "completion_tokens": 1111, + "reasoning_tokens": 1081, + "elapsed_seconds": 6.427784958999837 + }, + "grouped_lossless": { + "id": "confirm-fresh-25-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-26\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "b28268840d184524b7c875c94a0b0cab0d6e85da32c5b1e57cf4855124444b65", + "request_file_sha256": "30430c94fb13ba508e33d5a0792a78b15a20fb8166c10ce50ee60c8e4ee0c6bf", + "receipt_file_sha256": "026d645cd9eda92109052cce3aed52a540a867c13846490acd7f205ce049d9d5", + "receipt_body_text_sha256": "b5b57d34f8a44d4b6fe0af6c401b5e3989c645923b371ab784041bfe44440f44", + "result_file_sha256": "258b73b129b801b662376d5cb0f39961843f2127df41664b4c2a54a8e81537fa", + "raw_content_sha256": "bd59ff2841c464354faf7faa7146bbbf8c2dd82eef7abb3e9321e9a86bc87da6", + "response_id": "20261014_889f0d90-a2e7-43f1-abf5-482f476ece1c", + "receipt_status": 200, + "prompt_tokens": 1985, + "completion_tokens": 5158, + "reasoning_tokens": 5128, + "elapsed_seconds": 25.215050832994166 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-25-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "incomplete", + "finish_reason": "length", + "raw_content": null, + "raw_output": null, + "expected_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure.", + "status_review": "No valid raw action status because the final is incomplete; counted incorrect.", + "request_sha256": "b28268840d184524b7c875c94a0b0cab0d6e85da32c5b1e57cf4855124444b65", + "request_file_sha256": "30430c94fb13ba508e33d5a0792a78b15a20fb8166c10ce50ee60c8e4ee0c6bf", + "receipt_file_sha256": "81fa95b9ee9e29f8ff19575a0a125413415834f09289ac2673c28e95e9b2d3fc", + "receipt_body_text_sha256": "1cdb8ead394c6c4338cb855eba5899166e500a06f6bf741f0947b327e8c5d38c", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "raw_content_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "response_id": "20261014_6126a20a-fca3-4421-bbb5-1c3b0e6b02dc", + "receipt_status": 200, + "prompt_tokens": 1985, + "completion_tokens": 8192, + "reasoning_tokens": 8191, + "elapsed_seconds": 39.421729541005334 + } + } + }, + { + "history_id": "confirm-fresh-25-te-m1", + "case_id": "fresh-25-te", + "scenario": "fresh-25", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "question": "VISIT-2125 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-25-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-25-1", + "role": "customer", + "content": "VISIT-2125ని 2027-04-26కి మార్చే అభ్యర్థనను పరిశీలించండి." + }, + { + "id": "note-25-2", + "role": "customer", + "content": "ఇచ్చిన సర్వర్ సమయానికి అనుమతి చెల్లుతుందో చూడండి. ఆ స్థిర సమయం తర్వాత ఒక సెకనుకు అది ముగుస్తుంది." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The grant is valid at the fixed server instant and expires one second later; the trace confirms the update for VISIT-2125. The scoped final backend date is 2027-04-26.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "confirmed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-25", + "id": "VISIT-2125", + "date": "2027-03-26", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-25", + "id": "VISIT-2125", + "date": "2027-04-26", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9effddfd72d09184", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2125" + }, + "response": { + "date": "2027-03-26", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-9b0ed4a4af640b8a", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2125", + "approval_id": "AUTH-25-GRANT-19", + "date": "2027-04-26", + "expected_version": 5, + "idempotency_key": "REQUEST-25-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-26", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "34ccefae3e3903722517237cefebfcec74774539188d2c9181b966cee4f574d1", + "case_sha256": "236c0c96aa42519c77d85bed38d0e91f2241095beb3b0eeb9c8de3082b51c1b3", + "gold_label_sha256": "fb7a9658db2ed90120f725e1218792c4224ea1f7c4d84d75bfadd0ee19632581", + "reports": { + "chronological": { + "id": "confirm-fresh-25-te-m1-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-26\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "b5afc0a6d6d54cac8b67fad7fc30cc0b060d2076fa0d1c5d8016f4f412748208", + "request_file_sha256": "70116231fff9f40a75db0d2824eb38e6c9e370cde62197036c28e8be8ee6e10b", + "receipt_file_sha256": "f6b78be423d9b9da47209f2bb2378a0083cddf34ec0bcd1fc534f3e4d6fef1eb", + "receipt_body_text_sha256": "4b0b8c7479a7e4ac376ebf65f7b09eee3c935f400d5ff82426c8b3c574c5eb2a", + "result_file_sha256": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "raw_content_sha256": "b67eb49d345d551277e52d9817ebc266663e8242554bf45e8cec67eff31a99b9", + "response_id": "20261014_4bc5dc98-891d-4897-921d-156792f88138", + "receipt_status": 200, + "prompt_tokens": 1511, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.00234779099992 + }, + "grouped_lossless": { + "id": "confirm-fresh-25-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-26\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "d543d1b12189eb17d8a381a54a764fbadf416831f2237abba56bd97bc74f9219", + "request_file_sha256": "64438c22d4b8eb9124e8a8d286c1e539e0fdfd14c725baf7f10498268a86f50e", + "receipt_file_sha256": "97eb2c2941734115e5cf530f4660405b740db85f442255aea2652ae1263ae0bf", + "receipt_body_text_sha256": "0df1bb0d88e594b90e58ff5d22b393b76bc84b3362752a2193ef605e85b4b1aa", + "result_file_sha256": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "raw_content_sha256": "b67eb49d345d551277e52d9817ebc266663e8242554bf45e8cec67eff31a99b9", + "response_id": "20261014_0858baba-9920-4d27-b44d-d95d3465ed94", + "receipt_status": 200, + "prompt_tokens": 2273, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.9434035830054199 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-25-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-26\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-26", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "d543d1b12189eb17d8a381a54a764fbadf416831f2237abba56bd97bc74f9219", + "request_file_sha256": "64438c22d4b8eb9124e8a8d286c1e539e0fdfd14c725baf7f10498268a86f50e", + "receipt_file_sha256": "de3ff9cb955ec219196fd9ce1cb89d3f0cf2ec9b7446c9181af5e13553aa5249", + "receipt_body_text_sha256": "9327bf98f5149056935b1a9ecf43e8481f7b52ec99f9b7b8438ec5c8c68d4ab0", + "result_file_sha256": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "raw_content_sha256": "b67eb49d345d551277e52d9817ebc266663e8242554bf45e8cec67eff31a99b9", + "response_id": "20261014_b4689a24-7fdd-44a6-82fd-8617dbe81fcb", + "receipt_status": 200, + "prompt_tokens": 2273, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.0558867090003332 + } + } + }, + { + "history_id": "confirm-fresh-26-te-m0", + "case_id": "fresh-26-te", + "scenario": "fresh-26", + "model": "sarvam-105b", + "family": "approval_boundary", + "question": "VISIT-2126 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-26-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. VISIT-2126ని 2027-04-27కి మార్చే అభ్యర్థనను పరిశీలించండి. ఇచ్చిన సర్వర్ సమయానికి అనుమతి చెల్లుతుందో చూడండి. ఆ స్థిర సమయం తర్వాత ఒక సెకనుకు అది ముగుస్తుంది." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The grant is valid at the fixed server instant and expires one second later; the trace confirms the update for VISIT-2126. The scoped final backend date is 2027-04-27.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "confirmed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-26", + "id": "VISIT-2126", + "date": "2027-03-27", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-26", + "id": "VISIT-2126", + "date": "2027-04-27", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a64d9eecfca3ffbb", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2126", + "approval_id": "AUTH-26-GRANT-19", + "date": "2027-04-27", + "expected_version": 5, + "idempotency_key": "REQUEST-26-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-27", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "a245e1c3c2721d64e0e94ad0119877467a0551587bdbeec2663076514a3fba39", + "case_sha256": "9112928eccafbc3d94a22c844b46e5029fb7ab427fec3c441abff633754fe302", + "gold_label_sha256": "cb3699d385b184f348bf6aa0cf31d6a735c45598f3e10db7b8fc427d4e9847ef", + "reports": { + "chronological": { + "id": "confirm-fresh-26-te-m0-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-27\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "afb84111d7fc8a941bc22a11fb3bc69f7ad03915bdf294d3c6df020a44f81064", + "request_file_sha256": "a65cad85c8509b0bac58f34902ea5eb594964c442cb094dc5b4c783daa9cb896", + "receipt_file_sha256": "25227c8b90ad47ac9b118db15cbb23e9762bb821d491ba0098cdc8740a4e9af2", + "receipt_body_text_sha256": "b5917c2ce5d6f4a59c05eb77c07cd76f3645060c9a5293db419227e51cfe5a76", + "result_file_sha256": "fb6e951f5e2bc3a3ec400cf4d93a3dfb2910e24d4ec7d9480d665ad63036999a", + "raw_content_sha256": "c1e910975c3325f664d8f29ff0ec33c8158dfa51c312012dff2adc0d4cc8071a", + "response_id": "20261014_bb7c0250-03c4-4bc2-ba70-01ec056a7c2f", + "receipt_status": 200, + "prompt_tokens": 1333, + "completion_tokens": 961, + "reasoning_tokens": 931, + "elapsed_seconds": 5.778320624987828 + }, + "grouped_lossless": { + "id": "confirm-fresh-26-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "incomplete", + "finish_reason": "length", + "raw_content": null, + "raw_output": null, + "expected_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure.", + "status_review": "No valid raw action status because the final is incomplete; counted incorrect.", + "request_sha256": "a28c066a8c640eea3b84ed46ca9097f1c58a33e87dab1317eae4804add24d53b", + "request_file_sha256": "481c5cfa2d725e730a011fff19c7e7be7008163c421931eb110a1e618620ba2d", + "receipt_file_sha256": "b9eb8ecf3672423823e01c23ea523d36abe1dbf09a0315e56d31822f25fab596", + "receipt_body_text_sha256": "8751878879ad6ebe84e9535bf7d7fcb591123f3d7029c5860ac7146b109659b0", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "raw_content_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "response_id": "20261014_a5e45e2c-d725-4039-a51d-db82f5fa3d80", + "receipt_status": 200, + "prompt_tokens": 1836, + "completion_tokens": 8192, + "reasoning_tokens": 8191, + "elapsed_seconds": 33.65626324999903 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-26-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-27\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "a28c066a8c640eea3b84ed46ca9097f1c58a33e87dab1317eae4804add24d53b", + "request_file_sha256": "481c5cfa2d725e730a011fff19c7e7be7008163c421931eb110a1e618620ba2d", + "receipt_file_sha256": "dfd3a1bb4c304a1431e97541734f289877416e0e6d8270d9384f2a4a3eb2a815", + "receipt_body_text_sha256": "d777abbadc945a7e6f2d5438e83e5fabd8b1f16872ab15863f05ae5b668727ff", + "result_file_sha256": "fb6e951f5e2bc3a3ec400cf4d93a3dfb2910e24d4ec7d9480d665ad63036999a", + "raw_content_sha256": "c1e910975c3325f664d8f29ff0ec33c8158dfa51c312012dff2adc0d4cc8071a", + "response_id": "20261014_d6acc0ac-46d9-44c6-a549-d8de37c31bf5", + "receipt_status": 200, + "prompt_tokens": 1836, + "completion_tokens": 1683, + "reasoning_tokens": 1653, + "elapsed_seconds": 7.600142416995368 + } + } + }, + { + "history_id": "confirm-fresh-26-te-m1", + "case_id": "fresh-26-te", + "scenario": "fresh-26", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "question": "VISIT-2126 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-26-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. VISIT-2126ని 2027-04-27కి మార్చే అభ్యర్థనను పరిశీలించండి. ఇచ్చిన సర్వర్ సమయానికి అనుమతి చెల్లుతుందో చూడండి. ఆ స్థిర సమయం తర్వాత ఒక సెకనుకు అది ముగుస్తుంది." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The grant is valid at the fixed server instant and expires one second later; the trace confirms the update for VISIT-2126. The scoped final backend date is 2027-04-27.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "confirmed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-26", + "id": "VISIT-2126", + "date": "2027-03-27", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-26", + "id": "VISIT-2126", + "date": "2027-04-27", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-bb39b1cde7b53438", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2126" + }, + "response": { + "date": "2027-03-27", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-b1a62c49518937a9", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2126", + "approval_id": "AUTH-26-GRANT-19", + "date": "2027-04-27", + "expected_version": 5, + "idempotency_key": "REQUEST-26-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-27", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "3a201c1f249aa3a02a61d9943329e935f5bfeb2076ed8d8f9e6e58f93ee9367b", + "case_sha256": "9112928eccafbc3d94a22c844b46e5029fb7ab427fec3c441abff633754fe302", + "gold_label_sha256": "cb3699d385b184f348bf6aa0cf31d6a735c45598f3e10db7b8fc427d4e9847ef", + "reports": { + "chronological": { + "id": "confirm-fresh-26-te-m1-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-27\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "f8f7e48aede79e56d343d6d44eba2f0bbd4c2e741ce25b0a5d758ffa38cb48ce", + "request_file_sha256": "118dcb447e3ffc5e62938cf58e84809333f1a8749f83656272e7a3df6781779b", + "receipt_file_sha256": "d452d9585a54192824e9f752069fdaab954cf586d590e331efe426d555e98add", + "receipt_body_text_sha256": "7dba10951e4a8af944b159c1b6103e9f85cd819f3a5783601a58a9179ea5bf6c", + "result_file_sha256": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "raw_content_sha256": "fa5d023d08c3ca91b18204d2fb9daa32d70361a98f2d49b310618c10718bd925", + "response_id": "20261014_bd565aba-cdfc-47d2-bcd7-9a3c5dc02d5e", + "receipt_status": 200, + "prompt_tokens": 1481, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.945021417006501 + }, + "grouped_lossless": { + "id": "confirm-fresh-26-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-27\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "e8a8dfd3b56a09ded345df1ca3be441e1eaa488e1f202eb2a2698719f58aa556", + "request_file_sha256": "788b00abe0b74df821e9f261602c5b894f92e71bc1cd95e6b86184f21fc1cc0b", + "receipt_file_sha256": "27fefb6dead6552ca485e69738d9b96efc3c74ac146d82727f06073c20b300cd", + "receipt_body_text_sha256": "090cf6e8dfc8cc3ff674be290c082da63fb74956215c0fe39b339ced4a2e1c6c", + "result_file_sha256": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "raw_content_sha256": "fa5d023d08c3ca91b18204d2fb9daa32d70361a98f2d49b310618c10718bd925", + "response_id": "20261014_168ecdb3-4759-47ee-8bdb-939779360333", + "receipt_status": 200, + "prompt_tokens": 2123, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.1136486670002341 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-26-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-27\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-27", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "e8a8dfd3b56a09ded345df1ca3be441e1eaa488e1f202eb2a2698719f58aa556", + "request_file_sha256": "788b00abe0b74df821e9f261602c5b894f92e71bc1cd95e6b86184f21fc1cc0b", + "receipt_file_sha256": "bef52fc576a47cb54474cb17bbeabee51e2e0dadf808206856a79cbcc2578922", + "receipt_body_text_sha256": "3be3368266bf0673906b0beaacfa3364adcc3ff4d86e39b535e812f50d0ecea7", + "result_file_sha256": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "raw_content_sha256": "fa5d023d08c3ca91b18204d2fb9daa32d70361a98f2d49b310618c10718bd925", + "response_id": "20261014_eb6cedaa-d89e-435c-b3b0-b460228b1b77", + "receipt_status": 200, + "prompt_tokens": 2123, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.0787862920115003 + } + } + }, + { + "history_id": "confirm-fresh-27-te-m0", + "case_id": "fresh-27-te", + "scenario": "fresh-27", + "model": "sarvam-105b", + "family": "approval_boundary", + "question": "VISIT-2127 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-27-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-27-1", + "role": "customer", + "content": "VISIT-2127ని 2027-04-28కి మార్చే అభ్యర్థనను పరిశీలించండి." + }, + { + "id": "note-27-2", + "role": "customer", + "content": "ఇచ్చిన సర్వర్ సమయానికి అనుమతి చెల్లుతుందో చూడండి. ఆ స్థిర సమయం తర్వాత ఒక సెకనుకు అది ముగుస్తుంది." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The grant is valid at the fixed server instant and expires one second later; the trace confirms the update for VISIT-2127. The scoped final backend date is 2027-04-28.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "confirmed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-27", + "id": "VISIT-2127", + "date": "2027-03-28", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-27", + "id": "VISIT-2127", + "date": "2027-04-28", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-95846bff900c0e89", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2127", + "approval_id": "AUTH-27-GRANT-19", + "date": "2027-04-28", + "expected_version": 5, + "idempotency_key": "REQUEST-27-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-28", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "85b9b65bed2dcb9016db5cbbc83cc2749bc6b080078851654663adc7555184d0", + "case_sha256": "da4691577630e571639c487eb08a26158a996f95b4a958f03524c1c1af2214d0", + "gold_label_sha256": "30c0331a832a645c3ff0cb8f6a48d21394d0b7f92cadba7f160be82b5b1e901e", + "reports": { + "chronological": { + "id": "confirm-fresh-27-te-m0-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-28\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "6f49034e9d6f5bcda42eb49313a0192f2f57f396df2006d862ac2087e12c5ffb", + "request_file_sha256": "c41145be6361bd9ae870a138e9eec28036176d9b5258fa25aa35eb7fbb67c60d", + "receipt_file_sha256": "ea3808dbd87498faa2ccafc23fcd1e772869cf42df1b99d0a2848018052f4e08", + "receipt_body_text_sha256": "9c608afde37c70ae9d17be507d7b892521a2a3e7204c692d72db866754d63b53", + "result_file_sha256": "58dc4f555443172d26580e97135531691461045bdffc7d2a8802fe869294b394", + "raw_content_sha256": "9697ae0b043edc7459e3f0e7b3cf26431b534d4c4235ab6792946232f68e8762", + "response_id": "20261014_e19c0e94-5382-43c4-896c-7ee0d75c85c9", + "receipt_status": 200, + "prompt_tokens": 1377, + "completion_tokens": 2298, + "reasoning_tokens": 2268, + "elapsed_seconds": 10.898489665996749 + }, + "grouped_lossless": { + "id": "confirm-fresh-27-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\": \"confirmed\", \"answer\": \"2027-04-28\", \"needs_clarification\": false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "b817feb109976867316ab126aa755742fb39eab47f603199cb7f6a934377e810", + "request_file_sha256": "948ca72b0401ce24bb219a01ccc08e3160aa6cabd2bc83105716573905fec2a2", + "receipt_file_sha256": "0685132f24481e8aa308e3a3e65cb86b3c2269530919fb50f1b5d5fccbfdd4c1", + "receipt_body_text_sha256": "53883bebb477671b4583fac5ec2289d816810d6493682224b59a500aed0df81b", + "result_file_sha256": "04c15f2e05362259fb2c00f48ac7ea6f32c4898d1dbb0891c1d767bef29d25f3", + "raw_content_sha256": "ca5ffa82ffd7fbb61e75bdda4d8745aa04f7bd34f7a35241612277e52b52ef6a", + "response_id": "20261014_35241908-3622-40a3-a992-e49fc5139528", + "receipt_status": 200, + "prompt_tokens": 1996, + "completion_tokens": 2274, + "reasoning_tokens": 2240, + "elapsed_seconds": 9.866345625006943 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-27-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\": \"not_requested\", \"answer\": \"2027-04-28\", \"needs_clarification\": false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish confirmed.", + "request_sha256": "b817feb109976867316ab126aa755742fb39eab47f603199cb7f6a934377e810", + "request_file_sha256": "948ca72b0401ce24bb219a01ccc08e3160aa6cabd2bc83105716573905fec2a2", + "receipt_file_sha256": "dd7cd49fc9127ceaf8f683b74c7996fd8bf555f73a7708b33d4d7875eae04031", + "receipt_body_text_sha256": "9c52fff1363512004974dc10212be2e8d8c58cdac628ed5bd4a8e73e4e942473", + "result_file_sha256": "6dad95a7d69fa7ed7109363a832152dbf413d3cdb52e0fdfd1faa74c479378a2", + "raw_content_sha256": "26abba6e33580488183b594c9706fd521c559b2197f1a520f16f47d109d0d268", + "response_id": "20261014_97a7f130-0e51-443f-937a-000b19b2efdd", + "receipt_status": 200, + "prompt_tokens": 1996, + "completion_tokens": 2571, + "reasoning_tokens": 2535, + "elapsed_seconds": 11.015928041000734 + } + } + }, + { + "history_id": "confirm-fresh-27-te-m1", + "case_id": "fresh-27-te", + "scenario": "fresh-27", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "question": "VISIT-2127 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-27-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-27-1", + "role": "customer", + "content": "VISIT-2127ని 2027-04-28కి మార్చే అభ్యర్థనను పరిశీలించండి." + }, + { + "id": "note-27-2", + "role": "customer", + "content": "ఇచ్చిన సర్వర్ సమయానికి అనుమతి చెల్లుతుందో చూడండి. ఆ స్థిర సమయం తర్వాత ఒక సెకనుకు అది ముగుస్తుంది." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The grant is valid at the fixed server instant and expires one second later; the trace confirms the update for VISIT-2127. The scoped final backend date is 2027-04-28.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "confirmed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-27", + "id": "VISIT-2127", + "date": "2027-03-28", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-27", + "id": "VISIT-2127", + "date": "2027-04-28", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9b2628780f9f3fb1", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2127" + }, + "response": { + "date": "2027-03-28", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-b256f6e509372391", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2127", + "approval_id": "AUTH-27-GRANT-19", + "date": "2027-04-28", + "expected_version": 5, + "idempotency_key": "REQUEST-27-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-28", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "fe863445d6c241597cb2a8c4da24cae418c1b6d55dd7e39df5474a0164e10037", + "case_sha256": "da4691577630e571639c487eb08a26158a996f95b4a958f03524c1c1af2214d0", + "gold_label_sha256": "30c0331a832a645c3ff0cb8f6a48d21394d0b7f92cadba7f160be82b5b1e901e", + "reports": { + "chronological": { + "id": "confirm-fresh-27-te-m1-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-28\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "401ca2052c208ceeaaa7ac30b3aead539c9c3d07e0dac5edbbdb63fa34e429f9", + "request_file_sha256": "9fe1c607264cc0cf66665fe8586b3af1c993bcd8df04f5bc2c9bc1cd84d10345", + "receipt_file_sha256": "cc28ba94956035507a422e7be3ecb2e36514e22baf41336cac360c939e16fa92", + "receipt_body_text_sha256": "5097e5eb1f98d6bdc2104c7288f694db13939c522055dca8c8bf005258d15d43", + "result_file_sha256": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "raw_content_sha256": "8608e83321d9a3eac9053988d3073197a955bcce5974202ee13444130cb83da2", + "response_id": "20261014_cb5f9585-399e-41e8-973b-45b0f7af313a", + "receipt_status": 200, + "prompt_tokens": 1523, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.022177582999575 + }, + "grouped_lossless": { + "id": "confirm-fresh-27-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-28\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "e2c62f1a0143c7971698d1d190057fbacfec5a7539fe4a45ba94e29f0ddf5064", + "request_file_sha256": "2fb3971eec081cab265da36ebda00f17e00832852eabe107d2db88fedd695c32", + "receipt_file_sha256": "d99a0667734c53c07738ed7aef902a8ae0f0531f4c41727a6979887eae05914f", + "receipt_body_text_sha256": "3106b7b363374170eb56620658b4f4a49bd13f883fca5997cc1545543ce55f42", + "result_file_sha256": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "raw_content_sha256": "8608e83321d9a3eac9053988d3073197a955bcce5974202ee13444130cb83da2", + "response_id": "20261014_7ba0ab04-764b-4f94-bbdf-c6faaafa2e2e", + "receipt_status": 200, + "prompt_tokens": 2273, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.0070587079972029 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-27-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-28\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-28", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "e2c62f1a0143c7971698d1d190057fbacfec5a7539fe4a45ba94e29f0ddf5064", + "request_file_sha256": "2fb3971eec081cab265da36ebda00f17e00832852eabe107d2db88fedd695c32", + "receipt_file_sha256": "4bb3fc0b9e4c8fb7f2078c57e3cf30869b5a8a9b91240b25b7bd8e633745539c", + "receipt_body_text_sha256": "2aca9b909f6cbcab57f12aa88dd85e02822d7450bc6a26b111eb3d942123b957", + "result_file_sha256": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "raw_content_sha256": "8608e83321d9a3eac9053988d3073197a955bcce5974202ee13444130cb83da2", + "response_id": "20261014_3da3fae2-c243-4cb7-9250-f38805dc8d0a", + "receipt_status": 200, + "prompt_tokens": 2273, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.1062835000047926 + } + } + }, + { + "history_id": "confirm-fresh-28-te-m0", + "case_id": "fresh-28-te", + "scenario": "fresh-28", + "model": "sarvam-105b", + "family": "approval_boundary", + "question": "VISIT-2128 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-28-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. VISIT-2128ని 2027-04-29కి మార్చే అభ్యర్థనను పరిశీలించండి. ఇచ్చిన సర్వర్ సమయానికి అనుమతి చెల్లుతుందో చూడండి. ఆ స్థిర సమయం తర్వాత ఒక సెకనుకు అది ముగుస్తుంది." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The grant is valid at the fixed server instant and expires one second later; the trace confirms the update for VISIT-2128. The scoped final backend date is 2027-04-29.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "confirmed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-28", + "id": "VISIT-2128", + "date": "2027-03-29", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-28", + "id": "VISIT-2128", + "date": "2027-04-29", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-91737a7098e93fdb", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2128", + "approval_id": "AUTH-28-GRANT-19", + "date": "2027-04-29", + "expected_version": 5, + "idempotency_key": "REQUEST-28-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-29", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "72dee5d0398e6186b16633b8a731faa6e6eebe2bc020ae8157288b6966d730d7", + "case_sha256": "8f1e5f77bcea527761c08537ee29afd5efd63bdd491ee6ffe068a51f7a1dbd92", + "gold_label_sha256": "fa44409385a5ffc95af7330589206eee2da30beed32fa2ecfdaa61229b91b2e4", + "reports": { + "chronological": { + "id": "confirm-fresh-28-te-m0-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-29\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "6ab0a4be37931ed69914d1d167817ccb0655295243ee3ca92c081caf8ce96ee0", + "request_file_sha256": "86ffe108151f211eb821e3bfd781550b63421f9f5b301dff4f8be8492229c0a8", + "receipt_file_sha256": "5a44b97a675c789da363d5f8ce4121677d346f13a110bd47b78930d87c1f6742", + "receipt_body_text_sha256": "c8a5989326739307a725f3261f8d49e7297195a1fbde52932660bbea550b9363", + "result_file_sha256": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "raw_content_sha256": "d8820bdeab4784bd4b5a0ce76c7cf37112f3a832e5aa7de53fe26a44cbd97a5d", + "response_id": "20261014_9c229b0e-a387-45fe-bcf6-b52ecb5abed0", + "receipt_status": 200, + "prompt_tokens": 1339, + "completion_tokens": 519, + "reasoning_tokens": 489, + "elapsed_seconds": 3.13366279201 + }, + "grouped_lossless": { + "id": "confirm-fresh-28-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-29\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "898ff5691fa3504c07c67bb515785ffe5526f140289183846d901fca7841b668", + "request_file_sha256": "b90e20da052811b844a5701e6277ede17b7d40b9e8d8e80a91776f9db7d2475c", + "receipt_file_sha256": "f8437c3f278de57593ebb64779df2534a1d9df260879d46e56e5a7ad070f38f4", + "receipt_body_text_sha256": "abfd014759a0cca64c0d8f4e53e9d857fac02349c2807e272acd8b8853fb93fa", + "result_file_sha256": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "raw_content_sha256": "d8820bdeab4784bd4b5a0ce76c7cf37112f3a832e5aa7de53fe26a44cbd97a5d", + "response_id": "20261014_d8406e85-f2ce-4e14-96a5-95594b99876b", + "receipt_status": 200, + "prompt_tokens": 1837, + "completion_tokens": 4171, + "reasoning_tokens": 4139, + "elapsed_seconds": 23.696498665987747 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-28-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"confirmed\",\"answer\":\"2027-04-29\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "898ff5691fa3504c07c67bb515785ffe5526f140289183846d901fca7841b668", + "request_file_sha256": "b90e20da052811b844a5701e6277ede17b7d40b9e8d8e80a91776f9db7d2475c", + "receipt_file_sha256": "d1d77807a528854ac74cd3dbf3cff689b285ce0bd26d1fcf0f54f1fd4d2530c0", + "receipt_body_text_sha256": "44798ab15bdb7b62884bc1e26d36b3a7747089ae101a943beef58bc3e900d141", + "result_file_sha256": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "raw_content_sha256": "d8820bdeab4784bd4b5a0ce76c7cf37112f3a832e5aa7de53fe26a44cbd97a5d", + "response_id": "20261014_e63af974-91f1-4aa6-8aaa-fc2590eda434", + "receipt_status": 200, + "prompt_tokens": 1837, + "completion_tokens": 6869, + "reasoning_tokens": 6839, + "elapsed_seconds": 31.018074499996146 + } + } + }, + { + "history_id": "confirm-fresh-28-te-m1", + "case_id": "fresh-28-te", + "scenario": "fresh-28", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "question": "VISIT-2128 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-28-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. VISIT-2128ని 2027-04-29కి మార్చే అభ్యర్థనను పరిశీలించండి. ఇచ్చిన సర్వర్ సమయానికి అనుమతి చెల్లుతుందో చూడండి. ఆ స్థిర సమయం తర్వాత ఒక సెకనుకు అది ముగుస్తుంది." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The grant is valid at the fixed server instant and expires one second later; the trace confirms the update for VISIT-2128. The scoped final backend date is 2027-04-29.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "confirmed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-28", + "id": "VISIT-2128", + "date": "2027-03-29", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-28", + "id": "VISIT-2128", + "date": "2027-04-29", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a1cd89c7fe822b67", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2128" + }, + "response": { + "date": "2027-03-29", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-94b8feb79052237d", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2128", + "approval_id": "AUTH-28-GRANT-19", + "date": "2027-04-29", + "expected_version": 5, + "idempotency_key": "REQUEST-28-CHANGE-19" + }, + "response": { + "status": "confirmed", + "date": "2027-04-29", + "version": 6 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "41cfd00ce8d834eb4913c2fce96a57c8e7860577340d96d6084fb0ac441e84c0", + "case_sha256": "8f1e5f77bcea527761c08537ee29afd5efd63bdd491ee6ffe068a51f7a1dbd92", + "gold_label_sha256": "fa44409385a5ffc95af7330589206eee2da30beed32fa2ecfdaa61229b91b2e4", + "reports": { + "chronological": { + "id": "confirm-fresh-28-te-m1-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-29\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "a4505a354003dd9012f21bdc9067c78c16262d1c96ef35db3a20b95e4e0b08a7", + "request_file_sha256": "e6ce1b3ffd054994b308cef2410dd8766fa815cefb71e5250f56ccce353c6c89", + "receipt_file_sha256": "49052aa5b0c6a64d7ce79757fd97c9f6bb4ab62ffc2a11091c648195d9b55dda", + "receipt_body_text_sha256": "a126285954e487594626e36cf9825a8ecce5220e76d4c4146ef96e26eeecce97", + "result_file_sha256": "f70bc8efb6e5c2baaed099be3abea5bef92a529c53415c4cf1f762b65736aef1", + "raw_content_sha256": "824a909cf59990823dc60cd450be085d28013761e6c172af9edfa8a974c04ce2", + "response_id": "20261014_d18fb99a-2b5a-49f6-a9cd-cf4f172eb381", + "receipt_status": 200, + "prompt_tokens": 1477, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.9413197079993552 + }, + "grouped_lossless": { + "id": "confirm-fresh-28-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-29\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "7d86e3cc8efe64628da2566fa2496f76236ed33fe8d864332add0c262023fd6c", + "request_file_sha256": "136fdf58ffa2179f5377a58af7b1e27efb99a6942cd2c564bd7f17c599f5bf5d", + "receipt_file_sha256": "06edf7746c2e60c89707a239a30d6a39e81e729d8d1a216c04e90d79cdb4609e", + "receipt_body_text_sha256": "f4ff41d95a96f30cbde303ba5adbc7ddacd91a35152a701a09f9bce654916f57", + "result_file_sha256": "f70bc8efb6e5c2baaed099be3abea5bef92a529c53415c4cf1f762b65736aef1", + "raw_content_sha256": "824a909cf59990823dc60cd450be085d28013761e6c172af9edfa8a974c04ce2", + "response_id": "20261014_5943ed0a-8468-4e6a-9c64-74f81cc6249f", + "receipt_status": 200, + "prompt_tokens": 2120, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.9960944159975043 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-28-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"confirmed\",\n \"answer\": \"2027-04-29\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "confirmed", + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-29", + "needs_clarification": false + }, + "expected_action_status": "confirmed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status confirmed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "7d86e3cc8efe64628da2566fa2496f76236ed33fe8d864332add0c262023fd6c", + "request_file_sha256": "136fdf58ffa2179f5377a58af7b1e27efb99a6942cd2c564bd7f17c599f5bf5d", + "receipt_file_sha256": "97089578be8614b600bff9f10a46457d3e107e2b3edda52e4a6198d7aabff9de", + "receipt_body_text_sha256": "3bc44da15d7ef8b176edac9e55980679b7dcb66cc435b6d34b4c2b93afa4dfb2", + "result_file_sha256": "f70bc8efb6e5c2baaed099be3abea5bef92a529c53415c4cf1f762b65736aef1", + "raw_content_sha256": "824a909cf59990823dc60cd450be085d28013761e6c172af9edfa8a974c04ce2", + "response_id": "20261014_a0503d38-ea61-4496-a41e-ee60abf72f0b", + "receipt_status": 200, + "prompt_tokens": 2120, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.9589607079979032 + } + } + }, + { + "history_id": "confirm-fresh-29-te-m0", + "case_id": "fresh-29-te", + "scenario": "fresh-29", + "model": "sarvam-105b", + "family": "future_approval", + "question": "VISIT-2129 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-29-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-29-1", + "role": "customer", + "content": "ఇప్పుడే VISIT-2129ని 2027-04-30కి మార్చాలి." + }, + { + "id": "note-29-2", + "role": "customer", + "content": "ఇచ్చిన సర్వర్ సమయం తర్వాత ఒక గంటకు అనుమతి అమల్లోకి వస్తుంది. నా అభ్యర్థన దాని చెల్లుబాటు సమయాన్ని మార్చదు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The grant does not become effective until one hour after the fixed server time. This history has no update attempt; no date change was confirmed. The scoped current date remains 2027-03-30.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_performed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-29", + "id": "VISIT-2129", + "date": "2027-03-30", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-29", + "id": "VISIT-2129", + "date": "2027-03-30", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-82d8c17f95469987", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2129" + }, + "response": { + "date": "2027-03-30", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "c4f6588d46648e49a5af34345c752ff17dc78612e2181645618c835c7f5cd809", + "case_sha256": "98345f24869fe13876dae90994ac4aa346204f91c377356e8072603f29c51a55", + "gold_label_sha256": "98124b0d2d930eb900d70c8876a1dce676ea26522c7c6dcc7f84c87493be844c", + "reports": { + "chronological": { + "id": "confirm-fresh-29-te-m0-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-30\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_performed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "153fec59ad84969e5ad0f4672691e79c824a1d60b75115e8ffdfe7c06556c6d1", + "request_file_sha256": "0dca8427ba974b69a5fa0d1ba7488087abf18640742958b019a884a6cc4d8c9b", + "receipt_file_sha256": "b5a040b1ddb7709de25a6577cbd92b36c4e64ac81a69b742937cb3a252e36142", + "receipt_body_text_sha256": "2822eea813aefcdeef13b0d72fcc1f462331ebf2ba0dfc1507bf8c3fd3b74533", + "result_file_sha256": "f517b9fdb42a8c5fdc3536e2720b69744ec169f1684a3271023240b26b8c846f", + "raw_content_sha256": "6f60dea4694b6d52e1bbd9a2b2fe175afd3782c6eb1f13cf983f53e25c6b19cd", + "response_id": "20261014_790fc396-e26a-4dea-bef6-beafbaa8e6a9", + "receipt_status": 200, + "prompt_tokens": 1576, + "completion_tokens": 2566, + "reasoning_tokens": 2534, + "elapsed_seconds": 12.215514750001603 + }, + "grouped_lossless": { + "id": "confirm-fresh-29-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-03-30\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_performed", + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_performed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "3fdaf8bdd895a38390bde4150ff73432c1b9815c84ec8acfc1b03ef848014adb", + "request_file_sha256": "abb02752342e22ad6594132e6eb27b04f8b66c4423731b9bd713010daf952a38", + "receipt_file_sha256": "e1ba52bd127272feec14c167c5de0d17c166e7de643e4413e6e2009bace89448", + "receipt_body_text_sha256": "1e59d9eee9e7e81e7afec6e959da1b94f69ab3e04ea122030a089060619746e7", + "result_file_sha256": "f517b9fdb42a8c5fdc3536e2720b69744ec169f1684a3271023240b26b8c846f", + "raw_content_sha256": "6f60dea4694b6d52e1bbd9a2b2fe175afd3782c6eb1f13cf983f53e25c6b19cd", + "response_id": "20261014_3f9c665c-1937-4339-aa64-a0b931c5b6ad", + "receipt_status": 200, + "prompt_tokens": 2268, + "completion_tokens": 2444, + "reasoning_tokens": 2412, + "elapsed_seconds": 11.178991207998479 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-29-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-03-30\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "5627605f48e45ed2e1724a9f74b2435b96a79811fa1a772b4a5a68c306f6a55b", + "request_file_sha256": "338fae8b71dae9f117b0ef7ed488c7214cbb9b5d3ac6f94291ef80185eeff789", + "receipt_file_sha256": "d29f31f55e247d9889a74bda2a0b0854be2a0fa6e082802ad22e298ba395183a", + "receipt_body_text_sha256": "d7ec4c0976a7968046071f5ab4426fbe0c253c868ca8f7e9b3ad2a8137ebeb18", + "result_file_sha256": "9a33d005b8af04a9daae27cfbec0007280a387d73264252880042af4f86632aa", + "raw_content_sha256": "0163f4a41e8881b6b015044ce2a23ad1adb4ba560451442685979f2b5b6a7fff", + "response_id": "20261014_2028f48e-bcf7-48c4-a679-611d271b033d", + "receipt_status": 200, + "prompt_tokens": 2032, + "completion_tokens": 1531, + "reasoning_tokens": 1499, + "elapsed_seconds": 7.056660917005502 + } + } + }, + { + "history_id": "confirm-fresh-29-te-m1", + "case_id": "fresh-29-te", + "scenario": "fresh-29", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "question": "VISIT-2129 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-29-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-29-1", + "role": "customer", + "content": "ఇప్పుడే VISIT-2129ని 2027-04-30కి మార్చాలి." + }, + { + "id": "note-29-2", + "role": "customer", + "content": "ఇచ్చిన సర్వర్ సమయం తర్వాత ఒక గంటకు అనుమతి అమల్లోకి వస్తుంది. నా అభ్యర్థన దాని చెల్లుబాటు సమయాన్ని మార్చదు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The grant does not become effective until one hour after the fixed server time. This history has a rejected update receipt; no date change was confirmed. The scoped current date remains 2027-03-30.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "rejected", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-29", + "id": "VISIT-2129", + "date": "2027-03-30", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-29", + "id": "VISIT-2129", + "date": "2027-03-30", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a7ec05315ba4b6b4", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2129" + }, + "response": { + "date": "2027-03-30", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-aab7b0df58e5cbf3", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2129", + "approval_id": "AUTH-29-GRANT-20", + "date": "2027-04-30", + "expected_version": 5, + "idempotency_key": "REQUEST-29-CHANGE-20" + }, + "response": { + "status": "rejected", + "reason": "approval_not_effective", + "date": "2027-03-30", + "version": 5 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "ef2e96d07a161f8aefe3f23c9cc693d7c04f0a4ac1afc033cc0d8a0ccb2eb4db", + "case_sha256": "98345f24869fe13876dae90994ac4aa346204f91c377356e8072603f29c51a55", + "gold_label_sha256": "98124b0d2d930eb900d70c8876a1dce676ea26522c7c6dcc7f84c87493be844c", + "reports": { + "chronological": { + "id": "confirm-fresh-29-te-m1-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-03-30\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status rejected agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "2081e7540d8b4adcd42718ac6f137d1907a270d5dd0e64c775b720544437def6", + "request_file_sha256": "297aed1b8ae73df9b8d0ecc9fe20b4bce11ff81e44300a0ec8ca1cac9578314e", + "receipt_file_sha256": "9c2ae8aa782969bae38bfeb9c7fbae67fca3b728e72bc7ae7bd69b90f3d80205", + "receipt_body_text_sha256": "9adfb19c0cc4dea25b64a75046c52ec9c26cca4d69592cde9b4d9c4744bec0bd", + "result_file_sha256": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "raw_content_sha256": "c7b8d9498405aff859e7cd3c35981ee7d640e01038a3105a846ac123bac08074", + "response_id": "20261014_0ccd6efa-b2b6-4155-b178-dbea90501ac4", + "receipt_status": 200, + "prompt_tokens": 1517, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.0014116670063231 + }, + "grouped_lossless": { + "id": "confirm-fresh-29-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-03-30\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status rejected agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "bdf1f6a7cc73087785b24b21d0ff91674978a79f892b93c88949a7a5b964516d", + "request_file_sha256": "43d5c90ed1fe9a29eb73b60b86ff22298d5946e0427ab5bd51cd7c37ed381254", + "receipt_file_sha256": "4175d8e5803af5da643372f11c129a7ea4e755a214dbe92e4eeb408c192e36f5", + "receipt_body_text_sha256": "5d1f7f21231b5de20d79f61b4e5895c42f00bddc7b550b34cc57812c10d69e76", + "result_file_sha256": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "raw_content_sha256": "c7b8d9498405aff859e7cd3c35981ee7d640e01038a3105a846ac123bac08074", + "response_id": "20261014_b92892e3-0b03-40dc-92cd-a138b6c7ae1c", + "receipt_status": 200, + "prompt_tokens": 2279, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.9713660840061493 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-29-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-03-30\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "rejected", + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-03-30", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status rejected agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "bdf1f6a7cc73087785b24b21d0ff91674978a79f892b93c88949a7a5b964516d", + "request_file_sha256": "43d5c90ed1fe9a29eb73b60b86ff22298d5946e0427ab5bd51cd7c37ed381254", + "receipt_file_sha256": "e7d3bf0c0a8a11d5a5c7fa8b0d46a7e0761b4add3b4f5b03c22ead36334c9b55", + "receipt_body_text_sha256": "619fd48ef638b921f6ff201491e514e0864374cad70918aadc52880654976a15", + "result_file_sha256": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "raw_content_sha256": "c7b8d9498405aff859e7cd3c35981ee7d640e01038a3105a846ac123bac08074", + "response_id": "20261014_a9120cb8-ab2e-4a7d-b9ae-059b6d59abe2", + "receipt_status": 200, + "prompt_tokens": 2279, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.9608177500049351 + } + } + }, + { + "history_id": "confirm-fresh-30-te-m0", + "case_id": "fresh-30-te", + "scenario": "fresh-30", + "model": "sarvam-105b", + "family": "future_approval", + "question": "VISIT-2130 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-30-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. ఇప్పుడే VISIT-2130ని 2027-05-01కి మార్చాలి. ఇచ్చిన సర్వర్ సమయం తర్వాత ఒక గంటకు అనుమతి అమల్లోకి వస్తుంది. నా అభ్యర్థన దాని చెల్లుబాటు సమయాన్ని మార్చదు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The grant does not become effective until one hour after the fixed server time. This history has no update attempt; no date change was confirmed. The scoped current date remains 2027-03-31.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_performed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-30", + "id": "VISIT-2130", + "date": "2027-03-31", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-30", + "id": "VISIT-2130", + "date": "2027-03-31", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "e86db2ec8327a7099054adf79d1984e5d87e70f4e4c5062adaf9210a4996f130", + "case_sha256": "d139a0869df600e24f62fd85f8f0cbfb9f7d82ee49a776c2ed3ed689d4921c5a", + "gold_label_sha256": "a0e3b9a0dffc8cc6abc397a1298d2eb5ba40907a092896f7e7381a0881453c25", + "reports": { + "chronological": { + "id": "confirm-fresh-30-te-m0-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-03-31\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "b564f84805dadb082a1b98787bf9f55694e3ef4cfb282041d5e4d569cf3f30e7", + "request_file_sha256": "f30f4b5b42b0fcf7da6b3f78da8b1be1d7ace6766944814051f2c7c3514833b1", + "receipt_file_sha256": "a0dde3b5bafd309b4f54baf92a4e2733e4446aa071b9f434662be2e8a2c1df2e", + "receipt_body_text_sha256": "55f9057bf666f8174198d3615fc8aff42ccdc237603b7a7b97df2a763e05adc4", + "result_file_sha256": "c225aab8d19cca2515a377d9c5462cc34a003a0c3653544bba785e793db51707", + "raw_content_sha256": "cca5bd3578575b6d45284125d6a11cca2ba727e2b310e9f5ec6cd6bef932def8", + "response_id": "20261014_e1a6a610-35fd-445a-b9bb-691e91f7c802", + "receipt_status": 200, + "prompt_tokens": 1418, + "completion_tokens": 2969, + "reasoning_tokens": 2937, + "elapsed_seconds": 12.611806625005556 + }, + "grouped_lossless": { + "id": "confirm-fresh-30-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-03-31\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "2a69fefa7bdfbf3afaddc0efb4a918d7d93363f1c9ac2e833988f6d63a2065ea", + "request_file_sha256": "2e346afc550e2f32d48c8ad7a2700ce8270bca82d1424cae81509e5bbac10784", + "receipt_file_sha256": "5676fa64677404aefc8fc7b385a2cb36a98c0c0b8357af43f95307ee17cdd33a", + "receipt_body_text_sha256": "ed8db7fefce1d100c8068a7f65ee98fbd0406b350d3214742f0636a2a3ca62ce", + "result_file_sha256": "c225aab8d19cca2515a377d9c5462cc34a003a0c3653544bba785e793db51707", + "raw_content_sha256": "cca5bd3578575b6d45284125d6a11cca2ba727e2b310e9f5ec6cd6bef932def8", + "response_id": "20261014_5d611b05-ace9-46f3-ad5e-866da348db08", + "receipt_status": 200, + "prompt_tokens": 1856, + "completion_tokens": 1401, + "reasoning_tokens": 1369, + "elapsed_seconds": 8.085513083002297 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-30-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-03-31\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "49d961c97eb018daeea43efd5ceaf470f1e7c78aad57507154c56dc8a2288902", + "request_file_sha256": "c2d3904c52a4805fbfb5738ac0b4575c9a676ca9abf5a447301dd7d730a54aaa", + "receipt_file_sha256": "907babfb41c98fe819bfedd8393dcc06d9d4b45fb8043640c52a00d4ae166914", + "receipt_body_text_sha256": "a03c09136c286bb0ac742499db72d2cd01c70c052d72d939aae15a286aab606d", + "result_file_sha256": "c225aab8d19cca2515a377d9c5462cc34a003a0c3653544bba785e793db51707", + "raw_content_sha256": "cca5bd3578575b6d45284125d6a11cca2ba727e2b310e9f5ec6cd6bef932def8", + "response_id": "20261014_d0535662-ff53-4ade-b34e-6bb3ee784e3c", + "receipt_status": 200, + "prompt_tokens": 1601, + "completion_tokens": 3252, + "reasoning_tokens": 3220, + "elapsed_seconds": 14.983070792004582 + } + } + }, + { + "history_id": "confirm-fresh-30-te-m1", + "case_id": "fresh-30-te", + "scenario": "fresh-30", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "question": "VISIT-2130 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-30-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. ఇప్పుడే VISIT-2130ని 2027-05-01కి మార్చాలి. ఇచ్చిన సర్వర్ సమయం తర్వాత ఒక గంటకు అనుమతి అమల్లోకి వస్తుంది. నా అభ్యర్థన దాని చెల్లుబాటు సమయాన్ని మార్చదు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The grant does not become effective until one hour after the fixed server time. This history has a rejected update receipt; no date change was confirmed. The scoped current date remains 2027-03-31.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "rejected", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-30", + "id": "VISIT-2130", + "date": "2027-03-31", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-30", + "id": "VISIT-2130", + "date": "2027-03-31", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a4cca886167a258d", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2130" + }, + "response": { + "date": "2027-03-31", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-99d57cc2a5abb81c", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2130", + "approval_id": "AUTH-30-GRANT-20", + "date": "2027-05-01", + "expected_version": 5, + "idempotency_key": "REQUEST-30-CHANGE-20" + }, + "response": { + "status": "rejected", + "reason": "approval_not_effective", + "date": "2027-03-31", + "version": 5 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "4d90c819dd385551de4cd4a31f6e632acebf5c1982b5fc5ad2f11d4f6f7fa270", + "case_sha256": "d139a0869df600e24f62fd85f8f0cbfb9f7d82ee49a776c2ed3ed689d4921c5a", + "gold_label_sha256": "a0e3b9a0dffc8cc6abc397a1298d2eb5ba40907a092896f7e7381a0881453c25", + "reports": { + "chronological": { + "id": "confirm-fresh-30-te-m1-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-03-31\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "rejected", + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status rejected agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "fe4f205d21bc2a9b901f7e0bda3607051e0cb99da7f29554041f544e911350a0", + "request_file_sha256": "3d80d37f289c544112bfade0800235d63f804dfda0bcab8c3a7a13e88bbeb315", + "receipt_file_sha256": "7d4115c37deb9cd162b93318536843c76bac1cf7c3f8c540b76d251cb273a221", + "receipt_body_text_sha256": "b607732f8d52f8cd73d8630c4fc38dd4540529d7ce4be5022ba05665b632394e", + "result_file_sha256": "87902016aae6283bdff9207729e01b30ad77cad918dba806396649f58f68c74c", + "raw_content_sha256": "31b20e9d2fd9f0fa5256283e857194fb3888033a9b8acd91e893978a488f41a6", + "response_id": "20261014_a2438d06-4bb1-42b3-90cc-f24790cd5c25", + "receipt_status": 200, + "prompt_tokens": 1480, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.9327283329912461 + }, + "grouped_lossless": { + "id": "confirm-fresh-30-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-03-31\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_performed", + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_performed conflicts with trusted intent and actual receipts, which establish rejected.", + "request_sha256": "2da540733eeb2b84e69bdce8516a804682abb86adfd6a179c3da3aa781a81eb8", + "request_file_sha256": "dc5f2f3edbb78b4f9a2cf06b51c7c8308dab128419034ab512fb541f6676c0ad", + "receipt_file_sha256": "260deb414b93408a9b0058f650c1ee4d67404392b4af73a9971520e2f5c4d002", + "receipt_body_text_sha256": "3cb65e592831725deefb1bb58a7f987b45635bb8885c2f6a432446906a3ee96c", + "result_file_sha256": "da50423f0bc90841c2b9e138e9f4a9ad0340b0ec5d888b3091878471fa1a28a5", + "raw_content_sha256": "4a9f4305f0d97a49c966f980e6529f6c921798fc33742364cdcd76ba89fbee50", + "response_id": "20261014_889223d5-eea3-4f0b-bac5-5fff46e6f9b3", + "receipt_status": 200, + "prompt_tokens": 2128, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.037280082993675 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-30-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-03-31\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-03-31", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish rejected.", + "request_sha256": "2da540733eeb2b84e69bdce8516a804682abb86adfd6a179c3da3aa781a81eb8", + "request_file_sha256": "dc5f2f3edbb78b4f9a2cf06b51c7c8308dab128419034ab512fb541f6676c0ad", + "receipt_file_sha256": "a077d3ebec83b306740b6afbc2fe272dd5a1d3cfe96b9ab15a000f0a4cbd2f72", + "receipt_body_text_sha256": "3e17771d674f0ee75d12b83bc9d22e67faaea300bf0bf4c2e68954b69f67a64f", + "result_file_sha256": "fc8165ed660426c4e3123966abfd67ef96b0cbe2d8cd6f54949fa8d3ece0de02", + "raw_content_sha256": "a30aacee00152000aeb8c2565229b22fcef45fa466e22daf1ea4c8d587176f57", + "response_id": "20261014_1a373f7a-89d5-4681-92f7-1d867ae84761", + "receipt_status": 200, + "prompt_tokens": 2128, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.1079283750004834 + } + } + }, + { + "history_id": "confirm-fresh-31-te-m0", + "case_id": "fresh-31-te", + "scenario": "fresh-31", + "model": "sarvam-105b", + "family": "future_approval", + "question": "VISIT-2131 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-31-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-31-1", + "role": "customer", + "content": "ఇప్పుడే VISIT-2131ని 2027-05-02కి మార్చాలి." + }, + { + "id": "note-31-2", + "role": "customer", + "content": "ఇచ్చిన సర్వర్ సమయం తర్వాత ఒక గంటకు అనుమతి అమల్లోకి వస్తుంది. నా అభ్యర్థన దాని చెల్లుబాటు సమయాన్ని మార్చదు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The grant does not become effective until one hour after the fixed server time. This history has no update attempt; no date change was confirmed. The scoped current date remains 2027-04-01.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_performed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-31", + "id": "VISIT-2131", + "date": "2027-04-01", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-31", + "id": "VISIT-2131", + "date": "2027-04-01", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9a9e61d760ad6d68", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2131" + }, + "response": { + "date": "2027-04-01", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "3198992206bd25cdd77b86243dcf3f9ad15ece6a6db10731069a6246635a3997", + "case_sha256": "f196d0c2589f33f978e62f6254ea13b02bbe3c64a9c99d1c25bae1d09ce9eb4f", + "gold_label_sha256": "04626f9b5341ca9c4b080b085b6f094f9df63ec58f6f39753645925e2f6a5e3c", + "reports": { + "chronological": { + "id": "confirm-fresh-31-te-m0-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-04-01\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_performed", + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_performed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "8166fc7f2bb36894b8a7ab1e249a13186d0f2a3d739e613cb31b8ce950d75ea1", + "request_file_sha256": "b7e90b0c7d0f24e2f25bc900fab0b0aafeae731a3c6d3204652d5e729a41bb58", + "receipt_file_sha256": "4892fa785936fe22ae34cb2321f2468d45d7258e60a103d878fc8bd788f7b825", + "receipt_body_text_sha256": "277548ff345fff48aa21d3bd55600f357e05919ccc75186688840a5f697302fc", + "result_file_sha256": "fe4c3267a9cec82092e8ede130613eed251227abe98c2163619dbfcb8dccec96", + "raw_content_sha256": "a1f9fcadc6ba4e8eafc46ac98450dd9b630b034f69ac35fd2a9eecd2706d424e", + "response_id": "20261014_66473388-70ba-40bb-96c8-e4a8f4a3902c", + "receipt_status": 200, + "prompt_tokens": 1485, + "completion_tokens": 6889, + "reasoning_tokens": 6857, + "elapsed_seconds": 31.6984588750056 + }, + "grouped_lossless": { + "id": "confirm-fresh-31-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-01\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "40976b3b82a5dd478d5c0e42d0a094f400eaf97775991598f124dd305c6d9aed", + "request_file_sha256": "9568899c36e57c6ccdc7ad4850a7f51407f798d2f7eef99944563b8d66eee75e", + "receipt_file_sha256": "94dd382c6b98e8dcc4e114b28efe5f4b8e3bb0888bbd6646fbb7b378cab7da13", + "receipt_body_text_sha256": "8de12b410794009af2cd2709bac9a04b9ca40a57969aa50140763eb437a7f512", + "result_file_sha256": "b5770e7ca54498bbe9cbc192d959a98975dc03170a1aa14ce7055f82160458c4", + "raw_content_sha256": "564e8f5c169211638a274eece597eb4b2d5f7ccfac6aedfd9d936dc2a89cb610", + "response_id": "20261014_d9e2f38d-5262-4119-9eca-3200768acf74", + "receipt_status": 200, + "prompt_tokens": 2172, + "completion_tokens": 1033, + "reasoning_tokens": 1001, + "elapsed_seconds": 6.013302583000041 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-31-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "incomplete", + "finish_reason": "length", + "raw_content": null, + "raw_output": null, + "expected_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": false, + "clarification_match": false, + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure.", + "status_review": "No valid raw action status because the final is incomplete; counted incorrect.", + "request_sha256": "5c7927e19c02d4de816c6c39c53a96a2c26c9d4c5ef6f6c6e97cc9af63cc8ca2", + "request_file_sha256": "5be84070603b5e5f5d98e55212c08e928ebf893a3fd4352697bf7639ee059f31", + "receipt_file_sha256": "36dddc42e4faea6c52c935d799d77d78b1f4adf0119dc99edc738f8ec0ceef9b", + "receipt_body_text_sha256": "0338e8a68f004890b5682922f7106ad045e953b1a6af90a47922e2ff19bfe90a", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "raw_content_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "response_id": "20261014_4183984a-809a-4a1a-b265-11d0739239b6", + "receipt_status": 200, + "prompt_tokens": 2027, + "completion_tokens": 8192, + "reasoning_tokens": 8191, + "elapsed_seconds": 38.83640883299813 + } + } + }, + { + "history_id": "confirm-fresh-31-te-m1", + "case_id": "fresh-31-te", + "scenario": "fresh-31", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "question": "VISIT-2131 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-31-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-31-1", + "role": "customer", + "content": "ఇప్పుడే VISIT-2131ని 2027-05-02కి మార్చాలి." + }, + { + "id": "note-31-2", + "role": "customer", + "content": "ఇచ్చిన సర్వర్ సమయం తర్వాత ఒక గంటకు అనుమతి అమల్లోకి వస్తుంది. నా అభ్యర్థన దాని చెల్లుబాటు సమయాన్ని మార్చదు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The grant does not become effective until one hour after the fixed server time. This history has a rejected update receipt; no date change was confirmed. The scoped current date remains 2027-04-01.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "rejected", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-31", + "id": "VISIT-2131", + "date": "2027-04-01", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-31", + "id": "VISIT-2131", + "date": "2027-04-01", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-87ffbf233d15948d", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2131" + }, + "response": { + "date": "2027-04-01", + "version": 5, + "closed": false + } + }, + { + "call_id": "chatcmpl-tool-915f6fde6e456879", + "name": "update_appointment", + "arguments": { + "appointment_id": "VISIT-2131", + "approval_id": "AUTH-31-GRANT-20", + "date": "2027-05-02", + "expected_version": 5, + "idempotency_key": "REQUEST-31-CHANGE-20" + }, + "response": { + "status": "rejected", + "reason": "approval_not_effective", + "date": "2027-04-01", + "version": 5 + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "a38da3525d7d566751e113920c2b2e832ffad2022ac7ae346f144e6c18554355", + "case_sha256": "f196d0c2589f33f978e62f6254ea13b02bbe3c64a9c99d1c25bae1d09ce9eb4f", + "gold_label_sha256": "04626f9b5341ca9c4b080b085b6f094f9df63ec58f6f39753645925e2f6a5e3c", + "reports": { + "chronological": { + "id": "confirm-fresh-31-te-m1-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-04-01\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status rejected agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "180eca0b166aff19acb5e3ffe1943a6f1b482e383e580290023692c82fba868e", + "request_file_sha256": "af15d60f54aae9b1fa36dfb9174c05d5d888e3768011f89ca2ce19de4fd23686", + "receipt_file_sha256": "70f4510a76003e14e2d6c042f16c0105694fd1f900fb8925c5ae30f478052f64", + "receipt_body_text_sha256": "7114ce5475dc87500d52fa4e497f893f2b8e6647bba9c473f5428fe47949d966", + "result_file_sha256": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "raw_content_sha256": "0d3aa07f0f0dff3403e3ba14d6cb185cfb839350bb5d46026f3e377de4a0b70f", + "response_id": "20261014_f248cec5-feec-46f9-8ee1-79d0b7094db2", + "receipt_status": 200, + "prompt_tokens": 1523, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.0602819999912754 + }, + "grouped_lossless": { + "id": "confirm-fresh-31-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-04-01\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status rejected agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "789160e081f427632bb078b9d93c56f5b6b107314d4a5d388d5866252113771e", + "request_file_sha256": "02c1f081ab31b430698272b5745efb65ecb20367a07c25b29f1bb3f395f10ec1", + "receipt_file_sha256": "f7a462ddcb7c8d7d31305ea95e61c9362e6d17dbd874f50bc5b19c285bf3d690", + "receipt_body_text_sha256": "56f91e329e6608745d3b6211f90a0e247d1c6129500dc1804303cdbc6cf2251a", + "result_file_sha256": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "raw_content_sha256": "0d3aa07f0f0dff3403e3ba14d6cb185cfb839350bb5d46026f3e377de4a0b70f", + "response_id": "20261014_ee0e54d5-0832-49bd-add6-f9559665a8c7", + "receipt_status": 200, + "prompt_tokens": 2281, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.0018306670099264 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-31-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": false, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-04-01\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "rejected", + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-01", + "needs_clarification": false + }, + "expected_action_status": "rejected", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status rejected agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "789160e081f427632bb078b9d93c56f5b6b107314d4a5d388d5866252113771e", + "request_file_sha256": "02c1f081ab31b430698272b5745efb65ecb20367a07c25b29f1bb3f395f10ec1", + "receipt_file_sha256": "f5bf67261209ff721d2fe021fac9a71a96f033346fa7418b0c6a98ed3b1810ff", + "receipt_body_text_sha256": "2c6537c2957e05e34661d53fb728c2fda07b7e1f6b97ba3c5cc8f0bde22d78fa", + "result_file_sha256": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "raw_content_sha256": "0d3aa07f0f0dff3403e3ba14d6cb185cfb839350bb5d46026f3e377de4a0b70f", + "response_id": "20261014_ef7ba56f-6c1d-4986-927b-21286fc357c9", + "receipt_status": 200, + "prompt_tokens": 2281, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.9490969160106033 + } + } + }, + { + "history_id": "confirm-fresh-32-te-m0", + "case_id": "fresh-32-te", + "scenario": "fresh-32", + "model": "sarvam-105b", + "family": "future_approval", + "question": "VISIT-2132 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-32-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. ఇప్పుడే VISIT-2132ని 2027-05-03కి మార్చాలి. ఇచ్చిన సర్వర్ సమయం తర్వాత ఒక గంటకు అనుమతి అమల్లోకి వస్తుంది. నా అభ్యర్థన దాని చెల్లుబాటు సమయాన్ని మార్చదు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The grant does not become effective until one hour after the fixed server time. This history has no update attempt; no date change was confirmed. The scoped current date remains 2027-04-02.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_performed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-32", + "id": "VISIT-2132", + "date": "2027-04-02", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-32", + "id": "VISIT-2132", + "date": "2027-04-02", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "ca53f9672230dd53fd63c2a758ab50179ef602b9b331df435aaa263eab1816af", + "case_sha256": "88d9212e11db07959d33f6ce4c6a953ec7f76b5e24d3f2d211f4f8e76fddd9fd", + "gold_label_sha256": "439022a75020b9055f984f459b8eff6efe039c8629132d6259578274665d57c2", + "reports": { + "chronological": { + "id": "confirm-fresh-32-te-m0-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-04-02\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "51c76dc4c37439a0577655f1a2e9e080baf92c2993589da2843031e4ff0ffd77", + "request_file_sha256": "a164908edb1d18918dd01fdea44045180ec22a945b16eb56f087e18f7e5796de", + "receipt_file_sha256": "e30c2ec2d1007660b95af8a47378afbe90f29177be5d7643ddda5a7bc0bb0cc1", + "receipt_body_text_sha256": "de4c75d5bd7e37c75499cd7e974544f4f31efe282663e83322d326cdc8a47367", + "result_file_sha256": "2465b6c24f7e04b712333958ffe384554975d4fedbb6dadba79b616478a8eb65", + "raw_content_sha256": "995faa359044c9d6d14445c6b8c04426bf5850fc3e064c57f837cdce5fd3b5e5", + "response_id": "20261014_68856c5e-6fbb-44ba-8575-2547235da01c", + "receipt_status": 200, + "prompt_tokens": 1303, + "completion_tokens": 1861, + "reasoning_tokens": 1829, + "elapsed_seconds": 9.870290666993242 + }, + "grouped_lossless": { + "id": "confirm-fresh-32-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-04-02\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_performed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "00af19fa6f759b1027a1242db2b56c581dbefd24e85199129ad407f13e40741e", + "request_file_sha256": "a1fa4d38f3e7e74f5aa61679d0368d7d5bc64b988dd12c58ce7ffe33e2bb19ec", + "receipt_file_sha256": "dd84a2e3339cf4ecbb6eb3c45307b1c2dabb344c4fe348a1877b15981f254ba3", + "receipt_body_text_sha256": "a03099e9ba1cd88b2d761b683e04a1921605a7a761e74a1c8b7d0098f753a0ad", + "result_file_sha256": "b106ac717f61df19b67b9ff99d9131e9d442ef563917d74f9f34670967d64074", + "raw_content_sha256": "f08325c4871b8031e4648a53e1cec3da798b14ad1f6ccd0f41aae4349a10b13e", + "response_id": "20261014_2db46829-db6e-4c47-b0a6-717d55c689d7", + "receipt_status": 200, + "prompt_tokens": 1731, + "completion_tokens": 1972, + "reasoning_tokens": 1940, + "elapsed_seconds": 9.322261666995473 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-32-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\": \"not_performed\", \"answer\": \"2027-04-02\", \"needs_clarification\": false}", + "raw_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_performed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "1c078a86236e73529d2eb35946d35e00fbefdfe692dedce2cfbb8f03886bc41d", + "request_file_sha256": "d84251ae8f4dc921418046d42655a180ab539ddb725dde43dffad20a292b37aa", + "receipt_file_sha256": "4bb6c0a94cd86eacb7920d528d4e1f2ff57e3c3708fd3b27edc42723b4b9ddce", + "receipt_body_text_sha256": "acb65e2567822345a31f17bb0fc57ef23c382c0162717d7e4f90483dfbc41948", + "result_file_sha256": "c8610025a759bbe68179e78a62da3fb5774353b90330e6f59855fd3e75108541", + "raw_content_sha256": "6c8885ac0399db29e7fc2b456e2a7ea8317d20cc61747dfb28345ea9e01cd577", + "response_id": "20261014_515b18ac-5557-417a-a3c0-bae6293ee9a9", + "receipt_status": 200, + "prompt_tokens": 1591, + "completion_tokens": 1937, + "reasoning_tokens": 1901, + "elapsed_seconds": 10.579083415999776 + } + } + }, + { + "history_id": "confirm-fresh-32-te-m1", + "case_id": "fresh-32-te", + "scenario": "fresh-32", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "question": "VISIT-2132 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-32-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. ఇప్పుడే VISIT-2132ని 2027-05-03కి మార్చాలి. ఇచ్చిన సర్వర్ సమయం తర్వాత ఒక గంటకు అనుమతి అమల్లోకి వస్తుంది. నా అభ్యర్థన దాని చెల్లుబాటు సమయాన్ని మార్చదు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The grant does not become effective until one hour after the fixed server time. This history has no update attempt; no date change was confirmed. The scoped current date remains 2027-04-02.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_performed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-32", + "id": "VISIT-2132", + "date": "2027-04-02", + "version": 5, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-32", + "id": "VISIT-2132", + "date": "2027-04-02", + "version": 5, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a58950bb6d7cec67", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2132" + }, + "response": { + "date": "2027-04-02", + "version": 5, + "closed": false + } + } + ], + "prior_execution_receipts": [], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "dfd315d13675a52e9268ba74bfe7e3f87f316047c14efbc9e2b53f52c84cd6d2", + "case_sha256": "88d9212e11db07959d33f6ce4c6a953ec7f76b5e24d3f2d211f4f8e76fddd9fd", + "gold_label_sha256": "439022a75020b9055f984f459b8eff6efe039c8629132d6259578274665d57c2", + "reports": { + "chronological": { + "id": "confirm-fresh-32-te-m1-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-04-02\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_performed", + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_performed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "028f04d03b70113971af47303c9adb2216676838f46511cf08e5cf4e450f28ed", + "request_file_sha256": "9ee32977f53eb753647e2956407358f30d3e76b8e5d123a55db642cdbe5a65b8", + "receipt_file_sha256": "e7fc2f0439f5f6ff492564b3e01da6383e13ae1f6067d9129f5be588a726c312", + "receipt_body_text_sha256": "8e6271a043f0b2fa90353b4da177c86a5884003a434e006613e3cacf0d7a6abe", + "result_file_sha256": "b5537fea0aacf9bcedfc1a598c3b4e08a55b92f5c5cb181429cff1acda690630", + "raw_content_sha256": "1f2cad9a533345f41eea593348fa91f701d75caa054b15520f51e90a290da757", + "response_id": "20261014_bff6cc46-c285-448b-aa78-8d7fff03726c", + "receipt_status": 200, + "prompt_tokens": 1744, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9329089170059888 + }, + "grouped_lossless": { + "id": "confirm-fresh-32-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-02\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "7a8ace65adad7dffb4fa7b6339e940b74b34f0561244b14928312c1375a381eb", + "request_file_sha256": "5901b5c5e5fdb430ec768879a65f701eec187d10fc6626b4d225aa415cda93af", + "receipt_file_sha256": "f69ebbbc8e65ff57f126b715ba40f29c405875f6982acd15cc4aee927b5d474b", + "receipt_body_text_sha256": "f963f1e9217d84102c33c45fed74d2c3ff657981ba21e6fd600202d09d61a74c", + "result_file_sha256": "6339bfc3a8c7c5365b3d444ec9928fe27e6a132032e7c64c5a5b885479a0be7a", + "raw_content_sha256": "6d2640aaf6fb64c2eacb3520af3cebfc49d073ff5ea8b2b60feb45f961029035", + "response_id": "20261014_edec5c43-fecd-4dd5-b582-7506797a2ce0", + "receipt_status": 200, + "prompt_tokens": 2324, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9693154159904225 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-32-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-04-02\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-04-02", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "c508210e3fdb28b4a6f76292e6b2e2444fb369ac1e446874fd2851732c4d760c", + "request_file_sha256": "b1838644b04dfd4f59ea36c09e9ae46d03f96e65842d5f992081f2c79ea42de6", + "receipt_file_sha256": "a0d028fea9aedbfd066fc27727f8f0aa97867bf3f31e1248cbc7286a27d14a85", + "receipt_body_text_sha256": "53b837cf910a3f7255bed1b90793d52546253373bda67ec332ba7f9a15615ae3", + "result_file_sha256": "6339bfc3a8c7c5365b3d444ec9928fe27e6a132032e7c64c5a5b885479a0be7a", + "raw_content_sha256": "6d2640aaf6fb64c2eacb3520af3cebfc49d073ff5ea8b2b60feb45f961029035", + "response_id": "20261014_2bfeab1d-f124-4a45-9a3f-8b8eec2ade52", + "receipt_status": 200, + "prompt_tokens": 1881, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0257342499971855 + } + } + }, + { + "history_id": "confirm-fresh-33-te-m0", + "case_id": "fresh-33-te", + "scenario": "fresh-33", + "model": "sarvam-105b", + "family": "completed_older_request", + "question": "VISIT-2133 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-33-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-33-1", + "role": "customer", + "content": "VISIT-2133ని 2027-05-04కి మార్చిన అసలు అభ్యర్థనను దాని అసలు గుర్తింపుతో చూడండి." + }, + { + "id": "note-33-2", + "role": "customer", + "content": "అది ఇప్పటికే పూర్తయింది. తర్వాత వేరే మార్పు దాన్ని 2027-06-03కి మార్చింది. ఈ పునరావృత అభ్యర్థనతో తర్వాతి మార్పును వెనక్కి తిప్పవద్దు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The exact older request has a matching prior confirmed receipt, but a later distinct change leaves VISIT-2133 at 2027-06-03. Replaying the older request must not revert it; the truthful status is already_applied.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "already_applied", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-33", + "id": "VISIT-2133", + "date": "2027-06-03", + "version": 7, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-33", + "id": "VISIT-2133", + "date": "2027-06-03", + "version": 7, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2133", + "date": "2027-05-04", + "expected_version": 5, + "idempotency_key": "REQUEST-33-CHANGE-23", + "approval_id": "AUTH-33-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-04", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2133", + "date": "2027-06-03", + "expected_version": 6, + "idempotency_key": "REQUEST-33-LATER-23", + "approval_id": "AUTH-33-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-03", + "version": 7 + } + } + ], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "03f68cce95206c21951769f89b2705e730d78f1bb1e00019183c0fabe74e2e59", + "case_sha256": "e665b97d2d5c67ea41377122a77dd31fb1cbb8d11c5c5c6ea006f5eba09079c9", + "gold_label_sha256": "b9f65c9ee94fc6764c500821f48683e2a1c1c9412efc4dc43ca528aaa5f2b999", + "reports": { + "chronological": { + "id": "confirm-fresh-33-te-m0-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-03\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "3f881c3cebe8b5cacf33552daab7810ad2f9bb9766219d75e8b7bbc67095b029", + "request_file_sha256": "d3cdd4d90a63811dec5c655224daf118a673ef025060c2292832fd30b5340940", + "receipt_file_sha256": "f6139fe66e155f090fc826d7a0b35e577b343dca56f4dfb5eca25573e7dcb586", + "receipt_body_text_sha256": "22e4a6f881c890f841aea38cf3769ef86077a242acba690c77fdf0f92ccd48d6", + "result_file_sha256": "c1553fd0523b1529616e1962bbf6812c5dd94faf93dff31d113956ebebf274ad", + "raw_content_sha256": "6b7d837755fee0f4384fc2b064b5188c55989832ef709b05dbeab7ee721fed6c", + "response_id": "20261014_c1162ac5-2c24-4300-8a81-eca1ef7686dd", + "receipt_status": 200, + "prompt_tokens": 1815, + "completion_tokens": 1676, + "reasoning_tokens": 1644, + "elapsed_seconds": 9.552738250000402 + }, + "grouped_lossless": { + "id": "confirm-fresh-33-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-03\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "04664a5988d71c610bb70432f8cdd935c9e3062e4bf1eaa65fa1eeac4afc4477", + "request_file_sha256": "68ad7573ffab8bdf42e971dcce5b9faaf65edbb18013eae4d738f1e357bad0ac", + "receipt_file_sha256": "0978c52f6f5048446404cc73babcfbec19ac31033d0ebda3abe0778036973c36", + "receipt_body_text_sha256": "d71894c5dadbdd13c5a778b3eec55023c36e51df9716cbacddf49b9b3841f1af", + "result_file_sha256": "c1553fd0523b1529616e1962bbf6812c5dd94faf93dff31d113956ebebf274ad", + "raw_content_sha256": "6b7d837755fee0f4384fc2b064b5188c55989832ef709b05dbeab7ee721fed6c", + "response_id": "20261014_67b14c9f-ed81-48b2-ac51-f4d1a2b23b06", + "receipt_status": 200, + "prompt_tokens": 2343, + "completion_tokens": 1411, + "reasoning_tokens": 1379, + "elapsed_seconds": 9.608731583997724 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-33-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-03\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "e22677b6cfc7ba473752fb42cfcc2d97f8677292c3a146e41656a93ba7582076", + "request_file_sha256": "89975aa11eca7b0435d5c16df60a7336c5f25ea6ba3b0e2113642dc21e78ed25", + "receipt_file_sha256": "3f41a05585fdabdd99a616e4e7bcb2c2902b351b60e82bc17b39e7953d72c0c8", + "receipt_body_text_sha256": "e568008198a3038acb94f4c1d14bd77d3c647ada95951048ecb5d5378f107f41", + "result_file_sha256": "c1553fd0523b1529616e1962bbf6812c5dd94faf93dff31d113956ebebf274ad", + "raw_content_sha256": "6b7d837755fee0f4384fc2b064b5188c55989832ef709b05dbeab7ee721fed6c", + "response_id": "20261014_d5b72166-01ce-405a-8eaf-4a588dc157e4", + "receipt_status": 200, + "prompt_tokens": 2070, + "completion_tokens": 1583, + "reasoning_tokens": 1551, + "elapsed_seconds": 7.178025500004878 + } + } + }, + { + "history_id": "confirm-fresh-33-te-m1", + "case_id": "fresh-33-te", + "scenario": "fresh-33", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "question": "VISIT-2133 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-33-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-33-1", + "role": "customer", + "content": "VISIT-2133ని 2027-05-04కి మార్చిన అసలు అభ్యర్థనను దాని అసలు గుర్తింపుతో చూడండి." + }, + { + "id": "note-33-2", + "role": "customer", + "content": "అది ఇప్పటికే పూర్తయింది. తర్వాత వేరే మార్పు దాన్ని 2027-06-03కి మార్చింది. ఈ పునరావృత అభ్యర్థనతో తర్వాతి మార్పును వెనక్కి తిప్పవద్దు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The exact older request has a matching prior confirmed receipt, but a later distinct change leaves VISIT-2133 at 2027-06-03. Replaying the older request must not revert it; the truthful status is already_applied.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "already_applied", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-33", + "id": "VISIT-2133", + "date": "2027-06-03", + "version": 7, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-33", + "id": "VISIT-2133", + "date": "2027-06-03", + "version": 7, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-bcc777a0e55f625b", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2133" + }, + "response": { + "date": "2027-06-03", + "version": 7, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2133", + "date": "2027-05-04", + "expected_version": 5, + "idempotency_key": "REQUEST-33-CHANGE-23", + "approval_id": "AUTH-33-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-04", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2133", + "date": "2027-06-03", + "expected_version": 6, + "idempotency_key": "REQUEST-33-LATER-23", + "approval_id": "AUTH-33-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-03", + "version": 7 + } + } + ], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "210b887c85df4bf199b15cab6cdf525c47bd82cca8377c0fc7c85fc1cc4c3f9b", + "case_sha256": "e665b97d2d5c67ea41377122a77dd31fb1cbb8d11c5c5c6ea006f5eba09079c9", + "gold_label_sha256": "b9f65c9ee94fc6764c500821f48683e2a1c1c9412efc4dc43ca528aaa5f2b999", + "reports": { + "chronological": { + "id": "confirm-fresh-33-te-m1-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-03\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "917986cda566ed9997b577d852f7150b0b6554dacfb6ba8860534e8ecbc65745", + "request_file_sha256": "e08011474cebf6774f08d9ad71211fe7b8ce77bbd0daddb1765b1ad509e17e71", + "receipt_file_sha256": "cd74270303af65c5f5247db599859a11d408efcfba1420e2e11ea034789f9de7", + "receipt_body_text_sha256": "edd240b88ec9369946e44d670b350bbdfd7206d0b1b17ae2039e174dabe67715", + "result_file_sha256": "84391b7e3176e5c1aed2756ac5fc41fe066cc25886a30c9f667429bd8fa83637", + "raw_content_sha256": "b5a9b2a643044e8e02fbaa24273ac400f027c86f26590128838323f2715e255e", + "response_id": "20261014_32f699da-e077-441c-8735-eff31d435ea7", + "receipt_status": 200, + "prompt_tokens": 1979, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.065939709005761 + }, + "grouped_lossless": { + "id": "confirm-fresh-33-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-03\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "d3c1dc2b9286ce439f3cd21fc9c20b892af1a3d9dfea22b52f2b835a5cf9d920", + "request_file_sha256": "14cb39eaa4338fd9db01b32618591e12591a22d364d67e95f28aed3bb2aed75d", + "receipt_file_sha256": "fad76efe5e0e4df2cda611e2d6eb4cc0b69e0084bfb88a05d67890398d7eef08", + "receipt_body_text_sha256": "72f2a4b5abb893765357281d7cd74e4ffdc23ca08c32ee9548af0af730521c84", + "result_file_sha256": "0e529979621b57ba683527909c61a095b19f49f46d26c9741184bd4c71571e88", + "raw_content_sha256": "0d5919b20e564dc0d5fbc550dc3a613bc37b60a64f55e1b1d48adadf45f1ce1a", + "response_id": "20261014_2a3b1113-ed4b-4460-8c53-4c5ce08a0f21", + "receipt_status": 200, + "prompt_tokens": 2656, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0094782920059515 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-33-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-03\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-03", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "63582f020cff958aff44f687871e48796f21c4deaeeac21ac10cf50ccd6414da", + "request_file_sha256": "2ed4d565d7c62f9c4ae92dfe4a19dfcd51887d1ed0af7ad69a7c98deee67aff8", + "receipt_file_sha256": "1b9bb9dffac9e413ec4f5c394ad460cb662cce5cb2cb5d89288419f70758cec6", + "receipt_body_text_sha256": "f3042b1bfb71f4d288e112606cc80184370f6405da3058e98f6b85421744bb23", + "result_file_sha256": "84391b7e3176e5c1aed2756ac5fc41fe066cc25886a30c9f667429bd8fa83637", + "raw_content_sha256": "b5a9b2a643044e8e02fbaa24273ac400f027c86f26590128838323f2715e255e", + "response_id": "20261014_14bdc04c-c1b5-447b-8bac-c84503634c16", + "receipt_status": 200, + "prompt_tokens": 2359, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.029099874998792 + } + } + }, + { + "history_id": "confirm-fresh-34-te-m0", + "case_id": "fresh-34-te", + "scenario": "fresh-34", + "model": "sarvam-105b", + "family": "completed_older_request", + "question": "VISIT-2134 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-34-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. VISIT-2134ని 2027-05-05కి మార్చిన అసలు అభ్యర్థనను దాని అసలు గుర్తింపుతో చూడండి. అది ఇప్పటికే పూర్తయింది. తర్వాత వేరే మార్పు దాన్ని 2027-06-04కి మార్చింది. ఈ పునరావృత అభ్యర్థనతో తర్వాతి మార్పును వెనక్కి తిప్పవద్దు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The exact older request has a matching prior confirmed receipt, but a later distinct change leaves VISIT-2134 at 2027-06-04. Replaying the older request must not revert it; the truthful status is already_applied.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "already_applied", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-34", + "id": "VISIT-2134", + "date": "2027-06-04", + "version": 7, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-34", + "id": "VISIT-2134", + "date": "2027-06-04", + "version": 7, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a4554a2236e137b9", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2134" + }, + "response": { + "date": "2027-06-04", + "version": 7, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2134", + "date": "2027-05-05", + "expected_version": 5, + "idempotency_key": "REQUEST-34-CHANGE-23", + "approval_id": "AUTH-34-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-05", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2134", + "date": "2027-06-04", + "expected_version": 6, + "idempotency_key": "REQUEST-34-LATER-23", + "approval_id": "AUTH-34-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-04", + "version": 7 + } + } + ], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "8dd0bb5a1995db5035fe4c4e7faa9aea46e07bba701d1af6c54ebef9117165a8", + "case_sha256": "eebd25e788831623a6a2e3d8597c8fe2c688309e33674fc1ab8ec40937199e86", + "gold_label_sha256": "4354b6822986ff26e4e0d7ca27e394a301c03f271a281d55c6028e7c0b2e65c8", + "reports": { + "chronological": { + "id": "confirm-fresh-34-te-m0-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_performed\",\"answer\":\"2027-06-04\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_performed", + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_performed conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "6e07d04558d795f7a382853365c44b43829d767e343727b3f2903d855fe8a422", + "request_file_sha256": "98580c8134d688f3a08d855fa2bbc9ca630fb8aa1122e70003935bb3f9e83910", + "receipt_file_sha256": "fea7a582ae1f931d88cacfa74a7d5b40cacd75853e0e09b4d7b9794ae7340de9", + "receipt_body_text_sha256": "edb2e366521e8f97a2d001e57edc81dbc72bce5a0400a59eb53c49daa736c87d", + "result_file_sha256": "e06779a9cb208c2b4c218b7b33cb7b15f6617556fb3cb9cb92da244af97c2080", + "raw_content_sha256": "b6c41db1404ae24ceb4c157bac584d8a6eb112ca63ea75ab1e1815508b60ce23", + "response_id": "20261014_278f665c-9098-48c6-9637-ed1423607327", + "receipt_status": 200, + "prompt_tokens": 1837, + "completion_tokens": 3244, + "reasoning_tokens": 3212, + "elapsed_seconds": 13.686934375000419 + }, + "grouped_lossless": { + "id": "confirm-fresh-34-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-04\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "5bc011114a8132580f544f42df4e842e8868e38a85bb9b7be3c55803a1f22c98", + "request_file_sha256": "ef392f8820c1191236d4f7ccb30ce469f881547420981ac0782af9f4468c13cc", + "receipt_file_sha256": "d74a26b0a30b26c79ddb0838a96ac794b7bc91f3201a792a6722d7a1ed2232de", + "receipt_body_text_sha256": "796aececa25c9e76e24cb3d662f1fad3f4308225ec865c8177e5429a95dc7270", + "result_file_sha256": "abd46c530745df8728e321ec83dc35accf4d0d8c2c4080a3ad91e1f119565685", + "raw_content_sha256": "6586f51a831b7caa739d8a82e7b4fa0ab9b8b951e604255d1abf7cf47d787c1f", + "response_id": "20261014_86868935-a9ba-4f63-afbb-0a02c1032146", + "receipt_status": 200, + "prompt_tokens": 2407, + "completion_tokens": 1538, + "reasoning_tokens": 1506, + "elapsed_seconds": 6.980180041995482 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-34-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-06-04\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_performed", + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_performed conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "6c05fbd6a1290543314e8cb89116f0d5d27e4e291d76a1f822747f38a34a3e1b", + "request_file_sha256": "dd75db89662f92df9a00a557df636922d5f22affd97b5f67bc901f283f1f2a26", + "receipt_file_sha256": "16cfe45742468d1846f7d97bd10ca4986f5db87b6e327504151330f85162e410", + "receipt_body_text_sha256": "849ebae2632d5373e44627bfb8ff413a4333e48e8a5a7a8aca735d0ec20224e4", + "result_file_sha256": "724654f4a7af6c72796792e733aac468132ed9975139bc88bbc067193c76376a", + "raw_content_sha256": "1f9aae50f890487639284f3859153ccbb11bf0e46aae9b20c3faaaed2e29b79d", + "response_id": "20261014_d4ae937f-555c-4464-bcf1-ee1cdc00df8b", + "receipt_status": 200, + "prompt_tokens": 2216, + "completion_tokens": 5294, + "reasoning_tokens": 5250, + "elapsed_seconds": 24.093186958998558 + } + } + }, + { + "history_id": "confirm-fresh-34-te-m1", + "case_id": "fresh-34-te", + "scenario": "fresh-34", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "question": "VISIT-2134 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-34-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. VISIT-2134ని 2027-05-05కి మార్చిన అసలు అభ్యర్థనను దాని అసలు గుర్తింపుతో చూడండి. అది ఇప్పటికే పూర్తయింది. తర్వాత వేరే మార్పు దాన్ని 2027-06-04కి మార్చింది. ఈ పునరావృత అభ్యర్థనతో తర్వాతి మార్పును వెనక్కి తిప్పవద్దు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The exact older request has a matching prior confirmed receipt, but a later distinct change leaves VISIT-2134 at 2027-06-04. Replaying the older request must not revert it; the truthful status is already_applied.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "already_applied", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-34", + "id": "VISIT-2134", + "date": "2027-06-04", + "version": 7, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-34", + "id": "VISIT-2134", + "date": "2027-06-04", + "version": 7, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-8876f6872cb1a4bd", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2134" + }, + "response": { + "date": "2027-06-04", + "version": 7, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2134", + "date": "2027-05-05", + "expected_version": 5, + "idempotency_key": "REQUEST-34-CHANGE-23", + "approval_id": "AUTH-34-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-05", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2134", + "date": "2027-06-04", + "expected_version": 6, + "idempotency_key": "REQUEST-34-LATER-23", + "approval_id": "AUTH-34-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-04", + "version": 7 + } + } + ], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "4e688f86e99ca39a217396abf96dd9f8dc9c07a7566c83d39bb1542e4f002376", + "case_sha256": "eebd25e788831623a6a2e3d8597c8fe2c688309e33674fc1ab8ec40937199e86", + "gold_label_sha256": "4354b6822986ff26e4e0d7ca27e394a301c03f271a281d55c6028e7c0b2e65c8", + "reports": { + "chronological": { + "id": "confirm-fresh-34-te-m1-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-04\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "a6ee7b55ccff20fedbabc2448d55354ecf1a05dfffd2df9e740325fab87e1939", + "request_file_sha256": "a98b59849652e5ff1d63811064afdd12f667768fe849a258a795913c43b45dc6", + "receipt_file_sha256": "e5d9eb5dd4b43b30459dbda21628398eff36d686500be8f885e6e29a5d51e603", + "receipt_body_text_sha256": "9c6a0546c0b593ab1451eaa95447667c4981046336adbb3c3efec9512beb5b02", + "result_file_sha256": "fb15a37da4fb1c488e9b16ecb891c11ca2d0d116496c521e1c600d6a55309e25", + "raw_content_sha256": "d37ecd59c05b45200bc0a7393dd814c4e477d5816ec0bff4fe8eae3cc3ff484a", + "response_id": "20261014_cc246f63-af41-4bc3-a7cd-130789a55da6", + "receipt_status": 200, + "prompt_tokens": 1946, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9830327079980634 + }, + "grouped_lossless": { + "id": "confirm-fresh-34-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-04\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "6a184ccfea6590b48fa2ba409fa9f5526545c53a4836f192bfc5481e41a41118", + "request_file_sha256": "7f17c27b99a573b3cefc20275a3c8a848d0eabd73c22d5cf4ef854f5a6120c52", + "receipt_file_sha256": "3012e75d31ee63f7417032fa207685ed008f7c79ecabba1bb3dd7313b418fb54", + "receipt_body_text_sha256": "9bc4dd79d34d08240700d32c99bbb7e5698866006ae346f7b1ebbf11d3de33cb", + "result_file_sha256": "fb15a37da4fb1c488e9b16ecb891c11ca2d0d116496c521e1c600d6a55309e25", + "raw_content_sha256": "d37ecd59c05b45200bc0a7393dd814c4e477d5816ec0bff4fe8eae3cc3ff484a", + "response_id": "20261014_6eb0cc2a-b006-480d-9886-8434d89f650d", + "receipt_status": 200, + "prompt_tokens": 2518, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.064101041003596 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-34-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-04\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-04", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "dc86c952623299f58ac797675d258846b94bb7d97075178a52f0ef7d158c4366", + "request_file_sha256": "7910c11f2c726bddc789f993f6f04e39cc451f9a7149409ae877cd135f0f2491", + "receipt_file_sha256": "4e2a457d5951675cdebf8d73d76a8c1f62c296e0ec28fc6c53603215b02355ce", + "receipt_body_text_sha256": "8992f8adc3716667ca92455e9438f132e932ecd95a23a61166ddde3bbe48c3b8", + "result_file_sha256": "042713423da807bcfef49b2ccab9db9d7aee0d0c696115936fca95c315d2e3bb", + "raw_content_sha256": "0dede5205ccb97ca95a65809101dd03fc5a8724efaeaca1d97fffdd056f35a2f", + "response_id": "20261014_96dea3e7-d51e-4fd1-a5be-8b3d587d6eff", + "receipt_status": 200, + "prompt_tokens": 2218, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9962496660009492 + } + } + }, + { + "history_id": "confirm-fresh-35-te-m0", + "case_id": "fresh-35-te", + "scenario": "fresh-35", + "model": "sarvam-105b", + "family": "completed_older_request", + "question": "VISIT-2135 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-35-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-35-1", + "role": "customer", + "content": "VISIT-2135ని 2027-05-06కి మార్చిన అసలు అభ్యర్థనను దాని అసలు గుర్తింపుతో చూడండి." + }, + { + "id": "note-35-2", + "role": "customer", + "content": "అది ఇప్పటికే పూర్తయింది. తర్వాత వేరే మార్పు దాన్ని 2027-06-05కి మార్చింది. ఈ పునరావృత అభ్యర్థనతో తర్వాతి మార్పును వెనక్కి తిప్పవద్దు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The exact older request has a matching prior confirmed receipt, but a later distinct change leaves VISIT-2135 at 2027-06-05. Replaying the older request must not revert it; the truthful status is already_applied.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "already_applied", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-35", + "id": "VISIT-2135", + "date": "2027-06-05", + "version": 7, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-35", + "id": "VISIT-2135", + "date": "2027-06-05", + "version": 7, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2135", + "date": "2027-05-06", + "expected_version": 5, + "idempotency_key": "REQUEST-35-CHANGE-23", + "approval_id": "AUTH-35-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-06", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2135", + "date": "2027-06-05", + "expected_version": 6, + "idempotency_key": "REQUEST-35-LATER-23", + "approval_id": "AUTH-35-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-05", + "version": 7 + } + } + ], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "8718f5e6b21c1eef12243ec244e34a2d0da7b7bcbe5f13f09582d1a8c6614599", + "case_sha256": "67789f9a152b3511389dcb40d034042d0098f9e3a3127f90df231b4bb82362cd", + "gold_label_sha256": "6938a1f87dc9345840a0dc9b55c516d24c43849187a52bf543eb71d0b274c2c5", + "reports": { + "chronological": { + "id": "confirm-fresh-35-te-m0-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-05\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "80125d178d0901bf783d5334b7e465aa528a559e0b17a2c428c7fb1dccca377b", + "request_file_sha256": "c217ed8f84afb5fbf8fe26e770c66a81f52fdfa3b27f43172ba0cd0452b6ca66", + "receipt_file_sha256": "b6b4c46e23e90cfb76801ded4e40c5f19889f5ea7ca8ee8eda872445a816e625", + "receipt_body_text_sha256": "d913a8045994a7e79b6d9a954eca9deede95a259afdc6735fdc2ccae8ec5b018", + "result_file_sha256": "b31d0f9b5f8fe5765afbcb1eaf44d89c8c5492e0a28b6d2d4ff5fe02f1c09f85", + "raw_content_sha256": "a7a213c208ae83606f4551a683c7a4409c60837bc26f45ac19a6b96446e83525", + "response_id": "20261014_42ac34c0-9a27-44a1-8b15-8f9c38fbf652", + "receipt_status": 200, + "prompt_tokens": 1863, + "completion_tokens": 2694, + "reasoning_tokens": 2662, + "elapsed_seconds": 12.963635249994695 + }, + "grouped_lossless": { + "id": "confirm-fresh-35-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\": \"not_requested\", \"answer\": \"2027-06-05\", \"needs_clarification\": false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "abeb2f075b50adfe88e2522ac379a6bbe9c24ed82d98d8dc65fc3b0f7065dd70", + "request_file_sha256": "1738ee538dfd63813e75e57f9ba8a89624d6acbe365352fc2458c4b6ed569fc1", + "receipt_file_sha256": "f3630562f524845d21b3a8c4c6bc202a498da5c5c9905e4fbe3b904c21d59741", + "receipt_body_text_sha256": "1f1f6f90d1aed6177f0b924e71748b7656f6613b66241ceda11086aaf4d0c13e", + "result_file_sha256": "783d06630a3c4f84e56203d8144a5e4ccdbb51803d241e952be528d246e6346c", + "raw_content_sha256": "119943d853ec20d1fbebd8494e527c9f0412a40f3b41b6988593e958e52ea69d", + "response_id": "20261014_6f43d009-23fe-4b0f-917b-69439dd2d6f3", + "receipt_status": 200, + "prompt_tokens": 2400, + "completion_tokens": 4024, + "reasoning_tokens": 3988, + "elapsed_seconds": 19.921081625012448 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-35-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-06-05\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "94aa30fceb807032891bee1e4447774a2ff23deb81ea263bb1a35945213b3f03", + "request_file_sha256": "7b8ad2736375ba13ab1f83c5ce2029c52e5b6dcb4dad618dc659f8713330cd83", + "receipt_file_sha256": "a260c4bbf8988a3e8852f00125fc1a163565c73fed4b5eac490afbed1d2edf9c", + "receipt_body_text_sha256": "e461964446ce7b00cd24887639f306757b90338d086ddb996fb7832757cc2416", + "result_file_sha256": "296cb3c547ead0b0a5b30d3e7e77c560bc5d057059bb3ea526c0c2bd94e0c71e", + "raw_content_sha256": "0249c0fa5de0e696a7af9d1e4abe253d1025cd3505514ce0606fed5166205f00", + "response_id": "20261014_ed977408-6948-4ca7-ba35-291ba97ea5c6", + "receipt_status": 200, + "prompt_tokens": 2081, + "completion_tokens": 6210, + "reasoning_tokens": 6178, + "elapsed_seconds": 27.812365375008085 + } + } + }, + { + "history_id": "confirm-fresh-35-te-m1", + "case_id": "fresh-35-te", + "scenario": "fresh-35", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "question": "VISIT-2135 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-35-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-35-1", + "role": "customer", + "content": "VISIT-2135ని 2027-05-06కి మార్చిన అసలు అభ్యర్థనను దాని అసలు గుర్తింపుతో చూడండి." + }, + { + "id": "note-35-2", + "role": "customer", + "content": "అది ఇప్పటికే పూర్తయింది. తర్వాత వేరే మార్పు దాన్ని 2027-06-05కి మార్చింది. ఈ పునరావృత అభ్యర్థనతో తర్వాతి మార్పును వెనక్కి తిప్పవద్దు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The exact older request has a matching prior confirmed receipt, but a later distinct change leaves VISIT-2135 at 2027-06-05. Replaying the older request must not revert it; the truthful status is already_applied.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "already_applied", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-35", + "id": "VISIT-2135", + "date": "2027-06-05", + "version": 7, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-35", + "id": "VISIT-2135", + "date": "2027-06-05", + "version": 7, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-909746a667a3ab7c", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2135" + }, + "response": { + "date": "2027-06-05", + "version": 7, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2135", + "date": "2027-05-06", + "expected_version": 5, + "idempotency_key": "REQUEST-35-CHANGE-23", + "approval_id": "AUTH-35-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-06", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2135", + "date": "2027-06-05", + "expected_version": 6, + "idempotency_key": "REQUEST-35-LATER-23", + "approval_id": "AUTH-35-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-05", + "version": 7 + } + } + ], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "bdc6e209d6d1bb5ff6034737ad098e5ee2bc0b0a386fa0c24bd853f63d625cd4", + "case_sha256": "67789f9a152b3511389dcb40d034042d0098f9e3a3127f90df231b4bb82362cd", + "gold_label_sha256": "6938a1f87dc9345840a0dc9b55c516d24c43849187a52bf543eb71d0b274c2c5", + "reports": { + "chronological": { + "id": "confirm-fresh-35-te-m1-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-05\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "2e1a7e64c662adb40545aa2668a02ce336b355a2679b4da564e3d8ad3477ddc6", + "request_file_sha256": "2ff169ff8fd5756cb4c688dc7bfed4c8f9e94d6d95a91b61c2609ebe9f986b21", + "receipt_file_sha256": "23e87f8614af40713e3e51bddf0ad451cff813739ce65ddd1477fb9f75c37bb5", + "receipt_body_text_sha256": "466728d108d2d49e62808e46f1f78eafb6166d469b7e9f9c256b5e5d5ade7b58", + "result_file_sha256": "a10e59f23e1bcbeb10d0bb5925482d8103b4a143246bb28f303d672b12eeb2f2", + "raw_content_sha256": "baee28406332678362f125cfdb084fb35a70df7dda2b9c89c9ec8091f79aede6", + "response_id": "20261014_5168f452-54b0-4998-969d-5b8f49a0b4d5", + "receipt_status": 200, + "prompt_tokens": 1986, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0494226250011707 + }, + "grouped_lossless": { + "id": "confirm-fresh-35-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-05\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "d9269685a59c53a08a0c5e385e87f064b083eea9e158cdd8e0b76ed327a04fa7", + "request_file_sha256": "3f7fa9e86a11e72023da7ed0b9f1e9baa2b6c5bbc74f51eaf8c583560d1500e5", + "receipt_file_sha256": "aa70998152d2310679541173469dc2f2bd02affa726e6c244343987ad7f11e10", + "receipt_body_text_sha256": "1e550f723e524bfab7733f28cd8a60365168c40db464ea93cedf35a8c5ac7914", + "result_file_sha256": "10dca089506ffd53f9cf8d47ddf457b0596b2a838a3856eebeac46979cd679d9", + "raw_content_sha256": "5ccdaa467bf10798802cf3b20863dd51a0c1536186cc3eecf0a779a2bd89cb44", + "response_id": "20261014_1a69324b-5c69-4741-b72e-dcc0f754c740", + "receipt_status": 200, + "prompt_tokens": 2661, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9930545829993207 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-35-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-05\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-05", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "98834bf698aeaa69b91840d92c272f493e0b8c98d1e849773e553bea5dc6b43c", + "request_file_sha256": "459d7433cb1cdae2ab8929c4c973d9bfdebd74196c30bf6fca1c9a201b9d2982", + "receipt_file_sha256": "f13dd88425ee7f54cbb4c4d052163a0713b8134f0eb197b732924004780a476d", + "receipt_body_text_sha256": "171e4dedba51f19ee86815b5efb1674742aaa1a04a5e844da0a3bade5ee86d05", + "result_file_sha256": "a10e59f23e1bcbeb10d0bb5925482d8103b4a143246bb28f303d672b12eeb2f2", + "raw_content_sha256": "baee28406332678362f125cfdb084fb35a70df7dda2b9c89c9ec8091f79aede6", + "response_id": "20261014_4823b94a-f4b9-4533-bc4f-f676cad05f99", + "receipt_status": 200, + "prompt_tokens": 2361, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9639243749988964 + } + } + }, + { + "history_id": "confirm-fresh-36-te-m0", + "case_id": "fresh-36-te", + "scenario": "fresh-36", + "model": "sarvam-105b", + "family": "completed_older_request", + "question": "VISIT-2136 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-36-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. VISIT-2136ని 2027-05-07కి మార్చిన అసలు అభ్యర్థనను దాని అసలు గుర్తింపుతో చూడండి. అది ఇప్పటికే పూర్తయింది. తర్వాత వేరే మార్పు దాన్ని 2027-06-06కి మార్చింది. ఈ పునరావృత అభ్యర్థనతో తర్వాతి మార్పును వెనక్కి తిప్పవద్దు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The exact older request has a matching prior confirmed receipt, but a later distinct change leaves VISIT-2136 at 2027-06-06. Replaying the older request must not revert it; the truthful status is already_applied.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "already_applied", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-36", + "id": "VISIT-2136", + "date": "2027-06-06", + "version": 7, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-36", + "id": "VISIT-2136", + "date": "2027-06-06", + "version": 7, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2136", + "date": "2027-05-07", + "expected_version": 5, + "idempotency_key": "REQUEST-36-CHANGE-23", + "approval_id": "AUTH-36-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-07", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2136", + "date": "2027-06-06", + "expected_version": 6, + "idempotency_key": "REQUEST-36-LATER-23", + "approval_id": "AUTH-36-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-06", + "version": 7 + } + } + ], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "5ee0958da22ed2e917c0fb98fce93fe0f7ec24c9514e386d73af0f22821f6fc2", + "case_sha256": "9beebe22de6c7e29b6bd30df78fd585ec4bf7a0c5fe4166e0c7a67419372ae43", + "gold_label_sha256": "34697e6663cd49fdbeb58bfa41b7ec5c840da2e88f26994785624fa9bdfc826c", + "reports": { + "chronological": { + "id": "confirm-fresh-36-te-m0-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-06\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "a3ca18af6dfdc8b32691d1f46f197fa0d5338347cf2798d4ed475b429cb679dd", + "request_file_sha256": "2b6c19ac2d3aeea86ead17dbffdb02f6080e12fb568edf5670b6d9d95158260b", + "receipt_file_sha256": "f33ebbb90d0431f8d41f8d9d1b450fb96522b837793a11f7709f786fb54551cf", + "receipt_body_text_sha256": "4c9a85adf5f7f82d41ccb723f77174c7eb17ba68470cd22703627fd98f6e9c59", + "result_file_sha256": "f09ee14139dffa4254eb8299e58575fdc38b4acb79393af281e81167c8fe2c0b", + "raw_content_sha256": "2803662ff40dd4f63615eb045495ce5c39d4468bff1e77d7ea10ac817822af70", + "response_id": "20261014_2c09b4ad-0f61-432c-bf0c-14e369a1fd5a", + "receipt_status": 200, + "prompt_tokens": 1769, + "completion_tokens": 1646, + "reasoning_tokens": 1614, + "elapsed_seconds": 7.960662334007793 + }, + "grouped_lossless": { + "id": "confirm-fresh-36-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-06\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "af516f8ecd745a5f4c95d6ce217b412275016d4e607b38d1bdab8a91e23620e7", + "request_file_sha256": "2dfed078248f641a214655e98a1f90b66677e490e090e505405c758182a6c139", + "receipt_file_sha256": "360d069620f040ce15edd4f719c7d86f86ca8ae3dea7633feef6198fc33ab5b0", + "receipt_body_text_sha256": "6b479a242d4fb35e1650b5b076b9c0c9c5349f70446a6f5a57be7c263e868b42", + "result_file_sha256": "f09ee14139dffa4254eb8299e58575fdc38b4acb79393af281e81167c8fe2c0b", + "raw_content_sha256": "2803662ff40dd4f63615eb045495ce5c39d4468bff1e77d7ea10ac817822af70", + "response_id": "20261014_b7d54078-fc38-4e7a-a871-70b1ac8e336d", + "receipt_status": 200, + "prompt_tokens": 2199, + "completion_tokens": 2781, + "reasoning_tokens": 2749, + "elapsed_seconds": 11.999567209000816 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-36-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-06-06\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "bcb6a6876302f3da073ab1e1c38892aec331cccefc007a976a80d2b4740758b5", + "request_file_sha256": "abea862c490898fd7b295da1436f133b3c70b889800421614b44e89adbe97fc9", + "receipt_file_sha256": "f21f731f5ef2326733bdfef4fe840c4171623a40bf9014fcdbba306242295c8c", + "receipt_body_text_sha256": "62576ec79a52fa9ace7702e3a9a2ab7c7daf5ed9e7077c1bbed4b301a536064f", + "result_file_sha256": "f09ee14139dffa4254eb8299e58575fdc38b4acb79393af281e81167c8fe2c0b", + "raw_content_sha256": "2803662ff40dd4f63615eb045495ce5c39d4468bff1e77d7ea10ac817822af70", + "response_id": "20261014_d124164e-0937-43d7-847d-349cce36e92d", + "receipt_status": 200, + "prompt_tokens": 1936, + "completion_tokens": 4093, + "reasoning_tokens": 4061, + "elapsed_seconds": 17.289486125009716 + } + } + }, + { + "history_id": "confirm-fresh-36-te-m1", + "case_id": "fresh-36-te", + "scenario": "fresh-36", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "question": "VISIT-2136 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-36-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. VISIT-2136ని 2027-05-07కి మార్చిన అసలు అభ్యర్థనను దాని అసలు గుర్తింపుతో చూడండి. అది ఇప్పటికే పూర్తయింది. తర్వాత వేరే మార్పు దాన్ని 2027-06-06కి మార్చింది. ఈ పునరావృత అభ్యర్థనతో తర్వాతి మార్పును వెనక్కి తిప్పవద్దు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The exact older request has a matching prior confirmed receipt, but a later distinct change leaves VISIT-2136 at 2027-06-06. Replaying the older request must not revert it; the truthful status is already_applied.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "already_applied", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-36", + "id": "VISIT-2136", + "date": "2027-06-06", + "version": 7, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-36", + "id": "VISIT-2136", + "date": "2027-06-06", + "version": 7, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a54de1a387dfdf0a", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2136" + }, + "response": { + "date": "2027-06-06", + "version": 7, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2136", + "date": "2027-05-07", + "expected_version": 5, + "idempotency_key": "REQUEST-36-CHANGE-23", + "approval_id": "AUTH-36-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-07", + "version": 6 + } + }, + { + "arguments": { + "appointment_id": "VISIT-2136", + "date": "2027-06-06", + "expected_version": 6, + "idempotency_key": "REQUEST-36-LATER-23", + "approval_id": "AUTH-36-LATER-GRANT-23" + }, + "receipt": { + "status": "confirmed", + "date": "2027-06-06", + "version": 7 + } + } + ], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "fee6c3c3c7557c5080caa1924dd5be68975b1b0b73563bd34df42dd463f66bdc", + "case_sha256": "9beebe22de6c7e29b6bd30df78fd585ec4bf7a0c5fe4166e0c7a67419372ae43", + "gold_label_sha256": "34697e6663cd49fdbeb58bfa41b7ec5c840da2e88f26994785624fa9bdfc826c", + "reports": { + "chronological": { + "id": "confirm-fresh-36-te-m1-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-06\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "3b953b44aedd86b7608d6b1a338d8d5a853fa865b0385cce29e7bb2043358019", + "request_file_sha256": "f2286af0f0525d6fd257e0caac09096e178a1fa8049220d87a83e19a5f8a4ff4", + "receipt_file_sha256": "844ffcb34cffa289068555fbb548e7f46b58242757db36d3792c58dedd30bc54", + "receipt_body_text_sha256": "3d3a8b5c46c0de14d5a4d33d972911a6b6e1974b2dd4d12d57c37dfc4628d526", + "result_file_sha256": "5766d8a3d2817eb1b9e489c5c75934249fc394e5c0209fb689239f6b03244dca", + "raw_content_sha256": "efc6a4077028e288334cb3a79d28cec7f9704053928331652c0ff2cd591ea443", + "response_id": "20261014_199de64f-18ba-4cfd-894c-37b83605caa9", + "receipt_status": 200, + "prompt_tokens": 2052, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0162083329923917 + }, + "grouped_lossless": { + "id": "confirm-fresh-36-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-06-06\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "706ee00239572cbaf4e4ee56b6b1dfc8d0e12cc14634d3407053813e04254251", + "request_file_sha256": "ad6412b8abf218fa3f9e294c245ab0bd9762eeb91f7658ac89a8cd1396e2d759", + "receipt_file_sha256": "705d0d7f7a35976aee6e7e9a48936685b024dca3e8e721725b1404a5f2f041eb", + "receipt_body_text_sha256": "965e4dffb4553f5641c134f91f562b1b8d67755b1b52465c988626b5e5c75a10", + "result_file_sha256": "5766d8a3d2817eb1b9e489c5c75934249fc394e5c0209fb689239f6b03244dca", + "raw_content_sha256": "efc6a4077028e288334cb3a79d28cec7f9704053928331652c0ff2cd591ea443", + "response_id": "20261014_fecf3b4a-ac63-4782-90a9-a34fc4e84dca", + "receipt_status": 200, + "prompt_tokens": 2627, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9634353330038721 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-36-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-06-06\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-06-06", + "needs_clarification": false + }, + "expected_action_status": "already_applied", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied.", + "request_sha256": "7f916af4cc08583e8a2e0083ea8ab5d1701351658845f887b3a8f4ce0b7e707b", + "request_file_sha256": "b38494afebdd642b05ddbd2802e2d8a50b788f840a8bbce0bf9b72759b327fc9", + "receipt_file_sha256": "9f6d748f702a9aafafecc490c3d98d4f6893bc21b4726dd7918732136eb84c14", + "receipt_body_text_sha256": "3499e818656c84993c177a7fd8d0b63a2d847deec93ae0ff08e71c923998dcc4", + "result_file_sha256": "807ff01b1b38bc10bcd9060bd0d3b1ae9c42d960b9a0482d8c03411278d3cf2d", + "raw_content_sha256": "861aec082087961b93d13c7f30faddb78998f3b9bc4544a583396aa8fae6587a", + "response_id": "20261014_c0045f24-d6a6-47c7-aea4-e33867cab735", + "receipt_status": 200, + "prompt_tokens": 2219, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9523697499971604 + } + } + }, + { + "history_id": "confirm-fresh-37-te-m0", + "case_id": "fresh-37-te", + "scenario": "fresh-37", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "question": "VISIT-2137 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-37-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-37-1", + "role": "customer", + "content": "ఇప్పుడు VISIT-2137 కోసం 2027-06-07 కావాలి." + }, + { + "id": "note-37-2", + "role": "customer", + "content": "ఇచ్చిన అభ్యర్థన గుర్తింపును ఇప్పటికే 2027-05-08కి పూర్తయిన మార్పు కోసం వాడారు. అదే గుర్తింపుతో వేరే మార్పు అమలు చేయవద్దు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The supplied request key was used for an earlier, different completed change. The new requested date was not applied in this native trace; VISIT-2137 remains at 2027-05-08. An old receipt for different arguments is not proof this new request was applied.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_performed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-37", + "id": "VISIT-2137", + "date": "2027-05-08", + "version": 6, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-37", + "id": "VISIT-2137", + "date": "2027-05-08", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2137", + "date": "2027-05-08", + "expected_version": 5, + "idempotency_key": "REQUEST-37-CHANGE-24", + "approval_id": "AUTH-37-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-08", + "version": 6 + } + } + ], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "80fdfdbbc0ca4ba6a9de004de2991147d0a48b44eb5251e4139bb2c15002ade5", + "case_sha256": "303f26bd131bd7150e2d5ee339e368beb2e4906722672d6f27213909484e7ccb", + "gold_label_sha256": "a12ae7610534c0e237d91ff9e50dcac4be086443e0a1cf5d0c2d0fc1566371db", + "reports": { + "chronological": { + "id": "confirm-fresh-37-te-m0-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-08\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "3c0a512541ccac84f3dbac400a4275d73f0c46a89a995026da3cab2ea3fdae61", + "request_file_sha256": "e6af78e8ea868d92b1e0114459ed6936e43468198f47e79fc82a1397ac3b83fb", + "receipt_file_sha256": "900ab2f1b4f8a073286b18f91e9a6b56274608ce3e5f67289f2d68ee18df8ae5", + "receipt_body_text_sha256": "d8fe207b7273858a17b8d6a2342f35ba9157483bbd30b423c0433a6e4d9f553d", + "result_file_sha256": "69dde94d76c4033be2860a97d246a801af8bfcec0b584aa6c9f0986a935603bb", + "raw_content_sha256": "b91cbcd7f19d9fbab62d34814d050c116d17f3c029abf11a8767079f0385702d", + "response_id": "20261014_c2453df0-e130-4474-9d60-f7da8194548b", + "receipt_status": 200, + "prompt_tokens": 1673, + "completion_tokens": 2243, + "reasoning_tokens": 2211, + "elapsed_seconds": 10.327790666997316 + }, + "grouped_lossless": { + "id": "confirm-fresh-37-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-08\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "91521c8ec80acc24ef85f8efc4f92fb6efa1d25776f8a1b12a966d993c47baa3", + "request_file_sha256": "6305ac338023e95fd4c45a23835d9ec61c8ac8fdd1c635107856b0c8827a0b37", + "receipt_file_sha256": "cf4a78c8aa56781d4e407ada2f33620a3b9bbb4042c92ae0f60de194ed15bfd5", + "receipt_body_text_sha256": "db359ee88aed0bea8ff973459a0cc7354fc7f39b82deb66073de5a9c358d26b3", + "result_file_sha256": "69dde94d76c4033be2860a97d246a801af8bfcec0b584aa6c9f0986a935603bb", + "raw_content_sha256": "b91cbcd7f19d9fbab62d34814d050c116d17f3c029abf11a8767079f0385702d", + "response_id": "20261014_4dca3ecd-bb54-4082-835a-b7eed03cb239", + "receipt_status": 200, + "prompt_tokens": 2209, + "completion_tokens": 2117, + "reasoning_tokens": 2085, + "elapsed_seconds": 9.33684879100474 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-37-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-08\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "7c0ff7edb56293c0871ad9f0dccc5e19b5b3bb49b9a58e270434bfa63b151d5a", + "request_file_sha256": "a51a781cb506d7b0ee941532a2d82917e33b8dc753d7a2d39ed732f0abbb4ecb", + "receipt_file_sha256": "d118b489f86bd4de14c5a0301ccd1d4de826fff85673ec2471aab25c5a6f401c", + "receipt_body_text_sha256": "fd9a91e5e641912047cd9e406e29f6d4192d4c308739bf2417aca51649062685", + "result_file_sha256": "aff48058a24de4a1db9e0fa8c903c6c318fd62f91bba7bbd8e5d3168a70bc729", + "raw_content_sha256": "2292869e10a5a50d27fb1494433690cd1b3ef9e542b822350d3bd16e394d5c9e", + "response_id": "20261014_919387a8-1cc0-46f3-96e1-df9f2f0dd129", + "receipt_status": 200, + "prompt_tokens": 1958, + "completion_tokens": 4962, + "reasoning_tokens": 4930, + "elapsed_seconds": 21.5542608340038 + } + } + }, + { + "history_id": "confirm-fresh-37-te-m1", + "case_id": "fresh-37-te", + "scenario": "fresh-37", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "question": "VISIT-2137 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-37-0", + "role": "customer", + "content": "నేను మీరా; ఇది నా బుకింగ్ గురించిన తదుపరి సందేశం." + }, + { + "id": "note-37-1", + "role": "customer", + "content": "ఇప్పుడు VISIT-2137 కోసం 2027-06-07 కావాలి." + }, + { + "id": "note-37-2", + "role": "customer", + "content": "ఇచ్చిన అభ్యర్థన గుర్తింపును ఇప్పటికే 2027-05-08కి పూర్తయిన మార్పు కోసం వాడారు. అదే గుర్తింపుతో వేరే మార్పు అమలు చేయవద్దు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The supplied request key was used for an earlier, different completed change. The new requested date was not applied in this native trace; VISIT-2137 remains at 2027-05-08. An old receipt for different arguments is not proof this new request was applied.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_performed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-37", + "id": "VISIT-2137", + "date": "2027-05-08", + "version": 6, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-37", + "id": "VISIT-2137", + "date": "2027-05-08", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9576fa67ecc4f935", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2137" + }, + "response": { + "date": "2027-05-08", + "version": 6, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2137", + "date": "2027-05-08", + "expected_version": 5, + "idempotency_key": "REQUEST-37-CHANGE-24", + "approval_id": "AUTH-37-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-08", + "version": 6 + } + } + ], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "a1e09933fda0a506f1e12df21a1c802649ecafc6f783befa17410f35c9c75e25", + "case_sha256": "303f26bd131bd7150e2d5ee339e368beb2e4906722672d6f27213909484e7ccb", + "gold_label_sha256": "a12ae7610534c0e237d91ff9e50dcac4be086443e0a1cf5d0c2d0fc1566371db", + "reports": { + "chronological": { + "id": "confirm-fresh-37-te-m1-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-05-08\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_performed", + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_performed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "76e6f1cb741398869500604d30df3be0d0867607891ebf9d50849e55adf8c722", + "request_file_sha256": "32d5095c51e212d089678902e5da2f4c60859489581816e97baff7c6e418ba55", + "receipt_file_sha256": "47e33e7e75e21d4fbd16636d41fbf5bb68437d4b15bb54c0dfbe7849d94ee664", + "receipt_body_text_sha256": "dd838878326ba94efbad21b9e75a4ccd92898681a9a09aac6617adc2b91a8020", + "result_file_sha256": "2c3fdbb6856d232762072d5920f57471f718b0f9b72563994c9e7d9b1e6852d7", + "raw_content_sha256": "f025c2199b98f91f9bd992c7a9c8f85467b1b8ebc43dea9ba9165d4afb289c52", + "response_id": "20261014_9b8d14a4-e60a-4880-aded-1e2d9652621f", + "receipt_status": 200, + "prompt_tokens": 1915, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.959741375001613 + }, + "grouped_lossless": { + "id": "confirm-fresh-37-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-05-08\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "rejected", + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status rejected conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "26917c6d2aa984cecf7b463ce3f0a599bf22257e1fd5f0768d59403eee753628", + "request_file_sha256": "521e63b0227bf05f39a94a16839cb7c16b19628b5fe929110474fbb71c37ca64", + "receipt_file_sha256": "3e5366d8c341d273faaaef6a4a1f85cc3186191255fedde7c5f157d62736d393", + "receipt_body_text_sha256": "a19becee582a8f7d70ea4edc1085ea66337a64cecb4511fcf66d5d9ef35e5b9d", + "result_file_sha256": "cfd9d0213e93bbf6f5cb1029acb085a29aded09fa4637c7b47c8f4cf1e2d4a44", + "raw_content_sha256": "4c84e6bba619478a332d698ad106affca009aad59721341c37f525f7e76e7b83", + "response_id": "20261014_94eafc34-e984-4203-aa24-b11c3393c027", + "receipt_status": 200, + "prompt_tokens": 2592, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.025515582994558 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-37-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-08\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-08", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "498131307af61ad3f5320b54277c71f6fff12f5c723d6a8fe9697009764fec0c", + "request_file_sha256": "474e2aebac18ed24496b269378b28f96021efb7a3d0152fcd73be19212270d40", + "receipt_file_sha256": "d43dbc4139b9c07d52a4e85d60a148eb07c14337e1e9baf7ad985fb3b0096ebc", + "receipt_body_text_sha256": "0cc9b48e7b351895301d443d00f0b61ece33093b9cfb900e6c965a662d50fa86", + "result_file_sha256": "417418804f246865317d445869ff277d57e1f417198b5646e44534270893ef1f", + "raw_content_sha256": "404a69ff9e54f916db69a6ce6d90ff60c79f4d89c14a3a7f677443f9092339cc", + "response_id": "20261014_7a6526de-f8f9-4bf4-8268-13e48f7bc547", + "receipt_status": 200, + "prompt_tokens": 2237, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9568220000073779 + } + } + }, + { + "history_id": "confirm-fresh-38-te-m0", + "case_id": "fresh-38-te", + "scenario": "fresh-38", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "question": "VISIT-2138 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-38-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. ఇప్పుడు VISIT-2138 కోసం 2027-06-08 కావాలి. ఇచ్చిన అభ్యర్థన గుర్తింపును ఇప్పటికే 2027-05-09కి పూర్తయిన మార్పు కోసం వాడారు. అదే గుర్తింపుతో వేరే మార్పు అమలు చేయవద్దు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The supplied request key was used for an earlier, different completed change. The new requested date was not applied in this native trace; VISIT-2138 remains at 2027-05-09. An old receipt for different arguments is not proof this new request was applied.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_performed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-38", + "id": "VISIT-2138", + "date": "2027-05-09", + "version": 6, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-38", + "id": "VISIT-2138", + "date": "2027-05-09", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2138", + "date": "2027-05-09", + "expected_version": 5, + "idempotency_key": "REQUEST-38-CHANGE-24", + "approval_id": "AUTH-38-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-09", + "version": 6 + } + } + ], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "13cfc20073d520501af6e3ca6e7b906cf54693534bed0ba89906a6fa14012fe9", + "case_sha256": "6a9b7c9795fe779d2041e8f0cf2c7bcdb0009e7635a176a5805084e2b653e2ef", + "gold_label_sha256": "290fc7d30e91efc928981e8ca78d4d1c7a76e57ef8bf47ae2cc284cef7a06792", + "reports": { + "chronological": { + "id": "confirm-fresh-38-te-m0-source-chronological", + "arm": "chronological", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-09\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "0eb7baee557b69e0611cc2aed6d20016afb3db70f06f500b19f03e8acd704eda", + "request_file_sha256": "0d655e4479be29edc85982f95c5ef5e8f04c4d1cfce7aeca8a0d2ddf44c2a925", + "receipt_file_sha256": "3a1327d7e0ac0ab555744f72e226e1acd29ad88a997471061b683e4c641b021f", + "receipt_body_text_sha256": "061c0759995e0943978211e69922aa694a5c47b1b6f185d67889d815c6f5ccfd", + "result_file_sha256": "a83af413afe7cddcaa541b7076f09d4924ecc2e0a192a002771a74e3c3fadc75", + "raw_content_sha256": "0c4812369e0d55fa35c84fa8fee15ea63e76b40cbe4d6362cf99bfea51d639e0", + "response_id": "20261014_b1eb5aeb-f2b6-436e-a5ec-82c0555cb54a", + "receipt_status": 200, + "prompt_tokens": 1643, + "completion_tokens": 2288, + "reasoning_tokens": 2256, + "elapsed_seconds": 12.175992250005947 + }, + "grouped_lossless": { + "id": "confirm-fresh-38-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-09\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "14ac88ba246d4f75f13d4723e55d2e45b56b545e34ed7098b78396a1ac5111dd", + "request_file_sha256": "3c678801f37fc266f8be2abd21e8e5c99af4808bf89bb60a07998325a17cdcb3", + "receipt_file_sha256": "5aadb324f22578b827985d8cdccbc67ed1f482cf3133b74cfba9344000711a73", + "receipt_body_text_sha256": "62d45fcc5cc9913caafdcddf3e634bea26b8ce0efcbda44216c9baf0cc90e1eb", + "result_file_sha256": "a83af413afe7cddcaa541b7076f09d4924ecc2e0a192a002771a74e3c3fadc75", + "raw_content_sha256": "0c4812369e0d55fa35c84fa8fee15ea63e76b40cbe4d6362cf99bfea51d639e0", + "response_id": "20261014_8fd6dfb3-fc7e-41f2-b788-f21d78ed1794", + "receipt_status": 200, + "prompt_tokens": 2073, + "completion_tokens": 2919, + "reasoning_tokens": 2887, + "elapsed_seconds": 14.626250207991689 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-38-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-09\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "6c90bade7d34e01e2976e6bf4fb5c327a4edaa98418414b3b3923ce66d6c3309", + "request_file_sha256": "4a3cd0304c6609492b3f71064c3637f21b7b7a866f20ebc80e00adb548adfb14", + "receipt_file_sha256": "d6ad2d9807ab8ad8edb3ee800a60b2f377151fd9a1afa64ab38862d5fbd20ada", + "receipt_body_text_sha256": "ddb0cb08fee90c49a9ca464d0d8bb0d88e96ba313ee53c84b8a286afbca906e3", + "result_file_sha256": "06c2187ceae2c64dea1bf91ec923b84dd3573f34714a5aafeb636bfe711a773e", + "raw_content_sha256": "0875913962755e104e569968b4c0b381161a16bcb57f4ff6e928b39498b68ec5", + "response_id": "20261014_a81f4072-b72e-4a00-98b0-6c172291bc01", + "receipt_status": 200, + "prompt_tokens": 1815, + "completion_tokens": 4196, + "reasoning_tokens": 4164, + "elapsed_seconds": 20.786617167002987 + } + } + }, + { + "history_id": "confirm-fresh-38-te-m1", + "case_id": "fresh-38-te", + "scenario": "fresh-38", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "question": "VISIT-2138 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-38-0", + "role": "customer", + "content": "నేను అర్జున్. రెండు సందేశాలనూ వరుసగా చదవండి. ఇప్పుడు VISIT-2138 కోసం 2027-06-08 కావాలి. ఇచ్చిన అభ్యర్థన గుర్తింపును ఇప్పటికే 2027-05-09కి పూర్తయిన మార్పు కోసం వాడారు. అదే గుర్తింపుతో వేరే మార్పు అమలు చేయవద్దు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The supplied request key was used for an earlier, different completed change. The new requested date was not applied in this native trace; VISIT-2138 remains at 2027-05-09. An old receipt for different arguments is not proof this new request was applied.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_performed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-38", + "id": "VISIT-2138", + "date": "2027-05-09", + "version": 6, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-38", + "id": "VISIT-2138", + "date": "2027-05-09", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-9949a6cba868924b", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2138" + }, + "response": { + "date": "2027-05-09", + "version": 6, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2138", + "date": "2027-05-09", + "expected_version": 5, + "idempotency_key": "REQUEST-38-CHANGE-24", + "approval_id": "AUTH-38-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-09", + "version": 6 + } + } + ], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "892998984302902e08e87525602769e0bb382e798d6a32270c11b95ddaaafb2c", + "case_sha256": "6a9b7c9795fe779d2041e8f0cf2c7bcdb0009e7635a176a5805084e2b653e2ef", + "gold_label_sha256": "290fc7d30e91efc928981e8ca78d4d1c7a76e57ef8bf47ae2cc284cef7a06792", + "reports": { + "chronological": { + "id": "confirm-fresh-38-te-m1-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-05-09\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "rejected", + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status rejected conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "1f5541a44c60b7bb93517270c4a596c3cd8443bd2ab02f0a49d5057d6021770e", + "request_file_sha256": "68fd4854d2bcf2a27d98a6d281e253bbbbc28564365c8a64e353aab0965f3813", + "receipt_file_sha256": "3b1628931b6a0ed8afd6d827c91bc3111e03517b40b274baa3da3d558371235f", + "receipt_body_text_sha256": "a9bacb4570c1ad7ab6a012d4930e96d8008e45b1b123d72f7248b259c67052a4", + "result_file_sha256": "68edae63c30afcb8dede4b7e2cdfb966d8dced07732f78d2a0d7067c4a5c045e", + "raw_content_sha256": "5b31fb7ca17e78a90f4ad1615cc4f0f6f2ff976799a03a73715a9d74c4e47c88", + "response_id": "20261014_72ac4d06-8715-4815-911e-e239a603b4a7", + "receipt_status": 200, + "prompt_tokens": 1833, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.9656026250013383 + }, + "grouped_lossless": { + "id": "confirm-fresh-38-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-05-09\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "rejected", + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status rejected conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "8f4e63f0416f1579475f71236be2fdf4282d5aefe8ff520baa31e52653e71944", + "request_file_sha256": "4929a2a7fb5db954fbac8b6b67f60424762a0940cc51aeb6464c84a93dff507d", + "receipt_file_sha256": "4789fcac114838351607da1527feb822f3817fd77b7ae5bd9e0f1b9d7377579f", + "receipt_body_text_sha256": "b587f6c650ff4004c3735009ba3653b5735a4788a86dea0564ed7def017ddd7a", + "result_file_sha256": "68edae63c30afcb8dede4b7e2cdfb966d8dced07732f78d2a0d7067c4a5c045e", + "raw_content_sha256": "5b31fb7ca17e78a90f4ad1615cc4f0f6f2ff976799a03a73715a9d74c4e47c88", + "response_id": "20261014_0d2e8e48-6e87-48c7-8f2a-af4667c4f85b", + "receipt_status": 200, + "prompt_tokens": 2418, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 1.0040276660001837 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-38-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-09\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-09", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "c0477df2549bbda4d8dc6c80e99407cd5795bcce1972101794bce8e97b74cd61", + "request_file_sha256": "1c4ccae7a626e0667f950571ad8900947708336a8ea9c8f891e2510017cb26ac", + "receipt_file_sha256": "0741cd12b3a3c63e59f3715fdaf34fee78a70e70df5d121ef08b5e4b412d510b", + "receipt_body_text_sha256": "9e78c0e5b86788af6bc0b4151016d2c934ff57a5e116e3d978aba2b5f55bb094", + "result_file_sha256": "7bb3b716e7f26e456befff8ad0778da3cde694aa22a37e4e06202fb80e43454b", + "raw_content_sha256": "69e809991e2e6516871589d05968ed8d6dbfb4311f9ef072ce2282e6760d4834", + "response_id": "20261014_a20d150d-9096-4e8a-bcfc-6797bbbcfbde", + "receipt_status": 200, + "prompt_tokens": 2112, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0110378749959636 + } + } + }, + { + "history_id": "confirm-fresh-39-te-m0", + "case_id": "fresh-39-te", + "scenario": "fresh-39", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "question": "VISIT-2139 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-39-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-39-1", + "role": "customer", + "content": "ఇప్పుడు VISIT-2139 కోసం 2027-06-09 కావాలి." + }, + { + "id": "note-39-2", + "role": "customer", + "content": "ఇచ్చిన అభ్యర్థన గుర్తింపును ఇప్పటికే 2027-05-10కి పూర్తయిన మార్పు కోసం వాడారు. అదే గుర్తింపుతో వేరే మార్పు అమలు చేయవద్దు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The supplied request key was used for an earlier, different completed change. The new requested date was not applied in this native trace; VISIT-2139 remains at 2027-05-10. An old receipt for different arguments is not proof this new request was applied.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_performed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-39", + "id": "VISIT-2139", + "date": "2027-05-10", + "version": 6, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-39", + "id": "VISIT-2139", + "date": "2027-05-10", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2139", + "date": "2027-05-10", + "expected_version": 5, + "idempotency_key": "REQUEST-39-CHANGE-24", + "approval_id": "AUTH-39-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-10", + "version": 6 + } + } + ], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "be25deef9aa7af2eff657ea1983c9686f14843c51ca2bab6a1273c59ef1142f2", + "case_sha256": "7f98846024d80aebeb3b5a34a1d43c3a739b990d6afadf27e2d9b8698d0e19ff", + "gold_label_sha256": "2a2d36dc02ac4c29175d088c7ba411e5d416e3e38f9d65fae37f1d480331a6e7", + "reports": { + "chronological": { + "id": "confirm-fresh-39-te-m0-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-10\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "5a6c2295e28e264d0325f0b9baa8fd399641ea2550bdd817ef76d4125af29f6a", + "request_file_sha256": "f31829800064a6e86fb61545ba77849e676a09729ff5bf9b6a51157a3a29298c", + "receipt_file_sha256": "0c5d55e48ff5abc2a85e99ef64880b9e169ef98df562a0b85ecc9ebb54c5056e", + "receipt_body_text_sha256": "5af3e9a648a70807ab8bcef675dc164342054cd45671d7f3aadc89602ccead2d", + "result_file_sha256": "96b42d2cc3de69828ffb97c140cf9d47de4b7c1c2276847078932bfeadb2f1f8", + "raw_content_sha256": "b3ec4557109cc643fdfe76911b4b72140a78dbcd5ad0dc05a75ced885889e007", + "response_id": "20261014_170ab571-d833-4106-a641-e4a157392275", + "receipt_status": 200, + "prompt_tokens": 1653, + "completion_tokens": 1536, + "reasoning_tokens": 1504, + "elapsed_seconds": 6.939602875005221 + }, + "grouped_lossless": { + "id": "confirm-fresh-39-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-10\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "3a9f321121262b3ef97f24dc59d3cb2652df03497372612b2e5da353aee7cf52", + "request_file_sha256": "31b9fa38341b932f94b41428b1d96bd8d51c256d8314b79febd9ec93809b78e0", + "receipt_file_sha256": "c104ab864111fda600bba317e322e68e82ee4ac7995245364c9b8947915778f0", + "receipt_body_text_sha256": "0aa05c88c1f2dee2c4b671ce8491a9b1b74eefb0e659129dc5b6341226833bb2", + "result_file_sha256": "86ce10b21452933ea2f017456ad1d7f66046b05fe0cf423ddbe8220fa19420f2", + "raw_content_sha256": "4e0a3ab3a5c9a573a5295365d934a1f59cfa12169ca24dc63e30bc8ffd9495f4", + "response_id": "20261014_ea5f6589-657b-44f9-9284-e3a16111fa95", + "receipt_status": 200, + "prompt_tokens": 2190, + "completion_tokens": 4703, + "reasoning_tokens": 4671, + "elapsed_seconds": 21.842392959006247 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-39-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"not_requested\",\"answer\":\"2027-05-10\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "c4c43d1b8a0db11c8f5ec3ca5d432b6676c4862e6415ab8140fb6786707c7aa0", + "request_file_sha256": "622532448b6d6d2948381199e58975924f152c0a171745f77e968551d21129ee", + "receipt_file_sha256": "42fc4ca37301e9f7377ba74f341dcb4d03c9f8f7d8465bed0305c5a2766d6464", + "receipt_body_text_sha256": "419ce0a51d17ab7d8e2d0b8dc2068c0103e68c1958bf9b65368f038b6d2c9e2d", + "result_file_sha256": "96b42d2cc3de69828ffb97c140cf9d47de4b7c1c2276847078932bfeadb2f1f8", + "raw_content_sha256": "b3ec4557109cc643fdfe76911b4b72140a78dbcd5ad0dc05a75ced885889e007", + "response_id": "20261014_3f13c0d4-6410-4a61-ad33-24f3669af78a", + "receipt_status": 200, + "prompt_tokens": 1961, + "completion_tokens": 2102, + "reasoning_tokens": 2070, + "elapsed_seconds": 11.289832583002863 + } + } + }, + { + "history_id": "confirm-fresh-39-te-m1", + "case_id": "fresh-39-te", + "scenario": "fresh-39", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "question": "VISIT-2139 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-39-0", + "role": "customer", + "content": "నా పేరు కావ్య. ముందు పాత సందేశం, తర్వాత తదుపరి సందేశం ఉన్నాయి." + }, + { + "id": "note-39-1", + "role": "customer", + "content": "ఇప్పుడు VISIT-2139 కోసం 2027-06-09 కావాలి." + }, + { + "id": "note-39-2", + "role": "customer", + "content": "ఇచ్చిన అభ్యర్థన గుర్తింపును ఇప్పటికే 2027-05-10కి పూర్తయిన మార్పు కోసం వాడారు. అదే గుర్తింపుతో వేరే మార్పు అమలు చేయవద్దు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The supplied request key was used for an earlier, different completed change. The new requested date was not applied in this native trace; VISIT-2139 remains at 2027-05-10. An old receipt for different arguments is not proof this new request was applied.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_performed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-39", + "id": "VISIT-2139", + "date": "2027-05-10", + "version": 6, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-39", + "id": "VISIT-2139", + "date": "2027-05-10", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a940bf84bc95e42c", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2139" + }, + "response": { + "date": "2027-05-10", + "version": 6, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2139", + "date": "2027-05-10", + "expected_version": 5, + "idempotency_key": "REQUEST-39-CHANGE-24", + "approval_id": "AUTH-39-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-10", + "version": 6 + } + } + ], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "604968967b4559221e4b47b9f501ef8a13079ab7aa9687a83edd5a83bd301f76", + "case_sha256": "7f98846024d80aebeb3b5a34a1d43c3a739b990d6afadf27e2d9b8698d0e19ff", + "gold_label_sha256": "2a2d36dc02ac4c29175d088c7ba411e5d416e3e38f9d65fae37f1d480331a6e7", + "reports": { + "chronological": { + "id": "confirm-fresh-39-te-m1-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"rejected\",\n \"answer\": \"2027-05-10\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "rejected", + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status rejected conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "53f1fe7235189df79dbb5dd7d78bc112dd5f45309819bff95454d84e6891a452", + "request_file_sha256": "069ceab9e19bdb4a80b470e737d097585cf928b57cab8b288b0a8da34bc395fe", + "receipt_file_sha256": "4c044f8dd646b0830e96ac0fa0eef582bb2dee2e740878c4728fe8c8e2489535", + "receipt_body_text_sha256": "25397b10fd5fd57ef603d02ff5bc50a465934250db694d915b19e20d859a766a", + "result_file_sha256": "16e46d320572a55d27d1366cb2cc4864e58b19b8a19e6e0b5044ec39ab926afc", + "raw_content_sha256": "d899426e8ef4e447616ea48b973b02b64065c0a82d73ad786f4d64491f5ddae4", + "response_id": "20261014_79b6ae73-1aa7-4619-bef4-fdb2a3151de8", + "receipt_status": 200, + "prompt_tokens": 1928, + "completion_tokens": 40, + "reasoning_tokens": null, + "elapsed_seconds": 0.989755000002333 + }, + "grouped_lossless": { + "id": "confirm-fresh-39-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-10\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "7778f79009baf4eb9d98923e88f16ba7cf40161b21cc336a83e396a04d3fbe55", + "request_file_sha256": "f379b2c270cab9b5bed78be9e1d9720d64b05db696571b9bb58299d1fbf4666a", + "receipt_file_sha256": "4cc16abde2cff8127cc5bef60af2666f09f75d5551439ed92413d56ac8402246", + "receipt_body_text_sha256": "516a81d2fe86c8d21bf198c5cc4845923f6a0dfcfb08d3c483668b71f5750e4c", + "result_file_sha256": "f2371cd3e60dbad81739ea12fffa281dda1158ef039ec6a8984449c96e971832", + "raw_content_sha256": "057425dcb1fdd6d6ccad3bd4cad69751520540732ee2ff9949cb04440da4e058", + "response_id": "20261014_f9e5d99b-dee4-47df-bb44-2921129bbf1d", + "receipt_status": 200, + "prompt_tokens": 2614, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.038521458991454 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-39-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-10\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-10", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "f95e6cd3685329f10e532e5f86478b125d2d60b46274cd32f691473d42dc9700", + "request_file_sha256": "2c4bb5a60cd893c6b99a437510c748b37640c6dae0f0fb1b693d67a9f6267dd1", + "receipt_file_sha256": "38183f5201f8f007e08c2c5ff2b826a109d9357f10c75f2faa05a9e4170bde41", + "receipt_body_text_sha256": "ed20c718fc28fa1163be6b1264eb34f9da4119ba0a546a8c90fcc015441946e6", + "result_file_sha256": "f79a657b75b3fc410aff8d435eeaf3c5ba5d5672d55389524137ffcefa9943dc", + "raw_content_sha256": "6a4974639e1edd8ecb3f0201898f2d4823422d6695b0ad27708f93e554c2e3c1", + "response_id": "20261014_11fa66ca-ba2d-4526-b758-aafefc09d7dc", + "receipt_status": 200, + "prompt_tokens": 2250, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.1882771249947837 + } + } + }, + { + "history_id": "confirm-fresh-40-te-m0", + "case_id": "fresh-40-te", + "scenario": "fresh-40", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "question": "VISIT-2140 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-40-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. ఇప్పుడు VISIT-2140 కోసం 2027-06-10 కావాలి. ఇచ్చిన అభ్యర్థన గుర్తింపును ఇప్పటికే 2027-05-11కి పూర్తయిన మార్పు కోసం వాడారు. అదే గుర్తింపుతో వేరే మార్పు అమలు చేయవద్దు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The supplied request key was used for an earlier, different completed change. The new requested date was not applied in this native trace; VISIT-2140 remains at 2027-05-11. An old receipt for different arguments is not proof this new request was applied.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_performed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-40", + "id": "VISIT-2140", + "date": "2027-05-11", + "version": 6, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-40", + "id": "VISIT-2140", + "date": "2027-05-11", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a1d607daf740dc50", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2140" + }, + "response": { + "date": "2027-05-11", + "version": 6, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2140", + "date": "2027-05-11", + "expected_version": 5, + "idempotency_key": "REQUEST-40-CHANGE-24", + "approval_id": "AUTH-40-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-11", + "version": 6 + } + } + ], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "49d8e0c2cc252cad53bdc05fad7bd3de259f18935a7dbd41e19c4bedf1144f23", + "case_sha256": "cab6eeb14476976a7745d3ca78ded471bc1eafdfe87278113bbf9e6f3877ed53", + "gold_label_sha256": "221984dd73e7f6d8b3ea73979bdc9378a3748b585dee7266f2c3a11e7150ea9b", + "reports": { + "chronological": { + "id": "confirm-fresh-40-te-m0-source-chronological", + "arm": "chronological", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "90ddd22c8749e105611b3c4247e6da8275932a63fbf0ef7b6ff0eab4e08aecb7", + "request_file_sha256": "6989255c2663798e35016e7e782884e6acddb0cfbc30728a1eb61c0639636bcb", + "receipt_file_sha256": "8446122cf69f4957ccc9a6a819d6bfdc9e13ed35a2a694ce5f9801ad24b79da4", + "receipt_body_text_sha256": "1c7281cb70c576a2d87fb07134e6b004d4b3bf8e6e83a2e62271e617a302952e", + "result_file_sha256": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "raw_content_sha256": "9dac33ab5e4882ec69311a19f72c1330ec8f057e21a912a81f03d812c2e99127", + "response_id": "20261014_dfd1df16-5e24-4641-a8e4-a0a2a1344e3b", + "receipt_status": 200, + "prompt_tokens": 1812, + "completion_tokens": 4233, + "reasoning_tokens": 4201, + "elapsed_seconds": 20.638543459004723 + }, + "grouped_lossless": { + "id": "confirm-fresh-40-te-m0-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "0f3f478a3f2a933e4422d926965a52677266a6bdfe642d35610eb8493679dbde", + "request_file_sha256": "72c8a9dc90268587fb8a558d34a96f065eced108d1bb21c35697b272a2de7c29", + "receipt_file_sha256": "998944ffa61467b43c1b4e16123dac9f4e9302d45723e8e7e7b596a72102270a", + "receipt_body_text_sha256": "fbe1e99363797b7964c8076e73b72c3b9f434295358946a27d741e2f2bb8fcd7", + "result_file_sha256": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "raw_content_sha256": "9dac33ab5e4882ec69311a19f72c1330ec8f057e21a912a81f03d812c2e99127", + "response_id": "20261014_9c0c7fe3-6300-40be-bab4-8af2137d2370", + "receipt_status": 200, + "prompt_tokens": 2397, + "completion_tokens": 2583, + "reasoning_tokens": 2551, + "elapsed_seconds": 13.740064708996215 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-40-te-m0-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\"action_status\":\"already_applied\",\"answer\":\"2027-05-11\",\"needs_clarification\":false}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "884016adc3bf50eeec6ef96b486f5baa24869ca40606b6240a51d00c6e114988", + "request_file_sha256": "16e1fdeb9608c13429192d06b0b4a95bd56e9a99f4199839464b3ca686d314ea", + "receipt_file_sha256": "68788320b232c0f96089fe8919c9a2110c64dcaff53fbb1f0dad6f5fbaf16b77", + "receipt_body_text_sha256": "c751e51cf26d9a008a02c027b1ed97cef71bd80b9ebecc529a31aba19071a292", + "result_file_sha256": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "raw_content_sha256": "9dac33ab5e4882ec69311a19f72c1330ec8f057e21a912a81f03d812c2e99127", + "response_id": "20261014_a83f139d-04a1-453d-860a-f4f27d16635b", + "receipt_status": 200, + "prompt_tokens": 2104, + "completion_tokens": 4201, + "reasoning_tokens": 4169, + "elapsed_seconds": 20.65557820899994 + } + } + }, + { + "history_id": "confirm-fresh-40-te-m1", + "case_id": "fresh-40-te", + "scenario": "fresh-40", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "question": "VISIT-2140 కోసం ఇచ్చిన మార్పు అభ్యర్థనను పరిశీలించండి. ఇప్పుడు నిజంగా ఖరారైన తేదీ ఏమిటి?", + "source_messages": [ + { + "id": "note-40-0", + "role": "customer", + "content": "నేను రవి. ఈ రెండు సందేశాలు చూపిన బుకింగ్ గుర్తింపులకు సంబంధించినవి. ఇప్పుడు VISIT-2140 కోసం 2027-06-10 కావాలి. ఇచ్చిన అభ్యర్థన గుర్తింపును ఇప్పటికే 2027-05-11కి పూర్తయిన మార్పు కోసం వాడారు. అదే గుర్తింపుతో వేరే మార్పు అమలు చేయవద్దు." + } + ], + "question_target": "actual_current_date", + "source_interpretation": "The supplied request key was used for an earlier, different completed change. The new requested date was not applied in this native trace; VISIT-2140 remains at 2027-05-11. An old receipt for different arguments is not proof this new request was applied.", + "expected_answer_from_source_and_frozen_label": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status_from_actual_receipts": "not_performed", + "initial_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-40", + "id": "VISIT-2140", + "date": "2027-05-11", + "version": 6, + "closed": 0 + }, + "final_scoped_booking": { + "tenant": "fresh-confirmation", + "customer": "person-40", + "id": "VISIT-2140", + "date": "2027-05-11", + "version": 6, + "closed": 0 + }, + "actual_tool_trace": [ + { + "call_id": "chatcmpl-tool-a5e7c827d9141e7c", + "name": "read_current", + "arguments": { + "appointment_id": "VISIT-2140" + }, + "response": { + "date": "2027-05-11", + "version": 6, + "closed": false + } + } + ], + "prior_execution_receipts": [ + { + "arguments": { + "appointment_id": "VISIT-2140", + "date": "2027-05-11", + "expected_version": 5, + "idempotency_key": "REQUEST-40-CHANGE-24", + "approval_id": "AUTH-40-GRANT-24" + }, + "receipt": { + "status": "confirmed", + "date": "2027-05-11", + "version": 6 + } + } + ], + "source_gold_ambiguous": false, + "parent_paired_file_sha256": "e59da4a5d3b8a411ff189c6b882589213f489df7a3742b4f3b14bb5f04afaafd", + "case_sha256": "cab6eeb14476976a7745d3ca78ded471bc1eafdfe87278113bbf9e6f3877ed53", + "gold_label_sha256": "221984dd73e7f6d8b3ea73979bdc9378a3748b585dee7266f2c3a11e7150ea9b", + "reports": { + "chronological": { + "id": "confirm-fresh-40-te-m1-source-chronological", + "arm": "chronological", + "position": 1, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_performed\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_performed", + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_performed agrees with trusted intent and the actual prior/update receipts.", + "request_sha256": "31f71c44491f4b260c00c28beae02278ef951c6ea0502da384a4ab9eceab0626", + "request_file_sha256": "ab7febd9e447de0c027ee57a1ec08f86b53a4acb243091fc841ec032b0114565", + "receipt_file_sha256": "f1104509545157d78f745bbde56727f93066a35563e9a49a7e13d13daf8599d8", + "receipt_body_text_sha256": "437a37adbe07ad00eb0429e165c6181a48b9cf12f01a0c1c7be5709d35a866df", + "result_file_sha256": "e154d7d3fea97d5d74e19e49727b25d73019ead962c7b21da19391b8aa91e911", + "raw_content_sha256": "727a7f58f51aa4f83de71e27cab3c77950ad8058f008e847b9b26fc4f5c24860", + "response_id": "20261014_df410cbe-b492-4071-9ab7-c4bca47b114d", + "receipt_status": 200, + "prompt_tokens": 1843, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9605918339948403 + }, + "grouped_lossless": { + "id": "confirm-fresh-40-te-m1-source-grouped_lossless", + "arm": "grouped_lossless", + "position": 0, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"already_applied\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "already_applied", + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "f8c970127cd1e937518709fccc5e41a6f8d097fde95e53e0ecdfae882cd2481f", + "request_file_sha256": "5d2f945008763fef899111b75d41c3d713e912c5963418729fa382fe071fe92c", + "receipt_file_sha256": "edc3f948d4f827cc0515c7f1e4863df25f9cd7700c197275786820ac9b2f1461", + "receipt_body_text_sha256": "7d3b1c763a8c3c6a36753a85eadb4ea3b05053456a516d4aaf1d175410dd0e0e", + "result_file_sha256": "fca49d1de4a1905da389c1cef4610aee77922b07c9c45ea909d59d69c71cb0ab", + "raw_content_sha256": "c7f328d78597435701fad57703175eeab15cace2a91f54b5d85e68d150fb7bca", + "response_id": "20261014_932f6656-48c1-4677-9d4d-f4fa698c69f1", + "receipt_status": 200, + "prompt_tokens": 2413, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 1.0076746659906348 + }, + "grouped_actor_prose_withheld": { + "id": "confirm-fresh-40-te-m1-source-grouped_actor_prose_withheld", + "arm": "grouped_actor_prose_withheld", + "position": 2, + "treated_actor_prose": true, + "terminal": "answer", + "finish_reason": "stop", + "raw_content": "{\n \"action_status\": \"not_requested\",\n \"answer\": \"2027-05-11\",\n \"needs_clarification\": false\n}", + "raw_output": { + "action_status": "not_requested", + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_answer": { + "answer": "2027-05-11", + "needs_clarification": false + }, + "expected_action_status": "not_performed", + "date_only_match": true, + "clarification_match": true, + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "wrong_non_null_date": false, + "false_abstention": false, + "factual_review": "The raw date/null and clarification flag match the reviewed source or scoped final backend.", + "status_review": "Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed.", + "request_sha256": "75c476383b39dd2766dfbb9d9b27ee5a635febf94e26b94df7b085b1e260675f", + "request_file_sha256": "c71f995250e6a05a23059c2c9f83e5938078cd4d0ca0cbeb985a550314cbab85", + "receipt_file_sha256": "af0863b30838bd698da6518c8c20b79736f0679d6704ebeea16a89c75c7ffd0c", + "receipt_body_text_sha256": "bac9b3b82b93f92951866029105ea378a472ce4891ea5e7bd00fffa631f20403", + "result_file_sha256": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "raw_content_sha256": "2655156e40334761b11243498c952a7bfa50bffe056b93d1424ef09be4cb336c", + "response_id": "20261014_5ab4de7a-4483-403a-aa6e-b8cc32d7332a", + "receipt_status": 200, + "prompt_tokens": 2099, + "completion_tokens": 42, + "reasoning_tokens": null, + "elapsed_seconds": 0.9818256669968832 + } + } + } + ], + "judgments": [ + { + "job_id": "confirm-fresh-01-te-m0-source-chronological", + "history_id": "confirm-fresh-01-te-m0", + "case_id": "fresh-01-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "3a2f5328a1c35116ef142a3db1e78f87dbbc8f54ba09314ff44a7cf7652ea79b", + "result_file_sha256": "ce0b06081ae4177284f45db3e7222695e4995ff592024742d429c996d187fb1e", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_requested." + }, + { + "job_id": "confirm-fresh-01-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-01-te-m0", + "case_id": "fresh-01-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "8aaeb0f32dbe546730fa5f7e7419bc6071d73ddbc5f5b92f356e540d2f854cb2", + "result_file_sha256": "96797c104f7a78e3839f320aa377b1e928e67cc696b95a1e7c81bfcf267ef17b", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-01-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-01-te-m0", + "case_id": "fresh-01-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "request_correction", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": false, + "terminal": "answer", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "review": "False abstention: source or final backend uniquely establishes 2027-05-02, but raw answer is null. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-01-te-m1-source-chronological", + "history_id": "confirm-fresh-01-te-m1", + "case_id": "fresh-01-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "54f87f93350a2f520e59fad7cd323aeaa385120f331fae30b8483a7800211347", + "result_file_sha256": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-01-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-01-te-m1", + "case_id": "fresh-01-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "54f87f93350a2f520e59fad7cd323aeaa385120f331fae30b8483a7800211347", + "result_file_sha256": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-01-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-01-te-m1", + "case_id": "fresh-01-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "54f87f93350a2f520e59fad7cd323aeaa385120f331fae30b8483a7800211347", + "result_file_sha256": "ad5181f39ab9885dde3419041eaed66974e928cbf826cf14a8ca572e42126573", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-02-te-m0-source-chronological", + "history_id": "confirm-fresh-02-te-m0", + "case_id": "fresh-02-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "8a1e2997cc0301dc67f4599949b8916f49f477f5d893e801470b0c0fd38e7854", + "result_file_sha256": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-02-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-02-te-m0", + "case_id": "fresh-02-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "8a1e2997cc0301dc67f4599949b8916f49f477f5d893e801470b0c0fd38e7854", + "result_file_sha256": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-02-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-02-te-m0", + "case_id": "fresh-02-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "8a1e2997cc0301dc67f4599949b8916f49f477f5d893e801470b0c0fd38e7854", + "result_file_sha256": "6632e6bb430184eee56cea62784be7b3d400a139e629619000b96a6e11a93746", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-02-te-m1-source-chronological", + "history_id": "confirm-fresh-02-te-m1", + "case_id": "fresh-02-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "302302624fee023a6aedb4cd6f12fe82dc40383abb18b454d5e9be83a0d2c374", + "result_file_sha256": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-02-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-02-te-m1", + "case_id": "fresh-02-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "302302624fee023a6aedb4cd6f12fe82dc40383abb18b454d5e9be83a0d2c374", + "result_file_sha256": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-02-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-02-te-m1", + "case_id": "fresh-02-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "302302624fee023a6aedb4cd6f12fe82dc40383abb18b454d5e9be83a0d2c374", + "result_file_sha256": "91add740889506267ce47f6c6e2b9ff026e4ab812ed8f394e570019e7e7c88fc", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-03-te-m0-source-chronological", + "history_id": "confirm-fresh-03-te-m0", + "case_id": "fresh-03-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "212f85c3b1a2653eaa716d130aac4c6887ca2e4ae77cd416c3a0ee75931a3138", + "result_file_sha256": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-03-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-03-te-m0", + "case_id": "fresh-03-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "212f85c3b1a2653eaa716d130aac4c6887ca2e4ae77cd416c3a0ee75931a3138", + "result_file_sha256": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-03-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-03-te-m0", + "case_id": "fresh-03-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "212f85c3b1a2653eaa716d130aac4c6887ca2e4ae77cd416c3a0ee75931a3138", + "result_file_sha256": "6840dedb982f367dd49489f9366e0ec56a565c975ea95217a5ed4b238153f7d9", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-03-te-m1-source-chronological", + "history_id": "confirm-fresh-03-te-m1", + "case_id": "fresh-03-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "42098a7a7a4ef0ae49a0da19af03407227f6c6dda8858f3a01e7bd39c56ff795", + "result_file_sha256": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-03-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-03-te-m1", + "case_id": "fresh-03-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "42098a7a7a4ef0ae49a0da19af03407227f6c6dda8858f3a01e7bd39c56ff795", + "result_file_sha256": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-03-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-03-te-m1", + "case_id": "fresh-03-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "42098a7a7a4ef0ae49a0da19af03407227f6c6dda8858f3a01e7bd39c56ff795", + "result_file_sha256": "3a6bec092ea1abe3e8202b73a9399389c940473d9e8b32696d9d0361b935074c", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-04-te-m0-source-chronological", + "history_id": "confirm-fresh-04-te-m0", + "case_id": "fresh-04-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "c45d72a60c16dbd0e39fe78d180715170520e37c139792fb956a8819a538950b", + "result_file_sha256": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-04-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-04-te-m0", + "case_id": "fresh-04-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "c45d72a60c16dbd0e39fe78d180715170520e37c139792fb956a8819a538950b", + "result_file_sha256": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-04-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-04-te-m0", + "case_id": "fresh-04-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "c45d72a60c16dbd0e39fe78d180715170520e37c139792fb956a8819a538950b", + "result_file_sha256": "1a82ebaca1b24f34674b8ad160c27a4dd38e79e721905551136d031f2b9f359d", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-04-te-m1-source-chronological", + "history_id": "confirm-fresh-04-te-m1", + "case_id": "fresh-04-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "be956b2af283f28155cb87228f3c2833e326c59d589a68fb50d0a32eeaea2bd8", + "result_file_sha256": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-04-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-04-te-m1", + "case_id": "fresh-04-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "be956b2af283f28155cb87228f3c2833e326c59d589a68fb50d0a32eeaea2bd8", + "result_file_sha256": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-04-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-04-te-m1", + "case_id": "fresh-04-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "request_correction", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "be956b2af283f28155cb87228f3c2833e326c59d589a68fb50d0a32eeaea2bd8", + "result_file_sha256": "034e395929e9e1ee3ecf3492dfcc223a8de3ef2eeb61f69bbd57480935d6d71a", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-05-te-m0-source-chronological", + "history_id": "confirm-fresh-05-te-m0", + "case_id": "fresh-05-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-05-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-05-te-m0", + "case_id": "fresh-05-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "missing_replacement", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": false, + "terminal": "answer", + "raw_content_sha256": "837d7c4d0820dbbaaedba89ceca07d3f62ea5913ac395c7558beddcbae373aa6", + "result_file_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "review": "Null date is appropriate, but clarification=false misses the explicitly unresolved selection or condition. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-05-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-05-te-m0", + "case_id": "fresh-05-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-05-te-m1-source-chronological", + "history_id": "confirm-fresh-05-te-m1", + "case_id": "fresh-05-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-05-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-05-te-m1", + "case_id": "fresh-05-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-05-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-05-te-m1", + "case_id": "fresh-05-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-06-te-m0-source-chronological", + "history_id": "confirm-fresh-06-te-m0", + "case_id": "fresh-06-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "missing_replacement", + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": false, + "terminal": "answer", + "raw_content_sha256": "014cf05703fb39bb36034af11e8d93487393e9f563651792b7d462bd5fb9849d", + "result_file_sha256": "39d55d34faaf847cca522ee45ea4d0c2a58eb1da026891f66acd1aa872680135", + "review": "Null date is appropriate, but clarification=false misses the explicitly unresolved selection or condition. Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_requested." + }, + { + "job_id": "confirm-fresh-06-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-06-te-m0", + "case_id": "fresh-06-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "7d095b99976dab5ff227619f70a19f42f66d68a5a2c06af0e27aaa3031a4096e", + "result_file_sha256": "dd1eb69535233b7875007bb6fe801cf730d36ee48921e94b961cca63df5956d8", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_performed conflicts with trusted intent and actual receipts, which establish not_requested." + }, + { + "job_id": "confirm-fresh-06-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-06-te-m0", + "case_id": "fresh-06-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-06-te-m1-source-chronological", + "history_id": "confirm-fresh-06-te-m1", + "case_id": "fresh-06-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-06-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-06-te-m1", + "case_id": "fresh-06-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-06-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-06-te-m1", + "case_id": "fresh-06-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-07-te-m0-source-chronological", + "history_id": "confirm-fresh-07-te-m0", + "case_id": "fresh-07-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-07-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-07-te-m0", + "case_id": "fresh-07-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-07-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-te-m0", + "case_id": "fresh-07-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "missing_replacement", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": false, + "terminal": "answer", + "raw_content_sha256": "837d7c4d0820dbbaaedba89ceca07d3f62ea5913ac395c7558beddcbae373aa6", + "result_file_sha256": "7648d6fdbe9d14c2bea77064cb491e4f525be0c010dcea714f04e435f9cdf3b3", + "review": "Null date is appropriate, but clarification=false misses the explicitly unresolved selection or condition. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-07-te-m1-source-chronological", + "history_id": "confirm-fresh-07-te-m1", + "case_id": "fresh-07-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-07-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-07-te-m1", + "case_id": "fresh-07-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-07-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-07-te-m1", + "case_id": "fresh-07-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-08-te-m0-source-chronological", + "history_id": "confirm-fresh-08-te-m0", + "case_id": "fresh-08-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "7d095b99976dab5ff227619f70a19f42f66d68a5a2c06af0e27aaa3031a4096e", + "result_file_sha256": "dd1eb69535233b7875007bb6fe801cf730d36ee48921e94b961cca63df5956d8", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_performed conflicts with trusted intent and actual receipts, which establish not_requested." + }, + { + "job_id": "confirm-fresh-08-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-08-te-m0", + "case_id": "fresh-08-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-08-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-08-te-m0", + "case_id": "fresh-08-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-08-te-m1-source-chronological", + "history_id": "confirm-fresh-08-te-m1", + "case_id": "fresh-08-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-08-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-08-te-m1", + "case_id": "fresh-08-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-08-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-08-te-m1", + "case_id": "fresh-08-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "missing_replacement", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-09-te-m0-source-chronological", + "history_id": "confirm-fresh-09-te-m0", + "case_id": "fresh-09-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b5203780d91957fd9ffd4fed596dd501556d13bf3dc68e7bd2d239524382871b", + "result_file_sha256": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-09-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-09-te-m0", + "case_id": "fresh-09-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b5203780d91957fd9ffd4fed596dd501556d13bf3dc68e7bd2d239524382871b", + "result_file_sha256": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-09-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-09-te-m0", + "case_id": "fresh-09-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b5203780d91957fd9ffd4fed596dd501556d13bf3dc68e7bd2d239524382871b", + "result_file_sha256": "36145341b6b656ab94a554c587b7a270cea7cbb30cc71606547fc531c6647523", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-09-te-m1-source-chronological", + "history_id": "confirm-fresh-09-te-m1", + "case_id": "fresh-09-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "5f5a8175c7868d3fdb706311acadab639b8b5525cf4e1d717def813d9e5c10af", + "result_file_sha256": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-09-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-09-te-m1", + "case_id": "fresh-09-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "5f5a8175c7868d3fdb706311acadab639b8b5525cf4e1d717def813d9e5c10af", + "result_file_sha256": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-09-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-09-te-m1", + "case_id": "fresh-09-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "5f5a8175c7868d3fdb706311acadab639b8b5525cf4e1d717def813d9e5c10af", + "result_file_sha256": "82e62391e50063b847eeb58722cc3bc56537d7b609b90444fe9da5b00f88dfe8", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-10-te-m0-source-chronological", + "history_id": "confirm-fresh-10-te-m0", + "case_id": "fresh-10-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "9dac33ab5e4882ec69311a19f72c1330ec8f057e21a912a81f03d812c2e99127", + "result_file_sha256": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_requested." + }, + { + "job_id": "confirm-fresh-10-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-10-te-m0", + "case_id": "fresh-10-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "2362fc24e9ad026848ab17e8b29cf8890528a030252a998e2bd4367263250458", + "result_file_sha256": "727692fe8a92a3894a18a4126bc5cc2d92704c7d4c9a434233a0a5c3baf0a9dc", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-10-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-10-te-m0", + "case_id": "fresh-10-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "resolved_condition", + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "date_only_match": false, + "clarification_match": false, + "terminal": "answer", + "raw_content_sha256": "e3875ab43c67feba4f2cc12995ca2de6b5ea8d0758ee5e07393ef5123bf343f6", + "result_file_sha256": "08e1e6c34e14366850d3af627a73f020762ab2bf6e4855e1cf025b0428ef955c", + "review": "False abstention: source or final backend uniquely establishes 2027-05-11, but raw answer is null. Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_requested." + }, + { + "job_id": "confirm-fresh-10-te-m1-source-chronological", + "history_id": "confirm-fresh-10-te-m1", + "case_id": "fresh-10-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "2655156e40334761b11243498c952a7bfa50bffe056b93d1424ef09be4cb336c", + "result_file_sha256": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-10-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-10-te-m1", + "case_id": "fresh-10-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "2655156e40334761b11243498c952a7bfa50bffe056b93d1424ef09be4cb336c", + "result_file_sha256": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-10-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-10-te-m1", + "case_id": "fresh-10-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "2655156e40334761b11243498c952a7bfa50bffe056b93d1424ef09be4cb336c", + "result_file_sha256": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-11-te-m0-source-chronological", + "history_id": "confirm-fresh-11-te-m0", + "case_id": "fresh-11-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "16ab820924508bb6e38b21bb14605348846c4b7936adf81e2b291d962d86654f", + "result_file_sha256": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-11-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-11-te-m0", + "case_id": "fresh-11-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "16ab820924508bb6e38b21bb14605348846c4b7936adf81e2b291d962d86654f", + "result_file_sha256": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-11-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-11-te-m0", + "case_id": "fresh-11-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "16ab820924508bb6e38b21bb14605348846c4b7936adf81e2b291d962d86654f", + "result_file_sha256": "15510b825d0883c80f3255e5afcdf4802927c372bff2a68dff298dc8fd56b678", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-11-te-m1-source-chronological", + "history_id": "confirm-fresh-11-te-m1", + "case_id": "fresh-11-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b7f323c1d6229be3555907cf605c3036291995acf9fc575a5ef287331894401b", + "result_file_sha256": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-11-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-11-te-m1", + "case_id": "fresh-11-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b7f323c1d6229be3555907cf605c3036291995acf9fc575a5ef287331894401b", + "result_file_sha256": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-11-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-11-te-m1", + "case_id": "fresh-11-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b7f323c1d6229be3555907cf605c3036291995acf9fc575a5ef287331894401b", + "result_file_sha256": "ab87d7c7288c1fbb150700e6d133140f931794790459472441c39324b6577d48", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-12-te-m0-source-chronological", + "history_id": "confirm-fresh-12-te-m0", + "case_id": "fresh-12-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "9ef377dc92395cfd885732d67e7d06c1be62e2c646804b04ffde441ccef8c68e", + "result_file_sha256": "c4ff9927c3a55e64897349c37882c7d2896bdfbe26855686f0af32e779f96ce5", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-12-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-12-te-m0", + "case_id": "fresh-12-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "84abd2c69924b63d951b543aa83115e0e7b4550af4fa0d56780fc8fede890196", + "result_file_sha256": "bddfda0b83421ea19c31efae0763e4d882cd25d8f7fc51ecf50c2203c16f5d03", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-12-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-te-m0", + "case_id": "fresh-12-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "dcb1b840dbdcdf2d81c0a95349cc139654c19df93a9d0d71489f1b1f767761a5", + "result_file_sha256": "f6f15ce23c5eb8843b757dfdab575fa0efba2113144211af9b88ff80d2eff18e", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_performed conflicts with trusted intent and actual receipts, which establish not_requested." + }, + { + "job_id": "confirm-fresh-12-te-m1-source-chronological", + "history_id": "confirm-fresh-12-te-m1", + "case_id": "fresh-12-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "8d02e24f6a939eda3ceb1259da7123c36d6fce2ba7e80e6e78c63544f516f9c9", + "result_file_sha256": "8d9128c01cc9c00795ff8a0b689d4a4892c41abb9f5a6e7e79bc69704f2194fe", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-12-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-12-te-m1", + "case_id": "fresh-12-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "8d02e24f6a939eda3ceb1259da7123c36d6fce2ba7e80e6e78c63544f516f9c9", + "result_file_sha256": "8d9128c01cc9c00795ff8a0b689d4a4892c41abb9f5a6e7e79bc69704f2194fe", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-12-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-12-te-m1", + "case_id": "fresh-12-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "resolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "8d02e24f6a939eda3ceb1259da7123c36d6fce2ba7e80e6e78c63544f516f9c9", + "result_file_sha256": "8d9128c01cc9c00795ff8a0b689d4a4892c41abb9f5a6e7e79bc69704f2194fe", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-13-te-m0-source-chronological", + "history_id": "confirm-fresh-13-te-m0", + "case_id": "fresh-13-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "unresolved_condition", + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "date_only_match": false, + "clarification_match": false, + "terminal": "incomplete", + "raw_content_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure. No valid raw action status because the final is incomplete; counted incorrect." + }, + { + "job_id": "confirm-fresh-13-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-13-te-m0", + "case_id": "fresh-13-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "unresolved_condition", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": false, + "terminal": "answer", + "raw_content_sha256": "3be7e35ba83ddd02d3611c9f9823d34fcf7aef5b72761e62c517835aded0425b", + "result_file_sha256": "078d3f53025ce08cbdee6c9b66d1debeeb93a4fac3ea7fb3fdad08e9f2d8a614", + "review": "Raw 2027-03-14 selects a date despite the source leaving the replacement or condition unresolved. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-13-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-13-te-m0", + "case_id": "fresh-13-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "unresolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "1f143a6c338737b40763a356b6632537b85ce2afc83088c4f554fee275b1e164", + "result_file_sha256": "3a023f469ad2b66a2c3cccff96ecad9a50cb6437c0292e365161fb00067f87ce", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-13-te-m1-source-chronological", + "history_id": "confirm-fresh-13-te-m1", + "case_id": "fresh-13-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": false, + "terminal": "answer", + "raw_content_sha256": "7827a8c7b971979647ee14efc59af7031db8e64639d17be755062049fa079f44", + "result_file_sha256": "b6474bc8194ce66a7fd312e11be1d6465b06f536176a1bb68eebcb0a95a69d0d", + "review": "Raw 2027-03-14 selects a date despite the source leaving the replacement or condition unresolved. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-13-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-13-te-m1", + "case_id": "fresh-13-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-13-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-13-te-m1", + "case_id": "fresh-13-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-14-te-m0-source-chronological", + "history_id": "confirm-fresh-14-te-m0", + "case_id": "fresh-14-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "unresolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-14-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-14-te-m0", + "case_id": "fresh-14-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "unresolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-14-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-14-te-m0", + "case_id": "fresh-14-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "unresolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-14-te-m1-source-chronological", + "history_id": "confirm-fresh-14-te-m1", + "case_id": "fresh-14-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": false, + "terminal": "answer", + "raw_content_sha256": "f1cc3cfd197b822c6523ca628819fd8bab8da2efa0b004526ca96efcbb3be384", + "result_file_sha256": "318ed2b4a8a6682566450e991f6b72e55084ede01f0890501cedaa5bf0774042", + "review": "Raw 2027-04-15 selects a date despite the source leaving the replacement or condition unresolved. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-14-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-14-te-m1", + "case_id": "fresh-14-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": false, + "terminal": "answer", + "raw_content_sha256": "f1cc3cfd197b822c6523ca628819fd8bab8da2efa0b004526ca96efcbb3be384", + "result_file_sha256": "318ed2b4a8a6682566450e991f6b72e55084ede01f0890501cedaa5bf0774042", + "review": "Raw 2027-04-15 selects a date despite the source leaving the replacement or condition unresolved. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-14-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-14-te-m1", + "case_id": "fresh-14-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-15-te-m0-source-chronological", + "history_id": "confirm-fresh-15-te-m0", + "case_id": "fresh-15-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "unresolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-15-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-15-te-m0", + "case_id": "fresh-15-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "unresolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "1f143a6c338737b40763a356b6632537b85ce2afc83088c4f554fee275b1e164", + "result_file_sha256": "3a023f469ad2b66a2c3cccff96ecad9a50cb6437c0292e365161fb00067f87ce", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-15-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-15-te-m0", + "case_id": "fresh-15-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "unresolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-15-te-m1-source-chronological", + "history_id": "confirm-fresh-15-te-m1", + "case_id": "fresh-15-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": false, + "terminal": "answer", + "raw_content_sha256": "de515f172b9e6bd988e93a4bef206e9ce04f5942d82ca6219a2c20367efd4537", + "result_file_sha256": "1221a7e341e0ab559d66b45f568b82184bacfdecaa04abab155224785776dcee", + "review": "Raw 2027-03-16 selects a date despite the source leaving the replacement or condition unresolved. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-15-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-15-te-m1", + "case_id": "fresh-15-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": false, + "terminal": "answer", + "raw_content_sha256": "a3c47af1bf9caf574a51522c21d550db0a1e82e5621e992132138af3074ac320", + "result_file_sha256": "afd6ab4071b0df347bb5f8ee7e7e3b493e9f731b4ac18a0adbfee443737adcad", + "review": "Raw 2027-04-16 selects a date despite the source leaving the replacement or condition unresolved. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-15-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-15-te-m1", + "case_id": "fresh-15-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-16-te-m0-source-chronological", + "history_id": "confirm-fresh-16-te-m0", + "case_id": "fresh-16-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "unresolved_condition", + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "date_only_match": false, + "clarification_match": false, + "terminal": "incomplete", + "raw_content_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure. No valid raw action status because the final is incomplete; counted incorrect." + }, + { + "job_id": "confirm-fresh-16-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-16-te-m0", + "case_id": "fresh-16-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "unresolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-16-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-16-te-m0", + "case_id": "fresh-16-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "unresolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "debe6602cf176d0493e97c5781328d06f4bf761b38225a27bd95fc4456b03d7d", + "result_file_sha256": "b565e0bafafc68ef045e6a1a80a28d910d18f5cb2d1395cd4fc34e009f228214", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-16-te-m1-source-chronological", + "history_id": "confirm-fresh-16-te-m1", + "case_id": "fresh-16-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": false, + "terminal": "answer", + "raw_content_sha256": "75c6adfd0e8d53d51595c425215a8940b057c7b3c3e646327d765ea95b8653c8", + "result_file_sha256": "081f2013d93cd7b24a8f7a3eb964081ac78a1dfd0105add89dcdbd0bc3ba2e8d", + "review": "Raw 2027-04-17 selects a date despite the source leaving the replacement or condition unresolved. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-16-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-16-te-m1", + "case_id": "fresh-16-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": false, + "terminal": "answer", + "raw_content_sha256": "75c6adfd0e8d53d51595c425215a8940b057c7b3c3e646327d765ea95b8653c8", + "result_file_sha256": "081f2013d93cd7b24a8f7a3eb964081ac78a1dfd0105add89dcdbd0bc3ba2e8d", + "review": "Raw 2027-04-17 selects a date despite the source leaving the replacement or condition unresolved. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-16-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-16-te-m1", + "case_id": "fresh-16-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "unresolved_condition", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b146b30257265f5011b2a0913167ac94f8e83590857e72e0377bb6d62f6149b0", + "result_file_sha256": "3dc2d1d13053ffe46b623232da953ea67d5d574bade023b009b5b4f9636dbf47", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-17-te-m0-source-chronological", + "history_id": "confirm-fresh-17-te-m0", + "case_id": "fresh-17-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "separate_booking", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "9449807d1a496bfec71b5dbf1c86405ed0ae5d9745146d9437a26e0842b5006f", + "result_file_sha256": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-17-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-17-te-m0", + "case_id": "fresh-17-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "separate_booking", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "9449807d1a496bfec71b5dbf1c86405ed0ae5d9745146d9437a26e0842b5006f", + "result_file_sha256": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-17-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-17-te-m0", + "case_id": "fresh-17-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "separate_booking", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "9449807d1a496bfec71b5dbf1c86405ed0ae5d9745146d9437a26e0842b5006f", + "result_file_sha256": "760428af5173778aac2d9ee33cd7ca1160d42a46bef7564830989b40f6c7bde9", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-17-te-m1-source-chronological", + "history_id": "confirm-fresh-17-te-m1", + "case_id": "fresh-17-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "4070209a7773e6e396574c3cc304e3528574fb48855f2e8178fb6206d1c899c5", + "result_file_sha256": "87c81f64b32de0e465ec65f5672c4e2b281ceaddfdb2a11351a1e137c744f10c", + "review": "Raw 2027-03-18 differs from the source/actual-current target 2027-04-18; the raw date is unsupported for this question. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-17-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-17-te-m1", + "case_id": "fresh-17-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "4070209a7773e6e396574c3cc304e3528574fb48855f2e8178fb6206d1c899c5", + "result_file_sha256": "87c81f64b32de0e465ec65f5672c4e2b281ceaddfdb2a11351a1e137c744f10c", + "review": "Raw 2027-03-18 differs from the source/actual-current target 2027-04-18; the raw date is unsupported for this question. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-17-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-17-te-m1", + "case_id": "fresh-17-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "2d0061ed80efcc2974dfae76435db00a06c1b6479dfbefd7325a9093a9208008", + "result_file_sha256": "c906ece7cab4749ad05a7177d633850ff1d4fb81386b5fae720064c1ca673a6d", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-18-te-m0-source-chronological", + "history_id": "confirm-fresh-18-te-m0", + "case_id": "fresh-18-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "separate_booking", + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "date_only_match": false, + "clarification_match": false, + "terminal": "incomplete", + "raw_content_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure. No valid raw action status because the final is incomplete; counted incorrect." + }, + { + "job_id": "confirm-fresh-18-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-18-te-m0", + "case_id": "fresh-18-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "separate_booking", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "9f13b701995856fa703eec6041b4da4ba1c06e0ab9736e62b8ece9e50b06320e", + "result_file_sha256": "03bc7ffa97b58cb6893a53bd07ae07966616cd8c647d669421d85a572a958aae", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-18-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-te-m0", + "case_id": "fresh-18-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "separate_booking", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "9f13b701995856fa703eec6041b4da4ba1c06e0ab9736e62b8ece9e50b06320e", + "result_file_sha256": "03bc7ffa97b58cb6893a53bd07ae07966616cd8c647d669421d85a572a958aae", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-18-te-m1-source-chronological", + "history_id": "confirm-fresh-18-te-m1", + "case_id": "fresh-18-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "c89680404de922e7ad56887a56811c7ddb4eefcbb16c8b8689f6038edcee313e", + "result_file_sha256": "00d649cfc39ec358aa1d45e738f53feb9cb97e2c7b7fc9ca56a64eec5020214e", + "review": "Raw 2027-03-19 differs from the source/actual-current target 2027-04-19; the raw date is unsupported for this question. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-18-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-18-te-m1", + "case_id": "fresh-18-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "date_only_match": false, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "6e4c3efc9f6d0ac4c7148f8b28883e8b7b74253291cb26bd8f87cc4b99b2fddd", + "result_file_sha256": "c3381da56b5e4c327de6a23fba063f02f3de77e20c8c41a5ba4c99f2317591f0", + "review": "Raw 2027-03-19 differs from the source/actual-current target 2027-04-19; the raw date is unsupported for this question. Raw action status confirmed conflicts with trusted intent and actual receipts, which establish not_requested." + }, + { + "job_id": "confirm-fresh-18-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-18-te-m1", + "case_id": "fresh-18-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "c89680404de922e7ad56887a56811c7ddb4eefcbb16c8b8689f6038edcee313e", + "result_file_sha256": "00d649cfc39ec358aa1d45e738f53feb9cb97e2c7b7fc9ca56a64eec5020214e", + "review": "Raw 2027-03-19 differs from the source/actual-current target 2027-04-19; the raw date is unsupported for this question. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-19-te-m0-source-chronological", + "history_id": "confirm-fresh-19-te-m0", + "case_id": "fresh-19-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "separate_booking", + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "date_only_match": false, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "e414f0467a3a8ffd2922eb5949cd708aa0baeefb883102ebb36ed38ac393874b", + "result_file_sha256": "515ec2720e24ee39e5a4b433b38304fdcd549a00cc5b71768f7e357c2d975dfd", + "review": "Raw 2027-03-20 differs from the source/actual-current target 2027-04-20; the raw date is unsupported for this question. Raw action status confirmed conflicts with trusted intent and actual receipts, which establish not_requested." + }, + { + "job_id": "confirm-fresh-19-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-19-te-m0", + "case_id": "fresh-19-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "separate_booking", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "ccbb204d6da4516cd3f603b7585003889275c2ceab7bdb788b3288039143cf26", + "result_file_sha256": "4f2ff4c6daf1623e66caead1eec254086b77631cbd7ac1df8765b25a639a0c24", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-19-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-te-m0", + "case_id": "fresh-19-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "separate_booking", + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "date_only_match": false, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "dd8d02d6db895c5635fc959c9262b96346f67c6ef3ed6bc12ff404bf035f3754", + "result_file_sha256": "e1bb23a3fb2a08e8bd9ecf4790de66550406c987a5f2f7889494245ce06626c3", + "review": "Raw 2027-03-20 differs from the source/actual-current target 2027-04-20; the raw date is unsupported for this question. Raw action status not_performed conflicts with trusted intent and actual receipts, which establish not_requested." + }, + { + "job_id": "confirm-fresh-19-te-m1-source-chronological", + "history_id": "confirm-fresh-19-te-m1", + "case_id": "fresh-19-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "f4f4478b5c7f08706d0eb37b41fdb9c470082ee0f253ae4f0ca504e2eb97a9a7", + "result_file_sha256": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "review": "Raw 2027-03-20 differs from the source/actual-current target 2027-04-20; the raw date is unsupported for this question. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-19-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-19-te-m1", + "case_id": "fresh-19-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "f4f4478b5c7f08706d0eb37b41fdb9c470082ee0f253ae4f0ca504e2eb97a9a7", + "result_file_sha256": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "review": "Raw 2027-03-20 differs from the source/actual-current target 2027-04-20; the raw date is unsupported for this question. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-19-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-19-te-m1", + "case_id": "fresh-19-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "f4f4478b5c7f08706d0eb37b41fdb9c470082ee0f253ae4f0ca504e2eb97a9a7", + "result_file_sha256": "8d27d0988e48ee4a6128373c19e534121250951466f39892f0ddf3e8470e84f3", + "review": "Raw 2027-03-20 differs from the source/actual-current target 2027-04-20; the raw date is unsupported for this question. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-20-te-m0-source-chronological", + "history_id": "confirm-fresh-20-te-m0", + "case_id": "fresh-20-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "separate_booking", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "50dd30c16ff07995e873e614e9c42b88e61dd6fd45c0f360a0172c3453e458a3", + "result_file_sha256": "47da800200c0c2a40c8f6e6d4d56ab1df082b5204fe5af8fa9d8b032bf3fec83", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-20-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-20-te-m0", + "case_id": "fresh-20-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "separate_booking", + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "date_only_match": false, + "clarification_match": false, + "terminal": "answer", + "raw_content_sha256": "b0c8bdd4e4b8dbf54110b0365a94443a1ef48ea834bddaefce998503afaf7471", + "result_file_sha256": "44002938658ce12fa57d9ee0682ca5767a021114be6c0cbe8d8adbfea1b01f2a", + "review": "False abstention: source or final backend uniquely establishes 2027-04-21, but raw answer is null. Raw action status rejected conflicts with trusted intent and actual receipts, which establish not_requested." + }, + { + "job_id": "confirm-fresh-20-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-20-te-m0", + "case_id": "fresh-20-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "separate_booking", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "50dd30c16ff07995e873e614e9c42b88e61dd6fd45c0f360a0172c3453e458a3", + "result_file_sha256": "47da800200c0c2a40c8f6e6d4d56ab1df082b5204fe5af8fa9d8b032bf3fec83", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-20-te-m1-source-chronological", + "history_id": "confirm-fresh-20-te-m1", + "case_id": "fresh-20-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "efda99defce623b65285148dfd0b9814f4a3c673b764571d0d3827b37255a294", + "result_file_sha256": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "review": "Raw 2027-03-21 differs from the source/actual-current target 2027-04-21; the raw date is unsupported for this question. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-20-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-20-te-m1", + "case_id": "fresh-20-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "efda99defce623b65285148dfd0b9814f4a3c673b764571d0d3827b37255a294", + "result_file_sha256": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "review": "Raw 2027-03-21 differs from the source/actual-current target 2027-04-21; the raw date is unsupported for this question. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-20-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-20-te-m1", + "case_id": "fresh-20-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "separate_booking", + "factual_verdict": "incorrect", + "raw_status_verdict": "correct", + "date_only_match": false, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "efda99defce623b65285148dfd0b9814f4a3c673b764571d0d3827b37255a294", + "result_file_sha256": "d0198f6bf4f9507a0fedc67d042a6f0332d6129ceab838af8b8a5b307cb96749", + "review": "Raw 2027-03-21 differs from the source/actual-current target 2027-04-21; the raw date is unsupported for this question. Raw action status not_requested agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-21-te-m0-source-chronological", + "history_id": "confirm-fresh-21-te-m0", + "case_id": "fresh-21-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "2b038dfe210662433dc98cbece33644925efcac1dc025402d4110813a6ba5ac4", + "result_file_sha256": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-21-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-21-te-m0", + "case_id": "fresh-21-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "2b038dfe210662433dc98cbece33644925efcac1dc025402d4110813a6ba5ac4", + "result_file_sha256": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-21-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-21-te-m0", + "case_id": "fresh-21-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "2b038dfe210662433dc98cbece33644925efcac1dc025402d4110813a6ba5ac4", + "result_file_sha256": "6f04ba519ce62952b84725cde1345f3f7a7ea98b7cdde40833a2111e4ffd0a73", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-21-te-m1-source-chronological", + "history_id": "confirm-fresh-21-te-m1", + "case_id": "fresh-21-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "9649b662bcdbbb140ad57a183801a8f220b3a023c361cbd568d24381e0f098e1", + "result_file_sha256": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-21-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-21-te-m1", + "case_id": "fresh-21-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "9649b662bcdbbb140ad57a183801a8f220b3a023c361cbd568d24381e0f098e1", + "result_file_sha256": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-21-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-21-te-m1", + "case_id": "fresh-21-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "9649b662bcdbbb140ad57a183801a8f220b3a023c361cbd568d24381e0f098e1", + "result_file_sha256": "d400a4b0e4184f78d78b37892a79967d5ab6c6006d6ed2a7b80f981053b8ebe4", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-22-te-m0-source-chronological", + "history_id": "confirm-fresh-22-te-m0", + "case_id": "fresh-22-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "412d38ba18a68b8292a1112c6a6da160880d9f3ab5b1ae562dc88ac8bf6b7a42", + "result_file_sha256": "7a6034546c590ec5064db6d004a77a9dcf0a4f8eab5062272644c6750c85fef7", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-22-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-22-te-m0", + "case_id": "fresh-22-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "approved_change", + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "date_only_match": false, + "clarification_match": false, + "terminal": "incomplete", + "raw_content_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure. No valid raw action status because the final is incomplete; counted incorrect." + }, + { + "job_id": "confirm-fresh-22-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-22-te-m0", + "case_id": "fresh-22-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "412d38ba18a68b8292a1112c6a6da160880d9f3ab5b1ae562dc88ac8bf6b7a42", + "result_file_sha256": "7a6034546c590ec5064db6d004a77a9dcf0a4f8eab5062272644c6750c85fef7", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-22-te-m1-source-chronological", + "history_id": "confirm-fresh-22-te-m1", + "case_id": "fresh-22-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "3a77159e81f930d16df465bba0c47780df20a0d96e1d28d3144d74065e93efb7", + "result_file_sha256": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-22-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-22-te-m1", + "case_id": "fresh-22-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "3a77159e81f930d16df465bba0c47780df20a0d96e1d28d3144d74065e93efb7", + "result_file_sha256": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-22-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-22-te-m1", + "case_id": "fresh-22-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "3a77159e81f930d16df465bba0c47780df20a0d96e1d28d3144d74065e93efb7", + "result_file_sha256": "346ffe34786dcc73c365027c2240b09c753f43558f187f260f33c7892cd9286e", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-23-te-m0-source-chronological", + "history_id": "confirm-fresh-23-te-m0", + "case_id": "fresh-23-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "42e02e70a4fb69a05a1d57eab32d4ec5ccae0fd8c745b2c7d0f29e7d8ef48bb6", + "result_file_sha256": "0faf0e0192f0d7377eabd553b64abae59ded0e43cc4773b8a88de2b6c1adaeed", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-23-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-23-te-m0", + "case_id": "fresh-23-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "approved_change", + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "date_only_match": false, + "clarification_match": false, + "terminal": "incomplete", + "raw_content_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure. No valid raw action status because the final is incomplete; counted incorrect." + }, + { + "job_id": "confirm-fresh-23-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-23-te-m0", + "case_id": "fresh-23-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "approved_change", + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "date_only_match": false, + "clarification_match": false, + "terminal": "incomplete", + "raw_content_sha256": "7906ce5d89fbe60012b7b0dc72b1c11586a3529936de41abf7026d0c4b624947", + "result_file_sha256": "5e07ecfb62910b460fef6f452f1b4cf8e3705251a409fb26d17c729bb1105462", + "review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure. No valid raw action status because the final is incomplete; counted incorrect." + }, + { + "job_id": "confirm-fresh-23-te-m1-source-chronological", + "history_id": "confirm-fresh-23-te-m1", + "case_id": "fresh-23-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "2693a0032cdec8d9e8a477c15467e101f2a566f73bf63c4cb05aff66d210cfdc", + "result_file_sha256": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-23-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-23-te-m1", + "case_id": "fresh-23-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "2693a0032cdec8d9e8a477c15467e101f2a566f73bf63c4cb05aff66d210cfdc", + "result_file_sha256": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-23-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-23-te-m1", + "case_id": "fresh-23-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "2693a0032cdec8d9e8a477c15467e101f2a566f73bf63c4cb05aff66d210cfdc", + "result_file_sha256": "3a48b24e235452c70177ec3fbcd2eb736c39fb3a2ecbb2e673c7983ee55ded17", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-24-te-m0-source-chronological", + "history_id": "confirm-fresh-24-te-m0", + "case_id": "fresh-24-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "5f7884de70bae39ff454edda1e2dfc29c17845b488e477350cf0a7cc27c93739", + "result_file_sha256": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-24-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-24-te-m0", + "case_id": "fresh-24-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "5f7884de70bae39ff454edda1e2dfc29c17845b488e477350cf0a7cc27c93739", + "result_file_sha256": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-24-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-24-te-m0", + "case_id": "fresh-24-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "5f7884de70bae39ff454edda1e2dfc29c17845b488e477350cf0a7cc27c93739", + "result_file_sha256": "406b0242f87ac9095b4331a9b42586184b2f75cfd925da8637537b9ed57cb848", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-24-te-m1-source-chronological", + "history_id": "confirm-fresh-24-te-m1", + "case_id": "fresh-24-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "4913fc937b2d32f3dd8205703eb9608e0029f25eba1c8e31072557a2baa36b41", + "result_file_sha256": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-24-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-24-te-m1", + "case_id": "fresh-24-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "4913fc937b2d32f3dd8205703eb9608e0029f25eba1c8e31072557a2baa36b41", + "result_file_sha256": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-24-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-24-te-m1", + "case_id": "fresh-24-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "approved_change", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "4913fc937b2d32f3dd8205703eb9608e0029f25eba1c8e31072557a2baa36b41", + "result_file_sha256": "d7f6fa2c17818b5970fbd478fb7d91926409c7068d4c401b7f89c5a855597def", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-25-te-m0-source-chronological", + "history_id": "confirm-fresh-25-te-m0", + "case_id": "fresh-25-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "bd59ff2841c464354faf7faa7146bbbf8c2dd82eef7abb3e9321e9a86bc87da6", + "result_file_sha256": "258b73b129b801b662376d5cb0f39961843f2127df41664b4c2a54a8e81537fa", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-25-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-25-te-m0", + "case_id": "fresh-25-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "bd59ff2841c464354faf7faa7146bbbf8c2dd82eef7abb3e9321e9a86bc87da6", + "result_file_sha256": "258b73b129b801b662376d5cb0f39961843f2127df41664b4c2a54a8e81537fa", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-25-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-25-te-m0", + "case_id": "fresh-25-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "approval_boundary", + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "date_only_match": false, + "clarification_match": false, + "terminal": "incomplete", + "raw_content_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure. No valid raw action status because the final is incomplete; counted incorrect." + }, + { + "job_id": "confirm-fresh-25-te-m1-source-chronological", + "history_id": "confirm-fresh-25-te-m1", + "case_id": "fresh-25-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b67eb49d345d551277e52d9817ebc266663e8242554bf45e8cec67eff31a99b9", + "result_file_sha256": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-25-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-25-te-m1", + "case_id": "fresh-25-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b67eb49d345d551277e52d9817ebc266663e8242554bf45e8cec67eff31a99b9", + "result_file_sha256": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-25-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-25-te-m1", + "case_id": "fresh-25-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b67eb49d345d551277e52d9817ebc266663e8242554bf45e8cec67eff31a99b9", + "result_file_sha256": "9bfe622f47320700088a4380d6cfe8fabdf7c363cd9b471185977c86fc7d2454", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-26-te-m0-source-chronological", + "history_id": "confirm-fresh-26-te-m0", + "case_id": "fresh-26-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "c1e910975c3325f664d8f29ff0ec33c8158dfa51c312012dff2adc0d4cc8071a", + "result_file_sha256": "fb6e951f5e2bc3a3ec400cf4d93a3dfb2910e24d4ec7d9480d665ad63036999a", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-26-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-26-te-m0", + "case_id": "fresh-26-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "approval_boundary", + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "date_only_match": false, + "clarification_match": false, + "terminal": "incomplete", + "raw_content_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure. No valid raw action status because the final is incomplete; counted incorrect." + }, + { + "job_id": "confirm-fresh-26-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-26-te-m0", + "case_id": "fresh-26-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "c1e910975c3325f664d8f29ff0ec33c8158dfa51c312012dff2adc0d4cc8071a", + "result_file_sha256": "fb6e951f5e2bc3a3ec400cf4d93a3dfb2910e24d4ec7d9480d665ad63036999a", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-26-te-m1-source-chronological", + "history_id": "confirm-fresh-26-te-m1", + "case_id": "fresh-26-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "fa5d023d08c3ca91b18204d2fb9daa32d70361a98f2d49b310618c10718bd925", + "result_file_sha256": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-26-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-26-te-m1", + "case_id": "fresh-26-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "fa5d023d08c3ca91b18204d2fb9daa32d70361a98f2d49b310618c10718bd925", + "result_file_sha256": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-26-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-26-te-m1", + "case_id": "fresh-26-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "fa5d023d08c3ca91b18204d2fb9daa32d70361a98f2d49b310618c10718bd925", + "result_file_sha256": "977af60d492e4847953046a08f5fad87f3603a49753036b603cb3e4c87314ca3", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-27-te-m0-source-chronological", + "history_id": "confirm-fresh-27-te-m0", + "case_id": "fresh-27-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "9697ae0b043edc7459e3f0e7b3cf26431b534d4c4235ab6792946232f68e8762", + "result_file_sha256": "58dc4f555443172d26580e97135531691461045bdffc7d2a8802fe869294b394", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-27-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-27-te-m0", + "case_id": "fresh-27-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "ca5ffa82ffd7fbb61e75bdda4d8745aa04f7bd34f7a35241612277e52b52ef6a", + "result_file_sha256": "04c15f2e05362259fb2c00f48ac7ea6f32c4898d1dbb0891c1d767bef29d25f3", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-27-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-27-te-m0", + "case_id": "fresh-27-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "26abba6e33580488183b594c9706fd521c559b2197f1a520f16f47d109d0d268", + "result_file_sha256": "6dad95a7d69fa7ed7109363a832152dbf413d3cdb52e0fdfd1faa74c479378a2", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish confirmed." + }, + { + "job_id": "confirm-fresh-27-te-m1-source-chronological", + "history_id": "confirm-fresh-27-te-m1", + "case_id": "fresh-27-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "8608e83321d9a3eac9053988d3073197a955bcce5974202ee13444130cb83da2", + "result_file_sha256": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-27-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-27-te-m1", + "case_id": "fresh-27-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "8608e83321d9a3eac9053988d3073197a955bcce5974202ee13444130cb83da2", + "result_file_sha256": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-27-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-27-te-m1", + "case_id": "fresh-27-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "8608e83321d9a3eac9053988d3073197a955bcce5974202ee13444130cb83da2", + "result_file_sha256": "01e5caeb27990d6adc568b6f402bc9224ea1c61e77e9a44e19ea94e7e3ea8395", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-28-te-m0-source-chronological", + "history_id": "confirm-fresh-28-te-m0", + "case_id": "fresh-28-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "d8820bdeab4784bd4b5a0ce76c7cf37112f3a832e5aa7de53fe26a44cbd97a5d", + "result_file_sha256": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-28-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-28-te-m0", + "case_id": "fresh-28-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "d8820bdeab4784bd4b5a0ce76c7cf37112f3a832e5aa7de53fe26a44cbd97a5d", + "result_file_sha256": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-28-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-28-te-m0", + "case_id": "fresh-28-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "d8820bdeab4784bd4b5a0ce76c7cf37112f3a832e5aa7de53fe26a44cbd97a5d", + "result_file_sha256": "a9588372038c878ebe246c5513be5b1a0ea45b8e4d89e1fc70d7fbd41f8f87f0", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-28-te-m1-source-chronological", + "history_id": "confirm-fresh-28-te-m1", + "case_id": "fresh-28-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "824a909cf59990823dc60cd450be085d28013761e6c172af9edfa8a974c04ce2", + "result_file_sha256": "f70bc8efb6e5c2baaed099be3abea5bef92a529c53415c4cf1f762b65736aef1", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-28-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-28-te-m1", + "case_id": "fresh-28-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "824a909cf59990823dc60cd450be085d28013761e6c172af9edfa8a974c04ce2", + "result_file_sha256": "f70bc8efb6e5c2baaed099be3abea5bef92a529c53415c4cf1f762b65736aef1", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-28-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-28-te-m1", + "case_id": "fresh-28-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "approval_boundary", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "824a909cf59990823dc60cd450be085d28013761e6c172af9edfa8a974c04ce2", + "result_file_sha256": "f70bc8efb6e5c2baaed099be3abea5bef92a529c53415c4cf1f762b65736aef1", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status confirmed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-29-te-m0-source-chronological", + "history_id": "confirm-fresh-29-te-m0", + "case_id": "fresh-29-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "6f60dea4694b6d52e1bbd9a2b2fe175afd3782c6eb1f13cf983f53e25c6b19cd", + "result_file_sha256": "f517b9fdb42a8c5fdc3536e2720b69744ec169f1684a3271023240b26b8c846f", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_performed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-29-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-29-te-m0", + "case_id": "fresh-29-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "6f60dea4694b6d52e1bbd9a2b2fe175afd3782c6eb1f13cf983f53e25c6b19cd", + "result_file_sha256": "f517b9fdb42a8c5fdc3536e2720b69744ec169f1684a3271023240b26b8c846f", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_performed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-29-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-29-te-m0", + "case_id": "fresh-29-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "0163f4a41e8881b6b015044ce2a23ad1adb4ba560451442685979f2b5b6a7fff", + "result_file_sha256": "9a33d005b8af04a9daae27cfbec0007280a387d73264252880042af4f86632aa", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-29-te-m1-source-chronological", + "history_id": "confirm-fresh-29-te-m1", + "case_id": "fresh-29-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "c7b8d9498405aff859e7cd3c35981ee7d640e01038a3105a846ac123bac08074", + "result_file_sha256": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status rejected agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-29-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-29-te-m1", + "case_id": "fresh-29-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "c7b8d9498405aff859e7cd3c35981ee7d640e01038a3105a846ac123bac08074", + "result_file_sha256": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status rejected agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-29-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-29-te-m1", + "case_id": "fresh-29-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "c7b8d9498405aff859e7cd3c35981ee7d640e01038a3105a846ac123bac08074", + "result_file_sha256": "4d0aad268b9d864a6159bbf477834f3363f199fac9f9102d53d3eaa313643433", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status rejected agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-30-te-m0-source-chronological", + "history_id": "confirm-fresh-30-te-m0", + "case_id": "fresh-30-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "cca5bd3578575b6d45284125d6a11cca2ba727e2b310e9f5ec6cd6bef932def8", + "result_file_sha256": "c225aab8d19cca2515a377d9c5462cc34a003a0c3653544bba785e793db51707", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-30-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-30-te-m0", + "case_id": "fresh-30-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "cca5bd3578575b6d45284125d6a11cca2ba727e2b310e9f5ec6cd6bef932def8", + "result_file_sha256": "c225aab8d19cca2515a377d9c5462cc34a003a0c3653544bba785e793db51707", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-30-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-te-m0", + "case_id": "fresh-30-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "cca5bd3578575b6d45284125d6a11cca2ba727e2b310e9f5ec6cd6bef932def8", + "result_file_sha256": "c225aab8d19cca2515a377d9c5462cc34a003a0c3653544bba785e793db51707", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-30-te-m1-source-chronological", + "history_id": "confirm-fresh-30-te-m1", + "case_id": "fresh-30-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "31b20e9d2fd9f0fa5256283e857194fb3888033a9b8acd91e893978a488f41a6", + "result_file_sha256": "87902016aae6283bdff9207729e01b30ad77cad918dba806396649f58f68c74c", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status rejected agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-30-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-30-te-m1", + "case_id": "fresh-30-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "4a9f4305f0d97a49c966f980e6529f6c921798fc33742364cdcd76ba89fbee50", + "result_file_sha256": "da50423f0bc90841c2b9e138e9f4a9ad0340b0ec5d888b3091878471fa1a28a5", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_performed conflicts with trusted intent and actual receipts, which establish rejected." + }, + { + "job_id": "confirm-fresh-30-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-30-te-m1", + "case_id": "fresh-30-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "a30aacee00152000aeb8c2565229b22fcef45fa466e22daf1ea4c8d587176f57", + "result_file_sha256": "fc8165ed660426c4e3123966abfd67ef96b0cbe2d8cd6f54949fa8d3ece0de02", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish rejected." + }, + { + "job_id": "confirm-fresh-31-te-m0-source-chronological", + "history_id": "confirm-fresh-31-te-m0", + "case_id": "fresh-31-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "a1f9fcadc6ba4e8eafc46ac98450dd9b630b034f69ac35fd2a9eecd2706d424e", + "result_file_sha256": "fe4c3267a9cec82092e8ede130613eed251227abe98c2163619dbfcb8dccec96", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_performed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-31-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-31-te-m0", + "case_id": "fresh-31-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "564e8f5c169211638a274eece597eb4b2d5f7ccfac6aedfd9d936dc2a89cb610", + "result_file_sha256": "b5770e7ca54498bbe9cbc192d959a98975dc03170a1aa14ce7055f82160458c4", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-31-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-31-te-m0", + "case_id": "fresh-31-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "future_approval", + "factual_verdict": "incorrect", + "raw_status_verdict": "incorrect", + "date_only_match": false, + "clarification_match": false, + "terminal": "incomplete", + "raw_content_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "result_file_sha256": "b31ce1802301828daedfd3566225875b10cc1692a99f7d9e10ec5e63e8a3778d", + "review": "No valid final JSON answer (terminal incomplete); counted as a planned factual failure. No valid raw action status because the final is incomplete; counted incorrect." + }, + { + "job_id": "confirm-fresh-31-te-m1-source-chronological", + "history_id": "confirm-fresh-31-te-m1", + "case_id": "fresh-31-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "0d3aa07f0f0dff3403e3ba14d6cb185cfb839350bb5d46026f3e377de4a0b70f", + "result_file_sha256": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status rejected agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-31-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-31-te-m1", + "case_id": "fresh-31-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "0d3aa07f0f0dff3403e3ba14d6cb185cfb839350bb5d46026f3e377de4a0b70f", + "result_file_sha256": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status rejected agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-31-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-31-te-m1", + "case_id": "fresh-31-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "0d3aa07f0f0dff3403e3ba14d6cb185cfb839350bb5d46026f3e377de4a0b70f", + "result_file_sha256": "d2474e33b693b0b610ded914ba8af63b3522db7b3ad827ccd1f9435da55e4765", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status rejected agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-32-te-m0-source-chronological", + "history_id": "confirm-fresh-32-te-m0", + "case_id": "fresh-32-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "995faa359044c9d6d14445c6b8c04426bf5850fc3e064c57f837cdce5fd3b5e5", + "result_file_sha256": "2465b6c24f7e04b712333958ffe384554975d4fedbb6dadba79b616478a8eb65", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-32-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-32-te-m0", + "case_id": "fresh-32-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "f08325c4871b8031e4648a53e1cec3da798b14ad1f6ccd0f41aae4349a10b13e", + "result_file_sha256": "b106ac717f61df19b67b9ff99d9131e9d442ef563917d74f9f34670967d64074", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_performed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-32-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-te-m0", + "case_id": "fresh-32-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "6c8885ac0399db29e7fc2b456e2a7ea8317d20cc61747dfb28345ea9e01cd577", + "result_file_sha256": "c8610025a759bbe68179e78a62da3fb5774353b90330e6f59855fd3e75108541", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_performed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-32-te-m1-source-chronological", + "history_id": "confirm-fresh-32-te-m1", + "case_id": "fresh-32-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "1f2cad9a533345f41eea593348fa91f701d75caa054b15520f51e90a290da757", + "result_file_sha256": "b5537fea0aacf9bcedfc1a598c3b4e08a55b92f5c5cb181429cff1acda690630", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_performed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-32-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-32-te-m1", + "case_id": "fresh-32-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "6d2640aaf6fb64c2eacb3520af3cebfc49d073ff5ea8b2b60feb45f961029035", + "result_file_sha256": "6339bfc3a8c7c5365b3d444ec9928fe27e6a132032e7c64c5a5b885479a0be7a", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-32-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-32-te-m1", + "case_id": "fresh-32-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "future_approval", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "6d2640aaf6fb64c2eacb3520af3cebfc49d073ff5ea8b2b60feb45f961029035", + "result_file_sha256": "6339bfc3a8c7c5365b3d444ec9928fe27e6a132032e7c64c5a5b885479a0be7a", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-33-te-m0-source-chronological", + "history_id": "confirm-fresh-33-te-m0", + "case_id": "fresh-33-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "6b7d837755fee0f4384fc2b064b5188c55989832ef709b05dbeab7ee721fed6c", + "result_file_sha256": "c1553fd0523b1529616e1962bbf6812c5dd94faf93dff31d113956ebebf274ad", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-33-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-33-te-m0", + "case_id": "fresh-33-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "6b7d837755fee0f4384fc2b064b5188c55989832ef709b05dbeab7ee721fed6c", + "result_file_sha256": "c1553fd0523b1529616e1962bbf6812c5dd94faf93dff31d113956ebebf274ad", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-33-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-te-m0", + "case_id": "fresh-33-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "6b7d837755fee0f4384fc2b064b5188c55989832ef709b05dbeab7ee721fed6c", + "result_file_sha256": "c1553fd0523b1529616e1962bbf6812c5dd94faf93dff31d113956ebebf274ad", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-33-te-m1-source-chronological", + "history_id": "confirm-fresh-33-te-m1", + "case_id": "fresh-33-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b5a9b2a643044e8e02fbaa24273ac400f027c86f26590128838323f2715e255e", + "result_file_sha256": "84391b7e3176e5c1aed2756ac5fc41fe066cc25886a30c9f667429bd8fa83637", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-33-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-33-te-m1", + "case_id": "fresh-33-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "0d5919b20e564dc0d5fbc550dc3a613bc37b60a64f55e1b1d48adadf45f1ce1a", + "result_file_sha256": "0e529979621b57ba683527909c61a095b19f49f46d26c9741184bd4c71571e88", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-33-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-33-te-m1", + "case_id": "fresh-33-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b5a9b2a643044e8e02fbaa24273ac400f027c86f26590128838323f2715e255e", + "result_file_sha256": "84391b7e3176e5c1aed2756ac5fc41fe066cc25886a30c9f667429bd8fa83637", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-34-te-m0-source-chronological", + "history_id": "confirm-fresh-34-te-m0", + "case_id": "fresh-34-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b6c41db1404ae24ceb4c157bac584d8a6eb112ca63ea75ab1e1815508b60ce23", + "result_file_sha256": "e06779a9cb208c2b4c218b7b33cb7b15f6617556fb3cb9cb92da244af97c2080", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_performed conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-34-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-34-te-m0", + "case_id": "fresh-34-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "6586f51a831b7caa739d8a82e7b4fa0ab9b8b951e604255d1abf7cf47d787c1f", + "result_file_sha256": "abd46c530745df8728e321ec83dc35accf4d0d8c2c4080a3ad91e1f119565685", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-34-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-te-m0", + "case_id": "fresh-34-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "1f9aae50f890487639284f3859153ccbb11bf0e46aae9b20c3faaaed2e29b79d", + "result_file_sha256": "724654f4a7af6c72796792e733aac468132ed9975139bc88bbc067193c76376a", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_performed conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-34-te-m1-source-chronological", + "history_id": "confirm-fresh-34-te-m1", + "case_id": "fresh-34-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "d37ecd59c05b45200bc0a7393dd814c4e477d5816ec0bff4fe8eae3cc3ff484a", + "result_file_sha256": "fb15a37da4fb1c488e9b16ecb891c11ca2d0d116496c521e1c600d6a55309e25", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-34-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-34-te-m1", + "case_id": "fresh-34-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "d37ecd59c05b45200bc0a7393dd814c4e477d5816ec0bff4fe8eae3cc3ff484a", + "result_file_sha256": "fb15a37da4fb1c488e9b16ecb891c11ca2d0d116496c521e1c600d6a55309e25", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-34-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-34-te-m1", + "case_id": "fresh-34-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "0dede5205ccb97ca95a65809101dd03fc5a8724efaeaca1d97fffdd056f35a2f", + "result_file_sha256": "042713423da807bcfef49b2ccab9db9d7aee0d0c696115936fca95c315d2e3bb", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-35-te-m0-source-chronological", + "history_id": "confirm-fresh-35-te-m0", + "case_id": "fresh-35-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "a7a213c208ae83606f4551a683c7a4409c60837bc26f45ac19a6b96446e83525", + "result_file_sha256": "b31d0f9b5f8fe5765afbcb1eaf44d89c8c5492e0a28b6d2d4ff5fe02f1c09f85", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-35-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-35-te-m0", + "case_id": "fresh-35-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "119943d853ec20d1fbebd8494e527c9f0412a40f3b41b6988593e958e52ea69d", + "result_file_sha256": "783d06630a3c4f84e56203d8144a5e4ccdbb51803d241e952be528d246e6346c", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-35-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-te-m0", + "case_id": "fresh-35-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "0249c0fa5de0e696a7af9d1e4abe253d1025cd3505514ce0606fed5166205f00", + "result_file_sha256": "296cb3c547ead0b0a5b30d3e7e77c560bc5d057059bb3ea526c0c2bd94e0c71e", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-35-te-m1-source-chronological", + "history_id": "confirm-fresh-35-te-m1", + "case_id": "fresh-35-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "baee28406332678362f125cfdb084fb35a70df7dda2b9c89c9ec8091f79aede6", + "result_file_sha256": "a10e59f23e1bcbeb10d0bb5925482d8103b4a143246bb28f303d672b12eeb2f2", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-35-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-35-te-m1", + "case_id": "fresh-35-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "5ccdaa467bf10798802cf3b20863dd51a0c1536186cc3eecf0a779a2bd89cb44", + "result_file_sha256": "10dca089506ffd53f9cf8d47ddf457b0596b2a838a3856eebeac46979cd679d9", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-35-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-35-te-m1", + "case_id": "fresh-35-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "baee28406332678362f125cfdb084fb35a70df7dda2b9c89c9ec8091f79aede6", + "result_file_sha256": "a10e59f23e1bcbeb10d0bb5925482d8103b4a143246bb28f303d672b12eeb2f2", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-36-te-m0-source-chronological", + "history_id": "confirm-fresh-36-te-m0", + "case_id": "fresh-36-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "2803662ff40dd4f63615eb045495ce5c39d4468bff1e77d7ea10ac817822af70", + "result_file_sha256": "f09ee14139dffa4254eb8299e58575fdc38b4acb79393af281e81167c8fe2c0b", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-36-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-36-te-m0", + "case_id": "fresh-36-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "2803662ff40dd4f63615eb045495ce5c39d4468bff1e77d7ea10ac817822af70", + "result_file_sha256": "f09ee14139dffa4254eb8299e58575fdc38b4acb79393af281e81167c8fe2c0b", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-36-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-te-m0", + "case_id": "fresh-36-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "2803662ff40dd4f63615eb045495ce5c39d4468bff1e77d7ea10ac817822af70", + "result_file_sha256": "f09ee14139dffa4254eb8299e58575fdc38b4acb79393af281e81167c8fe2c0b", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-36-te-m1-source-chronological", + "history_id": "confirm-fresh-36-te-m1", + "case_id": "fresh-36-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "efc6a4077028e288334cb3a79d28cec7f9704053928331652c0ff2cd591ea443", + "result_file_sha256": "5766d8a3d2817eb1b9e489c5c75934249fc394e5c0209fb689239f6b03244dca", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-36-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-36-te-m1", + "case_id": "fresh-36-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "efc6a4077028e288334cb3a79d28cec7f9704053928331652c0ff2cd591ea443", + "result_file_sha256": "5766d8a3d2817eb1b9e489c5c75934249fc394e5c0209fb689239f6b03244dca", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-36-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-36-te-m1", + "case_id": "fresh-36-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "completed_older_request", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "861aec082087961b93d13c7f30faddb78998f3b9bc4544a583396aa8fae6587a", + "result_file_sha256": "807ff01b1b38bc10bcd9060bd0d3b1ae9c42d960b9a0482d8c03411278d3cf2d", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish already_applied." + }, + { + "job_id": "confirm-fresh-37-te-m0-source-chronological", + "history_id": "confirm-fresh-37-te-m0", + "case_id": "fresh-37-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b91cbcd7f19d9fbab62d34814d050c116d17f3c029abf11a8767079f0385702d", + "result_file_sha256": "69dde94d76c4033be2860a97d246a801af8bfcec0b584aa6c9f0986a935603bb", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-37-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-37-te-m0", + "case_id": "fresh-37-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b91cbcd7f19d9fbab62d34814d050c116d17f3c029abf11a8767079f0385702d", + "result_file_sha256": "69dde94d76c4033be2860a97d246a801af8bfcec0b584aa6c9f0986a935603bb", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-37-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-te-m0", + "case_id": "fresh-37-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "2292869e10a5a50d27fb1494433690cd1b3ef9e542b822350d3bd16e394d5c9e", + "result_file_sha256": "aff48058a24de4a1db9e0fa8c903c6c318fd62f91bba7bbd8e5d3168a70bc729", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-37-te-m1-source-chronological", + "history_id": "confirm-fresh-37-te-m1", + "case_id": "fresh-37-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "f025c2199b98f91f9bd992c7a9c8f85467b1b8ebc43dea9ba9165d4afb289c52", + "result_file_sha256": "2c3fdbb6856d232762072d5920f57471f718b0f9b72563994c9e7d9b1e6852d7", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_performed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-37-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-37-te-m1", + "case_id": "fresh-37-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "4c84e6bba619478a332d698ad106affca009aad59721341c37f525f7e76e7b83", + "result_file_sha256": "cfd9d0213e93bbf6f5cb1029acb085a29aded09fa4637c7b47c8f4cf1e2d4a44", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status rejected conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-37-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-37-te-m1", + "case_id": "fresh-37-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "404a69ff9e54f916db69a6ce6d90ff60c79f4d89c14a3a7f677443f9092339cc", + "result_file_sha256": "417418804f246865317d445869ff277d57e1f417198b5646e44534270893ef1f", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-38-te-m0-source-chronological", + "history_id": "confirm-fresh-38-te-m0", + "case_id": "fresh-38-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "0c4812369e0d55fa35c84fa8fee15ea63e76b40cbe4d6362cf99bfea51d639e0", + "result_file_sha256": "a83af413afe7cddcaa541b7076f09d4924ecc2e0a192a002771a74e3c3fadc75", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-38-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-38-te-m0", + "case_id": "fresh-38-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "0c4812369e0d55fa35c84fa8fee15ea63e76b40cbe4d6362cf99bfea51d639e0", + "result_file_sha256": "a83af413afe7cddcaa541b7076f09d4924ecc2e0a192a002771a74e3c3fadc75", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-38-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-te-m0", + "case_id": "fresh-38-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "0875913962755e104e569968b4c0b381161a16bcb57f4ff6e928b39498b68ec5", + "result_file_sha256": "06c2187ceae2c64dea1bf91ec923b84dd3573f34714a5aafeb636bfe711a773e", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-38-te-m1-source-chronological", + "history_id": "confirm-fresh-38-te-m1", + "case_id": "fresh-38-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "5b31fb7ca17e78a90f4ad1615cc4f0f6f2ff976799a03a73715a9d74c4e47c88", + "result_file_sha256": "68edae63c30afcb8dede4b7e2cdfb966d8dced07732f78d2a0d7067c4a5c045e", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status rejected conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-38-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-38-te-m1", + "case_id": "fresh-38-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "5b31fb7ca17e78a90f4ad1615cc4f0f6f2ff976799a03a73715a9d74c4e47c88", + "result_file_sha256": "68edae63c30afcb8dede4b7e2cdfb966d8dced07732f78d2a0d7067c4a5c045e", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status rejected conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-38-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-38-te-m1", + "case_id": "fresh-38-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "69e809991e2e6516871589d05968ed8d6dbfb4311f9ef072ce2282e6760d4834", + "result_file_sha256": "7bb3b716e7f26e456befff8ad0778da3cde694aa22a37e4e06202fb80e43454b", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-39-te-m0-source-chronological", + "history_id": "confirm-fresh-39-te-m0", + "case_id": "fresh-39-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b3ec4557109cc643fdfe76911b4b72140a78dbcd5ad0dc05a75ced885889e007", + "result_file_sha256": "96b42d2cc3de69828ffb97c140cf9d47de4b7c1c2276847078932bfeadb2f1f8", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-39-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-39-te-m0", + "case_id": "fresh-39-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "4e0a3ab3a5c9a573a5295365d934a1f59cfa12169ca24dc63e30bc8ffd9495f4", + "result_file_sha256": "86ce10b21452933ea2f017456ad1d7f66046b05fe0cf423ddbe8220fa19420f2", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-39-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-te-m0", + "case_id": "fresh-39-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "b3ec4557109cc643fdfe76911b4b72140a78dbcd5ad0dc05a75ced885889e007", + "result_file_sha256": "96b42d2cc3de69828ffb97c140cf9d47de4b7c1c2276847078932bfeadb2f1f8", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-39-te-m1-source-chronological", + "history_id": "confirm-fresh-39-te-m1", + "case_id": "fresh-39-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "d899426e8ef4e447616ea48b973b02b64065c0a82d73ad786f4d64491f5ddae4", + "result_file_sha256": "16e46d320572a55d27d1366cb2cc4864e58b19b8a19e6e0b5044ec39ab926afc", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status rejected conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-39-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-39-te-m1", + "case_id": "fresh-39-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "057425dcb1fdd6d6ccad3bd4cad69751520540732ee2ff9949cb04440da4e058", + "result_file_sha256": "f2371cd3e60dbad81739ea12fffa281dda1158ef039ec6a8984449c96e971832", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-39-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-39-te-m1", + "case_id": "fresh-39-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "6a4974639e1edd8ecb3f0201898f2d4823422d6695b0ad27708f93e554c2e3c1", + "result_file_sha256": "f79a657b75b3fc410aff8d435eeaf3c5ba5d5672d55389524137ffcefa9943dc", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-40-te-m0-source-chronological", + "history_id": "confirm-fresh-40-te-m0", + "case_id": "fresh-40-te", + "arm": "chronological", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "9dac33ab5e4882ec69311a19f72c1330ec8f057e21a912a81f03d812c2e99127", + "result_file_sha256": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-40-te-m0-source-grouped_lossless", + "history_id": "confirm-fresh-40-te-m0", + "case_id": "fresh-40-te", + "arm": "grouped_lossless", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "9dac33ab5e4882ec69311a19f72c1330ec8f057e21a912a81f03d812c2e99127", + "result_file_sha256": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-40-te-m0-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-te-m0", + "case_id": "fresh-40-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "9dac33ab5e4882ec69311a19f72c1330ec8f057e21a912a81f03d812c2e99127", + "result_file_sha256": "85f8497768e1dfdac3552a6c5a0e26282f0e0dbd4f92478717f948a312142d6a", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-40-te-m1-source-chronological", + "history_id": "confirm-fresh-40-te-m1", + "case_id": "fresh-40-te", + "arm": "chronological", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "correct", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "727a7f58f51aa4f83de71e27cab3c77950ad8058f008e847b9b26fc4f5c24860", + "result_file_sha256": "e154d7d3fea97d5d74e19e49727b25d73019ead962c7b21da19391b8aa91e911", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_performed agrees with trusted intent and the actual prior/update receipts." + }, + { + "job_id": "confirm-fresh-40-te-m1-source-grouped_lossless", + "history_id": "confirm-fresh-40-te-m1", + "case_id": "fresh-40-te", + "arm": "grouped_lossless", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "c7f328d78597435701fad57703175eeab15cace2a91f54b5d85e68d150fb7bca", + "result_file_sha256": "fca49d1de4a1905da389c1cef4610aee77922b07c9c45ea909d59d69c71cb0ab", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status already_applied conflicts with trusted intent and actual receipts, which establish not_performed." + }, + { + "job_id": "confirm-fresh-40-te-m1-source-grouped_actor_prose_withheld", + "history_id": "confirm-fresh-40-te-m1", + "case_id": "fresh-40-te", + "arm": "grouped_actor_prose_withheld", + "model": "sarvam-105b-conversations", + "family": "conflicting_request_key", + "factual_verdict": "correct", + "raw_status_verdict": "incorrect", + "date_only_match": true, + "clarification_match": true, + "terminal": "answer", + "raw_content_sha256": "2655156e40334761b11243498c952a7bfa50bffe056b93d1424ef09be4cb336c", + "result_file_sha256": "01c51cb45c19f670d63319073f42596d8ac9ae62157693f1179ca782c611a4e7", + "review": "The raw date/null and clarification flag match the reviewed source or scoped final backend. Raw action status not_requested conflicts with trusted intent and actual receipts, which establish not_performed." + } + ], + "method": "Assistant read all 40 Telugu question/source sets, both actual native histories per case, and all 240 new raw A/B/C outputs. Date/null, clarification, and raw action status were judged against source meaning, scoped backend state, and actual prior/update receipts. Independent stdlib receipt/meter replay was cross-checked. This is not native-speaker or human validation.", + "limitations": [ + "These are synthetic related development workflows; translations and model/arm repeats are not independent cases.", + "No actor or backend action was rerun in this phase. The 80 native histories are frozen from V3.", + "Output caps and invalid finals count as planned factual/status failures. A null date with false clarification is a contract miss even when date-only matches.", + "A raw action-status error is recorded separately from a correct date; no model-authored claim substitutes for an actual receipt.", + "No causal mechanism or model-wide result follows from this Telugu slice alone." + ] +} diff --git a/benchmarks/audits/sarvam-source-interpretation-2026-10-13/summary.json b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/summary.json new file mode 100644 index 0000000..ca18301 --- /dev/null +++ b/benchmarks/audits/sarvam-source-interpretation-2026-10-13/summary.json @@ -0,0 +1,2147 @@ +{ + "status": "DESCRIPTIVE_TABLES_FROM_FROZEN_AUDIT", + "model_arm": { + "sarvam-105b/chronological": { + "n": 120, + "factual_correct": 104, + "valid_final": 115, + "invalid_or_missing_final": 5, + "wrong_non_null_dates": 3, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 77, + "projected_status_correct": 115, + "raw_fully_correct_reports": 70, + "projected_fully_correct_reports": 104, + "known_prompt_tokens": 148412, + "known_completion_tokens": 393154, + "known_cost_microinr": 25815420, + "retained_microinr": 32269322, + "elapsed_seconds_sum": 1825.3474645020033, + "elapsed_seconds_count": 120, + "accuracy_percent": 86.66666666666667, + "terminal_counts": { + "answer": 115, + "incomplete": 5 + }, + "median_request_seconds": 12.83493852099491, + "p95_request_seconds": 31.6984588750056 + }, + "sarvam-105b/grouped_actor_prose_withheld": { + "n": 120, + "factual_correct": 105, + "valid_final": 114, + "invalid_or_missing_final": 6, + "wrong_non_null_dates": 1, + "false_abstentions": 2, + "unknown_condition_guesses": 0, + "raw_status_correct": 85, + "projected_status_correct": 114, + "raw_fully_correct_reports": 78, + "projected_fully_correct_reports": 105, + "known_prompt_tokens": 199853, + "known_completion_tokens": 415826, + "known_cost_microinr": 27947355, + "retained_microinr": 34934238, + "elapsed_seconds_sum": 1956.6855917031207, + "elapsed_seconds_count": 120, + "accuracy_percent": 87.5, + "terminal_counts": { + "answer": 114, + "incomplete": 6 + }, + "median_request_seconds": 13.584956645499915, + "p95_request_seconds": 34.90903183299815 + }, + "sarvam-105b/grouped_lossless": { + "n": 120, + "factual_correct": 99, + "valid_final": 112, + "invalid_or_missing_final": 8, + "wrong_non_null_dates": 4, + "false_abstentions": 1, + "unknown_condition_guesses": 2, + "raw_status_correct": 83, + "projected_status_correct": 112, + "raw_fully_correct_reports": 71, + "projected_fully_correct_reports": 99, + "known_prompt_tokens": 212692, + "known_completion_tokens": 445020, + "known_cost_microinr": 29891580, + "retained_microinr": 37364522, + "elapsed_seconds_sum": 2091.594803958971, + "elapsed_seconds_count": 120, + "accuracy_percent": 82.5, + "terminal_counts": { + "answer": 112, + "incomplete": 8 + }, + "median_request_seconds": 14.838648624994676, + "p95_request_seconds": 35.501241707999725 + }, + "sarvam-105b-conversations/chronological": { + "n": 120, + "factual_correct": 96, + "valid_final": 120, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 22, + "false_abstentions": 2, + "unknown_condition_guesses": 8, + "raw_status_correct": 109, + "projected_status_correct": 120, + "raw_fully_correct_reports": 86, + "projected_fully_correct_reports": 96, + "known_prompt_tokens": 163171, + "known_completion_tokens": 4808, + "known_cost_microinr": 2736045, + "retained_microinr": 3420099, + "elapsed_seconds_sum": 121.35232799799996, + "elapsed_seconds_count": 120, + "accuracy_percent": 80.0, + "terminal_counts": { + "answer": 120 + }, + "median_request_seconds": 0.9965718329985975, + "p95_request_seconds": 1.1151200840104138 + }, + "sarvam-105b-conversations/grouped_actor_prose_withheld": { + "n": 120, + "factual_correct": 103, + "valid_final": 120, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 8, + "false_abstentions": 9, + "unknown_condition_guesses": 2, + "raw_status_correct": 97, + "projected_status_correct": 120, + "raw_fully_correct_reports": 80, + "projected_fully_correct_reports": 103, + "known_prompt_tokens": 223650, + "known_completion_tokens": 4674, + "known_cost_microinr": 3635190, + "retained_microinr": 4544031, + "elapsed_seconds_sum": 124.40037220805243, + "elapsed_seconds_count": 120, + "accuracy_percent": 85.83333333333333, + "terminal_counts": { + "answer": 120 + }, + "median_request_seconds": 0.995008437501383, + "p95_request_seconds": 1.1882771249947837 + }, + "sarvam-105b-conversations/grouped_lossless": { + "n": 120, + "factual_correct": 100, + "valid_final": 120, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 18, + "false_abstentions": 2, + "unknown_condition_guesses": 6, + "raw_status_correct": 95, + "projected_status_correct": 120, + "raw_fully_correct_reports": 78, + "projected_fully_correct_reports": 100, + "known_prompt_tokens": 238962, + "known_completion_tokens": 4774, + "known_cost_microinr": 3870870, + "retained_microinr": 4838631, + "elapsed_seconds_sum": 122.39304328711296, + "elapsed_seconds_count": 120, + "accuracy_percent": 83.33333333333333, + "terminal_counts": { + "answer": 120 + }, + "median_request_seconds": 1.002929166505055, + "p95_request_seconds": 1.1136486670002341 + } + }, + "paired_contrasts": { + "A_to_B": { + "all/sarvam-105b": { + "n": 120, + "left_correct": 104, + "right_correct": 99, + "gain": 7, + "loss": 12, + "net": -5, + "restored_from_invalid": 4, + "corrected_usable_answer": 3, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.10833333333333334, + 0.025 + ], + "percentile_95_percentage_points": [ + -10.833333333333334, + 2.5 + ] + } + }, + "all/sarvam-105b-conversations": { + "n": 120, + "left_correct": 96, + "right_correct": 100, + "gain": 5, + "loss": 1, + "net": 4, + "restored_from_invalid": 0, + "corrected_usable_answer": 5, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.008333333333333333, + 0.08333333333333333 + ], + "percentile_95_percentage_points": [ + -0.8333333333333334, + 8.333333333333332 + ] + } + }, + "treated/sarvam-105b": { + "n": 98, + "left_correct": 82, + "right_correct": 79, + "gain": 7, + "loss": 10, + "net": -3, + "restored_from_invalid": 4, + "corrected_usable_answer": 3, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.1111111111111111, + 0.0449438202247191 + ], + "percentile_95_percentage_points": [ + -11.11111111111111, + 4.49438202247191 + ] + } + }, + "treated/sarvam-105b-conversations": { + "n": 86, + "left_correct": 62, + "right_correct": 66, + "gain": 5, + "loss": 1, + "net": 4, + "restored_from_invalid": 0, + "corrected_usable_answer": 5, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.011494252873563218, + 0.1125 + ], + "percentile_95_percentage_points": [ + -1.1494252873563218, + 11.25 + ] + } + }, + "no_treatment/sarvam-105b": { + "n": 22, + "left_correct": 22, + "right_correct": 20, + "gain": 0, + "loss": 2, + "net": -2, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.2, + 0.0 + ], + "percentile_95_percentage_points": [ + -20.0, + 0.0 + ] + } + }, + "no_treatment/sarvam-105b-conversations": { + "n": 34, + "left_correct": 34, + "right_correct": 34, + "gain": 0, + "loss": 0, + "net": 0, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + 0.0, + 0.0 + ], + "percentile_95_percentage_points": [ + 0.0, + 0.0 + ] + } + } + }, + "B_to_C": { + "all/sarvam-105b": { + "n": 120, + "left_correct": 99, + "right_correct": 105, + "gain": 15, + "loss": 9, + "net": 6, + "restored_from_invalid": 6, + "corrected_usable_answer": 9, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.03333333333333333, + 0.14166666666666666 + ], + "percentile_95_percentage_points": [ + -3.3333333333333335, + 14.166666666666666 + ] + } + }, + "all/sarvam-105b-conversations": { + "n": 120, + "left_correct": 100, + "right_correct": 103, + "gain": 12, + "loss": 9, + "net": 3, + "restored_from_invalid": 0, + "corrected_usable_answer": 12, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.041666666666666664, + 0.1 + ], + "percentile_95_percentage_points": [ + -4.166666666666666, + 10.0 + ] + } + }, + "treated/sarvam-105b": { + "n": 98, + "left_correct": 79, + "right_correct": 87, + "gain": 14, + "loss": 6, + "net": 8, + "restored_from_invalid": 5, + "corrected_usable_answer": 9, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.018867924528301886, + 0.19148936170212766 + ], + "percentile_95_percentage_points": [ + -1.8867924528301887, + 19.148936170212767 + ] + } + }, + "treated/sarvam-105b-conversations": { + "n": 86, + "left_correct": 66, + "right_correct": 69, + "gain": 12, + "loss": 9, + "net": 3, + "restored_from_invalid": 0, + "corrected_usable_answer": 12, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.05952380952380952, + 0.13636363636363635 + ], + "percentile_95_percentage_points": [ + -5.952380952380952, + 13.636363636363635 + ] + } + }, + "no_treatment/sarvam-105b": { + "n": 22, + "left_correct": 20, + "right_correct": 18, + "gain": 1, + "loss": 3, + "net": -2, + "restored_from_invalid": 1, + "corrected_usable_answer": 0, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.2222222222222222, + 0.0 + ], + "percentile_95_percentage_points": [ + -22.22222222222222, + 0.0 + ] + } + }, + "no_treatment/sarvam-105b-conversations": { + "n": 34, + "left_correct": 34, + "right_correct": 34, + "gain": 0, + "loss": 0, + "net": 0, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + 0.0, + 0.0 + ], + "percentile_95_percentage_points": [ + 0.0, + 0.0 + ] + } + } + }, + "A_to_C": { + "all/sarvam-105b": { + "n": 120, + "left_correct": 104, + "right_correct": 105, + "gain": 11, + "loss": 10, + "net": 1, + "restored_from_invalid": 5, + "corrected_usable_answer": 6, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.06666666666666667, + 0.08333333333333333 + ], + "percentile_95_percentage_points": [ + -6.666666666666667, + 8.333333333333332 + ] + } + }, + "all/sarvam-105b-conversations": { + "n": 120, + "left_correct": 96, + "right_correct": 103, + "gain": 14, + "loss": 7, + "net": 7, + "restored_from_invalid": 0, + "corrected_usable_answer": 14, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.016666666666666666, + 0.14166666666666666 + ], + "percentile_95_percentage_points": [ + -1.6666666666666667, + 14.166666666666666 + ] + } + }, + "treated/sarvam-105b": { + "n": 98, + "left_correct": 82, + "right_correct": 87, + "gain": 11, + "loss": 6, + "net": 5, + "restored_from_invalid": 5, + "corrected_usable_answer": 6, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.030303030303030304, + 0.13978494623655913 + ], + "percentile_95_percentage_points": [ + -3.0303030303030303, + 13.978494623655912 + ] + } + }, + "treated/sarvam-105b-conversations": { + "n": 86, + "left_correct": 62, + "right_correct": 69, + "gain": 14, + "loss": 7, + "net": 7, + "restored_from_invalid": 0, + "corrected_usable_answer": 14, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.025, + 0.19117647058823528 + ], + "percentile_95_percentage_points": [ + -2.5, + 19.11764705882353 + ] + } + }, + "no_treatment/sarvam-105b": { + "n": 22, + "left_correct": 22, + "right_correct": 18, + "gain": 0, + "loss": 4, + "net": -4, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + -0.3076923076923077, + -0.05 + ], + "percentile_95_percentage_points": [ + -30.76923076923077, + -5.0 + ] + } + }, + "no_treatment/sarvam-105b-conversations": { + "n": 34, + "left_correct": 34, + "right_correct": 34, + "gain": 0, + "loss": 0, + "net": 0, + "restored_from_invalid": 0, + "corrected_usable_answer": 0, + "scenario_cluster_bootstrap": { + "scenario_clusters": 40, + "draws": 10000, + "estimable_draws": 10000, + "zero_denominator_draws": 0, + "seed": 202610134, + "units": "fraction_of_selected_histories", + "percentile_95": [ + 0.0, + 0.0 + ], + "percentile_95_percentage_points": [ + 0.0, + 0.0 + ] + } + } + } + }, + "language_arm": { + "sarvam-105b/en/chronological": { + "n": 40, + "factual_correct": 36, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 1, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 26, + "projected_status_correct": 40, + "raw_fully_correct_reports": 23, + "projected_fully_correct_reports": 36, + "known_prompt_tokens": 48520, + "known_completion_tokens": 124156, + "known_cost_microinr": 8177160, + "retained_microinr": 10221464, + "elapsed_seconds_sum": 571.59208149802, + "elapsed_seconds_count": 40 + }, + "sarvam-105b/en/grouped_actor_prose_withheld": { + "n": 40, + "factual_correct": 36, + "valid_final": 39, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 29, + "projected_status_correct": 39, + "raw_fully_correct_reports": 26, + "projected_fully_correct_reports": 36, + "known_prompt_tokens": 66351, + "known_completion_tokens": 122348, + "known_cost_microinr": 8336145, + "retained_microinr": 10420195, + "elapsed_seconds_sum": 588.4635169550893, + "elapsed_seconds_count": 40 + }, + "sarvam-105b/en/grouped_lossless": { + "n": 40, + "factual_correct": 32, + "valid_final": 37, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 2, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 27, + "projected_status_correct": 37, + "raw_fully_correct_reports": 22, + "projected_fully_correct_reports": 32, + "known_prompt_tokens": 69880, + "known_completion_tokens": 136121, + "known_cost_microinr": 9215460, + "retained_microinr": 11519340, + "elapsed_seconds_sum": 634.3584105819755, + "elapsed_seconds_count": 40 + }, + "sarvam-105b/hi/chronological": { + "n": 40, + "factual_correct": 33, + "valid_final": 38, + "invalid_or_missing_final": 2, + "wrong_non_null_dates": 1, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 29, + "projected_status_correct": 38, + "raw_fully_correct_reports": 25, + "projected_fully_correct_reports": 33, + "known_prompt_tokens": 50160, + "known_completion_tokens": 140449, + "known_cost_microinr": 9179340, + "retained_microinr": 11474191, + "elapsed_seconds_sum": 651.8509295010299, + "elapsed_seconds_count": 40 + }, + "sarvam-105b/hi/grouped_actor_prose_withheld": { + "n": 40, + "factual_correct": 36, + "valid_final": 38, + "invalid_or_missing_final": 2, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 32, + "projected_status_correct": 38, + "raw_fully_correct_reports": 30, + "projected_fully_correct_reports": 36, + "known_prompt_tokens": 67036, + "known_completion_tokens": 155415, + "known_cost_microinr": 10330440, + "retained_microinr": 12913067, + "elapsed_seconds_sum": 723.8146057900158, + "elapsed_seconds_count": 40 + }, + "sarvam-105b/hi/grouped_lossless": { + "n": 40, + "factual_correct": 33, + "valid_final": 38, + "invalid_or_missing_final": 2, + "wrong_non_null_dates": 1, + "false_abstentions": 0, + "unknown_condition_guesses": 1, + "raw_status_correct": 31, + "projected_status_correct": 38, + "raw_fully_correct_reports": 26, + "projected_fully_correct_reports": 33, + "known_prompt_tokens": 71859, + "known_completion_tokens": 160099, + "known_cost_microinr": 10683825, + "retained_microinr": 13354798, + "elapsed_seconds_sum": 755.5494629169989, + "elapsed_seconds_count": 40 + }, + "sarvam-105b/te/chronological": { + "n": 40, + "factual_correct": 35, + "valid_final": 37, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 1, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 22, + "projected_status_correct": 37, + "raw_fully_correct_reports": 22, + "projected_fully_correct_reports": 35, + "known_prompt_tokens": 49732, + "known_completion_tokens": 128549, + "known_cost_microinr": 8458920, + "retained_microinr": 10573667, + "elapsed_seconds_sum": 601.9044535029534, + "elapsed_seconds_count": 40 + }, + "sarvam-105b/te/grouped_actor_prose_withheld": { + "n": 40, + "factual_correct": 33, + "valid_final": 37, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 1, + "false_abstentions": 2, + "unknown_condition_guesses": 0, + "raw_status_correct": 24, + "projected_status_correct": 37, + "raw_fully_correct_reports": 22, + "projected_fully_correct_reports": 33, + "known_prompt_tokens": 66466, + "known_completion_tokens": 138063, + "known_cost_microinr": 9280770, + "retained_microinr": 11600976, + "elapsed_seconds_sum": 644.4074689580157, + "elapsed_seconds_count": 40 + }, + "sarvam-105b/te/grouped_lossless": { + "n": 40, + "factual_correct": 34, + "valid_final": 37, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 1, + "false_abstentions": 1, + "unknown_condition_guesses": 1, + "raw_status_correct": 25, + "projected_status_correct": 37, + "raw_fully_correct_reports": 23, + "projected_fully_correct_reports": 34, + "known_prompt_tokens": 70953, + "known_completion_tokens": 148800, + "known_cost_microinr": 9992295, + "retained_microinr": 12490384, + "elapsed_seconds_sum": 701.6869304599968, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/en/chronological": { + "n": 40, + "factual_correct": 33, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 7, + "false_abstentions": 0, + "unknown_condition_guesses": 3, + "raw_status_correct": 38, + "projected_status_correct": 40, + "raw_fully_correct_reports": 31, + "projected_fully_correct_reports": 33, + "known_prompt_tokens": 55253, + "known_completion_tokens": 1606, + "known_cost_microinr": 925155, + "retained_microinr": 1156456, + "elapsed_seconds_sum": 40.76753279006516, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/en/grouped_actor_prose_withheld": { + "n": 40, + "factual_correct": 32, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 3, + "false_abstentions": 5, + "unknown_condition_guesses": 0, + "raw_status_correct": 34, + "projected_status_correct": 40, + "raw_fully_correct_reports": 26, + "projected_fully_correct_reports": 32, + "known_prompt_tokens": 76377, + "known_completion_tokens": 1534, + "known_cost_microinr": 1237695, + "retained_microinr": 1547133, + "elapsed_seconds_sum": 41.529686374007724, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/en/grouped_lossless": { + "n": 40, + "factual_correct": 32, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 7, + "false_abstentions": 1, + "unknown_condition_guesses": 3, + "raw_status_correct": 31, + "projected_status_correct": 40, + "raw_fully_correct_reports": 24, + "projected_fully_correct_reports": 32, + "known_prompt_tokens": 81507, + "known_completion_tokens": 1600, + "known_cost_microinr": 1318605, + "retained_microinr": 1648269, + "elapsed_seconds_sum": 40.718356665063766, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/hi/chronological": { + "n": 40, + "factual_correct": 31, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 7, + "false_abstentions": 2, + "unknown_condition_guesses": 1, + "raw_status_correct": 36, + "projected_status_correct": 40, + "raw_fully_correct_reports": 28, + "projected_fully_correct_reports": 31, + "known_prompt_tokens": 53921, + "known_completion_tokens": 1588, + "known_cost_microinr": 904095, + "retained_microinr": 1130134, + "elapsed_seconds_sum": 40.91175604193995, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/hi/grouped_actor_prose_withheld": { + "n": 40, + "factual_correct": 34, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 2, + "false_abstentions": 4, + "unknown_condition_guesses": 2, + "raw_status_correct": 32, + "projected_status_correct": 40, + "raw_fully_correct_reports": 26, + "projected_fully_correct_reports": 34, + "known_prompt_tokens": 73636, + "known_completion_tokens": 1560, + "known_cost_microinr": 1198140, + "retained_microinr": 1497689, + "elapsed_seconds_sum": 42.34330120701634, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/hi/grouped_lossless": { + "n": 40, + "factual_correct": 35, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 4, + "false_abstentions": 1, + "unknown_condition_guesses": 0, + "raw_status_correct": 32, + "projected_status_correct": 40, + "raw_fully_correct_reports": 28, + "projected_fully_correct_reports": 35, + "known_prompt_tokens": 78781, + "known_completion_tokens": 1570, + "known_cost_microinr": 1275915, + "retained_microinr": 1594909, + "elapsed_seconds_sum": 40.04242953909852, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/te/chronological": { + "n": 40, + "factual_correct": 32, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 8, + "false_abstentions": 0, + "unknown_condition_guesses": 4, + "raw_status_correct": 35, + "projected_status_correct": 40, + "raw_fully_correct_reports": 27, + "projected_fully_correct_reports": 32, + "known_prompt_tokens": 53997, + "known_completion_tokens": 1614, + "known_cost_microinr": 906795, + "retained_microinr": 1133509, + "elapsed_seconds_sum": 39.67303916599485, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/te/grouped_actor_prose_withheld": { + "n": 40, + "factual_correct": 37, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 3, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 31, + "projected_status_correct": 40, + "raw_fully_correct_reports": 28, + "projected_fully_correct_reports": 37, + "known_prompt_tokens": 73637, + "known_completion_tokens": 1580, + "known_cost_microinr": 1199355, + "retained_microinr": 1499209, + "elapsed_seconds_sum": 40.52738462702837, + "elapsed_seconds_count": 40 + }, + "sarvam-105b-conversations/te/grouped_lossless": { + "n": 40, + "factual_correct": 33, + "valid_final": 40, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 7, + "false_abstentions": 0, + "unknown_condition_guesses": 3, + "raw_status_correct": 32, + "projected_status_correct": 40, + "raw_fully_correct_reports": 26, + "projected_fully_correct_reports": 33, + "known_prompt_tokens": 78674, + "known_completion_tokens": 1604, + "known_cost_microinr": 1276350, + "retained_microinr": 1595453, + "elapsed_seconds_sum": 41.63225708295067, + "elapsed_seconds_count": 40 + } + }, + "family_arm": { + "sarvam-105b/approval_boundary/chronological": { + "n": 12, + "factual_correct": 11, + "valid_final": 11, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 11, + "projected_status_correct": 11, + "raw_fully_correct_reports": 11, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 16620, + "known_completion_tokens": 23884, + "known_cost_microinr": 1682340, + "retained_microinr": 2102930, + "elapsed_seconds_sum": 112.65253216601559, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/approval_boundary/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 9, + "valid_final": 9, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 7, + "projected_status_correct": 9, + "raw_fully_correct_reports": 7, + "projected_fully_correct_reports": 9, + "known_prompt_tokens": 23426, + "known_completion_tokens": 58987, + "known_cost_microinr": 3890610, + "retained_microinr": 4863267, + "elapsed_seconds_sum": 276.1262179579935, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/approval_boundary/grouped_lossless": { + "n": 12, + "factual_correct": 11, + "valid_final": 11, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 11, + "projected_status_correct": 11, + "raw_fully_correct_reports": 11, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 23621, + "known_completion_tokens": 53626, + "known_cost_microinr": 3571875, + "retained_microinr": 4464848, + "elapsed_seconds_sum": 257.84136253999895, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/approved_change/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 16451, + "known_completion_tokens": 18056, + "known_cost_microinr": 1330125, + "retained_microinr": 1662661, + "elapsed_seconds_sum": 87.49041983297502, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/approved_change/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 11, + "valid_final": 11, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 11, + "projected_status_correct": 11, + "raw_fully_correct_reports": 11, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 23349, + "known_completion_tokens": 44099, + "known_cost_microinr": 2996175, + "retained_microinr": 3745225, + "elapsed_seconds_sum": 203.3663537060056, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/approved_change/grouped_lossless": { + "n": 12, + "factual_correct": 10, + "valid_final": 10, + "invalid_or_missing_final": 2, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 10, + "projected_status_correct": 10, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 10, + "known_prompt_tokens": 23425, + "known_completion_tokens": 55117, + "known_cost_microinr": 3658395, + "retained_microinr": 4573000, + "elapsed_seconds_sum": 253.79363200101943, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/completed_older_request/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 3, + "projected_status_correct": 12, + "raw_fully_correct_reports": 3, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 21436, + "known_completion_tokens": 37494, + "known_cost_microinr": 2571180, + "retained_microinr": 3213980, + "elapsed_seconds_sum": 176.31832258298527, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/completed_older_request/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 3, + "projected_status_correct": 12, + "raw_fully_correct_reports": 3, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 24746, + "known_completion_tokens": 50458, + "known_cost_microinr": 3398670, + "retained_microinr": 4248341, + "elapsed_seconds_sum": 230.21405529203184, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/completed_older_request/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 3, + "projected_status_correct": 12, + "raw_fully_correct_reports": 3, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 27639, + "known_completion_tokens": 30288, + "known_cost_microinr": 2231865, + "retained_microinr": 2789838, + "elapsed_seconds_sum": 139.80573837601696, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/conflicting_request_key/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 1, + "projected_status_correct": 12, + "raw_fully_correct_reports": 1, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 19973, + "known_completion_tokens": 37830, + "known_cost_microinr": 2569395, + "retained_microinr": 3211749, + "elapsed_seconds_sum": 181.99868800201511, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/conflicting_request_key/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 11, + "valid_final": 11, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 2, + "projected_status_correct": 11, + "raw_fully_correct_reports": 2, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 23677, + "known_completion_tokens": 52673, + "known_cost_microinr": 3515535, + "retained_microinr": 4394423, + "elapsed_seconds_sum": 246.72375450000982, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/conflicting_request_key/grouped_lossless": { + "n": 12, + "factual_correct": 11, + "valid_final": 11, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 1, + "projected_status_correct": 11, + "raw_fully_correct_reports": 1, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 26339, + "known_completion_tokens": 51636, + "known_cost_microinr": 3493245, + "retained_microinr": 4366560, + "elapsed_seconds_sum": 239.8147215420031, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/future_approval/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 5, + "projected_status_correct": 12, + "raw_fully_correct_reports": 5, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 16755, + "known_completion_tokens": 41297, + "known_cost_microinr": 2729145, + "retained_microinr": 3411435, + "elapsed_seconds_sum": 195.9946642490395, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/future_approval/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 11, + "valid_final": 11, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 8, + "projected_status_correct": 11, + "raw_fully_correct_reports": 8, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 21149, + "known_completion_tokens": 38266, + "known_cost_microinr": 2613195, + "retained_microinr": 3266498, + "elapsed_seconds_sum": 177.69715537501907, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/future_approval/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 7, + "projected_status_correct": 12, + "raw_fully_correct_reports": 7, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 23251, + "known_completion_tokens": 40323, + "known_cost_microinr": 2768145, + "retained_microinr": 3460185, + "elapsed_seconds_sum": 196.18101524897793, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/missing_replacement/chronological": { + "n": 12, + "factual_correct": 4, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 8, + "projected_status_correct": 12, + "raw_fully_correct_reports": 2, + "projected_fully_correct_reports": 4, + "known_prompt_tokens": 11294, + "known_completion_tokens": 43160, + "known_cost_microinr": 2759010, + "retained_microinr": 3448767, + "elapsed_seconds_sum": 203.2952735429717, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/missing_replacement/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 6, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 6, + "projected_fully_correct_reports": 6, + "known_prompt_tokens": 16420, + "known_completion_tokens": 35871, + "known_cost_microinr": 2398560, + "retained_microinr": 2998204, + "elapsed_seconds_sum": 175.4489582910319, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/missing_replacement/grouped_lossless": { + "n": 12, + "factual_correct": 3, + "valid_final": 11, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 10, + "projected_status_correct": 11, + "raw_fully_correct_reports": 2, + "projected_fully_correct_reports": 3, + "known_prompt_tokens": 17483, + "known_completion_tokens": 45277, + "known_cost_microinr": 2978865, + "retained_microinr": 3723585, + "elapsed_seconds_sum": 214.70913649894646, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/request_correction/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 10, + "projected_status_correct": 12, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 10848, + "known_completion_tokens": 35690, + "known_cost_microinr": 2304120, + "retained_microinr": 2880155, + "elapsed_seconds_sum": 165.80868412599375, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/request_correction/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 11, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 1, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 11, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 16565, + "known_completion_tokens": 24461, + "known_cost_microinr": 1716135, + "retained_microinr": 2145175, + "elapsed_seconds_sum": 120.96716408400971, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/request_correction/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 17086, + "known_completion_tokens": 25483, + "known_cost_microinr": 1785270, + "retained_microinr": 2231592, + "elapsed_seconds_sum": 124.70271487400169, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/resolved_condition/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 10, + "projected_status_correct": 12, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 11317, + "known_completion_tokens": 39762, + "known_cost_microinr": 2555475, + "retained_microinr": 3194348, + "elapsed_seconds_sum": 180.207794499991, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/resolved_condition/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 11, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 1, + "unknown_condition_guesses": 0, + "raw_status_correct": 9, + "projected_status_correct": 12, + "raw_fully_correct_reports": 9, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 16297, + "known_completion_tokens": 33765, + "known_cost_microinr": 2270355, + "retained_microinr": 2837947, + "elapsed_seconds_sum": 167.5436829990358, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/resolved_condition/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 10, + "projected_status_correct": 12, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 17374, + "known_completion_tokens": 33831, + "known_cost_microinr": 2290470, + "retained_microinr": 2863093, + "elapsed_seconds_sum": 168.20070904299791, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/separate_booking/chronological": { + "n": 12, + "factual_correct": 8, + "valid_final": 11, + "invalid_or_missing_final": 1, + "wrong_non_null_dates": 3, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 9, + "projected_status_correct": 11, + "raw_fully_correct_reports": 8, + "projected_fully_correct_reports": 8, + "known_prompt_tokens": 11742, + "known_completion_tokens": 59672, + "known_cost_microinr": 3756450, + "retained_microinr": 4695568, + "elapsed_seconds_sum": 276.12464366802305, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/separate_booking/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 11, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 1, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 9, + "projected_status_correct": 12, + "raw_fully_correct_reports": 9, + "projected_fully_correct_reports": 11, + "known_prompt_tokens": 17981, + "known_completion_tokens": 38667, + "known_cost_microinr": 2589735, + "retained_microinr": 3237172, + "elapsed_seconds_sum": 180.1802984579699, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/separate_booking/grouped_lossless": { + "n": 12, + "factual_correct": 6, + "valid_final": 9, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 2, + "false_abstentions": 1, + "unknown_condition_guesses": 0, + "raw_status_correct": 8, + "projected_status_correct": 9, + "raw_fully_correct_reports": 6, + "projected_fully_correct_reports": 6, + "known_prompt_tokens": 18512, + "known_completion_tokens": 65391, + "known_cost_microinr": 4201140, + "retained_microinr": 5251430, + "elapsed_seconds_sum": 299.79583091697714, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/unresolved_condition/chronological": { + "n": 12, + "factual_correct": 9, + "valid_final": 9, + "invalid_or_missing_final": 3, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 8, + "projected_status_correct": 9, + "raw_fully_correct_reports": 8, + "projected_fully_correct_reports": 9, + "known_prompt_tokens": 11976, + "known_completion_tokens": 56309, + "known_cost_microinr": 3558180, + "retained_microinr": 4447729, + "elapsed_seconds_sum": 245.4564418319933, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/unresolved_condition/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 16243, + "known_completion_tokens": 38579, + "known_cost_microinr": 2558385, + "retained_microinr": 3197986, + "elapsed_seconds_sum": 178.41795104001358, + "elapsed_seconds_count": 12 + }, + "sarvam-105b/unresolved_condition/grouped_lossless": { + "n": 12, + "factual_correct": 10, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 2, + "false_abstentions": 0, + "unknown_condition_guesses": 2, + "raw_status_correct": 11, + "projected_status_correct": 12, + "raw_fully_correct_reports": 9, + "projected_fully_correct_reports": 10, + "known_prompt_tokens": 17962, + "known_completion_tokens": 44048, + "known_cost_microinr": 2912310, + "retained_microinr": 3640391, + "elapsed_seconds_sum": 196.7499429180316, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/approval_boundary/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 17948, + "known_completion_tokens": 482, + "known_cost_microinr": 298140, + "retained_microinr": 372678, + "elapsed_seconds_sum": 12.279806457983796, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/approval_boundary/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 11, + "projected_status_correct": 12, + "raw_fully_correct_reports": 11, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 26061, + "known_completion_tokens": 482, + "known_cost_microinr": 419835, + "retained_microinr": 524798, + "elapsed_seconds_sum": 12.268528750006226, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/approval_boundary/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 26329, + "known_completion_tokens": 482, + "known_cost_microinr": 423855, + "retained_microinr": 529823, + "elapsed_seconds_sum": 12.243598708009813, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/approved_change/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 17831, + "known_completion_tokens": 480, + "known_cost_microinr": 296265, + "retained_microinr": 370336, + "elapsed_seconds_sum": 11.836355124018155, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/approved_change/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 26302, + "known_completion_tokens": 480, + "known_cost_microinr": 423330, + "retained_microinr": 529168, + "elapsed_seconds_sum": 12.448434040998109, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/approved_change/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 26302, + "known_completion_tokens": 480, + "known_cost_microinr": 423330, + "retained_microinr": 529168, + "elapsed_seconds_sum": 11.99296229200263, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/completed_older_request/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 9, + "projected_status_correct": 12, + "raw_fully_correct_reports": 9, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 23591, + "known_completion_tokens": 504, + "known_cost_microinr": 384105, + "retained_microinr": 480136, + "elapsed_seconds_sum": 12.898333750010352, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/completed_older_request/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 7, + "projected_status_correct": 12, + "raw_fully_correct_reports": 7, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 27331, + "known_completion_tokens": 504, + "known_cost_microinr": 440205, + "retained_microinr": 550262, + "elapsed_seconds_sum": 13.1350625820196, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/completed_older_request/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 6, + "projected_status_correct": 12, + "raw_fully_correct_reports": 6, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 31117, + "known_completion_tokens": 504, + "known_cost_microinr": 496995, + "retained_microinr": 621248, + "elapsed_seconds_sum": 11.959902622038499, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/conflicting_request_key/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 6, + "projected_status_correct": 12, + "raw_fully_correct_reports": 6, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 22314, + "known_completion_tokens": 500, + "known_cost_microinr": 364710, + "retained_microinr": 455892, + "elapsed_seconds_sum": 12.270407874995726, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/conflicting_request_key/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 2, + "projected_status_correct": 12, + "raw_fully_correct_reports": 2, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 26017, + "known_completion_tokens": 504, + "known_cost_microinr": 420495, + "retained_microinr": 525623, + "elapsed_seconds_sum": 12.079574581992347, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/conflicting_request_key/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 2, + "projected_status_correct": 12, + "raw_fully_correct_reports": 2, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 29868, + "known_completion_tokens": 500, + "known_cost_microinr": 478020, + "retained_microinr": 597528, + "elapsed_seconds_sum": 12.204410330974497, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/future_approval/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 18203, + "known_completion_tokens": 482, + "known_cost_microinr": 301965, + "retained_microinr": 377461, + "elapsed_seconds_sum": 11.893214874973637, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/future_approval/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 5, + "projected_status_correct": 12, + "raw_fully_correct_reports": 5, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 26189, + "known_completion_tokens": 494, + "known_cost_microinr": 422475, + "retained_microinr": 528097, + "elapsed_seconds_sum": 12.045347375023994, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/future_approval/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 7, + "projected_status_correct": 12, + "raw_fully_correct_reports": 7, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 26632, + "known_completion_tokens": 490, + "known_cost_microinr": 428880, + "retained_microinr": 536103, + "elapsed_seconds_sum": 12.964495208027074, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/missing_replacement/chronological": { + "n": 12, + "factual_correct": 10, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 2, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 10, + "known_prompt_tokens": 12115, + "known_completion_tokens": 404, + "known_cost_microinr": 205965, + "retained_microinr": 257460, + "elapsed_seconds_sum": 12.17967891698936, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/missing_replacement/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 18168, + "known_completion_tokens": 384, + "known_cost_microinr": 295560, + "retained_microinr": 369453, + "elapsed_seconds_sum": 11.797906836029142, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/missing_replacement/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 19182, + "known_completion_tokens": 384, + "known_cost_microinr": 310770, + "retained_microinr": 388467, + "elapsed_seconds_sum": 11.954999916008092, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/request_correction/chronological": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 11, + "projected_status_correct": 12, + "raw_fully_correct_reports": 11, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 11335, + "known_completion_tokens": 504, + "known_cost_microinr": 200265, + "retained_microinr": 250337, + "elapsed_seconds_sum": 11.854606876018806, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/request_correction/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 8, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 4, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 8, + "projected_fully_correct_reports": 8, + "known_prompt_tokens": 17168, + "known_completion_tokens": 464, + "known_cost_microinr": 285360, + "retained_microinr": 356704, + "elapsed_seconds_sum": 13.21544787401217, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/request_correction/grouped_lossless": { + "n": 12, + "factual_correct": 12, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 12, + "projected_fully_correct_reports": 12, + "known_prompt_tokens": 17858, + "known_completion_tokens": 504, + "known_cost_microinr": 298110, + "retained_microinr": 372643, + "elapsed_seconds_sum": 12.309500044008018, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/resolved_condition/chronological": { + "n": 12, + "factual_correct": 10, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 2, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 10, + "known_prompt_tokens": 13860, + "known_completion_tokens": 484, + "known_cost_microinr": 236940, + "retained_microinr": 296178, + "elapsed_seconds_sum": 12.080073124001501, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/resolved_condition/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 8, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 4, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 8, + "projected_fully_correct_reports": 8, + "known_prompt_tokens": 19144, + "known_completion_tokens": 464, + "known_cost_microinr": 315000, + "retained_microinr": 393753, + "elapsed_seconds_sum": 11.855004083001404, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/resolved_condition/grouped_lossless": { + "n": 12, + "factual_correct": 10, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 0, + "false_abstentions": 2, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 10, + "known_prompt_tokens": 21354, + "known_completion_tokens": 484, + "known_cost_microinr": 349350, + "retained_microinr": 436692, + "elapsed_seconds_sum": 12.498722624019138, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/separate_booking/chronological": { + "n": 12, + "factual_correct": 0, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 12, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 11, + "projected_status_correct": 12, + "raw_fully_correct_reports": 0, + "projected_fully_correct_reports": 0, + "known_prompt_tokens": 12448, + "known_completion_tokens": 504, + "known_cost_microinr": 216960, + "retained_microinr": 271203, + "elapsed_seconds_sum": 11.991847333003534, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/separate_booking/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 5, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 6, + "false_abstentions": 1, + "unknown_condition_guesses": 0, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 5, + "projected_fully_correct_reports": 5, + "known_prompt_tokens": 18898, + "known_completion_tokens": 494, + "known_cost_microinr": 313110, + "retained_microinr": 391393, + "elapsed_seconds_sum": 11.78994375099137, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/separate_booking/grouped_lossless": { + "n": 12, + "factual_correct": 0, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 12, + "false_abstentions": 0, + "unknown_condition_guesses": 0, + "raw_status_correct": 9, + "projected_status_correct": 12, + "raw_fully_correct_reports": 0, + "projected_fully_correct_reports": 0, + "known_prompt_tokens": 19707, + "known_completion_tokens": 502, + "known_cost_microinr": 325725, + "retained_microinr": 407161, + "elapsed_seconds_sum": 12.05981566599803, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/unresolved_condition/chronological": { + "n": 12, + "factual_correct": 4, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 8, + "false_abstentions": 0, + "unknown_condition_guesses": 8, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 4, + "projected_fully_correct_reports": 4, + "known_prompt_tokens": 13526, + "known_completion_tokens": 464, + "known_cost_microinr": 230730, + "retained_microinr": 288418, + "elapsed_seconds_sum": 12.068003666005097, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/unresolved_condition/grouped_actor_prose_withheld": { + "n": 12, + "factual_correct": 10, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 2, + "false_abstentions": 0, + "unknown_condition_guesses": 2, + "raw_status_correct": 12, + "projected_status_correct": 12, + "raw_fully_correct_reports": 10, + "projected_fully_correct_reports": 10, + "known_prompt_tokens": 18372, + "known_completion_tokens": 404, + "known_cost_microinr": 299820, + "retained_microinr": 374780, + "elapsed_seconds_sum": 13.76512233397807, + "elapsed_seconds_count": 12 + }, + "sarvam-105b-conversations/unresolved_condition/grouped_lossless": { + "n": 12, + "factual_correct": 6, + "valid_final": 12, + "invalid_or_missing_final": 0, + "wrong_non_null_dates": 6, + "false_abstentions": 0, + "unknown_condition_guesses": 6, + "raw_status_correct": 11, + "projected_status_correct": 12, + "raw_fully_correct_reports": 5, + "projected_fully_correct_reports": 6, + "known_prompt_tokens": 20613, + "known_completion_tokens": 444, + "known_cost_microinr": 335835, + "retained_microinr": 419798, + "elapsed_seconds_sum": 12.204635876027169, + "elapsed_seconds_count": 12 + } + }, + "mechanical_gate": { + "checks": { + "all_720_clean": true, + "production_default_transport": true, + "treated_B_to_C_positive/sarvam-105b": true, + "two_positive_workflows/sarvam-105b": true, + "all_C_to_A_positive/sarvam-105b": true, + "C_no_higher_wrong_non_null_dates/sarvam-105b": true, + "C_no_higher_false_abstentions/sarvam-105b": false, + "C_no_lower_raw_status_correct/sarvam-105b": true, + "C_no_lower_projected_status_correct/sarvam-105b": false, + "treated_B_to_C_positive/sarvam-105b-conversations": true, + "two_positive_workflows/sarvam-105b-conversations": true, + "all_C_to_A_positive/sarvam-105b-conversations": true, + "C_no_higher_wrong_non_null_dates/sarvam-105b-conversations": true, + "C_no_higher_false_abstentions/sarvam-105b-conversations": false, + "C_no_lower_raw_status_correct/sarvam-105b-conversations": false, + "C_no_lower_projected_status_correct/sarvam-105b-conversations": true + }, + "mechanical_pass_pending_semantic_review": false, + "semantic_review_complete": false, + "forward_gate_pass": null + }, + "latency_scope": "Observed reporter request duration, not end-to-end native workflow or voice latency.", + "ci_scope": "Descriptive 40-scenario cluster bootstrap on a synthetic development-exposed bank.", + "known_microinr": 93896460, + "retained_microinr": 117370843 +} diff --git a/benchmarks/sarvam_source_interpretation.py b/benchmarks/sarvam_source_interpretation.py new file mode 100644 index 0000000..c6fd955 --- /dev/null +++ b/benchmarks/sarvam_source_interpretation.py @@ -0,0 +1,1242 @@ +"""Prospective all-history source interpretation screen; no actor or tool calls. + +Preparation and audit are offline. Paid execution is opt-in, one fixed reporter +request per job, with no continuation after any attempt. The saved histories are +development exposed; this module does not claim a memory or model improvement. +""" + +import argparse +import json +import random +import sqlite3 +from collections import Counter, defaultdict +from concurrent.futures import ThreadPoolExecutor +from itertools import permutations +from pathlib import Path + +from benchmarks import sarvam_evidence_ablation as ablation +from benchmarks.sarvam_fresh_confirmation.engine import report_view +from benchmarks.sarvam_handoff_diagnostic import ( + _census, + _copy_once, + _parent_files, + _sha, + _usage, +) +from benchmarks.sarvam_native_controls.engine import PROFILES, ControlMeter +from benchmarks.sarvam_native_controls.transcript_finalizer import classify +from benchmarks.sarvam_native_study.paced import Pacer +from benchmarks.sarvam_native_study.transport import ( + ENDPOINT, + call, + digest, + read_key, + save, +) +from benchmarks.sarvam_receipt_reporting import project +from benchmarks.sarvam_reporter_wording import reservation +from benchmarks.sarvam_reporting_handoff import prepare_plan +from benchmarks.sarvam_source_attribution import evaluate + +ARMS = ablation.ARMS +STAGE = "source_interpretation" +PARENT_EPISODES = 240 +PLANNED_CALLS = 720 +FULL_RESERVATION = 1_548_288_000 +PHASE_CAP = 1_550_000_000 +PRIOR_CALLS = 7037 +PRIOR_KNOWN = 271_224_630 +PRIOR_RETAINED = 371_289_339 +CAMPAIGN_CAP = 6_000_000_000 +ORDER_SEED = 202610134 +CODE = ( + "benchmarks/sarvam_source_interpretation.py", + "benchmarks/sarvam_evidence_ablation.py", + "benchmarks/sarvam_evidence_views.py", + "benchmarks/sarvam_handoff_diagnostic.py", + "benchmarks/sarvam_reporting_handoff.py", + "benchmarks/sarvam_reporter_wording.py", + "benchmarks/sarvam_ordinary_v2.py", + "benchmarks/sarvam_ordinary_study.py", + "benchmarks/sarvam_receipt_reporting.py", + "benchmarks/sarvam_source_attribution.py", + "benchmarks/sarvam_fresh_confirmation/engine.py", + "benchmarks/sarvam_native_controls/engine.py", + "benchmarks/sarvam_native_controls/integrated.py", + "benchmarks/sarvam_native_controls/transcript_finalizer.py", + "benchmarks/sarvam_native_controls/finalization_transport.py", + "benchmarks/sarvam_native_controls/separated.py", + "benchmarks/sarvam_native_controls/approval_backend.py", + "benchmarks/sarvam_native_study/backend.py", + "benchmarks/sarvam_native_study/runner.py", + "benchmarks/sarvam_native_study/diagnostic.py", + "benchmarks/sarvam_native_study/transport.py", + "benchmarks/sarvam_native_study/paced.py", + "benchmarks/sarvam_native_study/score.py", +) +PARENT_FILES = ( + "manifest.json", + "started.json", + "execution.json", + "verification.json", + "runtime.json", + "budget.json", + "review.json", + "PROTOCOL.md", + "pacing-control.json", + "semantic-review.json", +) + + +def _require(condition, reason): + if not condition: + raise ValueError(reason) + + +def _read(path): + return json.loads(Path(path).read_text()) + + +def _code_hashes(): + repo = Path(__file__).resolve().parents[1] + return {name: _sha(repo / name) for name in CODE} + + +def _source_inventory(parent, manifest, execution, verification, runtime): + """Build all histories from the original v3 artifacts, never a diagnostic answer.""" + parent = Path(parent) + ids = [job["id"] for job in manifest["jobs"]] + _require( + len(ids) == PARENT_EPISODES and len(set(ids)) == PARENT_EPISODES, + "Parent manifest incomplete", + ) + _require( + _read(parent / "started.json")["manifest_sha256"] == digest(manifest), + "Parent start/manifest mismatch", + ) + _require([o["id"] for o in execution["outcomes"]] == ids, "Parent execution order mismatch") + score = {(r["id"], r["variant"]): r for r in verification["rows"]} + _require(len(score) == 480, "Parent reporter verification incomplete") + inputs = [] + for job in manifest["jobs"]: + episode_id = job["id"] + pair_path = parent / "paired" / f"{episode_id}.json" + pair = _read(pair_path) + original = pair["original"] + _require( + original["id"] == episode_id + and original["case_id"] == job["case_id"] + and original["model"] == job["model"] + and original["initial_sha256"] == job["initial_sha256"], + f"Parent episode changed: {episode_id}", + ) + _require( + pair["actual_calls"] + == next(x for x in execution["outcomes"] if x["id"] == episode_id)["actual_calls"], + f"Parent actual calls changed: {episode_id}", + ) + for arm in ("control", "candidate"): + _require( + pair["reports"][arm]["terminal"] == score[(episode_id, arm)]["terminal"], + f"Parent report status changed: {episode_id}/{arm}", + ) + case = runtime[job["case_id"]] + _require( + case["language"] == job["case_id"].rsplit("-", 1)[-1] + and case["workflow"] == job["case_id"].rsplit("-", 1)[0] + and case["caller"] == _read_actor_caller(original), + f"Parent source/scope changed: {episode_id}", + ) + candidate_path = parent / "calls" / f"{episode_id}-final-candidate.request.json" + reconstructed = False + actor_plan_hash = None + if candidate_path.exists(): + record = _read(candidate_path) + _require( + record["endpoint"] == ENDPOINT and record["sha256"] == digest(record["request"]), + f"Saved reporter request changed: {episode_id}", + ) + request = record["request"] + request_file_sha = _sha(candidate_path) + _require( + pair["reports"]["candidate"]["terminal"] != "not_reached", + f"Saved candidate request but no parent report: {episode_id}", + ) + else: + _require( + episode_id == "confirm-fresh-13-en-m0" + and all( + pair["reports"][a]["terminal"] == "not_reached" + for a in ("control", "candidate") + ), + "Unexpected absent parent candidate request", + ) + actor_calls = pair["actual_calls"] + _require( + len(actor_calls) == 1 and actor_calls[0] == episode_id + "-a0", + "Capped actor history has other calls", + ) + actor_request_path = parent / "calls" / f"{actor_calls[0]}.request.json" + actor_receipt_path = parent / "calls" / f"{actor_calls[0]}.receipt.json" + recorded = _read(actor_request_path) + _require( + recorded["endpoint"] == ENDPOINT + and recorded["sha256"] == digest(recorded["request"]), + "Capped actor request changed", + ) + actor_plan = prepare_plan( + actor_job=actor_calls[0], + actor_request=recorded["request"], + native_receipt=_read(actor_receipt_path), + tool_trace=original["tool_trace"], + initial_backend=original["initial_backend"], + final_backend=original["final_backend"], + actor_calls_used=1, + caller_scope=case["caller"], + ) + request = actor_plan["reports"]["candidate"]["request"] + request_file_sha = None + actor_plan_hash = actor_plan["plan_sha256"] + reconstructed = True + _require(request["model"] == job["model"], "Reporter model mismatch") + _, sidecar, _ = ablation.render(request, original["tool_trace"], ARMS[0]) + _require( + sidecar["original"]["original_messages"] == original["history"], + f"Source history mismatch: {episode_id}", + ) + _require( + json.loads(original["history"][1]["content"])["source_messages"] + == case["source_messages"], + f"Original source messages changed: {episode_id}", + ) + inputs.append( + { + "id": episode_id, + "case_id": job["case_id"], + "scenario": case["workflow"], + "language": case["language"], + "family": case["family"], + "model": job["model"], + "parent_paired_sha256": _sha(pair_path), + "parent_request_file_sha256": request_file_sha, + "parent_request_payload_sha256": digest(request), + "capped_reconstruction": reconstructed, + "actor_plan_sha256": actor_plan_hash, + "request": request, + "tool_trace": original["tool_trace"], + } + ) + _require( + len(inputs) == PARENT_EPISODES + and Counter(i["model"] for i in inputs) == {m: 120 for m in PROFILES} + and Counter(i["language"] for i in inputs) + == {language: 80 for language in ("en", "hi", "te")}, + "Model/language coverage changed", + ) + scenarios = Counter(i["scenario"] for i in inputs) + _require( + len(scenarios) == 40 and set(scenarios.values()) == {6}, "40-by-6 scenario coverage changed" + ) + _require( + sum(i["capped_reconstruction"] for i in inputs) == 1, + "Capped source reconstruction count changed", + ) + return inputs + + +def _read_actor_caller(original): + supplied = json.loads(original["history"][1]["content"]) + current = original["initial_backend"]["appointments"] + rows = [row for row in current if row["id"] == supplied["appointment_id"]] + _require(len(rows) == 1, "Parent caller scope ambiguous") + return {key: rows[0][key] for key in ("tenant", "customer")} + + +def _ordered_jobs(inputs): + by_scenario = defaultdict(list) + for item in inputs: + by_scenario[item["scenario"]].append(item) + scenario_names = sorted(by_scenario) + six = tuple(permutations(ARMS)) + assignment = {} + for scenario_index, name in enumerate(scenario_names): + members = sorted(by_scenario[name], key=lambda i: (i["language"], i["model"])) + _require(len(members) == 6, "Scenario lacks six renderings") + for slot, item in enumerate(members): + assignment[item["id"]] = six[(scenario_index + slot) % 6] + shuffled = list(inputs) + random.Random(ORDER_SEED).shuffle(shuffled) + jobs = [] + for item in shuffled: + for position, arm in enumerate(assignment[item["id"]]): + request, sidecar, omissions = ablation.render(item["request"], item["tool_trace"], arm) + ablation.verify_omissions(request, sidecar, omissions, arm) + jobs.append( + { + "id": item["id"] + "-source-" + arm, + "history_id": item["id"], + "arm": arm, + "position": position, + "scenario": item["scenario"], + "request_sha256": digest(request), + "reservation_microinr": reservation(request), + "omission_count": len(omissions), + } + ) + _require( + len(jobs) == PLANNED_CALLS + and Counter(j["arm"] for j in jobs) == {arm: 240 for arm in ARMS} + and Counter((j["arm"], j["position"]) for j in jobs) + == {(arm, pos): 80 for arm in ARMS for pos in range(3)}, + "Three-arm order is unbalanced", + ) + # When C omits nothing, B and C have identical bytes and form a no-treatment control. + grouped = {(j["history_id"], j["arm"]): j for j in jobs} + for item in inputs: + b, c = (grouped[item["id"], arm] for arm in ARMS[1:]) + for arm in ARMS: + grouped[item["id"], arm]["treated_actor_prose"] = bool(c["omission_count"]) + _require( + bool(c["omission_count"]) or b["request_sha256"] == c["request_sha256"], + "No-op actor-prose arm differs from grouped control", + ) + return jobs + + +def _budget(jobs): + worst = sum(job["reservation_microinr"] for job in jobs) + _require( + worst == FULL_RESERVATION + and worst <= PHASE_CAP + and PRIOR_RETAINED + PHASE_CAP <= CAMPAIGN_CAP, + "Full-context budget changed", + ) + return { + "phase_cap_microinr": PHASE_CAP, + "stages": {STAGE: {"calls": PLANNED_CALLS, "cap_microinr": PHASE_CAP}}, + "prior_calls": PRIOR_CALLS, + "prior_known_microinr": PRIOR_KNOWN, + "prior_retained_microinr": PRIOR_RETAINED, + "campaign_cap_microinr": CAMPAIGN_CAP, + "authority": ( + "Founder original total INR6000 research approval plus current instruction " + "to proceed with proposed phase; not a renewed allowance" + ), + "maximum_full_context_reservation_microinr": worst, + } + + +def _completion(path): + result = _read(path) + _require( + result.get("all_required_publication_checks_complete") is True + and result.get("new_calls") == 2 + and result.get("new_native_or_tool_calls") == 0 + and result.get("research_calls") == PRIOR_CALLS + and round(result.get("research_known_inr", -1) * 1_000_000) == PRIOR_KNOWN + and round(result.get("research_retained_inr", -1) * 1_000_000) == PRIOR_RETAINED, + "Prior diagnostic completion or cumulative carry changed", + ) + return result + + +def _preflight(path, inputs, jobs): + reviewed = _read(path) + _require( + reviewed.get("status") == "INDEPENDENT_OFFLINE_SOURCE_PREFLIGHT_PASS" + and reviewed.get("scope", {}).get("planned_reporter_requests") == PLANNED_CALLS + and reviewed["scope"]["actor_prose_treated_histories"] == 184 + and reviewed["scope"]["actor_prose_no_treatment_histories"] == 56, + "Independent source preflight changed", + ) + cases = {row["id"]: row for row in reviewed["cases"]} + _require( + len(cases) == PARENT_EPISODES and set(cases) == {item["id"] for item in inputs}, + "Independent source preflight coverage changed", + ) + by_job = {(j["history_id"], j["arm"]): j for j in jobs} + treatment = Counter() + for item in inputs: + row = cases[item["id"]] + treated = by_job[item["id"], ARMS[2]]["treated_actor_prose"] + treatment[(item["model"], treated)] += 1 + _require( + row["paired_file_sha256"] == item["parent_paired_sha256"] + and row["saved_candidate_request_file_sha256"] == item["parent_request_file_sha256"] + and row["actor_prose_treatment_present"] is treated + and row["actor_prose_omission_ledger"] + == ablation.render(item["request"], item["tool_trace"], ARMS[2])[2] + and row["arm_request_sha256"] + == {arm: by_job[item["id"], arm]["request_sha256"] for arm in ARMS}, + f"Independent source preflight disagrees: {item['id']}", + ) + _require( + treatment + == { + ("sarvam-105b", True): 98, + ("sarvam-105b", False): 22, + ("sarvam-105b-conversations", True): 86, + ("sarvam-105b-conversations", False): 34, + }, + "Treatment split changed", + ) + return reviewed + + +def _flat_census(parent, manifest, execution, verification, runtime): + rows, selected, refusals = _census( + manifest, execution, verification, runtime, parent / "paired", parent / "calls" + ) + _require( + len(rows) == PARENT_EPISODES + and not refusals + and selected["id"] == "confirm-fresh-13-en-m0", + "Parent routing census changed", + ) + return rows + + +def prepare(root, parent, prior_completion): + """Freeze 240 original histories and 720 reporter requests without reading gold.""" + root, parent, prior_completion = map(Path, (root, parent, prior_completion)) + _require( + root.is_dir() + and all( + (root / name).is_file() + for name in ( + "PROTOCOL.md", + "budget.json", + "pacing-control.json", + "source-preflight.json", + ) + ), + "Reviewed protocol, budget and pacing files required", + ) + _require( + not any( + (root / name).exists() + for name in ("manifest.json", "inputs.json", "started.json", "calls", "provenance") + ), + "Study already prepared or attempted", + ) + _completion(prior_completion) + manifest, execution, verification, runtime = _parent_files(parent, full_call_inventory=True) + _require(verification["calls"] == 844, "Original v3 call inventory changed") + _require( + _read(parent / "started.json")["manifest_sha256"] == digest(manifest), + "Original v3 start binding changed", + ) + for job in manifest["jobs"]: + episode_id = job["id"] + _require( + (parent / "episodes" / episode_id / "paired.json").is_file(), + f"Parent episode missing: {episode_id}", + ) + candidate = parent / "calls" / f"{episode_id}-final-candidate.request.json" + if episode_id != "confirm-fresh-13-en-m0": + _require(candidate.is_file(), f"Saved candidate request missing: {episode_id}") + # From here onward every write is create-once; a partial freeze cannot resume. + archived = root / "provenance" / "parent" + for name in PARENT_FILES: + _copy_once(parent / name, archived / name) + _copy_once(parent / "gold.json", root / "sealed" / "gold.json") + _copy_once(prior_completion, root / "provenance" / "prior-completion.json") + semantic = _read(parent / "semantic-review.json") + _require( + semantic.get("status") == "ASSISTANT_LANGUAGE_REVIEW_COMPLETE", + "Parent semantic review incomplete", + ) + for entry in semantic["review_files"].values(): + relative = Path(entry["file"]) + _require( + not relative.is_absolute() and ".." not in relative.parts, + "Parent review path escapes archive", + ) + _copy_once(parent / relative, archived / relative) + for name in manifest["source_sha256"]: + _copy_once(parent / "source" / name, archived / "source" / name) + for job in manifest["jobs"]: + episode_id = job["id"] + _copy_once( + parent / "episodes" / episode_id / "paired.json", + archived / "paired" / f"{episode_id}.json", + ) + candidate = parent / "calls" / f"{episode_id}-final-candidate.request.json" + if candidate.exists(): + _copy_once(candidate, archived / "calls" / candidate.name) + selected_actor = "confirm-fresh-13-en-m0-a0" + for suffix in ("request", "receipt"): + name = f"{selected_actor}.{suffix}.json" + _copy_once(parent / "calls" / name, archived / "calls" / name) + copied_manifest, copied_execution, copied_verification, copied_runtime = _parent_files( + archived, gold_path=root / "sealed" / "gold.json" + ) + _flat_census(archived, copied_manifest, copied_execution, copied_verification, copied_runtime) + inputs = _source_inventory( + archived, copied_manifest, copied_execution, copied_verification, copied_runtime + ) + jobs = _ordered_jobs(inputs) + _preflight(root / "source-preflight.json", inputs, jobs) + budget = _budget(jobs) + _require(_read(root / "budget.json") == budget, "Reviewed budget differs from exact jobs") + _require( + _read(root / "pacing-control.json") + == { + "rpm": 90, + "user_reported_account_rpm": 120, + "workers": 4, + "batch_size": 4, + "stop_after_batch_errors": 2, + "no_retries": True, + }, + "Pacing must be 90 RPM", + ) + sidecars = {} + omissions = {} + for item in inputs: + _, sidecar, _ = ablation.render(item["request"], item["tool_trace"], ARMS[0]) + sidecars[item["id"]] = sidecar + by_id = {item["id"]: item for item in inputs} + for job in jobs: + item = by_id[job["history_id"]] + _, _, ledger = ablation.render(item["request"], item["tool_trace"], job["arm"]) + omissions[job["id"]] = ledger + save(root / "inputs.json", inputs) + save(root / "sidecars.json", sidecars) + save(root / "omissions.json", omissions) + frozen = [ + root / n + for n in ( + "PROTOCOL.md", + "budget.json", + "pacing-control.json", + "source-preflight.json", + "inputs.json", + "sidecars.json", + "omissions.json", + ) + ] + frozen += [p for p in (root / "provenance").rglob("*") if p.is_file()] + frozen += [root / "sealed" / "gold.json"] + package = { + "kind": "ALL_240_SAVED_HISTORY_SOURCE_INTERPRETATION_SCREEN", + "jobs": jobs, + "source_sha256": _code_hashes(), + "files_sha256": {str(p.relative_to(root)): _sha(p) for p in sorted(frozen)}, + "parent_census_sha256": digest( + _flat_census( + archived, copied_manifest, copied_execution, copied_verification, copied_runtime + ) + ), + "maximum_calls": PLANNED_CALLS, + "worst_case_retained_microinr": FULL_RESERVATION, + "workers": 4, + "batch_size": 4, + "rpm": 90, + "stop_after_batch_errors": 2, + "no_retries": True, + "new_native_action_trials": 0, + "capped_reconstruction_count": 1, + "treated_actor_prose_histories": sum( + j["treated_actor_prose"] for j in jobs if j["arm"] == ARMS[2] + ), + "development_exposed": True, + } + save(root / "manifest.json", package) + return package + + +def check(root): + """Recreate every paid request from the portable source records, without gold.""" + root = Path(root) + package = _read(root / "manifest.json") + _require( + package["kind"] == "ALL_240_SAVED_HISTORY_SOURCE_INTERPRETATION_SCREEN" + and package["source_sha256"] == _code_hashes(), + "Study source changed", + ) + for name, expected in package["files_sha256"].items(): + _require(_sha(root / name) == expected, f"Frozen input changed: {name}") + _completion(root / "provenance" / "prior-completion.json") + archived = root / "provenance" / "parent" + manifest, execution, verification, runtime = _parent_files( + archived, gold_path=root / "sealed" / "gold.json" + ) + _require( + digest(_flat_census(archived, manifest, execution, verification, runtime)) + == package["parent_census_sha256"], + "Portable parent census changed", + ) + inputs = _source_inventory(archived, manifest, execution, verification, runtime) + _require(inputs == _read(root / "inputs.json"), "Source inputs changed") + jobs = _ordered_jobs(inputs) + _preflight(root / "source-preflight.json", inputs, jobs) + _require( + jobs == package["jobs"] + and len(jobs) == PLANNED_CALLS + and _read(root / "budget.json") == _budget(jobs) + and package["worst_case_retained_microinr"] == FULL_RESERVATION, + "Frozen jobs or budget changed", + ) + sidecars, omissions = _read(root / "sidecars.json"), _read(root / "omissions.json") + by_id = {item["id"]: item for item in inputs} + _require( + set(sidecars) == set(by_id) and set(omissions) == {j["id"] for j in jobs}, + "Evidence sidecar/omission coverage changed", + ) + for job in jobs: + item = by_id[job["history_id"]] + request, sidecar, ledger = ablation.render(item["request"], item["tool_trace"], job["arm"]) + ablation.verify_omissions(request, sidecar, ledger, job["arm"]) + _require( + sidecars[item["id"]] == sidecar + and omissions[job["id"]] == ledger + and digest(request) == job["request_sha256"] + and reservation(request) == job["reservation_microinr"] + and len(ledger) == job["omission_count"], + f"Reporter evidence changed: {job['id']}", + ) + return package, by_id + + +def execute_job(job, item, dispatch): + try: + request, _, _ = ablation.render(item["request"], item["tool_trace"], job["arm"]) + _require(digest(request) == job["request_sha256"], "Dispatched request changed") + return classify(dispatch(job["id"], request)) + except Exception as exc: + return { + "terminal": "infrastructure_error", + "error_class": type(exc).__name__, + "fatal_abort": isinstance(exc, ValueError), + } + + +def run( + root, *, allow_paid=False, transport_call=call, credential_loader=read_key, pacer_factory=Pacer +): + """One no-resume launch; tests inject transport and never read a credential.""" + if not allow_paid: + raise ValueError("Paid study requires explicit --allow-paid") + root = Path(root) + package, inputs = check(root) + _require( + not any((root / name).exists() for name in ("started.json", "execution.json", "calls")), + "Study already started; no retry or continuation", + ) + _require( + package["maximum_calls"] == PLANNED_CALLS + and package["worst_case_retained_microinr"] == FULL_RESERVATION + and package["workers"] == package["batch_size"] == 4 + and package["stop_after_batch_errors"] == 2, + "Run configuration changed", + ) + mode = ( + "production_default_transport" + if transport_call is call and credential_loader is read_key and pacer_factory is Pacer + else "injected_offline_transport" + ) + save(root / "started.json", {"manifest_sha256": digest(package), "execution_mode": mode}) + meter, pacer, key = ControlMeter(root), pacer_factory(root), credential_loader() + + def dispatch(job_id, request): + timing = pacer.wait() + save(meter.root / f"{job_id}.dispatch.json", timing) + return transport_call(meter, job_id, STAGE, request, key) + + outcomes = [] + errors = fatal_aborts = 0 + with ThreadPoolExecutor(max_workers=4) as pool: + for offset in range(0, len(package["jobs"]), 4): + batch = package["jobs"][offset : offset + 4] + results = [ + future.result() + for future in [ + pool.submit(execute_job, job, inputs[job["history_id"]], dispatch) + for job in batch + ] + ] + for job, result in zip(batch, results): + save(root / f"{job['id']}.result.json", result) + outcomes.append({"id": job["id"], "terminal": result["terminal"]}) + errors += sum(result["terminal"] == "infrastructure_error" for result in results) + fatal_aborts += sum(bool(result.get("fatal_abort")) for result in results) + print( + { + "attempted": len(outcomes), + "infrastructure_errors": errors, + "meter": meter.summary(), + }, + flush=True, + ) + if errors >= 2 or fatal_aborts: + break + for job in package["jobs"][len(outcomes) :]: + outcomes.append({"id": job["id"], "terminal": "not_attempted_after_stop"}) + save( + root / "execution.json", + { + "outcomes": outcomes, + "meter": meter.summary(), + "attempted": sum(o["terminal"] != "not_attempted_after_stop" for o in outcomes), + "infrastructure_errors": errors, + "fatal_aborts": fatal_aborts, + }, + ) + return _read(root / "execution.json") + + +def _receipt_replay(root, job, result, ledger): + """Replay one attempted output and charge from immutable local artifacts.""" + root = Path(root) + request_path = root / "calls" / f"{job['id']}.request.json" + receipt_path = root / "calls" / f"{job['id']}.receipt.json" + if not request_path.exists(): + _require( + not receipt_path.exists() + and job["id"] not in ledger + and result["terminal"] == "infrastructure_error", + "Unrecorded or unmetered dispatch", + ) + return { + "usage": None, + "receipt": None, + "body": None, + "cost_microinr": 0, + "retained_microinr": 0, + } + request_record = _read(request_path) + _require( + request_record["endpoint"] == ENDPOINT + and request_record["sha256"] == job["request_sha256"] == digest(request_record["request"]), + "Recorded reporter request changed", + ) + _require(job["id"] in ledger, "Unmetered reporter request") + row = ledger[job["id"]] + _require( + row[0] == STAGE and row[1] == job["reservation_microinr"], + "Reporter meter reservation changed", + ) + if not receipt_path.exists(): + _require( + result["terminal"] == "infrastructure_error" + and row[2] == row[1] + and row[3] is None + and row[4] == "reserved", + "Missing receipt has uncertain ledger state", + ) + return { + "usage": { + "known": False, + "prompt_tokens": None, + "completion_tokens": None, + "known_cost_microinr": None, + "retained_microinr": row[2], + "full_reservation_microinr": row[1], + }, + "receipt": None, + "body": None, + "cost_microinr": 0, + "retained_microinr": row[2], + } + receipt = _read(receipt_path) + _require( + receipt["job"] == job["id"] and receipt["request_sha256"] == job["request_sha256"], + "Reporter receipt/request changed", + ) + body, usage = _usage(receipt, request_record["request"]) + _require( + row[2] == usage["retained_microinr"] + and row[3] == usage["known_cost_microinr"] + and row[4] == ("known" if usage["known"] else "unknown"), + "Reporter usage/meter mismatch", + ) + good = ( + receipt.get("status") == 200 + and usage["known"] + and body.get("model") == request_record["request"]["model"] + ) + if good: + try: + replay = classify(body) + except (ValueError, KeyError, TypeError, IndexError): + good = False + else: + _require(replay == result, "Saved reporter classification changed") + if not good: + _require( + result["terminal"] == "infrastructure_error", + "Reporter infrastructure failure not preserved", + ) + return { + "usage": usage, + "receipt": receipt, + "body": body, + "cost_microinr": usage["known_cost_microinr"] or 0, + "retained_microinr": usage["retained_microinr"], + } + + +def _score(job, item, result, case, label, original, usage, receipt): + projection = None + if result.get("parsed") is not None: + try: + projection = project( + intent=case["intent"], + requested_change=case["requested_change"], + prior_receipts=original["seed_receipts"], + tool_trace=original["tool_trace"], + raw_content=result["content"], + completed=True, + ) + except (ValueError, KeyError, TypeError): + pass + view = report_view( + {"original": original, "reports": {job["arm"]: {**result, "application": projection}}}, + job["arm"], + ) + attribution = evaluate( + case, + label, + view, + { + "runtime_sha256": digest(case), + "gold_sha256": digest(label), + "question_target": label["question_target"], + }, + ) + raw = result.get("parsed") + target = attribution["actual_question_answer"] + valid = raw is not None + wrong_non_null = bool(valid and raw["answer"] is not None and raw["answer"] != target["answer"]) + false_abstention = bool( + valid + and target["answer"] is not None + and (raw["answer"] is None or raw["needs_clarification"] is True) + ) + unsupported_non_null = bool(valid and target["answer"] is None and raw["answer"] is not None) + unknown_guess = bool(unsupported_non_null and item["family"] == "unresolved_condition") + return { + "id": job["id"], + "history_id": item["id"], + "case_id": item["case_id"], + "scenario": item["scenario"], + "language": item["language"], + "family": item["family"], + "model": item["model"], + "arm": job["arm"], + "position": job["position"], + "treated_actor_prose": job["treated_actor_prose"], + "terminal": result["terminal"], + "valid_final": valid, + "raw": raw, + "application_output": projection["application_output"] if projection else None, + "wrong_non_null_date": wrong_non_null, + "false_abstention": false_abstention, + "unsupported_non_null_on_null_target": unsupported_non_null, + "unknown_condition_guess": unknown_guess, + "raw_fully_correct": bool( + attribution["factual_answer_correct"] and attribution["raw_status_matches_execution"] + ), + "projected_fully_correct": bool( + attribution["factual_answer_correct"] + and attribution["application_status_matches_execution"] + ), + "usage": usage, + "elapsed_seconds": receipt.get("elapsed_seconds") if receipt else None, + **attribution, + } + + +def _aggregate(rows): + return { + "n": len(rows), + "factual_correct": sum(bool(r["factual_answer_correct"]) for r in rows), + "valid_final": sum(r["valid_final"] for r in rows), + "invalid_or_missing_final": sum(not r["valid_final"] for r in rows), + "wrong_non_null_dates": sum(r["wrong_non_null_date"] for r in rows), + "false_abstentions": sum(r["false_abstention"] for r in rows), + "unknown_condition_guesses": sum(r["unknown_condition_guess"] for r in rows), + "raw_status_correct": sum(r["raw_status_matches_execution"] for r in rows), + "projected_status_correct": sum(r["application_status_matches_execution"] for r in rows), + "raw_fully_correct_reports": sum(r["raw_fully_correct"] for r in rows), + "projected_fully_correct_reports": sum(r["projected_fully_correct"] for r in rows), + "known_prompt_tokens": sum((r["usage"] or {}).get("prompt_tokens") or 0 for r in rows), + "known_completion_tokens": sum( + (r["usage"] or {}).get("completion_tokens") or 0 for r in rows + ), + "known_cost_microinr": sum( + (r["usage"] or {}).get("known_cost_microinr") or 0 for r in rows + ), + "retained_microinr": sum((r["usage"] or {}).get("retained_microinr") or 0 for r in rows), + "elapsed_seconds_sum": sum(r["elapsed_seconds"] or 0 for r in rows), + "elapsed_seconds_count": sum(r["elapsed_seconds"] is not None for r in rows), + } + + +def _bootstrap(pairs, scenario_universe, *, seed=ORDER_SEED, draws=10_000): + """Percentile interval from paired scenario resampling, never row IID.""" + clusters = defaultdict(list) + for before, after in pairs: + clusters[before["scenario"]].append( + int(after["factual_answer_correct"]) - int(before["factual_answer_correct"]) + ) + names = sorted(scenario_universe) + _require( + len(names) == 40 and set(clusters) <= set(names), "Bootstrap scenario universe changed" + ) + randomizer = random.Random(seed) + estimates = [] + zero_denominator_draws = 0 + for _ in range(draws): + sample = [clusters[names[randomizer.randrange(len(names))]] for _ in names] + denominator = sum(len(cluster) for cluster in sample) + if denominator: + estimates.append(sum(sum(cluster) for cluster in sample) / denominator) + else: + zero_denominator_draws += 1 + estimates.sort() + return { + "scenario_clusters": len(names), + "draws": draws, + "estimable_draws": len(estimates), + "zero_denominator_draws": zero_denominator_draws, + "seed": seed, + "units": "fraction_of_selected_histories", + "percentile_95": [ + estimates[int(0.025 * (len(estimates) - 1))], + estimates[int(0.975 * (len(estimates) - 1))], + ], + "percentile_95_percentage_points": [ + 100 * estimates[int(0.025 * (len(estimates) - 1))], + 100 * estimates[int(0.975 * (len(estimates) - 1))], + ], + } + + +def _contrasts(rows): + by_history = defaultdict(dict) + for row in rows: + by_history[row["history_id"]][row["arm"]] = row + _require( + len(by_history) == PARENT_EPISODES + and all(set(arms) == set(ARMS) for arms in by_history.values()), + "Paired arm coverage changed", + ) + scenario_universe = {arms[ARMS[0]]["scenario"] for arms in by_history.values()} + _require(len(scenario_universe) == 40, "Paired scenario universe changed") + contrasts = {} + specs = ( + ("A_to_B", ARMS[0], ARMS[1]), + ("B_to_C", ARMS[1], ARMS[2]), + ("A_to_C", ARMS[0], ARMS[2]), + ) + for name, left, right in specs: + variants = {} + for scope, selected in ( + ("all", list(by_history.values())), + ("treated", [x for x in by_history.values() if x[ARMS[2]]["treated_actor_prose"]]), + ( + "no_treatment", + [x for x in by_history.values() if not x[ARMS[2]]["treated_actor_prose"]], + ), + ): + for model in ("all", *PROFILES): + pairs = [ + (x[left], x[right]) + for x in selected + if model == "all" or x[left]["model"] == model + ] + if not pairs: + continue + gains = [ + x + for x, y in pairs + if not x["factual_answer_correct"] and y["factual_answer_correct"] + ] + losses = [ + x + for x, y in pairs + if x["factual_answer_correct"] and not y["factual_answer_correct"] + ] + key = f"{scope}/{model}" + variants[key] = { + "n": len(pairs), + "left_correct": sum(x["factual_answer_correct"] for x, _ in pairs), + "right_correct": sum(y["factual_answer_correct"] for _, y in pairs), + "gain": len(gains), + "loss": len(losses), + "net": len(gains) - len(losses), + "restored_from_invalid": sum(not x["valid_final"] for x in gains), + "corrected_usable_answer": sum(x["valid_final"] for x in gains), + "workflow_net": { + scenario: sum( + int(y["factual_answer_correct"]) - int(x["factual_answer_correct"]) + for x, y in pairs + if x["scenario"] == scenario + ) + for scenario in sorted({x["scenario"] for x, _ in pairs}) + }, + "scenario_cluster_bootstrap": _bootstrap(pairs, scenario_universe), + } + contrasts[name] = variants + return contrasts + + +def _gate( + rows, + contrasts, + *, + receipts_complete, + unknown_calls, + infrastructure_errors, + production_transport, +): + all_complete = receipts_complete and unknown_calls == 0 and infrastructure_errors == 0 + checks = {"all_720_clean": all_complete, "production_default_transport": production_transport} + for model in PROFILES: + treated = contrasts["B_to_C"][f"treated/{model}"] + operational = contrasts["A_to_C"][f"all/{model}"] + checks[f"treated_B_to_C_positive/{model}"] = treated["net"] > 0 + checks[f"two_positive_workflows/{model}"] = ( + sum(value > 0 for value in treated["workflow_net"].values()) >= 2 + ) + checks[f"all_C_to_A_positive/{model}"] = operational["net"] > 0 + by_arm = { + arm: _aggregate([r for r in rows if r["model"] == model and r["arm"] == arm]) + for arm in ARMS + } + for metric in ("wrong_non_null_dates", "false_abstentions"): + checks[f"C_no_higher_{metric}/{model}"] = all( + by_arm[ARMS[2]][metric] <= by_arm[arm][metric] for arm in ARMS[:2] + ) + for metric in ("raw_status_correct", "projected_status_correct"): + checks[f"C_no_lower_{metric}/{model}"] = all( + by_arm[ARMS[2]][metric] >= by_arm[arm][metric] for arm in ARMS[:2] + ) + return { + "checks": checks, + "mechanical_pass_pending_semantic_review": all(checks.values()), + "semantic_review_complete": False, + "forward_gate_pass": None, + } + + +def audit(root, *, output=None): + """Independent offline receipt/charge replay; only here may gold be parsed.""" + root = Path(root) + package, inputs = check(root) + started = _read(root / "started.json") + _require( + started["manifest_sha256"] == digest(package) + and started["execution_mode"] + in ("production_default_transport", "injected_offline_transport"), + "Started manifest binding changed", + ) + execution = _read(root / "execution.json") + jobs = package["jobs"] + outcomes = execution["outcomes"] + _require( + len(outcomes) == PLANNED_CALLS + and [row["id"] for row in outcomes] == [job["id"] for job in jobs], + "Planned outcome inventory changed", + ) + attempted = [row for row in outcomes if row["terminal"] != "not_attempted_after_stop"] + _require( + outcomes[: len(attempted)] == attempted + and execution["attempted"] == len(attempted) + and len(attempted) <= PLANNED_CALLS, + "Stopped outcome prefix changed", + ) + with sqlite3.connect(f"file:{root / 'calls' / 'meter.sqlite'}?mode=ro", uri=True) as db: + ledger = {r[0]: r[1:] for r in db.execute("SELECT * FROM calls")} + cases = {c["id"]: c for c in _read(root / "provenance" / "parent" / "runtime.json")} + labels = _read(root / "sealed" / "gold.json") + rows, response_ids, dispatch_order = [], set(), [] + known = retained = unknown = infrastructure = 0 + attempted_ids = {row["id"] for row in attempted} + for job, outcome in zip(jobs, outcomes): + item = inputs[job["history_id"]] + pair = _read(root / "provenance" / "parent" / "paired" / f"{item['id']}.json") + original = pair["original"] + case = cases[item["case_id"]] + label = labels[item["case_id"]] + _require(label["runtime_sha256"] == digest(case), "Gold/runtime source binding changed") + if job["id"] in attempted_ids: + result = _read(root / f"{job['id']}.result.json") + _require(result["terminal"] == outcome["terminal"], "Outcome/result changed") + replay = _receipt_replay(root, job, result, ledger) + usage, receipt, body = replay["usage"], replay["receipt"], replay["body"] + known += replay["cost_microinr"] + retained += replay["retained_microinr"] + unknown += bool(job["id"] in ledger and ledger[job["id"]][4] != "known") + infrastructure += result["terminal"] == "infrastructure_error" + if receipt and receipt.get("status") == 200 and usage["known"] and body.get("id"): + _require(body["id"] not in response_ids, "Duplicate provider response ID") + response_ids.add(body["id"]) + marker = root / "calls" / f"{job['id']}.dispatch.json" + if marker.exists(): + timing = _read(marker) + stamp = timing.get("dispatch_time") + if started["execution_mode"] == "production_default_transport": + _require( + type(stamp) in (int, float) + and timing.get("pacing_control", {}).get("rpm") == 90, + "Production dispatch timing changed", + ) + dispatch_order.append( + {"id": job["id"], "dispatch_time": stamp, "planned_position": job["position"]} + ) + elif ( + started["execution_mode"] == "production_default_transport" + and (root / "calls" / f"{job['id']}.request.json").exists() + ): + raise ValueError("Production request missing dispatch marker") + else: + _require( + not (root / f"{job['id']}.result.json").exists(), "Stopped job has saved result" + ) + result = {"terminal": "not_attempted_after_stop"} + usage = receipt = None + rows.append(_score(job, item, result, case, label, original, usage, receipt)) + _require( + infrastructure == execution["infrastructure_errors"] + and sum( + bool(_read(root / f"{job['id']}.result.json").get("fatal_abort")) + for job in jobs[: len(attempted)] + ) + == execution["fatal_aborts"], + "Infrastructure error count changed", + ) + request_ids = { + p.name.removesuffix(".request.json") for p in (root / "calls").glob("*.request.json") + } + receipt_ids = { + p.name.removesuffix(".receipt.json") for p in (root / "calls").glob("*.receipt.json") + } + dispatch_ids = { + p.name.removesuffix(".dispatch.json") for p in (root / "calls").glob("*.dispatch.json") + } + result_ids = {p.name.removesuffix(".result.json") for p in root.glob("*.result.json")} + _require( + result_ids == attempted_ids + and request_ids <= attempted_ids + and receipt_ids <= request_ids + and dispatch_ids <= attempted_ids + and set(ledger) == request_ids, + "Reporter artifact or meter inventory changed", + ) + meter_summary = ( + { + STAGE: { + "calls": len(ledger), + "charged_with_headroom_inr": retained / 1e6, + "known_usage_estimate_inr": known / 1e6, + "unknown_or_pending": sum(row[4] != "known" for row in ledger.values()), + } + } + if ledger + else {} + ) + _require( + retained <= PHASE_CAP + and PRIOR_RETAINED + retained <= CAMPAIGN_CAP + and execution["meter"] == meter_summary, + "Budget or saved meter summary changed", + ) + groups = {} + for dimensions in ( + ("model", "arm"), + ("language", "arm"), + ("family", "arm"), + ("model", "language", "arm"), + ("model", "family", "arm"), + ("treated_actor_prose", "arm"), + ): + kind = "/".join(dimensions) + keys = sorted({tuple(row[k] for k in dimensions) for row in rows}) + groups[kind] = { + "/".join(map(str, key)): _aggregate( + [row for row in rows if tuple(row[k] for k in dimensions) == key] + ) + for key in keys + } + contrasts = _contrasts(rows) + clean = ( + len(attempted) == PLANNED_CALLS + and len(receipt_ids) == PLANNED_CALLS + and len(ledger) == PLANNED_CALLS + ) + gate = _gate( + rows, + contrasts, + receipts_complete=clean, + unknown_calls=unknown, + infrastructure_errors=infrastructure, + production_transport=started["execution_mode"] == "production_default_transport", + ) + report = { + "verified": True, + "complete": clean and not infrastructure and not unknown, + "kind": "ALL_240_SAVED_HISTORY_SOURCE_INTERPRETATION_NOT_PARENT_REPAIR", + "execution_mode": started["execution_mode"], + "planned_calls": PLANNED_CALLS, + "attempted": len(attempted), + "reporter_receipts": len(receipt_ids), + "new_native_action_trials": 0, + "known_microinr": known, + "retained_microinr": retained, + "unknown_calls": unknown, + "infrastructure_errors": infrastructure, + "cumulative_calls": PRIOR_CALLS + len(ledger), + "cumulative_known_microinr": PRIOR_KNOWN + known, + "cumulative_retained_microinr": PRIOR_RETAINED + retained, + "rows": rows, + "actual_dispatch_order": sorted( + dispatch_order, + key=lambda row: (row["dispatch_time"] is None, row["dispatch_time"] or 0, row["id"]), + ), + "groups": groups, + "contrasts": contrasts, + "screen_gate": gate, + "scenario_clusters": 40, + "treated_histories": 184, + "no_treatment_histories": 56, + "development_exposed": True, + "independent_native_review": False, + } + if output is not None: + save(output, report) + return report + + +def main(argv=None): + parser = argparse.ArgumentParser() + sub = parser.add_subparsers(dest="command", required=True) + preparing = sub.add_parser("prepare") + preparing.add_argument("root") + preparing.add_argument("--parent", required=True) + preparing.add_argument("--prior-completion", required=True) + checking = sub.add_parser("check") + checking.add_argument("root") + running = sub.add_parser("run") + running.add_argument("root") + running.add_argument("--allow-paid", action="store_true") + reviewing = sub.add_parser("audit") + reviewing.add_argument("root") + reviewing.add_argument("--output") + args = parser.parse_args(argv) + if args.command == "prepare": + prepare(args.root, args.parent, args.prior_completion) + elif args.command == "check": + check(args.root) + elif args.command == "run": + run(args.root, allow_paid=args.allow_paid) + else: + result = audit(args.root, output=args.output) + if args.output is None: + print(json.dumps(result, ensure_ascii=False, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/docs/INDIC_ROADMAP.md b/docs/INDIC_ROADMAP.md index ae4bf1e..70c8750 100644 --- a/docs/INDIC_ROADMAP.md +++ b/docs/INDIC_ROADMAP.md @@ -22,7 +22,7 @@ The English record remains historical 423/500 once under its original judge; pro ## Sequence -1. Complete fresh native-action/reporting confirmation and review the remaining failures. +1. Native confirmation, one-case recovery and the720-call source-interpretation comparison are complete. The latest candidate fails reliability checks; next specify a question-specific evidence contract offline before any new paid comparison. 2. Establish a strong ordinary-storage baseline, then investigate persistent, cross-session and cross-language evidence failures with newly reviewed cases. Do not assume a failure in advance. 3. Select the smallest product component only if it fixes a reproduced residual problem and improves an observable workflow outcome. 4. Publish a reviewed multilingual benchmark and matched baselines; expand language and domain coverage in measured stages. @@ -35,4 +35,8 @@ No date for a novel product or industry-leading score is guaranteed. A result sh The corrected 240-episode comparison found a fixed-model base105B reporting gain (90→105/120), but the candidate failed acceptance: two reporter slots were never reached and Conversations wrong non-null dates increased 18→22. The separately proposed 108-call evidence-view screen is blocked and unexecuted; its selected capped history cannot be silently replaced. Source/condition interpretation and a skipped reporting handoff are concrete integration gaps. They do not yet establish a persistence problem or a novel memory product. See the [complete report](../benchmarks/audits/sarvam-fresh-confirmation-v3-2026-10-13/REPORT.md) and [bounded next task](../benchmarks/audits/sarvam-fresh-confirmation-v3-2026-10-13/NEXT_STEP.md). -A separate two-call recovery diagnostic subsequently produced a correct final report in both original variants for the sole capped English case, with no repeated action. The old v3 scores/gate and blocked108-call proposal stay unchanged. This removes one observed reporting failure; it does not establish persistence as the remaining cause. Next is a proposed all240-history source-interpretation comparison with fixed models and paired ordinary controls. [Recovery result and next step](../benchmarks/audits/sarvam-capped-recovery-2026-10-13/REPORT.md). +A separate two-call recovery diagnostic subsequently produced a correct final report in both original variants for the sole capped English case, with no repeated action. The old v3 scores/gate and blocked108-call proposal stay unchanged. This removes one observed reporting failure; it does not establish persistence as the remaining cause. The subsequently completed all240-history source-interpretation comparison is described below. [Recovery result and next step](../benchmarks/audits/sarvam-capped-recovery-2026-10-13/REPORT.md). + +## Latest source-interpretation decision + +The720-call comparison measured base104→105/120 and Conversations96→103/120 date-plus-clarification versus chronology, but failed predeclared reliability checks: more false abstentions on both models and worse status reporting/coverage. No candidate promotion or novel memory claim. Stop blanket prose removal; next investigate question-specific source interpretation against strong ordinary controls, with actual compute and fresh-case evidence. The historical English score is unchanged. [Report](../benchmarks/audits/sarvam-source-interpretation-2026-10-13/REPORT.md) and [next proposal](../benchmarks/audits/sarvam-source-interpretation-2026-10-13/NEXT_STEP.md). diff --git a/docs/SARVAM_RESEARCH.md b/docs/SARVAM_RESEARCH.md index 45ab646..8d9adb7 100644 --- a/docs/SARVAM_RESEARCH.md +++ b/docs/SARVAM_RESEARCH.md @@ -2,7 +2,26 @@ Updated October 13, 2026. We are investigating which problem warrants a product. We have measured integration gains and concrete failures, but no demonstrated unique multilingual memory advantage. This work is separate from the English baseline. -## Latest corrected comparison +## Latest: 720-call source-interpretation comparison + +All 240 saved native histories received three new reporters with the same models and profile: chronological A, lossless grouped B, and grouped C with prior agent prose withheld. No new actor or backend action was run. All720 calls completed once; two raw audits and complete English/Hindi/Telugu assistant reviews agree. + +| Model | A chronological | B grouped | C prose withheld | +| --- | ---: | ---: | ---: | +| Base105B | 104/120 (86.7%) | 99/120 (82.5%) | 105/120 (87.5%) | +| Conversations | 96/120 (80.0%) | 100/120 (83.3%) | 103/120 (85.8%) | + +**The acceptance gate failed; C is not promoted.** C reduces wrong definite dates but increases false abstentions on both models, loses base projected-status coverage through incomplete outputs, and worsens Conversations raw status. Conversations' fully correct raw reports fall86→80. C-versus-A gains/losses are11/10 base and14/7 Conversations; all14 Conversations gains fix usable wrong answers. Descriptive scenario-cluster intervals include zero, so unseen-task efficacy is not established. + +The predeclared primary C−B contrast on184 treated histories has net+8 base and+3 Conversations, also with intervals including zero. The other56 histories have identical B=C payloads; base still shows1gain/3losses, demonstrating run variability. Source removal and shorter input remain confounded. This is a development bank of40 related scenarios, not720 independent customers or a vendor-wide rate. + +New cost: ₹93.896460 known / ₹117.370843 retained. Native cumulative:7,757 calls, ₹365.121090 known / ₹488.660182 retained under the standing total₹6,000 ceiling. These are token estimates, not invoices. No further paid phase was launched. + +[Complete report](../benchmarks/audits/sarvam-source-interpretation-2026-10-13/REPORT.md), [offline replay](../benchmarks/audits/sarvam-source-interpretation-2026-10-13/README.md), [next offline proposal](../benchmarks/audits/sarvam-source-interpretation-2026-10-13/NEXT_STEP.md), [prior-art boundary](../benchmarks/audits/sarvam-source-interpretation-2026-10-13/PRIOR_ART_UPDATE.md). + +Stop blanket prose removal and wording sweeps. Investigate a source-grounded query/evidence contract that distinguishes customer requests and conditional preferences from confirmed state. Typed records, temporal history and provenance are existing capabilities in other systems; no new product or persistence need follows merely from implementing them. A useful future candidate needs matched compute, fresh reviewed cases, acceptable workflow latency and a reproducible residual problem. Independent human/native review is still missing. + +## Earlier corrected native comparison All 240 native episodes were attempted once. Forty related scenarios, each rendered in English/Hindi/Telugu, were tested with two fixed Sarvam profiles. Both reporters share each native history; only their first instruction differs. All 120 episodes per model stay in the denominator, including unusable and unreached reports. @@ -46,9 +65,9 @@ The skipped reporter after a capped, call-free native response is an identified A separate, prospectively frozen two-call diagnostic succeeded on the sole eligible capped episode. Both existing Sarvam-105B reporters returned the correct null date, clarification required, and no action requested. No actor or backend action was repeated. All 240 prior routing records were checked; the original study and its failed gate remain unchanged. This is one English case and no candidate-over-control advantage was found here. [Result, protocol and raw evidence](../benchmarks/audits/sarvam-capped-recovery-2026-10-13/REPORT.md). -The opt-in recovery path is now exercised with real provider responses, not just software tests. It adds compute: candidate reporting alone took 17.63 seconds, so this does not establish a fast voice path. The diagnostic added ₹0.566850 known estimate / ₹0.708564 retained. Native research cumulative is now 7,037 calls, ₹271.224630 known / ₹371.289339 retained. No new English LongMemEval result or multilingual memory advantage follows. +The opt-in recovery path is now exercised with real provider responses, not just software tests. It adds compute: candidate reporting alone took 17.63 seconds, so this does not establish a fast voice path. The diagnostic added ₹0.566850 known estimate / ₹0.708564 retained. At the close of that diagnostic, native research cumulative was 7,037 calls, ₹271.224630 known / ₹371.289339 retained. No new English LongMemEval result or multilingual memory advantage follows. -The next proposed study compares all 240 saved histories across ordinary chronological evidence, lossless evidence groups, and those groups without nonempty actor prose. It targets source interpretation with fixed models and reports both gains and regressions. The 720-call proposal remains unexecuted; the older blocked 108-call screen is preserved. [Next protocol requirements](../benchmarks/audits/sarvam-capped-recovery-2026-10-13/NEXT_STEP.md). +That diagnostic proposed the all240-history/720-call study now completed above. Its original proposal and the older blocked108-call screen remain preserved; neither historical result is rewritten. ## Accounting and reproducibility diff --git a/docs/SARVAM_SOURCE_INTERPRETATION.md b/docs/SARVAM_SOURCE_INTERPRETATION.md new file mode 100644 index 0000000..fb57fd6 --- /dev/null +++ b/docs/SARVAM_SOURCE_INTERPRETATION.md @@ -0,0 +1,20 @@ +# Sarvam saved-history source interpretation screen + +Completed once:720/720 calls. The candidate increases some factual counts but fails reliability gates and is not promoted. [Results and replay package](../benchmarks/audits/sarvam-source-interpretation-2026-10-13/README.md). Historical studies are unchanged. + +`benchmarks/sarvam_source_interpretation.py` prepares a prospective, reporter-only comparison over all 240 frozen v3 native histories. It sends three fixed reporting views per history: chronological A, lossless grouped B, and grouped C with only nonempty actor prose withheld. The same `sarvam_evidence_ablation` renderer, model profiles, reporter instruction, schema and 8,192-token cap are used for every arm. No actor, tool or backend action is executed. + +Preparation takes the frozen v3 parent, the completed two-call diagnostic's `completion.json`, and an existing new study directory containing reviewed `PROTOCOL.md`, `budget.json`, `pacing-control.json` and `source-preflight.json`: + +```text +python -m benchmarks.sarvam_source_interpretation prepare STUDY_ROOT --parent V3_ROOT --prior-completion DIAGNOSTIC_COMPLETION +python -m benchmarks.sarvam_source_interpretation check STUDY_ROOT +python -m benchmarks.sarvam_source_interpretation run STUDY_ROOT --allow-paid +python -m benchmarks.sarvam_source_interpretation audit STUDY_ROOT --output NEW_AUDIT_JSON +``` + +Only the final two commands can follow preparation. `run` has no restart or retry path: it writes an exclusive start marker, uses fixed four-request batches and four workers with 90-RPM pacing, and stops before a new batch after two cumulative infrastructure errors or one fatal validation/budget error. The old v3 and blocked 108-call screen remain unchanged. Gold is copied and hashed as opaque bytes during preparation and execution; only offline audit parses it. Preparation saves all 240 source records, paired parent records, 239 original candidate requests, the capped actor's original request and receipt, omission ledgers, and hashes for each of 720 planned requests. The capped history is reconstructed from that actor receipt, never from the later diagnostic answers. + +Audit replays recorded requests, responses, per-call usage and meter charges. It retains all 720 planned slots if execution stops early. It reports raw and receipt-projected status separately, date-plus-clarification accuracy, wrong definite dates, false abstentions, invalid finals, cost, latency, observed dispatch order, and paired A/B/C contrasts. C versus B on the predeclared 184 actor-prose-treated histories is primary; 56 B=C no-treatment histories are reported separately. Scenario-cluster bootstrap intervals resample all 40 source scenarios. The mechanical forward gate stays distinct from independent semantic review, and injected offline transport cannot pass the live-transport gate. + +The study is a development-exposed synthetic reporting comparison with unequal actual input-token counts. A gain would support this reporting-boundary intervention on the fixed histories; it would not establish a new memory architecture, a production workflow result, or the unique cause of an actor-prose effect. The frozen protocol and preserved raw receipts are the source of truth for the actual phase. diff --git a/tests/test_sarvam_source_interpretation.py b/tests/test_sarvam_source_interpretation.py new file mode 100644 index 0000000..6657e3a --- /dev/null +++ b/tests/test_sarvam_source_interpretation.py @@ -0,0 +1,253 @@ +"""Offline checks for the 240-history, three-arm source study.""" + +import contextlib +import copy +import io +import json +import shutil +import time +from pathlib import Path + +import pytest + +from benchmarks import sarvam_source_interpretation as study +from benchmarks.sarvam_native_study.transport import ENDPOINT, digest, save +from benchmarks.sarvam_reporting_handoff import prepare_plan +from tests.test_sarvam_reporting_handoff import fixture, reply + +EXPERIMENTS = Path( + "/Volumes/Sahith_SSD/AgentMem-OS/codex-memory-2026-09-08/tracks/sarvam/experiments" +) + + +def synthetic_inputs(): + base = prepare_plan(**fixture())["reports"]["candidate"]["request"] + requests = {"sarvam-105b": base} + conversation = copy.deepcopy(base) + conversation["model"] = "sarvam-105b-conversations" + conversation["reasoning_effort"] = None + requests[conversation["model"]] = conversation + inputs = [] + for n in range(40): + for language in ("en", "hi", "te"): + for model in requests: + ident = f"synthetic-{n:02d}-{language}-{model}" + inputs.append( + { + "id": ident, + "case_id": ident, + "scenario": f"scenario-{n:02d}", + "language": language, + "family": "synthetic", + "model": model, + "request": requests[model], + "tool_trace": [], + } + ) + return inputs + + +def test_six_orders_per_scenario_and_exact_budget(): + jobs = study._ordered_jobs(synthetic_inputs()) + assert len(jobs) == 720 + assert study._budget(jobs)["maximum_full_context_reservation_microinr"] == 1_548_288_000 + by_scenario = {} + for job in jobs: + by_scenario.setdefault(job["scenario"], {}).setdefault(job["history_id"], []).append( + job["arm"] + ) + for histories in by_scenario.values(): + assert len(histories) == 6 + assert len({tuple(order) for order in histories.values()}) == 6 + assert all(not job["treated_actor_prose"] for job in jobs) + by_history = {(job["history_id"], job["arm"]): job for job in jobs} + for item in synthetic_inputs(): + assert by_history[item["id"], study.ARMS[1]]["request_sha256"] == ( + by_history[item["id"], study.ARMS[2]]["request_sha256"] + ) + + +def test_bootstrap_resamples_all_40_scenarios_even_with_sparse_treatment(): + before = {"scenario": "scenario-00", "factual_answer_correct": False} + after = {"scenario": "scenario-00", "factual_answer_correct": True} + result = study._bootstrap( + [(before, after)], {f"scenario-{n:02d}" for n in range(40)}, draws=1000 + ) + assert result["scenario_clusters"] == 40 + assert result["zero_denominator_draws"] > 0 + assert result["estimable_draws"] + result["zero_denominator_draws"] == 1000 + assert result["percentile_95_percentage_points"] == [100.0, 100.0] + + +def test_stop_after_two_cumulative_infrastructure_errors_and_refuse_resume(tmp_path, monkeypatch): + inputs = synthetic_inputs() + jobs = study._ordered_jobs(inputs) + package = { + "jobs": jobs, + "maximum_calls": 720, + "worst_case_retained_microinr": study.FULL_RESERVATION, + "workers": 4, + "batch_size": 4, + "stop_after_batch_errors": 2, + } + monkeypatch.setattr(study, "check", lambda _root: (package, {x["id"]: x for x in inputs})) + save(tmp_path / "budget.json", study._budget(jobs)) + calls = [] + + class NoWait: + def __init__(self, _root): + pass + + def wait(self): + return {"dispatch_time": time.time(), "pacing_control": {"rpm": 90}} + + def transport(_meter, job, _stage, _request, _key): + calls.append(job) + if len(calls) in (2, 4): + raise RuntimeError("offline simulated infrastructure failure") + return reply(None, status="not_requested", clarification=True) + + with contextlib.redirect_stdout(io.StringIO()): + execution = study.run( + tmp_path, + allow_paid=True, + transport_call=transport, + credential_loader=lambda: "offline", + pacer_factory=NoWait, + ) + assert execution["attempted"] == 4 and execution["infrastructure_errors"] == 2 + assert len(execution["outcomes"]) == 720 + assert all(row["terminal"] == "not_attempted_after_stop" for row in execution["outcomes"][4:]) + with pytest.raises(ValueError, match="already started"): + study.run( + tmp_path, + allow_paid=True, + transport_call=transport, + credential_loader=lambda: pytest.fail("credential accessed on resume"), + pacer_factory=NoWait, + ) + assert len(calls) == 4 + + +def test_one_fatal_value_error_stops_after_started_batch(tmp_path, monkeypatch): + inputs = synthetic_inputs() + jobs = study._ordered_jobs(inputs) + package = { + "jobs": jobs, + "maximum_calls": 720, + "worst_case_retained_microinr": study.FULL_RESERVATION, + "workers": 4, + "batch_size": 4, + "stop_after_batch_errors": 2, + } + monkeypatch.setattr(study, "check", lambda _root: (package, {x["id"]: x for x in inputs})) + save(tmp_path / "budget.json", study._budget(jobs)) + + class NoWait: + def __init__(self, _root): + pass + + def wait(self): + return {"dispatch_time": time.time(), "pacing_control": {"rpm": 90}} + + def transport(_meter, job, _stage, _request, _key): + if job == jobs[0]["id"]: + raise ValueError("budget/source validation failed") + return reply(None, status="not_requested", clarification=True) + + with contextlib.redirect_stdout(io.StringIO()): + execution = study.run( + tmp_path, + allow_paid=True, + transport_call=transport, + credential_loader=lambda: "offline", + pacer_factory=NoWait, + ) + assert execution["attempted"] == 4 and execution["fatal_aborts"] == 1 + assert execution["infrastructure_errors"] == 1 + + +def test_paid_path_refuses_by_default_before_source_or_key(tmp_path, monkeypatch): + monkeypatch.setattr(study, "check", lambda _root: pytest.fail("source checked")) + with pytest.raises(ValueError, match="allow-paid"): + study.run(tmp_path, credential_loader=lambda: pytest.fail("credential accessed")) + + +def test_actual_parent_offline_freeze_and_fake_720_replay_when_available(tmp_path, monkeypatch): + parent = EXPERIMENTS / "2026-10-13-fresh-workflow-confirmation-v3" + phase = EXPERIMENTS / "2026-10-13-source-interpretation-full" + prior = EXPERIMENTS / "2026-10-13-capped-actor-recovery-diagnostic/completion.json" + if not all(path.exists() for path in (parent, phase, prior)): + pytest.skip("Frozen local Sarvam study artifacts unavailable") + for name in ("PROTOCOL.md", "budget.json", "pacing-control.json", "source-preflight.json"): + shutil.copyfile(phase / name, tmp_path / name) + original_read = study._read + + def no_gold_read(path): + if Path(path).name == "gold.json": + pytest.fail("Gold parsed before offline audit") + return original_read(path) + + with monkeypatch.context() as guarded: + guarded.setattr(study, "_read", no_gold_read) + package = study.prepare(tmp_path, parent, prior) + checked, inputs = study.check(tmp_path) + assert checked == package and len(inputs) == 240 + assert package["treated_actor_prose_histories"] == 184 + assert len(package["jobs"]) == 720 + assert not (tmp_path / "started.json").exists() + frozen_inputs = (tmp_path / "inputs.json").read_bytes() + (tmp_path / "inputs.json").write_bytes(frozen_inputs + b" ") + with pytest.raises(ValueError, match="Frozen input changed"): + study.check(tmp_path) + (tmp_path / "inputs.json").write_bytes(frozen_inputs) + + class NoWait: + def __init__(self, _root): + pass + + def wait(self): + return {"dispatch_time": time.time(), "pacing_control": {"rpm": 90}} + + def transport(meter, job, stage, request, _key): + save( + meter.root / f"{job}.request.json", + {"endpoint": ENDPOINT, "request": request, "sha256": digest(request)}, + ) + meter.reserve(job, stage, request) + body = reply(None, status="not_requested", clarification=True) + body["id"] = "offline-" + job + body["model"] = request["model"] + body["usage"] = {"prompt_tokens": 100, "completion_tokens": 30, "total_tokens": 130} + save( + meter.root / f"{job}.receipt.json", + { + "job": job, + "request_sha256": digest(request), + "status": 200, + "body_text": json.dumps(body), + "elapsed_seconds": 0.01, + }, + ) + assert meter.settle(job, body, request["model"], request["max_tokens"]) + return body + + with monkeypatch.context() as guarded: + guarded.setattr(study, "_read", no_gold_read) + with contextlib.redirect_stdout(io.StringIO()): + execution = study.run( + tmp_path, + allow_paid=True, + transport_call=transport, + credential_loader=lambda: "offline", + pacer_factory=NoWait, + ) + assert execution["attempted"] == 720 + audit = study.audit(tmp_path) + assert audit["verified"] and audit["complete"] + assert len(audit["rows"]) == 720 and audit["reporter_receipts"] == 720 + assert audit["known_microinr"] == 2_376_000 + assert audit["screen_gate"]["checks"]["production_default_transport"] is False + assert audit["contrasts"]["B_to_C"]["treated/all"]["n"] == 184 + assert audit["contrasts"]["B_to_C"]["no_treatment/all"]["n"] == 56 + assert len(audit["actual_dispatch_order"]) == 720