Reporting metric coherence — deferred review findings (post-#6618 stack)

A three-perspective review (protocol, buyer-workflow, code) of the delivery-reporting coherence stack (#6618 + #6620/#6621/#6622/#6626/#6627, plus #6616) produced findings that were fixed in the PRs, and the following accepted-but-deferred items. Recording them so they don't silently vanish; none blocks the stack.

Single-qualifier-per-(package, metric_id) is asserted, not enforced. get-media-buy-delivery-response.json justifies flat per-buy totals with "each buy is single-qualifier by definition," but committed-metric.json validates two entries with the same metric_id and different viewability_standard on one package — making such a commitment structurally unsatisfiable at package grain. Either enforce uniqueness at commitment validation or drop the by-definition claim. WG-level decision.
missing_metrics doesn't state where a leaf must appear (package totals vs any breakdown row). The sane reading for quartile_75 is quartile_data.q3_views present in package totals — one clause would make it a rule instead of a reading.
time_based_views: absent vs [] is undefined on a mixed video/display package (quartile_data uses null for not-applicable; the array has no analog).
reporting_dimensions.demographic is the only closed dimension block (additionalProperties: false) — 3.1 sellers hard-reject new fields there (e.g. sort_direction) but silently ignore them on the other dimensions. Pre-existing closure asymmetry; degradation behavior differs per dimension.
No video_starts metric exists, so cross-channel completion rates are all computed over impressions — autoplay-inflated on social, start-deflated on click-to-play olv — and completion_rate's meaning shifts per package when view_duration_seconds is set. Candidate for one honesty sentence in optimization-reporting.mdx, or a metric addition if buyers ask.
Conformance-grading note: with requested_metrics (feat(reporting): add requested_metrics to get_media_buy_delivery #6626), a buyer can sort by a metric excluded from the payload — graders MUST NOT attempt order-verification from response bodies in that case (the applied-sort echo is the contract surface).

🤖 Generated with Claude Code

@torbenbrodt

Triage

Classification: Spec question (multi-item deferred review findings)
Bucket(s): spec / protocol, schema, compliance-suite
Status: partially advanced — item 1 split into two actionable parts (see below); items 2, 3, 5, 6 status unchanged

The #6618 stack has landed.

Item 1 (updated 2026-10-07 — see @torbenbrodt's analysis above)

#7210 settled the package-grain satisfiability problem by shipping by_package[].metric_values with the rule "Sellers report one row per full qualifier set at package grain; when a metric appears here, its flat scalar counterpart on the package MUST be omitted." The "enforce uniqueness vs. drop the claim" fork from the original triage is now closed. What remains splits:

1a — Sentence drop (non-breaking, ready to implement). aggregated_totals.metric_aggregates description (L255 region) contains: "Per-buy totals keeps its flat shape — each buy is single-qualifier by definition." That sentence is stale. vendor_metric_values on per-buy totals already allows multi-qualifier rows; brand_lift is described as "Multidimensional in production" with per-lift_dimension rows; viewability explicitly handles the multi-standard case. No semantic change — this is removing a description that contradicts existing schema behavior.

File: static/schemas/source/media-buy/get-media-buy-delivery-response.json (the sentence in the metric_aggregates description).

1b — Omission rule for per-buy and window totals (needs WG vote). Proposal: when by_package[N].metric_values carries multiple qualifier sets for a metric_id, sellers MUST omit that metric's flat carrier on per-buy and window totals (and in media-buy-delivery-webhook-result.json). This extends the existing metric_values omission pattern — which already applies at the package scalar level — to sub-aggregate surfaces.

Wire shape of the proposed rule:

// by_package[N] — two lift_dimension rows in metric_values
{
  "metric_values": [
    { "metric_id": "brand_lift", "qualifier": { "lift_dimension": "awareness" }, "value": 0.04 },
    { "metric_id": "brand_lift", "qualifier": { "lift_dimension": "purchase_intent" }, "value": 0.07 }
  ],
  "totals": {
-   "brand_lift": 0.055,  // MUST be omitted — metric_values owns this metric_id
    "impressions": 1000000
  }
}

This is new normative MUST language; existing sellers that emit both a flat carrier and multi-qualifier metric_values rows for the same metric would need to conform. RFC #7744 (@torbenbrodt's team) is blocked on this decision.

Item 4 is already governed by DR-0009 (published schemas default additionalProperties: true; tightening is breaking and policy-wide). The correct resolution is to remove the additionalProperties: false from demographic to align with the decision record — not to add it to the other dimensions.

Triaged by Claude Code. Session: https://claude.ai/code/session_01996c6guAjACp18zjnqVqNK

Generated by Claude Code

Adding two more audit findings recorded during the coherence work but not yet tracked anywhere durable:

canonical-forecast-point dropped normative constraints from its source twin — the source forecast-point.json enumerates 17 named metric properties with maximum: 1 bounds on rate fields and an anyOf forcing standard whenever any viewability value is present; the canonical variant is a bare additionalProperties: forecast-range with none of that. No tooling compares canonical pairs against their source twins (the enum-drift linter only covers inline enums), so this drift is silent. Candidate fix: a canonical-pair parity test in the spirit of metric-qualifier-parity.
reporting-webhook.json inlines a 3-value reporting_frequency subset (hourly|daily|monthly) of the 6-value enums/reporting-frequency.json that windowed_pull_granularities refs. If the subset is intentional (webhook cadence deliberately narrower), it should be documented and allowlisted in scripts/lint-schema-enum-drift.cjs; if not, the webhook should ref the enum.

🤖 Generated with Claude Code

Triage (items 7–8)

Classification: Spec question (additional coherence findings)
Bucket(s): schema, compliance-suite
Status: item 7 → ready-to-implement (independently); item 8 → needs-wg-review

Items 7 and 8 were recorded after the prior triage above.

Item 7 (canonical-forecast-point parity gap) is independently actionable — no dependency on the feat(reporting): add leaf metric identities for nested delivery values #6618 stack. Candidate fix: add a canonical-pair parity test asserting that the 17 named metric properties, maximum: 1 bounds on rate fields, and the anyOf/standard constraint from forecast-point.json are all present in the canonical variant. Non-breaking tooling addition; can be queued without waiting for the stack to land.
Item 8 (reporting-webhook.json enum subset) is a design decision: if the 3-value subset (hourly|daily|monthly) is intentional, document the narrower webhook cadence and add an allowlist entry in scripts/lint-schema-enum-drift.cjs; if not, point reporting-webhook.json's reporting_frequency to enums/reporting-frequency.json. Either path is non-breaking once intent is confirmed. Routing to the WG queue alongside item 1.

Triaged by Claude Code. Session: https://claude.ai/code/session_01FptX2daafsXixedTBXgJh9

Generated by Claude Code

On item 1: #7210 already settled this at package grain. What remains is one stale sentence and one omission rule.

This item was recorded on 2026-08-17. #7210 (3.2) then added by_package[].metric_values: "Sellers report one row per full qualifier set at package grain" (get-media-buy-delivery-response.json#L482). Two viewability_standard commitments on one package are now satisfiable at package grain.

The "each buy is single-qualifier by definition" sentence survives only in the description of aggregated_totals.metric_aggregates (#L255), which 3.2 deprecated. The schema per-buy totals is built from already contradicts it:

brand_lift is "Multidimensional in production", one row per lift_dimension (delivery-metrics.json#L88).
viewability covers the case "When a buy reports under more than one standard" (#L480).
vendor_metric_values, which per-buy totals already carries: "the same vendor metric MAY appear in multiple rows only when each carries a distinct qualifier" (#L703, Vendor-metric uniqueness lint ignores qualifier, rejecting rows the spec allows #7234).

Enforcing one qualifier per (package, metric_id) would contradict metric_values, and would remove the reason lift_dimension and attribution_window exist. If the WG wants an enforcement half, its key is the full (scope, metric_id, qualifier) tuple, as on vendor_metric_values, not metric_id (with methodology_version for vendor scope, see #7235).

Proposed resolution

Drop the by-definition sentence from L255.
On per-buy and window totals, here and in media-buy-delivery-webhook-result.json: when a buy's metric_values carry more than one qualifier set for a metric_id, sellers MUST omit that metric's flat carrier (viewability, brand_lift, …) rather than combine the rows, and buyers read the package rows. This is the pattern 3.2 already uses for mixed-currency buys.

No current qualifier applies to impressions or spend, so the requested_metrics "always included" rule and the storyboards that assert totals.impressions, totals.clicks and a single-standard totals.viewability are unaffected.

Disclosure: our RFC #7744 is blocked on this item. It adds a qualifier that does apply to impressions and clicks; how that meets "always included" is for #7744 to settle, not this item.

@torbenbrodt

WG decision brief (3.3)

Item 1b, consent-style. Draft PR with the recommended shape: #8055

Proposal: when the buy's by_package[].metric_values carry more than one qualifier set for a metric_id, sellers MUST omit that metric's flat carrier (e.g. viewability, brand_lift) from per-buy and window totals, here and in media-buy-delivery-webhook-result.json. Buyers read the package rows. This mirrors the shipped mixed-currency pattern. impressions, spend and "always included" are unaffected, since no qualifier applies to them today.

Questions

Ratify the omission rule (yes / no)?
Is the identity key for "one row" (scope, metric_id, qualifier), as on vendor_metric_values, not metric_id alone (yes / no)?
Cross-package divergence: package A reports viewability_standard=mrc and package B groupm, one row each. As drafted, the rule does not fire and flat totals would combine two standards. Should it fire when qualifier sets differ across the buy's packages, or stay within a single package's rows (pick one: across packages / within a package)?

Default if no objection by 2026-10-15: Q1 yes, Q2 yes, Q3 within a package (the shape in #8055).

Notes

RFC: ivt_filtration metric qualifier — the MRC filtration level of a row #7744 (ivt_filtration, @torbenbrodt) proceeds after this. How a qualifier on impressions meets "always included" is RFC: ivt_filtration metric qualifier — the MRC filtration level of a row #7744's question, not this one.
1a (the stale "each buy is single-qualifier by definition" sentence in metric_aggregates) is not in feat(schema): omit flat totals carriers when package metric_values are multi-qualifier #8055 and should land with or before it.
Not covered in the draft: daily_breakdown[].totals and a requested_metrics carve-out like the mixed-currency one. Both are candidates for a follow-up if ratified.

Thanks for picking this up, and for #8055.

Q1: yes.

Q2: yes. (scope, metric_id, qualifier), as on vendor_metric_values.

Q3: across packages. "Within a package" leaves the case the rule exists for. Package A mrc plus package B groupm would still combine two standards in per-buy totals.viewability, which the schema rules out elsewhere ("never sum across standards", delivery-metric-aggregate.json).

The trigger also needs to count a package's flat carrier, not only metric_values. A package reporting a single standard can use the flat viewability block with its own standard and no metric_values row, so A as a metric_values row (mrc) plus B as a flat block (groupm) would still slip through. Suggested trigger: "when the buy's packages report a metric_id under more than one qualifier set, whether in by_package[].metric_values or in a package's flat carrier."

One consistency note on #8055: the schema descriptions say "when the buy's by_package[].metric_values carry more than one qualifier set" (across packages), while the changeset says "when any by_package[].metric_values carries more than one" (within a package). Whichever reading the WG ratifies, the two should say the same.

Agreed that "always included" for impressions is #7744's question. I'll take it up there once this lands.

feat(schema): omit flat totals carriers when package metric_values ar…

fix(schema): apply omission rule across packages and count flat carriers

feat(schema): omit flat totals carriers when package metric_values ar…

@torbenbrodt

Thanks @torbenbrodt, applied in #8055 (consent closed 2026-10-16: Q1 yes, Q2 yes, Q3 across packages).

The trigger is now "when the buy's packages report a metric_id under more than one qualifier set, whether in by_package[].metric_values or in a package's flat carrier". Sellers MUST omit that metric's flat carrier from per-buy and window totals, and from the webhook result's per-buy totals.
The schema descriptions and the changeset now say the same thing (across packages). The multi_qualifier_totals verifier constraint reads the same way.
The identity key stays (scope, metric_id, qualifier).
RFC: ivt_filtration metric qualifier — the MRC filtration level of a row #7744 can take up how a qualifier on impressions meets "always included" once feat(schema): omit flat totals carriers when package metric_values are multi-qualifier #8055 lands.
Still not covered, as noted in the brief: daily_breakdown[].totals and a requested_metrics carve-out.

fix(schema): apply omission rule across packages and count flat carriers

fix(schema): drop the stale single-qualifier-per-buy sentence (#6628 …

feat(schema): omit flat totals carriers when package metric_values ar…

Part of this shipped: items 1a and 1b (flat totals omitted for multi-qualifier metrics) in #8055 and item 7 (canonical-forecast-point constraints) in #6868, both 3.3.0-beta.0. Decision: #6628 (comment).

Items 2, 3, 4, 5, 6 and 8 have no PR or decision, so per the WG rule ('anything not decided moves to 3.4') only the remainder moves to 3.4. Objections by Fri 2026-11-06. The milestone stays 3.3.0 for now; it will be moved after Nov 6.

Navigation Menu

Repository navigation

Reporting metric coherence — deferred review findings (post-#6618 stack) #6628

Triage

Triage (items 7–8)

WG decision brief (3.3)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Reporting metric coherence — deferred review findings (post-#6618 stack) #6628

Description

Activity

bokelley commented on Aug 17, 2026

Triage

bokelley commented on Aug 25, 2026

bokelley commented on Aug 25, 2026

Triage (items 7–8)

torbenbrodt commented on Oct 7, 2026

bokelley commented on Oct 8, 2026

WG decision brief (3.3)

torbenbrodt commented on Oct 9, 2026

bokelley commented on Oct 19, 2026

bokelley commented on Oct 31, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions