Skip to content

Reporting-coverage reference model: package_ids exclusions and in-place membership removal disagree with both SDKs #8678

Closed

Description

@bokelley

Summary

Differential testing of the reporting-subscription coverage rule (the reference model in scripts/reference/reporting-coverage/, owedSubscriptionPeriodsV1 in @adcp/sdk 17.0.0-beta.11, and owed_subscription_periods in adcp 10.0.0b4, all on tzdata 2026f) found two inputs where both SDKs agree with each other and disagree with the reference model. Reading the spec text, the SDKs look right and the reference model looks wrong. Neither case is covered by vectors.json, so the model and both SDKs pass every current vector.

The harness that found these is in the SDK interop PR (tests/interop/differential/). Replay either repro with node tests/interop/differential/cli-ts.mjs or cli_py.py, or call expectedSubscriptionPeriods directly.

1. package_ids exclusions are ignored even when they name every package of the buy

exceptionWindows() skips every excluded[] entry that has package_ids (the comment reads "package_ids entries remove packages, not periods"). The package_ids description in core/reporting-subscription-coverage.json says "these are removed from its coverage rather than its periods" and "The buy stays covered for its other packages". When the named packages are the only packages that resolve the offering, nothing keeps the buy covered for that offering, so its periods from from should not be owed.

Repro: take the baseline case input and add:

"coverage": { "excluded": [ { "reason": "overridden", "media_buy_id": "mb_1", "package_ids": ["pkg_1"],
  "overridden_by": { "subscription_id": "analytics-daily-2" }, "from": "2026-11-04T00:00:00Z" } ] }
Owed period starts
Reference model 11-03, 11-04, 11-05
TS SDK 11-03
Python SDK 11-03

The same happens with offering_unsupported plus package_ids. The only vector with a package_ids entry (exceptions-only-remove-periods) names pkg_9, which the buy doesn't have, so it doesn't tell the two readings apart.

2. An account removed in place keeps none of its pre-removal periods

membership (core/reporting-subscription.json x-adcp-validation) says a removed account "opens an excluded_by_buyer entry from E, so periods that started before T stay owed in full", and "stays listed in subscription_coverage with status excluded while its open entry is retained". The reference model's inScope() checks only the current configuration's scope. Once the scope no longer selects the account, every period is dropped, including the ones before the removal.

Repro: take the baseline case input, set versions[0].configuration.scope to {"account_ids": ["acct_other"]}, and add:

"coverage": { "excluded": [ { "reason": "excluded_by_buyer", "from": "2026-11-05T00:00:00Z" } ] }
Owed period starts
Reference model none
TS SDK 11-03, 11-04
Python SDK 11-03, 11-04

No vector has an excluded_by_buyer entry while the account is out of scope.

Suggested fix

  • Derive both cases from the text.
  • Add them to vectors.json.
  • Fix the reference model:
    • a package_ids entry should remove the (buy, offering) periods starting in [from, until) when the entry names every package of the buy that resolves the offering;
    • scope should be evaluated through the excluded_by_buyer intervals when the current scope excludes the account.

The nightly differential run in the interop PR will fail on these until they're settled.

🤖 Generated with Claude Code

Activity

  1. bokelley commented on Nov 1, 2026

    @bokelley
    ContributorAuthor

    The harness is in #8679 (tests/interop/differential/).

  2. bokelley commented on Nov 1, 2026

    @bokelley
    ContributorAuthor

    A related question for whoever settles this. The three implementations also differ on when a coverage input error is raised. The numeric-offset vector says a refusal must follow "from the coverage rule", but error-codes.json doesn't state that for every detail:

    One sentence in error-codes.json would settle every case: either "raised only when the rule reads the value", or "raised whenever present in the input".

  3. added a commit that references this issue on Nov 2, 2026
    13c5ba4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions