Skip to content

[ING-508] fix(alerts): re-arm current usage alerts at renewal - #6132

Open
aquinofb wants to merge 1 commit into
ing-509-recovery-enginefrom
ing-508-renewal-baseline
Open

aquinofb wants to merge 1 commit into
ing-509-recovery-enginefrom
ing-508-renewal-baseline

Conversation

@aquinofb

Copy link
Copy Markdown
Contributor

Stacked on the ING-509 PR. GitHub will retarget this to main once that merges.

Do not merge before the ING-505 measurement has run. Product's sign-off was conditional on counting the affected population first, because alerts that used to be silently swallowed will start firing. The queries are ready on ING-505.

Context

At renewal, usage starts again from nothing. To an alert that compares against the last value it saw, that looks like every threshold clearing at once. The same comparison hides a second problem that exists today: an evaluation early in a new period compares small new usage against the large figure carried over, finds nothing crossed, and stores the small figure, so a threshold the customer really did cross in those first moments is never reported.

Changes

When the last evaluation belongs to an earlier period, the comparison starts from nothing instead. The boundary then reads as usage climbing from zero, which reports whatever the new period has already passed and cannot read as a recovery.

Alerts counting lifetime usage are excluded, because that figure accumulates across periods by design.

Excluding them by type is not enough on its own: a recurring billable metric aggregates from the start of the subscription rather than from the period, so its usage does not restart either even though the alert watches current usage. Rather than enumerate which metrics behave that way, the baseline is only cleared when the measured value actually fell, which is what a genuine restart looks like. Usage that carried across the boundary keeps its baseline and is compared normally, so nothing is re-reported. Tests cover two consecutive boundaries for both cases.

@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from 0507afc to e0df1b5 Compare August 12, 2026 20:01
@lago-claude-ai-agent

Copy link
Copy Markdown
Contributor

Automated pre-review (advisory, not a required check) — verdict: HOLD · CI green

HOLD — period rollover is inferred from a value decrease, so a restarted period whose first observed usage is equal to or above the prior value is not fully re-armed. For example, with a prior value of 15 and thresholds at 10/20, a new-period value of 25 reports only 20 instead of both thresholds. Distinguish resetting usage from carry-over usage without relying on current < previous_value, and cover the non-decreasing rollover case while preserving recurring-metric carry-over behavior.

@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from e0df1b5 to 5c59654 Compare August 19, 2026 15:31
@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from 5c59654 to 48894c8 Compare August 19, 2026 16:43
@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from 48894c8 to d5fa801 Compare August 19, 2026 17:18
@lago-claude-ai-agent

Copy link
Copy Markdown
Contributor

Automated pre-review (advisory, not a required check) — verdict: HOLD · CI green

HOLD: period rollover is inferred from current < previous_value, which misses a reset when the new period's first observed usage already equals or exceeds the prior value, leaving already-crossed thresholds disarmed.

The same heuristic misclassifies a cumulative recurring metric that legitimately decreases after the boundary and can re-fire thresholds as though it reset. Determine rollover from the monitored metric's reset semantics and add tests for both cases.

@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch 2 times, most recently from 8d43e68 to 6c8b131 Compare August 19, 2026 20:39
@lago-claude-ai-agent

Copy link
Copy Markdown
Contributor

Automated pre-review (advisory, not a required check) — verdict: HOLD · CI green

HOLD: Recovery notifications are skipped at every period boundary, even when window_resets_each_period? is false. Preserve normal recovery handling for recurring metrics whose window is continuous, and add a boundary test where such a metric drops below a resolvable threshold.

@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from 6c8b131 to c14163a Compare August 19, 2026 20:53
@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from c14163a to 8bb3d5f Compare August 19, 2026 20:57
@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from 8bb3d5f to 91a0c8e Compare August 20, 2026 12:15
@lago-claude-ai-agent

Copy link
Copy Markdown
Contributor

Automated pre-review (advisory, not a required check) — verdict: HOLD · CI green

HOLD — the rollover behavior is not covered for non-recurring billable-metric alerts.

  • Add previous-period specs for non-recurring billable_metric_current_usage_amount and billable_metric_current_usage_units alerts that prove they reset the baseline and re-fire crossed thresholds. The current recurring-amount example would still pass if both subclass overrides were removed, and the units path is never exercised.

@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from 91a0c8e to 25a0d7c Compare August 20, 2026 12:40
@lago-claude-ai-agent

Copy link
Copy Markdown
Contributor

Automated pre-review (advisory, not a required check) — verdict: PASS · CI green

PASS — Period rebasing is correctly limited to resetting current-usage windows, preserves recurring and lifetime baselines, suppresses false boundary recoveries, and has focused coverage for each affected alert type.

The targeted spec could not be executed here because the required lago CLI is unavailable.

@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from 25a0d7c to 798ba14 Compare August 20, 2026 12:57
@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from 798ba14 to 641a422 Compare August 20, 2026 13:28
@lago-claude-ai-agent

Copy link
Copy Markdown
Contributor

Automated pre-review (advisory, not a required check) — verdict: PASS · CI green

PASS — Period rollover now re-baselines only period-scoped current-usage alerts, while preserving recurring and lifetime windows and avoiding false recovery records. Coverage exercises total and per-metric amount/units alerts, recurring metrics, threshold refiring, and recovery behavior. The targeted spec could not be run locally because the required lago CLI is unavailable.

@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from 641a422 to 601d27f Compare August 31, 2026 14:29
@lago-claude-ai-agent

Copy link
Copy Markdown
Contributor

Automated pre-review (advisory, not a required check) — verdict: PASS · CI green

PASS — The rollover rebaseline is scoped to resettable current-usage windows, preserves recurring and lifetime baselines, and focused specs cover threshold refiring and recovery behavior.

@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from 601d27f to 9702465 Compare September 9, 2026 13:49
@lago-claude-ai-agent

Copy link
Copy Markdown
Contributor

Automated pre-review (advisory, not a required check) — verdict: HOLD · CI green

HOLD — A billable-metric alert can permanently miss the renewal rebaseline when the metric is absent from the plan on the first evaluation. call skips evaluate for a nil value but still advances last_processed_at; if a matching charge appears later in that period, the stale prior-period previous_value is retained and newly crossed thresholds can be missed. Preserve/apply the pending rebaseline in this path and add a regression covering absent-at-boundary then present.

@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from 9702465 to 491d1c0 Compare September 14, 2026 15:32
@lago-claude-ai-agent

Copy link
Copy Markdown
Contributor

Automated pre-review (advisory, not a required check) — verdict: PASS · CI green

PASS — The period rollover logic is correctly scoped to current-usage alerts, preserves continuous recurring-metric baselines, suppresses false boundary recoveries, and has focused coverage across total, per-metric, units, recurring, and lifetime cases.

@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from 491d1c0 to 6a5c5b3 Compare September 16, 2026 12:59
@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from 6a5c5b3 to 08d6c19 Compare September 16, 2026 13:24
@lago-claude-ai-agent

Copy link
Copy Markdown
Contributor

Automated pre-review (advisory, not a required check) — verdict: PASS · CI green

PASS — The period rollover logic is consistent across current-usage alert variants, preserves recurring and lifetime windows, and has focused coverage for re-triggering, resolution suppression, and recurring-metric exceptions.

@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from 08d6c19 to b6d5bac Compare September 16, 2026 13:34
@lago-claude-ai-agent

Copy link
Copy Markdown
Contributor

Automated pre-review (advisory, not a required check) — verdict: PASS · CI green

PASS — Period rollover handling is consistent across all current-usage alert types, preserves recurring and lifetime baselines, and avoids false recovery events. The added specs cover reset and non-reset paths for total amount and metric-specific amount/units alerts.

@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from b6d5bac to 3932c6a Compare September 16, 2026 14:29
@lago-claude-ai-agent

Copy link
Copy Markdown
Contributor

Automated pre-review (advisory, not a required check) — verdict: HOLD · CI green

HOLD: Mixed recurring/non-recurring plans are mishandled because one recurring charge disables rollover rebasing for the entire current-usage total; the non-recurring portion can reset below a threshold and cross it again without an alert.

Handle the mixed-plan baseline correctly and add a regression test containing both recurring and non-recurring charges.

@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from 3932c6a to cd04ca4 Compare September 17, 2026 13:37
@lago-claude-ai-agent

Copy link
Copy Markdown
Contributor

Automated pre-review (advisory, not a required check) — verdict: HOLD · CI green

HOLD — a billable-metric alert can consume the period rollover without resetting its baseline.

When find_value returns nil because the metric is absent from the plan, ProcessAlertService#call still advances last_processed_at but never runs evaluate. If that metric is attached later in the same period, period_rolled_over? is then false, so the old period's previous_value can suppress a valid trigger or emit a false recovery. Preserve the pending rollover (or reset before advancing it) and add a regression spec for absence at the boundary followed by attachment in the same period.

@aquinofb
aquinofb requested review from mariohd and toommz September 17, 2026 17:35
@aquinofb
aquinofb force-pushed the ing-508-renewal-baseline branch from cd04ca4 to df85049 Compare September 23, 2026 15:27
@lago-claude-ai-agent

Copy link
Copy Markdown
Contributor

Automated pre-review (advisory, not a required check) — verdict: PASS · CI green

PASS — The rollover logic re-baselines period-scoped current-usage alerts while preserving recurring and lifetime baselines, and it avoids emitting a false recovery at the boundary. The specs cover total amount, metric amount and units, recurring metrics, and lifetime usage; no blocking sibling-site or safety issue is apparent.

@groyoh
groyoh added this pull request to stack #6613 October 8, 2026 07:41
At renewal, usage starts again from nothing. To an alert that compares
against the last value it saw, that looks like every threshold clearing
at once, which would announce a recovery for every affected customer at
the same moment.

The same comparison hides a second problem that exists today. An
evaluation early in a new period compares the small new usage against
the large figure carried over from the last one, finds nothing crossed,
and stores the small figure. A threshold the customer really did cross in
those first moments is never reported. Product approved fixing this in
the knowledge that previously silent alerts will start firing.

When the last evaluation belongs to an earlier period, the comparison
starts from nothing instead. The boundary then reads as usage climbing
from zero, which reports whatever the new period has already passed and
cannot read as a recovery. So the storm never happens, and the alerts
that used to be swallowed arrive.

This applies only to alerts that watch usage within a period. Alerts
counting lifetime usage are deliberately excluded: that figure
accumulates across periods by design, so starting from nothing would
recompute the whole history as newly crossed and re-report every
threshold at every boundary, forever, for exactly the customers using
them. They keep comparing against the value they stored, which is
correct for them, and the swallowed-alert problem does not arise there
either. A test covers two consecutive boundaries to hold that line.

Excluding lifetime alerts by type is not enough on its own, because a
recurring billable metric aggregates from the start of the subscription
rather than from the period, so its usage does not restart either even
though the alert watches current usage. Rather than enumerate which
metrics behave that way, the baseline is only cleared when the measured
value actually fell, which is what a genuine restart looks like. Usage
that carried across the boundary keeps its baseline and is compared
normally, so nothing is re-reported. A test covers two consecutive
boundaries for that case too.
@groyoh
groyoh force-pushed the ing-508-renewal-baseline branch from df85049 to db48b4f Compare October 9, 2026 21:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants