Skip to content

feat(had): cluster-robust event-study inference + clustered sup-t band#612

Merged
igerber merged 1 commit into
mainfrom
feature/had-event-study-cluster-band
Jul 4, 2026
Merged

feat(had): cluster-robust event-study inference + clustered sup-t band#612
igerber merged 1 commit into
mainfrom
feature/had-event-study-cluster-band

Conversation

@igerber

@igerber igerber commented Jul 4, 2026

Copy link
Copy Markdown
Owner

Summary

  • HeterogeneousAdoptionDiD.fit(aggregate="event_study") now honors cluster= end-to-end on both designs (continuous CCT local-linear + mass-point 2SLS): cluster-robust per-horizon pointwise CIs and a cluster-robust simultaneous sup-t confidence band. Previously cluster= was silently ignored on the nonparametric event-study path (with a UserWarning), and the weighted mass-point cband=True case raised NotImplementedError.
  • New clustered branch in _sup_t_multiplier_bootstrap: aggregates the per-unit influence function to cluster level and draws cluster-level Rademacher multipliers, so the perturbation variance is the raw cluster sandwich Σ_c s_c². This reconciles to each path's analytical cluster-robust SE via a path scalar — 1.0 for continuous (the lprobust cluster meat carries no g/(g-1) correction) and √(G/(G-1)) for mass-point (restoring the CR1 finite-sample factor the returned IF omits).
  • cluster= + survey= is rejected on both designs (route clustering through survey_design=SurveyDesign(psu=<col>)). Single cluster → NaN band + RuntimeWarning. Unclustered fits are byte-unchanged.

Methodology references (required if estimator / math changes)

  • Method name(s): HeterogeneousAdoptionDiD event-study inference (cluster-robust CCT / 2SLS SEs + clustered sup-t simultaneous band)
  • Paper / source link(s): de Chaisemartin, Ciccia, D'Haultfœuille & Knau (2026), arXiv:2405.04465v6 (HAD event-study, Appendix B.2). The clustered simultaneous band is a documented library extension (no core-paper derivation / R anchor exists for it).
  • Any intentional deviations from the source (and why): The clustered event-study sup-t band has no reference-package anchor; documented in docs/methodology/REGISTRY.md ("Note (HAD clustered event-study sup-t band)") with the variance-family reconciliation. The reconciliation identity is the validation: sqrt(Σ_c (scale·s_c)²) == se to atol=1e-10 on the real influence function for both paths, plus the H=1 → 1.96 sup-t reduction. cluster= + survey= rejected (Binder-TSL would override the cluster-robust SE).

Validation

  • Tests added/updated: tests/test_had.py — new TestEventStudyClusterBand (deterministic IF/SE reconciliation for both paths, end-to-end continuous + weighted mass-point clustered bands, cluster=+survey= rejection, single-cluster NaN, determinism); clustered variant + single-cluster case added to TestSupTReducesToNormalAtH1; two former mass-point rejection tests converted to assert the now-supported behavior; the stale "cluster ignored" test flipped to assert cluster-robust threading. Full test_had.py suite: 393 passed / 2 skipped.
  • Backtest / simulation / notebook evidence (if applicable): N/A (no tutorial changes; opt-in inference feature)

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown

Overall Assessment
✅ Looks good. No unmitigated P0/P1 findings found.

Executive Summary

  • Affected method: HeterogeneousAdoptionDiD.fit(aggregate="event_study"), for continuous CCT and mass-point 2SLS event-study inference.
  • The clustered sup-t band is a documented library extension in docs/methodology/REGISTRY.md:L3026, so the no-R-anchor deviation is mitigated.
  • The new pointwise inference path still uses safe_inference() at diff_diff/had.py:L4685-L4693; no inline inference or partial NaN guard defect found.
  • cluster= + survey_design= is rejected before variance-family mixing at diff_diff/had.py:L4425-L4435.
  • Remaining issues are documentation/test polish, not blockers.

Methodology

  • Severity: P3 informational
    Impact: The clustered event-study sup-t band is not derived in the cited HAD paper/R anchor, but the deviation is explicitly documented with the raw cluster sandwich and path scalars in docs/methodology/REGISTRY.md:L3026. Per review rules, this is not a defect.
    Concrete fix: No action required.

Code Quality

  • Severity: P3
    Impact: Single-cluster clustered event-study fits warn that a “NaN band” is returned, but _fit_event_study leaves cband_crit_value, cband_method, and cband_n_bootstrap as None at diff_diff/had.py:L4716-L4725. That is not misleading statistical output, but it makes “undefined band” look the same as “band skipped” unless the caller catches the warning.
    Concrete fix: Either set cband_crit_value=np.nan, cband_method="cluster_multiplier_bootstrap", and cband_n_bootstrap=n_bootstrap_eff in the _cl_G < 2 branch, or change the warning/docs/tests to say no band fields are populated.

Performance

  • No findings. The added cluster branch streams multiplier blocks and does not introduce an obvious avoidable full (B x G) allocation.

Maintainability

  • Severity: P3
    Impact: A few in-code comments/docstrings still describe the old weighted-only band behavior, e.g. diff_diff/had.py:L4125-L4126, diff_diff/had.py:L4695, and diff_diff/had.py:L778-L780. This can mislead future maintainers around the now-supported unweighted clustered band.
    Concrete fix: Update those phrases to “unweighted, unclustered” or “weighted/survey or clustered” as appropriate.

Tech Debt

  • No findings beyond the minor documentation drift above. The removed TODO entry is consistent with the newly documented implementation.

Security

  • No findings. No secrets or security-sensitive changes identified in the diff.

Documentation/Tests

  • Severity: P2
    Impact: REGISTRY.md now contains contradictory methodology statements: the new clustered band is documented at docs/methodology/REGISTRY.md:L3016-L3026 and checked off at L3213, but older notes still say sup-t only runs for weighted/survey fits and that event-study cluster= remains a follow-up/ignored path at L3180 and L3207. Since the registry is the methodology source of truth, this should be cleaned up.
    Concrete fix: Revise docs/methodology/REGISTRY.md:L3180 and L3207 to include cluster= and remove the obsolete “ignored/follow-up” wording.

  • Severity: P3
    Impact: Tests cover continuous unweighted clustered band and weighted mass-point clustered band, plus deterministic IF reconciliation, but I did not see end-to-end coverage for weighted continuous clustered event-study or unweighted mass-point clustered event-study.
    Concrete fix: Add one end-to-end test for design="continuous_at_zero", weights=..., cluster=..., cband=True, and one for design="mass_point", unweighted cluster=..., cband=True.

@igerber igerber force-pushed the feature/had-event-study-cluster-band branch from 773e1fe to f94625b Compare July 4, 2026 12:51
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: f94625b97ad1df36f25a3a9b4ef3a3a3c1ec55ed


Overall Assessment

✅ Looks good. No unmitigated P0/P1 findings found.

Executive Summary

  • Affected method: HeterogeneousAdoptionDiD.fit(aggregate="event_study"), continuous CCT and mass-point 2SLS paths with cluster=.
  • The clustered sup-t band is a documented library extension, not a paper/R-anchor mismatch, per docs/methodology/REGISTRY.md:L3024-L3026.
  • Prior registry contradictions appear resolved at docs/methodology/REGISTRY.md:L3180 and L3213.
  • Prior test gaps are addressed with weighted continuous and unweighted mass-point clustered-band coverage in tests/test_had.py:L4923-L4965.
  • No inline inference or partial NaN guard issue found; per-horizon inference still routes through safe_inference() at diff_diff/had.py:L4689-L4697.
  • Remaining issue is minor documentation/comment drift only.

Methodology

  • Severity: P3 informational
    Impact: The clustered event-study sup-t band has no core-paper/R reference anchor, but this is explicitly documented as a library extension with the raw cluster-sandwich reconciliation and path scalars in docs/methodology/REGISTRY.md:L3024-L3026. Per review rules, this is not a defect.
    Concrete fix: No action required.

Code Quality

No findings.

Performance

No findings. The clustered branch draws multiplier blocks and stores only (n_bootstrap, n_horizons) perturbations, not a full (n_bootstrap, n_clusters) matrix.

Maintainability

  • Severity: P3
    Impact: A few comments/docstrings still imply all cband metadata is None on unweighted fits or that the band block is “weighted + cband only,” but unweighted cluster= now populates clustered band metadata. See diff_diff/had.py:L745-L780 and diff_diff/had.py:L4699. This is not a behavioral defect, but it can mislead future maintainers.
    Concrete fix: Update those comments to say “unweighted, unclustered” and “weighted/survey or clustered cband.”

Tech Debt

No findings. The former event-study cluster= deferral was removed from TODO.md, and remaining joint covariance work is still tracked as non-blocking.

Security

No findings.

Documentation/Tests

No blocking findings. The previous registry drift is cleaned up, and the added tests cover the previously missing weighted continuous and unweighted mass-point clustered event-study paths.

Review note: I did not run the test suite locally because the environment failed on missing numpy; this assessment is based on the diff and source inspection.

@igerber igerber force-pushed the feature/had-event-study-cluster-band branch from f94625b to 1ec33ff Compare July 4, 2026 12:56
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 1ec33ffdb3d3d6c3d51736e8f681a0467870ef9b


Overall Assessment

✅ Looks good. No unmitigated P0/P1 findings.

Executive Summary

  • Affected method: HeterogeneousAdoptionDiD.fit(aggregate="event_study") for continuous CCT and mass-point 2SLS paths with cluster=.
  • The clustered sup-t band is a documented library extension, not an undocumented paper/R mismatch: docs/methodology/REGISTRY.md:L3024-L3026.
  • Pointwise inference still routes through safe_inference(): diff_diff/had.py:L4693-L4701.
  • The clustered bootstrap branch reconciles the IF scale to analytical SEs as documented: diff_diff/had.py:L4710-L4758.
  • Prior P1/P0 concerns appear resolved; only minor stale guide/comment text remains.
  • I could not run tests locally because pytest is not installed.

Methodology

  • Severity: P3 informational
    Impact: The clustered event-study sup-t band has no core-paper/R anchor, but the deviation is explicitly documented with the variance-family reconciliation: continuous scale 1.0, mass-point sqrt(G/(G-1)). Per review rules, this is not a defect.
    Concrete fix: No action required.

Code Quality

No findings.

Performance

No findings. The clustered branch aggregates IFs to cluster level and stores bootstrap perturbations as (n_bootstrap, n_horizons), not a full draw-by-unit matrix: diff_diff/had.py:L2175-L2181.

Maintainability

  • Severity: P3
    Impact: One to_dict() comment still says weighted/survey cband fields are “None on unweighted fits,” but unweighted clustered event-study now populates cband metadata. This is comment drift only.
    Concrete fix: Update diff_diff/had.py:L913-L924 to say cband fields may be populated on unweighted clustered fits.

Tech Debt

No findings. The prior TODO deferral for HAD event-study cluster= threading was removed, and remaining joint covariance work stays tracked.

Security

No findings.

Documentation/Tests

  • Severity: P3
    Impact: The LLM guide still says mass-point weights= + aggregate="event_study" + cband=True requires HC1, without the new cluster= exception where the path resolves to CR1 and is supported. This can mislead users, but does not affect runtime correctness.
    Concrete fix: Update diff_diff/guides/llms-full.txt:L802 to add “unless cluster= is supplied; clustered fits resolve to CR1 and use the clustered sup-t band.”

Local verification: attempted targeted pytest, but the command is unavailable in this environment.

@igerber igerber force-pushed the feature/had-event-study-cluster-band branch from 1ec33ff to f725991 Compare July 4, 2026 13:02
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: f725991f6c0c5cb5e7c0f511c227746a6fdf3a1e


Overall Assessment

✅ Looks good. No unmitigated P0/P1 findings.

Executive Summary

  • Affected method: HeterogeneousAdoptionDiD.fit(aggregate="event_study") with cluster=, for continuous CCT and mass-point 2SLS paths.
  • The clustered sup-t band is a documented library extension, not an undocumented methodology deviation: docs/methodology/REGISTRY.md:L3024-L3026.
  • Pointwise inference still routes through safe_inference(), and cband endpoints are NaN-gated on finite positive SEs: diff_diff/had.py:L4696-L4704, diff_diff/had.py:L2282-L2321.
  • Cluster IF aggregation and mass-point CR1 scaling match the registry contract: diff_diff/had.py:L2160-L2184, diff_diff/had.py:L4713-L4761.
  • Previous stale API/LLM-guide concerns appear addressed; only minor non-blocking prose drift remains.
  • I could not run tests here: pytest is unavailable, and even import smoke failed because numpy is not installed.

Methodology

  • Severity: P3 informational
    Impact: The clustered event-study sup-t band has no core-paper or R reference anchor, but the registry explicitly documents it as a library extension with the variance-family reconciliation: continuous scale 1.0, mass-point sqrt(G/(G-1)). Per review rules, this is not a defect.
    Concrete fix: No action required.

Code Quality

No findings.

Performance

No findings. The clustered bootstrap aggregates IFs to cluster level and stores only (n_bootstrap, n_horizons) perturbations, avoiding a draw-by-unit matrix: diff_diff/had.py:L2178-L2184.

Maintainability

No findings.

Tech Debt

No findings. The old HAD event-study cluster= deferral was removed, and remaining joint covariance work stays tracked in TODO.md:L127.

Security

No findings.

Documentation/Tests

  • Severity: P3
    Impact: Minor stale prose remains: the registry heading at docs/methodology/REGISTRY.md:L3006 still describes the sup-t bootstrap as “on the weighted event-study path,” and the test class docstring at tests/test_had.py:L3279-L3280 still mentions “cluster ignored.” Nearby text and behavior are correct, so this is not a user-facing correctness issue.
    Concrete fix: Update those phrases to “weighted/survey or clustered event-study path” and remove cluster from the ignored-warning test class description.

HeterogeneousAdoptionDiD.fit(aggregate="event_study") now honors cluster=
end-to-end on both designs: cluster-robust per-horizon pointwise CIs (continuous
CCT + mass-point 2SLS) AND a cluster-robust simultaneous sup-t confidence band.
Previously cluster= was ignored on the nonparametric event-study path (with a
UserWarning) and the weighted mass-point cband=True case raised NotImplementedError.

The clustered sup-t band adds a dedicated branch to _sup_t_multiplier_bootstrap:
it aggregates the per-unit influence function to cluster level and draws
cluster-level Rademacher multipliers, so the perturbation variance is the raw
cluster sandwich sum_c s_c^2. This reconciles to each path's analytical
cluster-robust SE via a path scalar: 1.0 for continuous (the lprobust cluster
meat carries no g/(g-1) correction) and sqrt(G/(G-1)) for mass-point (restoring
the CR1 finite-sample factor the returned IF omits). Validated bootstrap-free:
sqrt(sum_c (scale*s_c)^2) == se to atol=1e-10 on the real IF for both paths, plus
the H=1 -> 1.96 reduction on the clustered branch.

cluster= + survey= is rejected (both designs) - route clustering through
survey_design=SurveyDesign(psu=<col>). Single cluster -> NaN band + RuntimeWarning.
Unclustered fits are byte-unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LHDijzf8zHXk5T8ahS2mKi
@igerber igerber force-pushed the feature/had-event-study-cluster-band branch from f725991 to c436dc2 Compare July 4, 2026 13:07
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: c436dc2326627edd4318b13d78bcd32672e9ed21


Overall Assessment

✅ Looks good. No unmitigated P0/P1 findings.

Executive Summary

  • Affected method: HeterogeneousAdoptionDiD.fit(aggregate="event_study") with cluster=, for continuous CCT and mass-point 2SLS paths.
  • The clustered sup-t band is a documented library extension, not an undocumented methodology deviation: docs/methodology/REGISTRY.md:L3024-L3026.
  • The implementation matches the registry contract: cluster-level IF aggregation in _sup_t_multiplier_bootstrap, continuous scale 1.0, and mass-point sqrt(G/(G-1)): diff_diff/had.py:L2160-L2184, diff_diff/had.py:L4713-L4758.
  • Edge cases are covered: cluster= + survey= rejects, single-cluster bands return NaN with warning, and pointwise inference stays routed through safe_inference(): diff_diff/had.py:L4433-L4444, diff_diff/had.py:L4728-L4743, diff_diff/had.py:L4696-L4704.
  • Prior P3 prose drift appears addressed; the registry heading and warning test docstring now reflect clustered support.

Methodology

  • Severity: P3 informational
    Impact: The clustered event-study sup-t band has no core-paper or R-package anchor, but this is explicitly documented as a library extension with the variance-family reconciliation and validation identities. Under the review rules, this is not a defect.
    Concrete fix: No action required.

Code Quality

No findings.

Performance

No findings. The clustered bootstrap aggregates IFs to cluster level and generates (n_bootstrap, n_horizons) perturbations without materializing a draw-by-unit matrix: diff_diff/had.py:L2178-L2184.

Maintainability

No findings.

Tech Debt

No findings. The old HAD event-study cluster= deferral was removed from TODO.md; remaining cross-horizon covariance work stays tracked as low-priority deferred work: TODO.md:L126-L127.

Security

No findings.

Documentation/Tests

  • Severity: P3 informational
    Impact: I could not independently run tests in this environment because pytest is not installed (/usr/bin/python: No module named pytest). The added tests do cover deterministic IF/SE reconciliation, clustered H=1 reduction, supported weighted mass-point clustered cband, cluster= + survey= rejection, single-cluster NaN behavior, and determinism: tests/test_had.py:L4772-L5005.
    Concrete fix: No PR change required; rely on CI for the full tests/test_had.py run.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Jul 4, 2026
@igerber igerber merged commit d7a6a53 into main Jul 4, 2026
33 of 34 checks passed
@igerber igerber deleted the feature/had-event-study-cluster-band branch July 4, 2026 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant