Skip to content

fix(gitlab): Make project webhook creation retry-safe - #129175

Open
giovanni-guidini wants to merge 1 commit into
masterfrom
gio/fix/gitlab-webhook-create-retries
Open

giovanni-guidini wants to merge 1 commit into
masterfrom
gio/fix/gitlab-webhook-create-retries

Conversation

@giovanni-guidini

@giovanni-guidini giovanni-guidini commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

GitLab project webhook reconciliation now persists an opaque creation identity before POST, paginates project hooks to adopt an exact prior attempt, and serializes work per repository. This makes the asynchronous ensure task safe after ambiguous create responses while retaining the existing rule that only a definite 404 permits replacement; verified duplicate cleanup and synchronous repository lifecycle reconciliation remain follow-up work.

Before

missing/404 ──► POST hook ──► response lost
                                │
                                ▼ retry
                             POST again ──► duplicate

This PR

reload repository under lock
    │
    ├── integration changed ──► halt stale task
    │
    └── expected integration
            │
            ▼
        missing/404
    │
    ▼
persist reconciliation ID
    │
    ▼
list every hook page for exact marked URL
    │
    ├── match found ──► adopt + update
    │
    └── no match ─────► POST marked hook
                           │
                           └── response lost ──► retry lists and adopts it
    │
    └── 1,000-hook cap reached ──► halt without POST
    │
    ▼
CAS-store winning hook ID
    │
    ├── success ──► done
    └── race lost ► halt without replacing concurrent repository state

@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Oct 29, 2026
Comment thread src/sentry/integrations/gitlab/tasks.py Resolved
@giovanni-guidini
giovanni-guidini force-pushed the gio/fix/gitlab-webhook-create-retries branch 2 times, most recently from f471665 to f4a944d Compare October 29, 2026 13:22
@giovanni-guidini
giovanni-guidini force-pushed the gio/fix/gitlab-webhook-create-retries branch from f4a944d to f30bde7 Compare October 29, 2026 13:52
@giovanni-guidini
giovanni-guidini marked this pull request as ready for review October 29, 2026 13:56
@giovanni-guidini
giovanni-guidini requested review from a team as code owners October 29, 2026 13:56
@giovanni-guidini
giovanni-guidini force-pushed the gio/fix/gitlab-webhook-create-retries branch 2 times, most recently from e22d792 to e0d234c Compare October 29, 2026 15:02

@cursor cursor Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread src/sentry/integrations/gitlab/tasks.py Resolved
@giovanni-guidini
giovanni-guidini force-pushed the gio/fix/gitlab-webhook-create-retries branch from e0d234c to 64988f4 Compare October 29, 2026 15:40

@billyvg billyvg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI-assisted review, posted on behalf of @billyvg.

Comment thread src/sentry/integrations/gitlab/tasks.py Resolved
Comment thread src/sentry/integrations/gitlab/tasks.py Resolved
Persist a reconciliation identity before creating project hooks so retries can list and adopt the original hook instead of creating duplicates. Paginate hook discovery and serialize reconciliation per repository.
@giovanni-guidini
giovanni-guidini force-pushed the gio/fix/gitlab-webhook-create-retries branch from 64988f4 to 85778f9 Compare November 2, 2026 08:50
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Backend Test Failures

Failures on 8ef0069 in this run:

tests/sentry/notifications/notifications/test_digests.py::DigestNotificationTest::test_digest_email_uses_user_timezone — log
[gw1] linux -- Python 3.13.17 /home/runner/work/sentry/sentry/.venv/bin/python3
tests/sentry/notifications/notifications/test_digests.py:252: in test_digest_email_uses_user_timezone
    assert "PST" in pacific_body or "PDT" in pacific_body, (
E   AssertionError: Pacific email should use Pacific timezone, but body was: 
E     
E     Notifications for bar
E     Nov. 1, 2026, 1:53 a.m.  to Nov. 1, 2026, 1:53 a.m. 
E     
E     Lenient Bedbug
E     
E     * oh no (1 event, 0 users)
E       http://testserver/organizations/baz/issues/21/?referrer=digest_email
E     
E     * oh no (1 event, 0 users)
E       http://testserver/organizations/baz/issues/20/?referrer=digest_email
E     
E     
E     
E     Unsubscribe: http://testserver/unsubscribe/baz/project/4559109355143200/?_=1V:1xNnnX:RTb7Ln21pUMQ74-OtV03osSRCfNVPS_HuRDRNbLxgb4&referrer=alert_digest
E     
E   assert ('PST' in '\n\nNotifications for bar\nNov. 1, 2026, 1:53 a.m.  to Nov. 1, 2026, 1:53 a.m. \n\nLenient Bedbug\n\n* oh no (1 event...ribe/baz/project/4559109355143200/?_=1V:1xNnnX:RTb7Ln21pUMQ74-OtV03osSRCfNVPS_HuRDRNbLxgb4&referrer=alert_digest\n' or 'PDT' in '\n\nNotifications for bar\nNov. 1, 2026, 1:53 a.m.  to Nov. 1, 2026, 1:53 a.m. \n\nLenient Bedbug\n\n* oh no (1 event...ribe/baz/project/4559109355143200/?_=1V:1xNnnX:RTb7Ln21pUMQ74-OtV03osSRCfNVPS_HuRDRNbLxgb4&referrer=alert_digest\n')

@giovanni-guidini
giovanni-guidini requested review from a team and billyvg November 2, 2026 08:57

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 85778f9. Configure here.

hook_id,
reconciliation_id=new_reconciliation_id,
)
outcome = "adopted"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adopt 404 blocks hook replacement

Medium Severity

A 404 from the adopt update_project_webhook is not handled locally, so it falls through as a terminal not_found failure. That skips both a replacement create and a retry. The stored-id path treats a definite 404 as the signal that it is safe to create; after an ambiguous prior create the listed hook can disappear before the follow-up PUT, leaving the repository without a webhook until something else retriggers reconciliation.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 85778f9. Configure here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants