Skip to content

Expose per-operation idempotency in the OpenAPI schema (e.g. x-sentry-idempotent) #128927

Open

Description

@sentry-junior

API clients like the Sentry CLI and MCP need to know which POST/PUT/DELETE operations are safe to replay on transient failures (5xx, timeouts). Today that knowledge can't come from the API, so each client hard-codes it per call site. That drifts, and it can't be shared through @sentry/api.

Context: getsentry/toolkit#1677 (review thread: getsentry/toolkit#1677 (comment)). The CLI there only retries non-safe methods on 5xx when a call opts in with idempotent: true, for example DIF/ProGuard assemble (state is keyed by checksum, so resending only polls) and preprod build assemble.

Current state

  • @sentry/api (generated from getsentry/sentry-api-schema) has no machine-readable idempotency. At most it has prose in a docstring.
  • The assemble endpoints clients replay aren't in the public schema:
    • DifAssembleEndpoint (src/sentry/api/endpoints/debug_files.py): POST is ApiPublishStatus.PRIVATE.
    • OrganizationArtifactBundleAssembleEndpoint (src/sentry/api/endpoints/organization_artifactbundle_assemble.py): POST is PRIVATE.
    • ProjectPreprodArtifactAssembleEndpoint (src/sentry/preprod/api/endpoints/organization_preprod_artifact_assemble.py): POST is EXPERIMENTAL.
  • There's precedent for vendor extensions: custom_postprocessing_hook in src/sentry/apidocs/hooks.py already stamps x-sentry-routing and x-sentry-experimental on operations.

Proposal

  1. Let endpoints declare per-method idempotency, e.g. a class attribute next to publish_status (idempotent_methods = {"POST"}) or an extend_schema option.
  2. Emit it from custom_postprocessing_hook as an operation extension, e.g. x-sentry-idempotent: true. Leave it out when false.
  3. Mark known-safe operations, starting with the assemble endpoints above. Clients still need those operations in the schema they consume; either publish them or emit the extension for private/experimental operations into the schema @sentry/api builds from.
  4. Ensure getsentry/sentry-api-schema and @sentry/api preserve the flag so clients can read it instead of hard-coding behavior.

Open questions

  • Name and shape: a boolean x-sentry-idempotent, or a richer x-sentry-retry object that leaves room for retry hints?
  • Should private/experimental operations be included in the schema consumed by @sentry/api, or should this ship only once they are public?
  • Should Idempotency-Key support be in scope for endpoints that aren't naturally idempotent, or tracked separately?

via BYK.

--

View Junior Session [Sentry]

Activity

  1. linear-code commented on Oct 27, 2026

    @linear-code
  2. sentry-junior commented on Oct 28, 2026

    @sentry-junior
    ContributorAuthor

    Status: done end to end, waiting only on the CLI adoption PR.

    Left as is:

    • Preprod assemble stays EXPERIMENTAL and unmarked, on purpose: a resend after the upload stage can create a second artifact. The CLI keeps its own explicit opt-in for that poll loop.
    • Shape: we shipped a boolean x-sentry-idempotent. A richer x-sentry-retry object wasn't needed.
    • Idempotency-Key support for writes that aren't naturally idempotent is out of scope here. It needs its own issue if someone wants it.

    I'll close this once getsentry/toolkit#1706 merges.

  3. sentry-junior commented on Oct 29, 2026

    @sentry-junior
    ContributorAuthor

    Done: getsentry/toolkit#1706 merged, and toolkit main is on @sentry/api 0.423.0, so the CLI now retries the writes the schema marks idempotent. Preprod assemble and Idempotency-Key support stay out of scope, as noted above.

  4. sentry-junior commented on Oct 29, 2026

    @sentry-junior
    ContributorAuthor

    Audit of public POST operations

    Not done yet: #128999 marked only the 3 writes the CLI already retried. There are 122 published non-safe operations. This covers the POSTs that look like upserts or get-or-create, focusing on the ones the CLI and MCP call. The PUT/PATCH/DELETE audit follows in a separate comment.

    Safe to mark as-is

    • bulkUpsertOrganizationCodeMappings (integrations/api/endpoints/organization_code_mappings_bulk.py): upserts by project, stack_root, and source_root, and ProjectRepository is get_or_create. A resend flips created to updated in the response but leaves the same state. The CLI uses it.
    • linkProjectRepository (api/endpoints/project_repo.py): returns 200 if the link already exists, 201 if it's new.
    • addOrganizationMemberTeam (core/endpoints/organization_member_team_details.py): returns 204 if the member is already on the team. The access-request path is get_or_create and doesn't resend the email.
    • "Create or update an External Issue" (sentry_apps/api/endpoints/installation_external_issues.py): PlatformExternalIssue upsert.

    Safe with a small fix or caveat

    • addProjectTeam (core/endpoints/project_team_details.py): Project.add_team swallows the IntegrityError, but PROJECT_TEAM_ADD is audited on every call. Only audit when add_team() returns True.
    • createOrganizationRelease (api/endpoints/organization_releases.py) and updateOrganizationRelease (releases/endpoints/organization_release_details.py): the release is get_or_create and returns 208 when nothing changed. ReleaseProject and release activity are only created once (was_released guard on PUT). But a resend with refs calls set_refs(..., fetch=True) again, which re-enqueues fetch_commits. Commits are get_or_create and the head commit is update_or_create, so the state converges and the cost is duplicate work. These are the most valuable ones, because the CLI calls both.

    Not safe to resend

    • createOrganizationReleaseDeploy: creates a new Deploy every time.
    • uploadOrganizationReleaseFile / uploadProjectReleaseFile: a resend returns 409.
    • starOrganizationServiceSpan: get_or_create, but a resend returns 403.
    • Plain creates (projects, teams, dashboards, monitors, workflows, detectors, keys, hooks, …) and preprod assemble are tracked in Support Idempotency-Key for writes that aren't naturally idempotent #129168 (Idempotency-Key).
  5. sentry-junior commented on Oct 29, 2026

    @sentry-junior
    ContributorAuthor

    Audit of public PUT / PATCH / DELETE operations

    This covers all 77 remaining published non-safe operations, read at master af697a63. A write counts as safe when resending the identical request, after the first one already succeeded, leaves the same state, adds no audit entries, Activity, emails, webhooks, rows or non-converging tasks, and doesn't return an error. This is from reading the code; nothing was run.

    PUT / PATCH (37)

    Safe as-is (14): updateOrganizationCodeMapping, updateOrganizationDetectors, updateOrganizationDiscoverSavedQuery, updateOrganizationExternalUser, updateOrganizationForwarding, "Link an Existing External Issue to an Issue", updateOrganizationReleaseFile, updateOrganizationWorkflows, updateProjectCodeOwners, "Update a Custom Inbound Filter", "Bulk Update Project Environments", updateProjectEnvironment, updateProjectReleaseFile, updateTeamExternalTeam.

    Safe with a small fix (16). Most just need the audit entry written only when something changed:

    • Audit entry written on every call: updateProject (core/endpoints/project_details.py:1345), updateProjectFilter (api/endpoints/project_filter_details.py:106), "Update a Service Hook" (project_servicehook_details.py:80-88), updateProjectKey (project_key_details.py:155,164), updateProjectOwnership (issues/endpoints/project_ownership.py:344; save() already tracks changed), updateTeam (core/endpoints/team_details.py:149), updateOrganizationMemberTeam (organization_member_team_details.py:443), updateOrganizationNotificationsAction (notification_actions_details.py:165), updateOrganizationRepositorySettings (organization_repository_settings.py:160).
    • updateOrganizationMonitor / updateProjectMonitor: MONITOR_EDIT is written whenever any field is sent (monitors/validators.py:515-525).
    • updateOrganizationScimV2Group (PATCH): member add, remove and replace are already diffed. A displayName replace audits unconditionally (scim/teams.py:443-459).
    • updateOrganization: a replayed avatar upload creates a new File and avatar ident every time (organization_details.py:795). Skip it when the bytes match.
    • updateOrganizationIssue: isPublic=true always deletes and recreates the share (api/helpers/group_index/update.py:1044-1066). The code comment says that's intentional, so changing it is a behaviour change.
    • updateProjectSymbolSource: without a body id the source gets a new uuid4(), so a replay with ?id=<old> returns 404 (project_symbol_sources.py:454). Fix: source.setdefault("id", id).
    • updateSentryApp: sending schema always deletes and recreates SentryAppComponent rows with new UUIDs (sentry_apps/logic.py:335-354). Skip that when the schema is unchanged.

    State is the same but replay returns 404 (1): updateOrganizationScimV2User. active=false hard-deletes the member.

    Not safe (6):

    • updateOrganizationIssues and "Bulk Mutate a List of Issues": without id= a replay re-runs the search and mutates the next batch. merge re-runs start_merge_groups and adds another MERGE Activity, and isPublic rotates the share.
    • updateOrganizationDashboard: every PUT adds a DashboardRevision, which also prunes the oldest revisions. Widgets without an id are recreated with new IDs.
    • updateOrganizationDetector and updateOrganizationWorkflow: conditions, filters and actions without an id are deleted and recreated. Metric detectors also re-sync Snuba subscriptions and dynamic ones call Seer again.
    • updateOrganizationMember: reinvite resends the invite email and regenerate makes a new token. The role and team path alone would only need the audit fix.

    DELETE (40)

    Safe as-is (4): deleteOrganizationIssueExternalIssue, "Unlink an External Issue from an Issue", deleteOrganizationMemberTeam, unstarOrganizationServiceSpan.

    Safe when selected by ID, not safe with a search query (3): deleteOrganizationIssues, "Bulk Remove a List of Issues", deleteOrganizationWorkflows. A query-based replay deletes the next matching batch.

    Safe with a small fix (9):

    • deleteOrganizationIntegration: this is a bug even without retries. installation.uninstall() runs before the updated guard (organization_integration_details.py:117-119), and the lookup doesn't filter on status. If a Discord server is shared with one other org, a second DELETE makes the bot leave that org's server, because discord/integration.py:118-127 doesn't count pending rows. Cursor Origin has the same pattern. Fix: return 204 before uninstall() when the org integration is already PENDING_DELETION.
    • deleteProjectTeam: PROJECT_TEAM_REMOVE is audited on every call (project_team_details.py:132). Same fix as addProjectTeam.
    • deleteOrganizationDetectors (ID mode): returns 400 when the IDs are already pending. Return 204 instead. Query mode is not safe.
    • deleteOrganizationIssue, deleteSentryApp (403 on replay), deleteTeam (ID lookups only), deleteProjectReplay, deleteProjectSymbolSource, and the sentry-app "Delete an External Issue": each would return 2xx if it treated "already pending or already gone" as success.

    State is the same but replay returns 404 (23): deleteOrganizationCodeMapping, deleteOrganizationDashboard, deleteOrganizationDetector, deleteOrganizationDiscoverSavedQuery, deleteOrganizationExternalUser, deleteOrganizationForwarding, deleteOrganizationMember, deleteOrganizationMonitor, deleteOrganizationNotificationsAction, deleteOrganizationPreprodArtifactSnapshot, deleteOrganizationRelease, deleteOrganizationReleaseFile, deleteOrganizationScimV2Group, deleteOrganizationScimV2User, deleteOrganizationWorkflow, deleteProject, deleteProjectCodeOwners, "Delete a Custom Inbound Filter", "Remove a Service Hook", deleteProjectKey, deleteProjectMonitor, deleteProjectReleaseFile, deleteTeamExternalTeam.

    • Soft deletes also rename or randomize the slug (project, team, monitor), so slug-addressed replays can't be made to return 2xx.
    • For deleteProject there's an edge case: if a new project takes the released slug before the replay, the replay deletes the new project.

    Unclear (1): "Disable Spike Protection". It lives in getsentry and wasn't read.

    Takeaways

    1. The biggest win is a shared "only audit on change" helper. Most of the safe-with-fix PUTs only need that.
    2. Nested lists that drop children missing from the request and recreate children without an id (dashboard widgets, detector conditions, workflow filters and actions) aren't replay-safe unless clients send every child's id. These belong with Idempotency-Key (Support Idempotency-Key for writes that aren't naturally idempotent #129168).
    3. For the 404-on-replay DELETEs, there are two options: mark them with a documented "a 404 after a retry means it was already deleted" rule, or leave them unmarked.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions