Skip to content

fix(electric): stop restarting Electric on unrelated deploys - #1457

Merged
Makisuo merged 1 commit into
mainfrom
fix/electric-image-ref-no-lockfile-restart
Oct 14, 2026
Merged

Makisuo merged 1 commit into
mainfrom
fix/electric-image-ref-no-lockfile-restart

Conversation

@Makisuo

@Makisuo Makisuo commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Every PRD deploy that touches bun.lock restarts Electric. alchemy names a context-built image after a hash of the context plus the repo lockfile, so an unrelated dependency change gives Electric a new image tag, a new task definition and a restart. Electric is a singleton (minimumHealthyPercent: 0), so each restart is a sync gap.

On 2026-10-14 this happened twice in a row: the deploys for #1450 and #1456 (a landing-only change) each caused ~2 min of 503s from the load balancer, about 130 failed shape requests across dashboards and the alerts lists. The three latest task definitions differ only in image tag.

Fix

apps/electric/Dockerfile was a single pinned FROM. Pass that ref to the ECS service as image instead: alchemy mirrors it into ECR under a tag hashed from the ref and platform only, so Electric restarts only when the pin changes. The Dockerfile is gone; the pin and its rationale now live in ELECTRIC_IMAGE in apps/electric/alchemy.run.ts, and the publish workflow and runbook point there.

The deploy that ships this restarts Electric once more (the tag scheme changes), then it stays put.

apps/ingest and apps/sandbox also build from contexts and likely restart on lockfile changes too; not touched here.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Devin Review

Summary by CodeRabbit

  • Chores
    • Updated the Electric deployment to use a specific, pinned image version. The image reference is now shown directly in the build summary.
  • Documentation
    • Updated Electric sync guidance to reflect how the production image is specified.

alchemy hashes a Dockerfile build context together with bun.lock, so any
dependency change retagged the Electric image and restarted the singleton
task. On 2026-10-14 two back-to-back PRD deploys (one landing-only) each
caused a ~2 min sync outage this way.

The Dockerfile was a single pinned FROM, so pass that ref as `image`:
alchemy mirrors it to ECR under a tag derived from the ref and platform
only, and Electric restarts only when the pin changes. The deploy that
ships this restarts it once more, since the tag scheme changes.
@maple-review-bot

maple-review-bot Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Maple review

🟢 Confidence 8/10 · likely safe to merge
The pin moves unchanged into ELECTRIC_IMAGE; the only unverifiable step is alchemy 2.0.0-beta.83 mirroring a bare image ref to ECR, which the next prd deploy itself proves.
quality 100/100 · no findings · tests not needed · risk medium

Electric stops being built from a Dockerfile context and is instead handed to the ECS service as the same pinned ghcr ref, so a bun.lock change no longer retags it and restarts the singleton. The pin, the workflow summary and the runbook stay consistent; safe to merge.

  • ELECTRIC_IMAGE in apps/electric/alchemy.run.ts replaces the context+Dockerfile build, keeping the same digest pin
  • AWS.ECS.Service("electric") now takes image instead of context/dockerfile
  • apps/electric/Dockerfile deleted; the publish workflow and docs/electric-sync.md point at ELECTRIC_IMAGE

Before merge

ed590bc · Updated on every push. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.

@coderabbitai

coderabbitai Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b58c17c7-f24e-4f1c-9ea4-2c2790545c6d
📥 Commits

Reviewing files that changed from the base of the PR and between d7a6ee7 and ed590bc.

📒 Files selected for processing: 4
  • .github/workflows/publish-electric-image.yml
  • apps/electric/Dockerfile
  • apps/electric/alchemy.run.ts
  • docs/electric-sync.md
💤 Files with no reviewable changes: 1
  • apps/electric/Dockerfile

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

📝 Walkthrough

The Electric deployment now uses a digest-pinned image reference instead of building from a local Dockerfile. The publishing workflow and deployment guidance now point to the image configuration in apps/electric/alchemy.run.ts.

Changes: Electric image deployment

Layer / File(s) Summary
Configure the pinned Electric image
apps/electric/alchemy.run.ts, apps/electric/Dockerfile, .github/workflows/publish-electric-image.yml, docs/electric-sync.md
createMapleElectric now uses the digest-pinned ELECTRIC_IMAGE instead of building from the local Dockerfile, which was removed. The workflow and deployment guidance now identify the image configuration as the pin location. The workflow summary prints the image reference directly.

Priority: ⬆️ High

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to ed590

No concrete deployment defect is established. The digest match remains unconfirmed and can be checked against the successful publish run before deployment.

Pre-merge checks: Passed 5
Passed Passed checks: 5 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly identifies the Electric deployment fix and matches the main change: preventing restarts caused by unrelated deploys.
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@Makisuo
Makisuo merged commit aff4d5e into main Oct 14, 2026
37 checks passed
@Makisuo
Makisuo deleted the fix/electric-image-ref-no-lockfile-restart branch October 14, 2026 12:26
@maple-review-bot

Copy link
Copy Markdown

Merged with 1 step from "Before merge" still open. If it is done, tick it on the review comment; if not, now is the time.

  • Infra · Check every environment has what this config now expects

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant