Skip to content

fix(blocks): read draft files past shared caches (no CDN-cached 404s) - #804

Closed
tlgimenes wants to merge 3 commits into
mainfrom
fix/draft-read-no-cdn-cache
Closed

tlgimenes wants to merge 3 commits into
mainfrom
fix/draft-read-no-cdn-cache

Conversation

@tlgimenes

@tlgimenes tlgimenes commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

What

Every draft GET in the v8 draft reader (packages/blocks/src/v8/draftChanges.ts: the draft index and each file it lists) now carries a fresh ?r=<nonce>, so no shared cache can answer it.

Why

delivery.decocms.com answers a missing object with a 404 that has no Cache-Control, and Cloudflare caches it (cf-cache-status: HIT, age climbing to ~180 s). If the SDK reads a draft file before its save has landed, every later read of that URL gets the cached 404 for about 3 minutes. Because the draft index already lists the name, the preview quietly shows production for that block, with nothing logged.

This happened in production on 2026-11-10 (osklenbr-faststore-studio, draft SWYdvLFwqQ5t8D60VODzJQ): index.json naming Footer-Island was written at 07:30:18Z and Footer-Island.json only at 07:30:44Z. The ~26 s gap was a stalled R2 PUT between two back-to-back writes, and the cached 404 kept the block on production for minutes after the file landed.

Studio keeps its write order: the draft index first, then the file (a deliberate design decision). So a listed name with no file can be a save still in flight for a moment. The nonce makes sure the first read after the file lands sees it, instead of a 404 that Cloudflare cached during that moment.

How the CDN behaves today:

request result
GET …/drafts/<slug>/nope-x1.json twice 404 MISS, then 404 HIT age: 0 (404s are cached)
GET …/Footer-Island.json 200 REVALIDATED, etag deabf5b2…
GET …/Footer-Island.json?r=abc1 200 MISS: same etag and body (R2 ignores the query; Cloudflare keys on it)

Draft objects are no-cache, max-age=0, must-revalidate, so every read already went to storage. The nonce adds no extra origin requests. It does add origin bytes: an existing object now comes back as a full 200 from R2 where Cloudflare used to revalidate it with a conditional request (see Known limitations).

Loopback draft hosts (plain http: a local Studio or a test server, which never sit behind a CDN) are read at their plain URL, without the nonce. Some of them match the request URL exactly, as the nextjs-binding and tanstack-worker smoke draft servers do.

Why no retry for a listed name whose file is a 404

I looked at adding one short retry. The code says it isn't worth it:

  • The editor's own preview heals as soon as the save answers. Studio moves the preview's pointer only once the save answers, with a revision taken from the last file written (adoptHostedSave in apps/web/src/components/sections-editor/content-protocol-api.ts). By then the index and file are both in R2, so the reload's nonce'd read sees the file. The pointer's version doesn't pin content (fetchDraft always reads the current index and files), so if the editor navigates or reloads inside the preview while a save is in flight, that read can still see the new index without the file and show production for that block. The reload that adoptHostedSave triggers fixes it.
  • Any other reader that catches the gap gets a new pointer when the file lands. The pointer's version is draftToken (apps/api/src/hosted/cms-drafts.ts). It hashes the index ETag and the ETag of every listed file that exists, so it changes as soon as the file appears. A second editor's poll then moves its pointer and reloads its preview, and with the nonce that reload reads the file. Without the nonce, it would have read the cached 404.
  • A retry would cost the cases it can't fix. A name that stays listed-with-no-file is a failed save's leftover (the next save or a discard fixes it). A retry would add its delay to every server render of that draft, and to every page an editor opens in the preview, which only editors see. It would also only help within its own window: a few hundred ms couldn't have covered the 26 s stall in the incident.

So the reader keeps treating a 404 as "not in the draft" (production shows), and the pointer moving is what heals it.

Spike (real R2 + delivery.decocms.com)

Script: race-spike.mjs on BugMachine (~/cas-explore/race-spike.mjs, logs in ~/cas-explore/spike-race/). It writes a throwaway draft the way Studio does: index.json naming Barra promocional first, then the body (PromoBar text replaced with a marker) 20 s later. It polls every 2 s.

Run 1, storefront (before fix): https://osklenbr.vtex.app/?__draft=delivery.decocms.com/sites/osklenbr-faststore-studio/drafts/spike-race-1/index.json@spike1

  • index PUT 08:02:59Z, body PUT 08:03:20Z
  • the page rendered fallback (production) until 08:06:00Z, which is 159.7 s after the body existed. The raw body URL was 404 cf=HIT age=…179 the whole time.

Run 2, SDK before vs after (fetchDraft from @decocms/blocks@8.12.0, as in osklen's node_modules, vs this branch's dist), same draft shape (spike-race-2):

  • body PUT 08:32:45Z
  • patched SDK: draft read 2.5 s later (the first poll after the PUT)
  • 8.12.0 SDK: fallback until 159.7 s later. The plain URL stayed 404 cf=HIT up to age=179.

Spike objects were left in place on purpose (nothing deleted without an OK):

  • sites/osklenbr-faststore-studio/drafts/spike-race-1/index.json
  • sites/osklenbr-faststore-studio/drafts/spike-race-1/Barra%20promocional.json
  • sites/osklenbr-faststore-studio/drafts/spike-race-2/index.json
  • sites/osklenbr-faststore-studio/drafts/spike-race-2/Barra%20promocional.json

Tests

  • New cms.test.ts: every https draft GET carries a distinct ?r= nonce and the object address is unchanged. A loopback host is read at its plain URL.
  • URL assertions in cms.test.ts, __conformance__/{delivery,sdk,campaigns}.test.ts now compare URLs with the nonce removed.
  • tests/worker-hosted.test.ts ("a preview request reads its draft's folder once") counted draft GETs with endsWith, so it broke on the nonce. It now compares URLs with the nonce removed, like the others.
  • The branch is rebased onto main 6492e9c (linear history, no merge commits): three commits, head 97aefba9. On BugMachine, on that tree: vitest run packages/blocks/src/v8 tests/ 1840 passed, 5 skipped; bun run build clean.
  • An earlier rebase check onto main 76baef6 (after feat(blocks): allow a 5 s CMS check interval; 1 s jitter #805, feat(nextjs,tanstack): v8-only StudioBridge and DecoDocument, Cypress-safe "." shortcut #806, feat(nextjs,tanstack): let Studio's HTML swap re-render the page through its router #808), on BugMachine:
    • vitest run packages/blocks/: 71 files, 2285 passed, 126 skipped.
    • vitest run tests/ examples/tanstack-worker-smoke/src: 12 files, 201 passed.
    • bun run build, bun run typecheck and bun run lint are clean. Lint's one warning is already on main.
    • bun run smoke passes in both examples/tanstack-worker-smoke and examples/nextjs-binding-smoke.
  • On the pushed head cffcdf5d: bun run build is clean, and vitest run tests/ packages/blocks/src/v8/cms.test.ts packages/blocks/src/v8/__conformance__ passes (26 files, 865 tests).
  • One run of the draft suites on the rebased tree had 1 failure out of 705 under heavy load (96 s wall time). The next two runs were 705/705. It looks like a timeout flake. I haven't checked whether main has it too.

Known limitations

  • The fix depends on Cloudflare keeping the query string in the cache key. delivery.decocms.com does today: a nonce'd URL got MISS with the same weak etag. If someone later adds an "ignore query string" Cache Rule there, the fix would silently stop working.
  • A server-side fix would complement this one (not done here). A Cloudflare rule that stops caching 404s under /sites/*/drafts/* would fix the stale 404 for every reader, including older SDKs and custom readers. This PR changes no Cloudflare settings. A custom preview.draftHosts behind a CDN that drops query strings (for example, CloudFront without query forwarding) won't benefit, and a host that rejects unknown query params would break.
  • Every draft render adds N+1 unique Cloudflare cache entries, and full GETs to R2 instead of conditional ones. This is cheap but not free. Next.js isn't affected: in draft mode, Next 16.2.11 patch-fetch uses originFetch.

Not merged — waiting for review.

🤖 Generated with Claude Code

tlgimenes and others added 3 commits November 10, 2026 16:03
Every draft GET (the index and each file) now carries a fresh ?r=<nonce>.
delivery.decocms.com answers a missing object with a 404 that has no
Cache-Control, and Cloudflare keeps it (cf-cache-status: HIT, age up to
~180 s). A draft file read once before its save landed then read as
missing for minutes, and the preview silently showed production for that
block. R2 ignores the query; Cloudflare keys on it, so the read always
reaches storage (the draft objects are no-cache anyway).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A loopback draft host (a local Studio or test server over plain http)
has no CDN in front, and some match the request URL exactly (the
nextjs-binding and tanstack-worker smoke draft servers did): the
cache-busting nonce goes on https reads only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tlgimenes
tlgimenes force-pushed the fix/draft-read-no-cdn-cache branch from cffcdf5 to 97aefba Compare November 10, 2026 19:06
@tlgimenes

Copy link
Copy Markdown
Contributor Author

Closing: fixed server-side instead. A Cloudflare cache rule on the decocms.com zone ("delivery.decocms.com: bypass cache for CMS drafts", http.host eq "delivery.decocms.com" and http.request.uri.path wildcard r"/sites/*/drafts/*" → Bypass cache, ordered after the existing delivery rule) stops 404s for draft files from being cached, for every SDK version. Verified 2026-11-10 19:48Z: missing and existing draft objects answer cf-cache-status DYNAMIC, latest.json still HIT; the old write-order race spike on osklenbr.vtex.app (SDK without this PR) showed the draft 0.5 s after the file landed, vs 169 s before.

@tlgimenes tlgimenes closed this Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant