Repository navigation
feat(storage): presigned PUT uploads for stores without POST (Cloudflare R2) - #248
Merged
Merged
Conversation
Cloudflare R2 takes presigned PUT, GET, HEAD and DELETE but no POST form uploads. STORAGE_UPLOAD_METHOD=put (default post, so existing S3, MinIO and RustFS installs are unchanged) signs every upload whose request carries the new optional fileSize as a presigned PUT of that object, with Content-Type and the exact Content-Length signed and the same two-minute expiry, and answers uploadMethod, uploadUrl and uploadHeaders beside the unchanged POST fields. A request without fileSize gets the POST alone. A fileSize over the upload's limit (the element's for the client API, 5 MB for the management storage route) is refused with 400 and the limit in the details before anything is signed, in either mode. Files stay private and are read through the /storage route, so nothing depends on a public-read bucket.
The survey runtime and the admin app send the file's size with the signing request and PUT the file to uploadUrl with uploadHeaders when the answer says uploadMethod "put"; otherwise, and with any server that predates it, they POST the form as before. A 400 carrying maxFileSize reads as a file that is too large.
Adds a respondent's file upload on a public survey page and a workspace logo upload, tags the upload specs @Upload, and runs them a second time in CI against a server started with STORAGE_UPLOAD_METHOD=put, including a check that the store refuses a PUT body larger than the size it was signed for.
…T store check putStoreCheck signs through the Bucket as the server does and checks that the store refuses a body one byte longer or shorter than signed, another Content-Type and an expired URL with nothing stored, takes a key that needs URL encoding (請求書 100% a+b&c=d), keeps the object private, and, given origins, answers the CORS preflight and the PUT for them. It runs against RustFS in the suite and, as TestPutCanary, against a real store such as the R2 bucket when PUT_CANARY_ORIGINS and its S3_* settings are set. In put mode the server now logs a warning for each upload request without fileSize, which can only be answered with a POST, so the callers that still send no size can be found. .env.example describes the two-step switch: put while the bucket still takes POST, the new store no sooner than two hours later.
houko
force-pushed
the
feat/presigned-put-uploads
branch
from
October 28, 2026 02:37
75dc180 to
441f12b
Compare
…loads # Conflicts: # .github/workflows/e2e.yml # apps/server/internal/clientapi/contract_test.go # deploy/docker/docker-compose.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No ticket: first unit of the owner's 2026-10-28 decision to keep forma's files on Cloudflare R2 like the other yldm apps.
What & why
Was: every upload (respondent files, logos, editor images) was a presigned POST form, which R2 does not support.
Now: with
STORAGE_UPLOAD_METHOD=putthe server also signs a presigned PUT of the exact key, type and size, and the survey runtime and admin app upload with it. The default stayspost, so existing installs answer byte for byte as before.fileSize; over the element's (or the 5 MB public) limit it is refused with 400 andmaxFileSizebefore anything is signed.signedUrl/presignedFieldsand adduploadMethod: "put",uploadUrl,uploadHeadersonly in put mode and only whenfileSizewas sent, so older clients still get a POST they understand./storageroute (server proxy), so R2 needs no ACL or public bucket. CSPconnect-srcalready allows anyhttps:endpoint (loopback endpoints outside production), so it is unchanged.Where to look
Bucket.SignPut: Content-Type and Content-Length signed, two-minute expiryService.SignUpload: size check, when PUT fields are addedApiClient.uploadFile: PUT/POST choice in the runtimeBreaking changes
None:
fileSizeand the three answer fields are optional and additive, and the new variable defaults to today's behaviour. New contract exchanges pin both modes; no existing exchange changed.Migrations & env
STORAGE_UPLOAD_METHOD(postdefault,put), documented in.env.example. The R2 bucket's CORS rule must allowPUTwithContent-Typefrom the app origin and, for respondents on customer sites, any origin.fileSizegets the POST alone and R2 refuses POST:STORAGE_UPLOAD_METHOD=putwhile the bucket is still MinIO (minio-criticalafter P6), which takes both POST and PUT, so old and new clients both upload.S3_*settings to R2./js/surveys.umd.cjsis served withmax-age=3600, stale-while-revalidate=3600, so a browser can run the runtime from before step 1 for up to two hours; the Reloader rolling restart also leaves pods onpostfor a while during step 1.fileSizeand useuploadMethod/uploadUrl/uploadHeaderswhen present; withoutfileSizethey get the POST alone, which R2 refuses. In put mode the server logsUpload signed as a presigned POST only ... sent no fileSizewith theworkspaceIdfor each such request, so those integrations can be found.How this was tested
Rerun:
cd apps/server && go test ./internal/storage/ ./internal/clientapi/ ./internal/mgmtapi/ ./internal/platform/config/Coverage
TestSignPutStoresExactlyTheSignedFile,TestSignPutEmptyFilefileSizevalidated, over-limit refused, PUT fields only in put mode with a sizeTestServiceSignUploadPut,TestStorageUploadFileSize,TestStorageUploadPut,TestV1StorageFileSize*-put,*-file-size,*-file-too-largemaxFileSizeapi-client.test.ts,upload.test.tsTestPutStoreCheck(RustFS),TestPutCanary(any store, env-gated)SignUpload→ PUT →Service.OpenTestServiceSignUploadPutEncodedName請求書 100% a+b&c=d.pdfread back through itsfileUrlTestServiceSignUploadPutWarnsWithoutSizeWARNwithworkspaceId, none in post mode or with a size/s/…, workspace logo, editor image, oversized PUT refused@uploadspecs, local stack against MinIORELEASE.2025-09-07(production's image)putand withpostOpen gaps
Not run against real R2 yet: there is no forma R2 bucket or bucket-scoped token yet. The size limit of
putrests on the store refusing a body whose length is not the signedContent-Length, which Cloudflare documents forContent-Typebut not forContent-Length. Before step 2 above, runTestPutCanarywith the bucket's production settings; it signs through the sameBucketcode, checks that a body 1 byte longer, 1 byte shorter, anotherContent-Typeand an expired URL are refused with nothing stored, that a key needing URL encoding (請求書 100% a+b&c=d--fid--….pdf) is stored, that the object is not readable unsigned, and that the CORS preflight and the PUT are allowed for each origin, then deletes its object:Record the result here or in the yldm-platform R2 PR. If R2 does not refuse a body of another length,
putneeds another size guard before production uses it.Risks
fileSizekeeps getting POST only, which fails on R2; the warning log above finds them, and the cutover release note must tell them to sendfileSize.