[cjimti]

Forms built on endpoints (#2094) now store their submissions, protect sensitive fields, render inside assets, and report to their owner. This covers four child tickets (#2177, #2178, #2179, #2180) and one E2E Nightly fix. The epic stays open for #2331, #2332 and #2333.

Collection through a webhook source (#2178)

• Binding a form with manage_script command=endpoint_set makes a webhook source named form-<endpoint> with input: form (internal/webhook/whsource/form.go, internal/endpoint/epsource). Its events land in webhook_form_<endpoint> on a scratch catalog connection.
• endpoint_set takes a connection argument. With no argument it uses the only scratch connection; when there are several, it refuses and lists them. endpoint_list reports is_form, form and form_connections.
• An accepted submission is stored through the receiver's own spool path (internal/webhook/receiver/form.go, Deliver) as one event of type form_submission. The payload holds endpoint, version, values, fields, sensitive and submitter (internal/endpoint/epcollect/epcollect.go).
• On a signed-in form the submitter comes from the session or viewer token, never from a form field.
• If the store fails, the submission is answered 503 with Retry-After and nothing is recorded as accepted (internal/endpoint/eprun/form.go).
• Form sources take no posts:
  • POST /hooks/form-<endpoint> answers 404.
  • The admin API refuses to create a form source directly (whadmin.ErrFormSource).
  • The admin API refuses to switch an existing source to input: form, and refuses a form block on a source that isn't a form's.
  • The admin webhook API reports an empty path for a form source.

Sensitive field encryption (#2179)

• A spec field marked sensitive is sealed with fieldcrypt before the event reaches the spool. The ciphertext is what lands in the spool, the raw segment, the compacted Parquet and the Trino view.
• A form with sensitive fields is refused at spec time when the deployment has no encryption key (epcollect.ErrCannotSeal).
• fieldcrypt gains key rotation:
  • NewFieldEncryptor(key, previous...) decrypts with the current key, then each previous key.
  • ENCRYPTION_KEY_PREVIOUS is a list of earlier keys (pkg/platform/fieldcrypt/fieldcrypt.go, docs/server/deployment.md).
  • AES-GCM authenticates, so no key id is stored.
• platform.decrypt(payload) is a new script host call (internal/scripttoken/decrypt.go, internal/endpoint/epcollect/decrypt.go). It is allowed when the run author's persona is listed in the source's decrypt_personas, which endpoint_set and the portal binding editor set. Admins are always allowed.
  • It accepts the payload as platform.query returns it (a dict) and as JSON text.
  • Each call writes an audit row of the new kind form_decrypt naming the fields opened (pkg/audit/event.go).
  • A run's recording stores opened values as [encrypted].

Rendering and embedding in assets (#2177)

• GET /portal/forms/<endpoint> renders a form from its spec, with its bot check, and posts to the endpoint (internal/endpoint/epformpage).
• HTML and JSX assets embed a form by the mcp:form:<endpoint> reference (internal/portal/assetrefs, internal/portal/contentrefs). An asset may embed any number of forms.
  • A public form embeds anywhere.
  • A signed-in form can be embedded only by its owner or an admin.
• A frame nested in an asset inherits the asset's sandbox (allow-scripts allow-downloads, opaque origin, no cookie). Four changes follow from that, and the asset sandbox itself is unchanged:
  • A signed-in form inside an asset is given a short-lived viewer token in its URL. The token is an HMAC over endpoint and viewer, valid 12h (internal/endpoint/epform/viewer.go), and the endpoint takes it as the caller only for that form (ephttp.caller).
  • A form answers Origin: null with CORS, never with credentials. Endpoints that are not forms still refuse it.
  • The form page submits from its button and the Enter key, because a form's submit event does not fire without allow-forms.
  • Form frames point at the browsed origin (ui/src/components/renderers/formFrames.ts), because frame-src 'self' refuses public_base_url when it differs.
• Turnstile and hCaptcha may not work inside a sandboxed asset frame; proof of work does (docs/scripts/endpoints.md).

Submissions view, export and MCP authoring (#2180)

• GET /api/v1/portal/scripts/{id}/submissions lists a form's stored submissions to its owner or an admin, decrypted and audited.
• GET /api/v1/portal/scripts/{id}/submissions/export?format=csv|xlsx exports them (internal/endpoint/epsubmit, internal/httpserver/scripthttp/submissionshttp). CSV cells that start with a formula character are prefixed.
• The automation's page has a Submissions card: Received and Submitter columns plus one column per field; a lock icon ("Encrypted when stored") on sensitive fields; Refresh, Export CSV and Export Excel; and "No submissions yet." when empty.
• The endpoint editor gains Connection, which becomes read-only once the source exists, and Decrypt personas.
• The webhook detail page shows a form source's form and links back to it.
• An agent can author, bind and read a form through manage_script alone; acceptance criterion 2 of feat(forms): submissions view, export and MCP authoring #2180 does exactly that.

E2E Nightly

TestIssue2065_TheAuditTheCallRecordAndTheRecordingHoldThePlaceholder read the call record once. The record is written after the audit row, so the read could come too early. It now polls (test/acceptance/issue_2065_test.go).

Acceptance

All 13 criteria passed through the real surface on the dev stack, and each run is recorded at build/<n>/acceptance.jsonl:

• feat(webhooks): a browser-submission source for each form #2178 (4 criteria):
  • A public submission becomes a row.
  • A refused bot check stores nothing.
  • A signed-in submission records the submitter.
  • input: form is accepted only on a form's own source.
• feat(forms): encrypt sensitive fields at submit time #2179 (4 criteria):
  • Ciphertext appears in the spool, the bucket and the table.
  • The owner reads plaintext and the read is audited.
  • A script decrypts only under a listed persona.
  • A value sealed under the previous key opens after rotation.
• feat(portal): render forms and embed them in assets #2177 (3 criteria):
  • An anonymous browser submits two forms on a public share.
  • A signed-in form embedded in an asset works with its viewer token, and a peer is refused.
  • Screenshots in light and dark at desktop and phone width.
• feat(forms): submissions view, export and MCP authoring #2180 (2 criteria):
  • The owner sees a submission and exports CSV and XLSX.
  • An agent works through MCP alone while an anonymous browser submits.

The Playwright half is ui/e2e/forms. The #2065 criteria also re-pass.

Readers of this state

• whsource.Config.input gains form, and there is a new form block (whsource.Form).
  • Read by ui/src/api/admin/types/webhooks.ts, ui/src/pages/webhooks/webhookForm.ts, WebhookEditor.tsx and WebhookDetailPage.tsx, all updated.
  • The new section is FormSourceSection.tsx.
  • The editor does not offer form as an input to choose.
• bindinghttp.bindingRequest gains connection and decrypt_personas, and the endpoint route descriptions changed.
  • Read by ui/src/api/portal/hooks/scriptEndpoint.ts, ScriptEndpointCard.tsx, ScriptEndpointSections.tsx and endpoints/FormStorageFields.tsx, all updated.
  • The other writer is epcmd (endpoint_set), which takes the same two fields.
• GET /portal/scripts/{id}/submissions and its export route are new, along with epsubmit.Page, epsubmit.Submission and epcollect.Submitter.
  • Read by ui/src/api/portal/hooks/scriptSubmissions.ts and ScriptSubmissions.tsx.
• audit.EventType and auditfiltersResponse.event_kinds gain form_decrypt, which also changes the parameter descriptions on the audit event and metric routes.
  • The admin audit page takes event kinds from the server (ui/src/api/admin/hooks/system-audit.ts sends event_kind as a string), so it needs no change.

Built-in pages

make builtin-pages-check listed seven pages whose source paths this diff touches. I read each against the change:

• asset-references-and-the-refresh-loop.md now covers mcp:form: references.
• content-types-for-stored-files, documents-that-fit-any-screen, presentations, semi-dynamic-dashboards, managed-script-dialect and writing-managed-scripts say nothing the change alters. None of them lists the script host calls, which help (scriptcontract) documents and which now includes platform.decrypt.

New package imports

test/structure/testdata/allowed_internal_imports.txt gains 47 edges, all from packages this PR adds or extends:

• epcollect:
  • whsource and whevent: the event it builds.
  • scripttoken: the decrypt host call's interface.
  • pkg/audit and pkg/middleware: the form_decrypt row and the caller identity.
  • epform: the spec's sensitive fields.
  • logsan.
• epsubmit:
  • whsource and whtable: the source and the view it reads.
  • epcollect: decrypting.
  • tablexlsx: the Excel export.
  • pkg/audit and pkg/middleware.
  • logsan.
• epsource: whsource.
• epformpage:
  • epbinding, eprun and epform: the binding, the spec and the run.
  • assetrefs: the shared URI prefix.
  • logsan.
• epcmd, ephttp, endpointwire and pkg/portal to epform: the viewer token.
• epcmd to epsource.
• eprun to epcollect.
• epwire to assetrefs: the form targets an asset resolves.
• formwire, webhookwire and submissionshttp are wiring:
  • formwire connects pkg/platform, the Trino toolkit, webhookwire, epform, epsubmit, whsource and whtable.
  • webhookwire connects epcmd, epcollect, epsource, scripttoken, whevent and pkg/middleware.
  • submissionshttp serves epsubmit through scripthttp's ownership check, with httpjson and logsan.
• internal/httpserver mounts formwire and submissionshttp.

Package size budgets were held by moving code out, never by raising a budget:

• fieldcrypt.DecodeKey/DecodeKeys moved out of pkg/platform.
• The test-only script stub moved from scriptrun to scriptdialect.
• epsource split out of epcollect for cohesion.
• submissionshttp is its own package under scripthttp.

make verify passed.

Closes #2177
Closes #2178
Closes #2179
Closes #2180

[codecov]

Codecov Report

❌ Patch coverage is 94.91892% with 47 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.80%. Comparing base (7fe1c07) to head (9d5ff4b).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/httpserver/formwire/formwire.go	54.54%	15 Missing ⚠️
internal/endpoint/epsubmit/epsubmit.go	95.00%	7 Missing ⚠️
internal/endpoint/epcollect/decrypt.go	91.37%	5 Missing ⚠️
internal/endpoint/epcollect/epcollect.go	93.50%	5 Missing ⚠️
...rver/scripthttp/submissionshttp/submissionshttp.go	96.22%	2 Missing ⚠️
internal/webhook/receiver/form.go	93.93%	2 Missing ⚠️
internal/webhook/whadmin/service.go	94.59%	2 Missing ⚠️
pkg/platform/fieldcrypt/fieldcrypt.go	95.00%	2 Missing ⚠️
internal/endpoint/epform/viewer.go	96.77%	1 Missing ⚠️
internal/endpoint/epformpage/epformpage.go	96.77%	1 Missing ⚠️
... and 5 more

Additional details and impacted files

@@           Coverage Diff            @@
##             main    #2338    +/-   ##
========================================
  Coverage   94.80%   94.80%            
========================================
  Files        1422     1435    +13     
  Lines      115639   116477   +838     
========================================
+ Hits       109627   110430   +803     
- Misses       5992     6027    +35     
  Partials       20       20            

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.