Skip to content

feat(endpoints): form page, file fields, event runs (#2331-#2333, #2337) - #2340

Merged
cjimti merged 1 commit into
mainfrom
feat/2331-2332-2333-2337-forms-events-uploads
Oct 27, 2026
Merged

cjimti merged 1 commit into
mainfrom
feat/2331-2332-2333-2337-forms-events-uploads

Conversation

@cjimti

@cjimti cjimti commented Oct 27, 2026

Copy link
Copy Markdown
Member

Adds the /forms/{endpoint} page and its embed snippet, file form fields, runs per stored webhook event (manage_script update event_source, PUT /api/v1/portal/scripts/{id}/event-source, GET /api/v1/portal/scripts/{id}/events, migration 000222), and GET/POST /api/v1/portal/knowledge/changesets[/{id}[/rollback]], and fixes 3 bugs: 1) a signed-in non-admin opening Knowledge > Changesets was caught in a sign-in loop (#2337), 2) the webhook receiver could register a raw window twice, 3) the #2176 acceptance tests named no scratch connection, which a stack with two of them requires.

Closes #2331
Closes #2332
Closes #2333
Closes #2337
Closes #2094

Hosted form page and embed snippet (#2331)

  • The form page is served at /forms/{endpoint} (internal/endpoint/epformpage, epform.PagePrefix). /portal/forms/ is gone, with no alias.
  • The page's CSP frame-ancestors is 'self' plus the binding's allowed_origins. A form framed inside an asset carries a signed, 12-hour, endpoint-bound embed marker (form_embed, internal/endpoint/epform/embed.go) that exempts it, because the asset's sandboxed frame has no origin the list could name. The asset rewrite (pkg/portal/assetrefs.go, internal/portal/assetrefs) adds the marker to every form address, and adds the viewer token for a signed-in reader.
  • Opened directly, an authenticated form uses the browser session: a viewer token in the URL is used first, then the session. With neither, a top-level load gets a 303 to sign-in and back, and a framed load gets a sign-in link. If the platform refuses the signed-in account, the page says the account cannot fill the form and does not redirect again.
  • The page posts its height to the parent (mcp-form:height) as it changes. The automation's page shows a copyable snippet for a bound form (ui/src/pages/scripts/endpoints/FormEmbed.tsx, embedSnippet.ts) whose frame resizes to that height. manage_script endpoint_list and the binding route report the bound form's page_url.

File fields (#2333)

  • New field type file, with max_files (default 1, at most 20), max_bytes (default 10 MiB) and accept. A submission with a file field is multipart/form-data (at most 1000 parts). Each file's type is judged from its bytes, not its name. Prefilling a file field is refused.
  • The submission is assigned its event id first. form_submit is then handed each file's mcp:// address, and the files are saved only after it accepts (internal/endpoint/epcollect/files.go, at most 4 saves in parallel). If saving or storing the event fails, the files already saved are deleted, so a refused or unstored submission keeps none.
  • Files are saved as managed resources in the form source's library, under its attachments folder: <attachments>/<date>/<event id>/<field>. Deleting the form source deletes them.
  • The submissions view links each file to its resource page, and the CSV and Excel exports give each file as name <uri>.

A run for each stored webhook event (#2332)

  • A script names one webhook source (scripts.event_source), through manage_script update event_source=... or the new Events section on its page. After the receiver has stored an event (spool, segment, or an imap poll's segment), internal/platform/scriptevents enqueues one run with trigger event for each script naming that source. The run reads the event as run.event: id, type, key, received_at, source and payload.
  • A unique index on (script_id, event_source, event_id) makes it one run per event, even when a sender retries.
  • A script set to run one at a time (exclusive queue or refuse) queues its event runs and runs them in order.
  • When a script already has scripts.events.max_queued (default 100) event runs waiting, a new event is recorded as a finished run with status skipped_backlog and a reason, not executed. The Events section shows the runs waiting, the runs skipped in the last 7 days, and the bound.
  • A failed event run emails the owner, as a failed scheduled run does.
  • Who may name which source (internal/platform/scriptevents/sourcerule): an administrator may name any source; anyone else only the source of a form whose script they own, since naming a source means reading every event it stores.
  • The enqueue runs in one transaction, after the event is durable and before the receiver answers; no run executes inside that request. If it fails, the error is logged and the stored event stands.
  • Migration 000222 adds scripts.event_source, and script_runs.event_source, event_id and event (JSONB), plus the unique index and a pending index. It also adds event to the trigger check and skipped_backlog to the status check. The migration passed migrate-check on pg16 and pg17.
  • TriggerEvent and StatusSkippedBacklog live in internal/runstate, which keeps them off pkg/script's exported surface.

Changesets for apply_knowledge holders (#2337)

  • pkg/admin answers valid credentials that are not the admin persona with 403 (ErrNotAdmin), not 401. The portal reads 401 as an expired session, which is what started the loop.
  • internal/httpserver/changesethttp serves GET /api/v1/portal/knowledge/changesets, GET .../{id} and POST .../{id}/rollback to a portal caller whose persona holds apply_knowledge, and to administrators. With no caller it answers 401; with a caller who lacks apply_knowledge, 403. The routes run the admin knowledge handler with the portal caller as its user, so a rollback is recorded under the reviewer.
  • The Changesets tab and drawer read the portal routes (ui/src/api/portal/hooks/changesets.ts). The tab's audit-filter query runs only for administrators.
  • ui/src/api/admin/client.ts does not redirect to sign-in on a 401 within 60 seconds of its previous sign-in redirect.

Webhook receiver fix

  • A delivery landing while its raw window's registration was finishing could register that window twice. registerPending is now serialized, and registerOne skips a window that is already registered (internal/webhook/receiver). It showed up as TestNewWindowIsRegisteredOnce failing only under full verify load. It did not fail in 300 targeted runs before the fix or 1,200 after, so the fix rests on reading the code, plus a new test of the case.

Readers of this state

  • changesethttp.changesetList (data, page, per_page, total) and the three changeset routes: ui/src/api/portal/hooks/changesets.ts, ui/src/pages/knowledge/capture/ChangesetsTab.tsx, ChangesetDrawer.tsx, and the MSW handlers in ui/src/mocks/handlers.ts moved to the portal base.
  • eventshttp.sourceRequest/sourceResponse (source, message) and PUT .../event-source: ui/src/api/portal/hooks/scriptEvents.ts, ui/src/pages/scripts/ScriptEventsCard.tsx, ui/src/mocks/handlers/scripts.ts.
  • eventshttp.summaryResponse (pending, skipped_since, skipped_window_days, max_queued) and GET .../events: scriptEvents.ts, ScriptEventsCard.tsx (EventSummaryFacts), the scripts mock.
  • script.Script.event_source and script.Contract.event_source: ui/src/api/portal/hooks/scripts.ts, and the Events fact in ui/src/pages/scripts/ScriptDetailPage.tsx. The run status skipped_backlog is labelled in ui/src/pages/scripts/runFormat.ts.
  • GET /portal/scripts/{id}/endpoint (description changed; the form's page_url): ui/src/api/portal/hooks/scriptEndpoint.ts and ui/src/pages/scripts/ScriptEndpointCard.tsx.
  • The page path moving to /forms/: ui/src/components/renderers/formFrames.ts (FORM_PATH_PREFIX; it rewrites only an address that carries form_embed) and ui/src/pages/webhooks/FormSourceSection.tsx.

Docs

  • docs/scripts/endpoints.md: the form page, embedding a form on a site, file fields.
  • docs/scripts/running.md: runs per stored event.
  • docs/server/asset-references.md, docs/knowledge/admin-api.md, docs/portal/admin-knowledge.md.
  • docs/llms.txt and docs/llms-full.txt, CLAUDE.md, and the writing-managed-scripts built-in page (run.event).
  • Light and dark screenshots of the form page.

Verification

- #2331: the form page moves to /forms/{endpoint}; a site its binding's allowed_origins lists may frame it, an asset frames it with a signed 12-hour embed marker, and the automation's page offers an embed snippet that sizes the frame to the form.
- #2332: a script names a webhook source (manage_script update event_source, or the Events section on its page) and each event the source stores enqueues one run with trigger event reading run.event; past scripts.events.max_queued (default 100) pending event runs an event is recorded as a skipped_backlog run. Migration 000222.
- #2333: file fields (max_files, max_bytes, accept) on a form; files are saved as managed resources in the form source's attachments folder only after form_submit accepts, and the submissions view and exports name them.
- #2337: the admin API answers a signed-in non-admin 403 instead of 401, the Changesets tab reads new /api/v1/portal/knowledge/changesets routes open to apply_knowledge holders, and the portal no longer redirects to sign-in again within a minute of a sign-in redirect.
- The webhook receiver no longer registers a raw window twice when a delivery lands while that window's registration is finishing.
@codecov

codecov Bot commented Oct 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.62881% with 46 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.81%. Comparing base (e01b0be) to head (bdc9dd3).

Files with missing lines Patch % Lines
internal/httpserver/webhookwire/forms.go 23.07% 10 Missing ⚠️
internal/httpserver/changesethttp/changesethttp.go 80.00% 8 Missing ⚠️
...nal/httpserver/scripthttp/eventshttp/eventshttp.go 81.39% 8 Missing ⚠️
internal/httpserver/endpointwire/endpointwire.go 42.85% 4 Missing ⚠️
internal/httpserver/formwire/formwire.go 0.00% 4 Missing ⚠️
internal/platform/scriptevents/store.go 97.56% 2 Missing ⚠️
internal/platform/scriptstore/runs.go 71.42% 2 Missing ⚠️
internal/endpoint/epcollect/epcollect.go 95.00% 1 Missing ⚠️
internal/endpoint/epcollect/files.go 98.50% 1 Missing ⚠️
internal/endpoint/epform/check.go 97.91% 1 Missing ⚠️
... and 5 more
Additional details and impacted files
@@           Coverage Diff            @@
##             main    #2340    +/-   ##
========================================
  Coverage   94.81%   94.81%            
========================================
  Files        1435     1443     +8     
  Lines      116477   117111   +634     
========================================
+ Hits       110433   111035   +602     
- Misses       6024     6056    +32     
  Partials       20       20            

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cjimti
cjimti merged commit c0eb937 into main Oct 27, 2026
14 checks passed
@cjimti
cjimti deleted the feat/2331-2332-2333-2337-forms-events-uploads branch October 27, 2026 07:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment