Skip to content

feat(automation): add API Request node - #3251

Open
abnercon wants to merge 3 commits into
SEMOSS:devfrom
abnercon:feat/automation-api-request-node
Open

abnercon wants to merge 3 commits into
SEMOSS:devfrom
abnercon:feat/automation-api-request-node

Conversation

@abnercon

@abnercon abnercon commented Nov 3, 2026 •

Copy link
Copy Markdown

Description

Adds an API Request node to SEMOSS Automation: send one HTTP request to an external service and use the response in later nodes. Includes a TestAutomationRequest Pixel so the editor can try a request before the automation runs.

Pairs with the semoss-ui PR that adds the editor (Postman/Bruno-style). Ship both together: the UI rejects node categories it doesn't know, so an older UI cannot load this catalog.

Changes Made

  • New node type api.request in the constants, AutomationNodeType, the catalog, and the validator.
  • AutomationApiRequestExecutor sends the request:
    • all HTTP methods; none/json/form/raw bodies; JSON or text responses, capped at 1 MB with a truncated flag
    • every status code is a normal result; only no response (timeout, unreachable host, blocked address) fails the node
    • redirects are not followed; Location is returned
    • blocks loopback, link-local, private, CGNAT and IPv6 unique-local addresses after DNS resolution, and applies the existing WHITE_LIST_DOMAINS allowlist when set
    • hides Set-Cookie, Authorization and HTTP/2 pseudo-headers in responses
  • Credential headers (Authorization, X-Api-Key, Cookie, ...) must contain a ${...} reference; typed-in secrets are rejected at save time.
  • Optional Connection: a REST function engine the caller can view supplies the host and its saved headers. Validated on save.
  • This node accepts ${...} references embedded in text (Bearer ${token}, /items/${id}); other nodes keep exact-reference-only. Documented in the automation AGENTS.md.
  • TestAutomationRequestReactor (requires EDIT on the project; same validation and destination checks as a real run).
  • CONTEXT.md glossary for the new terms.
  • 27 new unit tests (executor tests run against a local HTTP server).

How to Test

  1. mvn -f pom.xml test -Dtest='AutomationApiRequest*UnitTests,AutomationDefinitionValidatorUnitTests,AutomationNodeCatalogUnitTests'
  2. Build a local Monolith image per the README, run it, and run the semoss-ui dev server against it.
  3. Create an automation, add API request (group "API"), GET https://httpbin.org/get, press Send: expect 200 and the JSON body.
  4. GET https://httpbin.org/status/404: expect a 404 result, not an error. .../redirect-to?url=/get&status_code=302: expect 302 with location.
  5. GET http://169.254.169.254/latest/meta-data/: expect "not allowed to call a private or internal address".
  6. Save a node with a typed Authorization value: expect a save error; Bearer ${token} is accepted.
  7. Save and Run: expect the run to succeed with status, headers, body in the node output.

Notes

  • Verified manually in a local Docker stack (run history, Send, blocked address, redirect, 404, cURL import in the UI). Not verified: the Connection picker (no REST function engine in the test instance) and live truncation (httpbin caps its byte responses at ~100 KB; a unit test covers the 1 MB cap).
  • pom.xml declares maven-compiler-plugin twice, which Maven 3.9.16 rejects; I used a local untracked copy to build and did not change it. AskStringModelEngineResponseUnitTests does not compile on dev (missing fromJson) and was excluded locally.
  • Known gap: DNS is checked before connecting, but the HTTP client resolves the name again, so a hostile DNS server could still race the check.
  • The test Pixel reports errors as "API request node 'test'" because it uses a placeholder node id.

Companion UI PR: SEMOSS/semoss-ui#3840

🤖 Generated with Claude Code

abrincon_deloitte and others added 2 commits October 27, 2026 12:07
Adds an API Request node that sends one HTTP request and publishes the
response, plus a TestAutomationRequest Pixel for trying a request from
the editor.

- All HTTP methods; none/json/form/raw bodies; JSON/text responses
- Every status code is a normal response; only no response fails the node
- Redirects are not followed; Location is returned
- Blocks loopback, link-local, private and CGNAT destinations after DNS
  resolution, and honours the admin domain allowlist
- Credential headers must contain a ${...} reference; an optional
  Connection (REST function engine) supplies host and credentials
- Embedded ${...} references are supported in this node only
- CONTEXT.md glossary for the new terms

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…r hint

Response headers from HTTP/2 services included a ":status" pseudo-header.
Drop pseudo-headers along with Set-Cookie and Authorization. The save-time
credential-header error now suggests ${token}, matching how trigger inputs
are referenced (by name, not under "inputs.").

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@snyk-io

snyk-io Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants