From 61c0d15dfe7377bdf93502b14728f700002e3550 Mon Sep 17 00:00:00 2001 From: Jakub Dzikowski Date: Thu, 6 Aug 2026 16:17:29 +0200 Subject: [PATCH 01/12] Docs: shorten v0.13.0 CHANGELOG Summary for operators Replace the long security-detail bullets with six human-facing headlines; All changes keeps the full technical record. Co-authored-by: Cursor --- CHANGELOG.md | 28 +++++++--------------------- 1 file changed, 7 insertions(+), 21 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 63b2df67..c5fd644b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,27 +8,13 @@ ## Summary -- **Shell steps no longer splice untrusted values into `sh -c`:** every value interpolated into a workflow shell step is shell-quoted first, so a workflow parameter, capture, `for` iterator, or channel payload that contains shell metacharacters is passed to the shell as data and cannot inject a command, including when the value is bound through `jaiph mcp` or `jaiph serve`. -- **`jaiph serve` and `jaiph mcp` no longer go host-only from an inherited `JAIPH_UNSAFE=true`:** the long-lived servers now require explicit consent for unsafe host-only execution on their own command line, so `--unsafe` (or `--yes`) must be passed to run every call on the host with no sandbox. An ambient `JAIPH_UNSAFE=true` inherited from the environment, for example a value left in a shell profile by an earlier host-only `jaiph run`, is refused at startup with `E_UNSAFE_NO_CONSENT` instead of silently disabling the sandbox. When consent is given, the server prints a loud multi-line startup banner that states sandboxing is disabled and every call runs on the host with full filesystem and credential access. Inside a container or Kubernetes pod the container is the sandbox, so the refusal is skipped and a factory VPS / standalone runtime-image deploy still runs host-only without `--unsafe` on the command line. -- **The `jaiph serve` operator token no longer crosses into workflow sandboxes:** the environment-forwarding allowlist now excludes the whole host-only `JAIPH_SERVE_*` family, so `JAIPH_SERVE_TOKEN` and the OIDC and server-config keys stay on the host instead of being forwarded into every Docker container and agent subprocess the server runs. -- **The run audit journal is now tamper-resistant and verified when it is read:** each `run_summary.jsonl` line is chained with a keyed HMAC under a per-run secret that never reaches the workflow's own script or agent subprocesses, so a workflow that rewrites, truncates, or deletes its journal can no longer forge a chain that verifies. The per-run key is now stored outside the workflow-writable run directory, in an operator-side store that no sandbox mount reaches, so a workflow can no longer squat the key path or delete its own tamper evidence, and a keyed run whose key later goes missing fails closed instead of silently passing. A completed journal must also end with a `WORKFLOW_END` marker, so deleting the last lines of a finished journal is rejected instead of leaving a shorter chain that still links and verifies. Run listing, the `GET /v1/runs/{id}/events` snapshot, and OTLP and Sentry export now verify the chain and reject a tampered journal instead of trusting it. -- **`jaiph install` and the library registry now verify integrity instead of trusting-on-first-use:** a remotely fetched registry index is signature-verified against a detached `.minisig` (minisign, `jaiph.pub` embedded as the trust anchor) and rejected when missing, unsigned, or tampered; remote registry and library URLs must use `https://`/`ssh://` (a `http://` or other disallowed scheme is refused before any fetch or clone); every shipped registry entry must now pin a `commit` that the cloned HEAD must match on the first install, `npm run registry:build` refuses to write an index with an unpinned entry, and `jaiph install` refuses a registry name whose entry has no pinned commit unless you pass `--allow-unpinned`; and an optional per-library detached signature is verified fail-closed. -- **A workflow file can no longer weaken the Docker sandbox it runs in:** the entry file's `runtime.docker_image` and any isolation-breaking `runtime.docker_network` value (`host`, `container:*`, `ns:*`) are now host-controlled. When Docker is the active sandbox, a file-declared image is rejected (`E_DOCKER_IMAGE_HOST_ONLY`) and a file-declared `host` / `container:*` / `ns:*` network is rejected (`E_DOCKER_NETWORK_HOST_ONLY`), so a repo- or model-supplied workflow can no longer point the sandbox at an arbitrary image or join the host network namespace while still appearing sandboxed. Host-safe in-file network values (`default`, `none`, a named bridge network) are still honoured, and only the operator's `JAIPH_DOCKER_IMAGE` / `JAIPH_DOCKER_NETWORK` can select an image or an isolation-breaking network. -- **The image presence check no longer runs unhardened image code:** the check that confirms a Docker image contains `jaiph` before a run now uses the same sandbox hardening as the run itself (every capability dropped, no new privileges, a non-root user, and no network) and a non-login shell, so it can no longer source or execute startup and profile scripts baked into a workflow-selected image at a higher privilege than the run. -- **The default sandbox image is now pinned by digest and verified on every run:** the official `ghcr.io/jaiphlang/jaiph-runtime` image ships an expected manifest digest with each release, and every Docker-backed `jaiph run` resolves and checks the local image's digest against it, including on a cache hit, so a re-pointed tag or a poisoned local image cache under the same tag can no longer swap the sandbox rootfs while the run still looks sandboxed. A mismatch fails closed with `E_DOCKER_DIGEST_MISMATCH` and a message that tells you how to re-pull the pinned image, and `JAIPH_DOCKER_IMAGE_DIGEST` lets you pin or override the digest for any image. -- **A workflow file can no longer pull arbitrary host secrets into the Docker sandbox by declaring them:** the entry file's `trusted_envs` keys cross the sandbox allowlist only when the operator opts in with `JAIPH_TRUSTED_ENVS=1`. Absent the opt-in, a file-declared `trusted_envs` is ignored under Docker with a pre-flight warning, so an untrusted or model-edited entry naming `AWS_SECRET_ACCESS_KEY` or `GITHUB_TOKEN` cannot forward that host secret across the allowlist on its own. Host modes have no allowlist to bypass, so they honour the declaration as before, and authoring the entry file is now a trust boundary equal to `--env`. -- **A `sub`-less OIDC token no longer collapses onto one shared identity:** the OIDC principal is the token `sub`, falling back to `client_id` for machine tokens (OAuth2 client-credentials) that omit `sub`, and a verified token carrying neither claim is rejected with `401` instead of authenticating as a shared `unknown` principal. Two machine callers on the same issuer can no longer share one run-visibility bucket or idempotency namespace, so neither can list, read, or cancel the other's runs. -- **Project-local `.jaiph/hooks.json` no longer runs on the host without a workspace-trust decision:** hook commands run in the host CLI process, before and outside any Docker sandbox, so a `/.jaiph/hooks.json` that arrives with a cloned or untrusted repository is now gated behind the operator opt-in `JAIPH_TRUST_PROJECT_HOOKS=1`. Absent the opt-in, `jaiph run`, `jaiph serve`, and `jaiph mcp` ignore the project file with a one-line stderr notice, so a cloned repo cannot execute arbitrary host commands on `workflow_start`. The global `~/.jaiph/hooks.json` is the operator's own and always runs. -- **Release install and the runtime image now verify every download instead of failing open:** the binary installer requires a valid minisign signature, so on a normal host a missing `minisign` aborts the install rather than degrading to checksum-only, an empty `JAIPH_MINISIGN_PUBLIC_KEY` fails closed, and only `JAIPH_ALLOW_UNSIGNED=1` proceeds on checksum alone (finding M-5 removed the earlier `CI` opt-out, so CI installs must make `minisign` available). The `jaiph run`, `jaiph init`, and `jaiph use` bootstraps fetch `docs/install` and its published `install.sha256`, verify the two match, and refuse to run a tampered script instead of piping `curl … | bash`. Every toolchain fetch in `runtime/Dockerfile` now goes through `runtime/fetch-verify.sh` with a required, pinned SHA-256, so a poisoned toolchain CDN fails the build. - -- **CI installs and the `setup-jaiph` action now require a verified release signature:** the installer no longer downgrades to a checksum-only install when `CI` is set, so a missing `minisign` aborts the install on every host, and only an explicit `JAIPH_ALLOW_UNSIGNED=1` proceeds on checksum alone with a prominent warning. The `setup-jaiph` GitHub Actions action installs `minisign` on the runner so the action path always verifies the signature, and the release build now fails when the signing key is unset instead of publishing unsigned artifacts. -- **Credential redaction now covers many more secret names and their encoded forms:** the run journal and every surface that reads it back (`GET /v1/runs/{id}/events`, the OTLP export, the Sentry export, and a failed call's returned `result_text`) redact the value of any env var whose name looks like a credential, which now includes names the earlier four-suffix rule missed such as `AWS_SECRET_ACCESS_KEY`, `STRIPE_SECRET_KEY`, `DB_PASSWORD`, `PASSPHRASE`, and `SSH_PRIVATE_KEY`, and each value is redacted in its base64, hex, and URL-encoded forms as well as its raw form. Redaction still works by literal-substring replacement, so a secret transformed some other way, such as split across output chunks or embedded inside an opaque connection string, is not guaranteed to be caught, and the raw per-step capture files stay sensitive. -- **`jaiph serve` now serves a self-contained Swagger UI:** `/docs` embeds the pinned `swagger-ui-dist` assets in the jaiph binary and serves them from same-origin `/docs/*` paths, so the built-in API UI renders and can invoke workflows with no browser internet access, including on an air-gapped network or behind a Content-Security-Policy that blocks third-party hosts. `JAIPH_SERVE_EXPOSE_DOCS=false` still returns `404` for `/docs`, `/openapi.json`, and the embedded assets. -- **OIDC token verification now pins an explicit signing-algorithm allowlist:** `jaiph serve` accepts an OIDC bearer JWT only when its header names one of the pinned asymmetric algorithms — the RSA (`RS*` / `PS*`), ECDSA (`ES256` / `ES384` / `ES512`), and EdDSA families that standard OIDC providers sign with — and rejects symmetric algorithms (`HS*`), `alg: none`, and the non-recommended secp256k1 curve (`ES256K`) even when the signing key is present in the JWKS. Pinning the allowlist means a future key-type or JWKS change can never make an algorithm-confusion or `alg: none` forgery reachable, though `jose` already rejected those cases today. -- **A host run can now be bounded by a wall-clock timeout and a max-step circuit breaker:** `JAIPH_RUN_TIMEOUT` (seconds) gives a host-mode run — a `jaiph run --unsafe` or host-only run, and the host spawn a `jaiph serve` or `jaiph mcp` call uses — a parent-enforced wall-clock cap that terminates the run's whole process group (`SIGTERM`, then `SIGKILL`) once the budget is reached, so it stops without a manual Ctrl-C where before Ctrl-C was the only automatic stop. Docker mode keeps using `JAIPH_DOCKER_TIMEOUT`. `JAIPH_MAX_STEPS` adds an optional circuit breaker in the runtime that counts every executed step across the whole run, including loop iterations and nested or recursive calls, and aborts a runaway workflow once the count exceeds the cap. Both are off by default. -- **A leaf script step that goes silent is now killed after an idle-output timeout:** when a script step's subprocess produces no stdout or stderr for `JAIPH_STEP_IDLE_KILL_SEC` (default 3600 seconds, one hour; `0` disables), the runtime records a `LOGERR` naming the step and how long it was silent, terminates the step's subprocess (SIGTERM, then SIGKILL), and fails the step, so an overnight run can no longer hang for hours on a stuck command that stopped producing output. Any new output resets the timer, the periodic idle warnings on `JAIPH_STEP_IDLE_WARN_SEC` are unchanged and run on their own independent cadence, and prompt steps still get warnings only. -- **Local-source builds now enforce the lockfile with `npm ci` and exact-pin the one runtime dependency:** the from-source installer (`docs/install`) runs `npm ci` when the checkout has a `package-lock.json`, so a clean-room install uses exactly the versions the lockfile pins instead of letting `npm install` re-resolve caret ranges, and it falls back to `npm install` only when no lockfile is present. The single runtime dependency `jose` is now exact-pinned to `5.10.0` in `package.json` (no `^` caret range); the dev dependencies keep their caret ranges. -- **The runtime sandbox image now pins its base images by digest and its global npm installs by exact version:** every `FROM` in `runtime/Dockerfile` references its base image by an `@sha256:` digest instead of a mutable tag, and the global `npm install -g` of pnpm, yarn, and the Claude Code CLI each pins an exact version through a build ARG, so the built image is reproducible and its registry-sourced layers are attested the same way the direct toolchain downloads already are. A CI check rejects any later edit that reintroduces a bare `FROM` tag or an unpinned global install. +- **Hardened install and supply chain:** release installs and the runtime image fail closed on missing signatures; registry entries are signature-verified and commit-pinned; the sandbox image is digest-pinned and checked on every run. +- **Workflows cannot weaken the sandbox:** a `.jh` file can no longer pick an arbitrary Docker image, join the host network, or pull host secrets via `trusted_envs` without an operator opt-in (`JAIPH_TRUSTED_ENVS=1`). +- **Secrets stay out of agents and logs:** shell steps quote interpolated values; broader credential redaction covers more secret names and encodings; `JAIPH_SERVE_*` tokens never enter workflow sandboxes; project-local hooks need `JAIPH_TRUST_PROJECT_HOOKS=1`. +- **Tamper-evident run journals:** `run_summary.jsonl` is HMAC-chained and verified on read; truncated or rewritten journals fail closed. +- **Safer long-lived servers:** bare-metal `jaiph serve` / `jaiph mcp` refuse ambient `JAIPH_UNSAFE` without `--unsafe` (container/k8s factory deploys unchanged); OIDC principals without `sub` no longer collide; idle script steps and optional host run timeouts stop runaway work. +- **Self-contained API docs:** `jaiph serve /docs` ships Swagger UI in-process — no CDN required. + ## All changes From 00b46230f992645a2cb1b34ec243f8326971fe5c Mon Sep 17 00:00:00 2001 From: Jakub Dzikowski Date: Tue, 25 Aug 2026 15:26:48 +0200 Subject: [PATCH 02/12] Change: execute workflows on the host; remove the Docker sandbox Jaiph is the language and orchestrator, not a sandbox product. Isolation stays an outer wrap (container, pod, CI). Co-authored-by: Cursor --- .dependency-cruiser.cjs | 2 +- .github/workflows/ci.yml | 115 +- .jaiph/ensure_ci_passes.jh | 11 +- .jaiph/gh_ci_passes.jh | 4 +- .jaiph/libs/jaiphlang/artifacts.jh | 3 +- .jaiph/libs/jaiphlang/gh_actions.sh | 2 +- .jaiph/main.jh | 4 +- .jaiph/prepare_release.jh | 13 +- .jaiph/security_review.jh | 9 +- CHANGELOG.md | 4 + README.md | 9 +- design/0001-jaiph-is-the-language.md | 51 + design/2026-05-12-agent-proxy.md | 2 +- design/2026-07-14-mcp-server.md | 2 +- design/2026-07-23-serve-http-api.md | 2 +- docs/_layouts/docs.html | 6 +- docs/agent-analyzability.md | 4 +- docs/agent-auth.md | 33 +- docs/architecture.md | 64 +- docs/artifacts.md | 8 +- docs/cli.md | 53 +- docs/configuration.md | 77 +- docs/contributing.md | 60 +- docs/deploy.md | 64 +- docs/deploy/k8s.yaml | 25 +- docs/env-vars.md | 114 +- docs/first-agent-run.md | 41 +- docs/first-workflow.md | 33 +- docs/grammar.md | 2 +- docs/hooks.md | 9 +- docs/index.html | 36 +- docs/install-from-local.sh | 41 - docs/jaiph-skill.md | 8 +- docs/language.md | 4 +- docs/mcp.md | 29 +- docs/observability.md | 11 +- docs/run | 5 +- docs/sandbox-run.md | 97 - docs/sandboxing.md | 272 -- docs/serve.md | 16 +- docs/setup.md | 2 +- docs/testing.md | 2 +- docs/why-jaiph.md | 12 +- e2e/lib/common.sh | 113 +- e2e/playwright/landing-page.spec.ts | 1 - e2e/test_all.sh | 65 - e2e/tests/07_installer_binary.sh | 8 +- e2e/tests/09_dockerfile_fetch_verify.sh | 145 - e2e/tests/139_agent_credentials_preflight.sh | 73 +- e2e/tests/139_mcp_server_session.sh | 5 +- e2e/tests/140_env_passthrough.sh | 58 +- e2e/tests/141_mcp_docker_sandbox.sh | 174 - e2e/tests/143_docker_toolchain.sh | 47 - e2e/tests/143_step_idle_warn.sh | 1 - e2e/tests/146_trusted_envs.sh | 48 +- e2e/tests/147_serve_http_api.sh | 5 +- e2e/tests/148_standalone_image.sh | 54 - e2e/tests/149_mcp_generation_lifecycle.sh | 9 +- e2e/tests/150_k8s_deploy.sh | 253 -- e2e/tests/151_serve_transports_docker.sh | 124 - e2e/tests/152_shell_injection_serve.sh | 5 +- e2e/tests/153_docker_network_host_control.sh | 144 - e2e/tests/210_standalone_binary.sh | 3 +- e2e/tests/72_docker_run_artifacts.sh | 177 -- e2e/tests/74_docker_lifecycle.sh | 103 - e2e/tests/74b_docker_signal_cleanup.sh | 157 - e2e/tests/74c_docker_prepull.sh | 103 - e2e/tests/74d_docker_snapshot_isolation.sh | 122 - e2e/tests/74e_docker_git_snapshot_content.sh | 109 - e2e/tests/74f_docker_probe_hardening.sh | 99 - e2e/tests/74g_docker_digest_verify.sh | 114 - e2e/tests/75_docker_live_step_output.sh | 130 - e2e/tests/76_docker_failure_parity.sh | 181 -- e2e/tests/77_unsafe_confirm.sh | 53 - e2e/tests/windows_native_smoke.ps1 | 19 +- eslint.config.mjs | 2 +- integration/docs-explanation-task3.test.ts | 95 +- integration/docs-how-to-task4.test.ts | 6 - integration/docs-tutorials-task6.test.ts | 14 +- integration/exec-policy.test.ts | 93 +- integration/mcp-server.test.ts | 86 - integration/otlp-export.test.ts | 1 - integration/release-workflow.test.ts | 84 +- integration/run-summary-jsonl.test.ts | 2 - integration/sample-build/cli-tree.test.ts | 8 +- .../sample-build/recover-handle.test.ts | 26 +- integration/sample-build/run-core.test.ts | 23 +- .../sample-build/run-prompt-agent.test.ts | 9 - .../sample-build/test-advanced.test.ts | 2 - integration/sentry-export.test.ts | 1 - integration/serve-auth.test.ts | 5 +- integration/serve-restart.test.ts | 1 - integration/serve-server.test.ts | 1 - integration/signal-lifecycle.test.ts | 2 - integration/tty-running-timer.test.ts | 5 +- integration/windows-native-smoke.test.ts | 17 +- package.json | 3 +- plugins/vscode/syntaxes/jaiph.tmLanguage.json | 2 +- plugins/vscode/test/fixtures/current.jh | 2 +- plugins/vscode/test/fixtures/regression.jh | 2 + plugins/vscode/test/grammar.test.ts | 7 +- plugins/zed/test/fixtures/current.jh | 2 +- runtime/Dockerfile | 279 -- runtime/fetch-verify.sh | 51 - src/cli/commands/init.ts | 2 +- src/cli/commands/mcp.ts | 26 +- src/cli/commands/run.test.ts | 191 +- src/cli/commands/run.ts | 327 +- src/cli/commands/serve.ts | 23 +- src/cli/run/display.test.ts | 43 +- src/cli/run/display.ts | 18 +- src/cli/run/env.ts | 77 +- src/cli/run/hooks.ts | 8 +- src/cli/run/lifecycle.ts | 7 +- src/cli/run/non-tty-heartbeat.test.ts | 3 - src/cli/run/preflight-credentials.test.ts | 301 +- src/cli/run/preflight-credentials.ts | 81 +- src/cli/run/sandbox-flags.test.ts | 149 - src/cli/run/stderr-handler.test.ts | 4 +- src/cli/run/trusted-envs.test.ts | 79 +- src/cli/run/trusted-envs.ts | 62 +- src/cli/serve/handler.ts | 4 +- src/cli/shared/errors.test.ts | 62 - src/cli/shared/errors.ts | 32 +- src/cli/shared/generation-posture.test.ts | 145 +- src/cli/shared/generation.ts | 11 +- src/cli/shared/log-format.ts | 30 +- src/cli/shared/run-meta.ts | 16 +- src/cli/shared/serve-bootstrap.ts | 26 +- src/cli/shared/server-log-call.test.ts | 16 +- src/cli/shared/server-log.test.ts | 20 +- src/cli/shared/server-log.ts | 33 +- src/cli/shared/startup-posture.ts | 112 +- src/cli/shared/usage.test.ts | 105 +- src/cli/shared/usage.ts | 80 +- src/cli/shared/workflow-call-exec.ts | 77 +- .../shared/workflow-call-run-timeout.test.ts | 10 +- src/cli/shared/workflow-call-types.ts | 33 +- src/cli/shared/workflow-call.test.ts | 12 +- src/cli/shared/workflow-call.ts | 36 +- src/cli/telemetry/otlp.ts | 2 +- src/cli/telemetry/sentry.ts | 4 +- src/config.ts | 9 - src/env-reserved.ts | 26 +- src/format/emit.ts | 17 +- src/parse/metadata.ts | 13 +- src/parse/parse-metadata.test.ts | 24 +- src/parse/workflow-brace.ts | 2 +- src/parse/workflows.ts | 3 - src/runtime/docker-config.ts | 265 -- src/runtime/docker-image.ts | 369 --- src/runtime/docker-inplace.test.ts | 77 - src/runtime/docker-inplace.ts | 130 - src/runtime/docker-run.ts | 493 --- src/runtime/docker-sandbox.ts | 291 -- src/runtime/docker.test.ts | 2810 ----------------- src/runtime/docker.ts | 81 - src/runtime/embedded-assets.test.ts | 2 +- src/runtime/embedded-assets.ts | 2 +- src/runtime/index.ts | 37 +- src/runtime/kernel/env-allowlist.test.ts | 12 +- src/runtime/kernel/env-allowlist.ts | 84 +- src/runtime/kernel/index.ts | 3 +- .../node-workflow-runtime.trusted-env.test.ts | 76 - src/runtime/kernel/portability.test.ts | 2 +- src/runtime/kernel/portability.ts | 7 +- src/runtime/kernel/prompt-backends.ts | 3 +- src/runtime/kernel/prompt-retry.ts | 6 +- src/runtime/kernel/prompt.test.ts | 52 - src/runtime/testing.ts | 6 - src/transpile/compiler-golden.test.ts | 23 +- src/transpile/validate-config.ts | 2 - src/transpile/validate-expr.ts | 2 +- src/types.ts | 8 - .../compiler-txtar/parse-errors-snapshot.json | 22 +- test-fixtures/compiler-txtar/parse-errors.txt | 8 +- test-fixtures/compiler-txtar/valid.txt | 14 +- tools/embed-assets.js | 13 - 178 files changed, 814 insertions(+), 11341 deletions(-) create mode 100644 design/0001-jaiph-is-the-language.md delete mode 100644 docs/sandbox-run.md delete mode 100644 docs/sandboxing.md delete mode 100644 e2e/tests/09_dockerfile_fetch_verify.sh delete mode 100755 e2e/tests/141_mcp_docker_sandbox.sh delete mode 100644 e2e/tests/143_docker_toolchain.sh delete mode 100755 e2e/tests/148_standalone_image.sh delete mode 100755 e2e/tests/150_k8s_deploy.sh delete mode 100755 e2e/tests/151_serve_transports_docker.sh delete mode 100755 e2e/tests/153_docker_network_host_control.sh delete mode 100755 e2e/tests/72_docker_run_artifacts.sh delete mode 100755 e2e/tests/74_docker_lifecycle.sh delete mode 100755 e2e/tests/74b_docker_signal_cleanup.sh delete mode 100755 e2e/tests/74c_docker_prepull.sh delete mode 100755 e2e/tests/74d_docker_snapshot_isolation.sh delete mode 100755 e2e/tests/74e_docker_git_snapshot_content.sh delete mode 100755 e2e/tests/74f_docker_probe_hardening.sh delete mode 100755 e2e/tests/74g_docker_digest_verify.sh delete mode 100644 e2e/tests/75_docker_live_step_output.sh delete mode 100755 e2e/tests/76_docker_failure_parity.sh delete mode 100755 e2e/tests/77_unsafe_confirm.sh delete mode 100644 runtime/Dockerfile delete mode 100755 runtime/fetch-verify.sh delete mode 100644 src/cli/run/sandbox-flags.test.ts delete mode 100644 src/runtime/docker-config.ts delete mode 100644 src/runtime/docker-image.ts delete mode 100644 src/runtime/docker-inplace.test.ts delete mode 100644 src/runtime/docker-inplace.ts delete mode 100644 src/runtime/docker-run.ts delete mode 100644 src/runtime/docker-sandbox.ts delete mode 100644 src/runtime/docker.test.ts delete mode 100644 src/runtime/docker.ts diff --git a/.dependency-cruiser.cjs b/.dependency-cruiser.cjs index dbba7aa0..dea6e3fe 100644 --- a/.dependency-cruiser.cjs +++ b/.dependency-cruiser.cjs @@ -95,7 +95,7 @@ module.exports = { { name: "no-deep-imports-into-runtime", comment: - "Runtime is a deep module: code OUTSIDE the runtime package imports only its public entries — src/runtime/index.ts (production surface) or src/runtime/testing.ts (named test seams for cross-package *.test.ts) — never src/runtime/** internals (docker, docker-inplace, embedded-assets, kernel/*). Add a named re-export to src/runtime/index.ts (production) or src/runtime/testing.ts (test-only seams) instead of reaching in.", + "Runtime is a deep module: code OUTSIDE the runtime package imports only its public entries — src/runtime/index.ts (production surface) or src/runtime/testing.ts (named test seams for cross-package *.test.ts) — never src/runtime/** internals (embedded-assets, kernel/*). Add a named re-export to src/runtime/index.ts (production) or src/runtime/testing.ts (test-only seams) instead of reaching in.", severity: "error", from: { pathNot: "^src/runtime/" }, to: { path: "^src/runtime/", pathNot: "^src/runtime/(index|testing)\\.ts$" }, diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6aefbc2b..4dc97a0c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -38,60 +38,13 @@ jobs: VERSION="$(node -p "require('./package.json').version")" git ls-remote --exit-code https://github.com/jaiphlang/jaiph.git "refs/tags/v${VERSION}" - k8s-manifest: - name: Validate Kubernetes deploy manifest - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v4 - - # `kubectl apply --dry-run=client` still needs an API server for resource - # discovery (RESTMapper), so provision a throwaway kind cluster for it. - # The same cluster then backs the real deploy test below. - - name: Create kind cluster - uses: helm/kind-action@v1 - with: - cluster_name: jaiph-e2e - - # Fast schema gate — cheap, but only proves the manifest parses. - - name: Dry-run apply the standalone deploy manifest - run: kubectl apply --dry-run=client -f docs/deploy/k8s.yaml - - - name: Build runtime image for the deploy test - run: docker build -t jaiph-e2e-runtime:local -f runtime/Dockerfile . - - # Real deployment contract: external Secret gate, pod hardening - # (non-root, no privilege escalation, dropped caps, no SA token, - # read-only rootfs), an authenticated HTTP run, and its journal on the - # writable runs volume. - - name: Deploy and exercise the manifest on kind - run: | - JAIPH_E2E_SKIP_INSTALL=1 \ - JAIPH_E2E_KIND_CLUSTER=jaiph-e2e \ - JAIPH_E2E_DOCKER_IMAGE=jaiph-e2e-runtime:local \ - bash e2e/tests/150_k8s_deploy.sh - e2e: - name: E2E (${{ matrix.os }}, ${{ matrix.label }}) + name: E2E (${{ matrix.os }}) runs-on: ${{ matrix.os }} - env: - # Host/safe split applies on Ubuntu only. macOS runners do not ship Docker the same way — keep host-only there. - # "docker": unset JAIPH_UNSAFE so resolveDockerConfig enables the sandbox (pulls ghcr.io/jaiphlang/jaiph-runtime). - # "host": explicit opt-out, same as a fast local `JAIPH_UNSAFE=true npm run test:e2e`. - JAIPH_UNSAFE: ${{ matrix.jaiph_unsafe }} strategy: fail-fast: false matrix: - include: - - os: ubuntu-latest - label: docker - jaiph_unsafe: "" - - os: ubuntu-latest - label: host - jaiph_unsafe: "true" - - os: macos-latest - label: host - jaiph_unsafe: "true" + os: [ubuntu-latest, macos-latest] steps: - name: Checkout uses: actions/checkout@v4 @@ -101,12 +54,6 @@ jobs: with: node-version: "20" - - name: Build runtime image for Docker E2E - if: matrix.label == 'docker' - run: | - docker build -t jaiph-ci-runtime:local -f runtime/Dockerfile . - echo "JAIPH_DOCKER_IMAGE=jaiph-ci-runtime:local" >> "$GITHUB_ENV" - - name: Run runtime acceptance E2E run: | npm ci @@ -141,11 +88,6 @@ jobs: - name: Setup Bun uses: oven-sh/setup-bun@v2 - - name: Build runtime image for docs sample Docker runs - run: | - docker build -t jaiph-ci-runtime:local -f runtime/Dockerfile . - echo "JAIPH_DOCKER_IMAGE=jaiph-ci-runtime:local" >> "$GITHUB_ENV" - - name: Install dependencies run: npm ci @@ -169,7 +111,7 @@ jobs: fi command -v cursor-agent - - name: Install Claude Code CLI (prompt backend parity with Docker image) + - name: Install Claude Code CLI run: | npm install -g @anthropic-ai/claude-code command -v claude @@ -273,7 +215,6 @@ jobs: $bashScript = @' set -euo pipefail export DEBIAN_FRONTEND=noninteractive - export JAIPH_UNSAFE=true # wsl.exe does not forward the parent runner's env into this Linux # session, so CI (which docs/install treats as "trusted toolchain, # checksum-only OK when minisign is absent") must be re-set here — @@ -382,53 +323,3 @@ jobs: run: | $env:JAIPH_TEST_WINDOWS_EXE = Join-Path $env:GITHUB_WORKSPACE "jaiph-windows-x64.exe" ./e2e/tests/windows_native_smoke.ps1 - - docker-publish: - name: Publish Docker runtime image - needs: [test, e2e, docs-local, e2e-wsl, installer-powershell, windows-native-smoke] - if: github.ref == 'refs/heads/nightly' || startsWith(github.ref, 'refs/tags/v') - runs-on: ubuntu-latest - permissions: - contents: read - packages: write - env: - REGISTRY: ghcr.io - IMAGE_NAME: jaiphlang/jaiph-runtime - steps: - - name: Checkout - uses: actions/checkout@v4 - - - name: Log in to GHCR - uses: docker/login-action@v3 - with: - registry: ${{ env.REGISTRY }} - username: ${{ github.actor }} - password: ${{ secrets.GITHUB_TOKEN }} - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@v3 - - - name: Image tags - id: meta - run: | - if [[ "${GITHUB_REF}" == refs/tags/v* ]]; then - VERSION="${GITHUB_REF_NAME#v}" - echo "tags=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${VERSION},${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest" >> "$GITHUB_OUTPUT" - else - echo "tags=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:nightly" >> "$GITHUB_OUTPUT" - fi - - - name: Build and push - uses: docker/build-push-action@v6 - with: - context: . - file: runtime/Dockerfile - push: true - platforms: linux/amd64,linux/arm64 - tags: ${{ steps.meta.outputs.tags }} - - - name: Verify pushed image contains jaiph - run: | - TAG="$(echo '${{ steps.meta.outputs.tags }}' | cut -d',' -f1)" - docker run --rm --entrypoint sh "${TAG}" -lc "command -v jaiph && jaiph --version" - docker run --rm --cap-drop ALL --entrypoint sh "${TAG}" -lc "command -v jaiph" diff --git a/.jaiph/ensure_ci_passes.jh b/.jaiph/ensure_ci_passes.jh index 1518c82d..71a35535 100755 --- a/.jaiph/ensure_ci_passes.jh +++ b/.jaiph/ensure_ci_passes.jh @@ -11,11 +11,8 @@ script npm_run_test_ci = ``` while IFS= read -r _v; do unset "$_v" 2>/dev/null || true done < <(compgen -e | grep '^JAIPH_' || true) -# Full Docker e2e (incl. kind) is required here — same as GitHub Actions. -# Do NOT re-add JAIPH_E2E_SKIP_DOCKER to dodge a flake; fix the harness -# (named waits, EXIT cleanup, probe flake classification, kind heartbeats). # Heartbeat so JAIPH_STEP_IDLE_KILL_SEC cannot kill a long-but-live test:ci -# when individual e2e scripts go quiet (kind pulls, large docker builds). +# when individual e2e scripts go quiet. ( elapsed=0 while true; do @@ -66,12 +63,6 @@ workflow ensure_ci_passes() { expectations, or removal of obsolete features. - Any test change must be minimal with a clear rationale. - Do NOT add speculative fixes. Fix only what the log shows is broken. - - Do NOT lengthen src/runtime/docker.ts probe timeouts to paper over - Docker Desktop load flakes; fix the e2e harness instead (named - jaiph-run container waits, EXIT cleanup of leftover containers, - surface stderr on failure, probe flake retries / E_DOCKER_PROBE_FAILED). - - Do NOT set JAIPH_E2E_SKIP_DOCKER here — overnight runs the full - suite like GitHub Actions. """ } diff --git a/.jaiph/gh_ci_passes.jh b/.jaiph/gh_ci_passes.jh index 3e6e5c7b..2ccfbc44 100755 --- a/.jaiph/gh_ci_passes.jh +++ b/.jaiph/gh_ci_passes.jh @@ -5,8 +5,8 @@ # loop with an agent until CI is green (or run.recover_limit is hit). # # Run as: -# jaiph run --unsafe --env GITHUB_TOKEN .jaiph/gh_ci_passes.jh -# jaiph run --unsafe --env GITHUB_TOKEN .jaiph/gh_ci_passes.jh -- my-feature-branch +# jaiph run --env GITHUB_TOKEN .jaiph/gh_ci_passes.jh +# jaiph run --env GITHUB_TOKEN .jaiph/gh_ci_passes.jh -- my-feature-branch # # Requires: GH_TOKEN or GITHUB_TOKEN (via --env), jq, git, network access. # Each retry waits on HEAD after commit+push — do not pin an old commit here. diff --git a/.jaiph/libs/jaiphlang/artifacts.jh b/.jaiph/libs/jaiphlang/artifacts.jh index fb68a0cb..175151ef 100644 --- a/.jaiph/libs/jaiphlang/artifacts.jh +++ b/.jaiph/libs/jaiphlang/artifacts.jh @@ -3,8 +3,7 @@ # # Artifact publishing for Jaiph workflows. # Copies files from the workspace into ${JAIPH_ARTIFACTS_DIR} so they -# survive sandbox teardown and are readable on the host at -# .jaiph/runs//artifacts/. +# are readable at .jaiph/runs//artifacts/. # # Usage: # import "jaiphlang/artifacts" as artifacts diff --git a/.jaiph/libs/jaiphlang/gh_actions.sh b/.jaiph/libs/jaiphlang/gh_actions.sh index d193f8cf..d0b43a22 100755 --- a/.jaiph/libs/jaiphlang/gh_actions.sh +++ b/.jaiph/libs/jaiphlang/gh_actions.sh @@ -72,7 +72,7 @@ require_gh_token() { export GH_TOKEN="$GITHUB_TOKEN" return 0 fi - die "GH_TOKEN or GITHUB_TOKEN is required; pass a token explicitly (e.g. jaiph run --unsafe --env GITHUB_TOKEN .jaiph/gh_ci_passes.jh)" + die "GH_TOKEN or GITHUB_TOKEN is required; pass a token explicitly (e.g. jaiph run --env GITHUB_TOKEN .jaiph/gh_ci_passes.jh)" } mark_git_workspace_safe() { diff --git a/.jaiph/main.jh b/.jaiph/main.jh index 72d0dcca..72dde7a8 100755 --- a/.jaiph/main.jh +++ b/.jaiph/main.jh @@ -6,10 +6,10 @@ # helpers in the imported modules stay hidden. # # Serve (HTTP + MCP Streamable HTTP on the same port): -# jaiph serve --inplace -y --env ANTHROPIC_API_KEY --env GITHUB_TOKEN .jaiph/main.jh +# jaiph serve --env ANTHROPIC_API_KEY --env GITHUB_TOKEN .jaiph/main.jh # # MCP over stdio: -# jaiph mcp --inplace -y --env ANTHROPIC_API_KEY --env GITHUB_TOKEN .jaiph/main.jh +# jaiph mcp --env ANTHROPIC_API_KEY --env GITHUB_TOKEN .jaiph/main.jh # # Optional args use "" for the workflow default (MCP/HTTP require every # declared parameter as a string). gh_ci_passes needs GITHUB_TOKEN or GH_TOKEN. diff --git a/.jaiph/prepare_release.jh b/.jaiph/prepare_release.jh index 8bf83798..790f0960 100755 --- a/.jaiph/prepare_release.jh +++ b/.jaiph/prepare_release.jh @@ -225,14 +225,9 @@ workflow review_changelog(version) { combine Summary bullets (dedupe), append All changes (dedupe by title), keep chronological/newest-first within All changes. - Otherwise rename # Unreleased to # ${version} in place. - 4. Fix stale All-changes wording superseded by later work, especially: - - overlay / copy / fuse / JAIPH_DOCKER_NO_OVERLAY → snapshot | inplace - - MCP "in-place by default" → same sandbox truth table as jaiph run - (snapshot default; inplace only with JAIPH_INPLACE) - - Docker Ctrl+C / signal cleanup: snapshot and inplace modes only - 5. Ensure Summary covers the headline themes: sandbox snapshot + git-defined - content, trusted_envs, language sugar (else if, match |), logging, - security hardening, and anything from the merged 0.11.0 tranche (MCP, + 4. Fix stale All-changes wording superseded by later work. + 5. Ensure Summary covers the headline themes: trusted_envs, language sugar (else if, match |), logging, + and anything from the merged 0.11.0 tranche (MCP, Windows, --env, agent.model) when that section is being folded in. 6. Replace the top of CHANGELOG.md with a fresh empty scaffold: # Unreleased @@ -298,7 +293,7 @@ workflow default(arg) { 3. Confirm MINISIGN_SECRET_KEY is set in GitHub Actions secrets (see docs/contributing.md → Release signing). 4. Tag: git tag v${version} - 5. Push branch + tag (tag push triggers docker-publish and release.yml) + 5. Push branch + tag (tag push triggers release.yml) 6. Smoke check: jaiph use ${version} """ return version diff --git a/.jaiph/security_review.jh b/.jaiph/security_review.jh index fa0472a1..a52608b8 100755 --- a/.jaiph/security_review.jh +++ b/.jaiph/security_review.jh @@ -38,7 +38,7 @@ script security_review_tasks_path = `echo ".jaiph/tmp/security_review_queue_task const reviewer_role = """ You are a senior security engineer reviewing Jaiph — a workflow DSL, - TypeScript CLI/runtime, Docker sandbox, and agent-backend runner that + TypeScript CLI/runtime, and agent-backend runner that executes tools and scripts on behalf of users. Methodology: follow OWASP Agentic Security Initiative (ASI) Top 10 as @@ -50,14 +50,13 @@ const reviewer_role = """ Jaiph attack surface to prioritize: - Prompt / agent backends (injection into tool/shell execution) - Script and shell step execution (command injection, unsafe spawn) - - Docker sandbox (mount allowlist, env allowlist, caps, isolation escape) + - Host execution of workflows (no jaiph-managed sandbox; outer wrap is the operator's) - Secrets and credentials in env, logs, artifacts, run summaries - - Privilege / --unsafe / permission-mode bypass paths - Supply chain of binaries, installers, skills, and libraries - Auditability of runs (events, artifacts, tamper resistance) Severity scale: - - HIGH: directly exploitable; leads to RCE, sandbox escape, secret + - HIGH: directly exploitable; leads to RCE, host compromise, secret exfiltration, or auth/policy bypass. - MEDIUM: exploitable under specific conditions, significant impact. - LOW: defense-in-depth gaps or low-impact weaknesses. @@ -151,7 +150,7 @@ workflow review_scope(mode, scope_detail, report_file) { workflow review_codebase(report_file) { const scope_detail = """ Review the ENTIRE repository against OWASP ASI Top 10. Explore agent - backends, prompt paths, script/shell execution, Docker sandbox, + backends, prompt paths, script/shell execution, host execution, env/secrets handling, artifacts/run logs, installers, and skills. Do not limit yourself to a diff — this is a full codebase scan. """ diff --git a/CHANGELOG.md b/CHANGELOG.md index c5fd644b..0f858635 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,8 +2,12 @@ ## Summary +- **Host-only execution:** `jaiph run`, `jaiph mcp`, and `jaiph serve` execute on the host. The Docker sandbox driver, runtime image, and sandbox flags/environment variables are removed. Isolation is an outer concern — wrap `jaiph` if you want a container or kernel sandbox. + ## All changes +- **Removed — first-party Docker sandbox:** no Docker driver, no digest-pinned runtime image, no snapshot/inplace modes, no `--unsafe` / `--inplace` / `--yes` sandbox consent, no `JAIPH_UNSAFE` / `JAIPH_INPLACE*` / `JAIPH_DOCKER_*`, no in-file `runtime.docker_*`. `prompt` env scrub, journal redaction, `--env`, and `trusted_envs` stay. + # 0.13.0 ## Summary diff --git a/README.md b/README.md index 97e6012f..5e40b550 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,8 @@ # ![Jaiph](docs/logo.png) -[jaiph.org](https://jaiph.org) · [Your first workflow](docs/first-workflow.md) · [Your first agent + sandboxed run](docs/first-agent-run.md) · [Install & switch versions](docs/setup.md) · [Agent Skill](https://raw.githubusercontent.com/jaiphlang/jaiph/refs/heads/main/docs/jaiph-skill.md) · [Architecture](docs/architecture.md) · [CLI](docs/cli.md) · [Contributing](docs/contributing.md) +[jaiph.org](https://jaiph.org) · [Your first workflow](docs/first-workflow.md) · [Your first agent run](docs/first-agent-run.md) · [Install & switch versions](docs/setup.md) · [Agent Skill](https://raw.githubusercontent.com/jaiphlang/jaiph/refs/heads/main/docs/jaiph-skill.md) · [Architecture](docs/architecture.md) · [CLI](docs/cli.md) · [Contributing](docs/contributing.md) -> **Docs note:** The Jaiph documentation site follows the [Diátaxis](https://diataxis.fr/) framework. Tutorials: [Your first workflow](docs/first-workflow.md), [Your first agent + sandboxed run](docs/first-agent-run.md). How-to: [Install & switch versions](docs/setup.md), [Run in a Docker sandbox](docs/sandbox-run.md), [Authenticate agent backends](docs/agent-auth.md), [Configure backend & model](docs/configure-backend.md), [Add a hook](docs/hooks.md), [Use & publish a library](docs/libraries.md), [Save artifacts](docs/artifacts.md), [Write & run tests](docs/testing.md), [Serve workflows as MCP tools](docs/mcp.md), [Serve workflows over HTTP](docs/serve.md), [Export traces to an OTLP collector](docs/observability.md), [Deploy the runtime image standalone](docs/deploy.md). Reference: [CLI](docs/cli.md), [Configuration](docs/configuration.md), [Grammar](docs/grammar.md), [Language](docs/language.md), [Environment variables](docs/env-vars.md). Explanation: [Why Jaiph](docs/why-jaiph.md), [Architecture](docs/architecture.md), [Sandboxing](docs/sandboxing.md), [Inbox & Dispatch](docs/inbox.md), [Async Handles](docs/spec-async-handles.md). Contributor: [Contributing](docs/contributing.md), [Agent Skill](https://raw.githubusercontent.com/jaiphlang/jaiph/refs/heads/main/docs/jaiph-skill.md). +> **Docs note:** The Jaiph documentation site follows the [Diátaxis](https://diataxis.fr/) framework. Tutorials: [Your first workflow](docs/first-workflow.md), [Your first agent run](docs/first-agent-run.md). How-to: [Install & switch versions](docs/setup.md), [Authenticate agent backends](docs/agent-auth.md), [Configure backend & model](docs/configure-backend.md), [Add a hook](docs/hooks.md), [Use & publish a library](docs/libraries.md), [Save artifacts](docs/artifacts.md), [Write & run tests](docs/testing.md), [Serve workflows as MCP tools](docs/mcp.md), [Serve workflows over HTTP](docs/serve.md), [Export traces to an OTLP collector](docs/observability.md), [Deploy jaiph](docs/deploy.md). Reference: [CLI](docs/cli.md), [Configuration](docs/configuration.md), [Grammar](docs/grammar.md), [Language](docs/language.md), [Environment variables](docs/env-vars.md). Explanation: [Why Jaiph](docs/why-jaiph.md), [Architecture](docs/architecture.md), [Inbox & Dispatch](docs/inbox.md), [Async Handles](docs/spec-async-handles.md). Contributor: [Contributing](docs/contributing.md), [Agent Skill](https://raw.githubusercontent.com/jaiphlang/jaiph/refs/heads/main/docs/jaiph-skill.md). --- @@ -24,13 +24,12 @@ - **Rules and scripts** — Rules stay structured (no raw shell lines); **`script`** steps run bash or polyglot code as subprocesses. - **Agents** — Backends include Cursor, Claude, Codex (HTTP), or a custom `agent.command`. - **Testing** — `*.test.jh` files run in-process (`jaiph test`) with mocks and `expect_*` assertions ([Write & run tests](docs/testing.md)). -- **Safety and inspectability** — Docker-backed sandbox for **`jaiph run`** (env-controlled; see [Sandboxing](docs/sandboxing.md) and [Run in a Docker sandbox](docs/sandbox-run.md)); live **`__JAIPH_EVENT__`** on stderr and durable **`.jaiph/runs/`** artifacts ([Architecture](docs/architecture.md)). +- **Safety and inspectability** — live **`__JAIPH_EVENT__`** on stderr and durable **`.jaiph/runs/`** artifacts ([Architecture](docs/architecture.md)). Isolation of the process from the rest of the machine is an outer concern: wrap `jaiph` in your own container, pod, or CI runner if you want a sandbox ([Deploy jaiph](docs/deploy.md)). - **Tooling** — `jaiph compile`, `jaiph format`, `jaiph install` / `.jaiph/libs/` ([Use & publish a library](docs/libraries.md)), and optional `hooks.json` ([CLI](docs/cli.md), [Add a hook](docs/hooks.md)). - **MCP server** — `jaiph mcp ./tools.jh` serves a file's workflows as [MCP](https://modelcontextprotocol.io/) tools over stdio, so any MCP client (Claude Code, Cursor) can call tested Jaiph workflows as tools ([Serve workflows as MCP tools](docs/mcp.md)). - **HTTP API** — `jaiph serve ./tools.jh` serves the same workflows over HTTP with a generated OpenAPI 3.1 document and a browser Swagger UI at `/docs`, so any HTTP client (CI, Kubernetes, another service) can invoke them and inspect runs. Production auth is either a static single-operator bearer token or OIDC/JWT with per-user identity and `invoke` / `inspect` / `cancel` scope authorization, and every run is audit-attributed to its principal and correlation id ([Serve workflows over HTTP](docs/serve.md)). - **OpenTelemetry** — set the standard `OTEL_EXPORTER_OTLP_ENDPOINT` and each run exports one span tree (workflow → steps → prompts) to any OTLP collector — Grafana Tempo, Honeycomb, Datadog. Host-side, end-of-run, credential-redacted, zero new dependencies, never load-bearing ([Export traces to an OTLP collector](docs/observability.md)). - **Sentry error reporting** — set the standard `SENTRY_DSN` and every failed run (nonzero exit or a signal) is pushed to Sentry as one error event — workflow, failing step, a redacted output excerpt, and a run-dir pointer — so operators get alerting and grouping without scraping run dirs. Host-side, redacted, zero new dependencies, never load-bearing; successful runs send nothing ([Report failed runs to Sentry](docs/observability.md#report-failed-runs-to-sentry)). -- **Standalone deployment** — the published `ghcr.io/jaiphlang/jaiph-runtime` image bakes `JAIPH_UNSAFE=true`, so `docker run … jaiph run flow.jh` (or a Kubernetes pod) runs workflows directly — put credentials plus `.jh` files and go, no host jaiph process and no Docker daemon inside the container. Here the container/pod boundary *is* the sandbox — there is no jaiph-managed isolation ([Deploy the runtime image standalone](docs/deploy.md)). ## Core components @@ -123,7 +122,7 @@ workflow default(task) { ./main.jh "add user authentication" ``` -For the full language reference, see [Grammar](docs/grammar.md) and [Language](docs/language.md). For install, libraries, sandboxing, hooks, testing, and artifacts, see the How-to quadrant: [Install & switch versions](docs/setup.md), [Use & publish a library](docs/libraries.md), [Run in a Docker sandbox](docs/sandbox-run.md), [Add a hook](docs/hooks.md), [Write & run tests](docs/testing.md), [Save artifacts](docs/artifacts.md). New to Jaiph? Start with the tutorials: [Your first workflow](docs/first-workflow.md) and [Your first agent + sandboxed run](docs/first-agent-run.md). Or visit [jaiph.org](https://jaiph.org). +For the full language reference, see [Grammar](docs/grammar.md) and [Language](docs/language.md). For install, libraries, hooks, testing, and artifacts, see the How-to quadrant: [Install & switch versions](docs/setup.md), [Use & publish a library](docs/libraries.md), [Add a hook](docs/hooks.md), [Write & run tests](docs/testing.md), [Save artifacts](docs/artifacts.md). New to Jaiph? Start with the tutorials: [Your first workflow](docs/first-workflow.md) and [Your first agent run](docs/first-agent-run.md). Or visit [jaiph.org](https://jaiph.org). ## Start here diff --git a/design/0001-jaiph-is-the-language.md b/design/0001-jaiph-is-the-language.md new file mode 100644 index 00000000..e8a34823 --- /dev/null +++ b/design/0001-jaiph-is-the-language.md @@ -0,0 +1,51 @@ +# ADR 0001 — Jaiph is the language, not a sandbox product + +*Status: accepted* +*Date (UTC): 2026-08-25* + +## Decision + +Jaiph is a small workflow language and orchestrator: `rule`, `script`, `prompt`, `workflow`, compile, run, test, format, durable artifacts. + +Jaiph is not a sandbox product. It does not own a container runtime, a kernel policy engine, or a toolchain image. + +`jaiph run`, `jaiph mcp`, and `jaiph serve` execute on the host. There is no sandbox mode, no unsafe flag, and no sandbox environment variables. Isolation of the process from the rest of the machine is an **outer** concern (Docker, nono, k8s, CI, Codespaces). The operator wraps `jaiph`, or the environment already is the sandbox. + +## Why + +`docs/sandboxing.md` already said the sandbox is a deployment choice, not a programming model. The code ignored that: default-on Docker, a digest-pinned image, snapshot/inplace modes, interrupt teardown, and a test corpus that dwarfs several language features. + +That work has no language semantics. It made the repo hard to maintain and split the product vision: every `run` / `mcp` / `serve` change also had to be a sandbox change. + +The k8s/standalone path already set `JAIPH_UNSAFE=true` because the outer container is the real sandbox. `jaiph test` never used Docker. Native Windows never got a sandbox. The language already ran without one. + +A leftover “unsafe” flag or `JAIPH_DOCKER_*` family would keep the sandbox product alive as a ghost. Cut it. + +## What stays in Jaiph (invariants) + +These are orchestration, not kernel: + +- `prompt` subprocess env is fail-closed. Injected secrets (`--env`, `trusted_envs`) reach trusted `run` steps only. +- Run journal redaction. +- Compiler: `rule` purity, `W_PROMPT_IN_SHELL`, a `.jh` file cannot disable host secret policy by itself. + +## What is out + +Removed, not deprecated: + +- Docker driver, runtime image, digest pin, snapshot/inplace, confirmation prompts +- `--unsafe`, `--inplace`, `--yes` as sandbox consent, `JAIPH_UNSAFE`, `JAIPH_INPLACE*`, `JAIPH_DOCKER_*`, in-file `runtime.docker_*` +- Agent credential proxy, hostname allowlists, nono/Landlock/Seatbelt adapters +- Any feature whose primary purpose is process isolation + +## Product filter + +A change lands only if it makes `.jh` files easier to compile, run, test, or understand. + +`jaiph mcp` and `jaiph serve` are adapters around `run`. Bugfixes only unless a later ADR says otherwise. Registry, install, and deploy stay as they are; they are not the vision. + +## Consequences + +- `why-jaiph.md` commitment “Sandbox by default” is deleted. +- Host execution is the language runtime. Document how to wrap with an outer sandbox if the operator wants one. +- Revisit a first-party driver only if operators will not wrap *and* bare-host `prompt` is an actual incident pattern. Speculation is not enough. diff --git a/design/2026-05-12-agent-proxy.md b/design/2026-05-12-agent-proxy.md index a5d428e7..775bb62b 100644 --- a/design/2026-05-12-agent-proxy.md +++ b/design/2026-05-12-agent-proxy.md @@ -2,7 +2,7 @@ *Phantom Token credential proxy for the jaiph Docker sandbox. Container holds only a placeholder; real credentials live on the host and never cross the sandbox boundary.* -**Status:** design — ready for implementation +**Status:** rejected — see [ADR 0001](0001-jaiph-is-the-language.md). Jaiph does not own a sandbox; there is no credential proxy. **Date (UTC):** 2026-05-12 ## Problem diff --git a/design/2026-07-14-mcp-server.md b/design/2026-07-14-mcp-server.md index 2512ebb3..c672058d 100644 --- a/design/2026-07-14-mcp-server.md +++ b/design/2026-07-14-mcp-server.md @@ -2,7 +2,7 @@ *`jaiph mcp ` serves the file's workflows as MCP tools over stdio. Any MCP client (Claude Code, Claude Desktop, Cursor) can call tested, deterministic Jaiph workflows as tools — a `.jh` file becomes an MCP server with zero boilerplate.* -**Status:** design — ready for implementation (an MVP was spiked and verified end-to-end; this doc records the verified contracts) +**Status:** shipped. Sandbox posture in this doc is superseded by [ADR 0001](0001-jaiph-is-the-language.md) (host-only; no Docker driver). **Date (UTC):** 2026-07-14 ## Problem diff --git a/design/2026-07-23-serve-http-api.md b/design/2026-07-23-serve-http-api.md index c94b2bc8..113d9459 100644 --- a/design/2026-07-23-serve-http-api.md +++ b/design/2026-07-23-serve-http-api.md @@ -2,7 +2,7 @@ *`jaiph serve ` serves the file's workflows as an HTTP API with a generated OpenAPI 3.1 document and an embedded Swagger UI. Anything that speaks HTTP — a CI job, a Kubernetes deployment, another service, a human with a browser — can invoke tested workflows and inspect their runs, without an MCP client or a local jaiph install.* -**Status:** design — ready for implementation (tasks queued in QUEUE.md) +**Status:** shipped. Sandbox posture in this doc is superseded by [ADR 0001](0001-jaiph-is-the-language.md) (host-only; no Docker driver). **Date (UTC):** 2026-07-23 ## Problem diff --git a/docs/_layouts/docs.html b/docs/_layouts/docs.html index 64110341..70422073 100644 --- a/docs/_layouts/docs.html +++ b/docs/_layouts/docs.html @@ -44,10 +44,9 @@
  • jaiph.org
  • Tutorials
  • Your first workflow
    • -
  • Your first agent + sandboxed run
    • +
  • Your first agent run
  • How-to guides
  • Install & switch versions
    • -
  • Run in a Docker sandbox
  • Authenticate agent backends
  • Configure backend & model
  • Add a hook
    • @@ -57,7 +56,7 @@
  • Serve workflows as MCP tools
  • Serve workflows over HTTP
  • Export traces (OTLP)
    • -
  • Deploy the runtime image
    • +
  • Deploy jaiph
  • Reference
  • CLI
  • Configuration
    • @@ -67,7 +66,6 @@
  • Explanation
  • Architecture
  • Agent analyzability
    • -
  • Sandboxing
  • Inbox & Dispatch
  • Async Handles
  • Why Jaiph
    • diff --git a/docs/agent-analyzability.md b/docs/agent-analyzability.md index dcbc3b62..d7cf54bb 100644 --- a/docs/agent-analyzability.md +++ b/docs/agent-analyzability.md @@ -64,7 +64,7 @@ Each package is a **deep module**: narrow public surface, large private capabili | Parse | `src/parser.ts` (sole external entry; re-exports the intentional public API — no `export *` barrel) | `src/parse/**` | | Format | `src/format/index.ts` (sole external entry; re-exports the intentional formatter API, no `export *` barrel) | other `src/format/**` | | Transpile | `src/transpiler.ts` (sole external entry; re-exports the compile/validate surface plus the full module-graph API, no `export *` barrel) | `module-graph.ts`, `validate-*.ts`, emit internals, etc. | -| Runtime | `src/runtime/index.ts` (launch, docker, runner, `buildRuntimeGraph`, shared types/helpers intended for CLI); `src/runtime/testing.ts` is a second entry for named test seams that cross-package `*.test.ts` files reach (kept off the production entry) | `src/runtime/kernel/**` and other internals | +| Runtime | `src/runtime/index.ts` (launch, runner, `buildRuntimeGraph`, shared types/helpers intended for CLI); `src/runtime/testing.ts` is a second entry for named test seams that cross-package `*.test.ts` files reach (kept off the production entry) | `src/runtime/kernel/**` and other internals | | CLI | `src/cli/index.ts` plus per-slice entries under `src/cli//` as needed | slice-private files | **Rule.** Code **outside** a package imports **only** that package’s public entry. Code **inside** a package may import siblings freely, subject to no-cycles, fan-out, and file-size caps. @@ -118,7 +118,7 @@ These are **guardrails**, not conventions. Violations fail CI. **Baseline policy.** If the tree already violates a new rule, do **not** weaken the rule. Commit a dependency-cruiser known-violations baseline (and an explicit ESLint grandfather list) so **new** violations fail while old ones are tracked. Follow-up work removes baseline entries; it does not relax severity. -**Landed today.** `.dependency-cruiser.cjs` and `npm run arch:check` now enforce `no-circular` and the layer DAG, meaning each layer's rule against upward imports, including the exception that lets runtime reuse compile only through the single public entry `src/transpiler.ts`. Orphan modules are reported as a warning. Pre-existing violations are grandfathered in `.dependency-cruiser-known-violations.json` and passed to the check with `--ignore-known`, so a new cycle or upward import fails the build; that baseline is now **empty** — every originally-grandfathered edge was fixed rather than kept, so no import-graph violations remain tracked. `eslint.config.mjs` and `npm run lint` now enforce the two caps below on `src/**/*.ts`: `import/max-dependencies` at 8 (type imports ignored) and `max-lines` at 400 (blank and comment lines skipped). Test files are out of scope, because they legitimately import many modules and run long. Most files that once exceeded a cap were split into sibling modules in the same directory and now pass under the global caps with no override. The four largest remaining files keep a per-file override in `eslint.config.mjs` that turns off only the rule they break, each with a justification naming why splitting it is larger follow-up work, and the global cap is never raised, so any new violation still fails. Deep imports past the parse public entry (`src/parser.ts`) are now enforced by the `no-deep-imports-into-parse` rule, and every production call site routes through the entry (the former `validate-string.ts` → `parse/core.ts` baseline is gone: the interpolation validator moved into parse, see below). Deep imports past the transpile public entry are now enforced by the `no-deep-imports-into-transpile` rule: code outside `src/transpile/` imports only the **single** public entry `src/transpiler.ts` (the compile/validate surface plus the full module-graph API: `buildScripts*`, `loadModuleGraph`/`readModuleGraph`/`writeModuleGraph`, `collectDiagnostics`, `walkjhFiles`, `ModuleGraph` types, …) — `src/transpile/module-graph.ts` is no longer a second door, so runtime reaches the graph API through `src/transpiler.ts` too and `layer3-runtime-only-transpile-public-graph` now forbids every `runtime` → `src/transpile/**` edge; the CLI `collectDiagnostics`/`walkjhFiles` call sites were retargeted to the entry, and the former parse→transpile leak is gone: `validateJaiphStringContent`/`extractInlineCaptures` (which need `parseCallRef`) moved down into `src/parse/validate-string-content.ts`, so `parse/metadata.ts` uses a parse sibling and `transpile/validate-string.ts` re-exports them through `src/parser.ts` — no production file under `src/parse/` imports `src/transpile/`. The runtime slice now has a public entry too: `src/runtime/index.ts` re-exports the curated CLI-facing surface (graph construction, launch/runner, the Docker sandbox, emit/redact/portability helpers, embedded assets, and run-tree param display) and `no-deep-imports-into-runtime` fails any outside import that reaches a `src/runtime/**` internal. The runtime→CLI leak is gone: `buildStepDisplayParamPairs` moved out of `src/cli/commands/format-params.ts` into `src/runtime/kernel/format-params.ts` (re-exported through the public entry), so no production runtime file imports `src/cli/**` and there are zero baselined `runtime`→`cli` edges. Every production CLI call site that reached a runtime internal (docker, emit, portability, redact, runner, launch, embedded-assets) was retargeted to `src/runtime/index.ts`; the former `src/config.ts` → `runtime/kernel/runtime-arg-parser` leak is gone too — the pure `interpolate` helper moved down into `src/config.ts` (which `runtime-arg-parser` now imports downward and re-exports), so `config.ts` imports nothing from `src/runtime/`. The former cross-package test-seam imports (`_dockerExec`, `_dockerSpawn`, `_inplacePrompt`, `CHAIN_GENESIS`, `chainHmac`, `RuntimeEventEmitter`) were retargeted to a second named public entry `src/runtime/testing.ts` (allowlisted beside `index.ts` in the `no-deep-imports-into-runtime` rule), so those seams stay off the production `index.ts` while no test reaches a raw `src/runtime/**` path; there are now **zero** baselined `no-deep-imports-into-runtime` edges. Two upward test edges were also cleared by moving the test to its correct layer rather than baselining: the parser-error snapshot test that needs `loadModuleGraph` moved `src/parse/` → `src/transpile/`, and the compile→runtime graph-reuse test that needs `buildRuntimeGraph` moved `src/transpile/` → `src/runtime/`. The format slice now has a public entry too: `src/format/index.ts` re-exports the formatter API (`emitModule` and the `EmitOptions` type) and `no-deep-imports-into-format` fails any outside import that reaches a `src/format/**` internal such as `emit.ts`. The one outside call site (`src/cli/commands/format.ts`) was retargeted to the entry, and format keeps importing only parse and types, so no format source imports `src/cli`, `src/runtime`, or `src/transpile`. CLI slice isolation is now enforced too: `no-cross-cli-slice-imports` fails any import from a **peer** slice (`run`, `serve`, `mcp`, `exec`, `telemetry`) into another slice's private tree, using a `$1` path-group backreference so same-slice imports and imports of `src/cli/shared/**` (or lower-layer public entries) stay allowed. `commands` is the **composition root** and is deliberately absent from the rule's `from` set: it wires the other slices together (each subcommand launches its feature), so `commands` → slice imports are allowed rather than baselined. The one back-edge that was a shared display helper (`run/display.ts` → `commands/format-params.ts`) was fixed earlier by moving `format-params.ts` into `src/cli/shared/`. There are now **zero** baselined `no-cross-cli-slice-imports` edges. The former 17 peer feature-composition edges (`serve` mounting `mcp` tools and `exec` over HTTP, `exec` reusing `run` lifecycle and `telemetry`) were not domain contracts of their home slices but shared CLI infrastructure misfiled inside peer slices — `shared/generation.ts` already reached up into `exec/call`, `mcp/tools`, and three `run/*` modules, an inverted dependency the baseline hid. The fix moves that infrastructure down into `src/cli/shared`: the MCP-protocol engine `mcp/server.ts` → `shared/mcp-server.ts` and `mcp/tools.ts` → `shared/mcp-tools.ts` (used by both the `jaiph mcp` stdio subcommand and `jaiph serve` over HTTP), and the workflow-call executor `exec/call.ts` → `shared/workflow-call.ts` (`callWorkflow`, `WorkflowCallResult`/`WorkflowCallContext`, used by `commands/mcp`, `commands/serve`, `serve/handler`, and `shared/generation`). Because `shared` is not in the rule's `from` set, `shared/workflow-call.ts` may import `run/*` and `telemetry/otlp` downward without a peer violation, so the run/telemetry primitives stay put. The `mcp` and `exec` slice directories no longer exist — their concerns became shared infrastructure — while the peer-slice regex still names them so a reintroduced private tree is still guarded. Zero baselined slice edges originate from `commands`. With parse, transpile, runtime, format, and the CLI slices all gated, no deep-import work remains queued: every layer sits behind a public-entry rule and the dependency-cruiser baseline is empty. +**Landed today.** `.dependency-cruiser.cjs` and `npm run arch:check` now enforce `no-circular` and the layer DAG, meaning each layer's rule against upward imports, including the exception that lets runtime reuse compile only through the single public entry `src/transpiler.ts`. Orphan modules are reported as a warning. Pre-existing violations are grandfathered in `.dependency-cruiser-known-violations.json` and passed to the check with `--ignore-known`, so a new cycle or upward import fails the build; that baseline is now **empty** — every originally-grandfathered edge was fixed rather than kept, so no import-graph violations remain tracked. `eslint.config.mjs` and `npm run lint` now enforce the two caps below on `src/**/*.ts`: `import/max-dependencies` at 8 (type imports ignored) and `max-lines` at 400 (blank and comment lines skipped). Test files are out of scope, because they legitimately import many modules and run long. Most files that once exceeded a cap were split into sibling modules in the same directory and now pass under the global caps with no override. The four largest remaining files keep a per-file override in `eslint.config.mjs` that turns off only the rule they break, each with a justification naming why splitting it is larger follow-up work, and the global cap is never raised, so any new violation still fails. Deep imports past the parse public entry (`src/parser.ts`) are now enforced by the `no-deep-imports-into-parse` rule, and every production call site routes through the entry (the former `validate-string.ts` → `parse/core.ts` baseline is gone: the interpolation validator moved into parse, see below). Deep imports past the transpile public entry are now enforced by the `no-deep-imports-into-transpile` rule: code outside `src/transpile/` imports only the **single** public entry `src/transpiler.ts` (the compile/validate surface plus the full module-graph API: `buildScripts*`, `loadModuleGraph`/`readModuleGraph`/`writeModuleGraph`, `collectDiagnostics`, `walkjhFiles`, `ModuleGraph` types, …) — `src/transpile/module-graph.ts` is no longer a second door, so runtime reaches the graph API through `src/transpiler.ts` too and `layer3-runtime-only-transpile-public-graph` now forbids every `runtime` → `src/transpile/**` edge; the CLI `collectDiagnostics`/`walkjhFiles` call sites were retargeted to the entry, and the former parse→transpile leak is gone: `validateJaiphStringContent`/`extractInlineCaptures` (which need `parseCallRef`) moved down into `src/parse/validate-string-content.ts`, so `parse/metadata.ts` uses a parse sibling and `transpile/validate-string.ts` re-exports them through `src/parser.ts` — no production file under `src/parse/` imports `src/transpile/`. The runtime slice now has a public entry too: `src/runtime/index.ts` re-exports the curated CLI-facing surface (graph construction, launch/runner, emit/redact/portability helpers, embedded assets, and run-tree param display) and `no-deep-imports-into-runtime` fails any outside import that reaches a `src/runtime/**` internal. The runtime→CLI leak is gone: `buildStepDisplayParamPairs` moved out of `src/cli/commands/format-params.ts` into `src/runtime/kernel/format-params.ts` (re-exported through the public entry), so no production runtime file imports `src/cli/**` and there are zero baselined `runtime`→`cli` edges. Every production CLI call site that reached a runtime internal (emit, portability, redact, runner, launch, embedded-assets) was retargeted to `src/runtime/index.ts`; the former `src/config.ts` → `runtime/kernel/runtime-arg-parser` leak is gone too — the pure `interpolate` helper moved down into `src/config.ts` (which `runtime-arg-parser` now imports downward and re-exports), so `config.ts` imports nothing from `src/runtime/`. The former cross-package test-seam imports (`CHAIN_GENESIS`, `chainHmac`, `RuntimeEventEmitter`) were retargeted to a second named public entry `src/runtime/testing.ts` (allowlisted beside `index.ts` in the `no-deep-imports-into-runtime` rule), so those seams stay off the production `index.ts` while no test reaches a raw `src/runtime/**` path; there are now **zero** baselined `no-deep-imports-into-runtime` edges. Two upward test edges were also cleared by moving the test to its correct layer rather than baselining: the parser-error snapshot test that needs `loadModuleGraph` moved `src/parse/` → `src/transpile/`, and the compile→runtime graph-reuse test that needs `buildRuntimeGraph` moved `src/transpile/` → `src/runtime/`. The format slice now has a public entry too: `src/format/index.ts` re-exports the formatter API (`emitModule` and the `EmitOptions` type) and `no-deep-imports-into-format` fails any outside import that reaches a `src/format/**` internal such as `emit.ts`. The one outside call site (`src/cli/commands/format.ts`) was retargeted to the entry, and format keeps importing only parse and types, so no format source imports `src/cli`, `src/runtime`, or `src/transpile`. CLI slice isolation is now enforced too: `no-cross-cli-slice-imports` fails any import from a **peer** slice (`run`, `serve`, `mcp`, `exec`, `telemetry`) into another slice's private tree, using a `$1` path-group backreference so same-slice imports and imports of `src/cli/shared/**` (or lower-layer public entries) stay allowed. `commands` is the **composition root** and is deliberately absent from the rule's `from` set: it wires the other slices together (each subcommand launches its feature), so `commands` → slice imports are allowed rather than baselined. The one back-edge that was a shared display helper (`run/display.ts` → `commands/format-params.ts`) was fixed earlier by moving `format-params.ts` into `src/cli/shared/`. There are now **zero** baselined `no-cross-cli-slice-imports` edges. The former 17 peer feature-composition edges (`serve` mounting `mcp` tools and `exec` over HTTP, `exec` reusing `run` lifecycle and `telemetry`) were not domain contracts of their home slices but shared CLI infrastructure misfiled inside peer slices — `shared/generation.ts` already reached up into `exec/call`, `mcp/tools`, and three `run/*` modules, an inverted dependency the baseline hid. The fix moves that infrastructure down into `src/cli/shared`: the MCP-protocol engine `mcp/server.ts` → `shared/mcp-server.ts` and `mcp/tools.ts` → `shared/mcp-tools.ts` (used by both the `jaiph mcp` stdio subcommand and `jaiph serve` over HTTP), and the workflow-call executor `exec/call.ts` → `shared/workflow-call.ts` (`callWorkflow`, `WorkflowCallResult`/`WorkflowCallContext`, used by `commands/mcp`, `commands/serve`, `serve/handler`, and `shared/generation`). Because `shared` is not in the rule's `from` set, `shared/workflow-call.ts` may import `run/*` and `telemetry/otlp` downward without a peer violation, so the run/telemetry primitives stay put. The `mcp` and `exec` slice directories no longer exist — their concerns became shared infrastructure — while the peer-slice regex still names them so a reintroduced private tree is still guarded. Zero baselined slice edges originate from `commands`. With parse, transpile, runtime, format, and the CLI slices all gated, no deep-import work remains queued: every layer sits behind a public-entry rule and the dependency-cruiser baseline is empty. **Scripts.** The import-graph gate and the ESLint caps gate are both live and wired to their committed configs: diff --git a/docs/agent-auth.md b/docs/agent-auth.md index c46d45e7..1552eb58 100644 --- a/docs/agent-auth.md +++ b/docs/agent-auth.md @@ -8,7 +8,7 @@ diataxis: how-to This recipe sets the credentials each agent backend needs so the CLI's credential pre-flight passes and `prompt` steps reach the model. -`jaiph run` runs a host-side credential pre-flight before it spawns the runner or the Docker container. The pre-flight is keyed to the backends the entry file declares. Missing credentials produce either `E_AGENT_CREDENTIALS`, which is a hard abort, or a `jaiph: warning:` on host-only runs for the `claude` and `cursor` backends (see the table below). Hard failures exit before any runner or container is launched. The behavior is implemented in `src/cli/run/preflight-credentials.ts`. +`jaiph run` runs a host-side credential pre-flight before it spawns the runner. The pre-flight is keyed to the backends the entry file declares. Missing credentials produce either `E_AGENT_CREDENTIALS`, which is a hard abort, or a `jaiph: warning:` for the `claude` and `cursor` backends (see the table below). Hard failures exit before any runner is launched. The behavior is implemented in `src/cli/run/preflight-credentials.ts`. ## Prerequisites @@ -16,13 +16,13 @@ This recipe sets the credentials each agent backend needs so the CLI's credentia ## Pick the backend's credential -| Backend | Required credentials | Host run (no Docker) | Docker run (any mode incl. `inplace`) | -|---|---|---|---| -| `claude` | `ANTHROPIC_API_KEY` or `CLAUDE_CODE_OAUTH_TOKEN` | warn only (a stored Claude CLI login may still work) | hard error `E_AGENT_CREDENTIALS` | -| `cursor` | `CURSOR_API_KEY` | warn only (a stored `cursor-agent login` may still work) | hard error `E_AGENT_CREDENTIALS` | -| `codex` | `OPENAI_API_KEY` | hard error `E_AGENT_CREDENTIALS` (no CLI-login fallback) | hard error `E_AGENT_CREDENTIALS` when `OPENAI_API_KEY` is unset on the host (`OPENAI_API_KEY` is forwarded into the container) | +| Backend | Required credentials | Host behaviour | +|---|---|---| +| `claude` | `ANTHROPIC_API_KEY` or `CLAUDE_CODE_OAUTH_TOKEN` | warn only (a stored Claude CLI login may still work) | +| `cursor` | `CURSOR_API_KEY` | warn only (a stored `cursor-agent login` may still work) | +| `codex` | `OPENAI_API_KEY` | hard error `E_AGENT_CREDENTIALS` (no CLI-login fallback) | -Under Docker sandboxing the host-side stored logins (Keychain entries, `~/.claude`, `cursor-agent login`) do not cross the container boundary. Only `JAIPH_*` run-control keys plus the credential keys in the table above are forwarded, and credential keys only for the backends the entry file selects (see [Sandboxing](sandboxing.md#what-docker-protects-against)). Set credentials on the host so the allowlist can forward them into the container. Forward anything else one key at a time with `--env`, which is an intentional allowlist bypass. +Set credentials on the host. Forward anything else one key at a time with `--env`. ### Which backends get checked @@ -31,7 +31,7 @@ The pre-flight validates every backend the entry file could reach, which is each The default is deduplicated against your declarations, so where you set the backend decides whether the `cursor` default is also checked: - **Module scope.** Putting `config { agent.backend = "claude" }` at the top of the file makes `claude` the effective default, so only `claude` is checked. -- **Workflow scope only.** Putting `config { agent.backend = "claude" }` inside a workflow, with no module-level backend, leaves `cursor` as the default. The pre-flight then checks both `claude` and `cursor`. Under Docker that makes a missing `CURSOR_API_KEY` a hard error even when every prompt targets Claude. +- **Workflow scope only.** Putting `config { agent.backend = "claude" }` inside a workflow, with no module-level backend, leaves `cursor` as the default. The pre-flight then checks both `claude` and `cursor`. To check only the backend you intend to use, set it at module scope or export `JAIPH_AGENT_BACKEND`. Either one becomes the default and absorbs the extra check. See [Configure backend/model](configure-backend.md) for the config scopes. @@ -50,7 +50,7 @@ claude setup-token export CLAUDE_CODE_OAUTH_TOKEN="..." ``` -On host runs (no Docker), a stored `~/.claude` or macOS Keychain login from a previous interactive `claude` session also works, but in that case the pre-flight emits a warning rather than failing. +A stored `~/.claude` or macOS Keychain login from a previous interactive `claude` session also works, but in that case the pre-flight emits a warning rather than failing. ## 2. Authenticate Cursor @@ -66,9 +66,9 @@ For host runs only, an interactive `cursor-agent login` (stored on disk) also sa export OPENAI_API_KEY="sk-..." ``` -`OPENAI_API_KEY` is required on both host and Docker runs. The `codex` backend has no CLI-login fallback, so there is no warning path. Under Docker, export the key on the host. It crosses the container boundary via the env allowlist when the entry file selects `codex`, the same per-backend rule as `ANTHROPIC_API_KEY` and `CURSOR_API_KEY`. +`OPENAI_API_KEY` is required. The `codex` backend has no CLI-login fallback, so there is no warning path. -To target an OpenAI-compatible endpoint instead of the default, set `JAIPH_CODEX_API_URL` to the chat-completions URL (`JAIPH_*` is forwarded under Docker). +To target an OpenAI-compatible endpoint instead of the default, set `JAIPH_CODEX_API_URL` to the chat-completions URL. ## 4. Run the pre-flight @@ -78,11 +78,11 @@ jaiph run ./flow.jh The pre-flight runs before the banner. Hard failures print a stderr message naming the backend, the model (when `agent.model` is set), the entry `.jh` file, the config scope that picked the backend (`module config`, `workflow `, `JAIPH_AGENT_BACKEND env`, or `default`), and the concrete remedy. The error code is `E_AGENT_CREDENTIALS`. Host-only warnings for `claude` and `cursor` use the same header fields with a `jaiph: warning:` prefix. -## Skip the pre-flight (escape hatch) +## Skip the pre-flight -`JAIPH_UNSAFE=true` (or `jaiph run --unsafe`) skips the pre-flight entirely. The host is in charge, a stored CLI login may work, and the runtime's per-backend guards remain as a backstop. The pre-flight is also skipped when the entry file neither declares an explicit backend nor uses any `prompt` step, because nothing would credential against. +The pre-flight is skipped when the entry file neither declares an explicit backend nor uses any `prompt` step, because nothing would credential against. -`jaiph run --raw` also skips the pre-flight. Raw mode is the passthrough the host uses to run the workflow inside the Docker container, so the outer `jaiph run` has already run the pre-flight before it spawns the inner raw run. +`jaiph run --raw` also skips the pre-flight. ## Verification @@ -92,10 +92,7 @@ When every required credential is present, the pre-flight is silent, with no std E_AGENT_CREDENTIALS: agent.backend "claude" selected by module config in /path/to/flow.jh — neither ANTHROPIC_API_KEY nor CLAUDE_CODE_OAUTH_TOKEN is set. Run `claude setup-token` and export CLAUDE_CODE_OAUTH_TOKEN, or set ANTHROPIC_API_KEY. ``` -Under Docker the message includes the suffix `(Docker is on — set the env var on the host so it is forwarded into the container.)`. - ## Related -- [Run a workflow in a Docker sandbox](sandbox-run.md) — how host env vars cross the container boundary. - [Configure backend/model](configure-backend.md) — picking which backend a workflow uses. -- [Sandboxing — What Docker protects against](sandboxing.md#what-docker-protects-against) — env allowlist and what crosses the container boundary. +- [Environment variables](env-vars.md) — `--env` and credential names. diff --git a/docs/architecture.md b/docs/architecture.md index 985ddbf1..18c8ff58 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -7,6 +7,9 @@ redirect_from: - /spec-async-isolated - /target-design - /reporting + - /sandboxing + - /sandbox-run + - /how-to/sandbox-run --- # Architecture @@ -21,16 +24,16 @@ For how to contribute, see [Contributing](contributing.md), which covers branche ## System overview -Workflow authors write `.jh` / `.test.jh` modules. The toolchain turns those files into validated modules plus extracted script files, and then the same AST interpreter runs the workflows whether you use local `jaiph run`, Docker, or `jaiph test`. +Workflow authors write `.jh` / `.test.jh` modules. The toolchain turns those files into validated modules plus extracted script files, and then the same AST interpreter runs the workflows for `jaiph run` and `jaiph test`. 1. Parse source into AST. Every CLI path walks the entry plus its transitive `.jh` import closure **once** through **`loadModuleGraph`** (`src/transpile/module-graph.ts`) and reuses that **`ModuleGraph`** for the banner (`metadataToConfig`), validation (**`validateModule`** inside **`emitScriptsForModuleFromGraph`**, invoked by **`buildScriptsFromGraph`**), script-body extraction, and, across the parent to child process boundary on the default local `jaiph run`, for **`buildRuntimeGraph(graph)`** in the spawned runner (see [Local module graph](#local-module-graph) and the sequence diagram below). `parsejaiph(source, filePath)` is I/O-pure, and validation and script emit operate entirely on the in-memory graph and never re-read `.jh` files. Inside this compile-and-run graph pipeline, `loadModuleGraph` is the only routine that reads `.jh` sources from disk. A few paths outside the graph pipeline still read `.jh` directly, such as `runWorkflowRaw` on the `jaiph run --raw` path (`src/cli/commands/run.ts`) and the exported `loadImportedModules` helper (`src/cli/shared/paths.ts`). 2. **Compile-time** validation runs before script extraction. The validator consumes the in-memory graph; imported ASTs are looked up by absolute path and never re-read from disk. Three validation entry points share the same per-module walk via **`validateModuleInto`**: **`validateModule(ast, graph)`** is the per-module throwing form (used by **`emitScriptsForModuleFromGraph`** / **`buildScriptsFromGraph()`** so the existing single-error path stays intact), **`validateReferences(graph)`** validates every reachable module then throws the first sorted error, and **`collectDiagnostics(graph)`** returns a populated `Diagnostics` collector (`src/diagnostics.ts`) with **every** recoverable error from every reachable module. The **`jaiph compile`** command walks the same import closure but routes through `collectDiagnostics`: it builds a graph per entry, collects diagnostics, prints them all (sorted by file/line/col, in `path:line:col CODE message` form on stderr — or as a single JSON array on stdout with `--json`), and exits non-zero if any diagnostic was collected. It **does not** emit **`scripts/`**, **does not** invoke **`buildRuntimeGraph()`**, and never spawns the workflow runner (`src/cli/commands/compile.ts`). For a **directory** argument it discovers `*.jh` via `walkjhFiles`, which **skips** `*.test.jh`; to validate a test module, pass that file explicitly. Imported modules in the closure are still validated recursively either way. 3. **CLI** (`dist/src/cli.js` via npm, or a **Bun-compiled** `dist/jaiph` binary) prepares script executables (scripts-only), then spawns a **detached child** through the internal **`__workflow-runner`** argv marker (**`spawnJaiphWorkflowProcess`** in `src/runtime/kernel/workflow-launch.ts`). The child entrypoint is **`runWorkflowRunner`** (`src/runtime/kernel/node-workflow-runner.ts`), which loads or deserializes the module graph, calls **`buildRuntimeGraph()`**, then runs **`NodeWorkflowRuntime`**. Under Node the spawn is **`process.execPath`** + **`dist/src/cli.js`** + **`__workflow-runner`**; under the Bun standalone binary, **`process.execPath`** is the **`jaiph`** binary itself with the same marker. Script steps execute as managed subprocesses; prompt, inbox I/O, and event/summary emission are handled by the kernel under `src/runtime/kernel/`. 4. Stream live events to the CLI and persist durable run artifacts. -Interactive **`jaiph run`** parses **`__JAIPH_EVENT__`** lines from the runner's stderr, renders the progress tree, and runs hooks. **`jaiph run --raw`** skips that shell. The child uses inherited stdio, so events still land on stderr unchanged. Use `--raw` when you embed Jaiph or when the host wraps a container (see [CLI, `jaiph run`](cli.md#jaiph-run) and [Sandboxing](sandboxing.md)). +Interactive **`jaiph run`** parses **`__JAIPH_EVENT__`** lines from the runner's stderr, renders the progress tree, and runs hooks. **`jaiph run --raw`** skips that shell. The child uses inherited stdio, so events still land on stderr unchanged. Use `--raw` when you embed Jaiph (see [CLI, `jaiph run`](cli.md#jaiph-run)). -All orchestration uses the Node workflow runtime, which is the AST interpreter, whether you run local `jaiph run`, `jaiph test`, or **Docker `jaiph run`**. Docker containers run the same **`jaiph run --raw`** / **`__workflow-runner`** dispatch with the compiled JS source tree and scripts mounted read-only. +All orchestration uses the Node workflow runtime, which is the AST interpreter, whether you run `jaiph run` or `jaiph test`. ### Import-graph layering @@ -40,7 +43,7 @@ The `src/` import graph is an acyclic layered DAG: parse/format → transpile - **CLI (`src/cli`, invoked via compiled `src/cli.ts` → `dist/src/cli.js`)** - Entry point (`run`, `test`, `compile`, `init`, `install`, `use`, `format`, `mcp`, `serve`). Paths ending in `.jh` / `.test.jh` are also accepted as implicit commands (see `src/cli/index.ts`). - - **Workflow launch** is owned in TypeScript (`src/runtime/kernel/workflow-launch.ts` + `src/cli/run/lifecycle.ts`): spawns the runner via **`process.execPath`** and the **`__workflow-runner`** argv marker. **`runWorkflowRunner`** (`src/runtime/kernel/node-workflow-runner.ts`) handles that argv, loads or reads the module graph, calls **`buildRuntimeGraph()`**, then **`NodeWorkflowRuntime.runDefault()`**. The **`default`** workflow name is wired in **`buildRunModuleLaunch`** (`workflow-launch.ts`). `setupRunSignalHandlers` accepts an optional `onSignalCleanup` callback for Docker sandbox teardown on SIGINT/SIGTERM — for a Docker-backed run it is `stopDockerRunOnSignal`, which stops and removes the container (`docker kill` then `docker rm -f`) before deleting the host sandbox clone so an interrupt cannot orphan a running container (see [Docker runtime helper](#core-components)). + - **Workflow launch** is owned in TypeScript (`src/runtime/kernel/workflow-launch.ts` + `src/cli/run/lifecycle.ts`): spawns the runner via **`process.execPath`** and the **`__workflow-runner`** argv marker. **`runWorkflowRunner`** (`src/runtime/kernel/node-workflow-runner.ts`) handles that argv, loads or reads the module graph, calls **`buildRuntimeGraph()`**, then **`NodeWorkflowRuntime.runDefault()`**. The **`default`** workflow name is wired in **`buildRunModuleLaunch`** (`workflow-launch.ts`). `setupRunSignalHandlers` terminates the runner process tree on SIGINT/SIGTERM. - Parses runtime events and renders progress (except `--raw`); dispatches hooks. - **Parser (`src/parser.ts`, `src/parse/*`)** @@ -77,8 +80,8 @@ The `src/` import graph is an acyclic layered DAG: parse/format → transpile - The pipeline contract is **`loadModuleGraph` → `buildScriptsFromGraph(graph, outDir)`**, which runs **`validateModule`** + **`buildScriptFiles`** per reachable module via **`emitScriptsForModuleFromGraph`**. `parsejaiph` is I/O-pure; validation and script emit never re-read `.jh` sources during graph work. Each reachable module is parsed exactly once per `jaiph run` (see [Local module graph](#local-module-graph)). - **Runtime public entry (`src/runtime/index.ts`)** - - **Public entry.** Code outside the runtime package imports the runtime slice only through `src/runtime/index.ts`, which re-exports a curated CLI-facing API: graph construction (`buildRuntimeGraph`, `RuntimeGraph`), the launch and runner entry points (`runWorkflowRunner`, `WORKFLOW_RUNNER_ARG`, `spawnJaiphWorkflowProcess`, `runTestFile`), the Docker sandbox surface (`spawnDockerProcess`, `resolveDockerConfig`, `prepareImage`, `selectSandboxMode`, the run-config env constants, and the `DockerRunConfig` / `SandboxMode` types, among others), the emit and audit-chain helpers the CLI reads after a run (`generateChainKey`, `verifyRunJournal`, `redactCredentials`), the terminal-portability helpers (`canUseAnsi`, `killProcessTree`, `resolveShell`), the embedded-asset accessors, and the run-tree param display helper (`buildStepDisplayParamPairs`). It is not an `export *` barrel of the tree. The runtime has two allowlisted public entries: `src/runtime/index.ts` (the production surface above) and `src/runtime/testing.ts` (test seams, described next). The `no-deep-imports-into-runtime` rule in `.dependency-cruiser.cjs` fails any outside import that reaches a `src/runtime/**` internal directly, such as a `kernel/*.ts` file or `docker.ts`, unless the import goes through one of those two entries. Add a named re-export to `src/runtime/index.ts` instead of reaching in. The baseline carries no `no-deep-imports-into-runtime` leftovers. The former `src/config.ts` → `src/runtime/kernel/runtime-arg-parser.ts` leak is gone: the pure `interpolate` helper moved down into `src/config.ts`, which `runtime-arg-parser.ts` imports downward and re-exports, so `src/config.ts` imports nothing from `src/runtime/`. - - **Test-seam entry (`src/runtime/testing.ts`).** A few runtime internals are private to production but needed by cross-package `*.test.ts` files that stub or inspect them: the Docker exec and spawn indirection (`_dockerExec`, `_dockerSpawn`), the in-place run prompt (`_inplacePrompt`), the audit-chain HMAC internals (`CHAIN_GENESIS`, `chainHmac`), and the live-event emitter (`RuntimeEventEmitter`). These are re-exported from a second public entry, `src/runtime/testing.ts`, which is allowlisted beside `src/runtime/index.ts` in the `no-deep-imports-into-runtime` rule. A cross-package test imports a seam from `src/runtime/testing.ts` and never a raw `src/runtime/**` path, so the seams stay off the production `index.ts` while no test deep-imports the tree. Keep this surface small: add a seam only when a cross-package test genuinely needs one. + - **Public entry.** Code outside the runtime package imports the runtime slice only through `src/runtime/index.ts`, which re-exports a curated CLI-facing API: graph construction (`buildRuntimeGraph`, `RuntimeGraph`), the launch and runner entry points (`runWorkflowRunner`, `WORKFLOW_RUNNER_ARG`, `spawnJaiphWorkflowProcess`, `runTestFile`), the emit and audit-chain helpers the CLI reads after a run (`generateChainKey`, `verifyRunJournal`, `redactCredentials`), the terminal-portability helpers (`canUseAnsi`, `killProcessTree`, `resolveShell`), the embedded-asset accessors, and the run-tree param display helper (`buildStepDisplayParamPairs`). It is not an `export *` barrel of the tree. The runtime has two allowlisted public entries: `src/runtime/index.ts` (the production surface above) and `src/runtime/testing.ts` (test seams, described next). The `no-deep-imports-into-runtime` rule in `.dependency-cruiser.cjs` fails any outside import that reaches a `src/runtime/**` internal directly, such as a `kernel/*.ts` file, unless the import goes through one of those two entries. Add a named re-export to `src/runtime/index.ts` instead of reaching in. The baseline carries no `no-deep-imports-into-runtime` leftovers. The former `src/config.ts` → `src/runtime/kernel/runtime-arg-parser.ts` leak is gone: the pure `interpolate` helper moved down into `src/config.ts`, which `runtime-arg-parser.ts` imports downward and re-exports, so `src/config.ts` imports nothing from `src/runtime/`. + - **Test-seam entry (`src/runtime/testing.ts`).** A few runtime internals are private to production but needed by cross-package `*.test.ts` files that stub or inspect them: the audit-chain HMAC internals (`CHAIN_GENESIS`, `chainHmac`), and the live-event emitter (`RuntimeEventEmitter`). These are re-exported from a second public entry, `src/runtime/testing.ts`, which is allowlisted beside `src/runtime/index.ts` in the `no-deep-imports-into-runtime` rule. A cross-package test imports a seam from `src/runtime/testing.ts` and never a raw `src/runtime/**` path, so the seams stay off the production `index.ts` while no test deep-imports the tree. Keep this surface small: add a seam only when a cross-package test genuinely needs one. - **No runtime → CLI edge.** The runtime is layer 3 and the CLI is layer 4, so imports may only point downward. The kernel emits the run-tree display param pairs on every managed step, so the `buildStepDisplayParamPairs` helper lives in the runtime at `src/runtime/kernel/format-params.ts`; `src/cli/commands/format-params.ts` re-exports it through the public entry so its CLI callers keep one import site. No production file under `src/runtime/` imports `src/cli/**`. - **Node Workflow Runtime (`src/runtime/kernel/node-workflow-runtime.ts`)** @@ -105,11 +108,6 @@ The `src/` import graph is an acyclic layered DAG: parse/format → transpile - **Public entry.** Code outside the format package imports the format slice only through `src/format/index.ts`, which re-exports the formatter API (`emitModule` and the `EmitOptions` type). It is not an `export *` barrel of the tree. The `no-deep-imports-into-format` rule in `.dependency-cruiser.cjs` fails any outside import that reaches a `src/format/**` internal directly, such as `emit.ts`. Add a named re-export to `src/format/index.ts` instead of reaching in. Format is layer 1 beside parse, so its sources import only parse and types, never `src/cli`, `src/runtime`, or `src/transpile`. - `jaiph format` rewrites `.jh` / `.test.jh` files into canonical style. `emitModule(ast, trivia, opts?)` reads the semantic AST together with the parallel **`Trivia`** store ([Trivia (CST layer)](#trivia-cst-layer)) to round-trip leading comments, top-level order, `config` body sequence, `"""..."""` and `bareSource` forms, the original quotedness of top-level `const` values (`EnvDeclDef.wasQuoted` — `true` for `"…"` / `"""…"""` sources, `undefined` for bare tokens — so a quoted value is never silently rewritten as bare based on whether it contains a space), and prompt / script body discriminators. Step emission switches on `WorkflowStepDef.type` (8 variants) and an `emitExpr` helper switches on `Expr.kind` (8 kinds) — there are no dual code paths for "managed sidecar vs literal value" because that branch was removed from the AST. Call arguments render straight off the typed `Arg[]` — `var` → bare name, `literal` → raw — so the formatter no longer re-parses any args string or consults a `bareIdentifierArgs` shadow field. Pure data→text emitter; no side-effects beyond file writes. Round-trip is bit-for-bit on every fixture under `examples/` and `test-fixtures/golden-ast/fixtures/` — pinned by `src/format/roundtrip.test.ts`, which asserts `parse → format → parse → format` converges in one step on every fixture. -- **Docker runtime helper (`src/runtime/docker.ts`)** - - Parses mount specs, resolves Docker config (image, network, timeout), and builds the `docker run` invocation when the CLI enables **Docker sandboxing** for `jaiph run` (environment-driven; there is no `jaiph run --docker` flag — see [Sandboxing](sandboxing.md)). **Host-controlled image/network (finding M-6):** an entry file is untrusted, so when Docker is the active sandbox `resolveDockerConfig` rejects a file-declared `runtime.docker_image` (`E_DOCKER_IMAGE_HOST_ONLY`) and a file-declared isolation-breaking `runtime.docker_network` — `host`, `container:*`, `ns:*`, anything that is not `default` / `none` / a plain named bridge network (`isHostSafeInFileNetwork`) — (`E_DOCKER_NETWORK_HOST_ONLY`). The operator's `JAIPH_DOCKER_IMAGE` / `JAIPH_DOCKER_NETWORK` remain trusted and are used verbatim (they may even select `host`); host-safe in-file network values are still honoured. When Docker is off these keys are inert and not enforced. On **`win32`** the Docker sandbox is out of scope: **`resolveDockerConfig`** forces host-only mode (same UX as an explicit **`JAIPH_UNSAFE=true`**) with a one-line notice, so the CLI never probes `docker` and never hard-fails on a missing daemon (`JAIPH_DOCKER_ENABLED=true` cannot override this). The container runs the same **`jaiph run --raw`** / **`__workflow-runner`** entry as local execution. The default image is the official `ghcr.io/jaiphlang/jaiph-runtime` GHCR image tagged with the CLI version (`ghcr.io/jaiphlang/jaiph-runtime:`); every selected image must already contain `jaiph` (no auto-install or derived-image build at runtime). Image preparation (`prepareImage`) runs before the CLI banner: it checks whether the image is local, pulls with `--quiet` if needed (short status lines on stderr instead of Docker's default pull UI), and verifies that `jaiph` exists in the image. **Digest pinning + fail-closed verification (finding M-6):** the mutable tag alone is not the trust boundary — a registry compromise, a re-pointed tag, or a poisoned local cache under the same tag would substitute the sandbox rootfs. The expected manifest digest ships with the release (`RUNTIME_IMAGE_DIGEST`, baked from `package.json`'s `runtimeImageDigest`) and an operator can override it with `JAIPH_DOCKER_IMAGE_DIGEST` (`resolveExpectedDigest`). When a digest is pinned, a cold pull resolves the digest-pinned reference (`repo@sha256:…`, content-addressed) and tags it back to the run reference (`pullPinnedImage`); then `verifyImageDigest` inspects the resolved local image's registry digest and fails closed (`E_DOCKER_DIGEST_MISMATCH`, with re-pull recovery guidance) on **every** run, including the cache-hit path, before the image is used. Enforcement is skipped when no digest is pinned (a custom operator image with no `JAIPH_DOCKER_IMAGE_DIGEST` / `@sha256:`, or the default image before the release bakes its digest). **Hardened presence probe (finding M-8):** the image is workflow-influenced and is pulled before the check, so the verification probe (`buildImageProbeArgs`) runs the image with the same hardening as a real run (`--cap-drop ALL`, `--security-opt no-new-privileges`, a non-root `--user`, and `--network none`) and a non-login `sh -c`, so `command -v jaiph` resolves only PATH and the check never sources `/etc/profile` or `/etc/profile.d/*` scripts baked into the image. `spawnDockerProcess` does not pull or verify — it receives a pre-resolved image. The spawn call uses `stdio: ["ignore", "pipe", "pipe"]` — stdin is ignored so the Docker CLI does not block on stdin EOF, which would stall event streaming and hang the host CLI after the container exits. - - **Workspace immutability:** By default Docker runs cannot modify the host workspace. In the default **snapshot** mode the host takes a writable point-in-time clone of the workspace at run start (`/sandbox`, via `cloneWorkspaceForSandbox` in `src/runtime/docker.ts`) and bind-mounts that clone read-write at `/jaiph/workspace`; the live host checkout is never mounted, and the clone is discarded on exit. The clone content is **git-defined**: for a git workspace it is exactly `git ls-files --cached --others --exclude-standard` plus `.git/` wholesale (gitignored files — `node_modules/`, `.env`, build output — are absent, never scanned); git is the sole ignore oracle (no reimplemented gitignore matcher). A non-git workspace (no `.git` at the root, or `git ls-files` fails) falls back to copying everything. See [Sandboxing — What the snapshot contains](sandboxing.md#snapshot-content). The only host-writable path is `/jaiph/run` (run artifacts), and the snapshot source under it is masked from the container with a tmpfs at `/jaiph/run/sandbox`. Workflows that need to capture workspace changes should write files (for example a `git diff` into a temp path) and publish them with `artifacts.save()`. The explicit opt-in **inplace** mode (truthy **`JAIPH_INPLACE`** — `1` or `true`, or `jaiph run --inplace`) breaks this contract on purpose — the host workspace itself is bind-mounted read-write so the run's edits persist live on the host, with the rest of the sandbox (caps, env allowlist, mount set) unchanged. See [Sandboxing](sandboxing.md) for the full contract and [Save artifacts](artifacts.md). - - **Container teardown on interrupt / timeout:** `spawnDockerProcess` assigns every container a deterministic `--name` (`jaiph-run-`, emitted immediately after `run --rm`) so it can be force-removed by name later. A `docker run --rm` container can outlive its host `docker` client (Docker Desktop / detached behaviour), so killing the client's process tree alone does not guarantee the container stops. On SIGINT/SIGTERM the run's `onSignalCleanup` calls **`stopDockerRunOnSignal`**, and the run-timeout kill (`E_TIMEOUT`) calls **`stopDockerContainer`** directly — both run `docker kill ` (bounded 5 s) then `docker rm -f ` (bounded 10 s), best-effort, so the `--rm` container disappears from `docker ps` within a bounded window. Splitting kill from rm avoids macOS Docker Desktop lock contention where a single `docker rm -f` on a still-running container can block for the full timeout. Order matters: the container is stopped **before** `cleanupDocker` removes the host workspace snapshot at `/sandbox`, because that snapshot is bind-mounted into the container. The per-call cancel path in the shared workflow-call executor (`src/cli/shared/workflow-call.ts`, which backs both `jaiph mcp` tool calls and `jaiph serve`) applies the same teardown — `stopDockerContainer` then `cancelRunProcess`. Both sandbox modes (snapshot, inplace) share this contract. See [Sandboxing — interrupting a Docker run](sandboxing.md#interrupting-a-docker-run). - ## Local module graph {: #local-module-graph} @@ -118,9 +116,8 @@ The toolchain has one canonical representation, **`ModuleGraph`**, for all `.jh` - **`loadModuleGraph(entryFile, workspaceRoot?)`** (`src/transpile/module-graph.ts`) walks the entry plus its transitive `import` edges through `resolveImportPath` and returns `{ entryFile, workspaceRoot?, modules: Map }> }`. **`/`** imports (for example `jaiphlang/queue`) resolve through the workspace library fallback under `.jaiph/libs/` when a relative path does not exist. Within the graph pipeline this is the only routine that reads `.jh` sources from disk, and `parsejaiph(source, filePath)` itself is I/O-pure. A couple of paths outside the graph pipeline read `.jh` on their own, such as `runWorkflowRaw` (`jaiph run --raw`) and the exported `loadImportedModules` helper. - **`src/cli/commands/run.ts`** calls `loadModuleGraph` once after path normalization. The entry AST is reused for the banner / `runtime` config via **`metadataToConfig(resolveModuleMetadata(mod, env))`** — `resolveModuleMetadata` resolves the `config { … }` block's interpolation before `metadataToConfig` flattens it. The same graph is passed to **`buildScriptsFromGraph(graph, outDir)`**, which calls **`emitScriptsForModuleFromGraph`** per reachable module; each call runs **`validateModule(ast, graph)`** against the in-memory ASTs. - **Process boundary.** The CLI serializes the graph with **`writeModuleGraph`** to **`/.jaiph-module-graph.json`** (deterministic JSON: entries sorted by absolute path; ASTs included verbatim). It points the spawned **`__workflow-runner`** child at the file through the internal env var **`JAIPH_MODULE_GRAPH_FILE`**. The runner reads it back with **`readModuleGraph`** and passes the result to **`buildRuntimeGraph(graph)`**, which produces the runtime view (with **`import script`** stub injection) without touching disk. Cross-module workflow / rule / script resolution matches the on-disk load path. -- **Scope of the env-var hand-off.** `JAIPH_MODULE_GRAPH_FILE` is set on the host (non-Docker) execution paths that spawn the local **`__workflow-runner`** child: interactive **`jaiph run`** when Docker sandboxing is disabled (`dockerConfigForBanner.enabled === false`), and the shared workflow-call executor (`src/cli/shared/workflow-call.ts`) that backs `jaiph mcp` tool calls and `jaiph serve`. It is **not** set on these paths, which load the graph from disk inside the runner instead: +- **Scope of the env-var hand-off.** `JAIPH_MODULE_GRAPH_FILE` is set on the execution paths that spawn the local **`__workflow-runner`** child: interactive **`jaiph run`**, and the shared workflow-call executor (`src/cli/shared/workflow-call.ts`) that backs `jaiph mcp` tool calls and `jaiph serve`. It is **not** set on these paths, which load the graph from disk inside the runner instead: - **`jaiph run --raw`** — `runWorkflowRaw` (`src/cli/commands/run.ts`) calls `buildScripts` directly without writing the graph file; the runner uses inherited stdio and falls back to `loadModuleGraph` from the source file. - - **Docker `jaiph run`** — the host writes the graph file under `outDir`, but skips the env var because the inner container command is `jaiph run --raw …` and the host bind-mount layout does not plumb the cache file inside the container. - **`jaiph test`** — `runSingleTestFile` builds the graph in `src/cli/commands/test.ts` and threads it through `runTestFile(graph, ...)` directly (no env var needed; same process). When the env var is absent, the runner falls back to the disk-walk parse path, which preserves the prior behavior. @@ -140,7 +137,7 @@ User-visible contracts (banner, hooks, run artifacts, `run_summary.jsonl`, `retu ### CLI responsibilities - Parse, validate, and launch workflows/tests. -- Own **process spawn** for `jaiph run` (detached workflow runner process group for signal propagation). Terminating a run means terminating the whole tree — the detached leader plus the agent backends and script children it spawned — routed through **`killProcessTree(pid, signal)`** (`src/runtime/kernel/portability.ts`), the single sanctioned home for group kills. On POSIX it signals the leader's process group with **`process.kill(-pid, signal)`**, falling back to a per-process kill if the group no longer exists (`ESRCH`). On **`win32`** a negative-PID group kill throws and a per-process kill would orphan the children, so it force-kills the tree with **`taskkill /pid /T /F`** (spawned, not shelled), degrading to a per-process kill if `taskkill` cannot be launched. Because `taskkill /F` is already forceful, a follow-up `SIGKILL` escalation after a `SIGTERM`/`SIGINT` is a **documented no-op** on `win32`. All group-kill call sites route through this helper: run teardown and the host run-timeout kill — `armRunTimeout`, the parent-enforced wall-clock cap for host mode (`JAIPH_RUN_TIMEOUT`; see [Configuration — Overall run timeout and step cap](configuration.md#overall-run-timeout-and-step-cap)) — both in `src/cli/run/lifecycle.ts`, the prompt watchdog (`src/runtime/kernel/prompt.ts`), and the Docker run-timeout kill (`src/runtime/docker.ts`). +- Own **process spawn** for `jaiph run` (detached workflow runner process group for signal propagation). Terminating a run means terminating the whole tree — the detached leader plus the agent backends and script children it spawned — routed through **`killProcessTree(pid, signal)`** (`src/runtime/kernel/portability.ts`), the single sanctioned home for group kills. On POSIX it signals the leader's process group with **`process.kill(-pid, signal)`**, falling back to a per-process kill if the group no longer exists (`ESRCH`). On **`win32`** a negative-PID group kill throws and a per-process kill would orphan the children, so it force-kills the tree with **`taskkill /pid /T /F`** (spawned, not shelled), degrading to a per-process kill if `taskkill` cannot be launched. Because `taskkill /F` is already forceful, a follow-up `SIGKILL` escalation after a `SIGTERM`/`SIGINT` is a **documented no-op** on `win32`. All group-kill call sites route through this helper: run teardown and the host run-timeout kill — `armRunTimeout`, the parent-enforced wall-clock cap (`JAIPH_RUN_TIMEOUT`; see [Configuration — Overall run timeout and step cap](configuration.md#overall-run-timeout-and-step-cap)) — both in `src/cli/run/lifecycle.ts`, and the prompt watchdog (`src/runtime/kernel/prompt.ts`). - Parse live runtime events; render terminal progress; trigger hooks — skipped in **`jaiph run --raw`** (child stdio inherited; see [CLI](cli.md#jaiph-run)). ## Contracts @@ -182,13 +179,13 @@ Step sequence numbers are monotonic and unique per run: `RuntimeEventEmitter` al Every line written to `run_summary.jsonl` by `RuntimeEventEmitter` carries a `prev_hash` field. The field holds a **keyed** HMAC-SHA256 (in hex) of the previous raw JSON line — `chainHmac(key, previousLine)` — with `chainHmac(key, CHAIN_GENESIS)` for the first line. The key is a per-run 256-bit secret (`generateChainKey`), so the chain is not reproducible from the public algorithm alone: rewriting a line, or dropping a line and re-linking the survivors, invalidates the chain and cannot be re-forged without the key. -**Key isolation (finding H-3).** The journal is written by the trusted kernel process, but the audited workflow — its `script` steps and prompt/agent subprocesses — must not be able to forge the chain. The key travels in the kernel process env under `JAIPH_CHAIN_KEY` (`CHAIN_KEY_ENV`) and is scrubbed from **every** subprocess env: `scrubTrustedKeys` (`node-workflow-runtime.ts`) removes it — along with the journal path `JAIPH_RUN_SUMMARY_FILE` — from every script scope, and `scrubPromptEnv` (`env-allowlist.ts`) drops it at the agent boundary even though the `JAIPH_` prefix otherwise forwards run-control keys into the Docker container (the in-container kernel legitimately needs it). The host (`src/cli/commands/run.ts`, `src/cli/shared/workflow-call.ts`) generates the key, forwards it to the runner, and — once the run is terminal — persists it in an operator-side store (`writeChainKey`) so read/export boundaries can verify it. +**Key isolation (finding H-3).** The journal is written by the trusted kernel process, but the audited workflow — its `script` steps and prompt/agent subprocesses — must not be able to forge the chain. The key travels in the kernel process env under `JAIPH_CHAIN_KEY` (`CHAIN_KEY_ENV`) and is scrubbed from **every** subprocess env: `scrubTrustedKeys` (`node-workflow-runtime.ts`) removes it — along with the journal path `JAIPH_RUN_SUMMARY_FILE` — from every script scope, and `scrubPromptEnv` (`env-allowlist.ts`) drops it at the agent boundary. The host (`src/cli/commands/run.ts`, `src/cli/shared/workflow-call.ts`) generates the key, forwards it to the runner, and — once the run is terminal — persists it in an operator-side store (`writeChainKey`) so read/export boundaries can verify it. -**Key storage outside the run directory (finding M-3).** The key is **not** written into the run directory, which is agent-writable (`$JAIPH_RUN_DIR` for script steps, bind-mounted `rw` at `/jaiph/run` under Docker). Storing it there let a workflow's first script step `mkdir "$JAIPH_RUN_DIR/.chain-key"` to squat the path so the host's write threw and was swallowed, then rewrite the journal freely with no integrity failure surfaced. Instead the key lives in an operator-side store — `resolveAuditKeyStore` (`emit.ts`), default `~/.jaiph/audit-keys`, override `JAIPH_AUDIT_KEY_DIR` — that is never mounted into a container. Each run gets one entry directory `/` holding the secret `key` file; the directory's existence is the durable "this run was keyed" marker. Persistence is a **hard error**: `writeChainKey` no longer swallows failures, and it creates the marker directory before the key, so even a partial write leaves the run marked keyed-but-keyless (which fails closed below) rather than silently unverifiable. +**Key storage outside the run directory (finding M-3).** The key is **not** written into the run directory, which is agent-writable (`$JAIPH_RUN_DIR` for script steps). Storing it there let a workflow's first script step `mkdir "$JAIPH_RUN_DIR/.chain-key"` to squat the path so the host's write threw and was swallowed, then rewrite the journal freely with no integrity failure surfaced. Instead the key lives in an operator-side store — `resolveAuditKeyStore` (`emit.ts`), default `~/.jaiph/audit-keys`, override `JAIPH_AUDIT_KEY_DIR` — . Each run gets one entry directory `/` holding the secret `key` file; the directory's existence is the durable "this run was keyed" marker. Persistence is a **hard error**: `writeChainKey` no longer swallows failures, and it creates the marker directory before the key, so even a partial write leaves the run marked keyed-but-keyless (which fails closed below) rather than silently unverifiable. **Verification at read/export boundaries.** `verifyRunSummaryChain(filePath, key, opts?)` walks each line, checks `prev_hash` against the recomputed keyed digest, and returns `{ ok: false, error }` at the first broken link (a missing/unreadable journal is a failure, not a silent pass). With `opts.requireTerminal` it additionally requires the journal to **end with the `WORKFLOW_END` terminal marker** (`TERMINAL_EVENT_TYPE`) — the chain commits to prefix integrity but not to length, so deleting the last *K* lines of a completed journal leaves a shorter-but-valid chain that would otherwise verify; requiring the terminal marker rejects any post-terminal tail truncation (finding L-3). `verifyRunJournal(runDir)` wraps it (always with `requireTerminal`, since a key is persisted only once the run is terminal): it looks up the run's store entry and returns `{ verified: false, ok: true }` when the run has **no** entry (an unkeyed/legacy run that cannot be verified — never blocked), `{ verified: true, ok: false }` when the run **was** keyed but the key is missing at verification time (**fail closed** — a keyed run whose key vanished must not downgrade to "not verified" and let a tampered journal through), or `{ verified: true, ok }` with the chain result otherwise. Every read/export boundary hard-fails when `verified && !ok`: run listing (`loadPersistedRuns` marks the run `failed` with `TAMPERED_RESULT_TEXT`), `GET /v1/runs/{id}/events` (`409 E_TAMPERED`), and OTLP/Sentry export (skip + warn, never POST a tampered journal). -**Scope of the guarantee.** A workflow script step cannot read the key or the journal path from its env, and cannot alter the journal in any way that verifies — any rewrite or omitted line is rejected, and any truncation *during* the run is caught because the kernel keeps appending under the pre-truncation head. A *post-run* clean truncation of a completed journal's tail is also rejected: the terminal-marker check (finding L-3) fails a keyed journal that no longer ends with `WORKFLOW_END`. Because a `.jh` host run and its `script` steps execute under the same OS user, a hash chain still cannot defend against a post-run same-user process that deletes the run's store entry, which makes the run unverifiable (`verified:false`) rather than a detectable tamper. Under Docker sandboxing the key never enters the container, and the key store lives outside every bind mount, so an in-sandbox workflow can neither read the key nor reach the store to squat or delete it. +**Scope of the guarantee.** A workflow script step cannot read the key or the journal path from its env, and cannot alter the journal in any way that verifies — any rewrite or omitted line is rejected, and any truncation *during* the run is caught because the kernel keeps appending under the pre-truncation head. A *post-run* clean truncation of a completed journal's tail is also rejected: the terminal-marker check (finding L-3) fails a keyed journal that no longer ends with `WORKFLOW_END`. Because a `.jh` host run and its `script` steps execute under the same OS user, a hash chain still cannot defend against a post-run same-user process that deletes the run's store entry, which makes the run unverifiable (`verified:false`) rather than a detectable tamper. A workflow cannot read the key from its env, and the key store lives outside the run directory. #### Secret redaction @@ -200,7 +197,7 @@ Before `RuntimeEventEmitter` writes an event line to `run_summary.jsonl`, it red - the `params` key/value pairs of every `STEP_END`, which hold the positional or named arguments passed to a `run`, tool, or `script` step, so a secret passed as an argument is redacted the same way as the step's captured output (`emitStep`), - the `message` field of every durable `LOG`, `LOGWARN`, and `LOGERR` event, so a value a workflow passes to `log`, `logwarn`, or `logerr` is redacted in the journal the same way as a step's captured output (`emitLog`). -The rule covers backend API keys such as `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, and `CURSOR_API_KEY` (the same names on the [Docker env allowlist](sandboxing.md)). +The rule covers backend API keys such as `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, and `CURSOR_API_KEY` . The same credential rule lives in one shared helper, **`redactCredentials`** (`src/runtime/kernel/redact.ts`). The helper is also the redaction boundary for returned call results. `composeResult` (`src/cli/shared/workflow-call.ts`) redacts a failed call's diagnostic capture (the failed-step detail, the raw stderr and stdout, and the collected `log` messages) before it becomes `jaiph serve`'s `result_text` or a `jaiph mcp` tool result. A successful workflow's return value is intentional API output rather than diagnostic capture, so it is returned verbatim. The journal that `redactCredentials` produces is what the OTLP export (`otlp.ts`), the Sentry export (`sentry.ts`), and `GET /v1/runs/{id}/events` (`handler.ts`) read back verbatim, so broadening the rule tightens all four surfaces at once. @@ -224,7 +221,7 @@ The progress UI combines a **static** step tree derived from the workflow AST (` ## Distribution: Node vs Bun standalone -- **Development / npm:** `npm run build` runs `npm run embed-assets` (regenerates **`src/runtime/embedded-assets.ts`** from `docs/jaiph-skill.md`, and **`src/version.ts`** from `package.json`'s `version` field), then `tsc`, and copies **`src/runtime/`** to **`dist/src/runtime/`** (kernel, `docker.ts`, etc.). The published `jaiph` bin is **`node dist/src/cli.js`**. +- **Development / npm:** `npm run build` runs `npm run embed-assets` (regenerates **`src/runtime/embedded-assets.ts`** from `docs/jaiph-skill.md`, and **`src/version.ts`** from `package.json`'s `version` field), then `tsc`, and copies **`src/runtime/`** to **`dist/src/runtime/`** (kernel JS for the compiled CLI). The published `jaiph` bin is **`node dist/src/cli.js`**. - **Standalone:** `npm run build:standalone` runs the same build, copies **`dist/src/runtime`** to **`dist/runtime`** beside the binary, then `bun build --compile ./src/cli.ts --outfile dist/jaiph`. Workflow launch self-spawns via **`process.execPath`** using the internal **`__workflow-runner`** argv marker (`src/runtime/kernel/workflow-launch.ts` + `src/cli/index.ts`): the node build invokes `node dist/src/cli.js __workflow-runner …`; the bun-compiled binary invokes itself, `jaiph __workflow-runner …`. The reserved marker is excluded from `--help`/usage and the file-shorthand path. `docs/jaiph-skill.md` is also embedded base64 inside the executable via **`src/runtime/embedded-assets.ts`**, so the standalone artifact is **fully self-contained** — no sibling `runtime/` or `docs/` files required. The displayed `jaiph --version` string is sourced from the generated **`src/version.ts`** (codegen'd from `package.json` by `embed-assets`), so the literal is statically baked into both the `tsc` and the `bun build --compile` outputs without a runtime read of `package.json`. **Bash** (or whatever shebang your `script` steps use) is still required on the host for script subprocesses. Ship **`dist/jaiph`** alone, or with **`dist/runtime`** alongside it for parity with the npm layout (table in [Contributing](contributing.md)). - **Release artifacts:** `.github/workflows/release.yml` cross-compiles the standalone binary for **darwin/linux × arm64/x64** plus **windows x64** (`jaiph-windows-x64.exe`; Bun has no windows arm64 target) on **`v*`** tag pushes and on pushes to the **`nightly`** branch, generates a `SHA256SUMS` covering the five binaries, signs it with **minisign** (`SHA256SUMS.minisig`), runs `--version` sanity gates on the linux-x64 and windows-x64 outputs, and uploads the seven assets (five binaries + `SHA256SUMS` + `SHA256SUMS.minisig`) to the matching GitHub Release (stable tag or rolling **`nightly`** prerelease). Asset filenames are fixed by the installer contract — see [Contributing — Release asset naming contract](contributing.md#release-asset-naming-contract). Two installers consume these assets: the POSIX **`docs/install`** (`curl … | bash`, darwin/linux; rejects Windows and points at the PowerShell one) and **`docs/install.ps1`** (`irm https://jaiph.org/install.ps1 | iex`, Windows x64), which downloads `jaiph-windows-x64.exe`, verifies it against `SHA256SUMS` with `Get-FileHash`, and installs to `%LOCALAPPDATA%\jaiph\bin`. @@ -250,18 +247,14 @@ flowchart TD BS2 --> Transpile LMG2 --> TR[Node Test Runner in-process] - Transpile -->|jaiph run local| RW[__workflow-runner child] - Transpile -->|jaiph run Docker| DC[Container: jaiph run --raw] - LMG1 -. JAIPH_MODULE_GRAPH_FILE (local non-Docker only) .-> RW + Transpile -->|jaiph run| RW[__workflow-runner child] + LMG1 -. JAIPH_MODULE_GRAPH_FILE (non --raw) .-> RW RW --> G[buildRuntimeGraph from graph] G --> GRAPH[RuntimeGraph] RW --> RT[NodeWorkflowRuntime] RT --> GRAPH - DC --> G - DC --> RT - TR -->|test_run_workflow| G TR --> RT @@ -305,21 +298,14 @@ sequenceDiagram Prep->>TF: loop: validateModule + emit (in-memory AST) TF-->>Prep: scripts/ atomic only Prep-->>CLI: scriptsDir + env JAIPH_SCRIPTS - alt local (non-Docker) - CLI->>CLI: writeModuleGraph(/.jaiph-module-graph.json) - Note over CLI: set JAIPH_MODULE_GRAPH_FILE on child env - CLI->>Runner: spawn detached __workflow-runner child - else Docker - CLI->>CLI: prepareImage (pull --quiet + verify jaiph) - Note over CLI: runs before banner so pull doesn't interleave - CLI->>Runner: spawn container running jaiph run --raw - Note over CLI: CLI parses events on stderr only - end - alt JAIPH_MODULE_GRAPH_FILE set (local non-Docker) + CLI->>CLI: writeModuleGraph(/.jaiph-module-graph.json) + Note over CLI: set JAIPH_MODULE_GRAPH_FILE on child env (skipped for --raw) + CLI->>Runner: spawn detached __workflow-runner child + alt JAIPH_MODULE_GRAPH_FILE set Runner->>Runner: readModuleGraph(file) Runner->>Graph: buildRuntimeGraph(graph) Note over Graph: no .jh re-reads - else absent (Docker / --raw / test runner) + else absent (--raw / test runner) Runner->>Runner: loadModuleGraph(sourceAbs, workspace) Runner->>Graph: buildRuntimeGraph(graph) end @@ -333,8 +319,6 @@ sequenceDiagram CLI-->>User: PASS/FAIL ``` -**Docker:** the inner container command is **`jaiph run --raw …`** (see [Sandboxing](sandboxing.md)): no banner or progress UI inside the container; **`__JAIPH_EVENT__`** lines still appear on stderr for the host CLI to parse. - ## Sequence diagram: `jaiph test` flow ```mermaid @@ -373,7 +357,7 @@ sequenceDiagram - `.jh` / `*.test.jh` share parser/AST. The pipeline is **`loadModuleGraph` → `buildScriptsFromGraph(graph, outDir)`** (per-module **`validateModule`** + **`buildScriptFiles`** via **`emitScriptsForModuleFromGraph`**); `parsejaiph` is I/O-pure and graph-based validation / emit operate entirely in-memory. **`buildRuntimeGraph`** consumes the same `ModuleGraph` (loaded in the runner from disk or — on the default local **`jaiph run`** path — deserialized from the parent CLI's graph file via **`JAIPH_MODULE_GRAPH_FILE`**; see [Local module graph](#local-module-graph)). - **`jaiph compile`** walks import closures through **`collectDiagnostics(graph)`** (the multi-error sibling of **`validateReferences`**), prints the full diagnostic set sorted by `(file, line, col)`, and exits non-zero on any non-empty set — no **`scripts/`** emission (no **`buildScriptFiles`** / **`buildScripts`**), no **`buildRuntimeGraph()`**, no runner spawn. Directory discovery omits **`*.test.jh`** unless you pass a test file explicitly. -- **Node-only runtime:** all execution — local `jaiph run`, Docker `jaiph run`, and `jaiph test` — goes through `NodeWorkflowRuntime`. Docker containers run **`jaiph run --raw`** / **`__workflow-runner`** with the compiled JS tree and scripts mounted, using the same semantics as local execution. +- **Node-only runtime:** all execution — `jaiph run` and `jaiph test` — goes through `NodeWorkflowRuntime`. - **CLI** owns launch, observation, hooks (except **`jaiph run --raw`**), and runtime preparation (`buildScripts`). **`jaiph run --raw`** still emits **`__JAIPH_EVENT__`** on stderr from the runtime; the CLI does not attach the interactive progress/hooks pipeline. **`jaiph test`** passes **`suppressLiveEvents: true`** into **`NodeWorkflowRuntime`** so **`RuntimeEventEmitter`** skips writing those live stderr lines while **`run_summary.jsonl`** still records workflow traffic where the emitter appends it. - Workflow execution runs in **`NodeWorkflowRuntime`**, with **script steps** as managed subprocesses. - No workflow-level `.sh` files or `jaiph_stdlib.sh` are produced or required. diff --git a/docs/artifacts.md b/docs/artifacts.md index b48dbecc..4e23d268 100644 --- a/docs/artifacts.md +++ b/docs/artifacts.md @@ -9,7 +9,7 @@ redirect_from: # Save artifacts -This recipe publishes files from a workflow into the run's `artifacts/` directory under the run logs root (`.jaiph/runs/` by default). Copying files into `artifacts/` is the supported way to export them when Docker sandboxing is on. In the default snapshot mode, workspace edits are discarded when the container exits, but anything copied into `artifacts/` stays on the host. +This recipe publishes files from a workflow into the run's `artifacts/` directory under the run logs root (`.jaiph/runs/` by default). Copying files into `artifacts/` is the supported way to export them from a run. The runtime always creates an `artifacts/` directory under the run log directory and exposes its absolute path as `JAIPH_ARTIFACTS_DIR`. The `jaiphlang/artifacts` library is the standard way to copy files into that directory, and you can also write there directly from a `script` step. @@ -78,13 +78,13 @@ After the run, list the artifacts directory: ls //-/artifacts/ ``` -Replace `` with `.jaiph/runs` when `JAIPH_RUNS_DIR` is unset, or with your configured runs directory otherwise. The date and time segments are UTC, and `` is the entry-file basename (or `JAIPH_SOURCE_FILE` when set). You should see the files your workflow saved. Under Docker sandboxing the host path is the same. The run mount at `/jaiph/run` inside the container is bound to the host runs root, so artifacts land on the host even though the run executed inside the container. +Replace `` with `.jaiph/runs` when `JAIPH_RUNS_DIR` is unset, or with your configured runs directory otherwise. The date and time segments are UTC, and `` is the entry-file basename (or `JAIPH_SOURCE_FILE` when set). You should see the files your workflow saved. `artifacts.save(...)` fails when the input list is empty after trimming, when any listed path is missing or not a regular file, or when `JAIPH_ARTIFACTS_DIR` is unset. Wrap the call in `recover` or `catch` if you want the workflow to tolerate that failure. ## Verify a run's integrity chain -Every line the runtime appends to `run_summary.jsonl` carries a `prev_hash` field. The field holds a **keyed** HMAC-SHA256 of the previous raw line (keyed genesis for the first line), computed under a per-run secret the audited workflow never sees. Rewriting a line, or dropping a line and re-linking the survivors, breaks the chain and cannot be re-forged without the key, so you can detect tampering with a run's audit trail. The key is persisted once the run is terminal — **not** in the run directory (which the workflow can write to), but in an operator-side store outside every sandbox mount (`~/.jaiph/audit-keys` by default, override `JAIPH_AUDIT_KEY_DIR`), keyed by run-directory identity (finding M-3). See [Architecture — Keyed hash chain](architecture.md#hash-chain) for the full contract, including the key-isolation and read/export-boundary guarantees. +Every line the runtime appends to `run_summary.jsonl` carries a `prev_hash` field. The field holds a **keyed** HMAC-SHA256 of the previous raw line (keyed genesis for the first line), computed under a per-run secret the audited workflow never sees. Rewriting a line, or dropping a line and re-linking the survivors, breaks the chain and cannot be re-forged without the key, so you can detect tampering with a run's audit trail. The key is persisted once the run is terminal — **not** in the run directory (which the workflow can write to), but in an operator-side store (`~/.jaiph/audit-keys` by default, override `JAIPH_AUDIT_KEY_DIR`), keyed by run-directory identity. See [Architecture — Keyed hash chain](architecture.md#hash-chain) for the full contract, including the key-isolation and read/export-boundary guarantees. To check a run directory, run this self-contained Node script. It resolves the run's key from the operator store, where the `sha256` of the run directory's canonical path names its entry. It then recomputes the keyed chain the same way the runtime does and confirms the journal still ends with its `WORKFLOW_END` terminal marker. No jaiph build is required: @@ -118,4 +118,4 @@ A clean, complete journal prints `chain intact and terminal (N lines)` and exits - [Architecture — Durable artifact layout](architecture.md#durable-artifact-layout) — the full run directory tree, including where `artifacts/` sits, plus the hash chain and secret-redaction contracts for `run_summary.jsonl`. - [Use & publish a library](libraries.md) — installing `jaiphlang/artifacts` and writing your own libraries. -- [Sandboxing — The two sandbox modes](sandboxing.md#the-two-sandbox-modes) — snapshot mode discards workspace edits; artifacts persist on the host in every mode. +- [CLI](cli.md) — `jaiph run` artifacts layout. diff --git a/docs/cli.md b/docs/cli.md index dfd5197b..5c40ffc7 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -31,7 +31,7 @@ The reserved internal marker `__workflow-runner` is excluded from `--help`/usage | Subcommand | Purpose | |---|---| -| `run` | Compile, launch, and observe one workflow run (with optional Docker sandboxing). | +| `run` | Compile, launch, and observe one workflow run on the host. | | `test` | Execute `*.test.jh` blocks in-process with mocks. | | `compile` | Multi-error validation pass — no `scripts/` emission, no runtime spawn. | | `format` | Rewrite `.jh` / `.test.jh` files into canonical style. | @@ -47,27 +47,24 @@ The reserved internal marker `__workflow-runner` is excluded from `--help`/usage Compile and execute a workflow's `default` entrypoint. ```text -jaiph run [--target ] [--raw] [--workspace ] [--inplace] [--unsafe] [--yes|-y] [--env KEY[=VALUE]]... [--] [args...] +jaiph run [--target ] [--raw] [--workspace ] [--env KEY[=VALUE]]... [--] [args...] ``` -Sandbox selection is environment-driven; there is no `--docker` flag. The boolean sandbox flags (`--inplace`, `--unsafe`, `--yes`) are CLI front-ends that mutate the launched runtime env for one run only, and are shared verbatim with `jaiph serve` and `jaiph mcp` (one execution-policy contract; precedence: CLI flags > `JAIPH_*` env vars > workflow config metadata > defaults) — see [Configuration — Precedence](configuration.md#precedence) and [Environment variables — Precedence](env-vars.md#precedence). Flags belonging to another command (`--host`, `--port`) and unknown flags are usage errors, never positionals. +Every run executes on the host. Isolation is an outer concern: wrap jaiph in a container, a pod, or a CI runner if wanted. Shared flags (`--workspace`, `--env`) mean the same thing on `jaiph run`, `jaiph serve`, and `jaiph mcp` (precedence: CLI flags > `JAIPH_*` env vars > workflow config metadata > defaults) — see [Configuration — Precedence](configuration.md#precedence) and [Environment variables — Precedence](env-vars.md#precedence). Flags belonging to another command (`--host`, `--port`) and unknown flags are usage errors, never positionals. ### Flags | Flag | Argument | Effect | |---|---|---| | `--target` | `` | Keep emitted script files and run metadata under `` instead of a temp directory. | -| `--raw` | — | Skip the banner, live progress tree, hooks, and PASS/FAIL footer. The runner child inherits stdio; `__JAIPH_EVENT__` JSON lines go to stderr unchanged. Host `--raw` never launches Docker even when `JAIPH_DOCKER_ENABLED=true`. | -| `--workspace` | `` | Override the workspace root used for library resolution and the Docker workspace mount. A missing value, missing path, or non-directory aborts with a specific message. There is no `JAIPH_WORKSPACE` env equivalent input — that name is reserved for the in-container remap output. | -| `--inplace` | — | Front-end for `JAIPH_INPLACE=1`. On a TTY, prints a destructive-edit warning that **leads with the access scope** (edits land in this workspace directory only — `` — while the rest of your machine stays inside the Docker sandbox) plus the git-tree recovery posture, then requires `Continue? [y/N]` (default **no**). Non-TTY requires `--yes` / `JAIPH_INPLACE_YES` or aborts with `E_DOCKER_INPLACE_NO_CONFIRM`. | -| `--unsafe` | — | Front-end for `JAIPH_UNSAFE=true`. Cannot be combined with `--inplace` (`E_FLAG_CONFLICT`). When this turns Docker off while it would otherwise be on, a **stronger** confirmation than `--inplace` fires: the warning states host-only / **no sandbox**, that filesystem access is your **entire machine** (not just the workspace), and that scripts and agent backends can read secrets from your environment and reach paths outside the project. On a TTY it requires `Continue? [y/N]` (default **no**); non-TTY requires `--yes` / `JAIPH_INPLACE_YES` or aborts with `E_UNSAFE_NO_CONFIRM`. No prompt fires when Docker is off for another reason (explicit `JAIPH_DOCKER_ENABLED=false`, or the Windows host-only override, which prints its own notice). `--raw` skips this prompt (embedding / Docker inner run). | -| `-y`, `--yes` | — | Front-end for `JAIPH_INPLACE_YES=1`. Skips **both** the `--inplace` and `--unsafe` confirmation prompts — required to use either mode non-interactively. | -| `--env` | `KEY=VALUE` or `KEY` | Repeatable per-key environment passthrough into the workflow process. `--env KEY=VALUE` defines `KEY` with that exact value (first `=` splits; the value may contain `=`; empty is allowed). `--env KEY` forwards the host's current value, aborting with `E_ENV_MISSING` before spawning if `KEY` is unset on the host. `KEY` must match `[A-Za-z_][A-Za-z0-9_]*` (else `E_ENV_INVALID`). Reserved sandbox-control keys (`JAIPH_UNSAFE`, `JAIPH_INPLACE`, `JAIPH_INPLACE_YES`, any `JAIPH_DOCKER_*`, the `JAIPH_TRUSTED_ENVS` opt-in, and the `JAIPH_TRUST_PROJECT_HOOKS` opt-in) and runtime-managed keys (`JAIPH_WORKSPACE`, `JAIPH_RUNS_DIR`, `JAIPH_RUN_ID`, `JAIPH_SCRIPTS`, `JAIPH_MODULE_GRAPH_FILE`, `JAIPH_SOURCE_ABS`, `JAIPH_META_FILE`, `JAIPH_AGENT_TRUSTED_WORKSPACE`, `JAIPH_RUN_WORKFLOW`) are rejected with `E_ENV_RESERVED` — use the sandbox flags or real env vars for those. **In a Docker sandbox `--env` is the per-key consent that crosses the fail-closed env allowlist verbatim** (added as explicit `-e KEY=VALUE` container args, winning over any allowlist-forwarded value); see [Sandboxing — Environment exposure](sandboxing.md#env-exposure). Values are never path-remapped. | +| `--raw` | — | Skip the banner, live progress tree, hooks, and PASS/FAIL footer. The runner child inherits stdio; `__JAIPH_EVENT__` JSON lines go to stderr unchanged. | +| `--workspace` | `` | Override the workspace root used for library resolution. A missing value, missing path, or non-directory aborts with a specific message. There is no `JAIPH_WORKSPACE` env equivalent input — that name is reserved for the runner. | +| `--env` | `KEY=VALUE` or `KEY` | Repeatable per-key environment passthrough into the workflow process. `--env KEY=VALUE` defines `KEY` with that exact value (first `=` splits; the value may contain `=`; empty is allowed). `--env KEY` forwards the host's current value, aborting with `E_ENV_MISSING` before spawning if `KEY` is unset on the host. `KEY` must match `[A-Za-z_][A-Za-z0-9_]*` (else `E_ENV_INVALID`). Runtime-managed keys (`JAIPH_WORKSPACE`, `JAIPH_RUNS_DIR`, `JAIPH_RUN_ID`, `JAIPH_SCRIPTS`, `JAIPH_MODULE_GRAPH_FILE`, `JAIPH_SOURCE_ABS`, `JAIPH_META_FILE`, `JAIPH_AGENT_TRUSTED_WORKSPACE`, `JAIPH_TRUST_PROJECT_HOOKS`) are rejected with `E_ENV_RESERVED`. Values are never path-remapped. | | `--` | — | End of Jaiph flags; remaining tokens are forwarded to `workflow default`. | ### Pre-flight -After module-graph load and Docker-mode resolution, before the runner / container is spawned, the host CLI runs a credential pre-flight (`src/cli/run/preflight-credentials.ts`). Missing credentials produce either `E_AGENT_CREDENTIALS` (hard error) or a warning depending on backend and Docker mode — see [Authenticate agent backends](agent-auth.md) and [Configuration — Credential pre-flight](configuration.md#credential-pre-flight). `jaiph run --raw` does not run the pre-flight. +After module-graph load, before the runner is spawned, the host CLI runs a credential pre-flight (`src/cli/run/preflight-credentials.ts`). Missing credentials produce either `E_AGENT_CREDENTIALS` (hard error) or a warning depending on backend — see [Authenticate agent backends](agent-auth.md) and [Configuration — Credential pre-flight](configuration.md#credential-pre-flight). `jaiph run --raw` does not run the pre-flight. ### Progress markers @@ -104,11 +101,11 @@ Step `.out` files are written incrementally; consumers may `tail -f` them. `.out ### Failure footer -Interactive `jaiph run` only (`--raw` omits this block). On non-zero exit, the CLI emits a stderr footer with `Logs:`, `Summary:`, `out:` / `err:` paths, and an `Output of failed step:` excerpt. The fields are resolved from the last `STEP_END` object with non-zero `status` in `run_summary.jsonl`; `out_content` / `err_content` are preferred over `out_file` / `err_file`. In Docker mode, container-internal `/jaiph/run/*` paths are remapped to host paths. +Interactive `jaiph run` only (`--raw` omits this block). On non-zero exit, the CLI emits a stderr footer with `Logs:`, `Summary:`, `out:` / `err:` paths, and an `Output of failed step:` excerpt. The fields are resolved from the last `STEP_END` object with non-zero `status` in `run_summary.jsonl`; `out_content` / `err_content` are preferred over `out_file` / `err_file`. ### Hook events -Hooks load from `~/.jaiph/hooks.json` (global) and `/.jaiph/hooks.json` (project-local; project overrides global per event). Hooks run on the **host** CLI process even in Docker mode. The project-local file runs only when the operator trusts the workspace with `JAIPH_TRUST_PROJECT_HOOKS=1`; absent the opt-in it is ignored with a stderr notice while the global file still runs (finding M-10). See [Add a hook](hooks.md) and [`JAIPH_TRUST_PROJECT_HOOKS`](env-vars.md). +Hooks load from `~/.jaiph/hooks.json` (global) and `/.jaiph/hooks.json` (project-local; project overrides global per event). Hooks run on the **host** CLI process. The project-local file runs only when the operator trusts the workspace with `JAIPH_TRUST_PROJECT_HOOKS=1`; absent the opt-in it is ignored with a stderr notice while the global file still runs. See [Add a hook](hooks.md) and [`JAIPH_TRUST_PROJECT_HOOKS`](env-vars.md). ## `jaiph test` @@ -290,7 +287,7 @@ Implementation: with no `JAIPH_INSTALL_COMMAND` override, `jaiph use` downloads Serve a file's workflows as [MCP](https://modelcontextprotocol.io/) tools over stdio. See [Serve workflows as MCP tools](mcp.md) for the recipe and client-registration steps. ```text -jaiph mcp [--workspace ] [--inplace] [--unsafe] [--yes|-y] [--env KEY[=VALUE]]... +jaiph mcp [--workspace ] [--env KEY[=VALUE]]... ``` `jaiph --mcp ` is an equivalent alias, dispatched after `compile` in `src/cli/index.ts`. @@ -298,10 +295,7 @@ jaiph mcp [--workspace ] [--inplace] [--unsafe] [--yes|-y] [--env KEY[=VALU | Flag | Argument | Effect | |---|---|---| | `--workspace` | `` | Workspace root for import resolution (default: auto-detected from the file's directory). A missing value or non-directory path aborts with a specific message. | -| `--env` | `KEY=VALUE` or `KEY` | Same per-key passthrough as `jaiph run --env` (same forms, validation, and reserved-key rejection), resolved once at startup and applied to **every** tool call for the server's lifetime. A bare `--env KEY` unset on the host aborts server startup with `E_ENV_MISSING`. In Docker mode the pairs cross the container boundary as explicit `-e` args bypassing the allowlist, exactly as for `jaiph run --env`. | -| `--inplace` | — | Front-end for `JAIPH_INPLACE=1`: every tool call's Docker sandbox bind-mounts the host workspace read-write. Mutually exclusive with `--unsafe` (`E_FLAG_CONFLICT` at startup, before anything is spawned). No interactive prompt — launching the server with the flag (or env var) is the consent; the effective posture is printed once at startup and applied to every call. | -| `--unsafe` | — | Front-end for `JAIPH_UNSAFE=true`: every tool call runs on the host with no sandbox. Host-only mode requires explicit consent on this command line, so pass `--unsafe` (or `--yes`). An inherited `JAIPH_UNSAFE=true` with no such flag is refused at startup with `E_UNSAFE_NO_CONSENT` (skipped inside a container, where the container is the sandbox). Mutually exclusive with `--inplace` (`E_FLAG_CONFLICT`). When consent is given, the server prints a prominent SANDBOXING DISABLED banner at startup. | -| `-y`, `--yes` | — | Front-end for `JAIPH_INPLACE_YES=1` (recorded on every call's env; servers themselves never prompt). | +| `--env` | `KEY=VALUE` or `KEY` | Same per-key passthrough as `jaiph run --env` (same forms, validation, and reserved-key rejection), resolved once at startup and applied to **every** tool call for the server's lifetime. A bare `--env KEY` unset on the host aborts server startup with `E_ENV_MISSING`. | | `-h`, `--help` | — | Print the subcommand usage and exit `0`. | Flags that belong to another command (for example `--raw` or `--port`) are usage errors naming the owning command — never silently ignored. Precedence across layers is the shared execution-policy order: CLI flags > `JAIPH_*` env vars > workflow config metadata > defaults (see [Environment variables — Precedence](env-vars.md#precedence)). @@ -310,19 +304,19 @@ Flags that belong to another command (for example `--raw` or `--port`) are usage - Loads the module graph and runs `collectDiagnostics` (the same compile-time pass as `jaiph compile`). Any diagnostic prints `file:line:col CODE message` lines to **stderr** and exits `1`. - A missing path, a non-`.jh` path, or a path that is not a file exits `1` with a message on stderr. -- On success the server runs until stdin closes or it receives `SIGINT` / `SIGTERM`. Shutdown is **drain-then-cancel**: stdin closing (or the first signal) stops accepting input and waits for in-flight calls to finish before cleaning up and exiting `0` — a draining call keeps its scripts until it settles. A **second** signal cancels the in-flight calls instead of waiting: each run's child process tree is terminated (`SIGINT`, then `SIGKILL` after a grace period) and, in Docker mode, its container is force-removed (`docker rm -f`), so no child process or container outlives the server; the killed calls settle with error results and the server still exits `0`. +- On success the server runs until stdin closes or it receives `SIGINT` / `SIGTERM`. Shutdown is **drain-then-cancel**: stdin closing (or the first signal) stops accepting input and waits for in-flight calls to finish before cleaning up and exiting `0` — a draining call keeps its scripts until it settles. A **second** signal cancels the in-flight calls instead of waiting: each run's child process tree is terminated (`SIGINT`, then `SIGKILL` after a grace period), so no child process outlives the server; the killed calls settle with error results and the server still exits `0`. ### stdout invariant -From the moment the server starts, **stdout carries only newline-delimited JSON-RPC**. Every banner, warning, workflow-exclusion notice, reload message, Docker notice, and credential-pre-flight warning goes to **stderr**. Each outbound protocol message is a single atomic write of `JSON.stringify(msg) + "\n"`. +From the moment the server starts, **stdout carries only newline-delimited JSON-RPC**. Every banner, warning, workflow-exclusion notice, reload message, and credential-pre-flight warning goes to **stderr**. Each outbound protocol message is a single atomic write of `JSON.stringify(msg) + "\n"`. ### Operator log (stderr) `jaiph mcp` and `jaiph serve` write an **operator log to stderr only**. They never write it to the protocol channel, so MCP stdout stays JSON-RPC and HTTP response bodies stay API payloads. The operator log is **not** a logging framework, and Jaiph adds no winston, pino, or bunyan for it. It is a thin labelled writer that prints one line at a time to stderr and reuses the same level and color formatting as the `jaiph run` progress tree. Colors are used only when the stderr sink is a terminal and `NO_COLOR` is unset. -On every tool call or run the operator log writes two lines. The start line names the workflow, the sandbox label, and the run id, for example `jaiph mcp: Running () run_id=…`. The sandbox label uses the same words as the startup banner, which are snapshot, in-place, unsafe, and no sandbox. The end line reports the terminal status, the exit code, the elapsed time, and the run dir when it is known, for example `jaiph mcp: Finished status=ok exit=0 elapsed_ms=… rundir=…`. On `jaiph serve` both lines also carry `principal=` and `correlation=`. +On every tool call or run the operator log writes two lines. The start line names the workflow and the run id, for example `jaiph mcp: Running run_id=…`. The end line reports the terminal status, the exit code, the elapsed time, and the run dir when it is known, for example `jaiph mcp: Finished status=ok exit=0 elapsed_ms=… rundir=…`. On `jaiph serve` both lines also carry `principal=` and `correlation=`. -You can change how much the operator log prints with two environment variables, both documented in [Environment variables](env-vars.md): +Two environment variables change how much the operator log prints, both documented in [Environment variables](env-vars.md): - `JAIPH_SERVER_LOG=debug` prints the servers' extra `debug` diagnostic lines. - `JAIPH_SERVER_LOG_WORKFLOW=1` mirrors each workflow `log`, `logwarn`, and `logerr` event to the operator log. Each mirrored line is colored by level and carries `run_id=` and the same depth and async-branch subscript indent as the run tree. Mirroring is off by default, so an MCP host is not flooded and the tool-result text is not repeated. Mirrored lines go through the same credential redaction as the durable run journal, so a secret is never printed to stderr. @@ -336,8 +330,8 @@ Newline-delimited JSON-RPC 2.0. Requests are handled concurrently (a long `tools | `initialize` | Replies with `protocolVersion`, `capabilities: {tools: {listChanged: true}}`, and `serverInfo: {name: "jaiph", title: "Jaiph workflows", version}`. Echoes the client's `protocolVersion` if it is one of `2024-11-05`, `2025-03-26`, `2025-06-18`; otherwise replies with the newest of that set. | | `ping` | Empty result. | | `tools/list` | `{tools: [{name, description, inputSchema}]}` from the current tool set (re-read per request, so hot reload needs no cache invalidation). | -| `tools/call` | Runs the workflow (Docker sandbox or host, per the env — see Execution below). Result: `{content: [{type: "text", text}], isError}`. When `params._meta.progressToken` is present, the run's `STEP_START` / `STEP_END` events stream as `notifications/progress` until the response is sent (see below). | -| `notifications/cancelled` | Cancels the matching in-flight `tools/call` (`params.requestId`): terminates the run's child process tree (SIGINT, then SIGKILL after a grace period) and, in Docker mode, force-removes the call's container (`docker rm -f`) so it cannot orphan; sends **no response** for that id, and keeps the server serving. A cancellation for an unknown or already-finished id is a no-op. | +| `tools/call` | Runs the workflow on the host. Result: `{content: [{type: "text", text}], isError}`. When `params._meta.progressToken` is present, the run's `STEP_START` / `STEP_END` events stream as `notifications/progress` until the response is sent (see below). | +| `notifications/cancelled` | Cancels the matching in-flight `tools/call` (`params.requestId`): terminates the run's child process tree (SIGINT, then SIGKILL after a grace period); sends **no response** for that id, and keeps the server serving. A cancellation for an unknown or already-finished id is a no-op. | | other notifications | Ignored (`notifications/initialized`, …); no response. | | unknown request | JSON-RPC error `-32601`. | @@ -370,8 +364,7 @@ Tool descriptions come from the `#` comment lines directly above each workflow ( ### Execution and hot reload -- Tool calls honor the same env-driven sandbox selection as `jaiph run` (`resolveDockerConfig`): Docker on macOS/Linux by default, and host-only on Windows or when you consent to unsafe mode with the explicit `--unsafe` (an inherited `JAIPH_UNSAFE=true` alone is refused; see the flag table). The image is prepared once at startup (`checkDockerAvailable` + `prepareImage`), not per call. Run artifacts land under `.jaiph/runs/` exactly as for `jaiph run`. -- **The workspace is isolated by default** for `jaiph mcp` — the same as `jaiph run`. Each tool call's container works on a writable point-in-time snapshot of the workspace, so edits are discarded on exit and the host tree is untouched. Pass `--inplace` (or set `JAIPH_INPLACE=1`) to bind the real workspace read-write so tool effects land live (opt-in), or `--unsafe` to run on the host with no sandbox (host-only mode needs the explicit flag; an inherited `JAIPH_UNSAFE=true` alone is refused, see the flag table). The posture is resolved and printed once at startup and applied to every call. +- Tool calls execute on the host, the same as `jaiph run`. Run artifacts land under `.jaiph/runs/` exactly as for `jaiph run`. Concurrent calls each get their own run id and run directory. Two calls that change the same files can race. - Source files in the module graph are watched (polling, ~750 ms). A valid edit re-derives tools and emits `notifications/tools/list_changed`; an edit that fails to compile keeps the previous tool set serving and logs diagnostics to stderr. - Calls bind to the generation (emitted scripts + serialized graph) live when they start; a superseded generation's scripts dir survives until its last in-flight call settles, so a call spanning a reload still runs its remaining steps — the same lease model `jaiph serve` uses for HTTP runs. @@ -381,7 +374,7 @@ Tool descriptions come from the `#` comment lines directly above each workflow ( Serve a file's workflows as an HTTP API with a generated OpenAPI 3.1 document and an embedded Swagger UI. Same exposure rules and execution layer as `jaiph mcp`, over HTTP instead of stdio. See [Serve workflows over HTTP](serve.md) for the recipe. ```text -jaiph serve [--host ] [--port ] [--workspace ] [--allow-anonymous] [--inplace] [--unsafe] [--yes|-y] [--env KEY[=VALUE]]... +jaiph serve [--host ] [--port ] [--workspace ] [--allow-anonymous] [--env KEY[=VALUE]]... ``` | Flag | Argument | Effect | @@ -391,14 +384,11 @@ jaiph serve [--host ] [--port ] [--workspace ] [--allow-anonymous] | `--allow-anonymous` | — | Explicit opt-in to run open with no authentication on loopback. Without it, a loopback bind with no `JAIPH_SERVE_TOKEN` and no OIDC aborts startup, because anonymous mode authorizes every local principal with all capabilities over all runs (loopback guards the network, not other local users — finding M-2). For a single-user workstation only; shared hosts must set `JAIPH_SERVE_TOKEN` or configure OIDC. When passed, the server prints a startup warning that it is open to all local principals. Ignored (no-op) when a token or OIDC is configured, and it never permits a non-loopback bind. | | `--workspace` | `` | Workspace root for import resolution (default: auto-detected). | | `--env` | `KEY=VALUE` or `KEY` | Same per-key passthrough as `jaiph run --env`, resolved once at startup and applied to every run for the server's lifetime. | -| `--inplace` | — | Front-end for `JAIPH_INPLACE=1`: every run's Docker sandbox bind-mounts the host workspace read-write. Mutually exclusive with `--unsafe` (`E_FLAG_CONFLICT` at startup, before anything is spawned). No interactive prompt — launching the server with the flag (or env var) is the consent; the effective posture is printed once at startup and applied to every run. | -| `--unsafe` | — | Front-end for `JAIPH_UNSAFE=true`: every run executes on the host with no sandbox. Host-only mode requires explicit consent on this command line, so pass `--unsafe` (or `--yes`). An inherited `JAIPH_UNSAFE=true` with no such flag is refused at startup with `E_UNSAFE_NO_CONSENT` (skipped inside a container, where the container is the sandbox). Mutually exclusive with `--inplace` (`E_FLAG_CONFLICT`). When consent is given, the server prints a prominent SANDBOXING DISABLED banner at startup. | -| `-y`, `--yes` | — | Front-end for `JAIPH_INPLACE_YES=1` (recorded on every run's env; servers themselves never prompt). | | `-h`, `--help` | — | Print the subcommand usage and exit `0`. | Flags that belong to another command (for example `--raw` or `--target`) are usage errors naming the owning command — never silently ignored. Precedence across layers is the shared execution-policy order: CLI flags > `JAIPH_*` env vars > workflow config metadata > defaults (see [Environment variables — Precedence](env-vars.md#precedence)). -Startup mirrors `jaiph mcp`: graph load + `collectDiagnostics` (diagnostics to stderr, exit `1`), one-time Docker image preparation, credential pre-flight as warnings, and a sandbox-posture notice. All logs go to stderr; one startup line prints the listen URL and the `/docs` URL. Per-run operator lines (a start line `Running … run_id=` and an end line `Finished … status=… elapsed_ms=…`) and the optional workflow-log mirror follow the same stderr-only operator-log contract as `jaiph mcp`, and HTTP response bodies stay API payloads. See [Operator log (stderr)](#operator-log-stderr) above, and `JAIPH_SERVER_LOG` and `JAIPH_SERVER_LOG_WORKFLOW` in [Environment variables](env-vars.md). +Startup mirrors `jaiph mcp`: graph load + `collectDiagnostics` (diagnostics to stderr, exit `1`), credential pre-flight as warnings, and a host-execution notice. All logs go to stderr; one startup line prints the listen URL and the `/docs` URL. Per-run operator lines (a start line `Running … run_id=` and an end line `Finished … status=… elapsed_ms=…`) and the optional workflow-log mirror follow the same stderr-only operator-log contract as `jaiph mcp`, and HTTP response bodies stay API payloads. See [Operator log (stderr)](#operator-log-stderr) above, and `JAIPH_SERVER_LOG` and `JAIPH_SERVER_LOG_WORKFLOW` in [Environment variables](env-vars.md). ### Endpoints @@ -438,8 +428,7 @@ Each run's public record is persisted beside its journal as `run.json` when it f See [Environment variables](env-vars.md) for the complete inventory. The variables most relevant to CLI behaviour: -- `JAIPH_DOCKER_ENABLED`, `JAIPH_UNSAFE`, `JAIPH_INPLACE`, `JAIPH_INPLACE_YES` — sandbox enablement and mode. -- `JAIPH_DOCKER_IMAGE`, `JAIPH_DOCKER_IMAGE_DIGEST`, `JAIPH_DOCKER_NETWORK`, `JAIPH_DOCKER_TIMEOUT` — Docker mode parameters. +- `JAIPH_RUN_TIMEOUT` — parent-enforced wall-clock cap for a run. - `JAIPH_NON_TTY_HEARTBEAT_FIRST_SEC`, `JAIPH_NON_TTY_HEARTBEAT_INTERVAL_MS` — non-TTY progress cadence. - `JAIPH_RUNS_DIR`, `JAIPH_WORKSPACE`, `JAIPH_SOURCE_FILE` — run-layout inputs. - `JAIPH_INSTALL_COMMAND`, `JAIPH_REGISTRY`, `JAIPH_SKILL_PATH` — install / init inputs. diff --git a/docs/configuration.md b/docs/configuration.md index 7981b328..d6f43abc 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -9,7 +9,7 @@ redirect_from: # Configuration -This page is the authoritative inventory of Jaiph configuration keys: every key, its value type, default, environment-variable equivalent, and precedence. For environment-variable details (defaults, scopes, sandbox forwarding) see [Environment variables](env-vars.md). For the CLI flags that front-end the same knobs see [CLI](cli.md). +This page is the authoritative inventory of Jaiph configuration keys: every key, its value type, default, environment-variable equivalent, and precedence. For environment-variable details (defaults, scopes) see [Environment variables](env-vars.md). For the CLI flags that front-end the same knobs see [CLI](cli.md). Configuration sources, in priority order: @@ -18,7 +18,6 @@ Configuration sources, in priority order: 3. **Module-level `config { … }`** — applies to all workflows in that file unless overridden. 4. **Built-in defaults** — lowest priority. -Docker enablement uses a separate, env-only resolution; see [Docker enablement](#docker-enablement). ## Config block syntax @@ -26,7 +25,7 @@ Docker enablement uses a separate, env-only resolution; see [Docker enablement]( |---|---| | Module-level | At most one `config { … }` block per `.jh` file. May appear anywhere among top-level constructs. | | Workflow-level | At most one nested `config { … }` per workflow body. Must be the first non-comment construct in the body. | -| Allowed module keys | `agent.*`, `run.*`, `runtime.*`, `module.*`, and `trusted_envs`. | +| Allowed module keys | `agent.*`, `run.*`, `module.*`, and `trusted_envs`. | | Allowed workflow keys | `agent.*`, `run.*`, and `trusted_envs`. `runtime.*` and `module.*` are `E_PARSE`. | | Duplicate block | `E_PARSE duplicate config block (only one allowed per file)` / `E_PARSE duplicate config block inside workflow (only one allowed per workflow)`. | | Unknown key | `E_PARSE unknown config key: . Allowed: …` (lists every allowed key). | @@ -115,39 +114,10 @@ workflow publish { Semantics: - Declared keys resolve from the **pristine host environment captured once at process start** — never from the calling workflow's scope env. A sub-workflow does not inherit a caller's keys by being called; it must declare `trusted_envs` itself. -- Resolved values are injected **only into `run`-step script subprocesses** of the declaring workflow. They are **never** forwarded to `prompt` agent subprocesses — the prompt env stays the fail-closed allowlist described in [Sandboxing](sandboxing.md), in every sandbox mode. +- Resolved values are injected **only into `run`-step script subprocesses** of the declaring workflow. They are **never** forwarded to `prompt` agent subprocesses — the prompt env stays the fail-closed allowlist (base env, `JAIPH_*` control keys, and that backend's own credential keys). - Declaring a key anywhere in the file (or an imported module) also **scrubs** it from every workflow's ambient scope env, so only the declaring workflow's `run` steps see it. -- Pre-flight: a declared key with no value on the host (and no `--env` override) aborts before anything is spawned (`E_ENV_MISSING`). Reserved keys (the `--env` `E_ENV_RESERVED` set, including `JAIPH_DOCKER_*`) are rejected at parse time. +- Pre-flight: a declared key with no value on the host (and no `--env` override) aborts before anything is spawned (`E_ENV_MISSING`). Reserved keys (the `--env` `E_ENV_RESERVED` set) are rejected at parse time. - `--env KEY=VALUE` remains the imperative override: it wins over the host-snapshot value for the same key. -- Docker: the entry file's resolved keys cross the sandbox boundary through the same explicit `-e` channel as `--env` pairs — **but only when the operator opts in** with `JAIPH_TRUSTED_ENVS=1`. **Authoring the entry file is a trust boundary equal to `--env`:** an untrusted or model-edited entry could name arbitrary host secrets (`AWS_SECRET_ACCESS_KEY`, `GITHUB_TOKEN`) and pull them across the allowlist the sandbox exists to enforce (finding M-7). Absent the opt-in, the entry file's `trusted_envs` is ignored under Docker (with a pre-flight warning) and nothing is forwarded. Host modes have no allowlist to bypass (the runner inherits the host env directly), so they honour the declaration regardless. See [`JAIPH_TRUSTED_ENVS`](env-vars.md). - -## Runtime (Docker) keys - -These configure the Docker sandbox. Allowed in **module-level** config only. They are read by the host CLI when it considers a Docker launch (`resolveDockerConfig` in `src/runtime/docker.ts`) and never affect `NodeWorkflowRuntime` directly. **Docker on/off is not a `runtime.*` key** — see [Docker enablement](#docker-enablement). - -| Key | Type | Default | Env equivalent | Notes | -|---|---|---|---|---| -| `runtime.docker_image` | string | `ghcr.io/jaiphlang/jaiph-runtime:` | `JAIPH_DOCKER_IMAGE` | Container image. Must already contain `jaiph` (`E_DOCKER_NO_JAIPH` otherwise). **Host-controlled:** an in-file value is rejected (`E_DOCKER_IMAGE_HOST_ONLY`) when Docker is the active sandbox; set a non-default image only through `JAIPH_DOCKER_IMAGE`. | -| `runtime.docker_network` | string | `default` | `JAIPH_DOCKER_NETWORK` | `docker run --network` value. `none` disables egress. **Host-controlled for isolation-breaking values:** an in-file `host`, `container:*`, or `ns:*` is rejected (`E_DOCKER_NETWORK_HOST_ONLY`) when Docker is the active sandbox — these dissolve the sandbox network boundary. Host-safe in-file values (`default`, `none`, a named bridge network) are honoured; the operator may still select any value, including `host`, through `JAIPH_DOCKER_NETWORK`. | -| `runtime.docker_timeout_seconds` | integer | `14400` | `JAIPH_DOCKER_TIMEOUT` | Container execution timeout in seconds. `0` disables. Negative or invalid env value produces `E_DOCKER_TIMEOUT`. | - -In-file `runtime.docker_enabled` is not supported (`E_PARSE`); use the env-only enablement below. In the same spirit, `runtime.docker_image` and isolation-breaking `runtime.docker_network` values are host-controlled: a repo- or model-supplied entry file cannot point the sandbox at an arbitrary image or gut its network isolation (finding M-6). When Docker is off (host / `JAIPH_UNSAFE` mode) these keys are inert and not enforced. - -The default official image is also pinned by manifest digest. The expected digest ships with the release, and every run verifies the local image against it and fails closed on a mismatch. There is no config-file key for the digest, so set or override it with the [`JAIPH_DOCKER_IMAGE_DIGEST`](env-vars.md) environment variable, which also lets you pin a custom `JAIPH_DOCKER_IMAGE`. - -## Docker enablement - -Checks are applied top to bottom; the first match wins. - -| Check | Result | -|---|---| -| Platform is Windows (`win32`) | Docker off (host-only mode, with a one-line notice). Overrides everything below, including `JAIPH_DOCKER_ENABLED=true`. | -| `JAIPH_DOCKER_ENABLED` is set to exact `true` | Docker on. | -| `JAIPH_DOCKER_ENABLED` is set to any other value | Docker off. | -| `JAIPH_DOCKER_ENABLED` is unset and `JAIPH_UNSAFE=true` | Docker off. | -| Default (no env) | Docker on. | - -`CI=true` does not change this default. Host `jaiph run --raw` never consults this branch — the workflow runner is local in that path. On Windows the Docker sandbox is out of scope, so `jaiph run` resolves to host-only mode automatically without probing `docker` or failing on a missing daemon — see [Sandboxing — Windows runs host-only](sandboxing.md#windows-runs-host-only) for the full model. ## Precedence {: #precedence} @@ -161,17 +131,6 @@ Checks are applied top to bottom; the first match wins. | Module-level `config` | Applies to workflows without their own block. | | Built-in defaults | Lowest priority. | -### Runtime (Docker) keys - -| Layer | Effect | -|---|---| -| CLI flags (`--inplace`, `--unsafe`, `--yes` on `jaiph run` / `jaiph serve` / `jaiph mcp`) | Set the corresponding `JAIPH_*` variable on the launched env for that process, so the env layer below stays the single source of truth. | -| Environment (`JAIPH_DOCKER_*`, `JAIPH_UNSAFE`, `JAIPH_INPLACE`) | Highest env-layer priority for `image`, `network`, `timeout`, and sandbox posture. | -| Module-level `config` (`runtime.*`) | Applies when no env override is set. | -| Built-in defaults | Lowest priority. | - -Workflow-level `config` cannot set `runtime.*` keys. Contradictory posture (`--inplace`/`JAIPH_INPLACE` together with `--unsafe`/`JAIPH_UNSAFE`) is rejected with `E_FLAG_CONFLICT` before anything is spawned rather than resolved by precedence — see [Environment variables — Precedence](env-vars.md#precedence). - ### Scoping across nested calls | Call type | Scope behaviour | @@ -222,9 +181,6 @@ The existing `JAIPH_AGENT_COMMAND_LOCKED=1` / `JAIPH_AGENT_BACKEND_LOCKED=1` fla | `run.logs_dir` | `JAIPH_RUNS_DIR` | | `run.debug` | `JAIPH_DEBUG` | | `run.recover_limit` | _(no env override)_ | -| `runtime.docker_image` | `JAIPH_DOCKER_IMAGE` | -| `runtime.docker_network` | `JAIPH_DOCKER_NETWORK` | -| `runtime.docker_timeout_seconds` | `JAIPH_DOCKER_TIMEOUT` | | `module.name` | _(no env override)_ | | `module.version` | _(no env override)_ | | `module.description` | _(no env override)_ | @@ -245,17 +201,17 @@ Backend-specific flags come from `agent.cursor_flags` / `agent.claude_flags` (or ### Credential pre-flight {: #credential-pre-flight} -Before `jaiph run` spawns the workflow runner or Docker container, the host CLI runs a credential pre-flight (`src/cli/run/preflight-credentials.ts`). It collects the distinct backend(s) declared in the entry file's module-level `config` block and each workflow-level block, plus the effective default (`JAIPH_AGENT_BACKEND` env, or `cursor` when unset). Deeper per-import overrides resolved at runtime are not followed. +Before `jaiph run` spawns the workflow runner, the host CLI runs a credential pre-flight (`src/cli/run/preflight-credentials.ts`). It collects the distinct backend(s) declared in the entry file's module-level `config` block and each workflow-level block, plus the effective default (`JAIPH_AGENT_BACKEND` env, or `cursor` when unset). Deeper per-import overrides resolved at runtime are not followed. -| Backend | Required credential | Host run (no Docker) | Docker run (any mode incl. `inplace`) | -|---|---|---|---| -| `codex` | `OPENAI_API_KEY` | hard error (`E_AGENT_CREDENTIALS`) | hard error (`E_AGENT_CREDENTIALS`) | -| `claude` | `ANTHROPIC_API_KEY` or `CLAUDE_CODE_OAUTH_TOKEN` | warn (CLI login may still work) | hard error (`E_AGENT_CREDENTIALS`) | -| `cursor` | `CURSOR_API_KEY` | warn (CLI login may still work) | hard error (`E_AGENT_CREDENTIALS`) | +| Backend | Required credential | Host behaviour | +|---|---|---| +| `codex` | `OPENAI_API_KEY` | hard error (`E_AGENT_CREDENTIALS`) | +| `claude` | `ANTHROPIC_API_KEY` or `CLAUDE_CODE_OAUTH_TOKEN` | warn (CLI login may still work) | +| `cursor` | `CURSOR_API_KEY` | warn (CLI login may still work) | -Hard errors exit non-zero with no runner or container launched. Warnings go to stderr and the run proceeds. Skip cases: entry file declares no explicit backend and uses no `prompt` step → no pre-flight; `jaiph run --raw` → no pre-flight; `JAIPH_UNSAFE=true` / `--unsafe` → no pre-flight (host escape hatch — runtime backend guards remain). +Hard errors exit non-zero with no runner launched. Warnings go to stderr and the run proceeds. Skip cases: entry file declares no explicit backend and uses no `prompt` step → no pre-flight; `jaiph run --raw` → no pre-flight. -Every error and warning names: the backend; the model when `agent.model` is set; the entry `.jh` file; the config scope (`module config`, `workflow `, `JAIPH_AGENT_BACKEND env`, or `default`); and the concrete remedy. Docker-mode messages also note that the variable must be set on the host so it gets forwarded. +Every error and warning names:Every error and warning names: the backend; the model when `agent.model` is set; the entry `.jh` file; the config scope (`module config`, `workflow `, `JAIPH_AGENT_BACKEND env`, or `default`); and the concrete remedy. ## Model resolution {: #model-resolution} @@ -287,7 +243,7 @@ For the Claude backend, when `agent.model` is set and `agent.claude_flags` does | 5 | 30m | | 6 | 2h | -Total worst-case wall-clock: ~2h41m. Under Docker, `runtime.docker_timeout_seconds` caps this. +Total worst-case wall-clock: ~2h41m. Only transport failures are retried (non-zero exit from cursor/claude, codex HTTP error, spawn failure). Deterministic post-processing failures — invalid JSON, schema validation — fail on the first attempt and return `{ ok: false }`. @@ -313,14 +269,14 @@ The retry backoff above handles a backend that *fails*. A separate set of watchd Set any variable to `0` to disable that layer. The idle timer resets on every chunk of backend output, so a slow-but-active run is bounded only by the absolute cap. -The completion-grace layer specifically addresses the known `claude -p` failure mode where the CLI streams its final answer (and the terminal `result` event) but the process never exits — often because a descendant it spawned is still holding the output pipe open. When a watchdog fires it terminates the backend's whole process tree (via `killProcessTree`; see [Architecture](architecture.md)) with `SIGTERM`, escalating to `SIGKILL` after 5s, and tears down the runtime's handles on the child's stdio so a lingering descendant cannot keep the run alive. On Windows the tree is force-killed with `taskkill /T` on the first signal, so the `SIGKILL` escalation is a no-op. Under Docker, `runtime.docker_timeout_seconds` remains the outer backstop for the whole container. +The completion-grace layer specifically addresses the known `claude -p` failure mode where the CLI streams its final answer (and the terminal `result` event) but the process never exits — often because a descendant it spawned is still holding the output pipe open. When a watchdog fires it terminates the backend's whole process tree (via `killProcessTree`; see [Architecture](architecture.md)) with `SIGTERM`, escalating to `SIGKILL` after 5s, and tears down the runtime's handles on the child's stdio so a lingering descendant cannot keep the run alive. On Windows the tree is force-killed with `taskkill /T` on the first signal, so the `SIGKILL` escalation is a no-op. ## Overall run timeout and step cap {: #overall-run-timeout-and-step-cap} The prompt watchdogs above bound a single backend call. Jaiph also has two controls that bound the whole run, and both are off by default, so existing runs behave as before. -`JAIPH_RUN_TIMEOUT` sets a parent-enforced wall-clock cap, in seconds, for a host-mode run. Host mode means a `jaiph run --unsafe` or host-only run, and the host spawn that a `jaiph serve` or `jaiph mcp` call uses. Without this cap, the only automatic stop for a host run is a manual Ctrl-C, because the host spawn installs only SIGINT and SIGTERM handlers and the prompt watchdogs cover a single backend call. When the cap is reached, the parent terminates the run child's whole process group with `SIGTERM` and escalates to `SIGKILL` after a short grace period (via `killProcessTree`; see [Architecture](architecture.md)), so the run stops without a manual Ctrl-C, and the failure footer shows `E_RUN_TIMEOUT`. Set it to `0`, leave it empty, or give it an invalid value to disable it, which restores the earlier behaviour where only a manual SIGINT or SIGTERM stops a host run. Docker mode does not use this variable, because a Docker run is already bounded by `runtime.docker_timeout_seconds` (`JAIPH_DOCKER_TIMEOUT`) inside the container. +`JAIPH_RUN_TIMEOUT` sets a parent-enforced wall-clock cap, in seconds, for a run. Without this cap, the only automatic stop is a manual Ctrl-C, because the host spawn installs only SIGINT and SIGTERM handlers and the prompt watchdogs cover a single backend call. When the cap is reached, the parent terminates the run child's whole process group with `SIGTERM` and escalates to `SIGKILL` after a short grace period (via `killProcessTree`; see [Architecture](architecture.md)), so the run stops without a manual Ctrl-C, and the failure footer shows `E_RUN_TIMEOUT`. Set it to `0`, leave it empty, or give it an invalid value to disable it, which restores the earlier behaviour where only a manual SIGINT or SIGTERM stops a run. `JAIPH_MAX_STEPS` sets an optional max-step circuit breaker in the runtime. When you set it to a positive integer, the runtime counts every executed step across the whole run, and it counts loop iterations and nested or recursive calls but skips trivia. Once the count goes past the cap, the runtime logs `E_MAX_STEPS`, aborts the run, and returns a failure, so a runaway workflow stops on its own without a manual signal. Set it to `0`, leave it empty, or give it an invalid value to disable the breaker. @@ -357,7 +313,6 @@ Custom commands still participate in `PROMPT_START` / `PROMPT_END`, write artifa Agent and run settings are visible inside workflows, rules, and scripts as `JAIPH_*` environment variables. In orchestration strings, `${IDENT}` resolves against workflow bindings first, then against the process environment. -`JAIPH_DOCKER_*` variables are not populated from in-file `runtime.*` inside the workflow runner. Docker config is consumed when the CLI spawns the runner (or container); if a script needs Docker-related variables in its environment, export them from the parent shell. ## Created by `jaiph init` @@ -367,5 +322,5 @@ Agent and run settings are visible inside workflows, rules, and scripts as `JAIP - [Environment variables](env-vars.md) — every variable Jaiph reads. - [CLI](cli.md) — flags that front-end these config knobs. -- [Sandboxing](sandboxing.md) — Docker sandbox model. +- [Deploy jaiph](deploy.md) — wrap jaiph in an image or pod for outer isolation. - [Grammar](grammar.md) — `config` block syntax in the formal grammar. diff --git a/docs/contributing.md b/docs/contributing.md index 867a8c41..77aad305 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -10,7 +10,7 @@ redirect_from: Contributor docs answer a narrow question: **where changes belong**, **how to run the same checks CI runs**, and **which test layer** should encode a behavior change. -At a high level, Jaiph is built as described in [Architecture](architecture.md) — **`loadModuleGraph`** → per-module **`validateModule`** + script emit via **`buildScriptsFromGraph`** / **`emitScriptsForModuleFromGraph`**, the same graph consumed by **`buildRuntimeGraph(graph)`**, validate-only **`jaiph compile`** (**`collectDiagnostics`**), **`NodeWorkflowRuntime`**, artifact layout, and Docker helper contracts. Treat that page as authoritative for pipelines and boundaries; if anything here diverges from it or from the implementation, prefer **architecture + source**. +At a high level, Jaiph is built as described in [Architecture](architecture.md) — **`loadModuleGraph`** → per-module **`validateModule`** + script emit via **`buildScriptsFromGraph`** / **`emitScriptsForModuleFromGraph`**, the same graph consumed by **`buildRuntimeGraph(graph)`**, validate-only **`jaiph compile`** (**`collectDiagnostics`**), **`NodeWorkflowRuntime`**, artifact layout. Treat that page as authoritative for pipelines and boundaries; if anything here diverges from it or from the implementation, prefer **architecture + source**. For workflow syntax, library usage, tooling setup, and grammar details, see [Language](language.md), [Install & switch versions](setup.md), [Grammar](grammar.md), and [Your first workflow](first-workflow.md). For the `*.test.jh` language and test blocks, see [Write & run tests](testing.md). @@ -38,11 +38,9 @@ jaiph --version jaiph --help ``` -The script builds the self-contained standalone binary via `docs/install` (`npm ci` when a lockfile is present, else `npm install`, plus `npm run build:standalone`, including uncommitted changes) and installs `dist/jaiph` to `~/.local/bin` by default (or `JAIPH_BIN_DIR` if set). It then builds `runtime/Dockerfile` from the same checkout and **retags it as the default sandbox image** — `ghcr.io/jaiphlang/jaiph-runtime:` plus `:nightly` — so Docker runs use your local build without setting `JAIPH_DOCKER_IMAGE`. The image build is **required** (the script exits if Docker is missing, the daemon is down, or the build fails). +The script builds the self-contained standalone binary via `docs/install` (`npm ci` when a lockfile is present, else `npm install`, plus `npm run build:standalone`, including uncommitted changes) and installs `dist/jaiph` to `~/.local/bin` by default (or `JAIPH_BIN_DIR` if set). -Set **`JAIPH_SKIP_DOCKER_BUILD=1`** only to skip the image build (installer acceptance tests). - -**From-source prerequisites:** **`npm`**, **[Bun](https://bun.sh)**, and a running **Docker** daemon (`docker info` must succeed). +**From-source prerequisites:** **`npm`** and **[Bun](https://bun.sh)**. ## Developing in the repository @@ -60,13 +58,13 @@ For day-to-day work on the compiler and CLI you usually stay inside the clone: i | `npm run build:standalone` | `npm run build`, then copies **`dist/src/runtime`** → **`dist/runtime`** and runs **`bun build --compile ./src/cli.ts --outfile ./dist/jaiph`**. Requires [Bun](https://bun.sh). The resulting **`dist/jaiph`** is **fully self-contained** — `jaiph-skill.md` is baked into the binary, and workflow launch self-spawns via the internal `__workflow-runner` argv marker, so the binary needs no sibling `runtime/` or `docs/` files and no `node` / `npm` on the host. The `dist/runtime` copy is kept for parity with the npm layout ([Architecture — Distribution](architecture.md#distribution-node-vs-bun-standalone)). | | `npm run arch:check` | Runs **dependency-cruiser** over `src/` with **`.dependency-cruiser.cjs`** to enforce the [Agent analyzability](agent-analyzability.md) import graph: no cycles, the layer DAG (each layer imports only downward, including `runtime` ↛ `cli`, and runtime reuses compile only through the single public entry `src/transpiler.ts`), `no-deep-imports-into-parse` (code outside the parse package imports only the public entry `src/parser.ts`, never a `src/parse/**` internal), `no-deep-imports-into-transpile` (code outside the transpile package imports only the single public entry `src/transpiler.ts`, which re-exports the module-graph API, never a `src/transpile/**` internal), `no-deep-imports-into-runtime` (code outside the runtime package imports only a public entry, `src/runtime/index.ts` for production or `src/runtime/testing.ts` for named test seams, never a `src/runtime/**` internal), `no-deep-imports-into-format` (code outside the format package imports only the public entry `src/format/index.ts`, never a `src/format/**` internal), and `no-cross-cli-slice-imports` (a file in one CLI slice such as `commands` or `serve` imports another slice's private tree only through `src/cli/shared/**` or a lower-layer public entry). Reads the TypeScript sources directly, so it needs no build. Pre-existing violations are grandfathered in **`.dependency-cruiser-known-violations.json`** (passed via `--ignore-known`), so old violations are tracked while a new cycle, upward import, parse, transpile, runtime, or format deep import, or cross-CLI-slice import fails the check. A required CI step on the Compiler and unit tests job. | | `npm run lint` | Runs **ESLint** over `src/` with **`eslint.config.mjs`** and `--max-warnings 0` to enforce the [Agent analyzability](agent-analyzability.md) fan-out and file-size caps: `import/max-dependencies` at 8 runtime imports per file (type imports ignored) and `max-lines` at 400 non-blank, non-comment lines. Test files are out of scope. Most files that once exceeded a cap were split into sibling modules and now pass the caps with no override; the four largest remaining files keep a per-file override in **`eslint.config.mjs`** that turns off only the rule they break, each with a fresh justification, and the global cap is never raised, so a new violation fails the check. A required CI step on the Compiler and unit tests job. | -| `npm test` | **`npm run clean`**, then **`npm run build`**, then the Node.js test runner with **`JAIPH_UNSAFE=true`**, **`NODE_OPTIONS`** including **`--enable-source-maps`** and a large heap limit, on every file under `dist/integration/` matching `*.test.js`, every file under `dist/src/` matching `*.test.js` or `*.acceptance.test.js` (via `find`), `scripts/build-registry.test.mjs`, `dist/test-infra/compiler-test-runner.js` (txtar compiler tests), and `dist/test-infra/golden-ast-runner.js` (golden AST tests). | +| `npm test` | **`npm run clean`**, then **`npm run build`**, then the Node.js test runner with **`NODE_OPTIONS`** including **`--enable-source-maps`** and a large heap limit, on every file under `dist/integration/` matching `*.test.js`, every file under `dist/src/` matching `*.test.js` or `*.acceptance.test.js` (via `find`), `scripts/build-registry.test.mjs`, `dist/test-infra/compiler-test-runner.js` (txtar compiler tests), and `dist/test-infra/golden-ast-runner.js` (golden AST tests). | | `npm run test:compiler` | **`npm run build`**, then **`node --test`** on `dist/test-infra/compiler-test-runner.js` — runs txtar-based compiler test fixtures from `test-fixtures/compiler-txtar/`. | | `npm run test:golden-ast` | **`npm run build`**, then **`node --test`** on `dist/test-infra/golden-ast-runner.js` — runs golden AST tests from `test-fixtures/golden-ast/`. Use `UPDATE_GOLDEN=1 npm run test:golden-ast` to regenerate goldens after intentional parser changes. | | `npm run test:acceptance:compiler` | **`npm run build`**, then **`node --test`** with only `*.acceptance.test.js` files under **`dist/src/`** — compiler acceptance tests without the full unit suite or E2E. | | `npm run test:acceptance:runtime` | **`bash ./e2e/test_all.sh`** only — same E2E driver as below **without** an implicit rebuild; ensure `dist/` is up to date before running. | | `npm run test:acceptance` | **`npm run test:acceptance:compiler`** then **`npm run test:acceptance:runtime`**. | -| `npm run test:e2e` | **`npm run build`**, then **`bash ./e2e/test_all.sh`**. Prefer this when you want a fresh `dist/` before E2E. **`e2e::prepare_shared_context`** in `e2e/lib/common.sh` exports **`JAIPH_DOCKER_ENABLED=false`** after clearing most **`JAIPH_*`** variables, so typical tests run on the **host**; Docker coverage lives in scripts that set **`JAIPH_DOCKER_ENABLED=true`** — see [E2E testing](#e2e-testing) and **`resolveDockerConfig`** in `src/runtime/docker.ts` / [Architecture — Core components](architecture.md#core-components). | +| `npm run test:e2e` | **`npm run build`**, then **`bash ./e2e/test_all.sh`**. Prefer this when you want a fresh `dist/` before E2E. **`e2e::prepare_shared_context`** in `e2e/lib/common.sh` clears most inherited **`JAIPH_*`** variables before each test. Host-only. See [E2E testing](#e2e-testing). | | `npm run test:samples` | **`npx playwright test`** — Playwright suite for the docs landing page (`e2e/playwright/`). Uses `http://127.0.0.1:4000` (see `playwright.config.ts`); starts Jekyll via `webServer` or reuses one already on that port. Requires Playwright (`npx playwright install chromium` once). | | `npm run test:ci` | `npm test` followed by `npm run test:e2e` — useful before pushing when you want the full local picture. | @@ -157,7 +155,7 @@ find src -type f \( -name '*.test.ts' -o -name '*.acceptance.test.ts' \) | sort | CLI and terminal UX | `src/cli/**/*.test.ts` | Commands, `jaiph run` lifecycle, progress, hooks, `resolve-env` | | Transpiler and validation | `src/transpile/*.test.ts` + `*.acceptance.test.ts` | `validateModule`, `emit`, golden compiler (`compiler-golden.test.ts`), cross-module edge cases (`compiler-edge.acceptance.test.ts`) | | Formatter | `src/format/*.test.ts` | `jaiph format` | -| Runtime and Docker | `src/runtime/kernel/*.test.ts`, `src/runtime/docker.test.ts` | Graph, emit, prompts, test runner, workflow launch, `docker` helper | +| Runtime | `src/runtime/kernel/*.test.ts` | Graph, emit, prompts, test runner, workflow launch | | Standalone root tests | e.g. `src/inline-script-name.test.ts` | Small colocated cases that are not under a feature subtree | When adding a new source module or extending an existing one, create or extend the corresponding `*.test.ts` in the same directory. For kernel internals, the compile path, and artifact contracts, see [Architecture](architecture.md). @@ -169,14 +167,14 @@ Tests that span multiple modules, require subprocess/PTY harnesses, exercise pro | Test file | Kind | What it covers | |-----------|------|----------------| | `integration/docs-structure.test.ts` | Integration | Diátaxis docs lint — valid `diataxis:` front matter, nav ↔ page bijection, internal link / permalink / `redirect_from` resolution | -| `integration/docs-explanation-task3.test.ts` | Integration | Four greenfield explanation pages (`why-jaiph`, `inbox`, `spec-async-handles`, `sandboxing`) — permalinks, nav placement; **`sandboxing.md`** shape (threat model present; no how-to procedure headings or config-key tables) | +| `integration/docs-explanation-task3.test.ts` | Integration | Three explanation pages (`why-jaiph`, `inbox`, `spec-async-handles`) — permalinks, nav placement | | `integration/docs-how-to-task4.test.ts` | Integration | How-to quadrant — permalinks, retired-path redirects, recipe shape, `agent-auth` credential / pre-flight error pinning | | `integration/docs-reference-task5.test.ts` | Integration | Reference quadrant — permalinks, nav placement, `env-vars.md` source parity against `src/`, anti-tutorial shape guards | | `integration/docs-tutorials-task6.test.ts` | Integration | Tutorial quadrant — permalinks, `/getting-started` redirect absorption, runnable `first-workflow` snippet with documented output | | `integration/docs-nav-structure-task7.test.ts` | Integration | Nav spine — five Diátaxis section headings in documented order; every published page under its quadrant exactly once | | `integration/release-workflow.test.ts` | Integration | Release matrix / asset-naming contract — five-binary matrix (no windows-arm64), `SHA256SUMS` + upload lists include `jaiph-windows-x64.exe`, shared version-gate script, naming contract ↔ matrix ↔ installer parity | | `integration/installer-powershell.test.ts` | Integration | Windows PowerShell installer (`docs/install.ps1`) contract — download/verify/install steps, bash↔PowerShell lockstep release ref, and `docs/setup.md` / main-page one-liner parity | -| `integration/windows-native-smoke.test.ts` | Integration | Host-portable guards for the `windows-native-smoke` CI job and its `e2e/tests/windows_native_smoke.ps1` harness — job shape (windows-latest, `bun --compile` build, gate membership alongside `test`/`e2e`/`e2e-wsl`), stdout/exit-code assertions, cancellation orphan check, `prompt` pre-flight error, and no-WSL enforcement | +| `integration/windows-native-smoke.test.ts` | Integration | Host-portable guards for the `windows-native-smoke` CI job and its `e2e/tests/windows_native_smoke.ps1` harness — job exists on windows-latest, `bun --compile` build, harness run, stdout/exit-code assertions, cancellation orphan check, `prompt` pre-flight error, and no-WSL enforcement | | `integration/sample-build/build.test.ts` | Integration | Build/transpile behavior — `buildScripts`, script extraction | | `integration/sample-build/cli-tree.test.ts` | Integration | CLI tree output rendering for sample workflows | | `integration/sample-build/run-core.test.ts` | Integration | Core runtime execution — workflow runs, step sequencing, artifacts | @@ -193,25 +191,23 @@ Tests that span multiple modules, require subprocess/PTY harnesses, exercise pro | `integration/serve-restart.test.ts` | Integration | `jaiph serve` run recovery and idempotency across a real process restart | | `integration/otlp-export.test.ts` | Integration | OTLP trace export — a run with OTLP env sends exactly one well-formed POST to `/v1/traces`, and delivery is detached so a hanging collector does not block the terminal result | | `integration/sentry-export.test.ts` | Integration | Sentry error reporting — a failed run delivers exactly one envelope carrying the failing step and an excerpt across `jaiph run`, `jaiph run --raw`, and `jaiph serve`; a succeeding run delivers nothing | -| `integration/exec-policy.test.ts` | Integration | One execution-policy contract across `jaiph run`, `jaiph serve`, and `jaiph mcp` — the same sandbox / env cases produce the same effective child env, the same filesystem outcome, and the same fail-before-spawn behavior for a conflicting posture | +| `integration/exec-policy.test.ts` | Integration | One execution-policy contract across `jaiph run`, `jaiph serve`, and `jaiph mcp` — `--env` / `trusted_envs` produce the same effective child env | | `integration/tty-running-timer.test.ts` | Acceptance | In a TTY, verifies the “RUNNING workflow” line updates over time (requires Python 3 PTY harness) | The `integration/sample-build/` directory also has a shared `helpers.ts` module used by the sample-build tests. Shared test fixtures (`.jh` source files and expected output) live in `test-fixtures/sample-build/`. ## CI pipeline -The project uses GitHub Actions (`.github/workflows/ci.yml`). The workflow defines **eight** jobs. On a typical feature-branch push, **seven** of them run. The remaining job, **Publish Docker runtime image**, runs only on pushes to **`nightly`** and on **`v*`** version tags, after the test, E2E, docs, WSL, PowerShell-installer, and native-Windows-smoke jobs succeed, and it builds and pushes `ghcr.io/jaiphlang/jaiph-runtime` (the default `runtime.docker_image` / `JAIPH_DOCKER_IMAGE` when Docker sandboxing is on; see **Docker runtime helper** in [Architecture](architecture.md#core-components)). The **Validate Kubernetes deploy manifest** job is not one of those gates, so it does not block the image publish. +The project uses GitHub Actions (`.github/workflows/ci.yml`). The workflow defines **six** jobs. All six run on a typical feature-branch push. | Job | Runner | Purpose | |-----|--------|---------| | **Compiler and unit tests** | `ubuntu-latest` | `npm run arch:check` (the dependency-cruiser import-graph gate for no cycles and the layer DAG) then `npm run lint` (the ESLint fan-out and file-size caps) then `npm test` (TypeScript unit + acceptance + golden tests), plus a `curl` check that the public install URL responds and a git-tag verification on `main`. | -| **Validate Kubernetes deploy manifest** | `ubuntu-latest` | Provisions a throwaway `kind` cluster, dry-run applies `docs/deploy/k8s.yaml` as a schema gate, builds the local `jaiph-e2e-runtime:local` image, then runs `e2e/tests/150_k8s_deploy.sh` to deploy and exercise the manifest on the cluster: the external `jaiph-credentials` Secret gate, pod hardening (non-root, no privilege escalation, dropped capabilities, read-only root filesystem, no service-account token), an authenticated HTTP run, and its journal on the writable runs volume. | -| **E2E** | Matrix: **`ubuntu-latest` twice** + **`macos-latest`** | Job id `e2e`; in the Actions UI each leg appears as **`E2E (,