Feature/remote pipe func bundle transport

[thomashebrard]

Summary by cubic

Adds method-bundle transport to /execute and /start so callers can send a full method via bundle_b64 (zip) or files (map). The bundle’s .mthds now feeds mthds_contents so main_pipe resolves, while only .py/deps are materialized to a temp library_dirs; custom .py is allowed only on sandbox-hosted deployments.

• New Features

  • Accept bundle_b64 (base64 zip) or files ({relpath: text}); exactly one allowed; mutually exclusive with mthds_contents; requires at least one .mthds.
  • Ingest guards: traversal/backslash/drive forms blocked; invalid base64/zip rejected; hard limits MAX_BUNDLE_FILES and MAX_BUNDLE_TOTAL_KIB; bounded decompression with a pre-decode size check for bundle_b64; all guards run before any disk write.
  • Split handling: bundle .mthds -> mthds_contents; other files -> temp library_dirs for ApiRunner (auto-cleaned; start only needs it during setup). You can still set pipe_code to pick a pipe in the bundle.
  • Sandbox gate: bundles with .py require sandbox-hosted deployments; otherwise 403 CustomCodeRequiresSandbox.
  • Dependencies/docs/tests: pin pipelex via uv sources to the 0.38 seam (bumped SHA); docs and unit tests added.

• Bug Fixes

  • Fixed a 500 when running a bundle by keeping the proven path: .mthds populates mthds_contents so main_pipe resolves; only non-.mthds files are written to disk.

^{Written for commit 87636c1. Summary will update on new commits.}

[greptile-apps]

Greptile Summary

This PR adds request-carried method bundles for pipeline runs. The main changes are:

• New bundle_b64 and files transports for method bundles.
• Temporary bundle materialization into library_dirs for /execute and /start.
• Sandbox-hosted gating for bundles that include custom Python sources.
• Bundle ingest guards for path safety, file count, total size, zip bombs, invalid base64, and corrupt zips.
• Route and unit tests plus documentation for the new transport.

Confidence Score: 4/5

One packaging issue needs to be fixed before merging for non-uv install paths.

The core bundle parsing, route integration, sandbox gate, and tests look sound. The score reflects a contained blocking issue in dependency metadata that can make the changed imports fail outside uv installs.

pyproject.toml

T-Rex Logs

What T-Rex did

• Reproduced the non-uv packaging harness and confirmed that pip resolves the pipelex dependency to a PyPI wheel (pipelex 0.38.0) rather than the git source during installation in a fresh virtual environment.
• Validated wheel metadata and provenance show the installed pipelex-api wheel declares no git-based requires and contains no direct_url.json provenance, confirming git source is not preserved in the distribution.
• Executed the bundle tests with uv run pytest and observed EXIT_CODE: 0 with no changed-behavior failures or blockers.

_{Ran code and verified through T-Rex}

Important Files Changed

Filename	Overview
api/bundle.py	Adds bundle parsing and materialization with path, count, size, zip, base64, and cleanup guards.
api/routes/pipelex/pipeline.py	Threads request-carried bundles into ApiRunner library dirs with sandbox gating for Python sources.
api/schemas/models.py	Extends run and start request models for bundle transports and rejects bundle plus inline MTHDS.
pyproject.toml	Moves pipelex to a uv-only git source while leaving the standard dependency unpinned.
tests/unit/test_bundle_materialize.py	Adds unit coverage for bundle materialization, cleanup, and ingest guard failures.
tests/unit/test_pipeline_bundle.py	Adds route-level coverage for bundle materialization, sandbox gating, and invalid transport combinations.
uv.lock	Updates locked mthds and pipelex resolution to support the new bundle transport.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant Client
participant API as /execute or /start
participant Bundle as api.bundle
participant Gate as Sandbox gate
participant Runner as ApiRunner
participant Orchestrator

Client->>API: Run request with bundle_b64 or files
API->>Bundle: parse_bundle(...)
Bundle-->>API: ParsedBundle(entries, has_python_sources)
API->>Gate: is_pipe_func_sandbox_hosted() when .py present
alt .py present and not sandbox-hosted
    Gate-->>API: false
    API-->>Client: 403 CustomCodeRequiresSandbox
else accepted bundle
    API->>Bundle: materialize_parsed(parsed)
    Bundle-->>API: temp library_dirs
    API->>Runner: ApiRunner(..., library_dirs)
    Runner->>Orchestrator: execute/start pipe job
    Orchestrator-->>Runner: run result or dispatch ack
    Runner-->>API: response model
    API->>Bundle: context exit cleanup
    API-->>Client: 200 execute or 202 start
end

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant Client
participant API as /execute or /start
participant Bundle as api.bundle
participant Gate as Sandbox gate
participant Runner as ApiRunner
participant Orchestrator

Client->>API: Run request with bundle_b64 or files
API->>Bundle: parse_bundle(...)
Bundle-->>API: ParsedBundle(entries, has_python_sources)
API->>Gate: is_pipe_func_sandbox_hosted() when .py present
alt .py present and not sandbox-hosted
    Gate-->>API: false
    API-->>Client: 403 CustomCodeRequiresSandbox
else accepted bundle
    API->>Bundle: materialize_parsed(parsed)
    Bundle-->>API: temp library_dirs
    API->>Runner: ApiRunner(..., library_dirs)
    Runner->>Orchestrator: execute/start pipe job
    Orchestrator-->>Runner: run result or dispatch ack
    Runner-->>API: response model
    API->>Bundle: context exit cleanup
    API-->>Client: 200 execute or 202 start
end

Loading

Prompt To Fix All With AI

Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
pyproject.toml:18-20
**Preserve git dependency source**
`pipelex` now imports branch-only symbols (`is_pipe_func_sandbox_hosted` and the bundle seam), but the standard dependency published at lines 18-20 no longer pins that git revision. `tool.uv.sources` is only honored by `uv`; installing or building this project with other PEP 621 consumers will resolve `pipelex` from PyPI and miss the branch code, causing import or runtime failures on the changed path. Encode the git URL in the project dependency itself or keep a released version pin that contains these APIs.

_{Reviews (1): Last reviewed commit: "Pin pipelex to 0.38 SHA (fixes hosted bo..." | Re-trigger Greptile}

[greptile-apps]

Preserve git dependency source
pipelex now imports branch-only symbols (is_pipe_func_sandbox_hosted and the bundle seam), but the standard dependency published at lines 18-20 no longer pins that git revision. tool.uv.sources is only honored by uv; installing or building this project with other PEP 621 consumers will resolve pipelex from PyPI and miss the branch code, causing import or runtime failures on the changed path. Encode the git URL in the project dependency itself or keep a released version pin that contains these APIs.

Artifacts

Repro: non-uv packaging and import harness

• Contains supporting evidence from the run (text/x-python; charset=utf-8).

Repro: pip resolver output and failing import traceback

• Keeps the command output available without making the summary code-heavy.

_{Ran code and verified through T-Rex}

Prompt To Fix With AI

This is a comment left during a code review.
Path: pyproject.toml
Line: 18-20

Comment:
**Preserve git dependency source**
`pipelex` now imports branch-only symbols (`is_pipe_func_sandbox_hosted` and the bundle seam), but the standard dependency published at lines 18-20 no longer pins that git revision. `tool.uv.sources` is only honored by `uv`; installing or building this project with other PEP 621 consumers will resolve `pipelex` from PyPI and miss the branch code, causing import or runtime failures on the changed path. Encode the git URL in the project dependency itself or keep a released version pin that contains these APIs.

How can I resolve this? If you propose a fix, please make it concise.

[cubic-dev-ai]

All reported issues were addressed across 13 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>