Skip to content

feat: add Databricks sandbox backend for the native executor - #3

Draft
HsingTang wants to merge 1 commit into
feat/codex-harnessfrom
hsingtang/databricks-codex
Draft

HsingTang wants to merge 1 commit into
feat/codex-harnessfrom
hsingtang/databricks-codex

Conversation

@HsingTang

Copy link
Copy Markdown
Owner

Summary

Draft integration that backs the native Environment (v1.0 / feat/codex-harness)
with a Databricks sandbox, so an agent driven by this SDK runs exec_command /
write_stdin / apply_patch inside the sandbox.

This PR is opened inside a personal fork (base feat/codex-harness, head
hsingtang/databricks-codex) for Databricks and OpenAI folks to review before
anything is proposed upstream. It is not a contribution request to
openai/openai-agents-python yet.

Approach

The native Environment dials an authenticated wss:// executor and relays the
exec-server protocol over it; a Databricks sandbox runs that exec-server in
stdio mode, reached through the databricks-sdk streaming command API
(WorkspaceClient.sandbox.execute_command). Nothing in the SDK speaks stdio and
nothing in the sandbox speaks WebSocket, so a small localhost gateway bridges
them - it terminates the WebSocket, authenticates the bearer token at the HTTP
handshake, opens the exec-server's stdio, and relays bytes both ways. Because
execute_command reaches the sandbox outbound through the Databricks control
plane, the gateway runs on loopback next to the application and the sandbox
needs no inbound ingress. The agent loop runs in-process via NativeProvider,
which is what makes the Environment approval / transform / NetworkPolicy
hooks available.

Contents

  • src/agents/extensions/sandbox/databricks/ - SandboxExecServerGateway
    (relay + self-signed loopback TLS helper) and databricks_sandbox_environment
    (async context manager that provisions the gateway and yields a configured
    Environment).
  • pyproject.toml - a databricks optional-dependency extra
    (databricks-sdk[sandbox], websockets, cryptography, certifi) and a
    mypy override. This reintroduces [project.optional-dependencies], which the
    v1.0 tree had dropped.
  • Tests for the relay, bearer auth, teardown paths, the context manager, and the
    SSL_CERT_FILE trust-bundle helper.
  • examples/databricks/runner.py and a package README.md.

Open items (please review)

These assumptions are documented in the README and need confirming against a
live sandbox:

  1. The sandbox image's codex binary serves exec-server in stdio mode (the
    default command is ("codex", "exec-server", "stdio")). The SDK's own
    offline_docker.py proves exec-server stdio is valid; the Databricks
    cookbook exercised only the outbound --remote mode.
  2. The streaming execute_command (CommandProcess) is a suitable bidirectional
    binary stdio stream for the exec-server protocol under load.
  3. The native WebSocket client treats the connection as an opaque byte stream
    (the relay does not preserve per-message frame boundaries).

Validation

ruff check and ruff format --check pass on all new files. pytest and mypy
were not run here: the v1.0 SDK is a GitHub prerelease wheel (off PyPI) with a
compiled native runtime, which is not installable through the available package
proxy. The tests are written to run where that preview wheel is installed.

This pull request and its description were written by Isaac.

Provide a native Environment backed by a Databricks sandbox for the
v1.0 (codex-harness) SDK. A localhost WSS gateway relays the SDK
executor connection to the sandbox exec-server over the databricks-sdk
streaming command API; databricks_sandbox_environment provisions the
gateway and yields a configured Environment.

Add a databricks optional-dependency extra, unit tests for the relay
and bearer auth, a runnable example, and a package README. The README
flags two prerequisites to confirm against a live sandbox: exec-server
stdio mode in the image, and bidirectional streaming via
execute_command.

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant