Phase 4 minimal server

[streaky]

This pull request introduces Phase 4 ("minimal server") support to the project, adding a new HTTP inference server with OpenAI-compatible endpoints, authentication, and OpenAPI documentation. It updates the documentation to reflect the new server, adds a new cusco-server crate, and integrates the server into the CLI and development workflows. Several dependencies are added for HTTP and async support, and the executor is extended with a token-to-piece conversion.

Major new features and updates:

Server implementation and integration:

• Added the new cusco-server crate implementing a minimal HTTP inference server, including OpenAI-compatible endpoints, authentication, durable state, scheduling, and OpenAPI documentation.
• Integrated the server with the CLI via a new serve subcommand, wiring up authentication, command-line options, and the inference engine. [1] [2] [3]
• Added a LlamaEngine inference engine implementation that uses the existing executor for prompt generation.

Documentation and workflow updates:

• Updated README.md and AGENTS.md to document the new Phase 4 server, its features, endpoints, and usage instructions. [1] [2] [3]
• Updated compose.yaml to add a server-report service for integration testing and reporting.

Executor enhancements:

• Added a token_to_piece method to the executor and underlying FFI, enabling token-to-string conversion for streaming and OpenAI compatibility. [1] [2] [3] [4]

Dependency and workspace changes:

• Added required HTTP, async, and utility dependencies (axum, tokio, tower, uuid, etc.) to the workspace for server support. [1] [2] [3]

These changes collectively establish a minimal but functional HTTP inference server, laying the groundwork for further production hardening and mapped execution in future phases.

[Copilot]

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated no new comments.

Suppressed comments (2)

crates/server/src/lib.rs:321

• persist() holds self.inner's mutex across filesystem operations (create_dir_all, write, rename). This can block unrelated requests (including cancellation/admission bookkeeping) on slow disks and makes it easier to accidentally introduce deadlocks if future code calls persist() while holding the lock.

    fn persist(&self) -> Result<(), Error> {
        let guard = self.inner.lock();
        let bytes = serde_json::to_vec_pretty(&guard.durable).map_err(state_err)?;
        if let Some(parent) = self.state_path.parent() {

crates/server/src/lib.rs:735

• The SSE streaming path uses unwrap() when serializing stream events. If serialization ever fails (e.g., a future event variant adds a non-serializable field), the handler will panic and crash the request/task. It’s safer to avoid panicking in request handlers and turn serialization failures into stream errors or explicit error events.

        let rows = events
            .into_iter()
            .map(|event| Ok::<_, Infallible>(Event::default().json_data(event).unwrap()));
        Ok(Sse::new(stream::iter(rows)).into_response())