diff --git a/.facts b/.facts index 255c378d..1db3f7d3 100644 --- a/.facts +++ b/.facts @@ -1554,23 +1554,23 @@ - the Host Daemon durably persists each newly observed upstream Codex thread or Turn identifier before the Codex adapter continues waiting for later upstream work @mvp @implemented - persisting an upstream Codex identifier does not change Session or Turn lifecycle state, state revisions, safe logs, idempotency outcomes, or Control Lease ownership @mvp @implemented - recording the same upstream Codex identifier again is idempotent, while recording a different identifier for the same Session or Turn fails closed without overwriting the original @mvp @implemented -- satelle session records include enough host, readiness, provider, goal, and lifecycle metadata to survive transport changes without changing the public session identifier @spec @mvp +- satelle session records include enough host, readiness, provider, goal, and lifecycle metadata to survive transport changes without changing the public session identifier @spec @mvp @implemented - satelle Host Daemon is the canonical store for session metadata because it owns the Codex thread, app-server lifecycle, Computer Use readiness, logs, and recovery state @mvp @implemented -- satelle CLI may keep read-only convenience cache for recently used sessions but must refresh from the Host Daemon before reporting authoritative status @spec @mvp +- satelle CLI may keep read-only convenience cache for recently used sessions but must refresh from the Host Daemon before reporting authoritative status @spec @mvp @implemented - satelle status reconnects to the session's remote host automatically when the local CLI has no active connection @spec @mvp @implemented - satelle status and logs are the MVP surfaces for later inspection after a user disconnects from a live event stream @implemented @mvp - satelle clients that reconnect after missing live Satelle Events read current session state and normalized logs instead of replaying missed events @spec @mvp @implemented - satelle status for SSH bootstrap hosts may restart the on-demand Host Daemon before reading stored remote session metadata @spec @mvp @implemented - satelle status reports a typed session-not-found error when the selected host cannot find the requested session metadata @implemented @mvp - a Session operation verifies that the requested Session belongs to the connected Host Identity @mvp @implemented -- the local CLI uses its non-authoritative Session-to-Host cache only to select a candidate Host @spec @mvp -- satelle requires --host when a Session identifier cannot be mapped to exactly one configured Host without probing unrelated Hosts @spec @mvp +- the local CLI uses its non-authoritative Session-to-Host cache only to select a candidate Host @spec @mvp @implemented +- satelle requires --host when a Session identifier cannot be mapped to exactly one configured Host without probing unrelated Hosts @spec @mvp @implemented - satelle remote host retains minimal session metadata for 7 days by default @implemented @mvp -- satelle session metadata retention can be configured per host or profile @spec @mvp +- satelle session metadata retention can be configured per host or profile @spec @mvp @implemented - session retention cleanup never deletes a nonterminal Session or Turn @implemented @mvp - session retention age begins when the Session's most recent Turn becomes terminal @implemented @mvp -- expiration of Satelle metadata does not claim to delete Codex, provider, or operating-system records outside Satelle ownership @spec @mvp -- satelle can export a redacted task artifact set containing plan.md, worklog.md, and goal.md for a selected session so another agent or human can audit what happened without replaying raw desktop content @spec @mvp +- expiration of Satelle metadata does not claim to delete Codex, provider, or operating-system records outside Satelle ownership @spec @mvp @implemented +- satelle can export a redacted task artifact set containing plan.md, worklog.md, and goal.md for a selected session so another agent or human can audit what happened without replaying raw desktop content @spec @mvp @implemented - label: satelle records local redacted command history for setup, repair, run, steer, status, stop, logs, doctor, host, config, and MCP installer commands when local state storage is available command: cargo test --locked -p satelle-cli --features test-support --test command-history records_redacted_command_metadata_and_typed_outcomes -- --exact tags: [spec, mvp, implemented, ci] @@ -1584,7 +1584,7 @@ - label: satelle local aggregate stats summarize command counts, success and failure counts, last-used hosts, last-used profiles, and common typed error codes from redacted command history command: cargo test --locked -p satelle-cli --features test-support --test command-history aggregate_views_summarize_outcomes_targets_profiles_and_errors -- --exact tags: [spec, mvp, implemented, ci] -- satelle local aggregate stats are diagnostic convenience data and are never treated as authoritative remote host state @spec @mvp +- satelle local aggregate stats are diagnostic convenience data and are never treated as authoritative remote host state @spec @mvp @implemented - satelle exposes a top-level logs command in MVP @implemented @mvp - satelle logs is a read-only diagnostic command that never starts prompt execution, setup, repair, host update, service mutation, or configuration mutation @implemented @mvp - satelle logs reads authoritative Satelle Log Entries from the remote Host Daemon instead of relying on a local CLI cache @spec @mvp @implemented @@ -1689,31 +1689,31 @@ - Host Daemon state directories are accessible only to the daemon OS user and required operating-system service principals @spec @mvp @implemented - Satelle creates sensitive POSIX directories with mode 0700 and sensitive POSIX files with mode 0600 @spec @mvp @implemented - Satelle applies equivalent current-user restricted ACLs to sensitive Windows state @spec @mvp @implemented -- SQLite database, WAL, SHM, token verifier, attachment staging, and sensitive export staging files inherit the same restricted access boundary @spec @mvp -- Satelle Operator Log Files, local CLI state and cache, migration backups, setup recovery metadata, and recording artifacts inherit the same OS-user-private directory and file boundary @spec @mvp +- SQLite database, WAL, SHM, token verifier, attachment staging, and sensitive export staging files inherit the same restricted access boundary @spec @mvp @implemented +- Satelle Operator Log Files, local CLI state and cache, migration backups, setup recovery metadata, and recording artifacts inherit the same OS-user-private directory and file boundary @spec @mvp @implemented - Satelle refuses to use a sensitive state path that resolves through an unsafe writable directory boundary @spec @mvp @implemented -- retention deletion does not claim cryptographic erasure from SQLite pages, filesystem snapshots, backups, or upstream services @spec @mvp -- Satelle documents that Codex and model providers may retain data independently from Satelle-owned retention @spec @mvp +- retention deletion does not claim cryptographic erasure from SQLite pages, filesystem snapshots, backups, or upstream services @spec @mvp @implemented +- Satelle documents that Codex and model providers may retain data independently from Satelle-owned retention @spec @mvp @implemented - satelle local CLI cache stores only convenience data such as redacted command history, aggregate stats, AI client installer state, downloaded release metadata, and non-authoritative recent host summaries @spec @mvp @implemented - satelle local CLI cache root is configurable through SATELLE_CACHE_DIR @spec @mvp @implemented -- satelle local CLI cache entries have explicit time-to-live or retention policies instead of persisting indefinitely by default @spec @mvp -- satelle local CLI cache never stores provider credentials, raw prompts, screenshots, desktop recordings, full transcripts, raw Codex protocol payloads, or raw provider request and response bodies by default @spec @mvp +- satelle local CLI cache entries have explicit time-to-live or retention policies instead of persisting indefinitely by default @spec @mvp @implemented +- satelle local CLI cache never stores provider credentials, raw prompts, screenshots, desktop recordings, full transcripts, raw Codex protocol payloads, or raw provider request and response bodies by default @spec @mvp @implemented - satelle Host Daemon stores its SQLite database under the resolved Satelle local mutable state root @spec @mvp @implemented - satelle Host Daemon stores recording artifacts under a recordings directory inside the resolved Satelle local mutable state root by default @spec @mvp @implemented -- satelle Host Daemon writes Satelle Operator Log Files by default when the resolved OS-native Satelle log directory is writable @spec @mvp +- satelle Host Daemon writes Satelle Operator Log Files by default when the resolved OS-native Satelle log directory is writable @spec @mvp @implemented - satelle Satelle Operator Log Files are a best-effort local inspection mirror and are never authoritative for satelle logs, satelle status, recovery, retention, or support bundle collection @spec @mvp @implemented - satelle logs command reads Satelle Log Entries from SQLite through the Host Daemon API instead of parsing Satelle Operator Log Files @spec @mvp @implemented - satelle Satelle Operator Log Files contain human-readable redacted summaries derived from Satelle Log Entries instead of raw protocol payloads or raw subprocess streams @spec @mvp @implemented - satelle Satelle Operator Log Files do not include provider request bodies, provider response bodies, full prompts, screenshots, desktop recordings, full transcripts, raw setup stdout, raw setup stderr, raw repair stdout, or raw repair stderr by default @spec @mvp @implemented - satelle Satelle Operator Log Files rotate at 10 MiB and retain at most 5 files total per Host Daemon instance by default @spec @mvp @implemented -- satelle Satelle Operator Log File retention can be configured per host or profile without changing SQLite log retention @spec @mvp +- satelle Satelle Operator Log File retention can be configured per host or profile without changing SQLite log retention @spec @mvp @implemented - satelle Satelle Operator Log Files are stored under an OS-native Satelle log directory on the remote host instead of project directories @spec @mvp @implemented - satelle default Linux Satelle Operator Log File root is ${XDG_STATE_HOME:-$HOME/.local/state}/satelle/logs @spec @mvp @implemented - satelle default Windows Satelle Operator Log File root is the Local AppData Known Folder joined with Microck\Satelle\data\state\logs @spec @mvp @implemented - satelle default macOS Satelle Operator Log File root is $HOME/Library/Logs/dev.Microck.Satelle @spec @mvp @implemented -- satelle Host Daemon continues running when Satelle Operator Log File writes fail while SQLite remains writable, and records the degraded log-file sink as a typed diagnostic finding @spec @mvp -- satelle Host Daemon writes startup, shutdown, and fatal-error notices to stdout or stderr so service managers and containers can capture minimal process diagnostics @spec @mvp -- satelle MVP does not require direct journald, Windows Event Log, or macOS unified logging API integration as a separate log sink @spec @mvp +- satelle Host Daemon continues running when Satelle Operator Log File writes fail while SQLite remains writable, and records the degraded log-file sink as a typed diagnostic finding @spec @mvp @implemented +- satelle Host Daemon writes startup, shutdown, and fatal-error notices to stdout or stderr so service managers and containers can capture minimal process diagnostics @spec @mvp @implemented +- satelle MVP does not require direct journald, Windows Event Log, or macOS unified logging API integration as a separate log sink @spec @mvp @implemented - satelle later adds Platform-Native Log Sinks only as optional mirrors of redacted Satelle Log Entries @spec @later - satelle Satelle Platform-Native Log Sinks never replace the Host Daemon SQLite store, Satelle Operator Log Files, or the Host Daemon API as the authoritative source for satelle logs, status, recovery, retention, or support bundle collection @spec @later - satelle Satelle Platform-Native Log Sinks emit the same redacted summaries and default exclusions as Satelle Operator Log Files instead of raw protocol payloads, provider bodies, prompts, screenshots, transcripts, recordings, or raw setup and repair subprocess streams @spec @later @@ -1837,8 +1837,8 @@ - satelle host storage backup cleanup --host is the explicit command for deleting older eligible migration backups after a dry-run plan and mutation consent @spec @mvp @implemented - satelle destructive store reset is available only through an explicit host maintenance command @spec @mvp @implemented - satelle host store reset deletes metadata only by default and does not delete recordings unless the user explicitly requests recording deletion @spec @mvp @implemented -- satelle does not use ad hoc JSON files as the canonical store for session metadata, readiness cache, provider smoke results, or log summaries @spec @mvp -- satelle storage remains local to the remote host unless the user explicitly exports diagnostics or recordings @spec @mvp +- satelle does not use ad hoc JSON files as the canonical store for session metadata, readiness cache, provider smoke results, or log summaries @spec @mvp @implemented +- satelle storage remains local to the remote host unless the user explicitly exports diagnostics or recordings @spec @mvp @implemented # bridge diff --git a/.spec-gaps.md b/.spec-gaps.md index 2c6a3551..b6086957 100644 --- a/.spec-gaps.md +++ b/.spec-gaps.md @@ -82,6 +82,51 @@ Blocking: yes - an abstract semantic catalog, repository-wide mutual-exclusion p Resolution: User configuration and named profiles use `log_verbosity = "off" | "info" | "debug" | "trace"`. The exact precedence is `--log-verbosity`, then `SATELLE_LOG`, then the selected profile over base user configuration, then the command default. Machine JSON output suppresses diagnostic logging. Command-specific `--quiet` and `--verbose` continue to control presentation and event detail rather than diagnostic tracing. A named profile refers to durable mutation consent only through an explicit non-empty `trusted_profile = "name"` field. The reference must name an existing user-level Trusted Profile, and it activates consent only when the profile is selected by a user default or explicit `--profile`; `SATELLE_PROFILE` and project selection cannot activate it. Same-name profiles have no implicit relationship. +### GAP-043: Session metadata retention policy + +- Status: resolved +- Packet/facts: source packet 22, `c58` / L1530 +- Decision: Add `session_metadata_retention = "7d"` to Operator-owned Host and Profile configuration. Use a new `RetentionDuration` grammar with positive `h` and `d` units, a 7-day minimum, a 365-day maximum, and no zero or disable value. The minimum matches packet 22's fixed normalized SQLite log retention, because retained logs keep their Session scope until packet 23 owns configurable SQLite log retention. The default is 7 days. A selected user-owned Profile overrides the user Host value through the existing profile overlay rules. Project config and project profiles cannot set destructive retention. Resolve one effective policy and inject it into Storage pruning. Do not broaden `ExplicitDuration`, whose ms/s/m grammar remains unchanged. + +### GAP-044: Operator Log File retention shape + +- Status: resolved +- Packet/facts: source packet 22, `jgo` / L1670 +- Decision: Add Operator-owned `operator_log_retained_files` to Host and Profile configuration with range 1 through 100 and default 5. A selected user-owned Profile overrides the user Host value. Project config cannot set it. Reuse `OperatorLogPolicy.retained_files`, keep the fixed 10 MiB rotation threshold, and keep SQLite log retention independent. Do not add age retention or a compound policy. + +### GAP-045: Local CLI cache retention + +- Status: resolved +- Packet/facts: source packet 22, `e70` / L1660 +- Decision: Treat owner-only `command-history.sqlite3` as the sole MVP local convenience cache for Session-to-Host candidates. Use a fixed `COMMAND_HISTORY_RETENTION` of 7 days, delete older rows in the same writer transaction before insert, and consider only rows newer than the same cutoff during candidate lookup. Add no config key and no second Session cache. Keep the existing closed redacted row schema. Managed SSH and release artifact caches retain their existing explicit cleanup policies. + +### GAP-046: Explicit redacted task artifact export + +- Status: resolved +- Packet/facts: source packet 22, `q69` / L1534 +- Decision: Add only `satelle session export --host --output `. Both options are required. Do not add implicit Host probing, stdout archives, overwrite, or format flags. Add an authenticated, identity-pinned, read-scoped `GET /v1/sessions/{id}/task-artifacts` route and a monotonic protocol bump after C1. The `satelle.task_artifacts.v1` response contains Host Identity, Session ID, and exactly three UTF-8 bodies: `plan.md`, `worklog.md`, and `goal.md`. +- Source and privacy: Generate the three bodies only from Host SQLite. Build `goal.md` from the persisted upstream goal reference and safe Session metadata, `plan.md` from ordered Turn model/provider/execution-policy/readiness references, and `worklog.md` from normalized redacted Session logs and terminal safe summaries. Never read arbitrary workspace files or include raw prompts, desktop content, screenshots, transcripts, credentials, Codex protocol payloads, or provider request/response bodies. Missing optional metadata renders a deterministic `not recorded` section. A missing or mismatched Session fails before content is returned. +- Destination and failure: The destination is the operator-selected Controller directory. Create a random sibling owner-only staging directory, write exactly three owner-only files, flush and sync them, then atomically rename the directory. Refuse an existing destination. Any failure removes staging and leaves the destination absent. This is the only task-artifact egress path in this packet. + +### GAP-047: Durable Session admission metadata + +- Status: resolved +- Packet/facts: source packet 22, `fd2` / L1518 +- Decision: Add a private immutable per-Turn `AdmissionReadinessRef` containing native result ID, native observed time, native source, and optional provider result ID, observed time, and source. Persist it with the Turn in SQLite. Existing Session Host/Desktop refs, Turn provider/model/execution policy, upstream goal ref, timestamps, and lifecycle state complete the required metadata. Do not persist readiness fingerprints again. Do not change `PublicSession`, the public Session ID, or the public schema. Historical references remain after reusable evidence expires. + +### GAP-048: Operator Log sink degradation ownership + +- Status: resolved +- Packet/facts: source packet 22, `wkh` / L1675; C3 Doctor integration consumes the result +- Decision: Packet 22 Storage owns `OperatorLogSinkHealth::{Healthy, Degraded(kind)}` using the existing failure-kind and write-outcome vocabulary. The first mirror failure marks runtime health degraded, coalesced failures do not duplicate it, and a later successful write clears it. Canonical SQLite commit results never depend on the mirror. Do not persist the mirror failure as a normalized SQLite log because that can recurse through the failing sink. +- C3 boundary: Doctor aggregation consumes the neutral Host/runtime health and emits finding ID `host.operator_log_file_degraded`, severity `warning`, fixability `manual_action_required`, readiness impact `degraded`, and one stable failure-kind token. Packet 22 owns the sink and health state; C3 owns Doctor presentation only. + +### GAP-049: Host-local storage and explicit egress + +- Status: resolved +- Packet/facts: source packet 22, `ju3` / L1798 +- Decision: Canonical SQLite, sidecars, attachments, recovery data, Operator Logs, and recordings remain on the Host. Ordinary run, status, log, and setup operations never copy Host storage to a Controller or service. Only an explicit Operator export command may copy a redacted artifact, diagnostic, or recording to its operator-selected destination. Document this boundary and prove ordinary read paths have no export or write side effect. The task-artifact exception is exactly GAP-046. + ### GAP-021 - Installer-managed standalone Codex contract (RESOLVED) Resolution: PR 06 owns fail-closed runtime admission of an existing managed Codex installation; the later setup-install and Codex-update packets own acquisition, update, and receipt creation. The acquisition contract uses the official OpenAI standalone Codex `0.144.0` full package from release tag `rust-v0.144.0`, mapping native Host targets to `aarch64-apple-darwin`, `x86_64-apple-darwin`, `aarch64-pc-windows-msvc`, or `x86_64-pc-windows-msvc`. The selected `codex-package-.tar.gz` must match `codex-package_SHA256SUMS` before the official versioned installer runs. The accepted package root is an immutable installer-owned directory under `/packages/standalone/releases`; mutable `current` links or junctions and visible-bin shims are never execution identities. Satelle atomically writes its owner-only `codex-install-receipt.json` under the resolved Satelle state root only after post-install verification. The receipt records schema version, manager, Codex version, target, release tag, exact artifact URL and SHA-256, Codex home, immutable package root, immutable binary path and SHA-256, and installation time. Installation and receipt creation are separate ordered ledger actions, not one atomic transaction. If receipt creation or post-verification fails, the installed package remains untrusted; setup or repair may adopt it only after repeating complete package, metadata, target, binary-digest, and version verification. Readiness fails closed when the receipt is absent or drifts from package metadata, binary bytes, `codex --version`, or the app-server capability handshake. The Host launches only the immutable receipt-recorded binary and sets the receipt-recorded `CODEX_HOME` only on Codex child processes; it ignores npm packages, mutable aliases, shims, and `PATH`. diff --git a/crates/satelle-cli/src/command-history.rs b/crates/satelle-cli/src/command-history.rs index 411e0689..8ea34d70 100644 --- a/crates/satelle-cli/src/command-history.rs +++ b/crates/satelle-cli/src/command-history.rs @@ -1,7 +1,7 @@ use rusqlite::{Connection, OpenFlags, TransactionBehavior, params}; use satelle_core::{ ErrorCode, SecureFileError, SessionId, open_or_create_owner_only_directory, - open_or_create_owner_only_file, + open_or_create_owner_only_file, open_owner_only_directory, }; #[cfg(not(unix))] use std::fs; @@ -12,6 +12,7 @@ use time::OffsetDateTime; const DATABASE_FILE_NAME: &str = "command-history.sqlite3"; const DATABASE_DIRECTORY_NAME: &str = "command-history"; const DATABASE_BUSY_TIMEOUT: Duration = Duration::from_secs(2); +const COMMAND_HISTORY_RETENTION: time::Duration = time::Duration::days(7); const SCHEMA: &str = r#" CREATE TABLE IF NOT EXISTS command_history ( @@ -122,6 +123,70 @@ fn format_started_at(timestamp: OffsetDateTime) -> String { ) } +/// Returns recent successful Host candidates without creating or updating the +/// optional cache. Any missing, unsafe, corrupt, or stale cache is equivalent +/// to no candidate, so the caller can require an explicit `--host`. +pub(super) fn session_host_candidates( + cache_root: &Path, + session_id: &SessionId, + observed_at: OffsetDateTime, +) -> Vec { + read_session_host_candidates(cache_root, session_id, observed_at).unwrap_or_default() +} + +fn read_session_host_candidates( + cache_root: &Path, + session_id: &SessionId, + observed_at: OffsetDateTime, +) -> Option> { + #[cfg(target_os = "macos")] + let resolved_cache_root = resolve_trusted_macos_aliases(cache_root).ok()?; + #[cfg(target_os = "macos")] + let cache_root = resolved_cache_root.as_path(); + let database_directory = cache_root.join(DATABASE_DIRECTORY_NAME); + if !database_directory.try_exists().ok()? { + return Some(Vec::new()); + } + let _database_directory_guard = open_owner_only_directory(&database_directory).ok()?; + let database_path = database_directory.join(DATABASE_FILE_NAME); + if !database_path.try_exists().ok()? { + return Some(Vec::new()); + } + let connection = Connection::open_with_flags( + database_path, + OpenFlags::SQLITE_OPEN_READ_ONLY + | OpenFlags::SQLITE_OPEN_NO_MUTEX + | OpenFlags::SQLITE_OPEN_NOFOLLOW, + ) + .ok()?; + connection.busy_timeout(DATABASE_BUSY_TIMEOUT).ok()?; + let cutoff = observed_at.checked_sub(COMMAND_HISTORY_RETENTION)?; + let mut statement = connection + .prepare( + "SELECT DISTINCT selected_host \ + FROM command_history \ + WHERE session_id = ?1 \ + AND selected_host IS NOT NULL \ + AND outcome_status = 'success' \ + AND started_at >= ?2 \ + AND started_at <= ?3 \ + ORDER BY selected_host", + ) + .ok()?; + statement + .query_map( + params![ + session_id.to_string(), + format_started_at(cutoff), + format_started_at(observed_at) + ], + |row| row.get(0), + ) + .ok()? + .collect::, _>>() + .ok() +} + /// Captures only redacted command metadata. The raw argument vector is never /// retained, so prompts, provider values, file contents, and secret sources /// cannot accidentally cross this persistence boundary. @@ -195,7 +260,6 @@ impl Recorder { // Windows scheduler contention. let transaction = connection.transaction_with_behavior(TransactionBehavior::Immediate)?; transaction.execute_batch(SCHEMA)?; - let session_id = final_session_id .map(ToString::to_string) .or(self.invocation.session_id); @@ -221,6 +285,16 @@ impl Recorder { env!("CARGO_PKG_VERSION"), ], )?; + // Prune from the writer transaction's wall-clock time, not the + // invocation start. Insert first so a command that ran past the + // retention window cannot leave its already-expired row behind. + let retention_cutoff = OffsetDateTime::now_utc() + .checked_sub(COMMAND_HISTORY_RETENTION) + .ok_or(rusqlite::Error::InvalidQuery)?; + transaction.execute( + "DELETE FROM command_history WHERE started_at < ?1", + [format_started_at(retention_cutoff)], + )?; transaction.commit()?; Ok(()) } @@ -245,7 +319,7 @@ fn prepare_cache_root(path: &Path) -> Result } #[cfg(target_os = "macos")] - let resolved_path = resolve_trusted_macos_aliases_for_creation(path)?; + let resolved_path = resolve_trusted_macos_aliases(path)?; #[cfg(target_os = "macos")] let path = resolved_path.as_path(); #[cfg(not(target_os = "macos"))] @@ -311,7 +385,7 @@ fn prepare_cache_root(path: &Path) -> Result } #[cfg(target_os = "macos")] -fn resolve_trusted_macos_aliases_for_creation(path: &Path) -> Result { +fn resolve_trusted_macos_aliases(path: &Path) -> Result { use std::os::unix::fs::MetadataExt; use std::path::Component; @@ -408,9 +482,13 @@ pub(super) enum HistoryWriteError { #[cfg(test)] mod tests { - use super::{Invocation, InvocationStart, Recorder, format_started_at}; + use super::{ + Invocation, InvocationStart, Recorder, format_started_at, session_host_candidates, + }; + use rusqlite::Connection; + use satelle_core::{ErrorCode, SessionId}; use std::path::PathBuf; - use std::time::Duration; + use std::time::{Duration, Instant}; use time::OffsetDateTime; #[test] @@ -437,4 +515,82 @@ mod tests { assert!(recorder.started.elapsed() >= Duration::from_millis(20)); } + + #[test] + fn persistence_drops_an_invocation_that_expired_while_running() { + let root = tempfile::tempdir().expect("create command history parent"); + let cache_root = root.path().join("cache"); + let started_at_time = OffsetDateTime::now_utc() - time::Duration::days(8); + let start = InvocationStart { + started_at: format_started_at(started_at_time), + started: Instant::now(), + }; + + Recorder::start( + cache_root.clone(), + Invocation::new("host-start", None, None, None), + start, + ) + .finish(None, None) + .expect("persist long-running invocation"); + + let retained_count: i64 = + Connection::open(cache_root.join("command-history/command-history.sqlite3")) + .expect("open command history") + .query_row("SELECT COUNT(*) FROM command_history", [], |row| row.get(0)) + .expect("count retained command history"); + assert_eq!(retained_count, 0); + } + + #[test] + fn candidate_lookup_is_read_only_recent_success_only_and_host_distinct() { + let root = tempfile::tempdir().expect("create candidate cache parent"); + let cache_root = root.path().join("cache"); + let session_id = SessionId::new(); + let now = OffsetDateTime::now_utc(); + assert!(session_host_candidates(&cache_root, &session_id, now).is_empty()); + assert!(!cache_root.exists()); + + for host in ["alpha", "alpha", "beta"] { + Recorder::start( + cache_root.clone(), + Invocation::new( + "status", + Some(host.to_string()), + None, + Some(session_id.to_string()), + ), + InvocationStart::capture(), + ) + .finish(None, None) + .expect("persist a successful candidate row"); + } + Recorder::start( + cache_root.clone(), + Invocation::new( + "status", + Some("gamma".to_string()), + None, + Some(session_id.to_string()), + ), + InvocationStart::capture(), + ) + .finish(None, Some(ErrorCode::InvalidUsage)) + .expect("persist a failed non-candidate row"); + assert_eq!( + session_host_candidates(&cache_root, &session_id, OffsetDateTime::now_utc()), + ["alpha".to_string(), "beta".to_string()] + ); + + Connection::open(cache_root.join("command-history/command-history.sqlite3")) + .expect("open candidate cache") + .execute( + "UPDATE command_history SET started_at = '2000-01-01T00:00:00.000000000Z'", + [], + ) + .expect("age candidate rows"); + assert!( + session_host_candidates(&cache_root, &session_id, OffsetDateTime::now_utc()).is_empty() + ); + } } diff --git a/crates/satelle-cli/src/logs.rs b/crates/satelle-cli/src/logs.rs index 954b061d..9d6e356c 100644 --- a/crates/satelle-cli/src/logs.rs +++ b/crates/satelle-cli/src/logs.rs @@ -182,6 +182,10 @@ impl LogReadPlan { }) } + const fn session_id(&self) -> Option<&SessionId> { + self.session_id.as_ref() + } + fn query(&self, query: LogPageQuery) -> LogPageQuery { let mut query = query.with_minimum_severity(self.minimum_severity); if let Some(session_id) = &self.session_id { @@ -309,7 +313,10 @@ pub(crate) fn show_logs( ) -> Result<(), CliFailure> { let request = LogReadRequest::from(command); let plan = LogReadPlan::resolve(&request)?; - let host = config.resolve_host(request.host.as_deref())?; + let host = match plan.session_id() { + Some(session_id) => config.resolve_session_host(request.host.as_deref(), session_id)?, + None => config.resolve_host(request.host.as_deref())?, + }; let transport = transport_for(&host)?; plan.emit(transport.as_ref(), format).map_err(failure) } diff --git a/crates/satelle-cli/src/main.rs b/crates/satelle-cli/src/main.rs index f075c8c8..0ba0263c 100644 --- a/crates/satelle-cli/src/main.rs +++ b/crates/satelle-cli/src/main.rs @@ -31,9 +31,10 @@ use logs::{LogsCommand, show_logs}; use notify::{Config as NotifyConfig, Event, RecommendedWatcher, RecursiveMode, Watcher}; use output::{EventOutput, OutputArgs, OutputFormat, SessionResultSchemaVersion, StatusReport}; #[cfg(any(windows, test))] -use satelle_core::daemon_service::WindowsServiceConfigV2; +use satelle_core::daemon_service::WindowsServiceConfigV3; use satelle_core::daemon_service::{ - DaemonServicePlatform, PersistentServiceDecision, SetupModeSelection, SetupModeSource, + DaemonServicePlatform, PersistentHostStoragePolicy, PersistentServiceDecision, + SetupModeSelection, SetupModeSource, }; use satelle_core::doctor::{DoctorScopeSelection, DoctorScopeSelectionError}; use satelle_core::session::{ @@ -49,9 +50,10 @@ use satelle_core::{ SatelleEvent, SatelleEventBody, SecureFileError, SessionId, SetupMode, SetupReadinessSummary, SetupReport, SetupRequiredInput, SetupSchemaVersion, SetupVerification, TransportKind, load_config, load_config_for_profile, load_config_without_profile, load_user_api_rate_limits, - open_or_create_owner_only_directory, open_or_create_owner_only_file, open_owner_only_directory, - read_owner_controlled_config_file, read_owner_only_secret_config_file, resolve_desktop_session, - resolve_path_set, utc_now, + open_new_owner_only_file, open_or_create_owner_only_directory, open_or_create_owner_only_file, + open_owner_only_directory, publish_new_owner_only_directory, read_owner_controlled_config_file, + read_owner_only_secret_config_file, resolve_desktop_session, resolve_path_set, + sync_owner_only_directory, utc_now, }; use satelle_host::{ ApiBearerToken, DoctorExecutionFailure, DoctorExecutionResult, HostService, @@ -262,6 +264,37 @@ impl<'a> ConfigContext<'a> { .map(SelectedHost::from) .map_err(failure) } + + fn resolve_session_host( + &self, + flag_host: Option<&str>, + session_id: &SessionId, + ) -> Result { + if flag_host.is_some() { + return self.resolve_host(flag_host); + } + let cwd = std::env::current_dir().unwrap_or_else(|_| PathBuf::from(".")); + let cache_root = resolve_path_set(&cwd).map_err(failure)?.cache_root; + let resolved = self.load()?; + let candidates = command_history::session_host_candidates( + &cache_root, + session_id, + OffsetDateTime::now_utc(), + ) + .into_iter() + .filter(|alias| resolved.config.hosts.contains_key(alias)) + .collect::>(); + let [candidate] = candidates.as_slice() else { + let mut error = SatelleError::invalid_usage(format!( + "Session {session_id} cannot be mapped to exactly one configured Host; pass --host " + )); + error + .details + .insert("candidate_count".to_string(), json!(candidates.len())); + return Err(failure(error)); + }; + self.resolve_host(Some(candidate)) + } } #[derive(Subcommand, Debug)] @@ -288,6 +321,10 @@ enum Command { Steer(SteerCommand), Status(StatusCommand), Stop(StopCommand), + Session { + #[command(subcommand)] + command: SessionCommand, + }, Logs(LogsCommand), Mcp { #[command(subcommand)] @@ -588,6 +625,12 @@ struct HostStartCommand { requires = "launchd_service" )] setup_ledger_retention_ms: Option, + /// Internal resolved Session metadata retention for a persistent launchd service. + #[arg(long, hide = true, value_name = "HOURS", requires = "launchd_service")] + session_metadata_retention_hours: Option, + /// Internal resolved Operator Log File generation count for launchd. + #[arg(long, hide = true, value_name = "COUNT", requires = "launchd_service")] + operator_log_retained_files: Option, /// Internal owner-only configuration used by the per-user Windows task. #[arg( long, @@ -1033,6 +1076,20 @@ struct StopCommand { output_args: OutputArgs, } +#[derive(Subcommand, Debug)] +enum SessionCommand { + Export(SessionExportCommand), +} + +#[derive(Args, Debug)] +struct SessionExportCommand { + session_id: String, + #[arg(long)] + host: String, + #[arg(long)] + output: PathBuf, +} + #[derive(Subcommand, Debug)] enum SupportCommand { Bundle(SupportBundleCommand), @@ -1403,6 +1460,9 @@ fn execute_command( Command::Steer(command) => steer_prompt(command, config, output).map(Some), Command::Status(command) => show_status(command, config, output).map(|_| None), Command::Stop(command) => stop_session(command, config, output).map(|_| None), + Command::Session { + command: SessionCommand::Export(command), + } => export_task_artifacts(command, config).map(|_| None), Command::Logs(command) => show_logs(command, config, output).map(|_| None), Command::Mcp { command: McpCommand::Serve, @@ -1525,6 +1585,13 @@ fn start_command_history( } if command.all ); let target = history_target(command)?; + if target.session_id.is_some() && target.explicit_host.is_none() { + // An implicit Session operation may consult command history only to + // select one candidate Host. Recording that same operation would turn + // the read-only routing cache path into a write and could reinforce a + // stale mapping before Host-authoritative status is known. + return None; + } let environment_preference = command_history_environment_preference(); if environment_preference == Some(false) { return None; @@ -1727,6 +1794,14 @@ fn history_target(command: &Command) -> Option> { explicit_host: command.host.as_deref(), session_id: canonical_history_session_id(&command.session_id), }, + Command::Session { + command: SessionCommand::Export(command), + } => HistoryTarget { + family: "session-export", + selects_host: true, + explicit_host: Some(&command.host), + session_id: canonical_history_session_id(&command.session_id), + }, Command::Logs(command) => HistoryTarget { family: "logs", selects_host: true, @@ -2065,6 +2140,8 @@ mod history_target_tests { bootstrap_provider_smoke_timeout_ms: None, on_demand_idle_timeout_ms: None, setup_ledger_retention_ms: None, + session_metadata_retention_hours: None, + operator_log_retained_files: None, service_config: None, output_args: OutputArgs::default(), }), @@ -2196,6 +2273,10 @@ mod history_target_tests { "--launchd-service", "--setup-ledger-retention-ms", "3600000", + "--session-metadata-retention-hours", + "720", + "--operator-log-retained-files", + "12", ]) .expect("parse internal launchd service start command"); assert!( @@ -2211,6 +2292,8 @@ mod history_target_tests { }; assert!(command.launchd_service); assert_eq!(command.setup_ledger_retention_ms, Some(3_600_000)); + assert_eq!(command.session_metadata_retention_hours, Some(720)); + assert_eq!(command.operator_log_retained_files, Some(12)); validate_host_start_mode(&command).expect("launchd service is a valid closed start mode"); } @@ -6662,7 +6745,9 @@ fn validate_host_start_mode(command: &HostStartCommand) -> Result<(), SatelleErr || command.bootstrap_native_readiness_timeout_ms.is_some() || command.bootstrap_provider_smoke_timeout_ms.is_some() || command.on_demand_idle_timeout_ms.is_some() - || command.setup_ledger_retention_ms.is_none()) + || command.setup_ledger_retention_ms.is_none() + || command.session_metadata_retention_hours.is_none() + || command.operator_log_retained_files.is_none()) { return Err(SatelleError::invalid_usage( "--launchd-service is an internal launchd input and cannot be combined with other internal or TLS Host start options", @@ -6678,7 +6763,9 @@ fn validate_host_start_mode(command: &HostStartCommand) -> Result<(), SatelleErr || command.bootstrap_native_readiness_timeout_ms.is_some() || command.bootstrap_provider_smoke_timeout_ms.is_some() || command.on_demand_idle_timeout_ms.is_some() - || command.setup_ledger_retention_ms.is_some()) + || command.setup_ledger_retention_ms.is_some() + || command.session_metadata_retention_hours.is_some() + || command.operator_log_retained_files.is_some()) { return Err(SatelleError::invalid_usage( "--service-config is an internal Windows service input and cannot be combined with ordinary Host start options", @@ -6737,6 +6824,40 @@ async fn bind_host_daemon( } } +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +enum DaemonProcessNotice { + Startup(SocketAddr), + Shutdown, + Fatal, +} + +fn write_daemon_process_notice( + writer: &mut impl Write, + notice: DaemonProcessNotice, +) -> io::Result<()> { + match notice { + DaemonProcessNotice::Startup(address) => { + writeln!(writer, "satelle-host: startup: listening on {address}") + } + DaemonProcessNotice::Shutdown => { + writeln!(writer, "satelle-host: shutdown: Host Daemon stopped") + } + DaemonProcessNotice::Fatal => { + writeln!( + writer, + "satelle-host: fatal: Host Daemon stopped with an error" + ) + } + } +} + +fn emit_daemon_process_notice(notice: DaemonProcessNotice) { + // Service managers and containers capture stderr independently from the + // bootstrap JSON protocol on stdout. A closed diagnostic sink must never + // change daemon lifecycle behavior. + let _ = write_daemon_process_notice(&mut io::stderr().lock(), notice); +} + fn start_host_daemon( command: HostStartCommand, config: ConfigContext<'_>, @@ -6764,7 +6885,7 @@ fn start_host_daemon_with( ) -> HostService, ) -> Result<(), CliFailure> { validate_host_start_mode(&command).map_err(failure)?; - let (service_path_overrides, service_setup_ledger_retention_ms) = + let (service_path_overrides, service_storage_policy) = if let Some(_service_config_path) = command.service_config.as_deref() { #[cfg(not(windows))] return Err(failure(SatelleError::invalid_usage( @@ -6778,15 +6899,25 @@ fn start_host_daemon_with( apply_windows_service_environment(&service_config); command.bind = service_config.bind().to_string(); command.foreground = true; - ( - Some(path_overrides), - Some(service_config.setup_ledger_retention_ms()), - ) + (Some(path_overrides), Some(service_config.storage_policy())) } } else if command.launchd_service { ( Some(daemon_path_overrides_from_process_environment()), - command.setup_ledger_retention_ms, + Some( + PersistentHostStoragePolicy::new( + command + .setup_ledger_retention_ms + .expect("launchd setup-ledger retention was validated"), + command + .session_metadata_retention_hours + .expect("launchd Session retention was validated"), + command + .operator_log_retained_files + .expect("launchd Operator Log retention was validated"), + ) + .map_err(|error| failure(SatelleError::invalid_usage(error.to_string())))?, + ), ) } else { (None, None) @@ -6909,7 +7040,7 @@ fn start_host_daemon_with( service_path_overrides .as_ref() .expect("persistent service path overrides were checked"), - service_setup_ledger_retention_ms.expect("persistent service retention was checked"), + service_storage_policy.expect("persistent service storage policy was checked"), ) .map_err(failure)?, (_, Some(token)) => { @@ -7005,6 +7136,7 @@ fn start_host_daemon_with( } else { println!("Host Daemon listening on {}", server.local_addr()); } + emit_daemon_process_notice(DaemonProcessNotice::Startup(server.local_addr())); let mut server_wait = Box::pin(server.wait()); let result = if command.foreground { @@ -7022,12 +7154,43 @@ fn start_host_daemon_with( server_wait.await.map_err(daemon_server_failure) }; state_release_task.abort(); + emit_daemon_process_notice(if result.is_ok() { + DaemonProcessNotice::Shutdown + } else { + DaemonProcessNotice::Fatal + }); result }) } +#[cfg(test)] +mod daemon_process_notice_tests { + use super::*; + + #[test] + fn process_notices_are_stable_and_do_not_use_the_bootstrap_protocol() { + let mut notices = Vec::new(); + for notice in [ + DaemonProcessNotice::Startup("127.0.0.1:3001".parse().unwrap()), + DaemonProcessNotice::Shutdown, + DaemonProcessNotice::Fatal, + ] { + write_daemon_process_notice(&mut notices, notice).expect("write process notice"); + } + + assert_eq!( + String::from_utf8(notices).unwrap(), + concat!( + "satelle-host: startup: listening on 127.0.0.1:3001\n", + "satelle-host: shutdown: Host Daemon stopped\n", + "satelle-host: fatal: Host Daemon stopped with an error\n", + ) + ); + } +} + #[cfg(any(windows, test))] -fn read_windows_service_config(path: &Path) -> Result { +fn read_windows_service_config(path: &Path) -> Result { if !path.is_absolute() { return Err(failure(SatelleError::invalid_usage( "Windows Host service config path must be absolute", @@ -7079,7 +7242,9 @@ mod windows_service_config_tests { log_dir: Some(PathBuf::from(r"C:\Users\owner\AppData\Local\Satelle\logs")), sources: BTreeMap::new(), }; - let expected = WindowsServiceConfigV2::new("127.0.0.1:3001", &overrides, 3_600_000) + let storage_policy = + PersistentHostStoragePolicy::new(3_600_000, 30 * 24, 12).expect("build storage policy"); + let expected = WindowsServiceConfigV3::new("127.0.0.1:3001", &overrides, storage_policy) .expect("build service config"); write_owner_only_config( &path, @@ -7097,7 +7262,7 @@ mod windows_service_config_tests { ["host", "start", "--foreground", "--bind", "127.0.0.1:3001"] ); assert_eq!(observed.environment().len(), 5); - assert_eq!(observed.setup_ledger_retention_ms(), 3_600_000); + assert_eq!(observed.storage_policy(), storage_policy); } #[test] @@ -7131,7 +7296,7 @@ mod windows_service_config_tests { } #[cfg(windows)] -fn apply_windows_service_environment(config: &WindowsServiceConfigV2) { +fn apply_windows_service_environment(config: &WindowsServiceConfigV3) { const PATH_OVERRIDES: [&str; 5] = [ "SATELLE_HOME", "SATELLE_CONFIG_FILE", @@ -7681,6 +7846,8 @@ mod daemon_tls_watcher_tests { bootstrap_provider_smoke_timeout_ms: None, on_demand_idle_timeout_ms: None, setup_ledger_retention_ms: None, + session_metadata_retention_hours: None, + operator_log_retained_files: None, service_config: None, output_args: OutputArgs::default(), } @@ -8205,6 +8372,8 @@ mod bootstrap_startup_tests { bootstrap_provider_smoke_timeout_ms: None, on_demand_idle_timeout_ms: Some(75_000), setup_ledger_retention_ms: None, + session_metadata_retention_hours: None, + operator_log_retained_files: None, service_config: None, output_args: OutputArgs::default(), }; @@ -10543,19 +10712,12 @@ fn steer_prompt( explicit_host_alias, SessionId::from_str(&command.session_id).map_err(|error| failure(error.into())), )?; - let config = report_not_admitted( - &mut event_output, - explicit_host_alias, - config_context.load(), - )?; let host = report_not_admitted( &mut event_output, explicit_host_alias, - config - .resolve_host_with_project_source(explicit_host_alias) - .map(SelectedHost::from) - .map_err(failure), + config_context.resolve_session_host(explicit_host_alias, &session_id), )?; + let config = report_not_admitted(&mut event_output, Some(&host.alias), config_context.load())?; let yolo_policy = resolve_yolo_policy( config, &host.alias, @@ -10815,7 +10977,7 @@ fn stop_session( let json = format.is_json(); let session_id = SessionId::from_str(&command.session_id).map_err(|error| failure(error.into()))?; - let host = config.resolve_host(command.host.as_deref())?; + let host = config.resolve_session_host(command.host.as_deref(), &session_id)?; let transport = transport_for(&host)?; let result = transport.stop(&session_id).map_err(failure)?; @@ -10836,6 +10998,170 @@ fn stop_session( } } +fn export_task_artifacts( + command: SessionExportCommand, + config: ConfigContext<'_>, +) -> Result<(), CliFailure> { + let session_id = + SessionId::from_str(&command.session_id).map_err(|error| failure(error.into()))?; + let host = config.resolve_host(Some(&command.host))?; + let artifacts = transport_for(&host)? + .task_artifacts(&session_id) + .map_err(failure)?; + let output = persist_task_artifacts(&command.output, &artifacts)?; + println!("Exported task artifacts: {}", output.display()); + Ok(()) +} + +fn persist_task_artifacts( + requested_output: &Path, + artifacts: &transport::TaskArtifacts, +) -> Result { + let file_name = requested_output.file_name().ok_or_else(|| { + failure(SatelleError::invalid_usage( + "--output must name a new destination directory", + )) + })?; + let requested_parent = task_artifact_output_parent(requested_output); + let parent = requested_parent.canonicalize().map_err(|error| { + failure(SatelleError::config_error( + format!( + "task artifact output parent {} is unavailable", + requested_parent.display() + ), + Some(error.to_string()), + )) + })?; + let output = parent.join(file_name); + if output.try_exists().map_err(|error| { + failure(SatelleError::config_error( + format!( + "could not inspect task artifact destination {}", + output.display() + ), + Some(error.to_string()), + )) + })? { + return Err(failure(SatelleError::invalid_usage(format!( + "task artifact destination {} already exists", + output.display() + )))); + } + + let _parent_guard = open_owner_only_directory(&parent).map_err(|error| { + failure(SatelleError::config_error( + format!( + "task artifact output parent {} is not owner-only", + parent.display() + ), + Some(error.to_string()), + )) + })?; + let staging_path = parent.join(format!( + ".satelle-task-artifacts-{}", + Uuid::now_v7().simple() + )); + let staging_guard = open_or_create_owner_only_directory(&staging_path).map_err(|error| { + failure(SatelleError::config_error( + "could not create owner-only task artifact staging directory", + Some(error.to_string()), + )) + })?; + let mut staging_cleanup = TaskArtifactStagingCleanup::new(staging_path.clone()); + for (name, body) in [ + ("plan.md", artifacts.plan.as_bytes()), + ("worklog.md", artifacts.worklog.as_bytes()), + ("goal.md", artifacts.goal.as_bytes()), + ] { + let path = staging_path.join(name); + let mut file = open_new_owner_only_file(&path).map_err(|error| { + failure(SatelleError::config_error( + format!("could not create owner-only task artifact {name}"), + Some(error.to_string()), + )) + })?; + file.write_all(body).map_err(|error| { + failure(SatelleError::config_error( + format!("could not write task artifact {name}"), + Some(error.to_string()), + )) + })?; + file.flush().map_err(|error| { + failure(SatelleError::config_error( + format!("could not flush task artifact {name}"), + Some(error.to_string()), + )) + })?; + file.sync_all().map_err(|error| { + failure(SatelleError::config_error( + format!("could not sync task artifact {name}"), + Some(error.to_string()), + )) + })?; + } + sync_owner_only_directory(&staging_path, &staging_guard).map_err(|error| { + failure(SatelleError::config_error( + "could not sync task artifact staging directory", + Some(error.to_string()), + )) + })?; + drop(staging_guard); + publish_new_owner_only_directory(&staging_path, &output).map_err(|error| { + failure(SatelleError::config_error( + format!( + "could not publish task artifact destination {}", + output.display() + ), + Some(error.to_string()), + )) + })?; + staging_cleanup.disarm(); + Ok(output) +} + +fn task_artifact_output_parent(requested_output: &Path) -> &Path { + requested_output + .parent() + .filter(|parent| !parent.as_os_str().is_empty()) + .unwrap_or_else(|| Path::new(".")) +} + +#[cfg(test)] +mod task_artifact_output_tests { + use super::*; + + #[test] + fn bare_relative_destination_uses_the_current_directory() { + assert_eq!( + task_artifact_output_parent(Path::new("task-artifacts")), + Path::new(".") + ); + } +} + +struct TaskArtifactStagingCleanup { + path: PathBuf, + armed: bool, +} + +impl TaskArtifactStagingCleanup { + fn new(path: PathBuf) -> Self { + Self { path, armed: true } + } + + fn disarm(&mut self) { + self.armed = false; + } +} + +impl Drop for TaskArtifactStagingCleanup { + fn drop(&mut self) { + if self.armed { + let _ = fs::remove_dir_all(&self.path); + } + } +} + fn effective_timeouts_json( host_config: &satelle_core::HostConfig, turn_execution_timeout_ms: u64, diff --git a/crates/satelle-cli/src/output.rs b/crates/satelle-cli/src/output.rs index 88eff572..cdc823a8 100644 --- a/crates/satelle-cli/src/output.rs +++ b/crates/satelle-cli/src/output.rs @@ -126,6 +126,7 @@ impl Command { ), Self::Status(command) => (command.output_args, EventOutput::None), Self::Stop(command) => (command.output_args, EventOutput::None), + Self::Session { .. } => (OutputArgs::default(), EventOutput::None), Self::Logs(command) => (command.output_args, EventOutput::None), Self::Mcp { command: super::McpCommand::Install(command), diff --git a/crates/satelle-cli/src/read.rs b/crates/satelle-cli/src/read.rs index ce83d13a..7e88bfca 100644 --- a/crates/satelle-cli/src/read.rs +++ b/crates/satelle-cli/src/read.rs @@ -330,7 +330,7 @@ pub(super) fn status( config: ConfigContext<'_>, ) -> Result<(PublicSession, String), CliFailure> { let session_id = SessionId::from_str(session_id).map_err(|error| failure(error.into()))?; - let host = config.resolve_host(host)?; + let host = config.resolve_session_host(host, &session_id)?; status_for_host(&session_id, &host).map(|session| (session, host.alias)) } diff --git a/crates/satelle-cli/src/ssh-bootstrap.rs b/crates/satelle-cli/src/ssh-bootstrap.rs index 7a139dbb..4edbd292 100644 --- a/crates/satelle-cli/src/ssh-bootstrap.rs +++ b/crates/satelle-cli/src/ssh-bootstrap.rs @@ -2553,7 +2553,7 @@ impl<'a> PersistentServiceRemote<'a> { pub(super) fn publish_windows_service_config( &mut self, task: &satelle_core::daemon_service::WindowsTaskDefinition, - config: &satelle_core::daemon_service::WindowsServiceConfigV2, + config: &satelle_core::daemon_service::WindowsServiceConfigV3, ) -> Result<(), SshBootstrapError> { self.require_platform(satelle_core::daemon_service::DaemonServicePlatform::Windows)?; let contents = serde_json::to_vec_pretty(config) @@ -2625,7 +2625,7 @@ impl<'a> PersistentServiceRemote<'a> { &self, artifact: &UploadedHostArtifact, overrides: &DaemonPathOverrides, - setup_ledger_retention_ms: u64, + storage_policy: satelle_core::daemon_service::PersistentHostStoragePolicy, ) -> Result { self.require_platform(satelle_core::daemon_service::DaemonServicePlatform::Macos)?; let binary = self.absolute_artifact_path(artifact); @@ -2633,7 +2633,7 @@ impl<'a> PersistentServiceRemote<'a> { Path::new(&binary), "127.0.0.1:3001", overrides, - setup_ledger_retention_ms, + storage_policy, ) .map_err(|_| SshBootstrapError::InvalidPersistentServiceDefinition)?; Ok(LaunchdServiceDefinition { @@ -3629,7 +3629,7 @@ fn parse_service_path_overrides( output: &[u8], ) -> Result { if target.is_windows() { - let config: satelle_core::daemon_service::WindowsServiceConfigV2 = + let config: satelle_core::daemon_service::WindowsServiceConfigV3 = serde_json::from_slice(output) .map_err(|_| SshBootstrapError::InvalidServiceObservation)?; if config.bind() != "127.0.0.1:3001" { @@ -3669,7 +3669,7 @@ struct ObservedLaunchdServiceDefinition { executable: String, bind: SocketAddr, path_overrides: DaemonPathOverrides, - setup_ledger_retention_ms: u64, + storage_policy: satelle_core::daemon_service::PersistentHostStoragePolicy, } fn parse_launchd_service_definition( @@ -3689,6 +3689,10 @@ fn parse_launchd_service_definition( "--bind", ); const RETENTION_PREFIX: &str = "--setup-ledger-retention-ms"; + const SESSION_RETENTION_PREFIX: &str = + "--session-metadata-retention-hours"; + const OPERATOR_LOG_RETENTION_PREFIX: &str = + "--operator-log-retained-files"; const ENVIRONMENT_PREFIX: &str = concat!("", "EnvironmentVariables",); const SUFFIX: &str = concat!( @@ -3717,13 +3721,26 @@ fn parse_launchd_service_definition( return Err(SshBootstrapError::InvalidServiceObservation); } let (setup_ledger_retention_ms, body) = body - .split_once(ENVIRONMENT_PREFIX) + .split_once(SESSION_RETENTION_PREFIX) .ok_or(SshBootstrapError::InvalidServiceObservation)?; - let setup_ledger_retention_ms = setup_ledger_retention_ms - .parse::() - .ok() - .filter(|value| *value > 0 && *value <= satelle_core::MAX_SETUP_LEDGER_RETENTION_MS) + let (session_metadata_retention_hours, body) = + body.split_once(OPERATOR_LOG_RETENTION_PREFIX) + .ok_or(SshBootstrapError::InvalidServiceObservation)?; + let (operator_log_retained_files, body) = body + .split_once(ENVIRONMENT_PREFIX) .ok_or(SshBootstrapError::InvalidServiceObservation)?; + let storage_policy = satelle_core::daemon_service::PersistentHostStoragePolicy::new( + setup_ledger_retention_ms + .parse::() + .map_err(|_| SshBootstrapError::InvalidServiceObservation)?, + session_metadata_retention_hours + .parse::() + .map_err(|_| SshBootstrapError::InvalidServiceObservation)?, + operator_log_retained_files + .parse::() + .map_err(|_| SshBootstrapError::InvalidServiceObservation)?, + ) + .map_err(|_| SshBootstrapError::InvalidServiceObservation)?; let environment = body .strip_suffix(SUFFIX) .ok_or(SshBootstrapError::InvalidServiceObservation)?; @@ -3749,7 +3766,7 @@ fn parse_launchd_service_definition( Ok(ObservedLaunchdServiceDefinition { executable: binary, bind, - setup_ledger_retention_ms, + storage_policy, path_overrides: daemon_path_overrides_from_environment( RemoteTarget::DarwinArm64, &entries, @@ -4093,7 +4110,7 @@ impl RemoteUserDirectories { path: &str, host_id: &str, expected_path_overrides: &DaemonPathOverrides, - expected_setup_ledger_retention_ms: u64, + expected_storage_policy: satelle_core::daemon_service::PersistentHostStoragePolicy, ) -> Result, SshBootstrapError> { self.probe_managed_service_executable_with_program( OsStr::new("ssh"), @@ -4101,7 +4118,7 @@ impl RemoteUserDirectories { path, host_id, expected_path_overrides, - expected_setup_ledger_retention_ms, + expected_storage_policy, ) } @@ -4113,7 +4130,7 @@ impl RemoteUserDirectories { path: &str, host_id: &str, expected_path_overrides: &DaemonPathOverrides, - expected_setup_ledger_retention_ms: u64, + expected_storage_policy: satelle_core::daemon_service::PersistentHostStoragePolicy, ) -> Result, SshBootstrapError> { self.probe_managed_service_executable_with_program( ssh_program.as_os_str(), @@ -4121,7 +4138,7 @@ impl RemoteUserDirectories { path, host_id, expected_path_overrides, - expected_setup_ledger_retention_ms, + expected_storage_policy, ) } @@ -4132,7 +4149,7 @@ impl RemoteUserDirectories { path: &str, host_id: &str, expected_path_overrides: &DaemonPathOverrides, - expected_setup_ledger_retention_ms: u64, + expected_storage_policy: satelle_core::daemon_service::PersistentHostStoragePolicy, ) -> Result, SshBootstrapError> { if host_id.is_empty() || !host_id @@ -4167,7 +4184,7 @@ impl RemoteUserDirectories { "$config=Get-Content -LiteralPath $path -Raw | ConvertFrom-Json; ", "$task=Get-ScheduledTask -TaskPath '\\Satelle\\' -TaskName {task_name} ", "-ErrorAction SilentlyContinue; ", - "if ($config.schema -cne 'satelle.host-service.v2' -or $null -eq $task) {{ ", + "if ($config.schema -cne 'satelle.host-service.v3' -or $null -eq $task) {{ ", "[Console]::Out.Write('absent'); exit 0 }}; ", "[xml]$xml=Export-ScheduledTask -TaskPath '\\Satelle\\' -TaskName {task_name}; ", "$root=$xml.Task; ", @@ -4257,14 +4274,14 @@ impl RemoteUserDirectories { .ok_or(SshBootstrapError::InvalidServiceObservation)?; let config = config.strip_suffix('\r').unwrap_or(config); let Ok(config) = serde_json::from_str::< - satelle_core::daemon_service::WindowsServiceConfigV2, + satelle_core::daemon_service::WindowsServiceConfigV3, >(config) else { return Ok(None); }; - let expected = satelle_core::daemon_service::WindowsServiceConfigV2::new( + let expected = satelle_core::daemon_service::WindowsServiceConfigV3::new( "127.0.0.1:3001", expected_path_overrides, - expected_setup_ledger_retention_ms, + expected_storage_policy, ) .map_err(|_| SshBootstrapError::InvalidServiceObservation)?; if config != expected { @@ -4283,7 +4300,7 @@ impl RemoteUserDirectories { }; if definition.bind != "127.0.0.1:3001".parse().expect("static socket address") || definition.path_overrides != *expected_path_overrides - || definition.setup_ledger_retention_ms != expected_setup_ledger_retention_ms + || definition.storage_policy != expected_storage_policy { return Ok(None); } @@ -5636,6 +5653,16 @@ mod tests { #[cfg(unix)] use std::os::unix::fs::{PermissionsExt, symlink}; + #[cfg(unix)] + fn persistent_storage_policy() -> satelle_core::daemon_service::PersistentHostStoragePolicy { + satelle_core::daemon_service::PersistentHostStoragePolicy::new( + satelle_core::daemon_service::DEFAULT_SETUP_LEDGER_RETENTION_MS, + satelle_core::DEFAULT_SESSION_METADATA_RETENTION_HOURS, + satelle_core::DEFAULT_OPERATOR_LOG_RETAINED_FILES, + ) + .expect("valid default persistent storage policy") + } + #[test] fn offline_cleanup_commands_bind_authorization_and_the_exact_approved_set() { let identity = OfflineStorageMaintenanceIdentity { @@ -5851,7 +5878,7 @@ mod tests { Path::new("/Users/operator/Library/Caches/Satelle/host/v0.1.0/darwin-arm64/satelle"), "127.0.0.1:3001", &DaemonPathOverrides::default(), - satelle_core::daemon_service::DEFAULT_SETUP_LEDGER_RETENTION_MS, + persistent_storage_policy(), ) .expect("render canonical macOS service definition"); fs::write( @@ -5886,7 +5913,7 @@ mod tests { &service_path, "host-123", &DaemonPathOverrides::default(), - satelle_core::daemon_service::DEFAULT_SETUP_LEDGER_RETENTION_MS, + persistent_storage_policy(), ) .expect("run audited read-only service probe") .as_ref() @@ -5901,7 +5928,12 @@ mod tests { &service_path, "host-123", &DaemonPathOverrides::default(), - 3_600_000, + satelle_core::daemon_service::PersistentHostStoragePolicy::new( + 3_600_000, + satelle_core::DEFAULT_SESSION_METADATA_RETENTION_HOURS, + satelle_core::DEFAULT_OPERATOR_LOG_RETAINED_FILES, + ) + .unwrap(), ) .expect("a drifted macOS retention is observable") .is_none() @@ -5945,7 +5977,7 @@ mod tests { state_dir: Some(PathBuf::from("/Users/operator/drifted-state")), ..DaemonPathOverrides::default() }, - satelle_core::daemon_service::DEFAULT_SETUP_LEDGER_RETENTION_MS, + persistent_storage_policy(), ) .expect("render drifted macOS service definition"); fs::write( @@ -5965,7 +5997,7 @@ mod tests { &service_path, "host-123", &DaemonPathOverrides::default(), - satelle_core::daemon_service::DEFAULT_SETUP_LEDGER_RETENTION_MS, + persistent_storage_policy(), ) .expect("a well-formed drifted launchd environment is observable") .is_none() @@ -5978,10 +6010,12 @@ mod tests { "#!/bin/sh\nprintf '%s\\n' \"$@\" > {}\n", "printf 'managed\\r\\n", "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\\r\\n", - "{{\"schema\":\"satelle.host-service.v2\",", + "{{\"schema\":\"satelle.host-service.v3\",", "\"daemon_arguments\":[\"host\",\"start\",\"--foreground\",\"--bind\",", "\"127.0.0.1:3001\"],\"environment\":{{}},", - "\"setup_ledger_retention_ms\":2592000000}}\\r\\n", + "\"storage_policy\":{{\"setup_ledger_retention_ms\":2592000000,", + "\"session_metadata_retention_hours\":168,", + "\"operator_log_retained_files\":5}}}}\\r\\n", "C:\\\\Users\\\\operator\\\\AppData\\\\Local\\\\Satelle\\\\host\\\\v0.1.0\\\\", "win32-x64-msvc\\\\satelle-", "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.exe'\n" @@ -6002,7 +6036,7 @@ mod tests { &windows_service_path, "host-123", &DaemonPathOverrides::default(), - satelle_core::daemon_service::DEFAULT_SETUP_LEDGER_RETENTION_MS, + persistent_storage_policy(), ) .expect("run audited Windows service probe") .as_ref() @@ -6019,7 +6053,12 @@ mod tests { &windows_service_path, "host-123", &DaemonPathOverrides::default(), - 3_600_000, + satelle_core::daemon_service::PersistentHostStoragePolicy::new( + 3_600_000, + satelle_core::DEFAULT_SESSION_METADATA_RETENTION_HOURS, + satelle_core::DEFAULT_OPERATOR_LOG_RETAINED_FILES, + ) + .unwrap(), ) .expect("a drifted Windows retention is observable") .is_none() @@ -6078,10 +6117,12 @@ mod tests { concat!( "#!/bin/sh\nprintf 'managed\\r\\n", "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\\r\\n", - "{\"schema\":\"satelle.host-service.v2\",", + "{\"schema\":\"satelle.host-service.v3\",", "\"daemon_arguments\":[\"host\",\"start\",\"--foreground\",\"--bind\",", "\"127.0.0.1:3002\"],\"environment\":{},", - "\"setup_ledger_retention_ms\":2592000000}\\r\\n", + "\"storage_policy\":{\"setup_ledger_retention_ms\":2592000000,", + "\"session_metadata_retention_hours\":168,", + "\"operator_log_retained_files\":5}}\\r\\n", "C:\\\\Users\\\\operator\\\\AppData\\\\Local\\\\Satelle\\\\host\\\\v0.1.0\\\\", "win32-x64-msvc\\\\satelle-", "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.exe'\n", @@ -6096,7 +6137,7 @@ mod tests { &windows_service_path, "host-123", &DaemonPathOverrides::default(), - satelle_core::daemon_service::DEFAULT_SETUP_LEDGER_RETENTION_MS, + persistent_storage_policy(), ) .expect("a well-formed drifted Windows service config is observable") .is_none() @@ -6107,11 +6148,13 @@ mod tests { concat!( "#!/bin/sh\nprintf 'managed\\r\\n", "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\\r\\n", - "{\"schema\":\"satelle.host-service.v2\",", + "{\"schema\":\"satelle.host-service.v3\",", "\"daemon_arguments\":[\"host\",\"start\",\"--foreground\",\"--bind\",", "\"127.0.0.1:3001\"],", "\"environment\":{\"SATELLE_HOME\":\"C:\\\\\\\\Drifted\"},", - "\"setup_ledger_retention_ms\":2592000000}\\r\\n", + "\"storage_policy\":{\"setup_ledger_retention_ms\":2592000000,", + "\"session_metadata_retention_hours\":168,", + "\"operator_log_retained_files\":5}}\\r\\n", "C:\\\\Users\\\\operator\\\\AppData\\\\Local\\\\Satelle\\\\host\\\\v0.1.0\\\\", "win32-x64-msvc\\\\satelle-", "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.exe'\n", @@ -6126,7 +6169,7 @@ mod tests { &windows_service_path, "host-123", &DaemonPathOverrides::default(), - satelle_core::daemon_service::DEFAULT_SETUP_LEDGER_RETENTION_MS, + persistent_storage_policy(), ) .expect("a well-formed drifted Windows service environment is observable") .is_none() @@ -6550,10 +6593,14 @@ mod tests { #[test] fn persistent_service_path_override_parsers_are_closed() { let windows = br#"{ - "schema":"satelle.host-service.v2", + "schema":"satelle.host-service.v3", "daemon_arguments":["host","start","--foreground","--bind","127.0.0.1:3001"], "environment":{"SATELLE_STATE_DIR":"C:\\Users\\operator\\AppData\\Local\\Satelle\\state"}, - "setup_ledger_retention_ms":3600000 + "storage_policy":{ + "setup_ledger_retention_ms":3600000, + "session_metadata_retention_hours":168, + "operator_log_retained_files":5 + } }"#; let parsed = parse_service_path_overrides(RemoteTarget::WindowsX64Msvc, windows) .expect("valid Windows service config"); @@ -6564,7 +6611,7 @@ mod tests { assert!( parse_service_path_overrides( RemoteTarget::WindowsX64Msvc, - br#"{"schema":"satelle.host-service.v2","daemon_arguments":["host","start","--foreground","--bind","127.0.0.1:3001"],"environment":{"OTHER":"C:\\safe"},"setup_ledger_retention_ms":3600000}"#, + br#"{"schema":"satelle.host-service.v3","daemon_arguments":["host","start","--foreground","--bind","127.0.0.1:3001"],"environment":{"OTHER":"C:\\safe"},"storage_policy":{"setup_ledger_retention_ms":3600000,"session_metadata_retention_hours":168,"operator_log_retained_files":5}}"#, ) .is_err() ); @@ -6580,7 +6627,12 @@ mod tests { Path::new("/Users/operator/Applications/Satelle & Host/satelle"), "127.0.0.1:4001", &overrides, - 3_600_000, + satelle_core::daemon_service::PersistentHostStoragePolicy::new( + 3_600_000, + satelle_core::DEFAULT_SESSION_METADATA_RETENTION_HOURS, + satelle_core::DEFAULT_OPERATOR_LOG_RETAINED_FILES, + ) + .unwrap(), ) .expect("valid launchd plist"); let parsed = parse_service_path_overrides(RemoteTarget::DarwinArm64, plist.as_bytes()) @@ -7878,6 +7930,8 @@ mod tests { provider_smoke_success_cache_ttl: None, provider_smoke_failure_cache_ttl: None, setup_ledger_retention: None, + session_metadata_retention: None, + operator_log_retained_files: None, daemon_idle_timeout: None, desktop_user: None, desktop_session_preference: None, diff --git a/crates/satelle-cli/src/tailscale-serve.rs b/crates/satelle-cli/src/tailscale-serve.rs index a57df11a..fe524af0 100644 --- a/crates/satelle-cli/src/tailscale-serve.rs +++ b/crates/satelle-cli/src/tailscale-serve.rs @@ -972,6 +972,8 @@ mod tests { provider_smoke_failure_cache_ttl: None, daemon_idle_timeout: None, setup_ledger_retention: None, + session_metadata_retention: None, + operator_log_retained_files: None, desktop_user: None, desktop_session_preference: None, desktop_session_native_selector: None, diff --git a/crates/satelle-cli/src/tailscale.rs b/crates/satelle-cli/src/tailscale.rs index e43ddc17..dc343fce 100644 --- a/crates/satelle-cli/src/tailscale.rs +++ b/crates/satelle-cli/src/tailscale.rs @@ -811,6 +811,8 @@ mod tests { provider_smoke_failure_cache_ttl: None, daemon_idle_timeout: None, setup_ledger_retention: None, + session_metadata_retention: None, + operator_log_retained_files: None, desktop_user: None, desktop_session_preference: None, desktop_session_native_selector: None, diff --git a/crates/satelle-cli/src/transport-tests.rs b/crates/satelle-cli/src/transport-tests.rs index a6d7761a..5595cbb0 100644 --- a/crates/satelle-cli/src/transport-tests.rs +++ b/crates/satelle-cli/src/transport-tests.rs @@ -5280,7 +5280,7 @@ fn authenticated_direct_protocol_mismatch_retains_daemon_version_for_maintenance "details": { "daemon_version": "0.0.9", "reason": "unsupported", - "supported_versions": ["13"], + "supported_versions": ["14"], "received_version": "11", }, "docs_url": null, @@ -6138,6 +6138,36 @@ fn admission_failures_preserve_definitive_and_ambiguous_phases() { assert!(api_failure.durable_handles().is_none()); } +#[test] +fn direct_session_resource_reads_preserve_session_not_found_identity() { + let session_id = SessionId::new(); + let api_error: satelle_transport::ApiError = serde_json::from_value(serde_json::json!({ + "schema_version": "satelle.error.v1", + "request_id": satelle_transport::RequestId::new().to_string(), + "host_identity": "host-direct-test", + "code": "session-not-found", + "category": "not_found", + "retryable": false, + "message": "Session was not found", + "details": null, + "docs_url": null, + "suggested_commands": [] + })) + .expect("deserialize Session-not-found API error"); + + let mapped = direct_session_resource_error( + "direct-test", + &session_id, + DaemonClientError::Api { + status: 404_u16.try_into().expect("404 is a valid HTTP status"), + error: Box::new(api_error), + }, + ); + + assert_eq!(mapped.code, ErrorCode::SessionNotFound); + assert!(mapped.message.contains(session_id.as_str())); +} + #[test] fn stop_not_confirmed_api_details_are_validated_and_preserved() { let session_id = SessionId::new(); diff --git a/crates/satelle-cli/src/transport.rs b/crates/satelle-cli/src/transport.rs index b491b70b..7630eda7 100644 --- a/crates/satelle-cli/src/transport.rs +++ b/crates/satelle-cli/src/transport.rs @@ -1,7 +1,7 @@ use crate::{CliFailure, SelectedHost, bootstrap_lock, failure, on_demand_idle_timeout}; use satelle_core::daemon_service::{ - DaemonArtifactPlan, DaemonServicePlan, DaemonServicePlatform, PersistentServiceDecision, - SetupModeSelection, WindowsServiceConfigV2, WindowsTaskDefinition, + DaemonArtifactPlan, DaemonServicePlan, DaemonServicePlatform, PersistentHostStoragePolicy, + PersistentServiceDecision, SetupModeSelection, WindowsServiceConfigV3, WindowsTaskDefinition, }; use satelle_core::doctor::DoctorScopeSelection; use satelle_core::session::{HostIdentityRef, PublicSession, TurnAdmissionFailure}; @@ -346,10 +346,35 @@ pub(crate) trait TransportClient { request: &TurnRequest, ) -> Result; fn status(&self, session_id: &SessionId) -> Result; + fn task_artifacts(&self, session_id: &SessionId) -> Result; fn stop(&self, session_id: &SessionId) -> Result; fn logs(&self, query: &LogPageQuery) -> Result; } +pub(crate) struct TaskArtifacts { + pub(crate) plan: String, + pub(crate) worklog: String, + pub(crate) goal: String, +} + +impl TaskArtifacts { + fn from_host(artifacts: satelle_host::TaskArtifactSet) -> Self { + Self { + plan: artifacts.plan().to_string(), + worklog: artifacts.worklog().to_string(), + goal: artifacts.goal().to_string(), + } + } + + fn from_response(artifacts: satelle_transport::TaskArtifactsResponse) -> Self { + Self { + plan: artifacts.plan().to_string(), + worklog: artifacts.worklog().to_string(), + goal: artifacts.goal().to_string(), + } + } +} + pub(crate) struct RepairLedgerPlan { pub(crate) available: bool, pub(crate) automatic_action_ids: Vec, @@ -947,6 +972,12 @@ impl TransportClient for LocalTransport { self.service.status(session_id) } + fn task_artifacts(&self, session_id: &SessionId) -> Result { + self.service + .task_artifacts(session_id) + .map(TaskArtifacts::from_host) + } + fn stop(&self, session_id: &SessionId) -> Result { self.service.stop(session_id) } @@ -1309,7 +1340,7 @@ fn coordinate_persistent_setup( enum PreparedPersistentService { Windows { task: Box, - config: Box, + config: Box, }, Launchd(ssh_bootstrap::LaunchdServiceDefinition), } @@ -2424,7 +2455,7 @@ impl SshSetupTransport { daemon_path_overrides: &DaemonPathOverrides, remote: &ssh_bootstrap::PersistentServiceRemote<'_>, ) -> Result { - let setup_ledger_retention_ms = resolved_setup_ledger_retention_ms(&self.host_config); + let storage_policy = PersistentHostStoragePolicy::from_host_config(&self.host_config); match target.service_platform() { DaemonServicePlatform::Windows => { let task = remote @@ -2433,10 +2464,10 @@ impl SshSetupTransport { artifact, ) .map_err(|error| map_ssh_daemon_bootstrap_error(&self.alias, error))?; - let config = WindowsServiceConfigV2::new( + let config = WindowsServiceConfigV3::new( "127.0.0.1:3001", daemon_path_overrides, - setup_ledger_retention_ms, + storage_policy, ) .map_err(|error| SatelleError::config_error(error.to_string(), None))?; Ok(PreparedPersistentService::Windows { @@ -2445,7 +2476,7 @@ impl SshSetupTransport { }) } DaemonServicePlatform::Macos => remote - .launchd_definition(artifact, daemon_path_overrides, setup_ledger_retention_ms) + .launchd_definition(artifact, daemon_path_overrides, storage_policy) .map(PreparedPersistentService::Launchd) .map_err(|error| map_ssh_daemon_bootstrap_error(&self.alias, error)), DaemonServicePlatform::Linux => Err(SatelleError::persistent_service_unsupported( @@ -3177,11 +3208,10 @@ pub(crate) fn preflight_ssh_storage_maintenance(host: &SelectedHost) -> Result<( Ok(()) } -fn resolved_setup_ledger_retention_ms(config: &satelle_core::HostConfig) -> u64 { - config.setup_ledger_retention.as_ref().map_or( - satelle_core::daemon_service::DEFAULT_SETUP_LEDGER_RETENTION_MS, - satelle_core::ExplicitDuration::milliseconds, - ) +fn resolved_persistent_storage_policy( + config: &satelle_core::HostConfig, +) -> PersistentHostStoragePolicy { + PersistentHostStoragePolicy::from_host_config(config) } pub(crate) fn preview_ssh_storage_restore( @@ -3220,7 +3250,7 @@ pub(crate) fn preview_ssh_storage_restore( &service_asset_path, host_id, &path_overrides, - resolved_setup_ledger_retention_ms(&host.config), + resolved_persistent_storage_policy(&host.config), ) .map_err(|error| map_ssh_daemon_bootstrap_error(&transport.alias, error))? .ok_or_else(SatelleError::state_conflict)?; @@ -3280,7 +3310,7 @@ pub(crate) fn plan_ssh_storage_backup_cleanup( &service_asset_path, host_id, &path_overrides, - resolved_setup_ledger_retention_ms(&host.config), + resolved_persistent_storage_policy(&host.config), ) .map_err(|error| map_ssh_daemon_bootstrap_error(&transport.alias, error))? .ok_or_else(SatelleError::state_conflict)?; @@ -3914,7 +3944,7 @@ fn inspect_host_maintenance( &destination, host_id, &expected_path_overrides, - resolved_setup_ledger_retention_ms(&host.config), + resolved_persistent_storage_policy(&host.config), ) .map_err(|error| { map_ssh_daemon_bootstrap_error(&transport.alias, error) @@ -6969,6 +6999,10 @@ impl TransportClient for SshSetupTransport { Err(self.unsupported("session status")) } + fn task_artifacts(&self, _session_id: &SessionId) -> Result { + Err(self.unsupported("task artifact export")) + } + fn stop(&self, _session_id: &SessionId) -> Result { Err(self.unsupported("session stop")) } @@ -7232,7 +7266,14 @@ impl TransportClient for DirectTransport { self.client .read_session(session_id) .map(|response| response.session().clone()) - .map_err(|error| direct_transport_error(&self.alias, error)) + .map_err(|error| direct_session_resource_error(&self.alias, session_id, error)) + } + + fn task_artifacts(&self, session_id: &SessionId) -> Result { + self.client + .read_task_artifacts(session_id) + .map(TaskArtifacts::from_response) + .map_err(|error| direct_session_resource_error(&self.alias, session_id, error)) } fn stop(&self, session_id: &SessionId) -> Result { @@ -8123,6 +8164,21 @@ fn direct_transport_error(host: &str, error: DaemonClientError) -> SatelleError } } +fn direct_session_resource_error( + host: &str, + session_id: &SessionId, + error: DaemonClientError, +) -> SatelleError { + if matches!( + &error, + DaemonClientError::Api { error, .. } if error.code() == ApiErrorCode::SessionNotFound + ) { + SatelleError::session_not_found(session_id) + } else { + direct_transport_error(host, error) + } +} + fn direct_run_transport_error(host: &str, error: DaemonClientError) -> SatelleError { match error { DaemonClientError::Transport(error) if error.is_connect() => { diff --git a/crates/satelle-cli/tests/cli.rs b/crates/satelle-cli/tests/cli.rs index 98c591dd..cd3d1509 100644 --- a/crates/satelle-cli/tests/cli.rs +++ b/crates/satelle-cli/tests/cli.rs @@ -351,9 +351,11 @@ fn session_id(stdout: &[u8]) -> String { .to_string() } -fn completed_log_session(state: &TestStateDir) -> String { +fn completed_log_session(state: &TestStateDir) -> (String, std::path::PathBuf) { + let cache = state.path().join("command-history-cache"); let run_output = satelle() .env("SATELLE_STATE_DIR", state.path()) + .env("SATELLE_CACHE_DIR", &cache) .args(["run", "--host", "local-demo", "Open the browser"]) .assert() .success() @@ -362,15 +364,17 @@ fn completed_log_session(state: &TestStateDir) -> String { let session = session_id(&run_output.stdout); satelle() .env("SATELLE_STATE_DIR", state.path()) + .env("SATELLE_CACHE_DIR", &cache) .args(["steer", &session, "Continue"]) .assert() .success(); satelle() .env("SATELLE_STATE_DIR", state.path()) + .env("SATELLE_CACHE_DIR", &cache) .args(["stop", &session]) .assert() .success(); - session + (session, cache) } fn combined_output(output: &assert_cmd::assert::Assert) -> String { @@ -554,7 +558,7 @@ fn production_status_and_stop_do_not_read_or_mutate_demo_state() { for command in ["status", "stop"] { let output = production_satelle() .env("SATELLE_STATE_DIR", state.path()) - .args([command, &session_id, "--json"]) + .args([command, &session_id, "--host", "local-demo", "--json"]) .assert() .code(66) .get_output() @@ -1456,6 +1460,8 @@ fn events_json_emits_newline_delimited_satelle_events() { .args([ "steer", &session_id, + "--host", + "local-demo", "--events", "json", "Continue from the same event stream", @@ -1530,6 +1536,8 @@ api_token = {{ kind = "file", path = {token_path} }} vec![ "steer".to_string(), SessionId::new().to_string(), + "--host".to_string(), + "remote".to_string(), "--events".to_string(), "json".to_string(), "Continue".to_string(), @@ -1903,7 +1911,15 @@ fn explicit_events_override_quiet_mode_for_run_and_steer() { satelle() .env("SATELLE_STATE_DIR", state.path()) .args([ - "steer", session, "--quiet", "--events", "human", "--json", "Continue", + "steer", + session, + "--host", + "local-demo", + "--quiet", + "--events", + "human", + "--json", + "Continue", ]) .assert() .success() @@ -2121,10 +2137,11 @@ fn detach_returns_starting_session_without_event_streaming() { #[test] fn logs_json_applies_tail_session_source_level_and_since_on_the_host() { let state = state_dir(); - let session = completed_log_session(&state); + let (session, cache) = completed_log_session(&state); let output = satelle() .env("SATELLE_STATE_DIR", state.path()) + .env("SATELLE_CACHE_DIR", &cache) .args([ "logs", "--session", @@ -2152,6 +2169,7 @@ fn logs_json_applies_tail_session_source_level_and_since_on_the_host() { let output = satelle() .env("SATELLE_STATE_DIR", state.path()) + .env("SATELLE_CACHE_DIR", &cache) .args([ "logs", "--source", @@ -2174,6 +2192,7 @@ fn logs_json_applies_tail_session_source_level_and_since_on_the_host() { let output = satelle() .env("SATELLE_STATE_DIR", state.path()) + .env("SATELLE_CACHE_DIR", &cache) .args(["logs", "--session", &session, "--level", "warn", "--json"]) .assert() .success() @@ -2185,6 +2204,7 @@ fn logs_json_applies_tail_session_source_level_and_since_on_the_host() { let output = satelle() .env("SATELLE_STATE_DIR", state.path()) + .env("SATELLE_CACHE_DIR", &cache) .args(["logs", "--session", &session, "--since", "30m", "--json"]) .assert() .success() @@ -2194,6 +2214,7 @@ fn logs_json_applies_tail_session_source_level_and_since_on_the_host() { let output = satelle() .env("SATELLE_STATE_DIR", state.path()) + .env("SATELLE_CACHE_DIR", &cache) .args([ "logs", "--session", @@ -2223,10 +2244,11 @@ fn logs_json_applies_tail_session_source_level_and_since_on_the_host() { #[test] fn logs_after_resumes_strictly_after_the_opaque_cursor() { let state = state_dir(); - let session = completed_log_session(&state); + let (session, cache) = completed_log_session(&state); let initial = satelle() .env("SATELLE_STATE_DIR", state.path()) + .env("SATELLE_CACHE_DIR", &cache) .args(["logs", "--session", &session, "--tail", "2", "--json"]) .assert() .success() @@ -2240,6 +2262,7 @@ fn logs_after_resumes_strictly_after_the_opaque_cursor() { let resumed = satelle() .env("SATELLE_STATE_DIR", state.path()) + .env("SATELLE_CACHE_DIR", &cache) .args(["logs", "--session", &session, "--after", cursor, "--json"]) .assert() .success() @@ -5170,8 +5193,14 @@ fn local_demo_outputs_do_not_include_old_product_name() { let session = session_id(&run.get_output().stdout); for args in [ vec!["doctor", "--host", "local-demo"], - vec!["steer", &session, "Continue from the same session"], - vec!["status", &session], + vec![ + "steer", + &session, + "--host", + "local-demo", + "Continue from the same session", + ], + vec!["status", &session, "--host", "local-demo"], vec!["logs", "--host", "local-demo"], ] { let assertion = satelle() diff --git a/crates/satelle-cli/tests/command-history.rs b/crates/satelle-cli/tests/command-history.rs index 15d98fe6..e592aada 100644 --- a/crates/satelle-cli/tests/command-history.rs +++ b/crates/satelle-cli/tests/command-history.rs @@ -817,6 +817,37 @@ fn history_database_wait_does_not_inflate_command_duration() { ); } +#[test] +fn successful_writer_prunes_command_history_older_than_seven_days() { + let fixture = Fixture::new(); + fixture + .command() + .args(["config", "check", "--json"]) + .assert() + .success(); + fixture + .connection() + .execute( + "UPDATE command_history SET started_at = '2000-01-01T00:00:00.000000000Z'", + [], + ) + .expect("age the existing redacted command-history row"); + + fixture + .command() + .args(["config", "check", "--json"]) + .assert() + .success(); + + let rows = fixture + .connection() + .query_row("SELECT COUNT(*) FROM command_history", [], |row| { + row.get::<_, i64>(0) + }) + .expect("count retained command-history rows"); + assert_eq!(rows, 1); +} + #[cfg(unix)] #[test] fn concurrent_first_run_creates_one_secure_cache_root_without_losing_rows() { diff --git a/crates/satelle-cli/tests/session-host-routing.rs b/crates/satelle-cli/tests/session-host-routing.rs index eb6e287f..226b8239 100644 --- a/crates/satelle-cli/tests/session-host-routing.rs +++ b/crates/satelle-cli/tests/session-host-routing.rs @@ -1,4 +1,6 @@ use assert_cmd::Command; +use rusqlite::{Connection, params}; +use satelle_core::SessionId; use satelle_host::ApiBearerToken; use satelle_host::test_support::TestStateDir; use serde_json::Value; @@ -175,6 +177,213 @@ api_token = {{ kind = "file", path = {token_path} }} } } +#[test] +fn implicit_session_host_uses_only_one_recent_cache_candidate() { + let state = state_dir(); + let cache_root = state.path().join("cache"); + let user_config = state.path().join("user-config.toml"); + let token_file = state.path().join("remote.token"); + let token = ApiBearerToken::generate().expect("generate remote API token"); + write_user_config(&token_file, token.expose().as_str()); + let token_path = toml::Value::String(token_file.to_string_lossy().into_owned()).to_string(); + write_user_config( + &user_config, + format!( + r#" +default_host = "remote" + +[hosts.local-demo] +transport = "local" +adapter = "fake" + +[hosts.remote] +transport = "direct" +adapter = "fake" +address = "https://127.0.0.1:9" +expected_host_id = "host-must-not-be-probed" +api_token = {{ kind = "file", path = {token_path} }} +"#, + ), + ); + + let unknown = SessionId::new().to_string(); + let absent = satelle() + .env("SATELLE_CONFIG_FILE", &user_config) + .env("SATELLE_STATE_DIR", state.path()) + .env("SATELLE_CACHE_DIR", &cache_root) + .args(["status", &unknown, "--json"]) + .assert() + .failure() + .get_output() + .clone(); + let absent_error = parse_json_output(&absent.stderr); + assert_eq!(absent_error["code"], "invalid-usage"); + assert!(absent_error["message"].as_str().unwrap().contains("--host")); + assert!(!cache_root.exists(), "candidate lookup must stay read-only"); + + let run = satelle() + .env("SATELLE_CONFIG_FILE", &user_config) + .env("SATELLE_STATE_DIR", state.path()) + .env("SATELLE_CACHE_DIR", &cache_root) + .args(["run", "--host", "local-demo", "Create a cached Session"]) + .assert() + .success() + .get_output() + .clone(); + let session = session_id(&run.stdout); + let implicit = satelle() + .env("SATELLE_CONFIG_FILE", &user_config) + .env("SATELLE_STATE_DIR", state.path()) + .env("SATELLE_CACHE_DIR", &cache_root) + .args(["status", &session, "--json"]) + .assert() + .success() + .get_output() + .clone(); + assert_eq!(parse_json_output(&implicit.stdout)["host"], "local-demo"); + + let database = cache_root.join("command-history/command-history.sqlite3"); + Connection::open(database) + .expect("open command-history cache") + .execute( + "INSERT INTO command_history (command_family, selected_host, selected_profile, session_id, started_at, duration_ms, outcome_status, error_code, cli_version) \ + SELECT command_family, 'remote', selected_profile, session_id, started_at, duration_ms, outcome_status, error_code, cli_version \ + FROM command_history WHERE session_id = ?1 AND selected_host = 'local-demo' LIMIT 1", + params![session], + ) + .expect("add a second configured Host candidate"); + let ambiguous = satelle() + .env("SATELLE_CONFIG_FILE", &user_config) + .env("SATELLE_STATE_DIR", state.path()) + .env("SATELLE_CACHE_DIR", &cache_root) + .args(["status", &session, "--json"]) + .assert() + .failure() + .get_output() + .clone(); + let ambiguous_error = parse_json_output(&ambiguous.stderr); + assert_eq!(ambiguous_error["code"], "invalid-usage"); + assert!( + ambiguous_error["message"] + .as_str() + .unwrap() + .contains("exactly one configured Host") + ); +} + +#[test] +fn session_export_writes_exact_owner_only_redacted_artifacts_without_overwrite() { + let state = state_dir(); + let user_config = state.path().join("user-config.toml"); + write_user_config( + &user_config, + r#" +[hosts.local-demo] +transport = "local" +adapter = "fake" +"#, + ); + let prompt_canary = "PRIVATE_CLI_TASK_ARTIFACT_PROMPT_CANARY"; + let run = satelle() + .env("SATELLE_CONFIG_FILE", &user_config) + .env("SATELLE_STATE_DIR", state.path()) + .args(["run", "--host", "local-demo", prompt_canary]) + .assert() + .success() + .get_output() + .clone(); + let session = session_id(&run.stdout); + let output = state.path().join("task-artifacts"); + + satelle() + .env("SATELLE_CONFIG_FILE", &user_config) + .env("SATELLE_STATE_DIR", state.path()) + .args([ + "session", + "export", + &session, + "--host", + "local-demo", + "--output", + output.to_str().unwrap(), + ]) + .assert() + .success(); + let mut names = fs::read_dir(&output) + .expect("read exported artifact directory") + .map(|entry| { + entry + .expect("read exported artifact entry") + .file_name() + .to_string_lossy() + .into_owned() + }) + .collect::>(); + names.sort(); + assert_eq!(names, ["goal.md", "plan.md", "worklog.md"]); + for name in &names { + let body = fs::read_to_string(output.join(name)).expect("read exported artifact body"); + assert!(!body.contains(prompt_canary)); + } + assert!( + fs::read_to_string(output.join("plan.md")) + .unwrap() + .contains("- Model: fake-model-v1") + ); + assert!( + fs::read_to_string(output.join("goal.md")) + .unwrap() + .contains("not recorded") + ); + #[cfg(unix)] + { + assert_eq!( + fs::metadata(&output).unwrap().permissions().mode() & 0o777, + 0o700 + ); + for name in &names { + assert_eq!( + fs::metadata(output.join(name)) + .unwrap() + .permissions() + .mode() + & 0o777, + 0o600 + ); + } + } + + let retained_plan = fs::read(output.join("plan.md")).expect("read retained plan"); + satelle() + .env("SATELLE_CONFIG_FILE", &user_config) + .env("SATELLE_STATE_DIR", state.path()) + .args([ + "session", + "export", + &session, + "--host", + "local-demo", + "--output", + output.to_str().unwrap(), + ]) + .assert() + .failure(); + assert_eq!(fs::read(output.join("plan.md")).unwrap(), retained_plan); + assert_eq!( + fs::read_dir(state.path()) + .unwrap() + .filter_map(Result::ok) + .filter(|entry| { + entry + .file_name() + .to_string_lossy() + .starts_with(".satelle-task-artifacts-") + }) + .count(), + 0 + ); +} + #[test] fn host_status_resolves_the_selected_host_before_contacting_transport() { let state = state_dir(); diff --git a/crates/satelle-core/src/daemon-service.rs b/crates/satelle-core/src/daemon-service.rs index 56a78ed4..c3c61d35 100644 --- a/crates/satelle-core/src/daemon-service.rs +++ b/crates/satelle-core/src/daemon-service.rs @@ -7,7 +7,7 @@ use std::net::SocketAddr; use std::path::{Path, PathBuf}; use thiserror::Error; -pub const WINDOWS_SERVICE_CONFIG_SCHEMA: &str = "satelle.host-service.v2"; +pub const WINDOWS_SERVICE_CONFIG_SCHEMA: &str = "satelle.host-service.v3"; pub const DEFAULT_SETUP_LEDGER_RETENTION_MS: u64 = 30 * 24 * 60 * 60 * 1_000; #[derive(Clone, Copy, Debug, Serialize, Deserialize, PartialEq, Eq)] @@ -441,7 +441,7 @@ pub fn render_launchd_user_plist( binary: &Path, bind: &str, overrides: &DaemonPathOverrides, - setup_ledger_retention_ms: u64, + storage_policy: PersistentHostStoragePolicy, ) -> Result { let binary = binary .to_str() @@ -453,7 +453,8 @@ pub fn render_launchd_user_plist( if !bind.ip().is_loopback() { return Err(WindowsServiceDefinitionError::InvalidServiceConfig); } - validate_setup_ledger_retention_ms(setup_ledger_retention_ms) + storage_policy + .validate() .map_err(|_| WindowsServiceDefinitionError::InvalidServiceConfig)?; let mut environment = String::new(); for entry in overrides.entries() { @@ -480,13 +481,17 @@ pub fn render_launchd_user_plist( "--foreground--launchd-service", "--bind{}", "--setup-ledger-retention-ms{}", + "--session-metadata-retention-hours{}", + "--operator-log-retained-files{}", "EnvironmentVariables{}", "RunAtLoadKeepAlive", "" ), xml_escape(binary), xml_escape(&bind.to_string()), - setup_ledger_retention_ms, + storage_policy.setup_ledger_retention_ms, + storage_policy.session_metadata_retention_hours, + storage_policy.operator_log_retained_files, environment )) } @@ -522,19 +527,88 @@ pub enum WindowsServiceDefinitionError { InvalidLocalAppDataPath, } +#[derive(Clone, Copy, Debug, Deserialize, Serialize, PartialEq, Eq)] +#[serde(deny_unknown_fields)] +pub struct PersistentHostStoragePolicy { + setup_ledger_retention_ms: u64, + session_metadata_retention_hours: u64, + operator_log_retained_files: usize, +} + +impl PersistentHostStoragePolicy { + pub fn from_host_config(config: &crate::HostConfig) -> Self { + Self { + setup_ledger_retention_ms: config.setup_ledger_retention.as_ref().map_or( + DEFAULT_SETUP_LEDGER_RETENTION_MS, + crate::ExplicitDuration::milliseconds, + ), + session_metadata_retention_hours: config.session_metadata_retention.as_ref().map_or( + crate::DEFAULT_SESSION_METADATA_RETENTION_HOURS, + |retention| retention.hours(), + ), + operator_log_retained_files: config + .operator_log_retained_files + .unwrap_or(crate::DEFAULT_OPERATOR_LOG_RETAINED_FILES), + } + } + + pub fn new( + setup_ledger_retention_ms: u64, + session_metadata_retention_hours: u64, + operator_log_retained_files: usize, + ) -> Result { + let policy = Self { + setup_ledger_retention_ms, + session_metadata_retention_hours, + operator_log_retained_files, + }; + policy + .validate() + .map_err(|_| WindowsServiceDefinitionError::InvalidServiceConfig)?; + Ok(policy) + } + + pub const fn setup_ledger_retention_ms(self) -> u64 { + self.setup_ledger_retention_ms + } + + pub const fn session_metadata_retention_hours(self) -> u64 { + self.session_metadata_retention_hours + } + + pub const fn operator_log_retained_files(self) -> usize { + self.operator_log_retained_files + } + + fn validate(self) -> Result<(), &'static str> { + validate_setup_ledger_retention_ms(self.setup_ledger_retention_ms)?; + if !(crate::MIN_SESSION_METADATA_RETENTION_HOURS + ..=crate::MAX_SESSION_METADATA_RETENTION_HOURS) + .contains(&self.session_metadata_retention_hours) + { + return Err("invalid session metadata retention"); + } + if !(1..=crate::MAX_OPERATOR_LOG_RETAINED_FILES).contains(&self.operator_log_retained_files) + { + return Err("invalid operator log retention"); + } + Ok(()) + } +} + #[derive(Clone, Debug, Serialize, PartialEq, Eq)] -pub struct WindowsServiceConfigV2 { +pub struct WindowsServiceConfigV3 { schema: String, daemon_arguments: Vec, environment: BTreeMap, - setup_ledger_retention_ms: u64, + storage_policy: PersistentHostStoragePolicy, } -impl WindowsServiceConfigV2 { +impl WindowsServiceConfigV3 { pub fn new( bind: &str, overrides: &DaemonPathOverrides, - setup_ledger_retention_ms: u64, + storage_policy: PersistentHostStoragePolicy, ) -> Result { let mut environment = BTreeMap::new(); insert_path_override(&mut environment, "SATELLE_HOME", overrides.home.as_ref()); @@ -569,7 +643,7 @@ impl WindowsServiceConfigV2 { bind.to_string(), ], environment, - setup_ledger_retention_ms, + storage_policy, }; config .validate() @@ -595,8 +669,8 @@ impl WindowsServiceConfigV2 { &self.environment } - pub fn setup_ledger_retention_ms(&self) -> u64 { - self.setup_ledger_retention_ms + pub const fn storage_policy(&self) -> PersistentHostStoragePolicy { + self.storage_policy } pub fn path_overrides(&self) -> DaemonPathOverrides { @@ -641,12 +715,12 @@ impl WindowsServiceConfigV2 { }) { return Err("invalid daemon environment"); } - validate_setup_ledger_retention_ms(self.setup_ledger_retention_ms)?; + self.storage_policy.validate()?; Ok(()) } } -impl<'de> Deserialize<'de> for WindowsServiceConfigV2 { +impl<'de> Deserialize<'de> for WindowsServiceConfigV3 { fn deserialize(deserializer: D) -> Result where D: serde::Deserializer<'de>, @@ -657,7 +731,7 @@ impl<'de> Deserialize<'de> for WindowsServiceConfigV2 { schema: String, daemon_arguments: Vec, environment: BTreeMap, - setup_ledger_retention_ms: u64, + storage_policy: PersistentHostStoragePolicy, } let wire = WireConfig::deserialize(deserializer)?; @@ -665,7 +739,7 @@ impl<'de> Deserialize<'de> for WindowsServiceConfigV2 { schema: wire.schema, daemon_arguments: wire.daemon_arguments, environment: wire.environment, - setup_ledger_retention_ms: wire.setup_ledger_retention_ms, + storage_policy: wire.storage_policy, }; config.validate().map_err(D::Error::custom)?; Ok(config) @@ -903,6 +977,11 @@ mod tests { .expect("verified Windows executable") } + fn storage_policy() -> PersistentHostStoragePolicy { + PersistentHostStoragePolicy::new(3_600_000, 30 * 24, 12) + .expect("valid persistent storage policy") + } + #[test] fn windows_task_definition_matches_gap_020_exactly() { let definition = windows_definition(); @@ -957,7 +1036,7 @@ mod tests { log_dir: Some(PathBuf::from(r"C:\Satelle\logs")), ..DaemonPathOverrides::default() }; - let config = WindowsServiceConfigV2::new("127.0.0.1:3001", &overrides, 3_600_000) + let config = WindowsServiceConfigV3::new("127.0.0.1:3001", &overrides, storage_policy()) .expect("valid service config"); assert_eq!(config.schema(), WINDOWS_SERVICE_CONFIG_SCHEMA); assert_eq!( @@ -965,7 +1044,7 @@ mod tests { ["host", "start", "--foreground", "--bind", "127.0.0.1:3001"] ); assert_eq!(config.environment().len(), 5); - assert_eq!(config.setup_ledger_retention_ms(), 3_600_000); + assert_eq!(config.storage_policy(), storage_policy()); assert_eq!( config.environment().keys().cloned().collect::>(), [ @@ -989,7 +1068,7 @@ mod tests { "daemon_arguments", "environment", "schema", - "setup_ledger_retention_ms", + "storage_policy", ] ); } @@ -1000,25 +1079,29 @@ mod tests { "schema": WINDOWS_SERVICE_CONFIG_SCHEMA, "daemon_arguments": ["host", "start", "--foreground", "--bind", "0.0.0.0:3001"], "environment": {}, - "setup_ledger_retention_ms": 3600000 + "storage_policy": storage_policy() }); - assert!(serde_json::from_value::(invalid_arguments).is_err()); + assert!(serde_json::from_value::(invalid_arguments).is_err()); let invalid_environment = serde_json::json!({ "schema": WINDOWS_SERVICE_CONFIG_SCHEMA, "daemon_arguments": ["host", "start", "--foreground", "--bind", "127.0.0.1:3001"], "environment": {"PATH": "C:\\attacker"}, - "setup_ledger_retention_ms": 3600000 + "storage_policy": storage_policy() }); - assert!(serde_json::from_value::(invalid_environment).is_err()); + assert!(serde_json::from_value::(invalid_environment).is_err()); let invalid_retention = serde_json::json!({ "schema": WINDOWS_SERVICE_CONFIG_SCHEMA, "daemon_arguments": ["host", "start", "--foreground", "--bind", "127.0.0.1:3001"], "environment": {}, - "setup_ledger_retention_ms": 0 + "storage_policy": { + "setup_ledger_retention_ms": 0, + "session_metadata_retention_hours": 720, + "operator_log_retained_files": 12 + } }); - assert!(serde_json::from_value::(invalid_retention).is_err()); + assert!(serde_json::from_value::(invalid_retention).is_err()); } #[test] @@ -1253,7 +1336,7 @@ mod tests { home: Some(PathBuf::from("/Users/operator/Satelle & Host")), ..DaemonPathOverrides::default() }, - 3_600_000, + storage_policy(), ) .expect("valid launchd definition"); assert!(plist.contains("EnvironmentVariables")); @@ -1262,6 +1345,8 @@ mod tests { assert!(plist.contains("127.0.0.1:3001")); assert!(plist.contains("--launchd-service")); assert!(plist.contains("--setup-ledger-retention-ms")); + assert!(plist.contains("--session-metadata-retention-hours")); + assert!(plist.contains("--operator-log-retained-files")); assert!(plist.contains("3600000")); assert!(!plist.contains("0.0.0.0")); assert!(!plist.contains("UserName")); @@ -1274,10 +1359,10 @@ mod tests { state_dir: Some(PathBuf::from(r"C:\Users\operator\Satelle\state")), ..DaemonPathOverrides::default() }; - let config = WindowsServiceConfigV2::new("127.0.0.1:3001", &overrides, 3_600_000) + let config = WindowsServiceConfigV3::new("127.0.0.1:3001", &overrides, storage_policy()) .expect("valid Windows service config"); assert_eq!(config.path_overrides(), overrides); - assert_eq!(config.setup_ledger_retention_ms(), 3_600_000); + assert_eq!(config.storage_policy(), storage_policy()); } } diff --git a/crates/satelle-core/src/lib.rs b/crates/satelle-core/src/lib.rs index c332248d..1f7dfcc0 100644 --- a/crates/satelle-core/src/lib.rs +++ b/crates/satelle-core/src/lib.rs @@ -1,5 +1,5 @@ use directories::ProjectDirs; -use serde::{Deserialize, Serialize}; +use serde::{Deserialize, Deserializer, Serialize}; use serde_json::{Map, Value}; use sha2::{Digest, Sha256}; use std::collections::{BTreeMap, BTreeSet}; @@ -60,10 +60,10 @@ pub use secure_file::{ keyed_secret_comparison_digest, open_new_owner_only_file, open_or_create_owner_only_directory, open_or_create_owner_only_file, open_owner_only_directory, owner_only_secret_destination_exists, persist_new_owner_only_secret_file, - publish_owner_only_secret_file, read_bounded_regular_file_no_follow, - read_owner_controlled_config_file, read_owner_only_secret_config_file, - read_owner_only_secret_file, read_trusted_ca_bundle_file, rollback_owner_only_secret_file, - stage_owner_only_secret_file, sync_owner_only_directory, + publish_new_owner_only_directory, publish_owner_only_secret_file, + read_bounded_regular_file_no_follow, read_owner_controlled_config_file, + read_owner_only_secret_config_file, read_owner_only_secret_file, read_trusted_ca_bundle_file, + rollback_owner_only_secret_file, stage_owner_only_secret_file, sync_owner_only_directory, }; pub const PRODUCT_NAME: &str = "Satelle"; @@ -113,6 +113,8 @@ impl SatelleConfig { provider_smoke_failure_cache_ttl: None, daemon_idle_timeout: None, setup_ledger_retention: None, + session_metadata_retention: None, + operator_log_retained_files: None, desktop_user: None, desktop_session_preference: None, desktop_session_native_selector: None, @@ -297,6 +299,8 @@ pub struct HostConfig { pub provider_smoke_failure_cache_ttl: Option, pub daemon_idle_timeout: Option, pub setup_ledger_retention: Option, + pub session_metadata_retention: Option, + pub operator_log_retained_files: Option, pub desktop_user: Option, pub desktop_session_preference: Option, pub desktop_session_native_selector: Option, @@ -1641,6 +1645,71 @@ impl ExplicitDuration { } } +pub const DEFAULT_SESSION_METADATA_RETENTION_HOURS: u64 = 7 * 24; +pub const MIN_SESSION_METADATA_RETENTION_HOURS: u64 = 7 * 24; +pub const MAX_SESSION_METADATA_RETENTION_HOURS: u64 = 365 * 24; +pub const DEFAULT_OPERATOR_LOG_RETAINED_FILES: usize = 5; +pub const MAX_OPERATOR_LOG_RETAINED_FILES: usize = 100; + +/// Destructive Satelle-owned retention uses a deliberately narrow grammar. +/// It does not share the broader timeout and cache-duration vocabulary. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct RetentionDuration { + raw: String, + hours: u64, +} + +impl RetentionDuration { + pub fn parse(value: &str) -> Option { + let (count, hours_per_unit) = value + .strip_suffix('h') + .map(|count| (count, 1)) + .or_else(|| value.strip_suffix('d').map(|count| (count, 24)))?; + let count = count.parse::().ok()?; + let hours = count.checked_mul(hours_per_unit)?; + if !(MIN_SESSION_METADATA_RETENTION_HOURS..=MAX_SESSION_METADATA_RETENTION_HOURS) + .contains(&hours) + { + return None; + } + Some(Self { + raw: value.to_string(), + hours, + }) + } + + pub fn as_str(&self) -> &str { + &self.raw + } + + pub const fn hours(&self) -> u64 { + self.hours + } +} + +impl Serialize for RetentionDuration { + fn serialize(&self, serializer: S) -> Result + where + S: serde::Serializer, + { + serializer.serialize_str(&self.raw) + } +} + +impl<'de> Deserialize<'de> for RetentionDuration { + fn deserialize(deserializer: D) -> Result + where + D: Deserializer<'de>, + { + let value = String::deserialize(deserializer)?; + Self::parse(&value).ok_or_else(|| { + serde::de::Error::custom( + "retention values require h or d units and must be between 7d and 365d", + ) + }) + } +} + pub const DEFAULT_TURN_EXECUTION_TIMEOUT_MS: u64 = 30 * 60 * 1_000; pub const MAX_TURN_EXECUTION_TIMEOUT_MS: u64 = 24 * 60 * 60 * 1_000; // Keep retention inside an i64 nanosecond span so subtracting it from a @@ -3164,6 +3233,7 @@ fn reject_interpolation(path: &Path, value: &toml::Value) -> Result<(), SatelleE "provider_smoke_failure_cache_ttl", "daemon_idle_timeout", "setup_ledger_retention", + "session_metadata_retention", ] { collect_interpolation_for_value( &format!("{host_path}.{key}"), @@ -3427,6 +3497,29 @@ fn reject_timeout_config_errors(path: &Path, value: &toml::Value) -> Result<(), return Err(SatelleError::duration_unit_required(path, &retention_path)); } } + if let Some(value) = host_table.get("session_metadata_retention") { + let retention_path = format!("{host_path}.session_metadata_retention"); + let Some(value) = value.as_str() else { + return Err(SatelleError::duration_unit_required(path, &retention_path)); + }; + if RetentionDuration::parse(value).is_none() { + return Err(SatelleError::duration_unit_required(path, &retention_path)); + } + } + if let Some(value) = host_table.get("operator_log_retained_files") { + let retained_files = value + .as_integer() + .and_then(|value| usize::try_from(value).ok()); + if !matches!(retained_files, Some(1..=MAX_OPERATOR_LOG_RETAINED_FILES)) { + return Err(SatelleError::config_error( + format!( + "config file {} value {host_path}.operator_log_retained_files must be between 1 and {MAX_OPERATOR_LOG_RETAINED_FILES}", + path.display() + ), + None, + )); + } + } let Some(timeouts) = host_table.get("timeouts").and_then(toml::Value::as_table) else { continue; }; @@ -3728,6 +3821,8 @@ fn reject_unknown_user_config_keys(path: &Path, value: &toml::Value) -> Result<( "provider_smoke_failure_cache_ttl", "daemon_idle_timeout", "setup_ledger_retention", + "session_metadata_retention", + "operator_log_retained_files", "desktop_user", "desktop_session_preference", "desktop_session_native_selector", @@ -3924,6 +4019,54 @@ mod setup_ledger_retention_duration_tests { } } +#[cfg(test)] +mod session_metadata_retention_tests { + use super::*; + + #[test] + fn destructive_retention_has_its_own_bounded_hour_and_day_grammar() { + assert_eq!(RetentionDuration::parse("7d").unwrap().hours(), 7 * 24); + assert_eq!( + RetentionDuration::parse("365d").unwrap().hours(), + MAX_SESSION_METADATA_RETENTION_HOURS + ); + for invalid in ["0h", "167h", "60m", "3600s", "366d", "1.5h", "1H"] { + assert!(RetentionDuration::parse(invalid).is_none(), "{invalid}"); + } + assert!( + ExplicitDuration::parse("1h").is_none(), + "retention hours must not broaden the existing duration grammar" + ); + } + + #[test] + fn host_retention_and_operator_log_count_validate_at_config_boundary() { + let parsed = parse_user_config( + Path::new("/test/config.toml"), + "[hosts.local-demo]\ntransport = \"local\"\nadapter = \"codex\"\nsession_metadata_retention = \"30d\"\noperator_log_retained_files = 12\n", + ) + .expect("parse bounded retention policy"); + let host = parsed.config.hosts.get(LOCAL_DEMO_HOST).unwrap(); + assert_eq!( + host.session_metadata_retention.as_ref().unwrap().hours(), + 30 * 24 + ); + assert_eq!(host.operator_log_retained_files, Some(12)); + + for field in [ + "session_metadata_retention = \"60m\"", + "operator_log_retained_files = 0", + "operator_log_retained_files = 101", + ] { + let raw = format!( + "[hosts.local-demo]\ntransport = \"local\"\nadapter = \"codex\"\n{field}\n" + ); + parse_user_config(Path::new("/test/config.toml"), &raw) + .expect_err("reject an invalid destructive retention policy"); + } + } +} + #[cfg(test)] mod pr08_turn_duration_tests { use super::*; diff --git a/crates/satelle-core/src/profiles.rs b/crates/satelle-core/src/profiles.rs index 780ff870..64155580 100644 --- a/crates/satelle-core/src/profiles.rs +++ b/crates/satelle-core/src/profiles.rs @@ -1,8 +1,9 @@ use super::{ ConfigInterpolation, ErrorCode, ExplicitDuration, HostConfig, LogVerbosity, - PresentationOutputFormat, SatelleConfig, SatelleError, TimeoutConfig, UnknownConfigKey, - collect_interpolation_for_value, collect_unknown_keys_for_table, finish_interpolation_check, - interpolation_syntax, optional_non_empty_env, + MAX_OPERATOR_LOG_RETAINED_FILES, PresentationOutputFormat, RetentionDuration, SatelleConfig, + SatelleError, TimeoutConfig, UnknownConfigKey, collect_interpolation_for_value, + collect_unknown_keys_for_table, finish_interpolation_check, interpolation_syntax, + optional_non_empty_env, }; use serde::{Deserialize, Serialize}; use serde_json::Value; @@ -25,6 +26,8 @@ const PROFILE_KEYS: &[&str] = &[ "provider_smoke_failure_cache_ttl", "daemon_idle_timeout", "setup_ledger_retention", + "session_metadata_retention", + "operator_log_retained_files", ]; const TIMEOUT_KEYS: &[&str] = &["native_readiness", "provider_smoke_test", "turn_execution"]; @@ -85,6 +88,8 @@ pub(super) struct ProfileConfig { provider_smoke_failure_cache_ttl: Option, daemon_idle_timeout: Option, setup_ledger_retention: Option, + session_metadata_retention: Option, + operator_log_retained_files: Option, } impl ProfileConfig { @@ -174,6 +179,14 @@ impl ProfileConfig { if let Some(retention) = &self.setup_ledger_retention { host.setup_ledger_retention = Some(retention.clone()); } + if source.allows_user_policy() { + if let Some(retention) = &self.session_metadata_retention { + host.session_metadata_retention = Some(retention.clone()); + } + if let Some(retained_files) = self.operator_log_retained_files { + host.operator_log_retained_files = Some(retained_files); + } + } if source.allows_user_policy() && let Some(enabled) = self.experimental_provider_computer_use { @@ -418,6 +431,11 @@ fn reject_profile_interpolation( table.get("setup_ledger_retention"), &mut interpolations, ); + collect_interpolation_for_value( + &format!("{profile_path}.session_metadata_retention"), + table.get("session_metadata_retention"), + &mut interpolations, + ); if let Some(timeouts) = table.get("timeouts").and_then(toml::Value::as_table) { for key in TIMEOUT_KEYS { collect_interpolation_for_value( @@ -476,6 +494,29 @@ fn reject_profile_duration_errors( return Err(SatelleError::duration_unit_required(path, &retention_path)); } } + if let Some(value) = table.get("session_metadata_retention") { + let retention_path = format!("{profile_path}.session_metadata_retention"); + let Some(value) = value.as_str() else { + return Err(SatelleError::duration_unit_required(path, &retention_path)); + }; + if RetentionDuration::parse(value).is_none() { + return Err(SatelleError::duration_unit_required(path, &retention_path)); + } + } + if let Some(value) = table.get("operator_log_retained_files") { + let retained_files = value + .as_integer() + .and_then(|value| usize::try_from(value).ok()); + if !matches!(retained_files, Some(1..=MAX_OPERATOR_LOG_RETAINED_FILES)) { + return Err(SatelleError::config_error( + format!( + "config file {} value {profile_path}.operator_log_retained_files must be between 1 and {MAX_OPERATOR_LOG_RETAINED_FILES}", + path.display() + ), + None, + )); + } + } let Some(timeouts) = table.get("timeouts").and_then(toml::Value::as_table) else { return Ok(()); }; @@ -734,4 +775,35 @@ mod timeout_profile_tests { 60 * 60 * 1_000 ); } + + #[test] + fn only_user_selected_profiles_override_destructive_retention() { + let profile: ProfileConfig = toml::from_str( + "session_metadata_retention = \"30d\"\noperator_log_retained_files = 12\n", + ) + .expect("parse profile retention policy"); + + for source in [ + ProfileSelectionSource::UserConfig, + ProfileSelectionSource::CliFlag, + ] { + let mut host = base_host(); + profile.apply_to_host(super::super::LOCAL_DEMO_HOST, &mut host, source); + assert_eq!( + host.session_metadata_retention.as_ref().unwrap().hours(), + 30 * 24 + ); + assert_eq!(host.operator_log_retained_files, Some(12)); + } + + for source in [ + ProfileSelectionSource::ProjectConfig, + ProfileSelectionSource::Environment, + ] { + let mut host = base_host(); + profile.apply_to_host(super::super::LOCAL_DEMO_HOST, &mut host, source); + assert!(host.session_metadata_retention.is_none()); + assert!(host.operator_log_retained_files.is_none()); + } + } } diff --git a/crates/satelle-core/src/secure-file.rs b/crates/satelle-core/src/secure-file.rs index 53831db0..483a29e4 100644 --- a/crates/satelle-core/src/secure-file.rs +++ b/crates/satelle-core/src/secure-file.rs @@ -879,6 +879,41 @@ fn constant_time_digest_eq(left: &[u8; 32], right: &[u8; 32]) -> bool { == 0 } +/// Atomically publishes one new owner-only sibling directory without +/// replacing an existing destination. +pub fn publish_new_owner_only_directory( + source: &Path, + destination: &Path, +) -> Result<(), SecureFileError> { + let parent = source + .parent() + .filter(|parent| Some(*parent) == destination.parent()) + .ok_or(SecureFileError::UnsafeOrUnavailable)?; + let directory = open_owner_only_directory(parent)?; + // Validate the complete staging boundary before dropping its handle for + // Windows rename semantics. The pinned parent still owns both names. + drop(open_owner_only_directory(source)?); + match move_sibling_file_without_replace(source, destination, &directory)? { + NoReplaceMoveOutcome::Moved => { + if let Err(error) = sync_owner_only_directory(parent, &directory) { + return match move_sibling_file_without_replace(destination, source, &directory)? { + NoReplaceMoveOutcome::Moved => { + sync_owner_only_directory(parent, &directory)?; + Err(error) + } + NoReplaceMoveOutcome::SourceMissing + | NoReplaceMoveOutcome::DestinationOccupied => { + Err(SecureFileError::RollbackFailed) + } + }; + } + Ok(()) + } + NoReplaceMoveOutcome::DestinationOccupied => Err(SecureFileError::OverwriteRequired), + NoReplaceMoveOutcome::SourceMissing => Err(SecureFileError::UnsafeOrUnavailable), + } +} + #[derive(Clone, Copy, Debug, Eq, PartialEq)] enum SecretArtifactKind { Missing, diff --git a/crates/satelle-core/src/session.rs b/crates/satelle-core/src/session.rs index 28bfd17a..6ed26d59 100644 --- a/crates/satelle-core/src/session.rs +++ b/crates/satelle-core/src/session.rs @@ -327,6 +327,17 @@ pub enum ApprovalPolicy { Never, } +impl ApprovalPolicy { + pub const fn as_str(self) -> &'static str { + match self { + Self::Untrusted => "untrusted", + Self::OnFailure => "on_failure", + Self::OnRequest => "on_request", + Self::Never => "never", + } + } +} + #[derive(Clone, Copy, Debug, Eq, PartialEq)] pub enum SandboxPolicy { ReadOnly, @@ -334,6 +345,16 @@ pub enum SandboxPolicy { DangerFullAccess, } +impl SandboxPolicy { + pub const fn as_str(self) -> &'static str { + match self { + Self::ReadOnly => "read_only", + Self::WorkspaceWrite => "workspace_write", + Self::DangerFullAccess => "danger_full_access", + } + } +} + /// Operator-selected execution posture for one Turn. /// /// This is explicit request intent. Adapters must consume the resulting @@ -352,6 +373,15 @@ pub enum FeatureChoice { Enabled, } +impl FeatureChoice { + pub const fn as_str(self) -> &'static str { + match self { + Self::Disabled => "disabled", + Self::Enabled => "enabled", + } + } +} + #[derive(Clone, Copy, Debug, Eq, PartialEq)] pub struct ExperimentalFeatureChoices { computer_use: FeatureChoice, diff --git a/crates/satelle-host/src/daemon.rs b/crates/satelle-host/src/daemon.rs index 22ccb7cd..4b8164d7 100644 --- a/crates/satelle-host/src/daemon.rs +++ b/crates/satelle-host/src/daemon.rs @@ -40,6 +40,7 @@ pub struct DaemonRuntimeStatus { session_count: usize, active_turn_count: usize, recovery_pending_turn_count: usize, + operator_log_health: crate::OperatorLogSinkHealth, } /// Authoritative Host state used when a Client reconnects after losing its @@ -83,6 +84,10 @@ impl DaemonRuntimeStatus { pub const fn recovery_pending_turn_count(&self) -> usize { self.recovery_pending_turn_count } + + pub const fn operator_log_health(&self) -> crate::OperatorLogSinkHealth { + self.operator_log_health + } } /// Host-owned capability evidence. Route availability and network limits stay @@ -1958,6 +1963,7 @@ fn daemon_status(snapshot: crate::runtime::RuntimeSnapshot) -> DaemonRuntimeStat session_count: snapshot.session_count(), active_turn_count: snapshot.active_turn_count(), recovery_pending_turn_count: snapshot.recovery_pending_turn_count(), + operator_log_health: snapshot.operator_log_health(), } } diff --git a/crates/satelle-host/src/lib-tests.rs b/crates/satelle-host/src/lib-tests.rs index 1037fc9c..2bc4bccb 100644 --- a/crates/satelle-host/src/lib-tests.rs +++ b/crates/satelle-host/src/lib-tests.rs @@ -86,7 +86,12 @@ fn production_service_reports_the_frozen_service_config_path_set() { state_dir: Some(configured_state_root.clone()), ..DaemonPathOverrides::default() }, - 3_600_000, + satelle_core::daemon_service::PersistentHostStoragePolicy::new( + 3_600_000, + satelle_core::DEFAULT_SESSION_METADATA_RETENTION_HOURS, + satelle_core::DEFAULT_OPERATOR_LOG_RETAINED_FILES, + ) + .expect("valid persistent storage policy"), ) .expect("valid persistent service configuration"); let paths = service diff --git a/crates/satelle-host/src/lib.rs b/crates/satelle-host/src/lib.rs index cf86b998..99a85525 100644 --- a/crates/satelle-host/src/lib.rs +++ b/crates/satelle-host/src/lib.rs @@ -88,12 +88,48 @@ use std::sync::{Arc, Condvar, Mutex, RwLock, RwLockReadGuard, Weak}; use std::time::{Duration, Instant}; use storage::Storage; pub use storage::{ - SetupActionPlan, SetupActionRecord, SetupActionSkipReason, SetupActionStatus, - SetupOperationKind, SetupRepairAction, SetupRepairDecision, SetupRepairPlan, - SetupRepairPostcondition, SetupRepairProbe, SetupRunPlan, SetupRunRecord, SetupRunStatus, + OperatorLogFailureKind, OperatorLogSinkHealth, SetupActionPlan, SetupActionRecord, + SetupActionSkipReason, SetupActionStatus, SetupOperationKind, SetupRepairAction, + SetupRepairDecision, SetupRepairPlan, SetupRepairPostcondition, SetupRepairProbe, SetupRunPlan, + SetupRunRecord, SetupRunStatus, }; use zeroize::Zeroizing; +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct TaskArtifactSet { + session_id: SessionId, + plan: String, + worklog: String, + goal: String, +} + +impl TaskArtifactSet { + pub(crate) fn new(session_id: SessionId, plan: String, worklog: String, goal: String) -> Self { + Self { + session_id, + plan, + worklog, + goal, + } + } + + pub const fn session_id(&self) -> &SessionId { + &self.session_id + } + + pub fn plan(&self) -> &str { + &self.plan + } + + pub fn worklog(&self) -> &str { + &self.worklog + } + + pub fn goal(&self) -> &str { + &self.goal + } +} + pub(crate) const DEFAULT_MODEL_BINDING: &str = "codex-default"; pub(crate) const DEFAULT_PROVIDER_BINDING: &str = "codex-default"; @@ -2653,10 +2689,7 @@ impl HostService { .provider_smoke_failure_cache_ttl .as_ref() .map_or(DEFAULT_PROVIDER_SMOKE_FAILURE_TTL, duration_to_time); - let setup_ledger_retention = config - .setup_ledger_retention - .as_ref() - .map_or(storage::DEFAULT_SETUP_LEDGER_RETENTION, duration_to_time); + let storage_policy = runtime::RuntimeStoragePolicy::from_host_config(config); let policy = runtime::ProductionAdapterPolicy { native_readiness_timeout: timeout, native_readiness_ttl: ttl, @@ -2676,7 +2709,7 @@ impl HostService { operator_log_root, adapter, runtime::RuntimeProviderPolicy::from_host_config(config), - setup_ledger_retention, + storage_policy, ), operation_capacity: Arc::new(OperationCapacity::default()), turn_execution_timeout: configured_turn_execution_timeout(config), @@ -2709,15 +2742,8 @@ impl HostService { /// its Satelle-owned launchd or Windows service configuration. pub fn production_for_service( overrides: &DaemonPathOverrides, - setup_ledger_retention_ms: u64, + storage_policy: satelle_core::daemon_service::PersistentHostStoragePolicy, ) -> Result { - if setup_ledger_retention_ms == 0 - || setup_ledger_retention_ms > satelle_core::MAX_SETUP_LEDGER_RETENTION_MS - { - return Err(SatelleError::invalid_usage( - "persistent Host service setup-ledger retention is invalid", - )); - } let mut config = satelle_core::SatelleConfig::defaults() .hosts .remove(LOCAL_DEMO_HOST) @@ -2727,8 +2753,23 @@ impl HostService { config.daemon_state_dir = overrides.state_dir.clone(); config.daemon_cache_dir = overrides.cache_dir.clone(); config.daemon_log_dir = overrides.log_dir.clone(); - config.setup_ledger_retention = - satelle_core::ExplicitDuration::parse(&format!("{setup_ledger_retention_ms}ms")); + config.setup_ledger_retention = Some( + satelle_core::ExplicitDuration::parse(&format!( + "{}ms", + storage_policy.setup_ledger_retention_ms() + )) + .ok_or_else(|| { + SatelleError::config_error("invalid service setup-ledger retention", None) + })?, + ); + config.session_metadata_retention = Some( + satelle_core::RetentionDuration::parse(&format!( + "{}h", + storage_policy.session_metadata_retention_hours() + )) + .ok_or_else(|| SatelleError::config_error("invalid service Session retention", None))?, + ); + config.operator_log_retained_files = Some(storage_policy.operator_log_retained_files()); Ok(Self::production_for_host(&config)) } @@ -3888,6 +3929,10 @@ impl HostService { self.runtime.status(session_id.clone()) } + pub fn task_artifacts(&self, session_id: &SessionId) -> Result { + self.runtime.task_artifacts(session_id.clone()) + } + pub fn stop(&self, session_id: &SessionId) -> Result { self.runtime.stop(StopCommand::new(session_id.clone())) } diff --git a/crates/satelle-host/src/runtime-model.rs b/crates/satelle-host/src/runtime-model.rs index e30c0080..27320817 100644 --- a/crates/satelle-host/src/runtime-model.rs +++ b/crates/satelle-host/src/runtime-model.rs @@ -2,8 +2,9 @@ use super::RequestIdentity; use super::adapter::AdapterReadiness; use crate::process_identity::{ProcessIdentity, ProcessIdentityError}; use crate::storage::{ - AdmissionContext, IDEMPOTENCY_RETENTION, IdempotencyInput, IdempotentOperation, LeaseOwner, - PrivateRequestToken, RecoverySubject, StorageError, StorageErrorKind, + AdmissionContext, AdmissionReadinessRef, IDEMPOTENCY_RETENTION, IdempotencyInput, + IdempotentOperation, LeaseOwner, PrivateRequestToken, RecoverySubject, StorageError, + StorageErrorKind, }; use satelle_core::session::{ExecutionPolicy, ExpectedRevisions, RetainedOwnership, Session}; use satelle_core::{ @@ -68,6 +69,26 @@ pub(super) fn admission( )) } +pub(super) fn admission_readiness_ref( + readiness: &AdapterReadiness, +) -> Result { + let native = readiness.evidence(); + let provider = readiness.provider_smoke_evidence(); + AdmissionReadinessRef::new( + native.result_id(), + native.observed_at(), + native.source().as_str(), + provider.map(|provider| { + ( + provider.result_id(), + provider.observed_at(), + provider.source().as_str(), + ) + }), + ) + .map_err(storage_failure) +} + pub(super) fn process_identity_failure(error: ProcessIdentityError) -> SatelleError { SatelleError { code: ErrorCode::StorageIntegrityFailed, diff --git a/crates/satelle-host/src/runtime-tests.rs b/crates/satelle-host/src/runtime-tests.rs index 547b912f..1243be32 100644 --- a/crates/satelle-host/src/runtime-tests.rs +++ b/crates/satelle-host/src/runtime-tests.rs @@ -115,10 +115,31 @@ fn production_runtime_with_host_policy( Ok(state_root.join("logs")), adapter, RuntimeProviderPolicy::from_host_config(config), - crate::storage::DEFAULT_SETUP_LEDGER_RETENTION, + super::RuntimeStoragePolicy::from_host_config(config), ) } +#[test] +fn host_retention_config_becomes_the_runtime_storage_policy() { + let mut config = satelle_core::SatelleConfig::defaults() + .hosts + .remove(LOCAL_DEMO_HOST) + .expect("the built-in local Host config exists"); + config.session_metadata_retention = + Some(satelle_core::RetentionDuration::parse("30d").expect("parse session retention")); + config.operator_log_retained_files = Some(12); + + let policy = super::RuntimeStoragePolicy::from_host_config(&config); + + assert_eq!(policy.session_metadata_retention, time::Duration::days(30)); + assert_eq!(policy.operator_log_retained_files, 12); + assert_eq!( + policy.setup_ledger_retention, + crate::storage::DEFAULT_SETUP_LEDGER_RETENTION, + "session and operator-log policy must not change setup-ledger retention", + ); +} + #[cfg(unix)] struct ProviderSecretRecoveryFixture { operation_id: String, @@ -659,6 +680,13 @@ fn runtime_mirrors_only_committed_normalized_log_entries() { assert!(operator_log.contains(&format!("cursor={cursor}"))); assert!(operator_log.contains("event=store_opened subject=host")); assert!(operator_log.contains("message=\"opened Host state store\"")); + assert_eq!( + runtime + .snapshot() + .expect("read runtime sink health") + .operator_log_health(), + crate::storage::OperatorLogSinkHealth::Healthy + ); } #[test] @@ -2010,6 +2038,69 @@ fn adapter_persists_upstream_refs_before_waiting_and_stop_keeps_them_durable() { assert_eq!(final_subject.upstream_goal_ref(), Some(&expected_goal_ref)); } +#[test] +fn task_artifacts_use_only_durable_redacted_session_state() { + let state = crate::TestStateDir::new().expect("temporary state directory should exist"); + let adapter = ReferencePersistingAdapter::default(); + let runtime = RuntimeHandle::new(Ok(state.path().to_path_buf()), adapter.clone()); + let raw_prompt = "PRIVATE_TASK_ARTIFACT_PROMPT_CANARY"; + let session = runtime + .run(RunCommand::detached(LOCAL_DEMO_HOST, raw_prompt)) + .expect("detached work should be admitted") + .session; + if !adapter.references_recorded.wait_for(WAIT_LIMIT) { + adapter.execute_release.signal(); + panic!("the adapter did not durably record its upstream references"); + } + + let artifacts = runtime + .task_artifacts(session.session_id().clone()) + .expect("the Host should render task artifacts from SQLite"); + assert_eq!(artifacts.session_id(), session.session_id()); + assert!(artifacts.plan().starts_with("# Plan\n\n")); + assert!( + artifacts + .plan() + .contains("- Model: fake-model-v1\n- Provider: fake-provider-v1") + ); + assert!(artifacts.plan().contains("- Native Readiness: result=")); + assert!(artifacts.plan().contains(", source=live")); + assert_eq!( + artifacts.goal(), + format!( + "# Goal\n\n- Session ID: {}\n- Upstream Goal Reference: not recorded\n", + session.session_id() + ) + ); + assert!(artifacts.worklog().starts_with("# Worklog\n\n")); + assert!(artifacts.worklog().contains("event=session_started")); + + let redacted = format!( + "{}\n{}\n{}", + artifacts.plan(), + artifacts.worklog(), + artifacts.goal() + ); + assert_privacy_canaries_absent( + "task artifact bodies", + redacted.as_bytes(), + &[ + raw_prompt, + PRIVATE_UPSTREAM_THREAD_REF, + PRIVATE_UPSTREAM_TURN_REF, + PRIVATE_UPSTREAM_GOAL_REF, + ], + ); + + runtime + .stop(StopCommand::new(session.session_id().clone())) + .expect("stop should consume the durable adapter references"); + adapter.execute_release.signal(); + runtime + .wait_for_background() + .expect("the losing execution worker should finish"); +} + #[test] fn adapter_receives_committed_policy_and_resumes_the_private_thread_reference() { let state = crate::TestStateDir::new().expect("temporary state directory should exist"); diff --git a/crates/satelle-host/src/runtime-tests/review-regressions.rs b/crates/satelle-host/src/runtime-tests/review-regressions.rs index 288a0ebc..8381bb31 100644 --- a/crates/satelle-host/src/runtime-tests/review-regressions.rs +++ b/crates/satelle-host/src/runtime-tests/review-regressions.rs @@ -603,7 +603,7 @@ fn stop_winning_before_running_skips_adapter_execution_and_returns_stopped() { execution_mode: satelle_core::session::TurnExecutionMode::Standard, work: super::super::worker::TurnWork { session, - subject: recovery_subject, + subject: *recovery_subject, _heartbeat: heartbeat, }, provider_smoke_event: None, diff --git a/crates/satelle-host/src/runtime.rs b/crates/satelle-host/src/runtime.rs index 4472c283..5147da53 100644 --- a/crates/satelle-host/src/runtime.rs +++ b/crates/satelle-host/src/runtime.rs @@ -50,7 +50,10 @@ use crate::storage::{ ReadinessProbeTerminal, SensitiveRequestDigest, SetupActionSkipReason, SetupRepairPlan, SetupRepairProbe, SetupRunPlan, SetupRunRecord, SetupRunStatus, Storage, StorageSnapshot, }; -use crate::{ApiBearerToken, ApiPrincipal, DaemonLogPage, LogCursor, LogPageQuery}; +use crate::{ + ApiBearerToken, ApiPrincipal, DaemonLogPage, LogCursor, LogPageQuery, LogSubject, + TaskArtifactSet, +}; use recovery::RecoveryQueue; pub(crate) use recovery::VerifiedSetupPostconditions; #[cfg(test)] @@ -63,6 +66,7 @@ use satelle_core::{ }; use serde::{Deserialize, Serialize}; use std::collections::BTreeMap; +use std::fmt::Write as _; use std::ops::{Deref, DerefMut}; use std::path::{Path, PathBuf}; use std::sync::{Arc, Mutex, MutexGuard}; @@ -72,6 +76,13 @@ pub(crate) fn storage_failure(error: crate::storage::StorageError) -> SatelleErr model::storage_failure(error) } +fn artifact_time(value: time::OffsetDateTime) -> Result { + value + .to_offset(time::UtcOffset::UTC) + .format(&time::format_description::well_known::Rfc3339) + .map_err(|_| model::integrity_failure("stored task artifact time is invalid")) +} + #[cfg(test)] thread_local! { static FAIL_NEXT_MAINTENANCE_START_AND_RETAIN: std::cell::Cell = const { @@ -615,7 +626,45 @@ pub(crate) struct RuntimeEngine { live_events: LiveEventHub, process_identity: ProcessIdentity, attachment_store: crate::attachment::AttachmentStore, + session_metadata_retention: time::Duration, + setup_ledger_retention: time::Duration, +} + +#[derive(Clone, Copy)] +pub(crate) struct RuntimeStoragePolicy { + session_metadata_retention: time::Duration, setup_ledger_retention: time::Duration, + operator_log_retained_files: usize, +} + +impl RuntimeStoragePolicy { + pub(crate) fn from_host_config(config: &satelle_core::HostConfig) -> Self { + Self { + session_metadata_retention: config + .session_metadata_retention + .as_ref() + .map_or(crate::storage::DEFAULT_SESSION_RETENTION, |retention| { + time::Duration::hours(retention.hours() as i64) + }), + setup_ledger_retention: config.setup_ledger_retention.as_ref().map_or( + crate::storage::DEFAULT_SETUP_LEDGER_RETENTION, + crate::duration_to_time, + ), + operator_log_retained_files: config + .operator_log_retained_files + .unwrap_or(satelle_core::DEFAULT_OPERATOR_LOG_RETAINED_FILES), + } + } +} + +impl Default for RuntimeStoragePolicy { + fn default() -> Self { + Self { + session_metadata_retention: crate::storage::DEFAULT_SESSION_RETENTION, + setup_ledger_retention: crate::storage::DEFAULT_SETUP_LEDGER_RETENTION, + operator_log_retained_files: satelle_core::DEFAULT_OPERATOR_LOG_RETAINED_FILES, + } + } } #[derive(Clone, Default)] @@ -643,6 +692,7 @@ impl RuntimeProviderPolicy { pub(crate) struct RuntimeSnapshot { host_identity: satelle_core::session::HostIdentityRef, storage: StorageSnapshot, + operator_log_health: crate::storage::OperatorLogSinkHealth, } impl RuntimeSnapshot { @@ -661,6 +711,10 @@ impl RuntimeSnapshot { pub(crate) const fn recovery_pending_turn_count(&self) -> usize { self.storage.recovery_pending_turn_count() } + + pub(crate) const fn operator_log_health(&self) -> crate::storage::OperatorLogSinkHealth { + self.operator_log_health + } } impl RuntimeEngine { @@ -670,7 +724,7 @@ impl RuntimeEngine { adapter: Arc, readiness_probe_driver: Option>, provider_policy: RuntimeProviderPolicy, - setup_ledger_retention: time::Duration, + storage_policy: RuntimeStoragePolicy, provider_smoke_fingerprinter: Option< crate::provider_auth::ProviderSmokeCredentialFingerprinter, >, @@ -693,7 +747,8 @@ impl RuntimeEngine { let engine = Arc::new(Self { storage: Arc::new(Mutex::new(storage)), operator_log: Mutex::new(OperatorLogMirror::new( - OperatorLogPolicy::new(operator_log_root), + OperatorLogPolicy::new(operator_log_root) + .with_retained_files(storage_policy.operator_log_retained_files), mirrored_cursor, )), adapter, @@ -705,7 +760,8 @@ impl RuntimeEngine { live_events: LiveEventHub::new(), process_identity, attachment_store, - setup_ledger_retention, + session_metadata_retention: storage_policy.session_metadata_retention, + setup_ledger_retention: storage_policy.setup_ledger_retention, }); Ok(engine) } @@ -1107,7 +1163,8 @@ impl RuntimeEngine { started_at, &command.identity, &self.process_identity, - )?; + )? + .with_readiness_ref(model::admission_readiness_ref(&readiness)?); let attachments = self.attachment_store.stage(command.attachments)?; let (outcome, provider_smoke_event) = command @@ -1167,7 +1224,8 @@ impl RuntimeEngine { started_at, &command.identity, &self.process_identity, - )?; + )? + .with_readiness_ref(model::admission_readiness_ref(&readiness)?); let attachments = self.attachment_store.stage(command.attachments)?; let (outcome, provider_smoke_event) = command.cancellation.with_commit_gate( command.session_id.clone(), @@ -2066,7 +2124,7 @@ impl RuntimeEngine { }; let work = TurnWork { session, - subject: recovery_subject, + subject: *recovery_subject, _heartbeat: heartbeat, }; let admitted = model::turn_outcome(&work.session, Vec::new()); @@ -2101,9 +2159,210 @@ impl RuntimeEngine { Ok(session.to_public()) } + fn task_artifacts(&self, session_id: &SessionId) -> Result { + // One export uses one retention boundary so a later page cannot expire + // a cursor that an earlier page in the same export already delivered. + let observed_at = time::OffsetDateTime::now_utc(); + self.maintain_session_retention(observed_at)?; + // Keep one storage guard for the full export so the Session, recovery + // subjects, and log pages all describe one coherent durable snapshot. + let mut storage = self.lock_storage()?; + let session = storage + .load_session(session_id) + .map_err(model::storage_failure)? + .ok_or_else(|| SatelleError::session_not_found(session_id))?; + + let mut plan = String::new(); + writeln!(plan, "# Plan\n").expect("writing to a String cannot fail"); + writeln!(plan, "- Session ID: {}", session.id()).expect("writing to a String cannot fail"); + writeln!( + plan, + "- Host Identity: {}", + session.host_identity().as_str() + ) + .expect("writing to a String cannot fail"); + writeln!( + plan, + "- Desktop Binding: {}\n", + session.desktop_binding().as_str() + ) + .expect("writing to a String cannot fail"); + writeln!(plan, "## Turns\n").expect("writing to a String cannot fail"); + + for (ordinal, turn) in session.turns().enumerate() { + let subject = storage + .recovery_subject(session.id(), turn.id()) + .map_err(model::storage_failure)?; + let policy = turn.execution_policy(); + writeln!(plan, "### Turn {}\n", ordinal + 1).expect("writing to a String cannot fail"); + writeln!(plan, "- Turn ID: {}", turn.id()).expect("writing to a String cannot fail"); + writeln!(plan, "- State: {}", turn.state().as_str()) + .expect("writing to a String cannot fail"); + writeln!(plan, "- Model: {}", policy.effective_model().as_str()) + .expect("writing to a String cannot fail"); + writeln!(plan, "- Provider: {}", policy.provider_binding().as_str()) + .expect("writing to a String cannot fail"); + writeln!( + plan, + "- Desktop Session: {}", + policy.desktop_target().session_id() + ) + .expect("writing to a String cannot fail"); + writeln!( + plan, + "- Approval Policy: {}", + policy.approval_policy().as_str() + ) + .expect("writing to a String cannot fail"); + writeln!( + plan, + "- Sandbox Policy: {}", + policy.sandbox_policy().as_str() + ) + .expect("writing to a String cannot fail"); + writeln!( + plan, + "- Turn Timeout Seconds: {}", + policy.timeout_policy().seconds() + ) + .expect("writing to a String cannot fail"); + writeln!( + plan, + "- Computer Use: {}", + policy.experimental_features().computer_use().as_str() + ) + .expect("writing to a String cannot fail"); + writeln!( + plan, + "- Provider Computer Use: {}", + policy + .experimental_features() + .provider_computer_use() + .as_str() + ) + .expect("writing to a String cannot fail"); + writeln!(plan, "- Started At: {}", artifact_time(turn.started_at())?) + .expect("writing to a String cannot fail"); + writeln!(plan, "- Updated At: {}", artifact_time(turn.updated_at())?) + .expect("writing to a String cannot fail"); + writeln!( + plan, + "- Terminal At: {}", + turn.terminal_at() + .map(artifact_time) + .transpose()? + .unwrap_or_else(|| "not recorded".to_string()) + ) + .expect("writing to a String cannot fail"); + if let Some(readiness) = subject.admission_readiness() { + writeln!( + plan, + "- Native Readiness: result={}, observed_at={}, source={}", + readiness.native_result_id(), + artifact_time(readiness.native_observed_at())?, + readiness.native_source() + ) + .expect("writing to a String cannot fail"); + match ( + readiness.provider_result_id(), + readiness.provider_observed_at(), + readiness.provider_source(), + ) { + (Some(result_id), Some(observed_at), Some(source)) => { + writeln!( + plan, + "- Provider Readiness: result={result_id}, observed_at={}, source={source}\n", + artifact_time(observed_at)? + ) + .expect("writing to a String cannot fail"); + } + _ => writeln!(plan, "- Provider Readiness: not recorded\n") + .expect("writing to a String cannot fail"), + } + } else { + writeln!(plan, "- Native Readiness: not recorded") + .expect("writing to a String cannot fail"); + writeln!(plan, "- Provider Readiness: not recorded\n") + .expect("writing to a String cannot fail"); + } + } + + let mut goal = String::new(); + writeln!(goal, "# Goal\n").expect("writing to a String cannot fail"); + writeln!(goal, "- Session ID: {}", session.id()).expect("writing to a String cannot fail"); + writeln!(goal, "- Upstream Goal Reference: not recorded") + .expect("writing to a String cannot fail"); + + let mut worklog = String::new(); + writeln!(worklog, "# Worklog\n").expect("writing to a String cannot fail"); + let mut cursor = None; + loop { + let query = LogPageQuery::forward(cursor, 10_000) + .expect("the artifact log page size is valid") + .with_session(session.id().clone()); + let page = match storage.log_page(&query, observed_at) { + Ok(page) => page, + Err(LogPageStorageError::Storage(error)) => { + return Err(model::storage_failure(error)); + } + Err(LogPageStorageError::CursorExpired { .. }) => { + return Err(model::integrity_failure( + "task artifact log pagination crossed the retention boundary", + )); + } + Err(LogPageStorageError::CursorAhead) => { + return Err(model::integrity_failure( + "task artifact log pagination crossed the Host log tail", + )); + } + }; + for entry in page.entries() { + let turn_id = match entry.subject() { + LogSubject::Turn { turn_id, .. } => turn_id.as_str(), + LogSubject::Host => "host", + }; + writeln!( + worklog, + "- {} [{}] source={} event={} turn={} cursor={}: {}", + artifact_time(entry.timestamp())?, + entry.severity().as_str(), + entry.source().as_str(), + entry.event().as_str(), + turn_id, + entry.cursor(), + entry.event().message(), + ) + .expect("writing to a String cannot fail"); + } + if !page.truncated() { + break; + } + cursor = Some(page.next_cursor()); + } + for turn in session.turns() { + if let Some(summary) = turn.safe_summary() { + writeln!( + worklog, + "- Turn {} terminal summary: {}", + turn.id(), + summary.as_str() + ) + .expect("writing to a String cannot fail"); + } + } + + Ok(TaskArtifactSet::new( + session.id().clone(), + plan, + worklog, + goal, + )) + } + fn log_page(&self, query: &LogPageQuery) -> Result { - self.maintain_session_retention(time::OffsetDateTime::now_utc())?; - match self.lock_storage()?.log_page(query) { + let observed_at = time::OffsetDateTime::now_utc(); + self.maintain_session_retention(observed_at)?; + match self.lock_storage()?.log_page(query, observed_at) { Ok(page) => Ok(page), Err(LogPageStorageError::Storage(error)) => Err(model::storage_failure(error)), Err(LogPageStorageError::CursorExpired { @@ -2150,9 +2409,15 @@ impl RuntimeEngine { fn snapshot(&self) -> Result { self.maintain_session_retention(time::OffsetDateTime::now_utc())?; let storage = self.lock_storage()?; + let operator_log_health = self + .operator_log + .lock() + .map_err(|_| model::integrity_failure("the operator log mirror lock was poisoned"))? + .health(); Ok(RuntimeSnapshot { host_identity: storage.host_identity().map_err(model::storage_failure)?, storage: storage.snapshot().map_err(model::storage_failure)?, + operator_log_health, }) } @@ -2173,8 +2438,9 @@ impl RuntimeEngine { observed_at: time::OffsetDateTime, ) -> Result<(), SatelleError> { self.lock_storage()? - .prune_expired_session_metadata_with_setup_retention( + .prune_expired_session_metadata_with_retention( observed_at, + self.session_metadata_retention, self.setup_ledger_retention, ) .map_err(model::storage_failure) @@ -2227,7 +2493,7 @@ struct LazyRuntime { operator_log_root: Result, engine: Option>, provider_policy: RuntimeProviderPolicy, - setup_ledger_retention: time::Duration, + storage_policy: RuntimeStoragePolicy, provider_smoke_fingerprinter: Option, } @@ -2635,7 +2901,7 @@ impl RuntimeHandle { operator_log_root, engine: None, provider_policy: RuntimeProviderPolicy::default(), - setup_ledger_retention: crate::storage::DEFAULT_SETUP_LEDGER_RETENTION, + storage_policy: RuntimeStoragePolicy::default(), provider_smoke_fingerprinter: None, })), } @@ -2660,7 +2926,7 @@ impl RuntimeHandle { operator_log_root, engine: None, provider_policy, - setup_ledger_retention: crate::storage::DEFAULT_SETUP_LEDGER_RETENTION, + storage_policy: RuntimeStoragePolicy::default(), provider_smoke_fingerprinter: Some( crate::provider_auth::ProviderSmokeCredentialFingerprinter::default(), ), @@ -2692,7 +2958,7 @@ impl RuntimeHandle { operator_log_root, engine: None, provider_policy, - setup_ledger_retention: crate::storage::DEFAULT_SETUP_LEDGER_RETENTION, + storage_policy: RuntimeStoragePolicy::default(), provider_smoke_fingerprinter: Some( crate::provider_auth::ProviderSmokeCredentialFingerprinter::default(), ), @@ -2705,7 +2971,7 @@ impl RuntimeHandle { operator_log_root: Result, adapter: ProductionComputerUseAdapter, provider_policy: RuntimeProviderPolicy, - setup_ledger_retention: time::Duration, + storage_policy: RuntimeStoragePolicy, ) -> Self { let provider_smoke_fingerprinter = adapter.provider_smoke_fingerprinter(); let adapter = Arc::new(adapter); @@ -2720,7 +2986,7 @@ impl RuntimeHandle { operator_log_root, engine: None, provider_policy, - setup_ledger_retention, + storage_policy, provider_smoke_fingerprinter: Some(provider_smoke_fingerprinter), })), } @@ -2731,6 +2997,7 @@ impl RuntimeHandle { self.lazy .lock() .unwrap_or_else(std::sync::PoisonError::into_inner) + .storage_policy .setup_ledger_retention } @@ -2757,7 +3024,7 @@ impl RuntimeHandle { operator_log_root, engine: None, provider_policy: RuntimeProviderPolicy::default(), - setup_ledger_retention: crate::storage::DEFAULT_SETUP_LEDGER_RETENTION, + storage_policy: RuntimeStoragePolicy::default(), provider_smoke_fingerprinter: Some( crate::provider_auth::ProviderSmokeCredentialFingerprinter::default(), ), @@ -3004,6 +3271,13 @@ impl RuntimeHandle { self.engine()?.status(&session_id) } + pub(crate) fn task_artifacts( + &self, + session_id: SessionId, + ) -> Result { + self.engine()?.task_artifacts(&session_id) + } + pub(crate) fn log_page(&self, query: &LogPageQuery) -> Result { self.engine()?.log_page(query) } @@ -3883,7 +4157,7 @@ impl RuntimeHandle { Arc::clone(&self.adapter), self.readiness_probe_driver.clone(), lazy.provider_policy.clone(), - lazy.setup_ledger_retention, + lazy.storage_policy, lazy.provider_smoke_fingerprinter.clone(), )?; lazy.engine = Some(Arc::clone(&engine)); diff --git a/crates/satelle-host/src/storage.rs b/crates/satelle-host/src/storage.rs index a5a943de..ca7bdd6d 100644 --- a/crates/satelle-host/src/storage.rs +++ b/crates/satelle-host/src/storage.rs @@ -33,11 +33,10 @@ use self::open::DATABASE_FILE_NAME; #[cfg(all(test, unix))] use self::open::LOCK_FILE_NAME; use self::open::sqlite_error; -#[cfg(test)] -pub(crate) use self::operator_log::{ - OperatorLogFailureKind, OperatorLogSink, OperatorLogWriteOutcome, -}; +pub use self::operator_log::{OperatorLogFailureKind, OperatorLogSinkHealth}; pub(crate) use self::operator_log::{OperatorLogMirror, OperatorLogPolicy}; +#[cfg(test)] +pub(crate) use self::operator_log::{OperatorLogSink, OperatorLogWriteOutcome}; pub(crate) use self::provider_secret_journal::{ BeginProviderSecretProvisioning, PROVIDER_SECRET_CANDIDATE_HMAC_DOMAIN, PROVIDER_SECRET_PRIOR_HMAC_DOMAIN, ProviderSecretProvisioningJournal, @@ -45,7 +44,7 @@ pub(crate) use self::provider_secret_journal::{ ProviderSecretProvisioningPreflight, ProviderSecretProvisioningReplay, provider_secret_file_paths, }; -pub(crate) use self::retention::DEFAULT_SETUP_LEDGER_RETENTION; +pub(crate) use self::retention::{DEFAULT_SESSION_RETENTION, DEFAULT_SETUP_LEDGER_RETENTION}; pub(crate) use self::setup_ledger::{ MaintenanceLeaseCapability, MaintenanceLeaseState, MaintenanceRecoverySubject, }; @@ -230,9 +229,9 @@ pub(crate) fn finish_completed_offline_storage_maintenance_if_present( } use self::sql::{ StoredIdempotency, ensure_control_lease_available, ensure_no_pending_stop, - insert_control_lease, insert_idempotency, insert_initial_session, insert_safe_log, - insert_terminal_json_idempotency, insert_turn, load_recovery_subject, matching_idempotency, - merge_observed_reference, persist_lifecycle_mutation, require_operation, + insert_admission_readiness, insert_control_lease, insert_idempotency, insert_initial_session, + insert_safe_log, insert_terminal_json_idempotency, insert_turn, load_recovery_subject, + matching_idempotency, merge_observed_reference, persist_lifecycle_mutation, require_operation, synchronize_control_lease, update_session_row, update_turn_idempotency, validate_initial_session, }; @@ -428,10 +427,11 @@ mod ssh_identity_commit_tests { (12, "fnv1a64:a5672c42bd40d2a8"), (13, "fnv1a64:5db2b0aa00a5f745"), (14, "fnv1a64:fb04115e0082c148"), + (15, "fnv1a64:efae7b5838392fa8"), ]; - const EXPECTED_SCHEMA_ROW_COUNT: usize = 69; + const EXPECTED_SCHEMA_ROW_COUNT: usize = 71; const EXPECTED_SCHEMA_SHA256: &str = - "e16065c8dd757275ff5085bb1a16e1edb9c27bb088af1ba0ae1a58fb52327f11"; + "87bd4af01a35dd9f7f5aced198f5a447e2d26f8a23bb66ef3dd78bd6bfd366de"; fn identity() -> HostIdentityRef { HostIdentityRef::new(HOST_IDENTITY.to_string()).expect("valid Host Identity fixture") @@ -580,7 +580,7 @@ mod ssh_identity_commit_tests { let user_version: i64 = connection .query_row("PRAGMA user_version", [], |row| row.get(0)) .expect("read schema user version"); - assert_eq!(user_version, 14); + assert_eq!(user_version, 15); let schema = connection .prepare( @@ -1124,6 +1124,7 @@ pub(crate) struct AdmissionContext { lease_owner: LeaseOwner, idempotency: IdempotencyInput, request_token: PrivateRequestToken, + readiness_ref: Option, } impl AdmissionContext { @@ -1136,9 +1137,15 @@ impl AdmissionContext { lease_owner, idempotency, request_token, + readiness_ref: None, } } + pub(crate) fn with_readiness_ref(mut self, readiness_ref: AdmissionReadinessRef) -> Self { + self.readiness_ref = Some(readiness_ref); + self + } + pub(crate) fn lease_owner(&self) -> &LeaseOwner { &self.lease_owner } @@ -1149,6 +1156,80 @@ impl AdmissionContext { } } +#[derive(Clone, Debug, Eq, PartialEq)] +pub(crate) struct AdmissionReadinessRef { + native_result_id: String, + native_observed_at: OffsetDateTime, + native_source: String, + provider: Option, +} + +#[derive(Clone, Debug, Eq, PartialEq)] +struct AdmissionProviderReadinessRef { + result_id: String, + observed_at: OffsetDateTime, + source: String, +} + +impl AdmissionReadinessRef { + pub(crate) fn new( + native_result_id: impl Into, + native_observed_at: OffsetDateTime, + native_source: &str, + provider: Option<(&str, OffsetDateTime, &str)>, + ) -> Result { + if !matches!(native_source, "cache" | "live") { + return Err(StorageError::new(StorageErrorKind::InvalidInput)); + } + let provider = provider + .map(|(result_id, observed_at, source)| { + if !matches!(source, "cache" | "live" | "refresh") { + return Err(StorageError::new(StorageErrorKind::InvalidInput)); + } + Ok(AdmissionProviderReadinessRef { + result_id: validated_private_reference(result_id.to_string())?, + observed_at, + source: source.to_string(), + }) + }) + .transpose()?; + Ok(Self { + native_result_id: validated_private_reference(native_result_id.into())?, + native_observed_at, + native_source: native_source.to_string(), + provider, + }) + } + + pub(crate) fn native_result_id(&self) -> &str { + &self.native_result_id + } + + pub(crate) const fn native_observed_at(&self) -> OffsetDateTime { + self.native_observed_at + } + + pub(crate) fn native_source(&self) -> &str { + &self.native_source + } + + pub(crate) fn provider_result_id(&self) -> Option<&str> { + self.provider + .as_ref() + .map(|provider| provider.result_id.as_str()) + } + + pub(crate) fn provider_observed_at(&self) -> Option { + self.provider.as_ref().map(|provider| provider.observed_at) + } + + pub(crate) fn provider_source(&self) -> Option<&str> { + self.provider + .as_ref() + .map(|provider| provider.source.as_str()) + } +} + pub(crate) enum ObservedUpstreamRef { Thread(PrivateUpstreamRef), Turn(PrivateUpstreamRef), @@ -1266,6 +1347,7 @@ pub(crate) struct RecoverySubject { upstream_thread_ref: Option, upstream_turn_ref: Option, upstream_goal_ref: Option, + admission_readiness: Option, } impl RecoverySubject { @@ -1304,6 +1386,10 @@ impl RecoverySubject { pub(crate) fn upstream_goal_ref(&self) -> Option<&PrivateUpstreamRef> { self.upstream_goal_ref.as_ref() } + + pub(crate) fn admission_readiness(&self) -> Option<&AdmissionReadinessRef> { + self.admission_readiness.as_ref() + } } impl fmt::Debug for RecoverySubject { @@ -1322,7 +1408,7 @@ impl fmt::Debug for RecoverySubject { pub(crate) enum AdmissionOutcome { Execute { session: Session, - recovery_subject: RecoverySubject, + recovery_subject: Box, }, InProgress(Session), Complete(Session), @@ -2880,6 +2966,9 @@ impl Storage { session.desktop_binding(), )?; insert_initial_session(&transaction, session, &context.request_token)?; + if let Some(readiness) = &context.readiness_ref { + insert_admission_readiness(&transaction, &turn_id, readiness)?; + } insert_control_lease(&transaction, session, &turn_id, &context.lease_owner)?; insert_idempotency( &transaction, @@ -2906,7 +2995,7 @@ impl Storage { .map_err(|source| sqlite_error(StorageErrorKind::OperationFailed, source))?; Ok(AdmissionOutcome::Execute { session: session.clone(), - recovery_subject, + recovery_subject: Box::new(recovery_subject), }) } @@ -2989,6 +3078,9 @@ impl Storage { turn, &context.request_token, )?; + if let Some(readiness) = &context.readiness_ref { + insert_admission_readiness(&transaction, &turn_id, readiness)?; + } insert_control_lease(&transaction, &session, &turn_id, &context.lease_owner)?; insert_idempotency( &transaction, @@ -3015,7 +3107,7 @@ impl Storage { .map_err(|source| sqlite_error(StorageErrorKind::OperationFailed, source))?; Ok(AdmissionOutcome::Execute { session, - recovery_subject, + recovery_subject: Box::new(recovery_subject), }) } diff --git a/crates/satelle-host/src/storage/0015_turn_admission_readiness.sql b/crates/satelle-host/src/storage/0015_turn_admission_readiness.sql new file mode 100644 index 00000000..1545b8e9 --- /dev/null +++ b/crates/satelle-host/src/storage/0015_turn_admission_readiness.sql @@ -0,0 +1,20 @@ +CREATE TABLE turn_admission_readiness ( + turn_id TEXT PRIMARY KEY + REFERENCES turns(turn_id) ON DELETE CASCADE, + native_result_id TEXT NOT NULL, + native_observed_at TEXT NOT NULL, + native_source TEXT NOT NULL CHECK (native_source IN ('cache', 'live')), + provider_result_id TEXT, + provider_observed_at TEXT, + provider_source TEXT CHECK ( + provider_source IS NULL OR provider_source IN ('cache', 'live', 'refresh') + ), + CHECK ( + (provider_result_id IS NULL + AND provider_observed_at IS NULL + AND provider_source IS NULL) + OR (provider_result_id IS NOT NULL + AND provider_observed_at IS NOT NULL + AND provider_source IS NOT NULL) + ) +) STRICT; diff --git a/crates/satelle-host/src/storage/logs.rs b/crates/satelle-host/src/storage/logs.rs index fbeaa536..2a95216b 100644 --- a/crates/satelle-host/src/storage/logs.rs +++ b/crates/satelle-host/src/storage/logs.rs @@ -188,8 +188,9 @@ impl Storage { pub(crate) fn log_page( &mut self, query: &LogPageQuery, + observed_at: OffsetDateTime, ) -> Result { - self.prune_logs_if_needed(OffsetDateTime::now_utc())?; + self.prune_logs_if_needed(observed_at)?; let (expired_through, earliest_available, high_water) = log_retention_bounds(&self.connection)?; if query.mode() == LogPageMode::Forward diff --git a/crates/satelle-host/src/storage/open.rs b/crates/satelle-host/src/storage/open.rs index 411a424b..b1d8c503 100644 --- a/crates/satelle-host/src/storage/open.rs +++ b/crates/satelle-host/src/storage/open.rs @@ -48,7 +48,7 @@ const BUSY_TIMEOUT: Duration = Duration::from_secs(5); const BACKUP_FORMAT_VERSION: u32 = 1; const RESTORE_ACTIVATION_JOURNAL: &str = ".satelle-restore-activation-v1"; const RESTORE_ACTIVATION_JOURNAL_LIMIT: usize = 64 * 1024; -const MIGRATIONS: [Migration; 14] = [ +const MIGRATIONS: [Migration; 15] = [ Migration { version: 1, sql: include_str!("0001_initial.sql"), @@ -133,6 +133,12 @@ const MIGRATIONS: [Migration; 14] = [ seeds_sensitive_state: false, irreversible: false, }, + Migration { + version: 15, + sql: include_str!("0015_turn_admission_readiness.sql"), + seeds_sensitive_state: false, + irreversible: false, + }, ]; #[derive(Clone, Copy)] diff --git a/crates/satelle-host/src/storage/operator-log.rs b/crates/satelle-host/src/storage/operator-log.rs index 0043b94d..601daf7f 100644 --- a/crates/satelle-host/src/storage/operator-log.rs +++ b/crates/satelle-host/src/storage/operator-log.rs @@ -10,7 +10,6 @@ use time::format_description::well_known::Rfc3339; const OPERATOR_LOG_FILE_NAME: &str = "satelle-host.log"; const DEFAULT_ROTATION_BYTES: u64 = 10 * 1024 * 1024; -const DEFAULT_RETAINED_FILES: usize = 5; const MIRROR_PAGE_SIZE: usize = 100; /// Runtime-owned cursor and sink for new authoritative log entries. @@ -24,8 +23,13 @@ pub(crate) struct OperatorLogMirror { impl OperatorLogMirror { pub(crate) fn new(policy: OperatorLogPolicy, mirrored_cursor: u64) -> Self { + let mut sink = OperatorLogSink::new(policy); + // Reconcile the file cap at process start. A read-only daemon may not + // commit another log entry, so write-time reconciliation alone can + // leave generations above a newly reduced cap indefinitely. + sink.reconcile_retention(); Self { - sink: OperatorLogSink::new(policy), + sink, mirrored_cursor, } } @@ -44,7 +48,7 @@ impl OperatorLogMirror { // never depend on retrying a local inspection artifact. match self.sink.write_committed(&entry) { OperatorLogWriteOutcome::Failure(kind) => { - let _failure_kind = kind; + debug_assert_eq!(self.sink.health(), OperatorLogSinkHealth::Degraded(kind)); } OperatorLogWriteOutcome::Written | OperatorLogWriteOutcome::FailureCoalesced => {} @@ -56,6 +60,10 @@ impl OperatorLogMirror { } } } + + pub(crate) const fn health(&self) -> OperatorLogSinkHealth { + self.sink.health() + } } /// File policy after another layer has resolved the OS-native log root. @@ -73,10 +81,15 @@ impl OperatorLogPolicy { Self { root, rotation_bytes: DEFAULT_ROTATION_BYTES, - retained_files: DEFAULT_RETAINED_FILES, + retained_files: satelle_core::DEFAULT_OPERATOR_LOG_RETAINED_FILES, } } + pub(crate) fn with_retained_files(mut self, retained_files: usize) -> Self { + self.retained_files = retained_files; + self + } + #[cfg(test)] pub(crate) fn for_test(root: PathBuf, rotation_bytes: u64, retained_files: usize) -> Self { assert!(rotation_bytes > 0); @@ -100,13 +113,19 @@ impl OperatorLogPolicy { } #[derive(Clone, Copy, Debug, Eq, PartialEq)] -pub(crate) enum OperatorLogFailureKind { +pub enum OperatorLogFailureKind { BoundaryUnavailable, FormatFailed, RotationFailed, WriteFailed, } +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum OperatorLogSinkHealth { + Healthy, + Degraded(OperatorLogFailureKind), +} + #[derive(Debug)] pub(crate) enum OperatorLogWriteOutcome { Written, @@ -133,7 +152,7 @@ pub(crate) struct OperatorLogSink { // The write handle must close before its pinned directory boundary. file: Option, directory: Option, - failure_active: bool, + active_failure: Option, } impl OperatorLogSink { @@ -142,7 +161,14 @@ impl OperatorLogSink { policy, file: None, directory: None, - failure_active: false, + active_failure: None, + } + } + + fn reconcile_retention(&mut self) { + match self.ensure_directory() { + Ok(()) => self.active_failure = None, + Err(kind) => self.active_failure = Some(kind), } } @@ -152,24 +178,31 @@ impl OperatorLogSink { pub(crate) fn write_committed(&mut self, entry: &DaemonLogEntry) -> OperatorLogWriteOutcome { match self.try_write_committed(entry) { Ok(()) => { - self.failure_active = false; + self.active_failure = None; OperatorLogWriteOutcome::Written } Err(kind) => { // Reopen from the pinned owner-only boundary on the next // record. Only the first failure in one uninterrupted outage - // is surfaced for conversion into the existing Doctor model. + // changes the typed health observed by the Doctor owner. self.file = None; - if self.failure_active { + if self.active_failure.is_some() { OperatorLogWriteOutcome::FailureCoalesced } else { - self.failure_active = true; + self.active_failure = Some(kind); OperatorLogWriteOutcome::Failure(kind) } } } } + pub(crate) const fn health(&self) -> OperatorLogSinkHealth { + match self.active_failure { + Some(kind) => OperatorLogSinkHealth::Degraded(kind), + None => OperatorLogSinkHealth::Healthy, + } + } + fn try_write_committed( &mut self, entry: &DaemonLogEntry, @@ -198,12 +231,7 @@ impl OperatorLogSink { } fn ensure_open(&mut self) -> Result<(), OperatorLogFailureKind> { - if self.directory.is_none() { - self.directory = Some( - open_or_create_owner_only_directory(&self.policy.root) - .map_err(|_| OperatorLogFailureKind::BoundaryUnavailable)?, - ); - } + self.ensure_directory()?; if self.file.is_none() { let mut file = open_or_create_owner_only_file(&self.current_path()) .map_err(|_| OperatorLogFailureKind::BoundaryUnavailable)?; @@ -214,6 +242,25 @@ impl OperatorLogSink { Ok(()) } + fn ensure_directory(&mut self) -> Result<(), OperatorLogFailureKind> { + if self.directory.is_none() { + let directory = open_or_create_owner_only_directory(&self.policy.root) + .map_err(|_| OperatorLogFailureKind::BoundaryUnavailable)?; + + // A lower retention setting takes effect as soon as this process + // acquires the log boundary, even when the active file does not + // rotate during this run. + for generation in + self.policy.retained_files..=satelle_core::MAX_OPERATOR_LOG_RETAINED_FILES + { + remove_if_present(&self.rotated_path(generation)) + .map_err(|_| OperatorLogFailureKind::RotationFailed)?; + } + self.directory = Some(directory); + } + Ok(()) + } + fn rotate(&mut self) -> Result<(), OperatorLogFailureKind> { self.file = None; let rotate = || -> std::io::Result<()> { diff --git a/crates/satelle-host/src/storage/retention.rs b/crates/satelle-host/src/storage/retention.rs index 482ad185..934a65be 100644 --- a/crates/satelle-host/src/storage/retention.rs +++ b/crates/satelle-host/src/storage/retention.rs @@ -7,7 +7,8 @@ use satelle_core::SessionId; use time::OffsetDateTime; use time::format_description::well_known::Rfc3339; -const DEFAULT_SESSION_RETENTION: time::Duration = time::Duration::days(7); +pub(crate) const DEFAULT_SESSION_RETENTION: time::Duration = + time::Duration::hours(satelle_core::DEFAULT_SESSION_METADATA_RETENTION_HOURS as i64); pub(crate) const DEFAULT_SETUP_LEDGER_RETENTION: time::Duration = time::Duration::milliseconds( satelle_core::daemon_service::DEFAULT_SETUP_LEDGER_RETENTION_MS as i64, ); @@ -26,19 +27,21 @@ impl Storage { &mut self, observed_at: OffsetDateTime, ) -> Result<(), StorageError> { - self.prune_expired_session_metadata_with_setup_retention( + self.prune_expired_session_metadata_with_retention( observed_at, + DEFAULT_SESSION_RETENTION, DEFAULT_SETUP_LEDGER_RETENTION, ) } - pub(crate) fn prune_expired_session_metadata_with_setup_retention( + pub(crate) fn prune_expired_session_metadata_with_retention( &mut self, observed_at: OffsetDateTime, + session_retention: time::Duration, setup_ledger_retention: time::Duration, ) -> Result<(), StorageError> { let session_cutoff = observed_at - .checked_sub(DEFAULT_SESSION_RETENTION) + .checked_sub(session_retention) .ok_or_else(|| StorageError::new(StorageErrorKind::InvalidInput))?; let session_cutoff_nanos = unix_timestamp_nanos(session_cutoff)?; let setup_cutoff = observed_at diff --git a/crates/satelle-host/src/storage/sql.rs b/crates/satelle-host/src/storage/sql.rs index 83b147dc..29cdc6e0 100644 --- a/crates/satelle-host/src/storage/sql.rs +++ b/crates/satelle-host/src/storage/sql.rs @@ -1,14 +1,15 @@ use super::codec::{ approval_policy_token, feature_choice_integer, format_revision, format_time, format_turn_revision, idempotent_operation_token, load_required_session, log_event_token, - log_severity_token, log_source_token, safe_summary_token, sandbox_policy_token, + log_severity_token, log_source_token, parse_time, safe_summary_token, sandbox_policy_token, turn_idempotency_token, turn_state_token, unix_timestamp_nanos, }; use super::logs::canonical_log; use super::{ - IDEMPOTENCY_RETENTION, IdempotencyInput, IdempotentOperation, LeaseOwner, LogEvent, - LogSeverity, ObservedUpstreamRef, PrivateRequestToken, PrivateUpstreamRef, RecoverySubject, - SafeLogRecord, Storage, StorageError, StorageErrorKind, sqlite_error, + AdmissionReadinessRef, IDEMPOTENCY_RETENTION, IdempotencyInput, IdempotentOperation, + LeaseOwner, LogEvent, LogSeverity, ObservedUpstreamRef, PrivateRequestToken, + PrivateUpstreamRef, RecoverySubject, SafeLogRecord, Storage, StorageError, StorageErrorKind, + sqlite_error, }; use crate::LogSubject; use rusqlite::{Connection, OptionalExtension, Transaction, TransactionBehavior, params}; @@ -277,6 +278,29 @@ pub(super) fn insert_turn( Ok(()) } +pub(super) fn insert_admission_readiness( + transaction: &Transaction<'_>, + turn_id: &TurnId, + readiness: &AdmissionReadinessRef, +) -> Result<(), StorageError> { + transaction + .execute( + "INSERT INTO turn_admission_readiness (turn_id, native_result_id, native_observed_at, native_source, provider_result_id, provider_observed_at, provider_source) \ + VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7)", + params![ + turn_id.as_str(), + readiness.native_result_id(), + format_time(readiness.native_observed_at())?, + readiness.native_source(), + readiness.provider_result_id(), + readiness.provider_observed_at().map(format_time).transpose()?, + readiness.provider_source(), + ], + ) + .map_err(|source| sqlite_error(StorageErrorKind::OperationFailed, source))?; + Ok(()) +} + pub(super) fn update_session_row( transaction: &Transaction<'_>, session: &Session, @@ -587,25 +611,89 @@ pub(super) fn load_recovery_subject( session: &Session, turn_id: &TurnId, ) -> Result { - let turn = session - .turn(turn_id) - .ok_or_else(|| StorageError::new(StorageErrorKind::InvalidStoredState))?; - let (request_token, upstream_thread_ref, upstream_turn_ref, upstream_goal_ref): ( + type StoredRecoverySubjectRefs = ( String, Option, Option, Option, - ) = connection + Option, + Option, + Option, + Option, + Option, + Option, + ); + + let turn = session + .turn(turn_id) + .ok_or_else(|| StorageError::new(StorageErrorKind::InvalidStoredState))?; + let ( + request_token, + upstream_thread_ref, + upstream_turn_ref, + upstream_goal_ref, + native_result_id, + native_observed_at, + native_source, + provider_result_id, + provider_observed_at, + provider_source, + ): StoredRecoverySubjectRefs = connection .query_row( - "SELECT t.request_token, s.upstream_thread_ref, t.upstream_turn_ref, s.upstream_goal_ref \ + "SELECT t.request_token, s.upstream_thread_ref, t.upstream_turn_ref, s.upstream_goal_ref, \ + r.native_result_id, r.native_observed_at, r.native_source, \ + r.provider_result_id, r.provider_observed_at, r.provider_source \ FROM turn_private_refs t \ JOIN turns u ON u.turn_id = t.turn_id \ JOIN session_private_refs s ON s.session_id = u.session_id \ + LEFT JOIN turn_admission_readiness r ON r.turn_id = u.turn_id \ WHERE u.session_id = ?1 AND u.turn_id = ?2", params![session.id().as_str(), turn_id.as_str()], - |row| Ok((row.get(0)?, row.get(1)?, row.get(2)?, row.get(3)?)), + |row| { + Ok(( + row.get(0)?, + row.get(1)?, + row.get(2)?, + row.get(3)?, + row.get(4)?, + row.get(5)?, + row.get(6)?, + row.get(7)?, + row.get(8)?, + row.get(9)?, + )) + }, ) .map_err(|source| sqlite_error(StorageErrorKind::InvalidStoredState, source))?; + let admission_readiness = match (native_result_id, native_observed_at, native_source) { + (Some(result_id), Some(observed_at), Some(source)) => { + let provider = match (provider_result_id, provider_observed_at, provider_source) { + (Some(result_id), Some(observed_at), Some(source)) => { + Some((result_id, parse_time(&observed_at)?, source)) + } + (None, None, None) => None, + _ => return Err(StorageError::new(StorageErrorKind::InvalidStoredState)), + }; + Some(AdmissionReadinessRef::new( + result_id, + parse_time(&observed_at)?, + &source, + provider.as_ref().map(|(result_id, observed_at, source)| { + (result_id.as_str(), *observed_at, source.as_str()) + }), + )?) + } + (None, None, None) => { + if provider_result_id.is_some() + || provider_observed_at.is_some() + || provider_source.is_some() + { + return Err(StorageError::new(StorageErrorKind::InvalidStoredState)); + } + None + } + _ => return Err(StorageError::new(StorageErrorKind::InvalidStoredState)), + }; Ok(RecoverySubject { session_id: session.id().clone(), turn_id: turn_id.clone(), @@ -621,6 +709,7 @@ pub(super) fn load_recovery_subject( .transpose()?, upstream_turn_ref: upstream_turn_ref.map(PrivateUpstreamRef::new).transpose()?, upstream_goal_ref: upstream_goal_ref.map(PrivateUpstreamRef::new).transpose()?, + admission_readiness, }) } diff --git a/crates/satelle-host/src/storage/stop.rs b/crates/satelle-host/src/storage/stop.rs index 50c17d41..f1bd5aee 100644 --- a/crates/satelle-host/src/storage/stop.rs +++ b/crates/satelle-host/src/storage/stop.rs @@ -23,7 +23,7 @@ use time::OffsetDateTime; pub(crate) struct StopClaim { idempotency: IdempotencyInput, - recovery_subject: RecoverySubject, + recovery_subject: Box, } pub(crate) struct StopAdmissionTarget { @@ -273,7 +273,7 @@ impl Storage { let recovery_subject = load_recovery_subject(&transaction, &session, &turn_id)?; let claim = StopClaim { idempotency: idempotency.clone(), - recovery_subject, + recovery_subject: Box::new(recovery_subject), }; transaction .commit() @@ -345,7 +345,7 @@ impl Storage { )?; let claim = StopClaim { idempotency: idempotency.clone(), - recovery_subject, + recovery_subject: Box::new(recovery_subject), }; transaction .commit() diff --git a/crates/satelle-host/src/storage/tests/lifecycle.rs b/crates/satelle-host/src/storage/tests/lifecycle.rs index ad434f9d..31099a80 100644 --- a/crates/satelle-host/src/storage/tests/lifecycle.rs +++ b/crates/satelle-host/src/storage/tests/lifecycle.rs @@ -6,7 +6,15 @@ fn terminal_session_round_trips_with_follow_up_and_exact_snapshot() { let (mut storage, recovery) = Storage::open(state.path()).expect("open storage"); assert!(recovery.is_empty()); let mut session = initial_session(&storage, SESSION_1, TURN_1, at(0)); - let run = admission(IdempotentOperation::Run, "run-1", "request-run-1", at(0)); + let run_readiness = AdmissionReadinessRef::new( + "native-readiness-1", + at(0), + "live", + Some(("provider-readiness-1", at(0), "refresh")), + ) + .expect("construct initial admission readiness"); + let run = admission(IdempotentOperation::Run, "run-1", "request-run-1", at(0)) + .with_readiness_ref(run_readiness.clone()); let admitted = storage .begin_session(&session, &run) @@ -41,6 +49,13 @@ fn terminal_session_round_trips_with_follow_up_and_exact_snapshot() { "thread-private-1", "turn-private-1", ); + storage + .record_upstream_ref( + session.id(), + &turn_id(TURN_1), + &ObservedUpstreamRef::goal("goal-private-1").unwrap(), + ) + .expect("record the durable Goal reference"); session = storage .commit_lifecycle( session.id(), @@ -51,12 +66,16 @@ fn terminal_session_round_trips_with_follow_up_and_exact_snapshot() { ) .expect("commit completion"); + let follow_up_readiness = + AdmissionReadinessRef::new("native-readiness-2", at(3), "cache", None) + .expect("construct follow-up admission readiness"); let steer = admission( IdempotentOperation::Steer, "steer-1", "request-steer-1", at(3), - ); + ) + .with_readiness_ref(follow_up_readiness.clone()); let follow_up = storage .begin_follow_up( session.id(), @@ -101,6 +120,29 @@ fn terminal_session_round_trips_with_follow_up_and_exact_snapshot() { .expect("load Session") .expect("stored Session"); assert_eq!(expected, restored.snapshot()); + assert_eq!(restored.id(), &session_id(SESSION_1)); + assert_eq!(restored.host_identity(), session.host_identity()); + assert_eq!( + storage + .recovery_subject(restored.id(), &turn_id(TURN_1)) + .expect("restore initial Turn metadata") + .admission_readiness(), + Some(&run_readiness) + ); + let follow_up_subject = storage + .recovery_subject(restored.id(), &turn_id(TURN_2)) + .expect("restore follow-up Turn metadata"); + assert_eq!( + follow_up_subject.admission_readiness(), + Some(&follow_up_readiness) + ); + assert_eq!( + follow_up_subject + .upstream_goal_ref() + .expect("restore Session Goal reference") + .as_str(), + "goal-private-1" + ); assert_eq!( "desktop-session-1", restored diff --git a/crates/satelle-host/src/storage/tests/logs.rs b/crates/satelle-host/src/storage/tests/logs.rs index 3c613c41..7b6a05b0 100644 --- a/crates/satelle-host/src/storage/tests/logs.rs +++ b/crates/satelle-host/src/storage/tests/logs.rs @@ -1,5 +1,6 @@ use super::*; use crate::DaemonLogEntry; +use crate::storage::operator_log::{OperatorLogFailureKind, OperatorLogSinkHealth}; use crate::{LogCursor, LogEvent, LogPageQuery, LogSeverity, LogSource}; use std::fs; use std::path::Path; @@ -199,6 +200,62 @@ fn operator_log_rotates_only_above_the_threshold_and_retains_newest_generations( cursor_strings(&[3, 4]) ); assert!(!log_root.join("satelle-host.log.5").exists()); + + operator_log.release_handles_for_test(); + drop(operator_log); + let twelfth = committed_host_log( + &mut storage, + at(0), + LogSource::HostDaemon, + LogSeverity::Warning, + ); + let mut reduced = + OperatorLogSink::new(OperatorLogPolicy::for_test(log_root.clone(), threshold, 2)); + assert!(matches!( + reduced.write_committed(&twelfth), + OperatorLogWriteOutcome::Written + )); + assert!(log_root.join("satelle-host.log.1").exists()); + for generation in 2..=4 { + assert!( + !log_root + .join(format!("satelle-host.log.{generation}")) + .exists(), + "generation {generation} survived the reduced retention cap" + ); + } +} + +#[test] +fn operator_log_mirror_applies_reduced_retention_without_a_new_record() { + let state = TempDir::new().expect("temporary state directory"); + let log_root = state.path().join("operator-logs"); + drop( + satelle_core::open_or_create_owner_only_directory(&log_root) + .expect("create owner-only operator log root"), + ); + fs::write(log_root.join("satelle-host.log"), b"current").expect("write current operator log"); + for generation in 1..=4 { + fs::write( + log_root.join(format!("satelle-host.log.{generation}")), + format!("generation {generation}"), + ) + .expect("write rotated operator log"); + } + + let mirror = OperatorLogMirror::new(OperatorLogPolicy::for_test(log_root.clone(), 1, 2), 0); + + assert_eq!(mirror.health(), OperatorLogSinkHealth::Healthy); + assert!(log_root.join("satelle-host.log").exists()); + assert!(log_root.join("satelle-host.log.1").exists()); + for generation in 2..=4 { + assert!( + !log_root + .join(format!("satelle-host.log.{generation}")) + .exists(), + "generation {generation} survived startup with the reduced retention cap" + ); + } } #[cfg(unix)] @@ -260,6 +317,10 @@ fn operator_log_failures_are_coalesced_and_do_not_roll_back_sqlite() { first_outcome.failure_kind(), Some(OperatorLogFailureKind::BoundaryUnavailable) ); + assert_eq!( + operator_log.health(), + OperatorLogSinkHealth::Degraded(OperatorLogFailureKind::BoundaryUnavailable) + ); assert_sqlite_log_cursors(&storage, &[1]); let second = committed_host_log( @@ -281,6 +342,7 @@ fn operator_log_failures_are_coalesced_and_do_not_roll_back_sqlite() { operator_log.write_committed(&third), OperatorLogWriteOutcome::Written )); + assert_eq!(operator_log.health(), OperatorLogSinkHealth::Healthy); assert_sqlite_log_cursors(&storage, &[1, 2, 3]); assert_eq!( operator_log_cursors(&unusable_log_root.join("satelle-host.log")), @@ -326,7 +388,7 @@ fn log_pages_filter_before_limiting_and_resume_after_the_delivered_cursor() { .expect("append third log"); let tail = storage - .log_page(&LogPageQuery::tail(2).expect("valid tail query")) + .log_page(&LogPageQuery::tail(2).expect("valid tail query"), now) .expect("read tail page"); assert_eq!( tail.entries() @@ -343,6 +405,7 @@ fn log_pages_filter_before_limiting_and_resume_after_the_delivered_cursor() { &LogPageQuery::forward(Some(LogCursor::from_position(first)), 1) .expect("valid forward query") .with_sources([LogSource::CodexAdapter]), + now, ) .expect("read filtered forward page"); assert_eq!(filtered.entries().len(), 1); @@ -354,21 +417,66 @@ fn log_pages_filter_before_limiting_and_resume_after_the_delivered_cursor() { .log_page( &LogPageQuery::forward(Some(LogCursor::from_position(third + 1)), 1) .expect("valid future-shaped query"), + now, ) .expect_err("a cursor above the store high-water mark must be rejected"); assert!(matches!(future, LogPageStorageError::CursorAhead)); } #[test] -fn log_pages_treat_future_since_values_as_an_empty_result() { +fn log_pagination_can_hold_one_retention_boundary_across_pages() { let state = TempDir::new().expect("temporary state directory"); let (mut storage, _) = Storage::open(state.path()).expect("open storage"); - let cursor = storage + let observed_at = at(0) + time::Duration::days(7); + let first = storage + .append_safe_log(&host_log(at(0), LogSource::Storage, LogSeverity::Info)) + .expect("append first boundary log"); + let second = storage + .append_safe_log(&host_log(at(0), LogSource::Storage, LogSeverity::Info)) + .expect("append second boundary log"); + storage .append_safe_log(&host_log( - OffsetDateTime::now_utc(), + at(0) + time::Duration::nanoseconds(1), LogSource::Storage, LogSeverity::Info, )) + .expect("append retained log"); + + let first_page = storage + .log_page( + &LogPageQuery::forward(None, 1).expect("valid first page query"), + observed_at, + ) + .expect("read first page at the export retention boundary"); + assert_eq!(first_page.entries()[0].cursor().position(), first); + assert!(first_page.truncated()); + + let continued = storage + .log_page( + &LogPageQuery::forward(Some(first_page.next_cursor()), 1) + .expect("valid continuation query"), + observed_at, + ) + .expect("continue at the same export retention boundary"); + assert_eq!(continued.entries()[0].cursor().position(), second); + + let crossed = storage + .log_page( + &LogPageQuery::forward(Some(first_page.next_cursor()), 1) + .expect("valid crossed-boundary query"), + observed_at + time::Duration::nanoseconds(1), + ) + .expect_err("advancing the retention boundary expires the first page cursor"); + assert!(matches!(crossed, LogPageStorageError::CursorExpired { .. })); +} + +#[test] +fn log_pages_treat_future_since_values_as_an_empty_result() { + let state = TempDir::new().expect("temporary state directory"); + let (mut storage, _) = Storage::open(state.path()).expect("open storage"); + let now = OffsetDateTime::now_utc(); + let cursor = storage + .append_safe_log(&host_log(now, LogSource::Storage, LogSeverity::Info)) .expect("append current log"); let future = OffsetDateTime::parse("2999-01-01T00:00:00Z", &Rfc3339).expect("future RFC 3339 timestamp"); @@ -378,6 +486,7 @@ fn log_pages_treat_future_since_values_as_an_empty_result() { &LogPageQuery::forward(None, 10) .expect("valid forward query") .with_since(future), + now, ) .expect("a future lower bound is a valid empty query"); @@ -406,6 +515,7 @@ fn retention_expires_only_cursors_that_can_no_longer_resume_the_retained_prefix( .log_page( &LogPageQuery::forward(Some(LogCursor::from_position(0)), 10) .expect("valid expired query"), + now, ) .expect_err("origin cursor must expire after retained history advances"); assert_eq!(expired_error.earliest_available_cursor(), Some(retained)); @@ -415,6 +525,7 @@ fn retention_expires_only_cursors_that_can_no_longer_resume_the_retained_prefix( .log_page( &LogPageQuery::forward(Some(LogCursor::from_position(expired)), 10) .expect("valid boundary query"), + now, ) .expect("the last expired cursor remains a valid resume boundary"); assert_eq!(resumed.entries()[0].cursor().position(), retained); @@ -559,7 +670,7 @@ fn appended_log_timestamps_cannot_move_backwards_behind_the_cursor_order() { assert_eq!(stored[1].record().recorded_at(), now); let page = storage - .log_page(&LogPageQuery::tail(10).expect("valid tail query")) + .log_page(&LogPageQuery::tail(10).expect("valid tail query"), now) .expect("read page after retention maintenance"); assert_eq!(page.entries().len(), 2); } @@ -609,16 +720,20 @@ fn persisted_log_rows_with_partial_subjects_are_rejected() { fn log_reads_enforce_retention_even_when_no_new_log_has_been_written() { let state = TempDir::new().expect("temporary state directory"); let (mut storage, _) = Storage::open(state.path()).expect("open storage"); + let observed_at = OffsetDateTime::now_utc(); let expired = storage .append_safe_log(&host_log( - OffsetDateTime::now_utc() - time::Duration::days(8), + observed_at - time::Duration::days(8), LogSource::Storage, LogSeverity::Info, )) .expect("append an old log"); let page = storage - .log_page(&LogPageQuery::tail(10).expect("valid tail query")) + .log_page( + &LogPageQuery::tail(10).expect("valid tail query"), + observed_at, + ) .expect("read after idle retention window"); assert!(page.entries().is_empty()); assert_eq!(page.next_cursor().position(), expired); @@ -627,6 +742,7 @@ fn log_reads_enforce_retention_even_when_no_new_log_has_been_written() { .log_page( &LogPageQuery::forward(Some(LogCursor::from_position(0)), 10) .expect("valid origin query"), + observed_at, ) .expect_err("the pre-retention origin must be expired"); assert_eq!(error.earliest_available_cursor(), None); diff --git a/crates/satelle-host/src/storage/tests/operational.rs b/crates/satelle-host/src/storage/tests/operational.rs index 31bcd3be..3cba9229 100644 --- a/crates/satelle-host/src/storage/tests/operational.rs +++ b/crates/satelle-host/src/storage/tests/operational.rs @@ -368,23 +368,24 @@ fn rebuild_storage_as_version_eleven_fixture(connection: &Connection) { expires_at ); + DROP TABLE turn_admission_readiness; DROP TABLE authorized_provider_bindings; DROP TABLE provider_smoke_hmac_key; ALTER TABLE setup_runs DROP COLUMN host_update_target_version; ALTER TABLE setup_runs DROP COLUMN host_update_artifact_digest; - DELETE FROM schema_migrations WHERE version IN (12, 13, 14); + DELETE FROM schema_migrations WHERE version IN (12, 13, 14, 15); PRAGMA user_version = 11;", ) .expect("restore the exact version eleven storage schema"); } #[test] -fn operational_evidence_schema_is_migrated_atomically_to_version_fourteen() { +fn operational_evidence_schema_is_migrated_atomically_to_version_fifteen() { let state = TempDir::new().expect("temporary state directory"); let (storage, _) = Storage::open(state.path()).expect("open storage"); let connection = storage.connection_for_test(); - assert_eq!(14_i64, pragma_integer(connection, "user_version")); + assert_eq!(15_i64, pragma_integer(connection, "user_version")); let versions = connection .prepare("SELECT version FROM schema_migrations ORDER BY version") .unwrap() @@ -395,7 +396,7 @@ fn operational_evidence_schema_is_migrated_atomically_to_version_fourteen() { assert_eq!( vec![ 1_i64, 2_i64, 3_i64, 4_i64, 5_i64, 6_i64, 7_i64, 8_i64, 9_i64, 10_i64, 11_i64, 12_i64, - 13_i64, 14_i64, + 13_i64, 14_i64, 15_i64, ], versions ); @@ -417,6 +418,7 @@ fn operational_evidence_schema_is_migrated_atomically_to_version_fourteen() { "logs", "authorized_provider_bindings", "provider_secret_provisioning_journal", + "turn_admission_readiness", ] { let exists: bool = connection .query_row( @@ -485,7 +487,7 @@ fn version_eleven_provider_smoke_rows_upgrade_to_credential_scoped_cache() { let mut upgraded = Storage::open_without_restart_recovery(state.path()) .expect("upgrade the version eleven store"); assert_eq!( - 14_i64, + 15_i64, pragma_integer(upgraded.connection_for_test(), "user_version") ); let credential_columns: i64 = upgraded @@ -578,7 +580,8 @@ fn version_ten_operation_rows_upgrade_without_data_loss_or_foreign_key_damage() storage .connection_for_test() .execute_batch( - "DROP TABLE authorized_provider_bindings; + "DROP TABLE turn_admission_readiness; + DROP TABLE authorized_provider_bindings; DROP TABLE provider_smoke_hmac_key; ALTER TABLE setup_runs DROP COLUMN host_update_target_version; ALTER TABLE setup_runs DROP COLUMN host_update_artifact_digest;", @@ -587,7 +590,7 @@ fn version_ten_operation_rows_upgrade_without_data_loss_or_foreign_key_damage() storage .connection_for_test() .execute( - "DELETE FROM schema_migrations WHERE version IN (11, 12, 13, 14)", + "DELETE FROM schema_migrations WHERE version IN (11, 12, 13, 14, 15)", [], ) .expect("remove version eleven and twelve history"); @@ -600,7 +603,7 @@ fn version_ten_operation_rows_upgrade_without_data_loss_or_foreign_key_damage() let upgraded = Storage::open_without_restart_recovery(state.path()) .expect("upgrade populated version ten storage"); let connection = upgraded.connection_for_test(); - assert_eq!(14_i64, pragma_integer(connection, "user_version")); + assert_eq!(15_i64, pragma_integer(connection, "user_version")); assert_eq!( ("run".to_string(), "in_progress".to_string()), connection @@ -778,13 +781,13 @@ fn newer_schema_history_is_rejected_without_downgrade() { .connection_for_test() .execute( "INSERT INTO schema_migrations (version, checksum, applied_at) - VALUES (15, ?1, '2026-07-21T00:00:00Z')", + VALUES (16, ?1, '2026-07-21T00:00:00Z')", ["aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"], ) .expect("insert future migration"); storage .connection_for_test() - .pragma_update(None, "user_version", 15) + .pragma_update(None, "user_version", 16) .expect("mark future schema"); drop(storage); @@ -795,7 +798,7 @@ fn newer_schema_history_is_rejected_without_downgrade() { assert_eq!(error.kind(), StorageErrorKind::MigrationIntegrity); let connection = Connection::open(state.path().join(DATABASE_FILE_NAME)) .expect("future database remains readable"); - assert_eq!(pragma_integer(&connection, "user_version"), 15); + assert_eq!(pragma_integer(&connection, "user_version"), 16); } #[test] @@ -821,7 +824,8 @@ fn version_seven_api_tokens_upgrade_to_explicit_active_state() { storage .connection_for_test() .execute_batch( - "DROP TABLE admission_cancellations; + "DROP TABLE turn_admission_readiness; + DROP TABLE admission_cancellations; DROP INDEX one_session_per_upstream_goal_ref; ALTER TABLE sessions DROP COLUMN display_name; ALTER TABLE session_private_refs DROP COLUMN upstream_goal_ref; @@ -830,7 +834,7 @@ fn version_seven_api_tokens_upgrade_to_explicit_active_state() { DROP TABLE provider_smoke_hmac_key; ALTER TABLE setup_runs DROP COLUMN host_update_target_version; ALTER TABLE setup_runs DROP COLUMN host_update_artifact_digest; - DELETE FROM schema_migrations WHERE version IN (8, 9, 10, 11, 12, 13, 14); + DELETE FROM schema_migrations WHERE version IN (8, 9, 10, 11, 12, 13, 14, 15); PRAGMA user_version = 7;", ) .expect("recreate the version seven token schema"); @@ -838,7 +842,7 @@ fn version_seven_api_tokens_upgrade_to_explicit_active_state() { let (storage, _) = Storage::open(state.path()).expect("upgrade version seven storage"); assert_eq!( - 14_i64, + 15_i64, pragma_integer(storage.connection_for_test(), "user_version") ); let token_state: String = storage @@ -2085,7 +2089,8 @@ fn version_one_store_upgrades_without_replacing_existing_state() { storage .connection_for_test() .execute_batch( - "DROP TABLE admission_cancellations; + "DROP TABLE turn_admission_readiness; + DROP TABLE admission_cancellations; DROP TABLE setup_actions; DROP TABLE setup_runs; DROP TABLE native_readiness_results; @@ -2096,7 +2101,7 @@ fn version_one_store_upgrades_without_replacing_existing_state() { ALTER TABLE api_tokens DROP COLUMN token_state; DROP TABLE authorized_provider_bindings; DROP TABLE provider_smoke_hmac_key; - DELETE FROM schema_migrations WHERE version IN (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14); + DELETE FROM schema_migrations WHERE version IN (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15); PRAGMA user_version = 1;", ) .unwrap(); @@ -2105,7 +2110,7 @@ fn version_one_store_upgrades_without_replacing_existing_state() { let (storage, _) = Storage::open(state.path()).expect("upgrade version one storage"); assert_eq!(expected_host, storage.host_identity().unwrap()); assert_eq!( - 14_i64, + 15_i64, pragma_integer(storage.connection_for_test(), "user_version") ); @@ -2190,7 +2195,8 @@ fn assert_version_one_corruption_rejected_before_migration( storage .connection_for_test() .execute_batch( - "DROP TABLE admission_cancellations; + "DROP TABLE turn_admission_readiness; + DROP TABLE admission_cancellations; DROP TABLE setup_actions; DROP TABLE setup_runs; DROP TABLE native_readiness_results; @@ -2203,7 +2209,7 @@ fn assert_version_one_corruption_rejected_before_migration( DROP INDEX idempotency_operation_identity; DROP TABLE authorized_provider_bindings; DROP TABLE provider_smoke_hmac_key; - DELETE FROM schema_migrations WHERE version IN (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14); + DELETE FROM schema_migrations WHERE version IN (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15); PRAGMA user_version = 1;", ) .expect("create a logically corrupt version one store"); @@ -2257,7 +2263,8 @@ fn failed_migration_rolls_back_partial_schema_and_preserves_existing_state() { storage .connection_for_test() .execute_batch( - "DROP TABLE admission_cancellations; + "DROP TABLE turn_admission_readiness; + DROP TABLE admission_cancellations; DROP TABLE setup_actions; DROP TABLE setup_runs; DROP TABLE native_readiness_results; @@ -2270,7 +2277,7 @@ fn failed_migration_rolls_back_partial_schema_and_preserves_existing_state() { DROP INDEX idempotency_operation_identity; DROP TABLE authorized_provider_bindings; DROP TABLE provider_smoke_hmac_key; - DELETE FROM schema_migrations WHERE version IN (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14); + DELETE FROM schema_migrations WHERE version IN (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15); PRAGMA user_version = 1; CREATE TABLE migration_sentinel (value TEXT NOT NULL) STRICT; INSERT INTO migration_sentinel (value) VALUES ('preserve-me'); diff --git a/crates/satelle-host/src/storage/tests/retention.rs b/crates/satelle-host/src/storage/tests/retention.rs index b02c6968..ef27ffc6 100644 --- a/crates/satelle-host/src/storage/tests/retention.rs +++ b/crates/satelle-host/src/storage/tests/retention.rs @@ -283,7 +283,11 @@ fn configured_setup_ledger_retention_changes_only_the_ledger_cutoff() { ); storage - .prune_expired_session_metadata_with_setup_retention(finished_at + retention, retention) + .prune_expired_session_metadata_with_retention( + finished_at + retention, + super::super::retention::DEFAULT_SESSION_RETENTION, + retention, + ) .unwrap(); assert_eq!( 1, @@ -291,8 +295,9 @@ fn configured_setup_ledger_retention_changes_only_the_ledger_cutoff() { ); storage - .prune_expired_session_metadata_with_setup_retention( + .prune_expired_session_metadata_with_retention( finished_at + retention + time::Duration::nanoseconds(1), + super::super::retention::DEFAULT_SESSION_RETENTION, retention, ) .unwrap(); @@ -302,6 +307,50 @@ fn configured_setup_ledger_retention_changes_only_the_ledger_cutoff() { ); } +#[test] +fn configured_session_retention_changes_only_the_session_cutoff() { + let state = TempDir::new().expect("temporary state directory"); + let (mut storage, _) = Storage::open(state.path()).expect("open storage"); + let terminal_at = at(10); + let retention = time::Duration::days(8); + let session = terminal_session( + &mut storage, + SESSION_1, + TURN_1, + terminal_at - time::Duration::seconds(1), + terminal_at, + ); + terminal_setup_run( + &mut storage, + "unrelated-setup-ledger", + SetupRunStatus::Completed, + terminal_at, + ); + + storage + .prune_expired_session_metadata_with_retention( + terminal_at + retention, + retention, + super::super::retention::DEFAULT_SETUP_LEDGER_RETENTION, + ) + .unwrap(); + assert!(storage.load_session(session.id()).unwrap().is_some()); + + storage + .prune_expired_session_metadata_with_retention( + terminal_at + retention + time::Duration::nanoseconds(1), + retention, + super::super::retention::DEFAULT_SETUP_LEDGER_RETENTION, + ) + .unwrap(); + assert!(storage.load_session(session.id()).unwrap().is_none()); + assert_eq!( + 1, + setup_rows(&storage, "setup_runs", "unrelated-setup-ledger"), + "session retention must not change setup-ledger retention", + ); +} + #[test] fn setup_ledger_retention_preserves_recovery_work_and_unrelated_host_state() { let state = TempDir::new().expect("temporary state directory"); @@ -797,6 +846,7 @@ fn expiring_the_session_that_owns_all_logs_preserves_the_cursor_high_water() { .log_page( &LogPageQuery::forward(Some(LogCursor::from_position(delivered_cursor)), 10) .expect("valid delivered cursor query"), + observed_at, ) .expect("a delivered cursor must not become cursor-ahead"); assert!(page.entries().is_empty()); diff --git a/crates/satelle-host/src/storage/tests/security.rs b/crates/satelle-host/src/storage/tests/security.rs index c7309a33..d31218b2 100644 --- a/crates/satelle-host/src/storage/tests/security.rs +++ b/crates/satelle-host/src/storage/tests/security.rs @@ -216,6 +216,7 @@ fn lifecycle_schema_excludes_raw_content_and_replayable_event_history() { "setup_actions", "setup_runs", "sqlite_sequence", + "turn_admission_readiness", "turn_policies", "turn_private_refs", "turns", @@ -362,6 +363,19 @@ fn lifecycle_schema_excludes_raw_content_and_replayable_event_history() { "provider_computer_use_enabled", ], ); + assert_table_columns( + &storage, + "turn_admission_readiness", + &[ + "turn_id", + "native_result_id", + "native_observed_at", + "native_source", + "provider_result_id", + "provider_observed_at", + "provider_source", + ], + ); assert_table_columns( &storage, "setup_runs", diff --git a/crates/satelle-transport/src/client.rs b/crates/satelle-transport/src/client.rs index 03a74178..80abe329 100644 --- a/crates/satelle-transport/src/client.rs +++ b/crates/satelle-transport/src/client.rs @@ -12,7 +12,7 @@ use crate::contract::{ ProviderSecretProvisioningPreviewResponse, ProviderSecretProvisioningResponse, ProviderSecretUploadEnvelope, RequestId, SessionResponse, SetupRepairPlanRequest, SetupRepairPlanResponse, SetupVerificationRequest, SetupVerificationResponse, StopRequest, - StopResponse, TurnRequest, provider_secret_upload_aad, + StopResponse, TaskArtifactsResponse, TurnRequest, provider_secret_upload_aad, }; use crate::transport_tls::{ ReqwestTrustError, TlsFailureKind, classify_tls_error, configure_reqwest_trust, @@ -750,6 +750,22 @@ impl DaemonClient { self.send_authenticated(request, request_id, StatusCode::OK) } + pub fn read_task_artifacts( + &self, + session_id: &SessionId, + ) -> Result { + let path = format!("/v1/sessions/{session_id}/task-artifacts"); + let (request, request_id) = self.protected_request(Method::GET, &path)?; + let response: TaskArtifactsResponse = + self.send_authenticated(request, request_id, StatusCode::OK)?; + // Bind the authenticated response body to the requested Session before + // any exported content can cross the transport boundary. + if response.session_id() != session_id { + return Err(DaemonClientError::ResponseContractViolation); + } + Ok(response) + } + pub fn stop_session( &self, session_id: &SessionId, @@ -1299,6 +1315,62 @@ mod tests { server.join().expect("join TLS server"); } + #[test] + fn task_artifact_read_rejects_response_for_another_session() { + let listener = TcpListener::bind("127.0.0.1:0").expect("bind artifact response fixture"); + let address = listener + .local_addr() + .expect("read artifact response fixture address"); + let token = ApiBearerToken::generate().expect("generate daemon token"); + let requested_session = SessionId::new(); + let requested_session_for_server = requested_session.clone(); + let returned_session = SessionId::new(); + let server = std::thread::spawn(move || { + let (mut stream, _) = listener.accept().expect("accept artifact client"); + let mut request = Vec::new(); + let mut chunk = [0_u8; 1024]; + while !request.windows(4).any(|bytes| bytes == b"\r\n\r\n") { + let read = stream.read(&mut chunk).expect("read artifact request"); + assert_ne!(read, 0, "artifact request ended before its headers"); + request.extend_from_slice(&chunk[..read]); + } + let request = String::from_utf8(request).expect("request headers should be UTF-8"); + assert!(request.starts_with(&format!( + "GET /v1/sessions/{requested_session_for_server}/task-artifacts HTTP/1.1\r\n" + ))); + let request_id = header_value(&request, "satelle-request-id") + .expect("request must carry a request ID"); + let body = serde_json::json!({ + "schema_version": "satelle.task_artifacts.v1", + "request_id": request_id, + "host_identity": "host-artifact-test", + "session_id": returned_session, + "plan": "# Plan\n", + "worklog": "# Worklog\n", + "goal": "# Goal\n", + }) + .to_string(); + write!( + stream, + "HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", + body.len() + ) + .expect("write artifact response"); + stream.flush().expect("flush artifact response"); + }); + + let client = DaemonClient::loopback(address, token, "host-artifact-test") + .expect("construct artifact client"); + let error = client + .read_task_artifacts(&requested_session) + .expect_err("another Session's artifacts must fail closed"); + assert!(matches!( + error, + DaemonClientError::ResponseContractViolation + )); + server.join().expect("join artifact response fixture"); + } + #[test] fn capabilities_rejects_protocol_mismatch_before_decoding_the_current_schema() { for protocol_header in ["", "Satelle-Protocol-Version: 5\r\n"] { diff --git a/crates/satelle-transport/src/contract.rs b/crates/satelle-transport/src/contract.rs index 05503e7e..493e0949 100644 --- a/crates/satelle-transport/src/contract.rs +++ b/crates/satelle-transport/src/contract.rs @@ -28,7 +28,8 @@ pub(crate) use session::TurnRequestParts; pub use session::{ AdmissionCancellationOutcome, AdmissionCancellationResponse, ImageAttachment, MAX_IMAGE_ATTACHMENT_BYTES, MAX_IMAGE_ATTACHMENT_BYTES_TOTAL, MAX_IMAGE_ATTACHMENT_COUNT, - SUPPORTED_IMAGE_MEDIA_TYPES, SessionResponse, StopRequest, StopResponse, TurnRequest, + SUPPORTED_IMAGE_MEDIA_TYPES, SessionResponse, StopRequest, StopResponse, TaskArtifactsResponse, + TurnRequest, }; pub use setup::{ BootstrapMaintenanceResponse, DURABLE_SETUP_PENDING_TTL, DurableTokenActivationResponse, @@ -48,10 +49,10 @@ pub(crate) use setup::{ }; pub(crate) const PROTOCOL_VERSION_HEADER: &str = "satelle-protocol-version"; -// Protocol v13 binds Host-update maintenance and selected-run recovery to the -// exact release artifact accepted before mutation. The protocol remains a -// hard cut because repair cannot safely infer an interrupted update target. -pub(crate) const PROTOCOL_VERSION: &str = "13"; +// Protocol v14 adds the authenticated, identity-pinned task artifact read. +// The protocol remains a hard cut because older peers cannot distinguish the +// closed redacted export contract from arbitrary Host file access. +pub(crate) const PROTOCOL_VERSION: &str = "14"; use serde::{Deserialize, Deserializer, Serialize, Serializer}; use std::fmt; @@ -189,7 +190,7 @@ mod tests { } #[test] - fn protocol_version_is_the_v13_hard_cut() { - assert_eq!(PROTOCOL_VERSION, "13"); + fn protocol_version_is_the_v14_hard_cut() { + assert_eq!(PROTOCOL_VERSION, "14"); } } diff --git a/crates/satelle-transport/src/contract/session.rs b/crates/satelle-transport/src/contract/session.rs index bb3af024..26739c18 100644 --- a/crates/satelle-transport/src/contract/session.rs +++ b/crates/satelle-transport/src/contract/session.rs @@ -10,6 +10,7 @@ use std::fmt; define_schema_token!(TurnRequestSchema, "satelle.api.v7"); define_schema_token!(StopRequestSchema, "satelle.api.v1"); define_schema_token!(SessionSchema, "satelle.session.v1"); +define_schema_token!(TaskArtifactsSchema, "satelle.task_artifacts.v1"); define_schema_token!(SessionStopSchema, "satelle.session.stop.v1"); define_schema_token!(AdmissionCancellationSchema, "satelle.admission.cancel.v1"); @@ -457,6 +458,75 @@ impl AuthenticatedResponseContract for SessionResponse { } } +/// Closed authenticated export envelope. Each body is rendered from the +/// Host's redacted SQLite projection and maps to one exact output filename. +#[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)] +#[serde(deny_unknown_fields)] +pub struct TaskArtifactsResponse { + schema_version: TaskArtifactsSchema, + request_id: RequestId, + host_identity: String, + session_id: SessionId, + plan: String, + worklog: String, + goal: String, +} + +impl TaskArtifactsResponse { + pub(crate) fn new( + request_id: RequestId, + host_identity: String, + session_id: SessionId, + plan: String, + worklog: String, + goal: String, + ) -> Self { + Self { + schema_version: TaskArtifactsSchema, + request_id, + host_identity, + session_id, + plan, + worklog, + goal, + } + } + + pub const fn request_id(&self) -> &RequestId { + &self.request_id + } + + pub fn host_identity(&self) -> &str { + &self.host_identity + } + + pub const fn session_id(&self) -> &SessionId { + &self.session_id + } + + pub fn plan(&self) -> &str { + &self.plan + } + + pub fn worklog(&self) -> &str { + &self.worklog + } + + pub fn goal(&self) -> &str { + &self.goal + } +} + +impl AuthenticatedResponseContract for TaskArtifactsResponse { + fn request_id(&self) -> &RequestId { + self.request_id() + } + + fn host_identity(&self) -> &str { + self.host_identity() + } +} + #[derive(Clone, Copy, Debug, Deserialize, Eq, PartialEq, Serialize)] #[serde(rename_all = "snake_case")] pub enum AdmissionCancellationOutcome { diff --git a/crates/satelle-transport/src/lib.rs b/crates/satelle-transport/src/lib.rs index 5c9dca18..bea47ce1 100644 --- a/crates/satelle-transport/src/lib.rs +++ b/crates/satelle-transport/src/lib.rs @@ -29,8 +29,8 @@ pub use contract::{ SetupRepairPlanAction, SetupRepairPlanRequest, SetupRepairPlanResponse, SetupRepairPostcondition, SetupRepairPreviousStatus, SetupRepairProbe, SetupRepairRunStatus, SetupVerificationRequest, SetupVerificationResponse, StopRequest, StopResponse, - SubscribeRequest, SubscribeRequestError, SubscribedResponse, TurnRequest, WsCloseReason, - WsControlError, WsServerControl, + SubscribeRequest, SubscribeRequestError, SubscribedResponse, TaskArtifactsResponse, + TurnRequest, WsCloseReason, WsControlError, WsServerControl, }; pub use server::{ DaemonServer, DaemonServerConfig, DaemonServerError, DaemonShutdownHandle, DaemonTlsConfig, diff --git a/crates/satelle-transport/src/server/mod.rs b/crates/satelle-transport/src/server/mod.rs index 1cb55235..817b971b 100644 --- a/crates/satelle-transport/src/server/mod.rs +++ b/crates/satelle-transport/src/server/mod.rs @@ -843,6 +843,10 @@ fn router(state: Arc) -> Router { .route("/v1/host/paths", get(host_paths)) .route("/v1/host/desktop-sessions", get(host_desktop_sessions)) .route("/v1/sessions/{session_id}", get(sessions::get_session)) + .route( + "/v1/sessions/{session_id}/task-artifacts", + get(sessions::get_task_artifacts), + ) .route("/v1/events", get(events::get_events)) .route_layer(middleware::from_fn_with_state( Arc::clone(&state), diff --git a/crates/satelle-transport/src/server/sessions.rs b/crates/satelle-transport/src/server/sessions.rs index bd0667e1..7057b395 100644 --- a/crates/satelle-transport/src/server/sessions.rs +++ b/crates/satelle-transport/src/server/sessions.rs @@ -3,7 +3,7 @@ use super::auth::AuthorizedRequest; use super::{ApiFailure, DaemonState, api_error_response, authenticated_json_response, host_error}; use crate::contract::{ AdmissionCancellationResponse, ApiErrorCategory, ApiErrorCode, RequestId, SessionResponse, - StopRequest, StopResponse, TurnRequest, TurnRequestParts, + StopRequest, StopResponse, TaskArtifactsResponse, TurnRequest, TurnRequestParts, }; use axum::extract::{Extension, FromRequestParts, Path, State}; use axum::http::request::Parts; @@ -191,6 +191,35 @@ pub(super) async fn get_session( ) } +pub(super) async fn get_task_artifacts( + State(state): State>, + Extension(authorized): Extension, + SessionPath(session_id): SessionPath, +) -> Response { + let service = Arc::clone(&state.service); + let artifacts = match host_call(&state, &authorized, move || { + service.task_artifacts(&session_id) + }) + .await + { + Ok(artifacts) => artifacts, + Err(response) => return response, + }; + authenticated_json_response( + StatusCode::OK, + &TaskArtifactsResponse::new( + authorized.request_id().clone(), + state.host_identity.clone(), + artifacts.session_id().clone(), + artifacts.plan().to_string(), + artifacts.worklog().to_string(), + artifacts.goal().to_string(), + ), + authorized.request_id(), + &state.host_identity, + ) +} + pub(super) async fn stop_session( State(state): State>, Extension(authorized): Extension, diff --git a/crates/satelle-transport/tests/http.rs b/crates/satelle-transport/tests/http.rs index f3c66f2d..2849724f 100644 --- a/crates/satelle-transport/tests/http.rs +++ b/crates/satelle-transport/tests/http.rs @@ -148,13 +148,13 @@ impl RunningServer { fn request(&self, path: &str) -> reqwest::RequestBuilder { self.protected_request(reqwest::Method::GET, path) - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") } fn mutation(&self, path: &str, idempotency_key: &str) -> reqwest::RequestBuilder { self.protected_request(reqwest::Method::POST, path) .header("Idempotency-Key", idempotency_key) - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") } fn mutation_with_request_id( @@ -165,7 +165,7 @@ impl RunningServer { ) -> reqwest::RequestBuilder { self.protected_request_with_request_id(reqwest::Method::POST, path, request_id) .header("Idempotency-Key", idempotency_key) - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") } fn protected_request(&self, method: reqwest::Method, path: &str) -> reqwest::RequestBuilder { @@ -371,7 +371,7 @@ fn setup_mutation_request( .header("Satelle-Expected-Host-Identity", host_identity) .header("Satelle-Request-Id", RequestId::new().to_string()) .header("Idempotency-Key", idempotency_key) - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") } fn replacement_token(token_id: &str) -> ApiBearerToken { @@ -1787,7 +1787,7 @@ async fn bootstrap_maintenance_routes_enforce_the_mutation_contract_before_ledge ), ( "missing-idempotency", - Some("13"), + Some("14"), None, false, false, @@ -1796,7 +1796,7 @@ async fn bootstrap_maintenance_routes_enforce_the_mutation_contract_before_ledge ), ( "query", - Some("13"), + Some("14"), Some("query-key"), true, false, @@ -1805,7 +1805,7 @@ async fn bootstrap_maintenance_routes_enforce_the_mutation_contract_before_ledge ), ( "cookie", - Some("13"), + Some("14"), Some("cookie-key"), false, true, @@ -1918,7 +1918,7 @@ async fn bootstrap_maintenance_routes_enforce_the_mutation_contract_before_ledge ), ( "missing-idempotency", - Some("13"), + Some("14"), None, false, false, @@ -1927,7 +1927,7 @@ async fn bootstrap_maintenance_routes_enforce_the_mutation_contract_before_ledge ), ( "query", - Some("13"), + Some("14"), Some("complete-query-key"), true, false, @@ -1936,7 +1936,7 @@ async fn bootstrap_maintenance_routes_enforce_the_mutation_contract_before_ledge ), ( "cookie", - Some("13"), + Some("14"), Some("complete-cookie-key"), false, true, diff --git a/crates/satelle-transport/tests/http/protocol.rs b/crates/satelle-transport/tests/http/protocol.rs index 4a82c6f4..4b4907e2 100644 --- a/crates/satelle-transport/tests/http/protocol.rs +++ b/crates/satelle-transport/tests/http/protocol.rs @@ -80,7 +80,7 @@ async fn protocol_version_gate_is_exact_sanitized_and_precedes_mutation_work() { // exposed. Send this case at the wire layer because reqwest deliberately // prevents callers from constructing padded header values. let whitespace_request = format!( - "POST /v1/sessions HTTP/1.1\r\nHost: localhost\r\nAuthorization: {}\r\nSatelle-Expected-Host-Identity: {}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 13 \r\nIdempotency-Key:\r\nContent-Length: 0\r\nConnection: close\r\n\r\n", + "POST /v1/sessions HTTP/1.1\r\nHost: localhost\r\nAuthorization: {}\r\nSatelle-Expected-Host-Identity: {}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 14 \r\nIdempotency-Key:\r\nContent-Length: 0\r\nConnection: close\r\n\r\n", bearer(&running.token), running.host_identity, RequestId::new(), @@ -149,12 +149,12 @@ async fn capabilities_handshake_rejects_old_and_missing_clients() { let current = running .protected_request(Method::GET, "/v1/capabilities") - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") .send() .await .expect("send capabilities request with current protocol version"); assert_eq!(current.status(), StatusCode::OK); - assert_eq!(current.headers()["satelle-protocol-version"], "13"); + assert_eq!(current.headers()["satelle-protocol-version"], "14"); } #[tokio::test] @@ -214,7 +214,7 @@ async fn assert_protocol_error( "details": { "daemon_version": env!("CARGO_PKG_VERSION"), "reason": reason, - "supported_versions": ["13"], + "supported_versions": ["14"], "received_version": received_version, }, "docs_url": null, diff --git a/crates/satelle-transport/tests/http/provider-auth.rs b/crates/satelle-transport/tests/http/provider-auth.rs index e7d2e442..ccd77376 100644 --- a/crates/satelle-transport/tests/http/provider-auth.rs +++ b/crates/satelle-transport/tests/http/provider-auth.rs @@ -486,7 +486,7 @@ async fn provider_binding_validation_requires_setup_or_control_authority() { let unauthenticated = reqwest::Client::new() .post(control.url(VALIDATION_PATH)) .header("Content-Type", "application/json") - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") .header("Satelle-Expected-Host-Identity", &control.host_identity) .header("Satelle-Request-Id", RequestId::new().as_str()) .header("Idempotency-Key", "provider-auth-unauthenticated") @@ -533,7 +533,7 @@ async fn bootstrap_admin_authorizes_and_control_validates_the_exact_path_aliases .header("Authorization", bearer(&bootstrap_token)) .header("Satelle-Expected-Host-Identity", &running.host_identity) .header("Satelle-Request-Id", RequestId::new().as_str()) - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") .header("Idempotency-Key", "provider-authorization-admin") .json(&authorization) .send() @@ -650,7 +650,7 @@ async fn validation_rejects_descriptor_material_and_control_cannot_authorize() { let forbidden = control .protected_request(reqwest::Method::PUT, AUTHORIZATION_PATH) .header("Idempotency-Key", "provider-authorization-control") - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") .json(&authorization) .send() .await @@ -670,7 +670,7 @@ fn bootstrap_mutation( .header("Authorization", bearer(token)) .header("Satelle-Expected-Host-Identity", &running.host_identity) .header("Satelle-Request-Id", RequestId::new().as_str()) - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") .header("Idempotency-Key", idempotency_key) } @@ -719,7 +719,7 @@ async fn provider_binding_mutations_require_admin() { let forbidden_delete = control .protected_request(reqwest::Method::DELETE, AUTHORIZATION_PATH) .header("Idempotency-Key", "provider-delete-control") - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") .send() .await .expect("send deletion as control principal"); @@ -736,7 +736,7 @@ async fn provider_binding_mutations_require_admin() { let authorized = ordinary_admin .protected_request(reqwest::Method::PUT, AUTHORIZATION_PATH) .header("Idempotency-Key", "provider-authorization-ordinary-admin") - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") .json(&authorization) .send() .await @@ -746,7 +746,7 @@ async fn provider_binding_mutations_require_admin() { let rejected_body = ordinary_admin .protected_request(reqwest::Method::DELETE, AUTHORIZATION_PATH) .header("Idempotency-Key", "provider-delete-body") - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") .json(&serde_json::json!({"unexpected": true})) .send() .await @@ -756,7 +756,7 @@ async fn provider_binding_mutations_require_admin() { let rejected_oversized_body = ordinary_admin .protected_request(reqwest::Method::DELETE, AUTHORIZATION_PATH) .header("Idempotency-Key", "provider-delete-oversized-body") - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") .body(vec![b'x'; 2 * 1024 * 1024]) .send() .await @@ -769,7 +769,7 @@ async fn provider_binding_mutations_require_admin() { let deleted = ordinary_admin .protected_request(reqwest::Method::DELETE, AUTHORIZATION_PATH) .header("Idempotency-Key", "provider-delete-body") - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") .send() .await .expect("reuse the rejected body idempotency key"); @@ -778,7 +778,7 @@ async fn provider_binding_mutations_require_admin() { let absent = ordinary_admin .protected_request(reqwest::Method::DELETE, AUTHORIZATION_PATH) .header("Idempotency-Key", "provider-delete-oversized-body") - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") .send() .await .expect("reuse the rejected oversized-body idempotency key"); @@ -975,7 +975,7 @@ async fn authorization_is_checked_before_the_durable_mutation_claim() { let rejected_before_body = running .protected_request(reqwest::Method::PUT, AUTHORIZATION_PATH) .header("Idempotency-Key", "provider-authority-before-body") - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") .body("{") .send() .await diff --git a/crates/satelle-transport/tests/http/raw-wire.rs b/crates/satelle-transport/tests/http/raw-wire.rs index e24ba217..46ce86b5 100644 --- a/crates/satelle-transport/tests/http/raw-wire.rs +++ b/crates/satelle-transport/tests/http/raw-wire.rs @@ -54,7 +54,7 @@ async fn provider_secret_raw_wire_rejects_identity_content_type_and_duplicate_me ), ] { let mut request = format!( - "POST /v1/setup/provider-secret HTTP/1.1\r\nHost: localhost\r\nAuthorization: {authorization}\r\nSatelle-Expected-Host-Identity: {expected_host}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 13\r\nIdempotency-Key: provider-secret-raw-wire\r\nContent-Type: {content_type}\r\nContent-Length: {}\r\n{metadata_headers}Connection: close\r\n\r\n", + "POST /v1/setup/provider-secret HTTP/1.1\r\nHost: localhost\r\nAuthorization: {authorization}\r\nSatelle-Expected-Host-Identity: {expected_host}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 14\r\nIdempotency-Key: provider-secret-raw-wire\r\nContent-Type: {content_type}\r\nContent-Length: {}\r\n{metadata_headers}Connection: close\r\n\r\n", RequestId::new(), secret_canary.len(), ) @@ -76,7 +76,7 @@ async fn chunked_oversize_body_returns_typed_413_without_admission() { ); let payload_bytes = body.len(); let head = format!( - "POST /v1/sessions HTTP/1.1\r\nHost: localhost\r\nAuthorization: {authorization}\r\nSatelle-Expected-Host-Identity: {}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 13\r\nIdempotency-Key: raw-chunked-limit\r\nContent-Type: application/json\r\nTransfer-Encoding: chunked\r\nConnection: close\r\n\r\n{payload_bytes:x}\r\n", + "POST /v1/sessions HTTP/1.1\r\nHost: localhost\r\nAuthorization: {authorization}\r\nSatelle-Expected-Host-Identity: {}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 14\r\nIdempotency-Key: raw-chunked-limit\r\nContent-Type: application/json\r\nTransfer-Encoding: chunked\r\nConnection: close\r\n\r\n{payload_bytes:x}\r\n", running.host_identity, RequestId::new(), ); @@ -160,7 +160,7 @@ async fn chunked_attachment_limit_and_log_privacy(trace_capture: TraceCapture) { let split = body.len() / 2; let request_id = RequestId::new(); let request_head = format!( - "POST /v1/sessions HTTP/1.1\r\nHost: localhost\r\nAuthorization: {authorization}\r\nSatelle-Expected-Host-Identity: {}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 13\r\nIdempotency-Key: attachment-limit-chunked\r\nContent-Type: application/json\r\nTransfer-Encoding: chunked\r\nConnection: close\r\n\r\n{split:x}\r\n", + "POST /v1/sessions HTTP/1.1\r\nHost: localhost\r\nAuthorization: {authorization}\r\nSatelle-Expected-Host-Identity: {}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 14\r\nIdempotency-Key: attachment-limit-chunked\r\nContent-Type: application/json\r\nTransfer-Encoding: chunked\r\nConnection: close\r\n\r\n{split:x}\r\n", running.host_identity, request_id, ); let mut request = request_head.into_bytes(); @@ -237,7 +237,7 @@ async fn bearer_tokens_in_http_trailers_are_rejected_without_admission() { let body = br#"{"schema_version":"satelle.api.v7","model_from_project":false,"provider_from_project":false,"prompt":"safe","execution_mode":"standard"}"#; let mutation_request = format!( - "POST /v1/sessions HTTP/1.1\r\nHost: localhost\r\nAuthorization: {}\r\nSatelle-Expected-Host-Identity: {}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 13\r\nIdempotency-Key: trailer-carrier\r\nContent-Type: application/json\r\nTransfer-Encoding: chunked\r\nTrailer: X-Api-Token\r\nConnection: close\r\n\r\n{:x}\r\n{}\r\n0\r\nX-Api-Token: {}\r\n\r\n", + "POST /v1/sessions HTTP/1.1\r\nHost: localhost\r\nAuthorization: {}\r\nSatelle-Expected-Host-Identity: {}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 14\r\nIdempotency-Key: trailer-carrier\r\nContent-Type: application/json\r\nTransfer-Encoding: chunked\r\nTrailer: X-Api-Token\r\nConnection: close\r\n\r\n{:x}\r\n{}\r\n0\r\nX-Api-Token: {}\r\n\r\n", bearer(&running.token), running.host_identity, RequestId::new(), @@ -258,7 +258,7 @@ async fn bearer_tokens_in_http_trailers_are_rejected_without_admission() { let admin = RunningServer::start(ApiScopes::ADMIN).await; let deletion_request = format!( - "DELETE /v1/setup/provider-bindings/openai/review HTTP/1.1\r\nHost: localhost\r\nAuthorization: {}\r\nSatelle-Expected-Host-Identity: {}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 13\r\nIdempotency-Key: provider-delete-trailer\r\nTransfer-Encoding: chunked\r\nTrailer: X-Api-Token\r\nConnection: close\r\n\r\n2\r\n{{}}\r\n0\r\nX-Api-Token: {}\r\n\r\n", + "DELETE /v1/setup/provider-bindings/openai/review HTTP/1.1\r\nHost: localhost\r\nAuthorization: {}\r\nSatelle-Expected-Host-Identity: {}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 14\r\nIdempotency-Key: provider-delete-trailer\r\nTransfer-Encoding: chunked\r\nTrailer: X-Api-Token\r\nConnection: close\r\n\r\n2\r\n{{}}\r\n0\r\nX-Api-Token: {}\r\n\r\n", bearer(&admin.token), admin.host_identity, RequestId::new(), @@ -272,7 +272,7 @@ async fn bearer_tokens_in_http_trailers_are_rejected_without_admission() { reqwest::Method::DELETE, "/v1/setup/provider-bindings/openai/review", ) - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") .header("Idempotency-Key", "provider-delete-trailer") .send() .await @@ -302,7 +302,7 @@ async fn stalled_upload_cannot_hold_daemon_shutdown_open_forever() { .await .expect("open stalled request connection"); let partial = format!( - "POST /v1/sessions HTTP/1.1\r\nHost: localhost\r\nAuthorization: {}\r\nSatelle-Expected-Host-Identity: {}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 13\r\nIdempotency-Key: stalled-shutdown\r\nContent-Type: application/json\r\nContent-Length: 1000\r\n\r\n{{", + "POST /v1/sessions HTTP/1.1\r\nHost: localhost\r\nAuthorization: {}\r\nSatelle-Expected-Host-Identity: {}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 14\r\nIdempotency-Key: stalled-shutdown\r\nContent-Type: application/json\r\nContent-Length: 1000\r\n\r\n{{", bearer(&token), initialized.host_identity(), RequestId::new(), @@ -362,7 +362,7 @@ async fn dropped_admission_response_is_recovered_without_stopping_or_duplicate_t let request = TurnRequest::new("PRIVATE_DROPPED_RESPONSE_CANARY"); let body = serde_json::to_vec(&request).expect("encode admission request"); let head = format!( - "POST /v1/sessions HTTP/1.1\r\nHost: localhost\r\nAuthorization: {}\r\nSatelle-Expected-Host-Identity: {}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 13\r\nIdempotency-Key: {IDEMPOTENCY_KEY}\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n", + "POST /v1/sessions HTTP/1.1\r\nHost: localhost\r\nAuthorization: {}\r\nSatelle-Expected-Host-Identity: {}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 14\r\nIdempotency-Key: {IDEMPOTENCY_KEY}\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n", bearer(&running.token), running.host_identity, RequestId::new(), @@ -674,7 +674,7 @@ async fn duplicate_header_case(header: DuplicateHeader, status: u16, code: &str) ), }; let mut request = format!( - "POST /v1/sessions HTTP/1.1\r\nHost: localhost\r\nSatelle-Expected-Host-Identity: {}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 13\r\nContent-Length: {}\r\n{duplicated}Connection: close\r\n\r\n", + "POST /v1/sessions HTTP/1.1\r\nHost: localhost\r\nSatelle-Expected-Host-Identity: {}\r\nSatelle-Request-Id: {}\r\nSatelle-Protocol-Version: 14\r\nContent-Length: {}\r\n{duplicated}Connection: close\r\n\r\n", running.host_identity, RequestId::new(), body.len(), diff --git a/crates/satelle-transport/tests/http/sessions.rs b/crates/satelle-transport/tests/http/sessions.rs index e1ec626a..9e3c44df 100644 --- a/crates/satelle-transport/tests/http/sessions.rs +++ b/crates/satelle-transport/tests/http/sessions.rs @@ -6,7 +6,8 @@ use satelle_core::session::{SessionActivity, TurnExecutionMode}; use satelle_test_contract::assert_privacy_canaries_absent; use satelle_transport::{ AdmissionCancellationOutcome, AdmissionCancellationResponse, ImageAttachment, - MAX_IMAGE_ATTACHMENT_BYTES, SessionResponse, StopRequest, StopResponse, TurnRequest, + MAX_IMAGE_ATTACHMENT_BYTES, SessionResponse, StopRequest, StopResponse, TaskArtifactsResponse, + TurnRequest, }; use sha2::Digest as _; @@ -344,6 +345,86 @@ async fn session_routes_complete_the_durable_reconnect_journey() { ); } +#[tokio::test] +async fn task_artifact_read_is_closed_redacted_and_identity_pinned() { + let running = RunningServer::start(ApiScopes::CONTROL).await; + let prompt_canary = "PRIVATE_TASK_ARTIFACT_HTTP_PROMPT_CANARY"; + let created: SessionResponse = running + .mutation("/v1/sessions", "task-artifact-export-session") + .json(&TurnRequest::new(prompt_canary)) + .send() + .await + .expect("create task artifact Session") + .json() + .await + .expect("decode task artifact Session"); + let session_id = created.session().session_id().clone(); + wait_until_idle(&running, session_id.as_str()).await; + let path = format!("/v1/sessions/{session_id}/task-artifacts"); + + let response = running + .request(&path) + .send() + .await + .expect("read task artifact response"); + assert_eq!(response.status(), StatusCode::OK); + let bytes = response.bytes().await.expect("read task artifact bytes"); + assert_privacy_canaries_absent("task artifact HTTP response", &bytes, &[prompt_canary]); + let json: Value = serde_json::from_slice(&bytes).expect("parse task artifact response"); + let keys = json + .as_object() + .expect("task artifact response is an object") + .keys() + .map(String::as_str) + .collect::>(); + assert_eq!( + keys, + std::collections::BTreeSet::from([ + "goal", + "host_identity", + "plan", + "request_id", + "schema_version", + "session_id", + "worklog", + ]) + ); + assert_eq!(json["schema_version"], "satelle.task_artifacts.v1"); + let artifacts: TaskArtifactsResponse = + serde_json::from_slice(&bytes).expect("decode task artifact contract"); + assert_eq!(artifacts.host_identity(), running.host_identity); + assert_eq!(artifacts.session_id(), &session_id); + assert!(artifacts.plan().starts_with("# Plan\n\n")); + assert!(artifacts.worklog().starts_with("# Worklog\n\n")); + assert!(artifacts.goal().contains("not recorded")); + + let mismatched = protected_at( + &reqwest::Client::new(), + Method::GET, + running.server.local_addr(), + &path, + &bearer(&running.token), + "unexpected-host", + ) + .send() + .await + .expect("reject mismatched task artifact Host identity"); + assert_api_error(mismatched, StatusCode::CONFLICT, "host-identity-mismatch").await; + + let diagnostics_only = RunningServer::start(ApiScopes::DIAGNOSTICS_SENSITIVE).await; + let forbidden = diagnostics_only + .request(&path) + .send() + .await + .expect("reject task artifact read without read scope"); + assert_api_error( + forbidden, + StatusCode::FORBIDDEN, + "authorization-insufficient-scope", + ) + .await; +} + #[tokio::test] async fn mutation_replays_preserve_operation_boundaries_and_reject_digest_drift() { let running = RunningServer::start(ApiScopes::CONTROL).await; @@ -1102,6 +1183,11 @@ async fn daemon_client_drives_the_complete_session_control_contract() { assert_eq!(steered.session().session_id(), &session_id); assert_eq!(steered.session().turns().len(), 2); wait_until_idle_with_client(&client, &session_id); + let artifacts = client + .read_task_artifacts(&session_id) + .expect("read task artifacts through DaemonClient"); + assert_eq!(artifacts.session_id(), &session_id); + assert!(artifacts.plan().contains("### Turn 2")); let stale = client .stop_session_for_turn(&session_id, &original_turn_id, STALE_STOP_KEY) @@ -1139,7 +1225,7 @@ async fn mutation_validation_fails_before_execution_with_typed_errors() { let missing_key = running .protected_request(Method::POST, "/v1/sessions") - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") .json(&TurnRequest::new("PRIVATE_MISSING_KEY_CANARY")) .send() .await @@ -1798,7 +1884,7 @@ fn protected_at( .header("Satelle-Expected-Host-Identity", host_identity) .header("Satelle-Request-Id", RequestId::new().to_string()); if is_mutation { - request.header("Satelle-Protocol-Version", "13") + request.header("Satelle-Protocol-Version", "14") } else { request } diff --git a/crates/satelle-transport/tests/http/setup-readiness.rs b/crates/satelle-transport/tests/http/setup-readiness.rs index b5d4ab06..54998e89 100644 --- a/crates/satelle-transport/tests/http/setup-readiness.rs +++ b/crates/satelle-transport/tests/http/setup-readiness.rs @@ -33,7 +33,7 @@ async fn setup_readiness_mutations_require_authenticated_control_scope() { let unauthenticated = reqwest::Client::new() .post(control.url(path)) .header("Content-Type", "application/json") - .header("Satelle-Protocol-Version", "13") + .header("Satelle-Protocol-Version", "14") .header("Satelle-Expected-Host-Identity", &control.host_identity) .header("Satelle-Request-Id", RequestId::new().as_str()) .header("Idempotency-Key", "setup-readiness-unauthenticated") diff --git a/docs/explanation/security-boundaries.mdx b/docs/explanation/security-boundaries.mdx index 0204f62b..219ff6a8 100644 --- a/docs/explanation/security-boundaries.mdx +++ b/docs/explanation/security-boundaries.mdx @@ -63,3 +63,20 @@ desktop content. Support bundle export is not implemented. The absence of that command is not permission to archive the Host state directory. Suspected vulnerabilities use the private reporting process in the repository security policy. + +## Retention has a defined ownership boundary + +Satelle retention deletes only Satelle-owned metadata and files. It does not +claim to delete Codex, model-provider, operating-system, backup, filesystem +snapshot, or other upstream records. Codex and model providers can apply their +own retention policies independently. + +Deletion is not cryptographic erasure. Deleted content can remain in SQLite +pages, filesystem snapshots, backups, or storage managed outside Satelle. + +Host storage stays local to the remote Host. An explicit Session artifact +export writes only the redacted plan, worklog, and Goal projection to the +owner-only operator-selected client path. Satelle MVP does not implement +diagnostic bundles, raw diagnostic exports, recordings, journald, Windows Event +Log, or macOS unified logging sinks. None of those later features is required +to preserve the authoritative SQLite log store. diff --git a/docs/reference/configuration.mdx b/docs/reference/configuration.mdx index 9a4a60a0..0a192867 100644 --- a/docs/reference/configuration.mdx +++ b/docs/reference/configuration.mdx @@ -29,6 +29,8 @@ daemon_idle_timeout = "10m" native_readiness_cache_ttl = "5m" provider_smoke_success_cache_ttl = "1440m" provider_smoke_failure_cache_ttl = "10m" +session_metadata_retention = "30d" +operator_log_retained_files = 12 yolo = false [hosts.local-demo.timeouts] @@ -40,7 +42,9 @@ Host fields include `transport`, `adapter`, `address`, `network`, `timeouts`, desktop selection, daemon path overrides, `setup_mode`, experimental-provider opt-in, daemon and readiness cache TTLs, Host-scoped `yolo`, project-selection permission, direct-transport identity and trust fields, and provider Secret -Source descriptors. +Source descriptors. `session_metadata_retention` accepts `h` or `d` values from +7 days through 365 days. `operator_log_retained_files` keeps 1 through 100 +rotated 10 MiB files and does not change SQLite log retention. The duration parser requires explicit units such as `500ms`, `30s`, or `2m`. Configuration does not perform environment-variable interpolation. @@ -60,6 +64,8 @@ provider_smoke_success_cache_ttl = "720m" provider_smoke_failure_cache_ttl = "5m" log_verbosity = "debug" trusted_profile = "maintenance" +session_metadata_retention = "30d" +operator_log_retained_files = 12 yolo = false [profiles.work.timeouts] @@ -78,6 +84,9 @@ implicitly by a same-name profile. Diagnostic verbosity accepts `off`, `info`, `debug`, or `trace`. Precedence is `--log-verbosity`, then `SATELLE_LOG`, then profile and user configuration. JSON command output never includes diagnostic logs. +User Host configuration sets the retention baseline. A selected user-owned +profile can override it. Project configuration cannot set destructive +retention. ## Project configuration @@ -98,7 +107,7 @@ native_readiness = "90s" The matching Host alias must already exist in user configuration and permit project selection. Project configuration cannot define addresses, TLS trust, tokens, desktop identity, daemon paths, Secret Sources, mutation consent, -Trusted Profiles, or YOLO enablement. +Trusted Profiles, destructive retention, or YOLO enablement. ## Inspect the result diff --git a/docs/reference/generated-cli.mdx b/docs/reference/generated-cli.mdx index bf23c6da..e26ff6e9 100644 --- a/docs/reference/generated-cli.mdx +++ b/docs/reference/generated-cli.mdx @@ -43,6 +43,7 @@ Commands: steer status stop + session logs mcp support @@ -439,6 +440,23 @@ Options: -h, --help Print help ``` +## `satelle session` + +```text +Usage: satelle session [OPTIONS] + +Commands: + export + help Print this message or the help of the given subcommand(s) + +Options: + --no-color Disable colored human output + --log-verbosity Set diagnostic log verbosity: off, info, debug, or trace [possible values: off, info, debug, trace] + --profile Apply a named user-level configuration profile + --error-format Format diagnostics as human-readable text or JSON [env: SATELLE_ERROR_FORMAT=] [possible values: human, json] + -h, --help Print help +``` + ## `satelle logs` ```text @@ -737,6 +755,24 @@ Options: -h, --help Print help ``` +## `satelle session export` + +```text +Usage: satelle session export [OPTIONS] --host --output + +Arguments: + + +Options: + --host + --no-color Disable colored human output + --log-verbosity Set diagnostic log verbosity: off, info, debug, or trace [possible values: off, info, debug, trace] + --output + --profile Apply a named user-level configuration profile + --error-format Format diagnostics as human-readable text or JSON [env: SATELLE_ERROR_FORMAT=] [possible values: human, json] + -h, --help Print help +``` + ## `satelle mcp serve` ```text