From 0fc5262c3f834b6106fa1ae731e435bc130b595c Mon Sep 17 00:00:00 2001 From: Seongjae Date: Thu, 24 Sep 2026 23:08:01 +0900 Subject: [PATCH 1/7] feat(install): identify artifacts and establish a bootstrap reference (DG1-C09) Install a packaged release as the current user's LaunchAgent, so the service runs a protected, identified artifact rather than a worktree build. The service keeps the same foreground `devguardd serve`. - scripts/package.py builds devguardd, devguard and devguard-launch in one release build of a clean tree and writes a manifest: release id, source commit/tree/tree digest, toolchain, each binary's SHA-256 and size, and the build's compiled compatibility (`devguardd version --json`: package, wire, protocol, capabilities, journal and configuration schemas). A package is functional, not SLO-qualified. - `devguardd install --package DIR` runs from the package itself (its executable must hash to the manifest's devguardd), validates exactly three matching regular binaries, and refuses while an authority serves or holds the lock, while the agent exists, or without the existing state; it never touches the journal. It freezes the release through a private sibling into a read-only releases/ (never overwritten), writes the plist (RunAtLoad, KeepAlive {Crashed}: a fail-closed exit is not restarted) and bootstraps it into gui/. - Selection only after verification: launchd's PID, the endpoint's handshake PID and that PID's executable image (proc_pidpath) must be the release's devguardd with the manifest's hash. Then selection.json records current and last-known-good and recovery/ keeps a copy; a failed verification boots the job out with nothing selected. - `devguardd status` reports the selection, launchd state, plist match and the running binary, exiting 0 only when healthy. devguard-macos gains executable_path (proc_pidpath) and core exposes JOURNAL_SCHEMA. The daemon depends on itself only to enable fixtures in its tests; validate.py allows exactly that. A fixture daemon can serve until SIGTERM as a service's program. Tests copy the test binary into packages so the installer truly runs from its package, with a fake manager and with launchd itself (a unique transient label, a crash restart onto the same journal, a SIGTERM stop, a fail-closed start not restarted). qualify.py gains dg1-bootstrap; CI runs it on macOS. Docs: contracts, operations, DG-1 ledger and verification (English authority, reviewed Korean), README and milestones. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 9 +- Cargo.lock | 1 + README.md | 5 +- crates/core/src/journal.rs | 12 +- crates/core/src/lib.rs | 1 + crates/daemon/Cargo.toml | 2 + crates/daemon/src/fixture.rs | 37 + crates/daemon/src/install.rs | 1003 +++++++++++++++++++++++++++ crates/daemon/src/lib.rs | 1 + crates/daemon/src/main.rs | 52 +- crates/daemon/src/paths.rs | 21 + crates/daemon/src/server.rs | 2 +- crates/daemon/tests/entrypoint.rs | 3 +- crates/daemon/tests/install.rs | 674 ++++++++++++++++++ crates/macos/src/lib.rs | 2 +- crates/macos/src/process.rs | 31 + docs/contracts.md | 44 +- docs/ko/contracts.md | 44 +- docs/ko/operations.md | 30 +- docs/ko/planning/milestones/DG-1.md | 6 +- docs/ko/planning/verification.md | 9 +- docs/milestones.md | 4 +- docs/operations.md | 30 +- docs/planning/milestones/DG-1.md | 6 +- docs/planning/verification.md | 9 +- docs/translations.json | 16 +- scripts/package.py | 89 +++ scripts/qualify.py | 9 +- scripts/validate.py | 8 +- 29 files changed, 2115 insertions(+), 45 deletions(-) create mode 100644 crates/daemon/src/install.rs create mode 100644 crates/daemon/tests/install.rs create mode 100644 scripts/package.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 23a7c8a..bfa2955 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -53,6 +53,11 @@ jobs: # Hosted runners' work capacity cannot fit one Cargo job (1 CPU and # 2 GiB), so real-build cases are recorded not run there; refusals still run. run: python3 scripts/qualify.py dg1-cargo --allow-incomplete --output target/qualification/dg1-cargo + - name: Native installation functional checks + if: runner.os == 'macOS' + # Transient launchd jobs need the user's gui domain; a runner without + # one records the launchd case not run, and the summary shows incomplete. + run: python3 scripts/qualify.py dg1-bootstrap --allow-incomplete --output target/qualification/dg1-bootstrap - name: Preserve qualification evidence if: always() uses: actions/upload-artifact@v6 @@ -67,9 +72,9 @@ jobs: import json, os from pathlib import Path with open(os.environ['GITHUB_STEP_SUMMARY'], 'a') as out: - for name in ['ci', 'dg1-authority', 'dg1-auth', 'dg1-probes', 'dg1-scopes', 'dg1-launch', 'dg1-reconcile', 'dg1-cli', 'dg1-cargo']: + for name in ['ci', 'dg1-authority', 'dg1-auth', 'dg1-probes', 'dg1-scopes', 'dg1-launch', 'dg1-reconcile', 'dg1-cli', 'dg1-cargo', 'dg1-bootstrap']: path = Path('target/qualification') / name / 'report.json' status = json.loads(path.read_text())['status'] if path.exists() else 'not_run' out.write(name + ': ' + status + '\n\n') - out.write('Contract regression, service/UDS, native host evidence, native scope, launch helper, reconciliation, command-line owner and Cargo adapter functional checks only; native suites run on macOS. Self-use and foreground SLO qualification are not run.\n') + out.write('Contract regression, service/UDS, native host evidence, native scope, launch helper, reconciliation, command-line owner, Cargo adapter and installation functional checks only; native suites run on macOS. Self-use and foreground SLO qualification are not run.\n') PY diff --git a/Cargo.lock b/Cargo.lock index 5f1f33b..fdac3ea 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -134,6 +134,7 @@ dependencies = [ "devguard-client", "devguard-contract", "devguard-core", + "devguard-daemon", "devguard-macos", "libc", "serde", diff --git a/README.md b/README.md index 45485ca..10df3d5 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ DevGuard centralizes resource admission for development workloads while preserving the resources needed to inspect and stop them. -The repository implements **DG-0 contracts and a durable authority core**, with DG-1 now in progress. C01 supplies canonical paths/bootstrap/storage checks; C02 supplies authenticated, bounded local communication with a small client and private credential-FD transfer; C03 supplies the native macOS boot clock, process identity and host pressure evidence that activate the service's journal; C04 supplies cooperative QoS/nice application with readback and observed process-group scope evidence; C05 supplies registration over the wire and the fenced `devguard-launch` helper; C06 supplies reconciliation, and with native evidence the service opens all three; C07 supplies the `devguard` command-line owner with doctor diagnostics, receipts and explicit bounded waits; C08 supplies the Cargo adapters, which fit compiler jobs to the reservation and share one jobserver. PR and post-merge main delivery evidence is tracked separately from implementation. Linux cgroups are not yet available. Use the operating guide for actual command availability; the design also contains future interfaces. +The repository implements **DG-0 contracts and a durable authority core**, with DG-1 now in progress. C01 supplies canonical paths/bootstrap/storage checks; C02 supplies authenticated, bounded local communication with a small client and private credential-FD transfer; C03 supplies the native macOS boot clock, process identity and host pressure evidence that activate the service's journal; C04 supplies cooperative QoS/nice application with readback and observed process-group scope evidence; C05 supplies registration over the wire and the fenced `devguard-launch` helper; C06 supplies reconciliation, and with native evidence the service opens all three; C07 supplies the `devguard` command-line owner with doctor diagnostics, receipts and explicit bounded waits; C08 supplies the Cargo adapters, which fit compiler jobs to the reservation and share one jobserver; C09 installs a packaged release as the current user's LaunchAgent, selected only after launchd is verified to run it. PR and post-merge main delivery evidence is tracked separately from implementation. Linux cgroups are not yet available. Use the operating guide for actual command availability; the design also contains future interfaces. - [Authoritative design reference](docs/design.md) · [Korean translation](docs/ko/design.md) - [Historical approved design (Korean, immutable)](docs/design.ko.md) @@ -26,11 +26,12 @@ python3 scripts/qualify.py dg1-launch --offline # macOS only python3 scripts/qualify.py dg1-reconcile --offline # macOS only python3 scripts/qualify.py dg1-cli --offline # macOS only python3 scripts/qualify.py dg1-cargo --offline # macOS only +python3 scripts/qualify.py dg1-bootstrap --offline # macOS only ``` Omit `--offline` when the locked crates have not been downloaded. Builds and tests use one Cargo job and one test thread by default. Results, source fingerprints and logs are written under `target/qualification/`. A newer compiler can be used with `--allow-toolchain-mismatch` for a supplemental check, which never counts as qualification for 1.95.0. -DG-0 tests use an explicitly fake OS backend. Their passing reports establish accounting, persistence and state-transition contracts. The additional C01–C08 suites check actual local storage, peer observations, credential transport, native macOS host evidence, cooperative scope evidence, the launch helper, reconciliation, the command-line owner and the Cargo adapters within bounded fixtures. They do not qualify Linux enforcement, browser responsiveness or self-governed execution; these remain `not_run`. +DG-0 tests use an explicitly fake OS backend. Their passing reports establish accounting, persistence and state-transition contracts. The additional C01–C09 suites check actual local storage, peer observations, credential transport, native macOS host evidence, cooperative scope evidence, the launch helper, reconciliation, the command-line owner, the Cargo adapters and installation within bounded fixtures. They do not qualify Linux enforcement, browser responsiveness or self-governed execution; these remain `not_run`. ## Repository boundaries diff --git a/crates/core/src/journal.rs b/crates/core/src/journal.rs index 1445eca..11f4add 100644 --- a/crates/core/src/journal.rs +++ b/crates/core/src/journal.rs @@ -7,6 +7,10 @@ use std::os::unix::fs::OpenOptionsExt; use std::path::Path; use std::time::Duration; +/// The only journal schema this build reads and writes. A release manifest +/// states it, so an incompatible downgrade can be refused before a switch. +pub const JOURNAL_SCHEMA: &str = "1"; + pub(crate) struct Journal { connection: Connection, } @@ -41,9 +45,9 @@ impl Journal { ) })?; let connection = Self::connection(path)?; - connection.execute_batch("BEGIN IMMEDIATE; + connection.execute_batch(&format!("BEGIN IMMEDIATE; CREATE TABLE metadata (name TEXT PRIMARY KEY, value TEXT NOT NULL); - INSERT INTO metadata VALUES ('schema', '1'); + INSERT INTO metadata VALUES ('schema', '{JOURNAL_SCHEMA}'); CREATE TABLE instances ( consumer TEXT NOT NULL, generation TEXT NOT NULL, instance TEXT NOT NULL, identity TEXT NOT NULL, state TEXT NOT NULL CHECK (state IN ('active','suspect','retired')), @@ -58,7 +62,7 @@ impl Journal { CREATE TABLE retired_generations ( consumer TEXT NOT NULL, generation TEXT NOT NULL, PRIMARY KEY (consumer, generation)); - COMMIT;").map_err(db_error)?; + COMMIT;")).map_err(db_error)?; Ok(()) } @@ -95,7 +99,7 @@ impl Journal { r.get(0) }) .map_err(db_error)?; - if schema != "1" { + if schema != JOURNAL_SCHEMA { return Err(Error::new( ErrorCode::JournalInvalid, "unsupported journal schema", diff --git a/crates/core/src/lib.rs b/crates/core/src/lib.rs index c9cfa40..87954e8 100644 --- a/crates/core/src/lib.rs +++ b/crates/core/src/lib.rs @@ -10,6 +10,7 @@ pub use authority::{ Authority, AuthorityStorage, InstanceRecord, LaunchDecision, Principal, Registration, RunDecision, TrustedPeer, }; +pub use journal::JOURNAL_SCHEMA; pub use policy::{ConsumerDefinition, ConsumerRole, Policy}; pub use pressure::{ DiskObservation, MemoryPressure, PressureController, PressureSample, PressureState, diff --git a/crates/daemon/Cargo.toml b/crates/daemon/Cargo.toml index 3969c1f..e83dea4 100644 --- a/crates/daemon/Cargo.toml +++ b/crates/daemon/Cargo.toml @@ -28,4 +28,6 @@ toml.workspace = true uuid.workspace = true [dev-dependencies] +# The installer's integration tests serve isolated fixture authorities. +devguard-daemon = { path = ".", features = ["test-fixtures"] } tempfile.workspace = true diff --git a/crates/daemon/src/fixture.rs b/crates/daemon/src/fixture.rs index 15aaa2b..59c1081 100644 --- a/crates/daemon/src/fixture.rs +++ b/crates/daemon/src/fixture.rs @@ -242,6 +242,43 @@ impl TestAuthority { } } +/// Serve the fixture authority under `base` in this process until SIGTERM or +/// SIGINT, as `devguardd serve` serves the canonical one. Tests run it as the +/// program of a service, under launchd or a fake manager. +pub fn serve_until_terminated(base: &Path) -> Result<()> { + static STOP: AtomicBool = AtomicBool::new(false); + extern "C" fn stop(_: libc::c_int) { + STOP.store(true, Ordering::Relaxed); + } + let server = Server::open_with( + &AuthorityPaths::fixture(base), + Options { + probe: Some(Box::new(HealthyProbe)), + reconcile_paused: None, + }, + )?; + // SAFETY: the handler only stores to a lock-free atomic. + unsafe { + libc::signal(libc::SIGTERM, stop as *const () as libc::sighandler_t); + libc::signal(libc::SIGINT, stop as *const () as libc::sighandler_t); + } + let flag = Arc::new(AtomicBool::new(false)); + let watched = flag.clone(); + let watcher = std::thread::spawn(move || { + while !watched.load(Ordering::Relaxed) { + if STOP.load(Ordering::Relaxed) { + watched.store(true, Ordering::Relaxed); + break; + } + std::thread::sleep(Duration::from_millis(10)); + } + }); + let result = server.run(flag.clone()); + flag.store(true, Ordering::Relaxed); + let _ = watcher.join(); + result +} + impl Drop for TestAuthority { fn drop(&mut self) { let _ = self.shutdown(); diff --git a/crates/daemon/src/install.rs b/crates/daemon/src/install.rs new file mode 100644 index 0000000..c106217 --- /dev/null +++ b/crates/daemon/src/install.rs @@ -0,0 +1,1003 @@ +//! Protected installation of a release as the current user's LaunchAgent. +//! +//! A package is a directory holding the three binaries and a manifest of their +//! hashes and compiled compatibility. The installer runs from the package's +//! own `devguardd`, so binaries from different builds are never mixed. It +//! copies the release into an immutable directory under the canonical root, +//! starts it through launchd with the same foreground `serve` entrypoint, and +//! only after verifying the running binary records it as current and last +//! known good and keeps a recovery copy. It never touches the journal: a +//! service that cannot open or reconcile its state fails closed, and launchd +//! restarts it only after a crash. + +use crate::paths::{secure_directory, AuthorityPaths}; +use devguard_client::protocol::{Hello, WIRE_VERSION}; +use devguard_client::Client; +use devguard_contract::{ + digest_bytes, Capability, Compatibility, Error, ErrorCode, Result, PROTOCOL_VERSION, +}; +use serde::{Deserialize, Serialize}; +use std::collections::{BTreeMap, BTreeSet}; +use std::fs::{self, File, OpenOptions}; +use std::io::{Read, Write}; +use std::os::unix::fs::{DirBuilderExt, OpenOptionsExt, PermissionsExt}; +use std::path::{Path, PathBuf}; +use std::time::{Duration, Instant}; + +/// The current user's agent. A reverse-DNS name of the project's repository host. +pub const LABEL: &str = "io.github.novelkr.devguard"; +pub const MANIFEST_SCHEMA: &str = "devguard-release-manifest/v1"; +pub const SELECTION_SCHEMA: &str = "devguard-release-selection/v1"; +/// The binaries a release consists of, all from one build. +pub const ARTIFACTS: [&str; 3] = ["devguardd", "devguard", "devguard-launch"]; +const MANIFEST_FILE: &str = "MANIFEST.json"; +const MANIFEST_BYTES: u64 = 64 * 1024; +const SELECTION_BYTES: usize = 256 * 1024; +const ARTIFACT_BYTES: u64 = 256 * 1024 * 1024; +const LAUNCHCTL: &str = "/bin/launchctl"; + +fn invalid(message: impl Into) -> Error { + Error::new(ErrorCode::InvalidRequest, message) +} + +fn unavailable(message: impl Into) -> Error { + Error::new(ErrorCode::ResourceControlUnavailable, message) +} + +/// What a build can serve and read, compiled in. A package states it, and the +/// installer refuses a package whose values differ from its own. +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +#[serde(deny_unknown_fields)] +pub struct BuildCompatibility { + pub package_version: String, + pub wire_version: u32, + pub protocol: u32, + pub capabilities: BTreeSet, + pub journal_schema: String, + pub config_schema: u32, +} + +/// This build's compatibility. +pub fn compiled() -> BuildCompatibility { + BuildCompatibility { + package_version: env!("CARGO_PKG_VERSION").into(), + wire_version: WIRE_VERSION, + protocol: PROTOCOL_VERSION, + capabilities: crate::server::launch_capabilities(), + journal_schema: devguard_core::JOURNAL_SCHEMA.into(), + config_schema: crate::config::CONFIG_SCHEMA, + } +} + +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +#[serde(deny_unknown_fields)] +pub struct Artifact { + pub sha256: String, + pub bytes: u64, +} + +/// A release's manifest. `source` and `build` are provenance written by the +/// packaging tool; the installer records them without interpreting them. +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)] +#[serde(deny_unknown_fields)] +pub struct Manifest { + pub schema: String, + pub release_id: String, + pub version: String, + pub source: serde_json::Value, + pub build: serde_json::Value, + pub artifacts: BTreeMap, + pub compatibility: BuildCompatibility, + /// `functional`: tested functionally, not measured for the SLO. + pub scope: String, + pub slo_qualified: bool, + pub created_at: String, +} + +/// A validated package. +#[derive(Debug, Clone)] +pub struct Package { + pub dir: PathBuf, + pub manifest: Manifest, + /// The digest of the manifest's exact bytes; it identifies the release. + pub manifest_sha256: String, +} + +fn valid_release_id(id: &str) -> bool { + !id.is_empty() + && id.len() <= 128 + && !id.starts_with('.') + && id + .bytes() + .all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'_' | b'-')) +} + +/// Read a regular file that is not a symlink, bounded by `limit`. +fn read_regular(path: &Path, limit: u64) -> Result> { + let meta = fs::symlink_metadata(path) + .map_err(|_| invalid(format!("{} is missing", path.display())))?; + if !meta.is_file() || meta.file_type().is_symlink() { + return Err(invalid(format!("{} is not a regular file", path.display()))); + } + let file = OpenOptions::new() + .read(true) + .custom_flags(libc::O_NOFOLLOW | libc::O_CLOEXEC) + .open(path) + .map_err(|_| invalid(format!("cannot read {}", path.display())))?; + let mut bytes = Vec::new(); + file.take(limit + 1) + .read_to_end(&mut bytes) + .map_err(|_| invalid(format!("cannot read {}", path.display())))?; + if bytes.len() as u64 > limit { + return Err(invalid(format!( + "{} exceeds its size limit", + path.display() + ))); + } + Ok(bytes) +} + +fn digest_file(path: &Path) -> Result { + Ok(digest_bytes(&read_regular(path, ARTIFACT_BYTES)?)) +} + +fn directory_entries(path: &Path) -> Result> { + let meta = fs::symlink_metadata(path) + .map_err(|_| invalid(format!("{} is missing", path.display())))?; + if !meta.is_dir() || meta.file_type().is_symlink() { + return Err(invalid(format!("{} is not a directory", path.display()))); + } + fs::read_dir(path) + .map_err(|_| invalid(format!("cannot list {}", path.display())))? + .map(|entry| { + entry + .map_err(|_| invalid("cannot list a package directory"))? + .file_name() + .into_string() + .map_err(|_| invalid("package entries must be UTF-8")) + }) + .collect() +} + +/// Check a package or an installed release: exactly the manifest and the three +/// binaries, each a regular executable file whose size and hash match, and a +/// manifest this build can install. +pub fn validate_package(dir: &Path) -> Result { + let entries = directory_entries(dir)?; + if entries != BTreeSet::from([MANIFEST_FILE.to_string(), "bin".to_string()]) { + return Err(invalid("a package holds exactly MANIFEST.json and bin/")); + } + let bytes = read_regular(&dir.join(MANIFEST_FILE), MANIFEST_BYTES)?; + let manifest: Manifest = serde_json::from_slice(&bytes) + .map_err(|_| invalid("the package manifest is not a valid release manifest"))?; + if manifest.schema != MANIFEST_SCHEMA { + return Err(invalid("unsupported release manifest schema")); + } + if !valid_release_id(&manifest.release_id) { + return Err(invalid( + "the release id may use only letters, digits, '.', '_' and '-'", + )); + } + let expected = compiled(); + if manifest.compatibility != expected || manifest.version != expected.package_version { + return Err(Error::new( + ErrorCode::ResourcePolicyUnsupported, + "the package's compatibility differs from this installer's build", + )); + } + if !matches!( + (manifest.scope.as_str(), manifest.slo_qualified), + ("functional", false) | ("slo-qualified", true) + ) { + return Err(invalid("the manifest scope and SLO qualification disagree")); + } + let names: BTreeSet<&str> = manifest.artifacts.keys().map(String::as_str).collect(); + if names != BTreeSet::from(ARTIFACTS) { + return Err(invalid( + "the manifest must list exactly devguardd, devguard and devguard-launch", + )); + } + let bin = dir.join("bin"); + if directory_entries(&bin)? != ARTIFACTS.iter().map(|name| name.to_string()).collect() { + return Err(invalid("bin/ holds exactly the three release binaries")); + } + for (name, artifact) in &manifest.artifacts { + let path = bin.join(name); + let meta = + fs::symlink_metadata(&path).map_err(|_| invalid("a release binary is missing"))?; + if !meta.is_file() || meta.permissions().mode() & 0o100 == 0 { + return Err(invalid(format!("{name} is not an executable regular file"))); + } + if meta.len() != artifact.bytes || digest_file(&path)? != artifact.sha256 { + return Err(invalid(format!("{name} does not match its manifest hash"))); + } + } + Ok(Package { + dir: dir.to_path_buf(), + manifest, + manifest_sha256: digest_bytes(&bytes), + }) +} + +/// How the service is registered with launchd. The installed service always +/// uses [`InstallOptions::canonical`]; tests use a unique label and paths. +#[derive(Debug, Clone)] +pub struct InstallOptions { + pub label: String, + pub plist: PathBuf, + pub log: PathBuf, + /// Arguments after the release's `devguardd`: `serve` for the service. + pub program_args: Vec, + pub environment: BTreeMap, + /// How long the started service may take to prove it is the release. + pub start_deadline: Duration, + /// Seconds launchd waits before restarting a crashed service. + pub throttle_seconds: u32, +} + +impl InstallOptions { + pub fn canonical(paths: &AuthorityPaths) -> Self { + Self { + label: LABEL.into(), + plist: paths.launch_agents().join(format!("{LABEL}.plist")), + log: paths.logs().join("devguardd.log"), + program_args: vec!["serve".into()], + environment: BTreeMap::new(), + start_deadline: Duration::from_secs(15), + throttle_seconds: 10, + } + } +} + +/// What launchd is asked to run. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ServiceSpec { + pub label: String, + pub plist: PathBuf, + pub program: PathBuf, + pub args: Vec, + pub environment: BTreeMap, +} + +/// The service's state as launchd reports it. +#[derive(Debug, Clone, Default, Serialize, PartialEq, Eq)] +pub struct ServiceState { + pub running: bool, + pub pid: Option, + pub runs: Option, + pub last_exit: Option, +} + +/// The seam to launchd. [`Launchctl`] is the real one. +pub trait ServiceManager { + fn bootstrap(&self, spec: &ServiceSpec) -> Result<()>; + fn bootout(&self, label: &str) -> Result<()>; + /// `None` when no job with the label is loaded. + fn state(&self, label: &str) -> Result>; +} + +/// The current user's GUI launchd domain. +pub struct Launchctl { + domain: String, +} + +impl Launchctl { + pub fn new(uid: u32) -> Self { + Self { + domain: format!("gui/{uid}"), + } + } + + fn run(&self, args: &[&str]) -> Result { + std::process::Command::new(LAUNCHCTL) + .args(args) + .output() + .map_err(|_| unavailable("cannot run launchctl")) + } +} + +/// launchctl's exit status for a service it cannot find. +const LAUNCHCTL_NO_SUCH_SERVICE: i32 = 113; + +impl ServiceManager for Launchctl { + fn bootstrap(&self, spec: &ServiceSpec) -> Result<()> { + let plist = spec + .plist + .to_str() + .ok_or_else(|| invalid("the plist path must be UTF-8"))?; + let output = self.run(&["bootstrap", &self.domain, plist])?; + if !output.status.success() { + return Err(unavailable(format!( + "launchctl bootstrap failed: {}", + String::from_utf8_lossy(&output.stderr).trim() + ))); + } + Ok(()) + } + + fn bootout(&self, label: &str) -> Result<()> { + let output = self.run(&["bootout", &format!("{}/{label}", self.domain)])?; + if !output.status.success() && output.status.code() != Some(LAUNCHCTL_NO_SUCH_SERVICE) { + return Err(unavailable(format!( + "launchctl bootout failed: {}", + String::from_utf8_lossy(&output.stderr).trim() + ))); + } + Ok(()) + } + + fn state(&self, label: &str) -> Result> { + let output = self.run(&["print", &format!("{}/{label}", self.domain)])?; + if output.status.code() == Some(LAUNCHCTL_NO_SUCH_SERVICE) { + return Ok(None); + } + if !output.status.success() { + return Err(unavailable("launchctl print failed")); + } + Ok(Some(parse_print(&String::from_utf8_lossy(&output.stdout)))) + } +} + +/// Read the service's own fields from `launchctl print`: only lines indented +/// by exactly one tab belong to the service rather than a nested section. +pub fn parse_print(text: &str) -> ServiceState { + let mut state = ServiceState::default(); + for line in text.lines() { + let Some(line) = line.strip_prefix('\t') else { + continue; + }; + if line.starts_with('\t') { + continue; + } + let Some((key, value)) = line.split_once(" = ") else { + continue; + }; + match key { + "state" => state.running = value == "running", + "pid" => state.pid = value.parse().ok(), + "runs" => state.runs = value.parse().ok(), + "last exit code" | "last terminating signal" => { + state.last_exit = Some(format!("{key} = {value}")) + } + _ => {} + } + } + if !state.running { + state.pid = None; + } + state +} + +fn xml_escape(text: &str) -> String { + text.replace('&', "&") + .replace('<', "<") + .replace('>', ">") + .replace('"', """) +} + +/// The agent's property list. A crash restarts the service; a clean or failing +/// exit, such as a fail-closed start on a missing journal, does not. +pub fn render_plist(spec: &ServiceSpec, log: &Path, throttle_seconds: u32) -> Result { + let text = |path: &Path| { + path.to_str() + .map(xml_escape) + .ok_or_else(|| invalid("service paths must be UTF-8")) + }; + let mut arguments = format!(" {}\n", text(&spec.program)?); + for arg in &spec.args { + arguments.push_str(&format!(" {}\n", xml_escape(arg))); + } + let mut environment = String::new(); + if !spec.environment.is_empty() { + environment.push_str(" EnvironmentVariables\n \n"); + for (name, value) in &spec.environment { + environment.push_str(&format!( + " {}\n {}\n", + xml_escape(name), + xml_escape(value) + )); + } + environment.push_str(" \n"); + } + let log = text(log)?; + Ok(format!( + " + + + + Label + {label} + ProgramArguments + +{arguments} +{environment} RunAtLoad + + KeepAlive + + Crashed + + + ThrottleInterval + {throttle_seconds} + ProcessType + Standard + StandardOutPath + {log} + StandardErrorPath + {log} + + +", + label = xml_escape(&spec.label), + )) +} + +/// Which release the service runs and which last ran verified. +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +#[serde(deny_unknown_fields)] +pub struct Selection { + pub schema: String, + pub current: String, + pub last_known_good: Option, + pub history: Vec, +} + +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +#[serde(deny_unknown_fields)] +pub struct SelectionEvent { + pub release_id: String, + pub event: String, + pub unix_ms: u128, +} + +fn unix_ms() -> u128 { + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|since| since.as_millis()) + .unwrap_or(0) +} + +pub fn read_selection(paths: &AuthorityPaths) -> Result> { + let path = paths.selection(); + if fs::symlink_metadata(&path).is_err() { + return Ok(None); + } + let bytes = crate::paths::read_private(&path, paths.uid(), SELECTION_BYTES)?; + let selection: Selection = serde_json::from_slice(&bytes).map_err(|_| { + Error::new( + ErrorCode::JournalInvalid, + "the release selection is not valid", + ) + })?; + if selection.schema != SELECTION_SCHEMA { + return Err(Error::new( + ErrorCode::JournalInvalid, + "unsupported release selection schema", + )); + } + Ok(Some(selection)) +} + +/// Replace a private file atomically: write a new sibling, sync it, rename it. +fn replace_private(path: &Path, bytes: &[u8]) -> Result<()> { + let parent = path + .parent() + .ok_or_else(|| invalid("a private file needs a directory"))?; + let temporary = parent.join(format!( + ".{}.{}", + path.file_name().and_then(|n| n.to_str()).unwrap_or("file"), + uuid::Uuid::new_v4().simple() + )); + let written = (|| -> std::io::Result<()> { + let mut file = OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .custom_flags(libc::O_NOFOLLOW | libc::O_CLOEXEC) + .open(&temporary)?; + file.write_all(bytes)?; + file.sync_all()?; + fs::rename(&temporary, path)?; + File::open(parent)?.sync_all() + })(); + if written.is_err() { + let _ = fs::remove_file(&temporary); + return Err(unavailable(format!("cannot write {}", path.display()))); + } + Ok(()) +} + +fn write_selection(paths: &AuthorityPaths, selection: &Selection) -> Result<()> { + let bytes = + serde_json::to_vec_pretty(selection).map_err(|_| invalid("selection encoding failed"))?; + replace_private(&paths.selection(), &bytes) +} + +/// Copy `from` (a validated package or release) into `destination` as an +/// immutable release: the copy is made in a private sibling, synced, made +/// read-only and renamed into place, so a partial copy never has the final +/// name. An existing destination is reused only when its manifest is identical. +fn freeze_copy(from: &Package, destination: &Path) -> Result { + if fs::symlink_metadata(destination).is_ok() { + let existing = validate_package(destination)?; + if existing.manifest_sha256 != from.manifest_sha256 { + return Err(invalid(format!( + "{} holds a different release; releases are never overwritten", + destination.display() + ))); + } + return Ok(true); + } + let parent = destination + .parent() + .ok_or_else(|| invalid("a release needs a parent"))?; + let incoming = parent.join(format!( + ".incoming-{}-{}", + std::process::id(), + uuid::Uuid::new_v4().simple() + )); + let copied = (|| -> std::io::Result<()> { + fs::DirBuilder::new().mode(0o700).create(&incoming)?; + fs::DirBuilder::new() + .mode(0o700) + .create(incoming.join("bin"))?; + for name in ARTIFACTS { + let target = incoming.join("bin").join(name); + fs::copy(from.dir.join("bin").join(name), &target)?; + fs::set_permissions(&target, fs::Permissions::from_mode(0o500))?; + File::open(&target)?.sync_all()?; + } + let manifest = incoming.join(MANIFEST_FILE); + fs::copy(from.dir.join(MANIFEST_FILE), &manifest)?; + fs::set_permissions(&manifest, fs::Permissions::from_mode(0o400))?; + File::open(&manifest)?.sync_all()?; + File::open(incoming.join("bin"))?.sync_all()?; + fs::set_permissions(incoming.join("bin"), fs::Permissions::from_mode(0o500))?; + Ok(()) + })(); + if copied.is_err() { + remove_incoming(&incoming); + return Err(unavailable(format!( + "cannot copy the release into {}", + parent.display() + ))); + } + let copy = validate_package(&incoming); + let copy = match copy { + Ok(copy) if copy.manifest_sha256 == from.manifest_sha256 => copy, + _ => { + remove_incoming(&incoming); + return Err(unavailable("the copied release does not match its package")); + } + }; + drop(copy); + if fs::rename(&incoming, destination).is_err() { + remove_incoming(&incoming); + // Another installer may have frozen the same release first. + let existing = validate_package(destination)?; + if existing.manifest_sha256 != from.manifest_sha256 { + return Err(invalid("a different release took this release's name")); + } + return Ok(true); + } + let _ = fs::set_permissions(destination, fs::Permissions::from_mode(0o500)); + let _ = File::open(parent).and_then(|dir| dir.sync_all()); + Ok(false) +} + +/// Remove an incoming copy, which may already be partly read-only. +fn remove_incoming(incoming: &Path) { + let _ = fs::set_permissions(incoming, fs::Permissions::from_mode(0o700)); + let _ = fs::set_permissions(incoming.join("bin"), fs::Permissions::from_mode(0o700)); + let _ = fs::remove_dir_all(incoming); +} + +/// Remove incoming copies left by installers that are no longer running. +fn sweep_incoming(parent: &Path) { + let Ok(entries) = fs::read_dir(parent) else { + return; + }; + for entry in entries.flatten() { + let name = entry.file_name().to_string_lossy().into_owned(); + let Some(rest) = name.strip_prefix(".incoming-") else { + continue; + }; + let pid: Option = rest.split('-').next().and_then(|pid| pid.parse().ok()); + // SAFETY: signal 0 only checks whether the process exists. + let alive = pid.is_some_and(|pid| unsafe { libc::kill(pid, 0) } == 0); + if !alive { + remove_incoming(&entry.path()); + } + } +} + +/// A handshake that demands nothing, to learn who serves the socket. +fn handshake(paths: &AuthorityPaths) -> Result { + let client = Client::connect( + &paths.socket(), + paths.uid(), + Compatibility { + minimum_protocol: PROTOCOL_VERSION, + maximum_protocol: PROTOCOL_VERSION, + required: BTreeSet::new(), + }, + )?; + Ok(client.hello.clone()) +} + +/// What the installed service was observed running. +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +pub struct RunningEvidence { + pub service_pid: u32, + pub authority_pid: u32, + pub executable: PathBuf, + pub sha256: String, + pub capabilities: BTreeSet, +} + +/// Prove that the service launchd runs is `release`'s `devguardd`, serving +/// the canonical endpoint. +fn observe_running( + paths: &AuthorityPaths, + manager: &dyn ServiceManager, + label: &str, + release: &Package, +) -> Result { + let state = manager + .state(label)? + .ok_or_else(|| unavailable("the service is not loaded"))?; + let service_pid = state + .pid + .filter(|_| state.running) + .ok_or_else(|| unavailable("the service is not running"))?; + let hello = handshake(paths)?; + if hello.authority.pid != service_pid { + return Err(unavailable("another process serves the authority endpoint")); + } + let executable = devguard_macos::executable_path(service_pid)? + .ok_or_else(|| unavailable("the service ended during verification"))?; + let expected = release.dir.join("bin/devguardd"); + let same = fs::canonicalize(&executable).ok() == fs::canonicalize(&expected).ok(); + let sha256 = digest_file(&executable)?; + if !same || sha256 != release.manifest.artifacts["devguardd"].sha256 { + return Err(unavailable( + "the running service is not the installed release", + )); + } + Ok(RunningEvidence { + service_pid, + authority_pid: hello.authority.pid, + executable, + sha256, + capabilities: hello.capabilities, + }) +} + +fn wait_running( + paths: &AuthorityPaths, + manager: &dyn ServiceManager, + options: &InstallOptions, + release: &Package, +) -> Result { + let deadline = Instant::now() + options.start_deadline; + loop { + match observe_running(paths, manager, &options.label, release) { + Ok(evidence) => return Ok(evidence), + Err(error) if Instant::now() >= deadline => return Err(error), + Err(_) => std::thread::sleep(Duration::from_millis(100)), + } + } +} + +#[derive(Debug, Clone, Serialize)] +pub struct InstallReport { + pub release_id: String, + pub release: PathBuf, + pub manifest_sha256: String, + /// An identical release was already installed and was reused. + pub reused_release: bool, + pub label: String, + pub plist: PathBuf, + pub running: RunningEvidence, + pub recovery: PathBuf, + pub selection: Selection, +} + +fn ensure_plain_directory(path: &Path, mode: u32) -> Result<()> { + match fs::symlink_metadata(path) { + Ok(meta) if meta.is_dir() && !meta.file_type().is_symlink() => Ok(()), + Ok(_) => Err(invalid(format!("{} is not a directory", path.display()))), + Err(_) => fs::DirBuilder::new() + .recursive(true) + .mode(mode) + .create(path) + .map_err(|_| unavailable(format!("cannot create {}", path.display()))), + } +} + +/// Install `package` as the service: refuse while another authority or an +/// agent exists, freeze the release, start it through `manager`, verify the +/// running binary, then record the selection and a recovery copy. +pub fn install( + paths: &AuthorityPaths, + package: &Path, + manager: &dyn ServiceManager, + options: &InstallOptions, +) -> Result { + if !cfg!(target_os = "macos") { + return Err(Error::new( + ErrorCode::ResourcePolicyUnsupported, + "the service is installed as a macOS LaunchAgent", + )); + } + let package = validate_package(package)?; + let running = + std::env::current_exe().map_err(|_| unavailable("cannot locate the running installer"))?; + if digest_file(&running)? != package.manifest.artifacts["devguardd"].sha256 { + return Err(invalid( + "run the installer from the package it installs: its devguardd must be this binary", + )); + } + // The service needs the existing state; installation never creates or repairs it. + paths.validate_existing()?; + if handshake(paths).is_ok() { + return Err(Error::new( + ErrorCode::ResourceUnavailable, + "an authority is serving; stop it before installing the service", + )); + } + drop( + devguard_core::AuthorityStorage::open(&paths.journal()).map_err(|_| { + Error::new( + ErrorCode::ResourceUnavailable, + "the authority state is held or invalid; installation needs it idle and valid", + ) + })?, + ); + if manager.state(&options.label)?.is_some() || fs::symlink_metadata(&options.plist).is_ok() { + return Err(Error::new( + ErrorCode::ResourceUnavailable, + "the service is already installed; replacing it is an upgrade", + )); + } + secure_directory(&paths.releases(), paths.uid(), true)?; + secure_directory(&paths.recovery(), paths.uid(), true)?; + let log_directory = options + .log + .parent() + .ok_or_else(|| invalid("the log needs a directory"))?; + secure_directory(log_directory, paths.uid(), true)?; + let agents = options + .plist + .parent() + .ok_or_else(|| invalid("the plist needs a directory"))?; + ensure_plain_directory(agents, 0o755)?; + sweep_incoming(&paths.releases()); + sweep_incoming(&paths.recovery()); + let id = package.manifest.release_id.clone(); + let release_dir = paths.releases().join(&id); + let reused = freeze_copy(&package, &release_dir)?; + let release = validate_package(&release_dir)?; + let spec = ServiceSpec { + label: options.label.clone(), + plist: options.plist.clone(), + program: release_dir.join("bin/devguardd"), + args: options.program_args.clone(), + environment: options.environment.clone(), + }; + let plist = render_plist(&spec, &options.log, options.throttle_seconds)?; + let written = OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o644) + .custom_flags(libc::O_NOFOLLOW | libc::O_CLOEXEC) + .open(&options.plist) + .and_then(|mut file| { + file.write_all(plist.as_bytes())?; + file.sync_all() + }); + if written.is_err() { + return Err(Error::new( + ErrorCode::ResourceUnavailable, + "the service plist already exists or cannot be written", + )); + } + let started = manager + .bootstrap(&spec) + .and_then(|()| wait_running(paths, manager, options, &release)); + let evidence = match started { + Ok(evidence) => evidence, + Err(error) => { + // Nothing was selected: stop and unload the unverified service. + let _ = manager.bootout(&options.label); + let _ = fs::remove_file(&options.plist); + return Err(error); + } + }; + let recovery = paths.recovery().join(&id); + freeze_copy(&release, &recovery)?; + let mut selection = read_selection(paths)?.unwrap_or(Selection { + schema: SELECTION_SCHEMA.into(), + current: id.clone(), + last_known_good: None, + history: Vec::new(), + }); + selection.current = id.clone(); + selection.last_known_good = Some(id.clone()); + selection.history.push(SelectionEvent { + release_id: id.clone(), + event: "installed and verified running".into(), + unix_ms: unix_ms(), + }); + write_selection(paths, &selection)?; + Ok(InstallReport { + release_id: id, + release: release_dir, + manifest_sha256: release.manifest_sha256, + reused_release: reused, + label: options.label.clone(), + plist: options.plist.clone(), + running: evidence, + recovery, + selection, + }) +} + +#[derive(Debug, Clone, Serialize)] +pub struct StatusReport { + pub label: String, + pub plist: PathBuf, + /// The plist is exactly what this build renders for the current release. + pub plist_matches_selection: bool, + pub service: Option, + pub selection: Option, + pub running: Option, + pub running_error: Option, + pub releases: Vec, + pub recovery: Vec, + /// The service runs the verified current release. + pub healthy: bool, +} + +fn listing(path: &Path) -> Vec { + fs::read_dir(path) + .map(|entries| { + let mut names: Vec = entries + .flatten() + .map(|entry| entry.file_name().to_string_lossy().into_owned()) + .filter(|name| !name.starts_with('.') && !name.ends_with(".json")) + .collect(); + names.sort(); + names + }) + .unwrap_or_default() +} + +/// Report the installed service without changing anything. +pub fn status( + paths: &AuthorityPaths, + manager: &dyn ServiceManager, + options: &InstallOptions, +) -> Result { + let selection = read_selection(paths)?; + let service = manager.state(&options.label)?; + let mut report = StatusReport { + label: options.label.clone(), + plist: options.plist.clone(), + plist_matches_selection: false, + service, + selection: selection.clone(), + running: None, + running_error: None, + releases: listing(&paths.releases()), + recovery: listing(&paths.recovery()), + healthy: false, + }; + let Some(selection) = selection else { + report.running_error = Some("no release has been installed".into()); + return Ok(report); + }; + let release_dir = paths.releases().join(&selection.current); + let release = match validate_package(&release_dir) { + Ok(release) => release, + Err(error) => { + report.running_error = Some(format!( + "the current release is not intact: {}", + error.message + )); + return Ok(report); + } + }; + let spec = ServiceSpec { + label: options.label.clone(), + plist: options.plist.clone(), + program: release_dir.join("bin/devguardd"), + args: options.program_args.clone(), + environment: options.environment.clone(), + }; + report.plist_matches_selection = render_plist(&spec, &options.log, options.throttle_seconds) + .ok() + .zip(fs::read_to_string(&options.plist).ok()) + .is_some_and(|(rendered, actual)| rendered == actual); + match observe_running(paths, manager, &options.label, &release) { + Ok(evidence) => report.running = Some(evidence), + Err(error) => report.running_error = Some(error.message), + } + report.healthy = report.plist_matches_selection && report.running.is_some(); + Ok(report) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn launchctl_print_is_read_from_the_service_level_only() { + let text = "gui/501/io.example = {\n\tactive count = 1\n\tpath = /x.plist\n\tstate = running\n\n\tprogram = /bin/sh\n\truns = 2\n\tpid = 4242\n\tlast terminating signal = Segmentation fault: 11\n\tendpoints = {\n\t\tstate = active\n\t\tpid = 7\n\t}\n}\n"; + let state = parse_print(text); + assert!(state.running); + assert_eq!(state.pid, Some(4242)); + assert_eq!(state.runs, Some(2)); + assert_eq!( + state.last_exit.as_deref(), + Some("last terminating signal = Segmentation fault: 11") + ); + let stopped = + parse_print("x = {\n\tstate = not running\n\truns = 3\n\tlast exit code = 1\n}\n"); + assert!(!stopped.running); + assert_eq!(stopped.pid, None); + assert_eq!(stopped.last_exit.as_deref(), Some("last exit code = 1")); + } + + #[test] + fn the_plist_restarts_only_a_crashed_service_and_escapes_its_values() { + let spec = ServiceSpec { + label: LABEL.into(), + plist: "/Users/u/Library/LaunchAgents/x.plist".into(), + program: "/Users/u/Library/Application Support/DevGuard/releases/r&1/bin/devguardd" + .into(), + args: vec!["serve".into()], + environment: BTreeMap::from([("A".to_string(), "".to_string())]), + }; + let plist = render_plist( + &spec, + Path::new("/Users/u/Library/Logs/DevGuard/devguardd.log"), + 10, + ) + .unwrap(); + assert!(plist.contains("/Users/u/Library/Application Support/DevGuard/releases/r&1/bin/devguardd\n serve")); + assert!( + plist.contains("KeepAlive\n \n Crashed\n ") + ); + assert!(plist.contains("A\n <b>")); + assert!(plist.contains("RunAtLoad\n ")); + let canonical = render_plist( + &ServiceSpec { + environment: BTreeMap::new(), + ..spec + }, + Path::new("/l"), + 10, + ) + .unwrap(); + assert!(!canonical.contains("EnvironmentVariables")); + } + + #[test] + fn release_ids_are_plain_names() { + for good in ["0.1.0-d30fbce-1a2b3c4d", "r_1", "a.b-c"] { + assert!(valid_release_id(good), "{good}"); + } + for bad in ["", ".hidden", "a/b", "../x", "a b", &"x".repeat(129)] { + assert!(!valid_release_id(bad), "{bad}"); + } + } + + #[test] + fn this_build_states_its_own_compatibility() { + let build = compiled(); + assert_eq!(build.package_version, env!("CARGO_PKG_VERSION")); + assert_eq!(build.journal_schema, devguard_core::JOURNAL_SCHEMA); + assert!(build.capabilities.contains(&Capability::FencedLaunch)); + } +} diff --git a/crates/daemon/src/lib.rs b/crates/daemon/src/lib.rs index 9591f4c..5ed2a91 100644 --- a/crates/daemon/src/lib.rs +++ b/crates/daemon/src/lib.rs @@ -2,5 +2,6 @@ pub mod config; #[cfg(any(test, feature = "test-fixtures"))] pub mod fixture; +pub mod install; pub mod paths; pub mod server; diff --git a/crates/daemon/src/main.rs b/crates/daemon/src/main.rs index ccf553f..83a105d 100644 --- a/crates/daemon/src/main.rs +++ b/crates/daemon/src/main.rs @@ -2,6 +2,7 @@ use devguard_contract::{Error, ErrorCode, Result}; use devguard_core::AuthorityStorage; use devguard_daemon::{ config::{self, HostConfig}, + install::{self, InstallOptions, Launchctl}, paths::AuthorityPaths, server::Server, }; @@ -18,17 +19,60 @@ extern "C" fn stop_signal(_: libc::c_int) { fn run() -> Result<()> { let args: Vec<_> = std::env::args().skip(1).collect(); if args == ["--help"] || args == ["help"] { - println!("devguardd paths | init | check | serve\nNormal authority paths come from the OS account. No path or test-budget override is accepted.\nserve observes native boot, process and host pressure evidence and, with it, opens registration, fenced launch through devguard-launch and reconciliation. There is no execution CLI yet; init and check never start workloads."); + println!("devguardd paths | init | check | serve | status | install --package DIR | version --json\nNormal authority paths come from the OS account. No path or test-budget override is accepted.\nserve observes native boot, process and host pressure evidence and, with it, opens registration, fenced launch through devguard-launch and reconciliation; `devguard` runs commands through it. install copies a package into a protected release and starts it as the current user's LaunchAgent; it must run from that package and refuses while an authority or the agent exists. status reports the installed service. init and check never start workloads."); return Ok(()); } - if args.len() != 1 || !matches!(args[0].as_str(), "paths" | "init" | "check" | "serve") { + let command = args.first().map(String::as_str).unwrap_or_default(); + let valid = match command { + "paths" | "init" | "check" | "serve" | "status" => args.len() == 1, + "install" => args.len() == 3 && args[1] == "--package", + "version" => args.len() == 2 && args[1] == "--json", + _ => false, + }; + if !valid { return Err(Error::new( ErrorCode::InvalidRequest, - "expected paths, init, check or serve; no alternate authority arguments are supported", + "expected paths, init, check, serve, status, install --package DIR or version --json; no alternate authority arguments are supported", )); } + if command == "version" { + println!( + "{}", + serde_json::to_string(&install::compiled()) + .map_err(|_| Error::new(ErrorCode::InvalidRequest, "version encoding failed"))? + ); + return Ok(()); + } let paths = AuthorityPaths::current_user()?; - match args[0].as_str() { + match command { + "install" => { + let report = install::install( + &paths, + std::path::Path::new(&args[2]), + &Launchctl::new(paths.uid()), + &InstallOptions::canonical(&paths), + )?; + println!( + "{}", + serde_json::to_string_pretty(&report) + .map_err(|_| Error::new(ErrorCode::InvalidRequest, "report encoding failed"))? + ); + } + "status" => { + let report = install::status( + &paths, + &Launchctl::new(paths.uid()), + &InstallOptions::canonical(&paths), + )?; + println!( + "{}", + serde_json::to_string_pretty(&report) + .map_err(|_| Error::new(ErrorCode::InvalidRequest, "report encoding failed"))? + ); + if !report.healthy { + std::process::exit(1); + } + } "paths" => println!( "{}", serde_json::to_string(&paths) diff --git a/crates/daemon/src/paths.rs b/crates/daemon/src/paths.rs index 41bea47..11cc3b1 100644 --- a/crates/daemon/src/paths.rs +++ b/crates/daemon/src/paths.rs @@ -86,6 +86,27 @@ impl AuthorityPaths { pub fn root(&self) -> &Path { &self.root } + pub fn home(&self) -> &Path { + &self.home + } + /// Installed immutable releases, with the selection of the current one. + pub fn releases(&self) -> PathBuf { + self.root.join("releases") + } + pub fn selection(&self) -> PathBuf { + self.releases().join("selection.json") + } + /// Independent copies that repair can use when a release is damaged. + pub fn recovery(&self) -> PathBuf { + self.root.join("recovery") + } + /// The current user's launchd agents. + pub fn launch_agents(&self) -> PathBuf { + self.home.join("Library/LaunchAgents") + } + pub fn logs(&self) -> PathBuf { + self.home.join("Library/Logs/DevGuard") + } pub fn runtime(&self) -> &Path { &self.runtime } diff --git a/crates/daemon/src/server.rs b/crates/daemon/src/server.rs index 665ca6d..9119e26 100644 --- a/crates/daemon/src/server.rs +++ b/crates/daemon/src/server.rs @@ -269,7 +269,7 @@ fn poisoned() -> Error { } /// The capabilities of a service with registration and fenced launch open. -fn launch_capabilities() -> BTreeSet { +pub(crate) fn launch_capabilities() -> BTreeSet { BTreeSet::from([ Capability::DurableAdmission, Capability::FencedLaunch, diff --git a/crates/daemon/tests/entrypoint.rs b/crates/daemon/tests/entrypoint.rs index 88f6e72..d7957e1 100644 --- a/crates/daemon/tests/entrypoint.rs +++ b/crates/daemon/tests/entrypoint.rs @@ -28,7 +28,8 @@ fn help_states_what_serve_opens_and_that_bootstrap_starts_no_workload() { assert!(result.status.success()); let help = String::from_utf8(result.stdout).unwrap(); assert!(help.contains("with it, opens registration, fenced launch")); - assert!(help.contains("There is no execution CLI yet; init and check never start workloads.")); + assert!(help.contains("install copies a package into a protected release")); + assert!(help.contains("init and check never start workloads.")); } #[cfg(target_os = "macos")] diff --git a/crates/daemon/tests/install.rs b/crates/daemon/tests/install.rs new file mode 100644 index 0000000..38375ea --- /dev/null +++ b/crates/daemon/tests/install.rs @@ -0,0 +1,674 @@ +#![cfg(target_os = "macos")] +//! DG1-C09: installing a release as the current user's LaunchAgent. +//! +//! The test binary is copied into each package as its `devguardd`, so the +//! installer really runs from the package it installs. The installed service +//! is that copy, re-executed as `daemon_child` to serve an isolated fixture +//! authority. A fake manager starts the program as launchd would; the native +//! test uses launchd itself, with a unique label and a plist under the test's +//! own directory, and always boots the job out. + +use devguard_contract::{digest_bytes, ErrorCode}; +use devguard_daemon::fixture::{serve_until_terminated, TestAuthority}; +use devguard_daemon::install::{ + self, Artifact, InstallOptions, Launchctl, Manifest, ServiceManager, ServiceSpec, ServiceState, + ARTIFACTS, MANIFEST_SCHEMA, +}; +use devguard_daemon::paths::AuthorityPaths; +use serde_json::{json, Value}; +use std::collections::BTreeMap; +use std::fs; +use std::os::unix::fs::PermissionsExt; +use std::os::unix::process::CommandExt; +use std::path::{Path, PathBuf}; +use std::process::{Child, Command, Stdio}; +use std::sync::{Arc, Mutex}; +use std::time::{Duration, Instant}; + +const DAEMON: &str = "DEVGUARD_TEST_DAEMON"; + +#[test] +#[ignore = "the service program of the installer tests"] +fn daemon_child() { + let Some(base) = std::env::var_os(DAEMON) else { + return; + }; + if let Err(error) = serve_until_terminated(Path::new(&base)) { + eprintln!("fixture daemon failed: {error:?}"); + std::process::exit(1); + } +} + +fn harness_args() -> Vec { + [ + "--exact", + "daemon_child", + "--ignored", + "--nocapture", + "--test-threads=1", + ] + .map(String::from) + .to_vec() +} + +/// A private base directory. Installed releases are read-only, so their +/// directories are made writable again before the directory is removed. +struct Base(tempfile::TempDir); + +impl Base { + fn new() -> Self { + Self( + tempfile::Builder::new() + .prefix("dg-i-") + .tempdir_in("/private/tmp") + .unwrap(), + ) + } + + fn path(&self) -> &Path { + self.0.path() + } +} + +fn make_writable(path: &Path) { + let Ok(meta) = fs::symlink_metadata(path) else { + return; + }; + if meta.is_dir() && !meta.file_type().is_symlink() { + let _ = fs::set_permissions(path, fs::Permissions::from_mode(0o700)); + if let Ok(entries) = fs::read_dir(path) { + for entry in entries.flatten() { + make_writable(&entry.path()); + } + } + } +} + +impl Drop for Base { + fn drop(&mut self) { + make_writable(self.0.path()); + } +} + +/// Bootstrap an authority under `base`, then stop serving it. +fn initialized(base: &Path) -> AuthorityPaths { + drop(TestAuthority::start(base).unwrap()); + AuthorityPaths::fixture(base) +} + +fn copy_executable(from: &Path, to: &Path) { + fs::copy(from, to).unwrap(); + fs::set_permissions(to, fs::Permissions::from_mode(0o755)).unwrap(); +} + +fn manifest_for(dir: &Path, release_id: &str, created_at: &str) -> Manifest { + let artifacts = ARTIFACTS + .iter() + .map(|name| { + let bytes = fs::read(dir.join("bin").join(name)).unwrap(); + ( + name.to_string(), + Artifact { + sha256: digest_bytes(&bytes), + bytes: bytes.len() as u64, + }, + ) + }) + .collect(); + Manifest { + schema: MANIFEST_SCHEMA.into(), + release_id: release_id.into(), + version: install::compiled().package_version, + source: json!({"test": true}), + build: json!({"profile": "test"}), + artifacts, + compatibility: install::compiled(), + scope: "functional".into(), + slo_qualified: false, + created_at: created_at.into(), + } +} + +fn write_manifest(dir: &Path, manifest: &Manifest) { + fs::write( + dir.join("MANIFEST.json"), + serde_json::to_vec_pretty(manifest).unwrap(), + ) + .unwrap(); +} + +/// A package whose `devguardd` is this test binary. +fn package(parent: &Path, name: &str, release_id: &str, created_at: &str) -> PathBuf { + let dir = parent.join(name); + fs::create_dir_all(dir.join("bin")).unwrap(); + copy_executable( + &std::env::current_exe().unwrap(), + &dir.join("bin/devguardd"), + ); + copy_executable(Path::new("/usr/bin/true"), &dir.join("bin/devguard")); + copy_executable(Path::new("/usr/bin/true"), &dir.join("bin/devguard-launch")); + let manifest = manifest_for(&dir, release_id, created_at); + write_manifest(&dir, &manifest); + dir +} + +fn options(paths: &AuthorityPaths, base: &Path, label: &str) -> InstallOptions { + InstallOptions { + label: label.into(), + plist: paths.launch_agents().join(format!("{label}.plist")), + log: paths.logs().join("devguardd.log"), + program_args: harness_args(), + environment: BTreeMap::from([(DAEMON.to_string(), base.to_string_lossy().into_owned())]), + start_deadline: Duration::from_secs(20), + throttle_seconds: 1, + } +} + +/// Starts the job's program as launchd would: its own process group, the job's +/// environment added, and a report while it runs. +#[derive(Default)] +struct FakeLaunchd { + job: Mutex>, + bootstraps: Mutex, +} + +impl ServiceManager for FakeLaunchd { + fn bootstrap(&self, spec: &ServiceSpec) -> devguard_contract::Result<()> { + let mut job = self.job.lock().unwrap(); + if job.is_some() { + return Err(devguard_contract::Error::new( + ErrorCode::ResourceUnavailable, + "a job with this label is already loaded", + )); + } + *self.bootstraps.lock().unwrap() += 1; + let child = Command::new(&spec.program) + .args(&spec.args) + .envs(&spec.environment) + .process_group(0) + .stdin(Stdio::null()) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .spawn() + .unwrap(); + *job = Some((spec.label.clone(), child)); + Ok(()) + } + + fn bootout(&self, label: &str) -> devguard_contract::Result<()> { + let mut job = self.job.lock().unwrap(); + if job.as_ref().is_some_and(|(loaded, _)| loaded == label) { + let (_, mut child) = job.take().unwrap(); + // SAFETY: signalling the child this fake started. + unsafe { libc::kill(child.id() as i32, libc::SIGTERM) }; + let _ = child.wait(); + } + Ok(()) + } + + fn state(&self, label: &str) -> devguard_contract::Result> { + let mut job = self.job.lock().unwrap(); + Ok(match job.as_mut() { + Some((loaded, child)) if loaded == label => Some(match child.try_wait().unwrap() { + None => ServiceState { + running: true, + pid: Some(child.id()), + runs: Some(1), + last_exit: None, + }, + Some(status) => ServiceState { + running: false, + pid: None, + runs: Some(1), + last_exit: Some(format!("last exit code = {:?}", status.code())), + }, + }), + _ => None, + }) + } +} + +impl Drop for FakeLaunchd { + fn drop(&mut self) { + // Release the lock before booting out, which takes it again. + let label = self + .job + .lock() + .unwrap() + .as_ref() + .map(|(label, _)| label.clone()); + if let Some(label) = label { + let _ = ServiceManager::bootout(self, &label); + } + } +} + +fn wait_for(limit: Duration, mut done: impl FnMut() -> bool) { + let deadline = Instant::now() + limit; + while !done() { + assert!(Instant::now() < deadline, "condition not met in {limit:?}"); + std::thread::sleep(Duration::from_millis(100)); + } +} + +fn mode(path: &Path) -> u32 { + fs::symlink_metadata(path).unwrap().permissions().mode() & 0o777 +} + +/// Qualification runs collect raw receipts; ordinary runs write nothing. +fn record(name: &str, value: Value) { + if let Some(directory) = std::env::var_os("DEVGUARD_EVIDENCE_DIR") { + let directory = PathBuf::from(directory); + fs::create_dir_all(&directory).unwrap(); + fs::write( + directory.join(format!("{name}.json")), + serde_json::to_string_pretty(&value).unwrap(), + ) + .unwrap(); + } +} + +const LABEL: &str = "io.github.novelkr.devguard.test"; + +#[test] +fn an_installed_release_is_verified_running_before_it_is_selected() { + let base = Base::new(); + let paths = initialized(base.path()); + let packages = base.path().join("packages"); + let pkg = package(&packages, "a", "0.1.0-test-a", "2026-09-24T00:00:00Z"); + let manager = FakeLaunchd::default(); + let options = options(&paths, base.path(), LABEL); + let report = install::install(&paths, &pkg, &manager, &options).unwrap(); + let release = paths.releases().join("0.1.0-test-a"); + assert_eq!(report.release, release); + assert!(!report.reused_release); + // The service launchd runs is the release's own devguardd, serving the endpoint. + assert_eq!(report.running.service_pid, report.running.authority_pid); + assert_eq!( + fs::canonicalize(&report.running.executable).unwrap(), + fs::canonicalize(release.join("bin/devguardd")).unwrap() + ); + assert_eq!(report.selection.current, "0.1.0-test-a"); + assert_eq!( + report.selection.last_known_good.as_deref(), + Some("0.1.0-test-a") + ); + // The release and its recovery copy are immutable. + for copy in [release.clone(), paths.recovery().join("0.1.0-test-a")] { + assert_eq!(mode(©), 0o500); + assert_eq!(mode(©.join("bin")), 0o500); + assert_eq!(mode(©.join("MANIFEST.json")), 0o400); + for name in ARTIFACTS { + assert_eq!(mode(©.join("bin").join(name)), 0o500); + } + } + let plist = fs::read_to_string(&options.plist).unwrap(); + assert!(plist.contains(&release.join("bin/devguardd").to_string_lossy().into_owned())); + assert_eq!(mode(&options.plist), 0o644); + assert_eq!(mode(&paths.selection()), 0o600); + let status = install::status(&paths, &manager, &options).unwrap(); + assert!(status.healthy, "{status:?}"); + assert_eq!(status.releases, ["0.1.0-test-a"]); + // While it runs, a second installation is refused. + let again = install::install(&paths, &pkg, &manager, &options).unwrap_err(); + assert_eq!(again.code, ErrorCode::ResourceUnavailable, "{again:?}"); + assert_eq!(*manager.bootstraps.lock().unwrap(), 1); + record( + "install-verified", + json!({"report": report, "status": status, "second_install": again.message}), + ); +} + +#[test] +fn an_identical_release_is_reused_and_a_different_one_never_overwrites_it() { + let base = Base::new(); + let paths = initialized(base.path()); + let packages = base.path().join("packages"); + let first = package(&packages, "a", "0.1.0-test-a", "2026-09-24T00:00:00Z"); + let manager = FakeLaunchd::default(); + let options = options(&paths, base.path(), LABEL); + install::install(&paths, &first, &manager, &options).unwrap(); + let uninstall = |manager: &FakeLaunchd| { + manager.bootout(LABEL).unwrap(); + fs::remove_file(&options.plist).unwrap(); + }; + uninstall(&manager); + let reused = install::install(&paths, &first, &manager, &options).unwrap(); + assert!(reused.reused_release); + uninstall(&manager); + // The same release id with different contents is refused, and the + // installed release is unchanged. + let installed = fs::read(paths.releases().join("0.1.0-test-a/MANIFEST.json")).unwrap(); + let other = package(&packages, "b", "0.1.0-test-a", "2026-09-25T00:00:00Z"); + let refused = install::install(&paths, &other, &manager, &options).unwrap_err(); + assert_eq!(refused.code, ErrorCode::InvalidRequest, "{refused:?}"); + assert!(refused.message.contains("never overwritten"), "{refused:?}"); + assert_eq!( + fs::read(paths.releases().join("0.1.0-test-a/MANIFEST.json")).unwrap(), + installed + ); + assert!(!options.plist.exists()); + assert!(manager.state(LABEL).unwrap().is_none()); + record( + "install-reuse", + json!({"reused": reused.reused_release, "different_release": refused.message}), + ); +} + +#[test] +fn packages_that_do_not_match_their_manifest_are_refused_before_anything_is_written() { + let base = Base::new(); + let paths = initialized(base.path()); + let packages = base.path().join("packages"); + let manager = FakeLaunchd::default(); + let options = options(&paths, base.path(), LABEL); + let mut cases = Vec::new(); + /// (case, how the package is changed, the refusal) + type Tamper = (&'static str, Box, ErrorCode); + let tamper: Vec = vec![ + ( + "modified-binary", + Box::new(|dir: &Path| { + let path = dir.join("bin/devguard"); + let mut bytes = fs::read(&path).unwrap(); + bytes.push(0); + fs::write(path, bytes).unwrap(); + }), + ErrorCode::InvalidRequest, + ), + ( + "extra-file", + Box::new(|dir: &Path| fs::write(dir.join("bin/extra"), b"x").unwrap()), + ErrorCode::InvalidRequest, + ), + ( + "missing-binary", + Box::new(|dir: &Path| fs::remove_file(dir.join("bin/devguard-launch")).unwrap()), + ErrorCode::InvalidRequest, + ), + ( + "symlinked-binary", + Box::new(|dir: &Path| { + fs::remove_file(dir.join("bin/devguard")).unwrap(); + std::os::unix::fs::symlink("/usr/bin/true", dir.join("bin/devguard")).unwrap(); + }), + ErrorCode::InvalidRequest, + ), + ( + "other-capabilities", + Box::new(|dir: &Path| { + let mut manifest: Manifest = + serde_json::from_slice(&fs::read(dir.join("MANIFEST.json")).unwrap()).unwrap(); + manifest.compatibility.capabilities.clear(); + write_manifest(dir, &manifest); + }), + ErrorCode::ResourcePolicyUnsupported, + ), + ( + "inconsistent-scope", + Box::new(|dir: &Path| { + let mut manifest: Manifest = + serde_json::from_slice(&fs::read(dir.join("MANIFEST.json")).unwrap()).unwrap(); + manifest.slo_qualified = true; + write_manifest(dir, &manifest); + }), + ErrorCode::InvalidRequest, + ), + ( + "installer-not-in-package", + Box::new(|dir: &Path| { + copy_executable(Path::new("/usr/bin/true"), &dir.join("bin/devguardd")); + let manifest = manifest_for(dir, "0.1.0-test-x", "2026-09-24T00:00:00Z"); + write_manifest(dir, &manifest); + }), + ErrorCode::InvalidRequest, + ), + ]; + for (name, change, code) in tamper { + let pkg = package(&packages, name, "0.1.0-test-x", "2026-09-24T00:00:00Z"); + change(&pkg); + let error = install::install(&paths, &pkg, &manager, &options).unwrap_err(); + assert_eq!(error.code, code, "{name}: {error:?}"); + assert!(!paths.releases().exists(), "{name}: nothing is installed"); + assert!(!options.plist.exists(), "{name}"); + cases.push(json!({"case": name, "code": error.code, "message": error.message})); + } + assert_eq!(*manager.bootstraps.lock().unwrap(), 0); + record("install-refusals", json!({"cases": cases})); +} + +#[test] +fn installation_is_refused_while_an_authority_serves_or_without_its_state() { + let base = Base::new(); + let authority = TestAuthority::start(base.path()).unwrap(); + let paths = AuthorityPaths::fixture(base.path()); + let pkg = package( + &base.path().join("packages"), + "a", + "0.1.0-test-a", + "2026-09-24T00:00:00Z", + ); + let manager = FakeLaunchd::default(); + let options = options(&paths, base.path(), LABEL); + let serving = install::install(&paths, &pkg, &manager, &options).unwrap_err(); + assert_eq!(serving.code, ErrorCode::ResourceUnavailable, "{serving:?}"); + assert!(!options.plist.exists()); + drop(authority); + // Installation never creates the authority state it needs. + let empty = Base::new(); + let missing = AuthorityPaths::fixture(empty.path()); + let pkg = package( + &empty.path().join("packages"), + "a", + "0.1.0-test-a", + "2026-09-24T00:00:00Z", + ); + let unbootstrapped = install::install(&missing, &pkg, &manager, &options).unwrap_err(); + assert!(!missing.releases().exists()); + assert!(!missing.journal().exists()); + assert_eq!(*manager.bootstraps.lock().unwrap(), 0); + record( + "install-preconditions", + json!({"authority_serving": serving.message, "missing_state": unbootstrapped.message}), + ); +} + +#[test] +fn a_service_that_never_proves_itself_is_unloaded_and_nothing_is_selected() { + let base = Base::new(); + let paths = initialized(base.path()); + let pkg = package( + &base.path().join("packages"), + "a", + "0.1.0-test-a", + "2026-09-24T00:00:00Z", + ); + let manager = FakeLaunchd::default(); + let mut options = options(&paths, base.path(), LABEL); + // The program exits at once without serving anything. + options.program_args = vec!["--exact".into(), "no_such_test".into()]; + options.start_deadline = Duration::from_secs(2); + let error = install::install(&paths, &pkg, &manager, &options).unwrap_err(); + assert!( + manager.state(LABEL).unwrap().is_none(), + "the unverified job is unloaded" + ); + assert!(!options.plist.exists()); + assert!( + install::read_selection(&paths).unwrap().is_none(), + "nothing is selected" + ); + assert!(!paths.recovery().join("0.1.0-test-a").exists()); + let status = install::status(&paths, &manager, &options).unwrap(); + assert!(!status.healthy); + record( + "install-unverified", + json!({"error": error.message, "status": status}), + ); +} + +#[test] +fn concurrent_installers_leave_exactly_one_service() { + let base = Base::new(); + let paths = initialized(base.path()); + let pkg = package( + &base.path().join("packages"), + "a", + "0.1.0-test-a", + "2026-09-24T00:00:00Z", + ); + let manager = Arc::new(FakeLaunchd::default()); + let options = options(&paths, base.path(), LABEL); + let outcomes: Vec<_> = (0..2) + .map(|_| { + let (paths, pkg, manager, options) = + (paths.clone(), pkg.clone(), manager.clone(), options.clone()); + std::thread::spawn(move || install::install(&paths, &pkg, &*manager, &options)) + }) + .collect::>() + .into_iter() + .map(|thread| thread.join().unwrap()) + .collect(); + let succeeded = outcomes.iter().filter(|outcome| outcome.is_ok()).count(); + assert_eq!(succeeded, 1, "{outcomes:?}"); + assert_eq!(*manager.bootstraps.lock().unwrap(), 1); + let entries: Vec = fs::read_dir(paths.releases()) + .unwrap() + .flatten() + .map(|entry| entry.file_name().to_string_lossy().into_owned()) + .collect(); + assert!( + entries.iter().all(|name| !name.starts_with(".incoming")), + "{entries:?}" + ); + assert!( + install::status(&paths, &*manager, &options) + .unwrap() + .healthy + ); + let refused = outcomes + .iter() + .find_map(|outcome| outcome.as_ref().err()) + .unwrap(); + record( + "install-concurrent", + json!({"succeeded": succeeded, "refused": refused.message, "releases": entries}), + ); +} + +/// Boots the transient job out even when the test fails. +struct Bootout { + manager: Launchctl, + label: String, +} + +impl Drop for Bootout { + fn drop(&mut self) { + let _ = self.manager.bootout(&self.label); + } +} + +fn gui_domain_available(uid: u32) -> bool { + Command::new("/bin/launchctl") + .args(["print", &format!("gui/{uid}")]) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .status() + .is_ok_and(|status| status.success()) +} + +#[test] +fn launchd_runs_the_release_restarts_it_after_a_crash_and_leaves_a_failed_start_down() { + let base = Base::new(); + let paths = initialized(base.path()); + if !gui_domain_available(paths.uid()) { + record( + "launchd-lifecycle", + json!({"status": "not_run", "reason": "this session has no launchd gui domain"}), + ); + return; + } + let label = format!("{LABEL}.{}", std::process::id()); + let manager = Launchctl::new(paths.uid()); + let _bootout = Bootout { + manager: Launchctl::new(paths.uid()), + label: label.clone(), + }; + let options = options(&paths, base.path(), &label); + let pkg = package( + &base.path().join("packages"), + "a", + "0.1.0-test-a", + "2026-09-24T00:00:00Z", + ); + let report = install::install(&paths, &pkg, &manager, &options).unwrap(); + let first = report.running.service_pid; + let loaded = manager.state(&label).unwrap().unwrap(); + assert_eq!(loaded.pid, Some(first)); + // A crash, as an abort after a panic, is restarted after the throttle + // interval, and the new process reopens and reconciles the same journal. + // (A SIGSEGV sent with kill only resets the handler Rust installs for stack + // overflows, so the process would survive it.) + // SAFETY: crashing the service this test installed. + assert_eq!(unsafe { libc::kill(first as i32, libc::SIGABRT) }, 0); + let deadline = Instant::now() + Duration::from_secs(30); + while !install::status(&paths, &manager, &options).is_ok_and(|status| { + status.healthy + && status + .running + .is_some_and(|running| running.service_pid != first) + }) { + assert!( + Instant::now() < deadline, + "no healthy restart: {:?}\n{}", + manager.state(&label), + fs::read_to_string(&options.log).unwrap_or_default() + ); + std::thread::sleep(Duration::from_millis(100)); + } + let restarted = manager.state(&label).unwrap().unwrap(); + assert!(restarted.runs >= Some(2), "{restarted:?}"); + assert!( + restarted + .last_exit + .as_deref() + .is_some_and(|exit| exit.contains("Abort trap")), + "{restarted:?}" + ); + // Stopping through launchd sends SIGTERM: a clean exit that removes the endpoint. + manager.bootout(&label).unwrap(); + wait_for(Duration::from_secs(10), || { + manager.state(&label).unwrap().is_none() && !paths.socket().exists() + }); + // With its journal made unreadable the service fails closed, and launchd + // does not restart a failed exit. + let journal = fs::read(paths.journal()).unwrap(); + fs::write(paths.journal(), vec![0x55; journal.len()]).unwrap(); + let spec = ServiceSpec { + label: label.clone(), + plist: options.plist.clone(), + program: report.release.join("bin/devguardd"), + args: options.program_args.clone(), + environment: options.environment.clone(), + }; + manager.bootstrap(&spec).unwrap(); + wait_for(Duration::from_secs(15), || { + manager.state(&label).unwrap().is_some_and(|state| { + !state.running && state.last_exit.as_deref() == Some("last exit code = 1") + }) + }); + std::thread::sleep(Duration::from_secs(3)); + let failed = manager.state(&label).unwrap().unwrap(); + assert!(!failed.running, "{failed:?}"); + assert_eq!( + failed.runs, + Some(1), + "a failed start is not restarted: {failed:?}" + ); + assert!(!paths.socket().exists()); + manager.bootout(&label).unwrap(); + record( + "launchd-lifecycle", + json!({"label": label, "installed": report, "after_crash": restarted, "failed_start": failed}), + ); +} diff --git a/crates/macos/src/lib.rs b/crates/macos/src/lib.rs index f276fb9..06fe548 100644 --- a/crates/macos/src/lib.rs +++ b/crates/macos/src/lib.rs @@ -18,7 +18,7 @@ mod table; pub use backend::NativeBackend; pub use clock::BootClock; pub use host::{HostCapacity, HostProbe, HostReading, NativeProbe, VolumeReading}; -pub use process::process_identity; +pub use process::{executable_path, process_identity}; pub use root::{become_scope_root, exec_with_workload_qos}; pub use sampler::{SampleReceipt, Sampler, SamplerOutcome, SAMPLE_INTERVAL_MS, WINDOW_MS}; pub use scope::{ diff --git a/crates/macos/src/process.rs b/crates/macos/src/process.rs index ba68770..0076196 100644 --- a/crates/macos/src/process.rs +++ b/crates/macos/src/process.rs @@ -3,6 +3,7 @@ //! unchanged by `exec` and distinguishes a reused PID within one boot. use devguard_contract::{ProcessIdentity, Result}; +use std::path::PathBuf; /// A live process observed in one consistent snapshot. #[cfg_attr(not(target_os = "macos"), allow(dead_code))] @@ -79,6 +80,36 @@ pub(crate) fn presence(_: u32) -> Result { Err(crate::unsupported()) } +/// The executable image of a live process, as the kernel records it. An +/// absent PID is `None`; a refused observation is an error, never absence. +#[cfg(target_os = "macos")] +pub fn executable_path(pid: u32) -> Result> { + use std::os::unix::ffi::OsStrExt; + let Ok(native) = libc::pid_t::try_from(pid) else { + return Ok(None); + }; + if native <= 0 { + return Ok(None); + } + let mut buffer = vec![0u8; libc::PROC_PIDPATHINFO_MAXSIZE as usize]; + // SAFETY: the buffer is live and its length is passed with it. + let length = + unsafe { libc::proc_pidpath(native, buffer.as_mut_ptr().cast(), buffer.len() as u32) }; + if length <= 0 { + return match std::io::Error::last_os_error().raw_os_error() { + Some(libc::ESRCH) => Ok(None), + _ => Err(crate::failed("cannot observe a process's executable")), + }; + } + buffer.truncate(length as usize); + Ok(Some(PathBuf::from(std::ffi::OsStr::from_bytes(&buffer)))) +} + +#[cfg(not(target_os = "macos"))] +pub fn executable_path(_: u32) -> Result> { + Err(crate::unsupported()) +} + #[cfg(target_os = "macos")] fn start_ticks(pid: libc::pid_t) -> Result> { // SAFETY: rusage_info_v0 is a plain C output structure. diff --git a/docs/contracts.md b/docs/contracts.md index a432397..c2175a3 100644 --- a/docs/contracts.md +++ b/docs/contracts.md @@ -1,6 +1,6 @@ # Implemented authority contract -This document describes the implemented DG-0 authority, C01/C02 service, storage and transport behavior, C03 native macOS host evidence, C04 cooperative policy and scope evidence, the C05 fenced launch helper, C06 reconciliation, the C07 command-line owner and the C08 Cargo adapters. PR and post-merge main delivery evidence is tracked separately from implementation. [Operations](operations.md) lists actual command availability. With native host evidence the service opens registration, launch and reconciliation; CodeSpace integration remains unimplemented. [Korean translation](ko/contracts.md). +This document describes the implemented DG-0 authority, C01/C02 service, storage and transport behavior, C03 native macOS host evidence, C04 cooperative policy and scope evidence, the C05 fenced launch helper, C06 reconciliation, the C07 command-line owner, the C08 Cargo adapters and C09 installation as the current user's LaunchAgent. PR and post-merge main delivery evidence is tracked separately from implementation. [Operations](operations.md) lists actual command availability. With native host evidence the service opens registration, launch and reconciliation; CodeSpace integration remains unimplemented. [Korean translation](ko/contracts.md). ## Authority and transport boundaries @@ -146,6 +146,48 @@ DG1-C08 adds the Cargo adapters (`devguard-cargo`), which fit Cargo's compiler p - **Inherited jobservers.** A jobserver the CLI inherited through `CARGO_MAKEFLAGS`, `MAKEFLAGS` or `MFLAGS` is checked the way Cargo will read it: the first variable present must name an open, inheritable pipe pair or a FIFO the user owns. A valid one is preserved and bounds parallelism in both modes, as the approved design requires, and the receipt states that its size cannot be observed; no second pool is created. Under it, direct mode inserts no jobs value, because Cargo would only warn that it ignores one, but an explicit value is still clamped. A descriptor-pair jobserver reaches only programs that keep inherited descriptors open. In pipeline mode, a Cargo started by a program that closes them, such as Python's `subprocess` by default, falls back to `CARGO_BUILD_JOBS`, and the receipt says so. A stale one, such as closed descriptors, is removed from the environment together with the job counts that came with it, so Cargo does not silently fall back to its own pool. - **Nested Cargo.** Cargo passes its jobserver to build scripts, so a nested Cargo shares the outer pool rather than creating another. +## Installation and the current-user service + +DG1-C09 installs a packaged release as the current user's LaunchAgent. The service still runs the same foreground `devguardd serve`; it is not a privileged daemon, and a worktree `target` binary is never the installed service. + +- **Package.** `scripts/package.py` requires a clean tree and Rust 1.95.0. It builds `devguardd`, `devguard` and `devguard-launch` in one release build, labelled bootstrap until C10, and writes `devguard-release-manifest/v1` with: + - a release id: `--`; + - the source commit, tree and qualification tree digest; + - the build command and toolchain; + - each binary's SHA-256 and size; + - the build's compiled compatibility, which `devguardd version --json` prints: package version, wire version, protocol, capabilities, journal schema and configuration schema. + + A package is a functional artifact (`scope: functional`, `slo_qualified: false`); only C12 qualifies a release. +- **Validation.** + - A package or installed release holds exactly `MANIFEST.json` and `bin/` with the three binaries. They must be regular executable files, not symlinks, whose sizes and hashes match the manifest. + - The manifest's compatibility must equal the installer's own build. + - The installer must run from the package: its own executable must hash to the manifest's `devguardd`. Binaries from different builds are therefore never mixed. +- **Install.** `devguardd install --package DIR` refuses while: + - an authority serves the canonical endpoint or holds the lock; + - the agent is loaded or its plist exists; + - the authority state is absent. It never creates, repairs or rewrites the journal. + + It copies the release into a private sibling, syncs it and makes it read-only: directories and binaries 0500, the manifest 0400. Only then does it rename the copy to `releases/`, so a partial copy never has the final name. An existing release is reused only when its manifest is byte-identical; a different release never overwrites it. Finally it writes `~/Library/LaunchAgents/io.github.novelkr.devguard.plist` (0644) and bootstraps it into the user's `gui/` domain. +- **Agent.** + - The program is the release's `devguardd serve`, and `RunAtLoad` starts it at load and login. + - `KeepAlive {Crashed: true}` restarts it only after a crash, after a 10-second throttle. + - A clean exit leaves it stopped. That includes launchd's SIGTERM and a fail-closed exit, such as a missing or corrupt journal or a second authority refused by the lock. + - Output goes to `~/Library/Logs/DevGuard/devguardd.log`; rotation is manual. +- **Verification before selection.** Within 15 seconds the installer must observe: + - the service launchd reports, with the same PID as the endpoint's handshake; + - that PID's executable image (`proc_pidpath`) being the release's `devguardd`, with the manifest's hash. + + Only then does it record `releases/selection.json` (0600: current and last known good, with a history) and keep an immutable recovery copy under `recovery/`. If verification fails, the job is booted out and the plist removed; nothing is selected. +- **Status.** `devguardd status` changes nothing. It reports: + - the selection and the launchd state; + - whether the plist is exactly the one this build renders for the current release; + - the running PID, executable and hash against the manifest; + - the releases and recovery copies. + + It exits 0 only when the service runs the verified current release. +- **Restart.** A restarted service reopens and reconciles the existing journal, as any start does. A missing or corrupt journal fails closed and stays down. +- **Limits.** Replacing the installed release is an upgrade (C11); C09 refuses it. There is no uninstall command. `launchctl bootout gui//io.github.novelkr.devguard` and removing the plist stop the service and keep every release, recovery copy, the selection and the journal. + ## Durable admission and launch The journal schema is version 1. Initialization is explicit and only creates a new file. Opening a missing, corrupt, unknown-schema or inconsistent journal fails closed. SQLite uses WAL, FULL synchronous durability and immediate transactions. A startup scan validates the accounting index against every stored record. diff --git a/docs/ko/contracts.md b/docs/ko/contracts.md index 9770cce..c769fc4 100644 --- a/docs/ko/contracts.md +++ b/docs/ko/contracts.md @@ -1,6 +1,6 @@ # 구현된 authority 계약 -이 문서는 구현된 DG-0 authority, C01/C02 서비스·저장소·transport 동작, C03 native macOS 호스트 증거, C04 협조적 정책·scope 증거, C05 fenced launch helper, C06 대조, C07 명령행 owner, C08 Cargo adapter를 설명한다. PR과 병합 후 main 전달 증거는 구현과 별도로 추적한다. 실제 제공 명령은 [운영 문서](operations.md)에 있다. Native 호스트 증거가 있으면 서비스는 등록·launch·대조를 열며, CodeSpace 결합은 아직 구현하지 않았다. [영문 정본](../contracts.md). +이 문서는 구현된 DG-0 authority, C01/C02 서비스·저장소·transport 동작, C03 native macOS 호스트 증거, C04 협조적 정책·scope 증거, C05 fenced launch helper, C06 대조, C07 명령행 owner, C08 Cargo adapter, 현재 사용자 LaunchAgent로의 C09 설치를 설명한다. PR과 병합 후 main 전달 증거는 구현과 별도로 추적한다. 실제 제공 명령은 [운영 문서](operations.md)에 있다. Native 호스트 증거가 있으면 서비스는 등록·launch·대조를 열며, CodeSpace 결합은 아직 구현하지 않았다. [영문 정본](../contracts.md). ## Authority와 transport 경계 @@ -146,6 +146,48 @@ DG1-C08은 command가 실행되는 예약에 맞춰 Cargo의 compiler 병렬도 - **상속된 jobserver.** CLI가 `CARGO_MAKEFLAGS`·`MAKEFLAGS`·`MFLAGS`로 상속한 jobserver는 Cargo가 읽는 방식대로 검사한다. 처음 존재하는 변수가 열려 있고 상속 가능한 pipe 쌍이나 사용자 소유 FIFO를 가리켜야 한다. 유효한 jobserver는 승인 설계대로 두 mode 모두에서 보존되어 병렬도를 제한하며, receipt에는 그 크기를 관측할 수 없다고 기록한다. 두 번째 pool은 만들지 않는다. 이때 direct mode는 job 수를 넣지 않는다. Cargo가 무시한다는 경고만 낼 것이기 때문이다. 다만 명시적 값은 여전히 조정한다. Descriptor 쌍 jobserver는 상속 descriptor를 열어 두는 프로그램에만 전달된다. Pipeline mode에서 Python `subprocess`의 기본 동작처럼 descriptor를 닫는 프로그램이 시작한 Cargo는 `CARGO_BUILD_JOBS`로 돌아가며, receipt에 그렇게 기록한다. 닫힌 descriptor처럼 오래된 참조는 함께 온 job 수와 함께 환경에서 제거하므로, Cargo가 조용히 자체 pool로 돌아가지 않는다. - **중첩 Cargo.** Cargo는 build script에 jobserver를 넘기므로, 중첩 Cargo는 새 pool을 만들지 않고 바깥 pool을 공유한다. +## 설치와 현재 사용자 서비스 + +DG1-C09는 패키지로 만든 release를 현재 사용자의 LaunchAgent로 설치한다. 서비스는 여전히 같은 foreground `devguardd serve`를 실행한다. 권한 있는 daemon이 아니며, worktree `target`의 바이너리를 설치된 서비스로 쓰지 않는다. + +- **패키지.** `scripts/package.py`는 깨끗한 tree와 Rust 1.95.0을 요구한다. 한 번의 release build로 `devguardd`·`devguard`·`devguard-launch`를 만들며, C10 전까지 이 build는 bootstrap으로 표시한다. 그리고 `devguard-release-manifest/v1`에 다음을 기록한다. + - release id: `--`; + - source commit·tree·qualification tree digest; + - build 명령과 toolchain; + - 각 바이너리의 SHA-256과 크기; + - build에 컴파일된 호환성. `devguardd version --json`이 이를 출력하며, 패키지 버전·wire 버전·protocol·capability·journal schema·설정 schema를 담는다. + + 패키지는 기능 artifact(`scope: functional`, `slo_qualified: false`)이며, release를 qualification하는 것은 C12뿐이다. +- **검증.** + - 패키지나 설치된 release에는 정확히 `MANIFEST.json`과 세 바이너리가 든 `bin/`만 있어야 한다. 바이너리는 symlink가 아닌 실행 가능한 일반 파일이어야 하며, 크기와 hash가 manifest와 일치해야 한다. + - Manifest의 호환성은 installer 자신의 build와 같아야 한다. + - Installer는 그 패키지에서 실행해야 한다. 즉 installer 자신의 실행 파일 hash가 manifest의 `devguardd`와 같아야 한다. 따라서 서로 다른 build의 바이너리가 섞이지 않는다. +- **설치.** `devguardd install --package DIR`는 다음 경우 거절한다. + - authority가 정상 endpoint를 제공하거나 잠금을 보유한 동안; + - agent가 load되어 있거나 plist가 있을 때; + - authority 상태가 없을 때. Journal을 만들거나 고치거나 다시 쓰지 않는다. + + Release를 private 형제 디렉터리로 복사해 sync하고 읽기 전용으로 만든다(디렉터리와 바이너리 0500, manifest 0400). 그다음에야 `releases/`로 rename하므로, 부분 복사본은 최종 이름을 갖지 않는다. 이미 있는 release는 manifest가 byte 단위로 같을 때만 재사용하며, 다른 release로 덮어쓰지 않는다. 끝으로 `~/Library/LaunchAgents/io.github.novelkr.devguard.plist`(0644)를 쓰고 사용자의 `gui/` domain에 bootstrap한다. +- **Agent.** + - 프로그램은 release의 `devguardd serve`이며, `RunAtLoad`로 load와 login 때 시작한다. + - `KeepAlive {Crashed: true}`이므로 crash 뒤에만 10초 throttle 후 다시 시작한다. + - 정상 종료에서는 멈춘 채로 둔다. launchd의 SIGTERM, 그리고 journal이 없거나 손상되었거나 두 번째 authority가 잠금에 거절된 경우처럼 닫힌 채 끝나는 종료가 여기에 포함된다. + - 출력은 `~/Library/Logs/DevGuard/devguardd.log`에 남으며, rotation은 수동이다. +- **선택 전 검증.** Installer는 15초 안에 다음을 관측해야 한다. + - launchd가 보고하는 서비스의 PID가 endpoint handshake의 PID와 같을 것; + - 그 PID의 실행 image(`proc_pidpath`)가 release의 `devguardd`이고 manifest의 hash와 같을 것. + + 그다음에야 `releases/selection.json`(0600: current와 last known good, 이력)을 기록하고 `recovery/`에 변경 불가능한 복구 사본을 보존한다. 검증에 실패하면 job을 bootout하고 plist를 제거하며, 아무것도 선택하지 않는다. +- **상태.** `devguardd status`는 아무것도 바꾸지 않고 다음을 보고한다. + - 선택과 launchd 상태; + - plist가 이 build가 현재 release에 대해 만드는 plist와 정확히 같은지; + - 실행 중인 PID·실행 파일·hash와 manifest의 비교; + - release와 복구 사본. + + 서비스가 검증된 current release를 실행할 때만 0으로 끝난다. +- **재시작.** 다시 시작한 서비스는 다른 시작과 마찬가지로 기존 journal을 다시 열어 대조한다. Journal이 없거나 손상되었으면 닫힌 채 멈춰 있다. +- **범위.** 설치된 release를 교체하는 것은 upgrade(C11)이며 C09는 이를 거절한다. 제거 명령은 없다. `launchctl bootout gui//io.github.novelkr.devguard`를 실행하고 plist를 지우면 서비스가 멈추며, 모든 release·복구 사본·선택·journal은 보존된다. + ## 내구성 admission과 launch Journal schema는 1이다. 초기화는 명시적으로 새 파일만 만든다. 누락·손상·미지원 schema·불일치 journal은 fail-closed다. SQLite는 WAL, FULL synchronous와 immediate transaction을 사용한다. 시작 시 모든 저장 record와 회계 index를 대조한다. diff --git a/docs/ko/operations.md b/docs/ko/operations.md index 4d5d793..5f6020a 100644 --- a/docs/ko/operations.md +++ b/docs/ko/operations.md @@ -2,7 +2,7 @@ [English](../operations.md) | [한국어](operations.md) -C01은 명시 bootstrap·고정 저장소를, C02는 인증된 로컬 transport를, C03은 native macOS boot·프로세스·호스트 압력 증거를, C04는 협조적 정책 readback과 scope 증거를, C05는 fenced launch helper를, C06은 대조를, C07은 `devguard` 명령행 owner를, C08은 Cargo adapter를 제공한다. PR과 병합 후 main 전달 증거는 구현과 별도로 추적한다. devguardd serve는 이 증거로 journal을 활성화하는 foreground 서비스를 실행하고 wire 등록·fenced launch·대조를 연다. `devguard exec`는 이 서비스를 통해 명령을 실행하고, `devguard doctor`는 이를 진단한다. 정상 authority는 현재 macOS만 지원하며 Linux CI는 이식 가능한 계약과 제한된 fixture를 검사한다. DG-LINUX 제어 자격을 부여하지 않는다. +C01은 명시 bootstrap·고정 저장소를, C02는 인증된 로컬 transport를, C03은 native macOS boot·프로세스·호스트 압력 증거를, C04는 협조적 정책 readback과 scope 증거를, C05는 fenced launch helper를, C06은 대조를, C07은 `devguard` 명령행 owner를, C08은 Cargo adapter를, C09는 패키지로 만든 release를 현재 사용자 LaunchAgent로 설치하는 기능을 제공한다. PR과 병합 후 main 전달 증거는 구현과 별도로 추적한다. devguardd serve는 이 증거로 journal을 활성화하는 foreground 서비스를 실행하고 wire 등록·fenced launch·대조를 연다. `devguard exec`는 이 서비스를 통해 명령을 실행하고, `devguard doctor`는 이를 진단한다. 정상 authority는 현재 macOS만 지원하며 Linux CI는 이식 가능한 계약과 제한된 fixture를 검사한다. DG-LINUX 제어 자격을 부여하지 않는다. ## 제공 명령 @@ -39,7 +39,17 @@ Native 증거를 사용할 수 있으면 `serve`는 관측한 호스트로 정 Native 증거가 없는 플랫폼이거나 macOS 관측이 실패하면 `serve`는 저장소만 보유한 닫힌 동작을 유지하고 어느 경우인지 보고한다. -인증된 status는 storage_validated, registration_ready, execution_ready, 이유와 설정 fingerprint를 보고한다. Native 증거가 있으면 등록과 실행이 준비된 상태이며, admission은 여전히 호스트 압력과 용량을 따른다. 그렇지 않으면 둘 다 false이며 이유는 미지원 플랫폼과 native 관측 실패를 구분한다. `devguard`는 자기 옆의 `devguard-launch`를 찾으므로 둘을 함께 빌드한다. 설치·LaunchAgent·repair는 후속 작업이다. 설정이나 handshake 성공만으로 명령이 관리되지는 않으며 bootstrap 빌드는 자기 적용 증거가 아니다. 영속 서비스는 제거 가능한 target 경로를 사용하지 않는다. 보호된 설치는 C09의 범위다. +인증된 status는 storage_validated, registration_ready, execution_ready, 이유와 설정 fingerprint를 보고한다. Native 증거가 있으면 등록과 실행이 준비된 상태이며, admission은 여전히 호스트 압력과 용량을 따른다. 그렇지 않으면 둘 다 false이며 이유는 미지원 플랫폼과 native 관측 실패를 구분한다. `devguard`는 자기 옆의 `devguard-launch`를 찾으므로 둘을 함께 빌드한다. 설정이나 handshake 성공만으로 명령이 관리되지는 않으며 bootstrap 빌드는 자기 적용 증거가 아니다. 영속 서비스를 제거 가능한 target 경로에서 실행하지 않는다. 대신 패키지를 설치한다. + +영속 서비스는 설치된 release를 실행한다. + +```sh +python3 scripts/package.py --offline +target/package//bin/devguardd install --package target/package/ +devguardd status +``` + +`package.py`는 깨끗한 tree에서 release 패키지를 만든다. `install`은 그 패키지에서 실행해야 하며, authority가 제공 중이거나 agent가 있거나 기존 authority 상태가 없으면 거절한다. Release를 변경 불가능한 `releases/`로 복사하고, 현재 사용자 LaunchAgent `io.github.novelkr.devguard`로 시작한다. 이 agent는 `releases//bin/devguardd serve`를 실행한다. launchd가 정확히 그 바이너리를 실행한다고 검증한 뒤에만 release를 선택한다. `devguardd status`는 설치된 서비스를 보고하며, 검증된 current release를 실행하지 않으면 1로 끝난다. Release 교체는 upgrade이며 repair와 함께 후속 작업(C11)이다. [계약 문서](contracts.md#설치와-현재-사용자-서비스)를 참조한다. `devguard exec [--project ID] [--adapter auto|generic|cargo|cargo-pipeline] [--wait DURATION] [--cpu MILLICPU] [--memory SIZE] [--tasks N] [--receipt PATH] -- PROGRAM [ARGS...]`는 명령을 admission하고 `devguard-launch`로 시작한 뒤 끝날 때까지 기다린다. 프로그램과 인자는 `--` 뒤에 두며 shell은 사용하지 않는다. 명령의 종료값으로 끝나거나 그 signal로 끝나며, 아무것도 시작하지 않았으면 125로 끝난다. `--wait`가 없으면 거절 즉시 끝나고, `--wait`가 있으면 용량·압력 거절을 기한까지 다시 시도하며, 호스트의 작업 용량보다 큰 요청은 즉시 거절한다. `--receipt`는 private JSON receipt를 기록한다. `devguard doctor`는 JSON 진단을 출력하며, `--require admission,registration,macos-cooperative`를 주면 요구 사항이 하나라도 충족되지 않을 때 실패한다. [계약 문서](contracts.md#명령행-owner)를 참조한다. @@ -111,6 +121,7 @@ python3 scripts/qualify.py dg1-launch --offline python3 scripts/qualify.py dg1-reconcile --offline python3 scripts/qualify.py dg1-cli --offline python3 scripts/qualify.py dg1-cargo --offline +python3 scripts/qualify.py dg1-bootstrap --offline python3 scripts/validate.py --offline ``` @@ -200,6 +211,17 @@ Core·launcher 증거·서비스·wire 시험은 이전 boot 회수, 옛 PID를 각 Cargo 실행과 pipeline script는 표준 descriptor만 상속한다. 호출자 자신의 descriptor 쌍 jobserver를 상속한 경우만 예외다. Shim은 각 Cargo가 받은 인자와 jobserver·fallback 변수도 기록한다. Hosted macOS 14 runner처럼 호스트의 작업 용량이 해당 경우에 필요한 Cargo job을 수용할 수 없으면 그 경우를 `not_run`으로 기록하고 suite는 `incomplete`를 보고한다. CI는 `--allow-incomplete`로 실행한다. -이 suite들은 Linux 강제, 자기 적용, foreground SLO를 not_run으로 남긴다. 전체 검증은 기존 44개 시험과 workspace crate 8개의 명시적 전체 의존 그래프를 검사한다. Launch crate는 시험의 격리 authority를 위해서만 daemon에 의존한다. CLI는 daemon의 경로·설정과 Cargo adapter에 의존하며, daemon fixture에는 시험에서만 의존한다. Cargo adapter는 contract에만 의존한다. Core·contract는 daemon 설정과 native adapter에 독립적이며 client는 core에 의존하지 않는다. +`dg1-bootstrap`도 macOS에서만 실행한다. 시험은 각 패키지의 `devguardd`로 시험 바이너리를 복사하므로, installer는 실제로 자기 패키지에서 실행된다. 설치된 서비스는 그 사본이며 격리된 fixture authority를 제공한다. 검사 항목은 다음과 같다. +- 선택 전에 실행이 검증된 설치 release, 변경 불가능한 release·복구 사본과 정상 status +- 같은 release의 재사용, 그리고 같은 id의 다른 release는 설치된 release를 건드리지 않고 거절하는지 +- 아무것도 쓰기 전에 거절되는 패키지: 바뀌었거나 남거나 빠졌거나 symlink인 바이너리, 다른 capability, 맞지 않는 scope, 패키지 자신의 것이 아닌 installer +- authority가 제공 중이거나 authority 상태가 없을 때의 거절 +- 끝내 자신을 입증하지 못해 아무것도 선택하지 않고 unload되는 서비스 +- 서비스 하나만 남기는 동시 installer +- launchd에서 고유 label의 임시 job으로: 같은 journal로 다시 시작하는 crash, SIGTERM 정지, journal을 읽을 수 없어 닫힌 채 끝나고 다시 시작하지 않는 시작 + +launchd gui domain이 없는 session에서는 launchd 경우를 `not_run`으로 기록하고 suite는 `incomplete`를 보고한다. + +이 suite들은 Linux 강제, 자기 적용, foreground SLO를 not_run으로 남긴다. 전체 검증은 기존 44개 시험과 workspace crate 8개의 명시적 전체 의존 그래프를 검사한다. Daemon은 자기 시험에서 fixture를 켜기 위해서만 자신에게 의존한다. Launch crate는 시험의 격리 authority를 위해서만 daemon에 의존한다. CLI는 daemon의 경로·설정과 Cargo adapter에 의존하며, daemon fixture에는 시험에서만 의존한다. Cargo adapter는 contract에만 의존한다. Core·contract는 daemon 설정과 native adapter에 독립적이며 client는 core에 의존하지 않는다. -복귀할 때 작업 소유 foreground 프로세스를 중지하고 보존한 설정과 schema 1 journal에 호환되는 source/artifact를 선택하며 영속 상태·자격을 유지한다. C03 활성화는 새 record 종류를 추가하지 않고 기존 복구만 수행하므로 C02 artifact로 같은 상태를 다시 열 수 있다. C06부터는 정상 서비스를 통해 workload가 시작될 수 있다. 대조가 없는 artifact로 복귀하기 전에 새 작업 시작을 멈추고 과금 중인 attempt가 종결 phase에 이를 때까지 기다린다. C05 artifact는 같은 journal을 읽지만 launch를 닫아 두고 대조하지 않으므로 남은 attempt는 과금된 Suspect로 남는다. Scope가 실행 중일 때 서비스가 멈추면 다시 시작한다. 그 attempt들은 owner가 claim되지 않은 grant를 보고하거나 reboot가 종료를 입증할 때까지 과금된 Suspect로 남는다. C07은 서비스 상태를 바꾸지 않는다. CLI를 되돌리려면 CLI로 명령을 시작하는 것을 멈춘다. 이미 시작한 명령은 scope가 끝날 때까지 과금되며, 관리되지 않는 실행으로 대체하지 않는다. C08도 서비스 상태를 바꾸지 않는다. Cargo job 조정을 멈추려면 `--adapter generic`을 명시하며, 기존 target과 cache는 유지된다. 복귀나 재시작을 성공시키려고 journal이나 tombstone을 삭제하지 않는다. +복귀할 때 작업 소유 foreground 프로세스를 중지하고 보존한 설정과 schema 1 journal에 호환되는 source/artifact를 선택하며 영속 상태·자격을 유지한다. C03 활성화는 새 record 종류를 추가하지 않고 기존 복구만 수행하므로 C02 artifact로 같은 상태를 다시 열 수 있다. C06부터는 정상 서비스를 통해 workload가 시작될 수 있다. 대조가 없는 artifact로 복귀하기 전에 새 작업 시작을 멈추고 과금 중인 attempt가 종결 phase에 이를 때까지 기다린다. C05 artifact는 같은 journal을 읽지만 launch를 닫아 두고 대조하지 않으므로 남은 attempt는 과금된 Suspect로 남는다. Scope가 실행 중일 때 서비스가 멈추면 다시 시작한다. 그 attempt들은 owner가 claim되지 않은 grant를 보고하거나 reboot가 종료를 입증할 때까지 과금된 Suspect로 남는다. C07은 서비스 상태를 바꾸지 않는다. CLI를 되돌리려면 CLI로 명령을 시작하는 것을 멈춘다. 이미 시작한 명령은 scope가 끝날 때까지 과금되며, 관리되지 않는 실행으로 대체하지 않는다. C08도 서비스 상태를 바꾸지 않는다. Cargo job 조정을 멈추려면 `--adapter generic`을 명시하며, 기존 target과 cache는 유지된다. C09 installer는 release가 검증되기 전에는 job을 bootout하고 plist를 제거하며 아무것도 선택하지 않는다. 설치된 서비스를 멈추려면 `launchctl bootout gui//io.github.novelkr.devguard`를 실행하고 plist를 지운다. Release·복구 사본·선택·journal은 보존되며, 보존한 artifact의 foreground `devguardd serve`가 같은 상태를 읽는다. 복귀나 재시작을 성공시키려고 journal이나 tombstone을 삭제하지 않는다. diff --git a/docs/ko/planning/milestones/DG-1.md b/docs/ko/planning/milestones/DG-1.md index bf4771c..9f015e2 100644 --- a/docs/ko/planning/milestones/DG-1.md +++ b/docs/ko/planning/milestones/DG-1.md @@ -4,7 +4,7 @@ 아래 ID·제목·PR 묶음은 **예정 값**이다. 실제 SHA나 GitHub PR 번호가 아니다. 모듈 경로는 구현 전까지 예정 책임을 나타낸다. 현재 daemon/client crate는 존재하며 launcher·platform-macos·cli·adapters는 후속 책임이다. crate 추가는 해당 PR에서 명시적 의존 allowlist와 전체 그래프 검증을 함께 확장하고 검사를 제거하지 않는다. 실제 상태는 ledger가 소유한다. -구현된 C01은 정상 경로·명시 bootstrap·배타 journal 검사를 제공한다. C02는 foreground devguardd serve, 제한된 인증 UDS 통신, OS UID/PID 관측, 엄격한 client 호환성과 private 자격 FD 전달을 추가한다. C03은 `devguard-macos`의 boot 시계, native PID/start 정체성, 호스트 용량과 serve에서 journal을 활성화하는 2초 압력 sampler를 추가한다. C04는 협조적 QoS/nice 적용과 readback, 이탈·추적 상실이 고정되는 관측 process group scope, 정체성을 확인한 종료를 추가한다. C05는 wire 등록과 fenced `devguard-launch` helper를 추가한다. Grant마다 claim된 helper는 하나이며, READY와 exec 전에 scope binding과 authorization을 거치고, transcript와 exec 실패 보고를 분리하며, payload descriptor를 정리한다. C06은 서비스 reconciler를 추가한다. 관측된 scope 종료, helper가 없다는 owner 보고, 이전 boot에서만 회수하며, reap 전 owner 관측, scope 종료, instance 폐기, 재시작 전후의 Suspect 회계를 제공한다. Native 증거가 있으면 서비스는 등록·launch·대조를 연다. C07은 `devguard` 명령행 owner를 추가한다. 명시적이고 제한된 대기, terminal·signal 전달, reap 전 관측, receipt, doctor 진단을 갖춘 관리 실행을 제공하며, 관리되지 않는 대체 실행은 없다. C08은 Cargo adapter를 추가한다. Compiler job을 예약에 맞춰 조정하거나 거절하며, pipeline과 중첩 Cargo 실행이 jobserver 하나를 공유하게 한다. PR과 병합 후 main 전달 증거는 별도로 추적한다. [운영 문서](../../operations.md)를 참조한다. +구현된 C01은 정상 경로·명시 bootstrap·배타 journal 검사를 제공한다. C02는 foreground devguardd serve, 제한된 인증 UDS 통신, OS UID/PID 관측, 엄격한 client 호환성과 private 자격 FD 전달을 추가한다. C03은 `devguard-macos`의 boot 시계, native PID/start 정체성, 호스트 용량과 serve에서 journal을 활성화하는 2초 압력 sampler를 추가한다. C04는 협조적 QoS/nice 적용과 readback, 이탈·추적 상실이 고정되는 관측 process group scope, 정체성을 확인한 종료를 추가한다. C05는 wire 등록과 fenced `devguard-launch` helper를 추가한다. Grant마다 claim된 helper는 하나이며, READY와 exec 전에 scope binding과 authorization을 거치고, transcript와 exec 실패 보고를 분리하며, payload descriptor를 정리한다. C06은 서비스 reconciler를 추가한다. 관측된 scope 종료, helper가 없다는 owner 보고, 이전 boot에서만 회수하며, reap 전 owner 관측, scope 종료, instance 폐기, 재시작 전후의 Suspect 회계를 제공한다. Native 증거가 있으면 서비스는 등록·launch·대조를 연다. C07은 `devguard` 명령행 owner를 추가한다. 명시적이고 제한된 대기, terminal·signal 전달, reap 전 관측, receipt, doctor 진단을 갖춘 관리 실행을 제공하며, 관리되지 않는 대체 실행은 없다. C08은 Cargo adapter를 추가한다. Compiler job을 예약에 맞춰 조정하거나 거절하며, pipeline과 중첩 Cargo 실행이 jobserver 하나를 공유하게 한다. C09는 패키지로 만든 release를 현재 사용자 LaunchAgent로 설치한다. hash와 컴파일된 호환성을 담은 manifest, 변경 불가능한 release·복구 사본을 두며, launchd가 그 release의 바이너리를 실행한다고 검증한 뒤에만 선택한다. PR과 병합 후 main 전달 증거는 별도로 추적한다. [운영 문서](../../operations.md)를 참조한다. ## PR 순서와 활성화 경계 @@ -17,7 +17,7 @@ | DG1-P5 | DG1-C09, DG1-C10, DG1-C11 | DG1-P4 | 설치·후보·독립 복구를 묶어 자기 적용 활성화 | | DG1-P6 | DG1-C12 | DG1-P5 | 기능 기준 artifact와 SLO 안정 artifact를 구분해 승격 | -공통 현재 명령 python3 scripts/validate.py --offline은 Rust 1.95.0 계약 회귀를 확인하며 fake backend로 native 동작을 입증하지 않는다. C01 authority, C02 인증·transport, C03 native probe, C04 native scope, C05 native launch, C06 native 대조, C07 CLI, C08 Cargo suite는 현재 제공한다. 아래에서 예정이라고 명시한 나머지 명령은 미제공이며 각 PR에서 fixture·실행 case 수·log·정리를 함께 구현하고 제공 상태를 갱신한다. 이름만 있는 테스트나 0개 실행을 통과로 처리하지 않는다. 공통 toolchain·증거·SLO 규칙은 상위 검증 문서에 있다. +공통 현재 명령 python3 scripts/validate.py --offline은 Rust 1.95.0 계약 회귀를 확인하며 fake backend로 native 동작을 입증하지 않는다. C01 authority, C02 인증·transport, C03 native probe, C04 native scope, C05 native launch, C06 native 대조, C07 CLI, C08 Cargo, C09 설치 suite는 현재 제공한다. 아래에서 예정이라고 명시한 나머지 명령은 미제공이며 각 PR에서 fixture·실행 case 수·log·정리를 함께 구현하고 제공 상태를 갱신한다. 이름만 있는 테스트나 0개 실행을 통과로 처리하지 않는다. 공통 toolchain·증거·SLO 규칙은 상위 검증 문서에 있다. P1은 runtime을 닫아 두고 P2는 실제 probe, P3는 launch·안전 정리, P4는 개발 진입점, P5는 설치·부모 예산·repair, P6는 측정·승격을 제공한다. C08까지 foreground daemon과 최소 단일 Cargo job·test thread bootstrap을 사용한다. P4 bundle은 삭제할 build 경로 밖에 보존한다. C10에서 부모 예산을 포함한 artifact를 먼저 기능 시험·동결한 직후 제한된 실제 자기 적용을 시작하며 SLO qualification은 C12에서 확립한다. @@ -133,7 +133,7 @@ P1은 runtime을 닫아 두고 P2는 실제 probe, P3는 launch·안전 정리, - 대상/산출물: artifact manifest/hash·daemon/helper 호환 정보, 예정 installer/service 시작·상태 확인, immutable 복구 사본. - 불변 조건: 후보가 기준 artifact 덮어쓰기 금지; bootstrap 기준은 기능 시험만 통과한 상태로 표시; 정상 authority는 하나. - 시험: 정상 최초 설치/재시작; hash mismatch·부분 설치·지원되지 않는 host; 동시 시작/설치 경쟁. -- 검증 명령: 현재 공통 회귀 + 예정 `python3 scripts/qualify.py dg1-bootstrap`. +- 검증 명령: 현재 공통 회귀 + **제공** `python3 scripts/qualify.py dg1-bootstrap --offline` (macOS 전용이며 다른 플랫폼은 `not_run`으로 기록한다). launchd gui domain이 없는 session은 launchd 경우를 `not_run`으로 기록한다. - 완료 증거: 실제 실행 binary hash와 manifest 일치, 서비스 PID/endpoint, 기능 기준 artifact의 한정된 검증 범위. - rollback: 서비스 시작 전 검증 실패 시 이전 보존 사본 선택; 활성 journal을 과거 snapshot으로 교체하지 않는다. - 인계: DG1-C10에 보호된 설치·복구 artifact를 제공한다. C09 artifact가 새 C10 상위 lease 기능을 이미 지원한다고 가정하지 않는다. P5 전체 전 일상 자기 적용으로 홍보하지 않는다. diff --git a/docs/ko/planning/verification.md b/docs/ko/planning/verification.md index 281e11c..2b07194 100644 --- a/docs/ko/planning/verification.md +++ b/docs/ko/planning/verification.md @@ -34,6 +34,7 @@ python3 scripts/qualify.py dg1-launch --offline python3 scripts/qualify.py dg1-reconcile --offline python3 scripts/qualify.py dg1-cli --offline python3 scripts/qualify.py dg1-cargo --offline +python3 scripts/qualify.py dg1-bootstrap --offline git diff --check ``` @@ -75,7 +76,7 @@ V-DOC-DG는 scripts/check_docs.py와 수동 의미 검토로 영어/한국어 ha ## 향후 suite와 장애 주입 계약 -scripts/qualify.py dg1-authority --offline은 C01 설정·저장소, scripts/qualify.py dg1-auth --offline은 C02 인증·transport, scripts/qualify.py dg1-probes --offline은 C03 native 증거, scripts/qualify.py dg1-scopes --offline은 C04 정책·scope 증거, scripts/qualify.py dg1-launch --offline은 C05 launch helper, scripts/qualify.py dg1-reconcile --offline은 C06 대조, scripts/qualify.py dg1-cli --offline은 C07 명령행 owner, scripts/qualify.py dg1-cargo --offline은 C08 Cargo adapter 검증을 제공한다. 그 밖의 DevGuard suite와 CodeSpace scripts/qualify-devguard.py는 해당 작업에서 제공할 예정 인터페이스다. 각 구현 PR이 실제 CLI·case inventory·nonzero case assertion·timeout·log 수집·격리 cleanup을 구현하고 제공 명령을 갱신해야 한다. macOS/Ubuntu CI는 전체 validator와 이식 가능한 두 기능 suite를 실행하고 각각의 report·log를 보존한다. Native suite는 macOS CI에서만 실행하고 그 밖의 환경에서는 `not_run`으로 기록한다. 증거를 제공할 수 없는 플랫폼에서 통과로 처리하지 않는다. 각 native 단계는 만들어야 할 raw receipt를 선언한다. 어떤 경우를 `not_run`으로 기록한 receipt가 있으면 suite는 `passed`가 아니라 `incomplete`가 된다. +scripts/qualify.py dg1-authority --offline은 C01 설정·저장소, scripts/qualify.py dg1-auth --offline은 C02 인증·transport, scripts/qualify.py dg1-probes --offline은 C03 native 증거, scripts/qualify.py dg1-scopes --offline은 C04 정책·scope 증거, scripts/qualify.py dg1-launch --offline은 C05 launch helper, scripts/qualify.py dg1-reconcile --offline은 C06 대조, scripts/qualify.py dg1-cli --offline은 C07 명령행 owner, scripts/qualify.py dg1-cargo --offline은 C08 Cargo adapter, scripts/qualify.py dg1-bootstrap --offline은 C09 설치 검증을 제공한다. 그 밖의 DevGuard suite와 CodeSpace scripts/qualify-devguard.py는 해당 작업에서 제공할 예정 인터페이스다. 각 구현 PR이 실제 CLI·case inventory·nonzero case assertion·timeout·log 수집·격리 cleanup을 구현하고 제공 명령을 갱신해야 한다. macOS/Ubuntu CI는 전체 validator와 이식 가능한 두 기능 suite를 실행하고 각각의 report·log를 보존한다. Native suite는 macOS CI에서만 실행하고 그 밖의 환경에서는 `not_run`으로 기록한다. 증거를 제공할 수 없는 플랫폼에서 통과로 처리하지 않는다. 각 native 단계는 만들어야 할 raw receipt를 선언한다. 어떤 경우를 `not_run`으로 기록한 receipt가 있으면 suite는 `passed`가 아니라 `incomplete`가 된다. C03 증거는 다음 파일에 기록한다. - **Raw receipt** (보고서의 `raw/` 디렉터리): 단위를 포함한 boot ID·시계 읽기, 호스트 용량, zombie·reap·거부 관측을 포함한 반복 프로세스 정체성, 계산된 비율을 포함한 native 압력 읽기, 마지막 sample 이후 admission이 닫히기까지 측정한 시간, 주입한 실패부터 Critical까지 걸린 서비스 loop 시간과 멈춘 probe에 대한 서비스 loop의 동작. @@ -117,6 +118,12 @@ C08 증거는 다음을 포함한다. - **실제 프로세스**: CLI owner와 실제 helper를 통한 작은 offline workspace의 실제 Cargo build. Compiler wrapper가 각 compile의 시간을 재고, `cargo` shim이 각 실행이 상속한 descriptor를 기록한다. 호스트의 작업 용량이 Cargo job을 수용할 수 없는 경우는 `not_run`으로 기록한다. - **단위 시험**: job 추정, subcommand와 `--` 전후의 인자 parsing, Cargo가 읽는 방식대로 해석한 job 값, jobserver 아래를 포함한 우선순위·조정·거절, fallback 변수, 닫혔거나 close-on-exec인 참조·descriptor 쌍·FIFO 참조에 대한 상속 jobserver 검사, FIFO pool과 그 크기 상한·token 수, adapter 선택. +C09 증거는 다음을 포함한다. +- **Raw receipt**: 실행 중인 PID·실행 파일·hash를 포함한 검증된 설치, 재사용한 같은 release와 거절한 다른 release, 거절된 패키지, authority 제공 중이거나 상태가 없을 때의 거절, 아무것도 선택하지 않고 unload된 미검증 서비스, 동시 installer, crash 재시작·정지·닫힌 채 끝나는 시작을 포함한 launchd 수명 주기. +- **실제 프로세스**: 각 패키지의 `devguardd`로 복사한 시험 바이너리가 격리된 fixture authority를 제공한다. Fake manager는 launchd와 같은 방식으로 이를 시작한다. Native 경우는 launchd 자체를 사용한다. 고유한 임시 label과 시험 디렉터리 아래의 plist를 쓰며, `~/Library/LaunchAgents`는 쓰지 않고, 모든 경로에서 bootout한다. +- **단위 시험**: `launchctl print` parsing, escape와 crash 전용 재시작을 포함한 plist 생성, release id 규칙, 이 build에 컴파일된 호환성. +- **실제 설치**: 사용자 확인이 필요한 qualification 호스트의 설치는 패키지 manifest, status 보고, 실행 중인 바이너리 hash와 함께 C09 완료 증거로 기록한다. + C02 증거는 양쪽 실제 OS socket UID/PID 관측, 분리된 consumer·관리 자격, helper-role 인증 거절, 현재·미래 wire fixture의 엄격한 decoding, 64 KiB frame, 세션 32개 제한, idle 대기를 포함한 frame별 절대 250 ms 기한, 부분·느린·마지막 응답과 private 자격 FD 전달을 포함한다. 전용 subprocess helper는 후속 exec 전 FD 닫기를 관측하고 argv·환경·debug·출력의 secret 누출을 확인한다. Parent test가 helper를 실행하며 별도의 ignored test를 독립 qualification 성공으로 세지 않는다. 0개가 아닌 parent case 수와 프로세스 정리를 함께 기록한다. Framing은 poll·descriptor O_NONBLOCK·호출별 nonblocking I/O로 Darwin timeout 옵션 변경 없이 peer 종료 뒤 버퍼 데이터를 보존한다. 느린 writer와 느린 reader를 모두 시험한다. 인증과 닫힌 등록 응답은 boot/start 정체성, native 등록/Principal, lease, OS 정책, helper 권한·launch를 입증하지 않으며 해당 범위는 P2/P3까지 미검증이다. system_tasks >= 48도 검증된 회계 추정치이지 kernel task 제한이나 측정된 충분성이 아니다. 같은 schema 1에서 이전 값 16을 거절하는 시험을 명시하며 자동 migration을 의미하지 않는다. diff --git a/docs/milestones.md b/docs/milestones.md index 2fd2ba0..d3f159a 100644 --- a/docs/milestones.md +++ b/docs/milestones.md @@ -7,14 +7,14 @@ The [detailed planning index](planning/README.md) owns the 46 proposed commit un | Milestone | Deliverable and gate | Current implementation | |---|---|---| | DG-0 | Independent repository, approved design, resource and compatibility contracts, durable state transitions, fake-backend fault tests, reproducible 1.95.0 validation | Implemented; use the exact-source qualification report for validation status | -| DG-1 | Real macOS host probes, daemon and CLI, launch gate, Cargo/generic adapters, parent-budget candidate tests, safe service update/repair, three measured SLO runs per target backend | In progress: C01 configuration/storage, C02 authenticated foreground transport, C03 native boot/process/pressure evidence, C04 cooperative policy/scope evidence, C05 fenced launch helper, C06 reconciliation, C07 command-line owner and C08 Cargo adapters; self-use and SLO unqualified | +| DG-1 | Real macOS host probes, daemon and CLI, launch gate, Cargo/generic adapters, parent-budget candidate tests, safe service update/repair, three measured SLO runs per target backend | In progress: C01 configuration/storage, C02 authenticated foreground transport, C03 native boot/process/pressure evidence, C04 cooperative policy/scope evidence, C05 fenced launch helper, C06 reconciliation, C07 command-line owner, C08 Cargo adapters and C09 installation as the current user's LaunchAgent; self-use and SLO unqualified | | CS-RG | Full-SHA consumer pin, pre-spawn slots, PrepareExec/ExecPrepared, approval preservation, bounded control/data lanes and replay, InProcess/UDS parity and upstream regression qualification | Not started | | P1-RECOVERY | Opt-in Gateway restart/reconnection while an independent Runner retains processes and I/O; reconcile workspace, approvals and DevGuard leases | Not started; depends on CS-RG; InProcess and Runner-loss I/O restoration excluded | | DG-LINUX | Actual Linux controller, ancestor capacity, complete sandbox/proxy scope, control protection and reclaim evidence | Not started; required for product completion | | DG-CACHE | Registered-root lease/reclaim exclusion, protected artifacts/evidence, interrupted trash sweep and measured physical recovery | Not started; not a P1 prerequisite | | DG-ADAPTERS | Tool-specific Python, Node/Bun, make/ninja and container/VM verification, without language branches in core | Not started; not a P1 prerequisite | -DG-0's SQLite and fake-backend tests do not qualify a running resource governor. The `devguardd` bootstrap/path/check commands and foreground `serve` with authenticated status and native C03 host evidence are available. With native evidence the service opens registration over the wire, fenced launch through `devguard-launch` and reconciliation. The `devguard` command-line owner (C07) runs commands through that service, with Cargo adapters (C08). CodeSpace resources settings and Runner wire changes remain future work; see [operations](operations.md). The approved design's examples must not be represented as currently runnable commands. +DG-0's SQLite and fake-backend tests do not qualify a running resource governor. The `devguardd` bootstrap/path/check commands and foreground `serve` with authenticated status and native C03 host evidence are available. With native evidence the service opens registration over the wire, fenced launch through `devguard-launch` and reconciliation. The `devguard` command-line owner (C07) runs commands through that service, with Cargo adapters (C08). A packaged release can be installed as the current user's LaunchAgent (C09). CodeSpace resources settings and Runner wire changes remain future work; see [operations](operations.md). The approved design's examples must not be represented as currently runnable commands. Record design acceptance, implementation and platform qualification separately. Do not change Linux `not_run` to passed because a fake cgroup test passed on macOS, or call normal Cargo bootstrap self-governed development. The reports from `scripts/validate.py` include those distinctions explicitly. diff --git a/docs/operations.md b/docs/operations.md index 3cb6c95..7d89321 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -2,7 +2,7 @@ [English](operations.md) | [한국어](ko/operations.md) -C01 provides explicit bootstrap/canonical storage, C02 authenticated local transport, C03 native macOS boot, process and host pressure evidence, C04 cooperative policy readback and scope evidence, C05 the fenced launch helper, C06 reconciliation, C07 the `devguard` command-line owner and C08 the Cargo adapters. PR and post-merge main delivery evidence is tracked separately from implementation. `devguardd serve` runs a foreground service that activates the journal with that evidence and then opens registration over the wire, fenced launch and reconciliation. `devguard exec` runs commands through that service, and `devguard doctor` diagnoses it. The normal authority is currently macOS-only; Linux CI checks portable contracts and bounded fixtures, not DG-LINUX controls. +C01 provides explicit bootstrap/canonical storage, C02 authenticated local transport, C03 native macOS boot, process and host pressure evidence, C04 cooperative policy readback and scope evidence, C05 the fenced launch helper, C06 reconciliation, C07 the `devguard` command-line owner, C08 the Cargo adapters and C09 installation of a packaged release as the current user's LaunchAgent. PR and post-merge main delivery evidence is tracked separately from implementation. `devguardd serve` runs a foreground service that activates the journal with that evidence and then opens registration over the wire, fenced launch and reconciliation. `devguard exec` runs commands through that service, and `devguard doctor` diagnoses it. The normal authority is currently macOS-only; Linux CI checks portable contracts and bounded fixtures, not DG-LINUX controls. ## Available commands @@ -39,7 +39,17 @@ When native evidence is available, `serve` derives the policy from the observed On a platform without native evidence, or when the macOS observation fails, `serve` keeps the storage-only closed behavior and reports which case applies. -Authenticated status reports `storage_validated`, `registration_ready`, `execution_ready`, a reason and the configuration fingerprint. With native evidence registration and execution are ready, and admission still follows host pressure and capacity. Otherwise both are false, and the reason distinguishes an unsupported platform from a failed native observation. `devguard` finds `devguard-launch` beside itself, so build them together. Installation, a LaunchAgent and repair remain future work. Configuration or a successful handshake alone does not govern commands. Necessary bootstrap builds are not self-use evidence. Do not use the disposable `target` path for a persistent service; protected installation is C09 work. +Authenticated status reports `storage_validated`, `registration_ready`, `execution_ready`, a reason and the configuration fingerprint. With native evidence registration and execution are ready, and admission still follows host pressure and capacity. Otherwise both are false, and the reason distinguishes an unsupported platform from a failed native observation. `devguard` finds `devguard-launch` beside itself, so build them together. Configuration or a successful handshake alone does not govern commands. Necessary bootstrap builds are not self-use evidence. Do not run a persistent service from the disposable `target` path; install a package instead. + +A persistent service runs an installed release: + +```sh +python3 scripts/package.py --offline +target/package//bin/devguardd install --package target/package/ +devguardd status +``` + +`package.py` builds a release package from a clean tree. `install` must run from that package. It refuses while an authority serves, while the agent exists, or without the existing authority state. It copies the release to an immutable `releases/` and starts it as the current user's LaunchAgent `io.github.novelkr.devguard`, which runs `releases//bin/devguardd serve`. It selects the release only after verifying that launchd runs exactly that binary. `devguardd status` reports the installed service and exits 1 unless it runs the verified current release. Replacing a release is an upgrade, and repair is future work (C11). See [contracts](contracts.md#installation-and-the-current-user-service). `devguard exec [--project ID] [--adapter auto|generic|cargo|cargo-pipeline] [--wait DURATION] [--cpu MILLICPU] [--memory SIZE] [--tasks N] [--receipt PATH] -- PROGRAM [ARGS...]` admits the command, starts it through `devguard-launch` and waits for it. The program and its arguments follow `--`, and no shell is used. It exits with the command's status or ends by its signal, and exits 125 when nothing was started. Without `--wait` a denial ends it at once; `--wait` retries capacity and pressure denials until the deadline, and refuses at once a request larger than the host's work capacity. `--receipt` writes a private JSON receipt. `devguard doctor` prints a JSON diagnosis, and `--require admission,registration,macos-cooperative` makes it fail unless each requirement holds. See [contracts](contracts.md#command-line-owner). @@ -113,6 +123,7 @@ python3 scripts/qualify.py dg1-launch --offline python3 scripts/qualify.py dg1-reconcile --offline python3 scripts/qualify.py dg1-cli --offline python3 scripts/qualify.py dg1-cargo --offline +python3 scripts/qualify.py dg1-bootstrap --offline python3 scripts/validate.py --offline ``` @@ -202,6 +213,17 @@ Argument, preparation, entry-point and scripted-authority tests cover strict par Each Cargo launch and pipeline script inherits only its standard descriptors, except the caller's own descriptor-pair jobserver when one is inherited. The shim also records the arguments and the jobserver and fallback variables each Cargo received. Where the host's work capacity cannot fit the Cargo jobs a case needs, as on the hosted macOS 14 runner, the case is recorded as `not_run` and the suite reports `incomplete`; CI runs it with `--allow-incomplete`. -These suites leave Linux enforcement, self-use and foreground SLO `not_run`. The full validator retains the 44 original tests and validates all eight workspace crates and their explicit dependency graph. The launch crate depends on the daemon only for its tests' isolated authorities. The CLI depends on the daemon's paths and configuration and on the Cargo adapter, and on the daemon's fixtures only in tests. The Cargo adapter depends only on the contract. Core/contract remain independent of daemon configuration and native adapters, and client does not depend on core. +These suites leave Linux enforcement, self-use and foreground SLO `not_run`. `dg1-bootstrap` also runs only on macOS. Its tests copy the test binary into each package as its `devguardd`, so the installer really runs from its package. The installed service is that copy, serving an isolated fixture authority. It checks: +- an installed release verified running before it is selected, with immutable release and recovery copies and a healthy status +- an identical release reused, and a different release under the same id refused without touching the installed one +- packages refused before anything is written: a modified, extra, missing or symlinked binary, other capabilities, an inconsistent scope, and an installer that is not the package's own +- refusal while an authority serves, or without the authority state +- a service that never proves itself, which is unloaded with nothing selected +- concurrent installers, which leave exactly one service +- under launchd itself, a transient job with a unique label: a crash restarted onto the same journal, a SIGTERM stop, and a start with an unreadable journal that fails closed and is not restarted + +A session without a launchd gui domain records the launchd case as `not_run`, and the suite reports `incomplete`. + +The full validator retains the 44 original tests and validates all eight workspace crates and their explicit dependency graph. The daemon depends on itself only to enable its fixtures in its own tests. The launch crate depends on the daemon only for its tests' isolated authorities. The CLI depends on the daemon's paths and configuration and on the Cargo adapter, and on the daemon's fixtures only in tests. The Cargo adapter depends only on the contract. Core/contract remain independent of daemon configuration and native adapters, and client does not depend on core. -To roll back this boundary, stop its task-owned foreground process and select a source/artifact compatible with the preserved configuration and schema-1 journal. Keep persistent state and credentials. A C02 artifact can reopen the same state: C03 activation adds no record type and only runs the existing recovery. From C06, workloads can start through the normal service. Before rolling back to an artifact without reconciliation, stop starting new work and let charged attempts reach a terminal phase. A C05 artifact reads the same journal but keeps launch closed and does not reconcile, so any remaining attempt stays charged and Suspect. If the service stops while scopes run, start it again: their attempts stay Suspect and charged until their owners report unclaimed grants or a reboot proves termination. C07 changes no service state: to roll back the CLI, stop starting commands through it. Commands it already started stay charged until their scopes end, and it never falls back to unmanaged execution. C08 changes no service state either: to stop fitting Cargo's jobs, select `--adapter generic` explicitly; existing targets and caches are kept. Never delete the journal or its tombstones to make a rollback or restart succeed. +To roll back this boundary, stop its task-owned foreground process and select a source/artifact compatible with the preserved configuration and schema-1 journal. Keep persistent state and credentials. A C02 artifact can reopen the same state: C03 activation adds no record type and only runs the existing recovery. From C06, workloads can start through the normal service. Before rolling back to an artifact without reconciliation, stop starting new work and let charged attempts reach a terminal phase. A C05 artifact reads the same journal but keeps launch closed and does not reconcile, so any remaining attempt stays charged and Suspect. If the service stops while scopes run, start it again: their attempts stay Suspect and charged until their owners report unclaimed grants or a reboot proves termination. C07 changes no service state: to roll back the CLI, stop starting commands through it. Commands it already started stay charged until their scopes end, and it never falls back to unmanaged execution. C08 changes no service state either: to stop fitting Cargo's jobs, select `--adapter generic` explicitly; existing targets and caches are kept. Before a release is verified, the C09 installer boots its job out and removes the plist, and nothing is selected. To stop the installed service, run `launchctl bootout gui//io.github.novelkr.devguard` and remove the plist. Releases, recovery copies, the selection and the journal are kept, and a preserved artifact's foreground `devguardd serve` reads the same state. Never delete the journal or its tombstones to make a rollback or restart succeed. diff --git a/docs/planning/milestones/DG-1.md b/docs/planning/milestones/DG-1.md index b2c4441..28294cf 100644 --- a/docs/planning/milestones/DG-1.md +++ b/docs/planning/milestones/DG-1.md @@ -4,7 +4,7 @@ Owner: DevGuard. Current implementation: `in-progress`; qualification: `not-run` All work IDs, commit titles and logical PR labels below are **proposed values**, not future SHAs or GitHub numbers. Module paths describe planned responsibilities until implemented. Each added workspace crate updates the explicit dependency allowlist in the same PR without removing full-graph validation. The ledger owns actual status. -Implemented behavior: C01 provides canonical paths, explicit bootstrap and locked journal validation. C02 adds foreground `devguardd serve`, authenticated bounded UDS communication, OS-observed UID/PID, strict client compatibility and private credential-FD transfer. C03 adds the `devguard-macos` boot clock, native PID/start identity, host capacity and a two-second pressure sampler that activates the journal in `serve`. C04 adds cooperative QoS/nice application with readback, observed process-group scopes with sticky escape and tracking loss, and identity-checked termination. C05 adds registration over the wire and the fenced `devguard-launch` helper: one claimed helper per grant, scope binding and authorization before READY and exec, a separate transcript and exec-failure report, and payload descriptor hygiene. C06 adds the service reconciler: releases only on observed scope termination, an owner's report that no helper exists or a previous boot; owner observation before reap; scope termination; instance retirement; and Suspect accounting across a restart. With native evidence the service opens registration, launch and reconciliation. C07 adds the `devguard` command-line owner: managed execution with explicit bounded waits, terminal and signal forwarding, observation before reap, receipts and doctor diagnostics, and never an unmanaged fallback. C08 adds the Cargo adapters: compiler jobs fitted to the reservation, with clamping and refusals, and one jobserver shared across a pipeline's and nested Cargo runs. PR and post-merge main delivery evidence is tracked separately. See [operations](../../operations.md). +Implemented behavior: C01 provides canonical paths, explicit bootstrap and locked journal validation. C02 adds foreground `devguardd serve`, authenticated bounded UDS communication, OS-observed UID/PID, strict client compatibility and private credential-FD transfer. C03 adds the `devguard-macos` boot clock, native PID/start identity, host capacity and a two-second pressure sampler that activates the journal in `serve`. C04 adds cooperative QoS/nice application with readback, observed process-group scopes with sticky escape and tracking loss, and identity-checked termination. C05 adds registration over the wire and the fenced `devguard-launch` helper: one claimed helper per grant, scope binding and authorization before READY and exec, a separate transcript and exec-failure report, and payload descriptor hygiene. C06 adds the service reconciler: releases only on observed scope termination, an owner's report that no helper exists or a previous boot; owner observation before reap; scope termination; instance retirement; and Suspect accounting across a restart. With native evidence the service opens registration, launch and reconciliation. C07 adds the `devguard` command-line owner: managed execution with explicit bounded waits, terminal and signal forwarding, observation before reap, receipts and doctor diagnostics, and never an unmanaged fallback. C08 adds the Cargo adapters: compiler jobs fitted to the reservation, with clamping and refusals, and one jobserver shared across a pipeline's and nested Cargo runs. C09 installs a packaged release as the current user's LaunchAgent: a manifest of hashes and compiled compatibility, immutable release and recovery copies, and selection only after launchd is verified to run that release's binary. PR and post-merge main delivery evidence is tracked separately. See [operations](../../operations.md). ## PR sequence and activation @@ -17,7 +17,7 @@ Implemented behavior: C01 provides canonical paths, explicit bootstrap and locke | DG1-P5 | DG1-C09, DG1-C10, DG1-C11 | DG1-P4 | | DG1-P6 | DG1-C12 | DG1-P5 | -Available regression: `python3 scripts/validate.py --offline` (Rust 1.95.0 contract regression; fake backends do not prove native behavior). C01 authority, C02 authentication/transport, C03 native probe, C04 native scope, C05 native launch, C06 native reconciliation, C07 CLI and C08 Cargo suites are now available; commands explicitly labelled planned below remain unavailable until implemented. Each PR must supply real fixtures, nonzero case counts, logs and cleanup, then update command availability. See [verification](../verification.md). +Available regression: `python3 scripts/validate.py --offline` (Rust 1.95.0 contract regression; fake backends do not prove native behavior). C01 authority, C02 authentication/transport, C03 native probe, C04 native scope, C05 native launch, C06 native reconciliation, C07 CLI, C08 Cargo and C09 installation suites are now available; commands explicitly labelled planned below remain unavailable until implemented. Each PR must supply real fixtures, nonzero case counts, logs and cleanup, then update command availability. See [verification](../verification.md). DG1-P1 keeps runtime readiness closed; P2 provides actual probes; P3 ships launch with safe cleanup; P4 provides development entrypoints; P5 installation/parent-budget/repair; P6 measures and promotes. Through C08 use foreground daemons and minimum one-job/one-thread bootstrap. Preserve the P4 bundle outside disposable output. At C10, first test and freeze a parent containing parent-budget support, then immediately begin bounded real self-use; C12 alone establishes SLO qualification. @@ -136,7 +136,7 @@ DG1-P1 keeps runtime readiness closed; P2 provides actual probes; P3 ships launc - Completion evidence: Running binary hashes match manifest, service PID/endpoint and explicit functional-only artifact scope. - Rollback: Select a preserved compatible copy on pre-start failure; never replace an active journal with stale snapshots. - Handoff: DG1-C10 receives protected installation and independent repair. Do not assume the C09 artifact already supports C10 parent operations. -- Verification command: available regression above plus **planned, not yet provided** `python3 scripts/qualify.py dg1-bootstrap`. +- Verification command: available regression above plus **available** `python3 scripts/qualify.py dg1-bootstrap --offline` (macOS only; other platforms record `not_run`). A session without a launchd gui domain records the launchd case as `not_run`. ### DG1-C10 — constrain candidate authorities within parent leases diff --git a/docs/planning/verification.md b/docs/planning/verification.md index 4a8fbb1..50a1268 100644 --- a/docs/planning/verification.md +++ b/docs/planning/verification.md @@ -34,6 +34,7 @@ python3 scripts/qualify.py dg1-launch --offline python3 scripts/qualify.py dg1-reconcile --offline python3 scripts/qualify.py dg1-cli --offline python3 scripts/qualify.py dg1-cargo --offline +python3 scripts/qualify.py dg1-bootstrap --offline git diff --check ``` @@ -71,7 +72,7 @@ For the preparation PRs, preserve runtime/Cargo/journal state. Documentation che ## Planned suites and fault injection -`scripts/qualify.py dg1-authority --offline` is available for C01 configuration/storage, `scripts/qualify.py dg1-auth --offline` for C02 authentication/transport, `scripts/qualify.py dg1-probes --offline` for C03 native evidence, `scripts/qualify.py dg1-scopes --offline` for C04 policy and scope evidence, `scripts/qualify.py dg1-launch --offline` for the C05 launch helper, `scripts/qualify.py dg1-reconcile --offline` for C06 reconciliation, `scripts/qualify.py dg1-cli --offline` for the C07 command-line owner, and `scripts/qualify.py dg1-cargo --offline` for the C08 Cargo adapters. Other DevGuard suites and CodeSpace `scripts/qualify-devguard.py ` remain planned interfaces until supplied by their work units. Each implementation PR supplies the actual interface, nonzero case inventory, timeouts, logs, isolation and cleanup, then updates its task command documentation. macOS/Ubuntu CI retains the full validator and both portable functional suites, preserving their separate reports and logs. Native suites run on macOS CI only and record `not_run` elsewhere; they never pass on a platform that cannot supply the evidence. Each native stage declares the raw receipts it must produce. A receipt that records a case as `not_run` makes the suite `incomplete`, not `passed`. +`scripts/qualify.py dg1-authority --offline` is available for C01 configuration/storage, `scripts/qualify.py dg1-auth --offline` for C02 authentication/transport, `scripts/qualify.py dg1-probes --offline` for C03 native evidence, `scripts/qualify.py dg1-scopes --offline` for C04 policy and scope evidence, `scripts/qualify.py dg1-launch --offline` for the C05 launch helper, `scripts/qualify.py dg1-reconcile --offline` for C06 reconciliation, `scripts/qualify.py dg1-cli --offline` for the C07 command-line owner, `scripts/qualify.py dg1-cargo --offline` for the C08 Cargo adapters, and `scripts/qualify.py dg1-bootstrap --offline` for C09 installation. Other DevGuard suites and CodeSpace `scripts/qualify-devguard.py ` remain planned interfaces until supplied by their work units. Each implementation PR supplies the actual interface, nonzero case inventory, timeouts, logs, isolation and cleanup, then updates its task command documentation. macOS/Ubuntu CI retains the full validator and both portable functional suites, preserving their separate reports and logs. Native suites run on macOS CI only and record `not_run` elsewhere; they never pass on a platform that cannot supply the evidence. Each native stage declares the raw receipts it must produce. A receipt that records a case as `not_run` makes the suite `incomplete`, not `passed`. C03 evidence is recorded in these files: - **Raw receipts** (in the report's `raw/` directory): boot ID and clock readings with units, host capacity, repeated process identities, including zombie, reaped and refused observations, native pressure readings with their derived rates, the measured time from the last sample to closed admission, and the service loop's time from an injected failure to Critical and its behavior with a stuck probe. @@ -113,6 +114,12 @@ C08 evidence covers: - **Real processes**: real Cargo builds of small offline workspaces through the CLI owner and the real helper. A compiler wrapper times each compilation, and a `cargo` shim records the descriptors each launch inherited. Cases whose Cargo jobs the host's work capacity cannot fit are recorded as `not_run`. - **Unit tests**: the job estimate; argument parsing around the subcommand and `--`; jobs values resolved as Cargo reads them; precedence, clamping and refusals, also under a jobserver; the fallback variable; inherited-jobserver checks for closed, close-on-exec, descriptor-pair and FIFO references; the FIFO pool, its size limit and its token count; and adapter selection. +C09 evidence covers: +- **Raw receipts**: a verified installation with its running PID, executable and hash; a reused identical release and a refused different one; refused packages; refusals while an authority serves or without state; an unverified service unloaded with nothing selected; concurrent installers; and the launchd lifecycle, with its crash restart, stop and fail-closed start. +- **Real processes**: the test binary copied into each package as its `devguardd` serves an isolated fixture authority. A fake manager starts it as launchd would. The native case uses launchd itself with a unique transient label, a plist under the test directory, never `~/Library/LaunchAgents`, and a bootout on every path. +- **Unit tests**: `launchctl print` parsing, plist rendering with escaping and the crash-only restart, release id rules, and this build's compiled compatibility. +- **Real installation**: the installation on the qualification host, which needs the user's confirmation, is recorded with the package manifest, the status report and the running binary's hash as C09 completion evidence. + C02 evidence covers actual OS socket UID/PID observations at both ends, distinct consumer/admin credentials, rejected helper-role authentication, strict current/future wire fixtures, 64 KiB frames, a 32-session limit, absolute 250 ms per-frame deadlines including idle waits, partial/slow/final responses and private credential-FD transport. Dedicated subprocess helpers verify FD closure before a subsequent exec and inspect argv/environment/debug/output for secret leakage. They are executed by parent tests and are not independent ignored qualification successes. Record process cleanup as well as the nonzero parent-case inventory. The framing implementation uses `poll` with descriptor `O_NONBLOCK` and per-call nonblocking I/O, preserving buffered data after peer closure without Darwin timeout-option mutation. Test slow readers as well as slow writers. Authentication and a closed registration response do not prove boot/start identity, native registration/principals, leases, OS policy application, helper authorization or launch. Those remain unqualified until P2/P3. Likewise, `system_tasks >= 48` is a validated accounting estimate, not a kernel task cap or a measured sufficiency claim. Explicitly test rejection of the previous value 16 under unchanged schema 1; no automatic migration is implied. diff --git a/docs/translations.json b/docs/translations.json index 590dda6..cd3e8fa 100644 --- a/docs/translations.json +++ b/docs/translations.json @@ -56,8 +56,8 @@ "id": "planning-milestones-dg-1", "source": "docs/planning/milestones/DG-1.md", "translation": "docs/ko/planning/milestones/DG-1.md", - "reviewed_source_sha256": "ada6d3eec2ffdebd9bb5e7afdebf2c8f0e095fdd610444b5bae9e0b33aa3a3ab", - "reviewed_translation_sha256": "2f8fa73dcb46238f23d660fb74db8ea4f212b921acb189d0795e93de357423e6" + "reviewed_source_sha256": "65ad40fe3febc141551f4e893365e60f3be54bd0a1243a7e4a5ad999255ee0a3", + "reviewed_translation_sha256": "af00fecfb3c3f701782584d00bf1655c2085c0df9e7e89bbcb17759f6481b9df" }, { "id": "planning-milestones-dg-adapters", @@ -98,22 +98,22 @@ "id": "planning-verification", "source": "docs/planning/verification.md", "translation": "docs/ko/planning/verification.md", - "reviewed_source_sha256": "02c4c3bd32e9dbe2f18cc3c0fe6ef7b745171dd207c7b67a55e1ff8f1dbb51c4", - "reviewed_translation_sha256": "2272f073d9284d5b3061ace985534f61e923c4e0294867c8ac77053af8fa2188" + "reviewed_source_sha256": "d95b7391badceadc7cffc824a18e5016086f31c817114b5fd2208695f4ab30d6", + "reviewed_translation_sha256": "cba8d5fd716105abcb31c6ef93cdae82993e91719f7255d5c2d8bd065eedbca4" }, { "id": "operations", "source": "docs/operations.md", "translation": "docs/ko/operations.md", - "reviewed_source_sha256": "f614b4451ed324ac8c8fd6f57134e9a82cfd6eb91727eb2c53be0bddca802cbb", - "reviewed_translation_sha256": "d8447f11bb7d5403f631b3f021caf75ae6c282005df28a92961cb830cd093d9e" + "reviewed_source_sha256": "b83172718e3e16969aa06e967afba6b6e02cd5040fac2e3b6b7a6667ebb142de", + "reviewed_translation_sha256": "dd0b3611f0f82913689be628dc86a356812bc577da6391af37fb16bf209ddf15" }, { "id": "contracts", "source": "docs/contracts.md", "translation": "docs/ko/contracts.md", - "reviewed_source_sha256": "f264a1099bba6bba0b6b66ad030b2ba58dbdc244c345f847db16ae76fbeecbe9", - "reviewed_translation_sha256": "f8ec5b7375eaee021995a38a1518d606f7389230d2ff914b14251f1b3d48b8b3" + "reviewed_source_sha256": "004fb8e882980b9b0f2cfff9e6a0ed63f0b87dae6cd67d3b434f06ec683a59e4", + "reviewed_translation_sha256": "bd5ea1ba73c4b0e2e5aa5d27ccc34cea9d1ff376b535005596390be9262c0fa1" } ] } diff --git a/scripts/package.py b/scripts/package.py new file mode 100644 index 0000000..febca00 --- /dev/null +++ b/scripts/package.py @@ -0,0 +1,89 @@ +#!/usr/bin/env python3 +"""Build a DevGuard release package for installation as the current user's LaunchAgent. + +The package holds `devguardd`, `devguard` and `devguard-launch` from one release build of a +clean tree, and a manifest of their hashes, the build's compiled compatibility and its source +provenance. Install it with `/bin/devguardd install --package `; the installer +verifies every hash and refuses to run from any other build. A package is a functional artifact: +it is not SLO-qualified until DG1-C12 measures it. +""" +import argparse +import datetime +import hashlib +import json +import os +from pathlib import Path +import shutil +import subprocess +import sys + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from validate import PINNED_RUST, source_fingerprint # noqa: E402 + +ROOT = Path(__file__).resolve().parents[1] +BINARIES = ["devguardd", "devguard", "devguard-launch"] +PACKAGES = ["devguard-daemon", "devguard-cli", "devguard-launch"] + + +def capture(args, environment=None): + return subprocess.check_output(args, cwd=ROOT, text=True, env=environment).strip() + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--offline", action="store_true") + parser.add_argument("--output", help="the package directory (default target/package/)") + args = parser.parse_args() + if capture(["git", "status", "--porcelain"]): + raise SystemExit("package requires a clean tree") + environment = dict(os.environ) + environment.setdefault("CARGO_BUILD_JOBS", "1") + rustc = capture(["rustc", "-vV"], environment) + if f"release: {PINNED_RUST}" not in rustc: + raise SystemExit(f"package requires Rust {PINNED_RUST}") + source = source_fingerprint() + command = ["cargo", "build", "--release", "--locked"] + for package in PACKAGES: + command += ["-p", package] + if args.offline: + command.append("--offline") + subprocess.run(command, cwd=ROOT, env=environment, check=True) + built = ROOT / "target" / "release" + compatibility = json.loads(capture([str(built / "devguardd"), "version", "--json"])) + artifacts = {} + for name in BINARIES: + data = (built / name).read_bytes() + artifacts[name] = {"sha256": hashlib.sha256(data).hexdigest(), "bytes": len(data)} + digest = hashlib.sha256(json.dumps(artifacts, sort_keys=True).encode()).hexdigest() + commit = source["head"] + release_id = f"{compatibility['package_version']}-{commit[:7]}-{digest[:8]}" + manifest = { + "schema": "devguard-release-manifest/v1", + "release_id": release_id, + "version": compatibility["package_version"], + "source": {"commit": commit, "tree": capture(["git", "rev-parse", "HEAD^{tree}"]), + "tree_digest": source["tree_digest"], "clean": True}, + "build": {"profile": "release", "command": command, + "environment": {"CARGO_BUILD_JOBS": environment["CARGO_BUILD_JOBS"]}, + "bootstrap": True, "rustc": rustc, "cargo": capture(["cargo", "-V"], environment)}, + "artifacts": artifacts, + "compatibility": compatibility, + "scope": "functional", + "slo_qualified": False, + "created_at": datetime.datetime.now(datetime.timezone.utc).isoformat(timespec="seconds"), + } + output = Path(args.output) if args.output else ROOT / "target" / "package" / release_id + if output.exists(): + raise SystemExit(f"refusing to overwrite {output}") + (output / "bin").mkdir(parents=True) + for name in BINARIES: + shutil.copy2(built / name, output / "bin" / name) + os.chmod(output / "bin" / name, 0o755) + if hashlib.sha256((output / "bin" / name).read_bytes()).hexdigest() != artifacts[name]["sha256"]: + raise SystemExit(f"the packaged {name} does not match its build") + (output / "MANIFEST.json").write_text(json.dumps(manifest, indent=2) + "\n") + print(json.dumps({"package": str(output), "release_id": release_id})) + + +if __name__ == "__main__": + main() diff --git a/scripts/qualify.py b/scripts/qualify.py index 2f64e2c..e74e90d 100644 --- a/scripts/qualify.py +++ b/scripts/qualify.py @@ -82,6 +82,12 @@ "nested-cargo", "inherited-jobserver", "inherited-pipe-jobserver", "stale-jobserver", "concurrent-consumers", "pipeline-cancelled"]), ], + "dg1-bootstrap": [ + ("install-logic", ["-p", "devguard-daemon", "--lib", "install::tests"]), + ("native-install", ["-p", "devguard-daemon", "--test", "install"], + ["install-verified", "install-reuse", "install-refusals", "install-preconditions", + "install-unverified", "install-concurrent", "launchd-lifecycle"]), + ], "dg1-reconcile": [ ("reconcile-contract", ["-p", "devguard-core", "--test", "authority_contract", "reconcile_"]), ("launcher-evidence", ["-p", "devguard-macos", "--lib", "backend::tests"]), @@ -110,12 +116,13 @@ "dg1-launch": "native macOS launch helper through an isolated authority: one claimed helper per grant, and scope binding and authorization before READY and exec", "dg1-cli": "the devguard command-line owner against isolated authorities with the real launch helper: preserved argv, directory, environment and exit status, signal forwarding and terminal job control, observation before reap, refusals, bounded waits, projects, the instance pool and doctor diagnostics", "dg1-cargo": "the Cargo adapters through the devguard owner against isolated authorities with real Cargo builds: jobs fitted to the reservation, clamping and refusals, one jobserver shared by a pipeline and by nested Cargo, inherited and stale jobservers, concurrent consumers and cancellation; Cargo jobs are compilation parallelism, not a cap on test threads or measured memory", + "dg1-bootstrap": "installation of a release as the current user's LaunchAgent against isolated authorities: package validation, immutable release and recovery copies, verification that launchd runs the release's own binary before it is selected, refusals, concurrent installers, and under launchd itself a crash restart, a SIGTERM stop and a fail-closed start that is not restarted; functional artifacts only, not SLO qualification", "dg1-reconcile": "native macOS reconciliation through isolated authorities: prepared cancellation and expiry, owner reports that no helper exists, releases only on scope termination, sticky escape and tracking loss, scope termination signals, dead owners, and a daemon crash with restart", } # Suites that start real workloads through the launch helper (in fixtures). LAUNCHING = {"dg1-launch", "dg1-reconcile", "dg1-cli", "dg1-cargo"} # Native suites observe the actual host; elsewhere they are not run, never passed. -NATIVE = {"dg1-probes", "dg1-scopes", "dg1-launch", "dg1-reconcile", "dg1-cli", "dg1-cargo"} +NATIVE = {"dg1-probes", "dg1-scopes", "dg1-launch", "dg1-reconcile", "dg1-cli", "dg1-cargo", "dg1-bootstrap"} def host_facts(): diff --git a/scripts/validate.py b/scripts/validate.py index 8d2b9be..4c1b41c 100644 --- a/scripts/validate.py +++ b/scripts/validate.py @@ -68,7 +68,9 @@ def dependency_boundary(offline, environment, output): "devguard-contract": set(), "devguard-core": {"devguard-contract"}, "devguard-macos": {"devguard-contract", "devguard-core"}, - "devguard-daemon": {"devguard-contract", "devguard-core", "devguard-client", "devguard-macos"}, + # The self edge only enables the daemon's fixtures in its own tests. + "devguard-daemon": {"devguard-contract", "devguard-core", "devguard-client", "devguard-macos", + "devguard-daemon"}, "devguard-client": {"devguard-contract"}, # The daemon edge is a test-only dependency for isolated fixture authorities. "devguard-launch": {"devguard-contract", "devguard-client", "devguard-macos", "devguard-daemon"}, @@ -85,6 +87,10 @@ def dependency_boundary(offline, environment, output): edges = {d["name"] for d in package["dependencies"]} & roots if edges != allowed[package["name"]]: raise RuntimeError("workspace dependency boundary changed: " + package["name"]) + daemon = next(p for p in packages if p["name"] == "devguard-daemon") + if any(d["name"] == "devguard-daemon" and (d.get("kind") != "dev" or d.get("features") != ["test-fixtures"]) + for d in daemon["dependencies"]): + raise RuntimeError("devguard-daemon may depend on itself only to enable its fixtures in tests") # The launch crate uses the daemon only for isolated test authorities. launch = next(p for p in packages if p["name"] == "devguard-launch") if any(d["name"] == "devguard-daemon" and d.get("kind") != "dev" for d in launch["dependencies"]): From 11acc804146e6385ac6c98f8e72b823220c1bbf1 Mon Sep 17 00:00:00 2001 From: Seongjae Date: Fri, 25 Sep 2026 00:09:04 +0900 Subject: [PATCH 2/7] feat(self-use): constrain candidate authorities within parent leases (DG1-C10) Let the stable authority lend one bounded budget to a candidate build's verification instead of issuing the host's budget a second time. - Parent leases in core: AdmitLease charges its whole budget against the host at the current pressure and returns a one-time token (only its digest is journaled). AdmitChild admits an ordinary attempt against the lease's remainder, never the host, with the token; a wrong token or unknown lease is Unauthorized, a child beyond the remainder is denied ResourceUnavailable and a fenced lease denies InvalidTransition. A lease is Active, Ending (ended by its owner, its owner gone or of an earlier boot, or its deadline passed) and Released once no child is charged; a Suspect child keeps it charged. Additive tables leases and lease_children keep journal schema 1; committed capacity counts lease budgets and non-child attempts. A generation with an unreleased lease cannot be retired. - Wire: parent_lease is stated only to a client that requires it. LeaseStatus is a token-only holder session that makes no other request. The reconciler settles leases on every pass. - Candidate authorities: `devguardd candidate --id --lease --capacity --token-fd` reads the lease token from a private descriptor, confirms the lease with its parent, initializes new state under candidates/ and serves admission within the leased capacity (no host headroom, only its own daemon reserved). It states neither fenced launch nor parent leases, refuses launch and lease requests, names its lease in its status and closes when the lease is no longer active, failing closed when the parent cannot confirm it. - CLI: `devguard exec --lease KEY --lease-token-fd N` admits a command as a lease child; the token descriptor is consumed before anything else, so the workload never inherits it. `devguard test-candidate` admits one lease, runs a candidate tree's build, applicable tests and its own candidate authority as lease children through the stable helper, checks the candidate's admission from outside, ends the lease, waits for its release, removes the candidate's area and writes report.json. Workloads that create their own process groups stay out of lease children. Tests: the core lease contract, strict lease wire fixtures, service lease sessions, served candidates and their refusals, entry points, the plan, and real lease children and candidate processes under isolated parents. qualify.py gains dg1-self-use; CI runs it on macOS. Docs: contracts, operations, DG-1 ledger and verification (English authority, reviewed Korean), README and milestones. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 7 +- README.md | 5 +- crates/cli/src/args.rs | 156 +++- crates/cli/src/authority.rs | 76 +- crates/cli/src/candidate.rs | 985 ++++++++++++++++++++++ crates/cli/src/exec.rs | 49 +- crates/cli/src/lib.rs | 24 +- crates/cli/src/preflight.rs | 2 + crates/cli/src/receipt.rs | 8 + crates/cli/tests/candidate.rs | 362 ++++++++ crates/cli/tests/entrypoint.rs | 19 + crates/cli/tests/support/mod.rs | 42 +- crates/client/src/credential.rs | 5 + crates/client/src/lib.rs | 47 +- crates/client/src/protocol.rs | 39 +- crates/client/tests/wire_compatibility.rs | 96 +++ crates/contract/src/lib.rs | 59 ++ crates/core/src/authority.rs | 332 +++++++- crates/core/src/journal.rs | 222 ++++- crates/core/src/lib.rs | 4 +- crates/core/tests/lease_contract.rs | 337 ++++++++ crates/daemon/src/candidate.rs | 773 +++++++++++++++++ crates/daemon/src/config.rs | 83 +- crates/daemon/src/fixture.rs | 42 +- crates/daemon/src/install.rs | 5 +- crates/daemon/src/lib.rs | 1 + crates/daemon/src/main.rs | 67 +- crates/daemon/src/paths.rs | 34 + crates/daemon/src/server.rs | 408 ++++++++- crates/daemon/tests/entrypoint.rs | 37 + docs/contracts.md | 45 +- docs/ko/contracts.md | 45 +- docs/ko/operations.md | 25 +- docs/ko/planning/milestones/DG-1.md | 6 +- docs/ko/planning/verification.md | 9 +- docs/milestones.md | 4 +- docs/operations.md | 25 +- docs/planning/milestones/DG-1.md | 6 +- docs/planning/verification.md | 9 +- docs/translations.json | 16 +- scripts/qualify.py | 24 +- 41 files changed, 4396 insertions(+), 144 deletions(-) create mode 100644 crates/cli/src/candidate.rs create mode 100644 crates/cli/tests/candidate.rs create mode 100644 crates/core/tests/lease_contract.rs create mode 100644 crates/daemon/src/candidate.rs diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index bfa2955..028feb6 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -58,6 +58,9 @@ jobs: # Transient launchd jobs need the user's gui domain; a runner without # one records the launchd case not run, and the summary shows incomplete. run: python3 scripts/qualify.py dg1-bootstrap --allow-incomplete --output target/qualification/dg1-bootstrap + - name: Native parent lease and candidate functional checks + if: runner.os == 'macOS' + run: python3 scripts/qualify.py dg1-self-use --output target/qualification/dg1-self-use - name: Preserve qualification evidence if: always() uses: actions/upload-artifact@v6 @@ -72,9 +75,9 @@ jobs: import json, os from pathlib import Path with open(os.environ['GITHUB_STEP_SUMMARY'], 'a') as out: - for name in ['ci', 'dg1-authority', 'dg1-auth', 'dg1-probes', 'dg1-scopes', 'dg1-launch', 'dg1-reconcile', 'dg1-cli', 'dg1-cargo', 'dg1-bootstrap']: + for name in ['ci', 'dg1-authority', 'dg1-auth', 'dg1-probes', 'dg1-scopes', 'dg1-launch', 'dg1-reconcile', 'dg1-cli', 'dg1-cargo', 'dg1-bootstrap', 'dg1-self-use']: path = Path('target/qualification') / name / 'report.json' status = json.loads(path.read_text())['status'] if path.exists() else 'not_run' out.write(name + ': ' + status + '\n\n') - out.write('Contract regression, service/UDS, native host evidence, native scope, launch helper, reconciliation, command-line owner, Cargo adapter and installation functional checks only; native suites run on macOS. Self-use and foreground SLO qualification are not run.\n') + out.write('Contract regression, service/UDS, native host evidence, native scope, launch helper, reconciliation, command-line owner, Cargo adapter, installation and parent lease functional checks only; native suites run on macOS. Real self-use under an installed parent and foreground SLO qualification are not run.\n') PY diff --git a/README.md b/README.md index 10df3d5..544f2be 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ DevGuard centralizes resource admission for development workloads while preserving the resources needed to inspect and stop them. -The repository implements **DG-0 contracts and a durable authority core**, with DG-1 now in progress. C01 supplies canonical paths/bootstrap/storage checks; C02 supplies authenticated, bounded local communication with a small client and private credential-FD transfer; C03 supplies the native macOS boot clock, process identity and host pressure evidence that activate the service's journal; C04 supplies cooperative QoS/nice application with readback and observed process-group scope evidence; C05 supplies registration over the wire and the fenced `devguard-launch` helper; C06 supplies reconciliation, and with native evidence the service opens all three; C07 supplies the `devguard` command-line owner with doctor diagnostics, receipts and explicit bounded waits; C08 supplies the Cargo adapters, which fit compiler jobs to the reservation and share one jobserver; C09 installs a packaged release as the current user's LaunchAgent, selected only after launchd is verified to run it. PR and post-merge main delivery evidence is tracked separately from implementation. Linux cgroups are not yet available. Use the operating guide for actual command availability; the design also contains future interfaces. +The repository implements **DG-0 contracts and a durable authority core**, with DG-1 now in progress. C01 supplies canonical paths/bootstrap/storage checks; C02 supplies authenticated, bounded local communication with a small client and private credential-FD transfer; C03 supplies the native macOS boot clock, process identity and host pressure evidence that activate the service's journal; C04 supplies cooperative QoS/nice application with readback and observed process-group scope evidence; C05 supplies registration over the wire and the fenced `devguard-launch` helper; C06 supplies reconciliation, and with native evidence the service opens all three; C07 supplies the `devguard` command-line owner with doctor diagnostics, receipts and explicit bounded waits; C08 supplies the Cargo adapters, which fit compiler jobs to the reservation and share one jobserver; C09 installs a packaged release as the current user's LaunchAgent, selected only after launchd is verified to run it; C10 supplies parent leases, whose children are admitted only against the lease, and candidate authorities bounded by them. PR and post-merge main delivery evidence is tracked separately from implementation. Linux cgroups are not yet available. Use the operating guide for actual command availability; the design also contains future interfaces. - [Authoritative design reference](docs/design.md) · [Korean translation](docs/ko/design.md) - [Historical approved design (Korean, immutable)](docs/design.ko.md) @@ -27,11 +27,12 @@ python3 scripts/qualify.py dg1-reconcile --offline # macOS only python3 scripts/qualify.py dg1-cli --offline # macOS only python3 scripts/qualify.py dg1-cargo --offline # macOS only python3 scripts/qualify.py dg1-bootstrap --offline # macOS only +python3 scripts/qualify.py dg1-self-use --offline # macOS only ``` Omit `--offline` when the locked crates have not been downloaded. Builds and tests use one Cargo job and one test thread by default. Results, source fingerprints and logs are written under `target/qualification/`. A newer compiler can be used with `--allow-toolchain-mismatch` for a supplemental check, which never counts as qualification for 1.95.0. -DG-0 tests use an explicitly fake OS backend. Their passing reports establish accounting, persistence and state-transition contracts. The additional C01–C09 suites check actual local storage, peer observations, credential transport, native macOS host evidence, cooperative scope evidence, the launch helper, reconciliation, the command-line owner, the Cargo adapters and installation within bounded fixtures. They do not qualify Linux enforcement, browser responsiveness or self-governed execution; these remain `not_run`. +DG-0 tests use an explicitly fake OS backend. Their passing reports establish accounting, persistence and state-transition contracts. The additional C01–C10 suites check actual local storage, peer observations, credential transport, native macOS host evidence, cooperative scope evidence, the launch helper, reconciliation, the command-line owner, the Cargo adapters, installation and parent leases within bounded fixtures. They do not qualify Linux enforcement, browser responsiveness or real self-use under an installed parent; these remain `not_run`. ## Repository boundaries diff --git a/crates/cli/src/args.rs b/crates/cli/src/args.rs index ddc9fb6..e8b22f3 100644 --- a/crates/cli/src/args.rs +++ b/crates/cli/src/args.rs @@ -1,7 +1,8 @@ //! Strict command-line parsing. The program and its arguments follow `--` and //! are passed as an argv vector; nothing is interpreted by a shell. -use devguard_contract::{validate_id, Error, ErrorCode, Result}; +use devguard_contract::{validate_id, AttemptKey, Error, ErrorCode, Result}; +use devguard_daemon::candidate::parse_key; use devguard_daemon::config::AdapterSelection; use std::collections::BTreeSet; use std::ffi::OsString; @@ -16,6 +17,7 @@ pub const MAX_WAIT: Duration = Duration::from_secs(24 * 60 * 60); pub enum Command { Exec(ExecArgs), Doctor(DoctorArgs), + TestCandidate(CandidateArgs), Help, Version, } @@ -29,10 +31,26 @@ pub struct ExecArgs { pub memory_bytes: Option, pub tasks: Option, pub receipt: Option, + /// Run as a child of this parent lease, admitted against its remainder + /// with the token read from `lease_token_fd`. + pub lease: Option, + pub lease_token_fd: Option, pub program: OsString, pub args: Vec, } +/// `devguard test-candidate`: a candidate tree verified under a parent lease. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct CandidateArgs { + pub candidate: PathBuf, + pub report: PathBuf, + pub cpu_milli: Option, + pub memory_bytes: Option, + pub tasks: Option, + pub ttl: Option, + pub wait: Option, +} + /// What `doctor --require` can demand of the service. #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] pub enum Requirement { @@ -175,15 +193,27 @@ pub fn parse(args: impl IntoIterator) -> Result { match first.to_str() { Some("exec") => parse_exec(raw).map(Command::Exec), Some("doctor") => parse_doctor(raw).map(Command::Doctor), + Some("test-candidate") => parse_candidate(raw).map(Command::TestCandidate), Some("--help" | "-h" | "help") if raw.next().is_none() => Ok(Command::Help), Some("--version" | "-V") if raw.next().is_none() => Ok(Command::Version), - _ => Err(usage("expected exec, doctor, --help or --version")), + _ => Err(usage( + "expected exec, doctor, test-candidate, --help or --version", + )), } } +fn descriptor(value: &str) -> Result { + let fd = parse_count(value, "descriptors are whole numbers above 2")?; + i32::try_from(fd) + .ok() + .filter(|fd| *fd > 2) + .ok_or_else(|| usage("descriptors are whole numbers above 2")) +} + fn parse_exec(mut raw: impl Iterator) -> Result { let (mut project, mut adapter, mut wait, mut cpu, mut memory, mut tasks, mut receipt) = (None, None, None, None, None, None, None); + let (mut lease, mut lease_token_fd) = (None, None); loop { let option = raw .next() @@ -203,6 +233,8 @@ fn parse_exec(mut raw: impl Iterator) -> Result { parse_count(&value(&mut raw)?, "--tasks takes a whole number")?, )?, Some("--receipt") => once(&mut receipt, PathBuf::from(value(&mut raw)?))?, + Some("--lease") => once(&mut lease, parse_key(&value(&mut raw)?)?)?, + Some("--lease-token-fd") => once(&mut lease_token_fd, descriptor(&value(&mut raw)?)?)?, _ => { return Err(usage( "unknown exec option; the program must follow `--` and no shell is used", @@ -210,6 +242,9 @@ fn parse_exec(mut raw: impl Iterator) -> Result { } } } + if lease.is_some() != lease_token_fd.is_some() { + return Err(usage("--lease and --lease-token-fd are given together")); + } let program = raw .next() .ok_or_else(|| usage("expected a program after `--`"))?; @@ -224,11 +259,49 @@ fn parse_exec(mut raw: impl Iterator) -> Result { memory_bytes: memory, tasks, receipt, + lease, + lease_token_fd, program, args: raw.collect(), }) } +fn parse_candidate(mut raw: impl Iterator) -> Result { + let (mut candidate, mut report, mut cpu, mut memory, mut tasks, mut ttl, mut wait) = + (None, None, None, None, None, None, None); + while let Some(option) = raw.next() { + match option.to_str() { + Some("--candidate") => once(&mut candidate, PathBuf::from(value(&mut raw)?))?, + Some("--report") => once(&mut report, PathBuf::from(value(&mut raw)?))?, + Some("--cpu") => once( + &mut cpu, + parse_count(&value(&mut raw)?, "--cpu takes whole millicpu")?, + )?, + Some("--memory") => once(&mut memory, parse_size(&value(&mut raw)?)?)?, + Some("--tasks") => once( + &mut tasks, + parse_count(&value(&mut raw)?, "--tasks takes a whole number")?, + )?, + Some("--ttl") => once(&mut ttl, parse_duration(&value(&mut raw)?)?)?, + Some("--wait") => once(&mut wait, parse_duration(&value(&mut raw)?)?)?, + _ => { + return Err(usage( + "test-candidate accepts --candidate, --report, --cpu, --memory, --tasks, --ttl and --wait", + )) + } + } + } + Ok(CandidateArgs { + candidate: candidate.ok_or_else(|| usage("test-candidate needs --candidate DIR"))?, + report: report.ok_or_else(|| usage("test-candidate needs --report DIR"))?, + cpu_milli: cpu, + memory_bytes: memory, + tasks, + ttl, + wait, + }) +} + fn parse_doctor(mut raw: impl Iterator) -> Result { let (mut require, mut project) = (None, None); while let Some(option) = raw.next() { @@ -276,6 +349,85 @@ mod tests { assert_eq!(exec.args, args(&["test", "--", "--nocapture"])); } + #[test] + fn lease_children_and_candidate_runs_take_explicit_options() { + let Command::Exec(exec) = parse(args(&[ + "exec", + "--lease", + "dev-cli/g/lease-1", + "--lease-token-fd", + "5", + "--", + "true", + ])) + .unwrap() else { + panic!("expected exec") + }; + assert_eq!(exec.lease.unwrap().attempt_id, "lease-1"); + assert_eq!(exec.lease_token_fd, Some(5)); + for invalid in [ + &["exec", "--lease", "dev-cli/g/lease-1", "--", "true"][..], + &["exec", "--lease-token-fd", "5", "--", "true"], + &[ + "exec", + "--lease", + "dev-cli/g", + "--lease-token-fd", + "5", + "--", + "true", + ], + &[ + "exec", + "--lease", + "dev-cli/g/l", + "--lease-token-fd", + "2", + "--", + "true", + ], + &["test-candidate", "--candidate", "/x"], + &["test-candidate", "--report", "/r"], + &[ + "test-candidate", + "--candidate", + "/x", + "--report", + "/r", + "--cpu", + "1.5", + ], + &[ + "test-candidate", + "--candidate", + "/x", + "--report", + "/r", + "--", + "true", + ], + ] { + assert!(parse(args(invalid)).is_err(), "{invalid:?}"); + } + let Command::TestCandidate(candidate) = parse(args(&[ + "test-candidate", + "--candidate", + "/x", + "--report", + "/r", + "--memory", + "4GiB", + "--ttl", + "2h", + ])) + .unwrap() else { + panic!("expected test-candidate") + }; + assert_eq!(candidate.memory_bytes, Some(4 << 30)); + assert_eq!(candidate.ttl, Some(Duration::from_secs(7_200))); + assert_eq!(candidate.wait, None); + } + #[test] fn exec_refuses_ambiguous_or_shell_like_invocations() { for invalid in [ diff --git a/crates/cli/src/authority.rs b/crates/cli/src/authority.rs index 16bcafe..5448888 100644 --- a/crates/cli/src/authority.rs +++ b/crates/cli/src/authority.rs @@ -5,12 +5,12 @@ use devguard_client::launch::HelperTicket; use devguard_client::protocol::{ - AbandonReason, CallerCredential, Hello, LaunchGrant, ServiceStatus, + AbandonReason, CallerCredential, Hello, LaunchGrant, LeaseGranted, ServiceStatus, }; use devguard_client::Client; use devguard_contract::{ - AdmissionRequest, AttemptKey, AttemptRecord, Capability, Compatibility, Error, ErrorCode, - InstanceIdentity, Result, Secret, + AdmissionRequest, AttemptKey, AttemptRecord, Budget, Capability, Compatibility, Error, + ErrorCode, InstanceIdentity, LeaseView, Result, Secret, }; use devguard_daemon::config::HostConfig; use devguard_daemon::paths::{read_private, AuthorityPaths}; @@ -32,6 +32,19 @@ pub fn compatibility() -> Compatibility { } } +/// What an owner of a parent lease, or of one of its children, needs. +pub fn lease_compatibility() -> Compatibility { + let mut compatibility = compatibility(); + compatibility.required.insert(Capability::ParentLease); + compatibility +} + +/// A parent lease this owner admits its attempts under, with its token. +pub struct LeaseHold { + pub key: AttemptKey, + pub token: Secret, +} + /// A handshake that demands nothing, for diagnostics. pub fn diagnostic_compatibility() -> Compatibility { Compatibility { @@ -52,6 +65,8 @@ pub struct Endpoint { /// This process's registered instance, unique per CLI process because an /// instance identity can never be reused. pub instance_id: String, + /// When set, admissions are children of this lease. + pub lease: Option, } /// The non-secret part of an endpoint, for receipts and diagnostics. @@ -87,6 +102,7 @@ impl Endpoint { generation, secret, instance_id: format!("cli-{}", uuid::Uuid::new_v4().simple()), + lease: None, }) } @@ -116,13 +132,55 @@ impl Endpoint { Ok(client) } - /// A fresh session registered as this process's instance. + /// A fresh session registered as this process's instance. A lease child + /// requires parent leases, so a service without them refuses at once. pub fn session(&self) -> Result { - let mut client = self.authenticated(compatibility())?; + let mut client = self.authenticated(if self.lease.is_some() { + lease_compatibility() + } else { + compatibility() + })?; client.register(self.instance_id.clone())?; Ok(client) } + /// A fresh registered session of a parent lease's owner. + fn lease_session(&self) -> Result { + let mut client = self.authenticated(lease_compatibility())?; + client.register(self.instance_id.clone())?; + Ok(client) + } + + /// Reserve `budget` from the host as a parent lease owned by this instance. + pub fn admit_lease( + &self, + key: &AttemptKey, + budget: Budget, + ttl_ms: Option, + ) -> Result { + self.lease_session()? + .admit_lease(key.clone(), budget, ttl_ms) + } + + /// Admit no further children under the lease. + pub fn end_lease(&self, key: &AttemptKey) -> Result { + self.lease_session()?.end_lease(key.clone()) + } + + /// The lease as its token holder sees it. + pub fn lease_status(&self, key: &AttemptKey, token: &Secret) -> Result { + let mut client = Client::connect( + &self.socket, + self.uid, + Compatibility { + minimum_protocol: 1, + maximum_protocol: 1, + required: BTreeSet::from([Capability::ParentLease]), + }, + )?; + client.lease_status(key.clone(), token.clone()) + } + /// Register and report the identity the authority observed. pub fn register(&self) -> Result { let mut client = self.authenticated(compatibility())?; @@ -165,7 +223,13 @@ impl Service for Endpoint { } fn admit(&self, request: AdmissionRequest) -> Result { - self.session()?.admit(request) + match &self.lease { + Some(lease) => { + self.session()? + .admit_child(lease.key.clone(), lease.token.clone(), request) + } + None => self.session()?.admit(request), + } } fn begin_launch(&self, key: &AttemptKey) -> Result { diff --git a/crates/cli/src/candidate.rs b/crates/cli/src/candidate.rs new file mode 100644 index 0000000..3a9251a --- /dev/null +++ b/crates/cli/src/candidate.rs @@ -0,0 +1,985 @@ +//! `devguard test-candidate`: verify a candidate build under a parent lease of +//! the stable authority. +//! +//! The stable CLI admits one lease from the host and runs every candidate +//! workload as a child of it, each through `devguard exec --lease` and the +//! stable launch helper: first the candidate tree's applicable build and +//! tests, then the candidate authority itself (`devguardd candidate`), whose +//! admission is checked from here. It then ends the lease, waits for the +//! candidate to close and the lease to be released, and writes a report. +//! Nothing runs outside the lease, and any failure ends the lease before +//! anything else happens. +//! +//! Workloads that create their own process groups or sessions would leave a +//! child's scope and stay Suspect under the cooperative macOS model, so the +//! candidate's native launch, CLI, terminal and scope suites are not run +//! here; they remain bootstrap and CI qualification runs. + +use crate::args::CandidateArgs; +use crate::authority::{Endpoint, Service}; +use crate::receipt::{unix_ms, Exit}; +use devguard_client::credential::CredentialHandoff; +use devguard_client::protocol::CallerCredential; +use devguard_client::Client; +use devguard_contract::{ + AdmissionRequest, AttemptKey, AttemptPhase, AttemptRecord, Budget, Capability, Compatibility, + Error, ErrorCode, LeasePhase, LeaseRecord, LeaseView, ResourceIntent, ResourceLevels, Result, + Secret, +}; +use devguard_daemon::candidate::{budget_text, key_text, CandidateSpec}; +use devguard_daemon::config::{self, HostConfig}; +use devguard_daemon::paths::{read_private, AuthorityPaths}; +use serde::Serialize; +use serde_json::{json, Value}; +use std::collections::{BTreeMap, BTreeSet}; +use std::ffi::OsString; +use std::io::Write; +use std::os::fd::RawFd; +use std::os::unix::fs::{DirBuilderExt, OpenOptionsExt}; +use std::path::PathBuf; +use std::process::{Child, Command, Stdio}; +use std::time::{Duration, Instant}; + +pub const SCHEMA: &str = "devguard-test-candidate/v1"; +const MIB: u64 = 1 << 20; +const GIB: u64 = 1 << 30; +/// The lease when none is given: two Cargo jobs, then the candidate. +pub const DEFAULT_LEASE: Budget = Budget { + cpu_milli: 2_000, + memory_bytes: 4 * GIB, + tasks: 96, +}; +/// The candidate authority's capacity, where the lease holds it. +pub const DEFAULT_CAPACITY: Budget = Budget { + cpu_milli: 1_000, + memory_bytes: GIB, + tasks: 64, +}; +pub const DEFAULT_TTL: Duration = Duration::from_secs(2 * 60 * 60); +/// Crates whose tests create no process group or session and observe no +/// host capacity, so they stay within a lease child's scope. +pub const APPLICABLE_TESTS: [&str; 4] = [ + "devguard-contract", + "devguard-core", + "devguard-client", + "devguard-cargo", +]; +/// How long the candidate may take to serve its endpoint. +const ENDPOINT_LIMIT: Duration = Duration::from_secs(60); +/// How long the candidate's admission may stay closed by its pressure. +const ADMISSION_LIMIT: Duration = Duration::from_secs(90); +/// How long the candidate may take to close once its lease has ended, and +/// again after it is asked to stop. +const CLOSE_LIMIT: Duration = Duration::from_secs(30); +/// How long the lease may take to be released once its children end. +const RELEASE_LIMIT: Duration = Duration::from_secs(60); +const FIRST_BACKOFF: Duration = Duration::from_millis(250); +const MAX_BACKOFF: Duration = Duration::from_secs(10); + +/// One governed workload, run by `devguard exec --lease` as its own process. +#[derive(Debug, Clone, Serialize)] +pub struct Workload { + pub name: String, + pub cwd: PathBuf, + /// `devguard exec` options, apart from the lease and the receipt. + pub options: Vec, + pub program: String, + pub args: Vec, + /// Variables set for the workload; the rest of the environment is + /// inherited. They are recorded, so they never carry a secret. + pub env: Vec<(String, String)>, +} + +/// The candidate authority's workload for its spec and token descriptor. +pub type DaemonWorkload = Box Workload>; + +/// What `test-candidate` runs. +pub struct Plan { + pub id: String, + pub lease: Budget, + pub ttl: Duration, + /// How long a lease refused for capacity or pressure is asked for again. + pub wait: Duration, + pub capacity: Budget, + /// Run in order, each to its end, before the candidate authority. + pub children: Vec, + pub daemon: DaemonWorkload, + pub report: PathBuf, + /// The candidate tree, when the plan was made from one. + pub tree: Option, +} + +/// The authority and the CLI a plan runs with. +pub struct Environment<'a> { + pub paths: &'a AuthorityPaths, + /// This CLI with the given arguments, as its own process. + pub cli: &'a dyn Fn(&[String]) -> Command, +} + +#[derive(Debug, Default, Serialize)] +struct LeaseSection { + key: Option, + budget: Option, + ttl_ms: u64, + admissions: u32, + granted: Option, + token_issued: bool, + ended: Option, + released: Option, +} + +#[derive(Debug, Serialize)] +struct ChildReport { + workload: Workload, + receipt: PathBuf, + log: PathBuf, + /// How the `devguard exec` process ended. + status: Option, + /// The receipt's result, reason and exit. + result: Value, + /// The admitted attempt as last observed, with its reservation. + attempt: Option, + reserved: Option, + within_lease: bool, + passed: bool, +} + +#[derive(Debug, Serialize)] +struct Report { + schema: &'static str, + managed: bool, + candidate: Value, + lease: LeaseSection, + children: Vec, + smoke: BTreeMap, + cleanup: Value, + failures: Vec, + verdict: &'static str, + started_unix_ms: u128, + finished_unix_ms: u128, +} + +fn failed(message: impl Into) -> Error { + Error::new(ErrorCode::ResourceControlUnavailable, message) +} + +fn text(error: &Error) -> String { + format!("{:?}: {}", error.code, error.message) +} + +/// The candidate authority's own credential, from its own area. +fn candidate_credential(paths: &AuthorityPaths) -> Result { + let config = HostConfig::load(paths)?; + let generation = config + .consumers + .get(crate::authority::CONSUMER) + .ok_or_else(|| failed("the candidate has no dev-cli consumer"))? + .generation + .clone(); + let bytes = read_private(&paths.cli_credential(), paths.uid(), 1024)?; + let secret = Secret::new( + String::from_utf8(bytes).map_err(|_| failed("the candidate credential is not valid"))?, + )?; + Ok(CallerCredential::Consumer { + consumer_id: crate::authority::CONSUMER.into(), + generation, + secret, + }) +} + +fn requiring(required: &[Capability]) -> Compatibility { + Compatibility { + minimum_protocol: 1, + maximum_protocol: 1, + required: required.iter().copied().collect(), + } +} + +/// A new authenticated and registered session with the candidate: every +/// frame has a short deadline, so each check uses its own. +fn candidate_session(paths: &AuthorityPaths, credential: &CallerCredential) -> Result { + let mut client = Client::connect(&paths.socket(), paths.uid(), requiring(&[]))?; + client.authenticate(credential.clone())?; + client.register("test-candidate".into())?; + Ok(client) +} + +fn request(key: AttemptKey, requested: Budget) -> AdmissionRequest { + AdmissionRequest { + key, + execution_digest: devguard_contract::digest_bytes(b"devguard test-candidate smoke"), + intent: ResourceIntent { + profile: "interactive".into(), + requested, + minimum: ResourceLevels::MACOS, + }, + } +} + +fn wait_with_limit(child: &mut Child, limit: Duration) -> Option { + let deadline = Instant::now() + limit; + loop { + match child.try_wait() { + Ok(Some(status)) => return Some(status), + Ok(None) if Instant::now() < deadline => std::thread::sleep(Duration::from_millis(50)), + _ => return None, + } + } +} + +fn status_text(status: std::process::ExitStatus) -> String { + use std::os::unix::process::ExitStatusExt; + match (status.code(), status.signal()) { + (Some(code), _) => format!("exit {code}"), + (None, Some(signal)) => format!("signal {signal}"), + _ => "unknown".into(), + } +} + +struct Orchestrator<'a> { + plan: &'a Plan, + env: &'a Environment<'a>, + report: Report, + endpoint: Option, + token: Option, + /// The candidate authority's `devguard exec`, while it may run. + candidate: Option<(Child, Workload)>, +} + +impl Orchestrator<'_> { + fn fail(&mut self, message: impl Into) { + self.report.failures.push(message.into()); + } + + fn endpoint(&self) -> Result<&Endpoint> { + self.endpoint + .as_ref() + .ok_or_else(|| failed("the authority is not open")) + } + + /// Admit the lease, asking again while the wait lasts when it does not + /// fit the host's capacity at the current pressure. + fn admit_lease(&mut self) -> Result<()> { + let endpoint = self.endpoint()?; + let key = endpoint.key(format!("lease-{}", uuid::Uuid::new_v4().simple())); + self.report.lease.key = Some(key.clone()); + let ttl_ms = u64::try_from(self.plan.ttl.as_millis()).unwrap_or(u64::MAX); + let deadline = Instant::now() + self.plan.wait; + let mut backoff = FIRST_BACKOFF; + loop { + self.report.lease.admissions += 1; + let endpoint = self.endpoint()?; + let admit = || endpoint.admit_lease(&key, self.plan.lease, Some(ttl_ms)); + // A lost reply is replayed once with the same key; a lease that + // was granted then comes back without its token. + let result = match admit() { + Err(error) if error.code == ErrorCode::ResourceControlUnavailable => admit(), + other => other, + }; + match result { + Ok(granted) => { + self.report.lease.token_issued = granted.token.is_some(); + self.report.lease.granted = Some(granted.lease); + self.token = granted.token; + if self.token.is_none() { + return Err(failed( + "the lease token was lost with a reply; the lease is ended unused", + )); + } + return Ok(()); + } + Err(error) + if error.code == ErrorCode::ResourceUnavailable + && Instant::now() + backoff < deadline => + { + std::thread::sleep(backoff); + backoff = (backoff * 2).min(MAX_BACKOFF); + } + Err(error) => return Err(error), + } + } + } + + /// Start `workload` as a lease child through `devguard exec --lease`. + /// `extra` is inherited by the workload itself. + fn start( + &mut self, + workload: &Workload, + extra: Option, + ) -> Result<(Child, PathBuf, PathBuf)> { + let key = self + .report + .lease + .key + .clone() + .ok_or_else(|| failed("no lease is held"))?; + let token = self + .token + .as_ref() + .ok_or_else(|| failed("no lease token is held"))?; + let handoff = CredentialHandoff::new(token)?; + let receipt = self + .plan + .report + .join(format!("{}.receipt.json", workload.name)); + let log = self.plan.report.join(format!("{}.log", workload.name)); + let mut args = vec![ + "exec".to_string(), + "--lease".into(), + key_text(&key), + "--lease-token-fd".into(), + handoff.descriptor().to_string(), + "--receipt".into(), + receipt.to_string_lossy().into_owned(), + ]; + args.extend(workload.options.iter().cloned()); + args.push("--".into()); + args.push(workload.program.clone()); + args.extend(workload.args.iter().cloned()); + let output = std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .custom_flags(libc::O_NOFOLLOW) + .open(&log) + .map_err(|_| failed("cannot create a workload log"))?; + let errors = output + .try_clone() + .map_err(|_| failed("cannot create a workload log"))?; + let mut command = (self.env.cli)(&args); + command + .current_dir(&workload.cwd) + .envs(workload.env.iter().map(|(name, value)| (name, value))) + .stdin(Stdio::null()) + .stdout(output) + .stderr(errors); + handoff.attach(&mut command); + if let Some(extra) = extra { + extra.attach(&mut command); + } + let child = command + .spawn() + .map_err(|_| failed(format!("cannot start devguard exec for {}", workload.name)))?; + // Dropping the command closes this process's copies of the tokens. + drop(command); + Ok((child, receipt, log)) + } + + /// Record a finished lease child from its receipt. + fn record( + &mut self, + workload: Workload, + receipt: PathBuf, + log: PathBuf, + status: Option, + ) -> bool { + let parsed: Option = std::fs::read(&receipt) + .ok() + .and_then(|bytes| serde_json::from_slice(&bytes).ok()); + let result = parsed.as_ref().map_or(Value::Null, |receipt| { + json!({"result": receipt["result"], "reason": receipt["reason"], + "exit": receipt["exit"], "lease": receipt["lease"]}) + }); + let attempt: Option = parsed.as_ref().and_then(|receipt| { + let admitted = receipt["attempts"] + .as_array() + .and_then(|attempts| { + attempts + .iter() + .rev() + .find(|entry| entry["record"]["reservation"].is_object()) + }) + .map(|entry| entry["record"].clone()); + let observed = receipt["observed_after_reap"].clone(); + let chosen = if observed.is_object() { + observed + } else { + admitted? + }; + serde_json::from_value(chosen).ok() + }); + let reserved = attempt + .as_ref() + .and_then(|attempt| attempt.reservation.as_ref()) + .map(|reservation| reservation.quantities); + let within_lease = reserved.is_some_and(|reserved| reserved.fits(self.plan.lease)); + let completed = result["result"] == "completed" && result["exit"] == json!({"code": 0}); + let passed = completed && within_lease && status.is_some_and(|status| status.success()); + if !passed { + self.fail(format!( + "the lease child {} did not complete within the lease: {}", + workload.name, result + )); + } + self.report.children.push(ChildReport { + workload, + receipt, + log, + status: status.map(status_text), + result, + attempt, + reserved, + within_lease, + passed, + }); + passed + } + + fn orchestrate(&mut self) -> Result<()> { + self.endpoint = Some(Endpoint::open(self.env.paths)?); + self.admit_lease()?; + for workload in &self.plan.children { + let (mut child, receipt, log) = self.start(workload, None)?; + let status = child.wait().ok(); + if !self.record(workload.clone(), receipt, log, status) { + return Err(failed(format!( + "{} failed; later children are not run", + workload.name + ))); + } + } + let key = self + .report + .lease + .key + .clone() + .ok_or_else(|| failed("no lease is held"))?; + let spec = CandidateSpec { + id: self.plan.id.clone(), + lease: key, + capacity: self.plan.capacity, + }; + let token = self + .token + .clone() + .ok_or_else(|| failed("no lease token is held"))?; + let handoff = CredentialHandoff::new(&token)?; + let workload = (self.plan.daemon)(&spec, handoff.descriptor()); + let (child, _, _) = self.start(&workload, Some(handoff))?; + self.candidate = Some((child, workload)); + self.smoke(&spec) + } + + fn check(&mut self, name: &str, ok: bool, detail: Value) -> Result<()> { + self.report + .smoke + .insert(name.into(), json!({"ok": ok, "detail": detail})); + if ok { + Ok(()) + } else { + Err(failed(format!( + "the candidate check {name} failed: {detail}" + ))) + } + } + + fn candidate_running(&mut self) -> bool { + self.candidate + .as_mut() + .is_some_and(|(child, _)| matches!(child.try_wait(), Ok(None))) + } + + /// Check the candidate authority from outside, through its own endpoint + /// and credential. + fn smoke(&mut self, spec: &CandidateSpec) -> Result<()> { + let paths = self.env.paths.candidate(&spec.id)?; + let deadline = Instant::now() + ENDPOINT_LIMIT; + let hello = loop { + match Client::connect(&paths.socket(), paths.uid(), requiring(&[])) { + Ok(client) => break client.hello, + Err(error) => { + if !self.candidate_running() || Instant::now() >= deadline { + return Err(failed(format!( + "the candidate never served its endpoint: {}", + text(&error) + ))); + } + std::thread::sleep(Duration::from_millis(200)); + } + } + }; + let capabilities: BTreeSet = hello.capabilities.clone(); + self.check( + "capabilities", + capabilities.contains(&Capability::DurableAdmission) + && !capabilities.contains(&Capability::FencedLaunch) + && !capabilities.contains(&Capability::ParentLease), + json!({"authority": hello.authority, "capabilities": capabilities}), + )?; + let credential = candidate_credential(&paths)?; + let status = { + let mut client = Client::connect(&paths.socket(), paths.uid(), requiring(&[]))?; + client.authenticate(credential.clone())?; + client.status()? + }; + self.check( + "status", + status.registration_ready + && !status.execution_ready + && status.reason.contains(&spec.id) + && status.reason.contains(&key_text(&spec.lease)), + json!(status), + )?; + let CallerCredential::Consumer { + consumer_id, + generation, + .. + } = &credential + else { + return Err(failed("the candidate credential is not a consumer's")); + }; + let key = |attempt: &str| AttemptKey { + consumer_id: consumer_id.clone(), + consumer_generation: generation.clone(), + attempt_id: attempt.into(), + }; + let work = spec.capacity.remaining_after(config::candidate_reservation( + config::BOOTSTRAP_SYSTEM_TASKS, + )); + let within = Budget { + cpu_milli: work.cpu_milli.min(100), + memory_bytes: work.memory_bytes.min(64 * MIB), + tasks: work.tasks.min(4), + }; + // A new authority admits nothing until its pressure is normal. + let deadline = Instant::now() + ADMISSION_LIMIT; + let mut attempts = 0; + let admitted = loop { + attempts += 1; + let record = candidate_session(&paths, &credential)? + .admit(request(key(&format!("within-{attempts}")), within))?; + if record.phase == AttemptPhase::Prepared + || record.denial != Some(ErrorCode::ResourceUnavailable) + || Instant::now() >= deadline + { + break record; + } + std::thread::sleep(Duration::from_secs(1)); + }; + let prepared = admitted.phase == AttemptPhase::Prepared; + self.check( + "admission_within_capacity", + prepared, + json!({"requested": within, "attempts": attempts, "record": admitted}), + )?; + let launch = candidate_session(&paths, &credential)?.begin_launch(admitted.key.clone()); + self.check( + "launch_refused", + matches!(&launch, Err(error) if error.code == ErrorCode::ResourcePolicyUnsupported), + json!(launch.as_ref().err()), + )?; + let cancelled = candidate_session(&paths, &credential)?.cancel(admitted.key.clone())?; + self.check( + "cancelled", + cancelled.phase == AttemptPhase::Cancelled, + json!(cancelled), + )?; + let beyond = + candidate_session(&paths, &credential)?.admit(request(key("beyond"), spec.capacity))?; + self.check( + "admission_beyond_capacity", + beyond.denial == Some(ErrorCode::ResourceUnavailable), + json!({"requested": spec.capacity, "work_capacity": work, "record": beyond}), + )?; + let nested = + candidate_session(&paths, &credential)?.admit_lease(key("nested-lease"), within, None); + self.check( + "parent_lease_refused", + matches!(&nested, Err(error) if error.code == ErrorCode::ResourcePolicyUnsupported), + json!(nested.as_ref().err()), + ) + } + + /// End the lease, let the candidate close, wait for the lease to be + /// released and remove the candidate's disposable area. Runs after every + /// outcome. + fn finish(&mut self) { + let (Some(key), Some(endpoint)) = (self.report.lease.key.clone(), self.endpoint.take()) + else { + return; + }; + self.settle(&key, &endpoint); + self.endpoint = Some(endpoint); + } + + fn settle(&mut self, key: &AttemptKey, endpoint: &Endpoint) { + if self.report.lease.granted.is_some() { + let ended = endpoint.end_lease(key).or_else(|_| endpoint.end_lease(key)); + match ended { + Ok(view) => self.report.lease.ended = Some(view), + Err(error) => self.fail(format!("cannot end the lease: {}", text(&error))), + } + } + if let Some((mut child, workload)) = self.candidate.take() { + let mut status = wait_with_limit(&mut child, CLOSE_LIMIT); + if status.is_none() { + self.fail("the candidate did not close when its lease ended; it was asked to stop"); + // SAFETY: the pid is this process's unreaped child; the CLI + // forwards the signal to the candidate's group. + unsafe { + libc::kill(child.id() as libc::pid_t, libc::SIGTERM); + } + status = wait_with_limit(&mut child, CLOSE_LIMIT); + } + let receipt = self + .plan + .report + .join(format!("{}.receipt.json", workload.name)); + let log = self.plan.report.join(format!("{}.log", workload.name)); + match status { + Some(status) => { + self.record(workload, receipt, log, Some(status)); + } + None => { + self.fail("the candidate is still running; its area is kept"); + // The child stays charged under the ended lease until its + // scope is observed empty. + self.candidate = Some((child, workload)); + } + } + } + if let Some(token) = self.token.clone() { + let deadline = Instant::now() + RELEASE_LIMIT; + loop { + match endpoint.lease_status(key, &token) { + Ok(view) if view.lease.phase == LeasePhase::Released => { + self.report.lease.released = Some(view); + break; + } + Ok(view) if Instant::now() >= deadline => { + self.fail(format!( + "the lease was not released: {:?} with {} children", + view.lease.phase, + view.children.len() + )); + self.report.lease.released = Some(view); + break; + } + Err(error) if Instant::now() >= deadline => { + self.fail(format!("cannot observe the lease: {}", text(&error))); + break; + } + _ => std::thread::sleep(Duration::from_millis(200)), + } + } + } + self.report.cleanup = if self.candidate.is_some() { + json!({"removed": false, "reason": "the candidate is still running"}) + } else { + match self.env.paths.candidate(&self.plan.id) { + Ok(paths) => { + let mut removed = Vec::new(); + for path in [paths.root().to_path_buf(), paths.runtime().to_path_buf()] { + if std::fs::symlink_metadata(&path).is_ok() { + match std::fs::remove_dir_all(&path) { + Ok(()) => removed.push(path), + Err(_) => self.fail(format!( + "cannot remove the candidate area {}", + path.display() + )), + } + } + } + json!({"removed": removed}) + } + Err(error) => json!({"removed": false, "reason": text(&error)}), + } + }; + } +} + +/// Run `plan` in `env` and report it. Exits 0 only when every child +/// completed within the lease, every candidate check passed, the candidate +/// closed with its lease and the lease was released. +pub fn run(plan: &Plan, env: &Environment) -> Exit { + let report = Report { + schema: SCHEMA, + managed: true, + candidate: json!({"id": plan.id, "capacity": plan.capacity, "tree": plan.tree}), + lease: LeaseSection { + budget: Some(plan.lease), + ttl_ms: u64::try_from(plan.ttl.as_millis()).unwrap_or(u64::MAX), + ..LeaseSection::default() + }, + children: Vec::new(), + smoke: BTreeMap::new(), + cleanup: Value::Null, + failures: Vec::new(), + verdict: "failed", + started_unix_ms: unix_ms(), + finished_unix_ms: 0, + }; + if let Err(error) = std::fs::DirBuilder::new().mode(0o700).create(&plan.report) { + eprintln!( + "devguard: cannot create the report directory {}: {error}; nothing was started", + plan.report.display() + ); + return Exit::Code(crate::exec::NOT_STARTED); + } + let mut orchestrator = Orchestrator { + plan, + env, + report, + endpoint: None, + token: None, + candidate: None, + }; + if let Err(error) = orchestrator.orchestrate() { + orchestrator.fail(text(&error)); + } + orchestrator.finish(); + let mut report = orchestrator.report; + let released = report + .lease + .released + .as_ref() + .is_some_and(|view| view.lease.phase == LeasePhase::Released); + let candidate_ran = report + .children + .iter() + .any(|child| child.workload.name == "candidate"); + if report.failures.is_empty() && released && candidate_ran { + report.verdict = "passed"; + } + report.finished_unix_ms = unix_ms(); + let path = plan.report.join("report.json"); + let written = serde_json::to_vec_pretty(&report) + .map_err(|error| std::io::Error::other(error.to_string())) + .and_then(|mut text| { + text.push(b'\n'); + let mut file = std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .open(&path)?; + file.write_all(&text)?; + file.sync_all() + }); + if let Err(error) = written { + eprintln!("devguard: cannot write the report: {error}"); + return Exit::Code(1); + } + println!( + "{}", + json!({"report": path, "verdict": report.verdict, "failures": report.failures}) + ); + Exit::Code(if report.verdict == "passed" { 0 } else { 1 }) +} + +/// The plan for a candidate tree: its build and applicable tests, then its +/// own `devguardd candidate`, all under one lease. +pub fn plan(args: &CandidateArgs, environment: &BTreeMap) -> Result { + let invalid = |message: &'static str| Error::new(ErrorCode::InvalidRequest, message); + let tree = args + .candidate + .canonicalize() + .map_err(|_| invalid("the candidate tree is not accessible"))?; + if !tree.join("Cargo.toml").is_file() || !tree.join("Cargo.lock").is_file() { + return Err(invalid( + "the candidate tree must be a Cargo workspace with a lock file", + )); + } + let lease = Budget { + cpu_milli: args.cpu_milli.unwrap_or(DEFAULT_LEASE.cpu_milli), + memory_bytes: args.memory_bytes.unwrap_or(DEFAULT_LEASE.memory_bytes), + tasks: args.tasks.unwrap_or(DEFAULT_LEASE.tasks), + }; + lease.validate_workload()?; + let jobs = devguard_cargo::required_jobs(lease)?; + let capacity = Budget { + cpu_milli: DEFAULT_CAPACITY.cpu_milli.min(lease.cpu_milli), + memory_bytes: DEFAULT_CAPACITY.memory_bytes.min(lease.memory_bytes), + tasks: DEFAULT_CAPACITY.tasks.min(lease.tasks), + }; + capacity + .remaining_after(config::candidate_reservation( + config::BOOTSTRAP_SYSTEM_TASKS, + )) + .validate_workload() + .map_err(|_| config::too_small_for_a_candidate())?; + let target = match environment.get(&OsString::from("CARGO_TARGET_DIR")) { + Some(target) => tree.join(target), + None => tree.join("target"), + }; + let daemon = target.join("debug/devguardd"); + let quantities = |budget: Budget| { + vec![ + "--cpu".to_string(), + budget.cpu_milli.to_string(), + "--memory".into(), + budget.memory_bytes.to_string(), + "--tasks".into(), + budget.tasks.to_string(), + ] + }; + let cargo = |name: &str, args: Vec, env: Vec<(String, String)>| { + let mut options = vec!["--adapter".to_string(), "cargo".into()]; + options.extend(quantities(lease)); + Workload { + name: name.into(), + cwd: tree.clone(), + options, + program: "cargo".into(), + args, + env, + } + }; + let mut build = vec!["build".to_string(), "--offline".into(), "--locked".into()]; + for package in ["devguard-daemon", "devguard-cli", "devguard-launch"] { + build.extend(["-p".to_string(), package.into()]); + } + let mut test = vec!["test".to_string(), "--offline".into(), "--locked".into()]; + for package in APPLICABLE_TESTS { + test.extend(["-p".to_string(), package.into()]); + } + let cwd = tree.clone(); + let daemon_options = { + let mut options = vec!["--adapter".to_string(), "generic".into()]; + options.extend(quantities(capacity)); + options + }; + Ok(Plan { + id: format!("c-{}", &uuid::Uuid::new_v4().simple().to_string()[..12]), + lease, + ttl: args.ttl.unwrap_or(DEFAULT_TTL), + wait: args.wait.unwrap_or(Duration::ZERO), + capacity, + children: vec![ + cargo("build", build, Vec::new()), + cargo( + "tests", + test, + vec![("RUST_TEST_THREADS".into(), jobs.to_string())], + ), + ], + daemon: Box::new(move |spec, fd| Workload { + name: "candidate".into(), + cwd: cwd.clone(), + options: daemon_options.clone(), + program: daemon.to_string_lossy().into_owned(), + args: vec![ + "candidate".into(), + "--id".into(), + spec.id.clone(), + "--lease".into(), + key_text(&spec.lease), + "--capacity".into(), + budget_text(spec.capacity), + "--token-fd".into(), + fd.to_string(), + ], + env: Vec::new(), + }), + report: args.report.clone(), + tree: Some(tree), + }) +} + +/// `devguard test-candidate` with the running `devguard` as the stable CLI. +pub fn run_args(args: &CandidateArgs, paths: &AuthorityPaths) -> Exit { + let environment: BTreeMap = std::env::vars_os().collect(); + let plan = match plan(args, &environment) { + Ok(plan) => plan, + Err(error) => { + eprintln!("devguard: {}; nothing was started", error.message); + return Exit::Code(crate::exec::NOT_STARTED); + } + }; + let executable = match std::env::current_exe() { + Ok(executable) => executable, + Err(error) => { + eprintln!("devguard: cannot locate this CLI: {error}; nothing was started"); + return Exit::Code(crate::exec::NOT_STARTED); + } + }; + let cli = move |args: &[String]| { + let mut command = Command::new(&executable); + command.args(args); + command + }; + run(&plan, &Environment { paths, cli: &cli }) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::path::Path; + + fn args(candidate: &Path) -> CandidateArgs { + CandidateArgs { + candidate: candidate.into(), + report: "/nonexistent/report".into(), + cpu_milli: None, + memory_bytes: None, + tasks: None, + ttl: None, + wait: None, + } + } + + #[test] + fn a_tree_is_planned_as_its_build_applicable_tests_and_candidate_under_one_lease() { + let tree = tempfile::tempdir().unwrap(); + assert!(plan(&args(tree.path()), &BTreeMap::new()).is_err()); + std::fs::write(tree.path().join("Cargo.toml"), "[workspace]\n").unwrap(); + std::fs::write(tree.path().join("Cargo.lock"), "").unwrap(); + let planned = plan(&args(tree.path()), &BTreeMap::new()).unwrap(); + assert_eq!(planned.lease, DEFAULT_LEASE); + assert_eq!(planned.capacity, DEFAULT_CAPACITY); + assert!(planned.capacity.fits(planned.lease)); + let names: Vec<&str> = planned.children.iter().map(|w| w.name.as_str()).collect(); + assert_eq!(names, ["build", "tests"]); + for child in &planned.children { + assert_eq!(child.program, "cargo"); + assert!(child.args.contains(&"--locked".to_string())); + assert!(child.args.contains(&"--offline".to_string())); + } + let tests = &planned.children[1]; + for package in APPLICABLE_TESTS { + assert!(tests.args.contains(&package.to_string())); + } + // Suites that create process groups or observe the host are not run. + for package in ["devguard-daemon", "devguard-cli", "devguard-macos"] { + assert!(!tests.args.contains(&package.to_string()), "{package}"); + } + assert_eq!(tests.env, [("RUST_TEST_THREADS".into(), "2".into())]); + let spec = CandidateSpec { + id: planned.id.clone(), + lease: AttemptKey { + consumer_id: "dev-cli".into(), + consumer_generation: "g".into(), + attempt_id: "lease-1".into(), + }, + capacity: planned.capacity, + }; + let daemon = (planned.daemon)(&spec, 7); + assert!(daemon.program.ends_with("target/debug/devguardd")); + assert_eq!( + daemon.args, + [ + "candidate", + "--id", + &planned.id, + "--lease", + "dev-cli/g/lease-1", + "--capacity", + "1000,1073741824,64", + "--token-fd", + "7" + ] + ); + // A lease that fits no Cargo job, or no candidate, is refused. + for (cpu, memory) in [(999, 4 * GIB), (2_000, GIB)] { + let small = CandidateArgs { + cpu_milli: Some(cpu), + memory_bytes: Some(memory), + ..args(tree.path()) + }; + assert!(plan(&small, &BTreeMap::new()).is_err(), "{cpu} {memory}"); + } + let environment = + BTreeMap::from([(OsString::from("CARGO_TARGET_DIR"), OsString::from("/t/x"))]); + let planned = plan(&args(tree.path()), &environment).unwrap(); + assert_eq!((planned.daemon)(&spec, 7).program, "/t/x/debug/devguardd"); + } +} diff --git a/crates/cli/src/exec.rs b/crates/cli/src/exec.rs index 85a01fc..01a59ea 100644 --- a/crates/cli/src/exec.rs +++ b/crates/cli/src/exec.rs @@ -7,7 +7,7 @@ //! known not to have started, and only while an explicit `--wait` lasts. use crate::args::ExecArgs; -use crate::authority::{Endpoint, Service}; +use crate::authority::{Endpoint, LeaseHold, Service}; use crate::preflight::{self, Preflight, Refusal}; use crate::receipt::{ AttemptEntry, AuthoritySummary, BudgetSummary, ExecReceipt, ExecResult, Exit, LaunchSummary, @@ -558,6 +558,13 @@ fn lossy(values: &[OsString]) -> Vec { /// Run `devguard exec` against the authority at `paths` with `helper`. pub fn run(args: ExecArgs, paths: &AuthorityPaths, helper: &Path) -> Exit { + // A lease token's descriptor is consumed first and closed on every path, + // so the executable never inherits it. + let token = args.lease_token_fd.map(|fd| { + // SAFETY: the caller passed this descriptor to the CLI for the lease + // token only; nothing else in this process owns it. + unsafe { devguard_client::credential::take_inherited(fd) } + }); let receipt_file = match &args.receipt { Some(path) => match ReceiptFile::create(path) { Ok(file) => Some(file), @@ -577,10 +584,20 @@ pub fn run(args: ExecArgs, paths: &AuthorityPaths, helper: &Path) -> Exit { }; run.receipt.wait.requested_ms = args.wait.map(|wait| wait.as_millis() as u64); run.receipt.command.args = lossy(&args.args); + run.receipt.lease = args.lease.clone(); + let lease = match (args.lease.clone(), token) { + (Some(key), Some(Ok(token))) => Ok(Some(LeaseHold { key, token })), + (_, Some(Err(error))) => Err(error), + _ => Ok(None), + }; let signals = Signals::install(); - let exit = match &signals { - Ok(signals) => execute(&mut run, &args, paths, helper, signals), - Err(error) => run.not_started(format!("cannot install signal forwarding: {error}")), + let exit = match (&signals, lease) { + (Ok(signals), Ok(lease)) => execute(&mut run, &args, lease, paths, helper, signals), + (_, Err(error)) => run.not_started(format!( + "cannot read the lease token: {}", + error_text(&error) + )), + (Err(error), _) => run.not_started(format!("cannot install signal forwarding: {error}")), }; if let Ok(signals) = &signals { run.receipt.signals.received = signals.received(); @@ -603,6 +620,7 @@ pub fn run(args: ExecArgs, paths: &AuthorityPaths, helper: &Path) -> Exit { fn execute( run: &mut Run, args: &ExecArgs, + lease: Option, paths: &AuthorityPaths, helper: &Path, signals: &Signals, @@ -610,12 +628,13 @@ fn execute( if !helper.is_absolute() || !helper.is_file() { return run.not_started(format!("the launch helper {} is missing", helper.display())); } - let endpoint = match Endpoint::open(paths) { + let mut endpoint = match Endpoint::open(paths) { Ok(endpoint) => endpoint, Err(error) => { return run.not_started(format!("cannot use the authority: {}", error_text(&error))) } }; + endpoint.lease = lease; let environment: BTreeMap = std::env::vars_os().collect(); let mut pre = match preflight::prepare(args, &endpoint.config, &environment) { Ok(pre) => pre, @@ -656,7 +675,25 @@ fn execute( capabilities: Default::default(), }, }); - if args.wait.is_some() { + if let (Some(_), Some(lease)) = (args.wait, &endpoint.lease) { + // A lease child is admitted against its lease, never the host: a + // request larger than the lease can never be admitted. + match endpoint.lease_status(&lease.key, &lease.token) { + Ok(view) => { + run.receipt.wait.lease_budget = Some(view.lease.budget); + if !pre.intent.requested.fits(view.lease.budget) { + let budget = view.lease.budget; + return run.not_started(format!( + "the request exceeds its parent lease's budget of {} mCPU, {} bytes and {} tasks, so no wait can admit it", + budget.cpu_milli, budget.memory_bytes, budget.tasks + )); + } + } + Err(error) => { + run.receipt.wait.work_capacity_unknown = Some(error_text(&error)); + } + } + } else if args.wait.is_some() { // A wait cannot make room the host does not have: refuse at once // rather than leave a denied attempt behind every backoff. match endpoint.config.observed_work_capacity(endpoint.uid) { diff --git a/crates/cli/src/lib.rs b/crates/cli/src/lib.rs index b873ab6..252e787 100644 --- a/crates/cli/src/lib.rs +++ b/crates/cli/src/lib.rs @@ -10,6 +10,7 @@ pub mod adapter; pub mod args; pub mod authority; +pub mod candidate; pub mod doctor; pub mod exec; pub mod preflight; @@ -25,16 +26,24 @@ use std::path::PathBuf; pub const USAGE: &str = "\ devguard exec [--project ID] [--adapter auto|generic|cargo|cargo-pipeline] [--wait DURATION] - [--cpu MILLICPU] [--memory SIZE] [--tasks N] [--receipt PATH] -- PROGRAM [ARGS...] + [--cpu MILLICPU] [--memory SIZE] [--tasks N] [--receipt PATH] + [--lease CONSUMER/GENERATION/ATTEMPT --lease-token-fd N] -- PROGRAM [ARGS...] devguard doctor [--require admission,registration,macos-cooperative] [--project ID] +devguard test-candidate --candidate DIR --report DIR [--cpu MILLICPU] [--memory SIZE] [--tasks N] + [--ttl DURATION] [--wait DURATION] devguard --help | --version exec admits the command at the central authority, starts it through devguard-launch under the reserved budget and reports its own exit. A command is never run unmanaged: when it cannot be admitted or started, nothing runs and devguard exits 125. --wait retries a -refused admission until the given time (ms, s, m or h) has passed. Durations and sizes are -whole numbers; sizes take KiB, MiB or GiB. The authority comes from the operating account; -there is no path or authority override."; +refused admission until the given time (ms, s, m or h) has passed. With --lease the command +is a child of that parent lease, admitted against its remainder with the token read from +descriptor N. Durations and sizes are whole numbers; sizes take KiB, MiB or GiB. The +authority comes from the operating account; there is no path or authority override. + +test-candidate admits one parent lease and runs a candidate tree's build, its applicable +tests and its own candidate authority as children of it, checks the candidate's admission, +ends the lease and writes report.json in the new report directory."; /// Run the command line `args` (without the program name). `locate` supplies /// the authority paths and the launch helper, and is consulted only by @@ -73,6 +82,13 @@ pub fn run( Exit::Code(1) } }, + args::Command::TestCandidate(candidate) => match locate() { + Ok((paths, _)) => candidate::run_args(&candidate, &paths), + Err(error) => { + eprintln!("devguard: {}; nothing was started", error.message); + Exit::Code(exec::NOT_STARTED) + } + }, } } diff --git a/crates/cli/src/preflight.rs b/crates/cli/src/preflight.rs index eee0dd3..b944c6a 100644 --- a/crates/cli/src/preflight.rs +++ b/crates/cli/src/preflight.rs @@ -324,6 +324,8 @@ mod tests { memory_bytes: memory, tasks, receipt: None, + lease: None, + lease_token_fd: None, program: "true".into(), args: Vec::new(), } diff --git a/crates/cli/src/receipt.rs b/crates/cli/src/receipt.rs index 2cd8052..a9f48aa 100644 --- a/crates/cli/src/receipt.rs +++ b/crates/cli/src/receipt.rs @@ -78,6 +78,10 @@ pub struct WaitSummary { pub work_capacity: Option, /// Why the capacity could not be checked; the authority still decides. pub work_capacity_unknown: Option, + /// The budget of the parent lease a lease child's wait was checked + /// against, instead of the host's capacity. + #[serde(skip_serializing_if = "Option::is_none")] + pub lease_budget: Option, } #[derive(Debug, Clone, Serialize)] @@ -120,6 +124,9 @@ pub struct ExecReceipt { pub budget: Option, pub project: Option, pub authority: Option, + /// The parent lease each attempt was admitted under, for a lease child. + #[serde(skip_serializing_if = "Option::is_none")] + pub lease: Option, pub attempts: Vec, pub wait: WaitSummary, pub launch: Option, @@ -156,6 +163,7 @@ impl ExecReceipt { budget: None, project: None, authority: None, + lease: None, attempts: Vec::new(), wait: WaitSummary::default(), launch: None, diff --git a/crates/cli/tests/candidate.rs b/crates/cli/tests/candidate.rs new file mode 100644 index 0000000..2160e1e --- /dev/null +++ b/crates/cli/tests/candidate.rs @@ -0,0 +1,362 @@ +#![cfg(target_os = "macos")] +//! DG1-C10: parent leases from the owner's side. Real `devguard exec --lease` +//! processes run real workloads as lease children through the real +//! `devguard-launch`, and `test-candidate` runs a real candidate authority +//! process as one of them, against isolated fixture parents. + +mod support; +use devguard_cli::candidate::{self, DaemonWorkload, Environment, Plan, Workload}; +use devguard_cli::Exit; +use devguard_client::credential::CredentialHandoff; +use devguard_client::Client; +use devguard_contract::{AttemptKey, Budget, Capability, Compatibility, LeasePhase, Secret}; +use devguard_daemon::candidate::{budget_text, key_text}; +use devguard_daemon::paths::AuthorityPaths; +use serde_json::{json, Value}; +use std::path::{Path, PathBuf}; +use std::time::Duration; +use support::*; + +#[test] +#[ignore = "the CLI process of the candidate tests"] +fn cli_child() { + support::cli_child(); +} + +#[test] +#[ignore = "the workload of the candidate tests"] +fn payload() { + support::payload(); +} + +#[test] +#[ignore = "the candidate authority of the candidate tests"] +fn candidate_child() { + support::candidate_child(); +} + +/// Fits a 3-CPU host's 500 mCPU of work capacity. +const LEASE: Budget = Budget { + cpu_milli: 450, + memory_bytes: 512 * MIB, + tasks: 96, +}; +/// Leaves 150 mCPU, 128 MiB and 16 tasks after the candidate's own daemon. +const CAPACITY: Budget = Budget { + cpu_milli: 400, + memory_bytes: 256 * MIB, + tasks: 64, +}; +const SMALL: [&str; 6] = ["--cpu", "50", "--memory", "16MiB", "--tasks", "2"]; + +fn strings(values: &[&str]) -> Vec { + values.iter().map(|value| value.to_string()).collect() +} + +fn payload_workload(name: &str, cwd: &Path, probe: Value) -> Workload { + Workload { + name: name.into(), + cwd: cwd.into(), + options: strings(&SMALL), + program: exe(), + args: probe_args("payload"), + env: vec![(PROBE.into(), probe.to_string())], + } +} + +/// The candidate authority as a lease child: this harness in its role. +fn daemon_workload(base: PathBuf, cwd: PathBuf) -> DaemonWorkload { + Box::new(move |spec, fd| Workload { + name: "candidate".into(), + cwd: cwd.clone(), + options: strings(&[ + "--cpu", + &CAPACITY.cpu_milli.to_string(), + "--memory", + &CAPACITY.memory_bytes.to_string(), + "--tasks", + &CAPACITY.tasks.to_string(), + ]), + program: exe(), + args: probe_args("candidate_child"), + env: vec![( + CANDIDATE.into(), + json!({"base": base, "id": spec.id, "lease": key_text(&spec.lease), + "capacity": budget_text(spec.capacity), "token_fd": fd}) + .to_string(), + )], + }) +} + +fn plan(work: &Path, children: Vec, daemon: DaemonWorkload) -> Plan { + Plan { + id: "c-1".into(), + lease: LEASE, + ttl: Duration::from_secs(600), + wait: Duration::ZERO, + capacity: CAPACITY, + children, + daemon, + report: work.join("report"), + tree: None, + } +} + +/// Run `plan` in this process, as the lease owner, against the fixture. +fn run_plan(fixture: &Fixture, plan: &Plan) -> (Exit, Value) { + let paths = AuthorityPaths::fixture(fixture.base()); + let base = fixture.base().to_path_buf(); + let cli = move |args: &[String]| cli_command(&base, args); + let exit = candidate::run( + plan, + &Environment { + paths: &paths, + cli: &cli, + }, + ); + let report = read_json(&plan.report.join("report.json")); + (exit, report) +} + +/// Every file a run left in `directory`, as text. +fn texts(directory: &Path) -> Vec<(PathBuf, String)> { + std::fs::read_dir(directory) + .unwrap() + .map(|entry| { + let path = entry.unwrap().path(); + let text = std::fs::read_to_string(&path).unwrap(); + (path, text) + }) + .collect() +} + +#[test] +fn a_candidate_and_its_workloads_run_as_children_of_one_lease_that_is_then_released() { + let fixture = Fixture::start(); + let work = fixture.work("work"); + let out = work.join("payload.json"); + let plan = plan( + &work, + vec![payload_workload("workload", &work, json!({"out": out}))], + daemon_workload(fixture.base().to_path_buf(), work.clone()), + ); + let (exit, report) = run_plan(&fixture, &plan); + assert_eq!(exit, Exit::Code(0), "{report:#}"); + assert_eq!(report["verdict"], "passed"); + assert_eq!(report["failures"], json!([])); + // The workload ran through the stable helper and inherited no token. + let seen = read_json(&out); + assert_eq!(seen["descriptors"], json!([0, 1, 2])); + assert_eq!(seen["pgid"], seen["pid"]); + let children = report["children"].as_array().unwrap(); + let names: Vec<&str> = children + .iter() + .map(|child| child["workload"]["name"].as_str().unwrap()) + .collect(); + assert_eq!(names, ["workload", "candidate"]); + for child in children { + assert_eq!(child["passed"], true, "{child:#}"); + assert_eq!(child["within_lease"], true); + assert_eq!(child["result"]["lease"], report["lease"]["key"]); + assert_eq!(child["attempt"]["phase"], "released"); + assert_eq!(child["attempt"]["release_reason"], "scope_terminated"); + } + assert_eq!( + children[1]["reserved"], + serde_json::to_value(CAPACITY).unwrap() + ); + for check in [ + "capabilities", + "status", + "admission_within_capacity", + "launch_refused", + "cancelled", + "admission_beyond_capacity", + "parent_lease_refused", + ] { + assert_eq!(report["smoke"][check]["ok"], true, "{check}"); + } + assert_eq!(report["lease"]["token_issued"], true); + assert_eq!(report["lease"]["ended"]["lease"]["end_reason"], "ended"); + assert_eq!(report["lease"]["released"]["lease"]["phase"], "released"); + assert_eq!(fixture.authority.committed().unwrap(), Budget::ZERO); + // The candidate's disposable area is gone, and the parent's state is its own. + let paths = AuthorityPaths::fixture(fixture.base()); + assert!(!paths.candidates().join("c-1").exists()); + assert!(!paths.candidate_runtime().join("c-1").exists()); + assert!(paths.journal().is_file()); + for (path, text) in texts(&plan.report) { + assert!(!text.contains(&fixture.secret()), "{path:?}"); + } + let log = std::fs::read_to_string(plan.report.join("candidate.log")).unwrap(); + assert!(log.contains("\"event\":\"candidate_opened\"")); + assert!(log.contains("\"closure\":\"lease_ended\"")); + record("test-candidate", report, &fixture.secret()); +} + +fn owner(fixture: &Fixture) -> Client { + let mut client = Client::connect( + &fixture.authority.socket(), + fixture.authority.uid(), + Compatibility { + minimum_protocol: 1, + maximum_protocol: 1, + required: [Capability::DurableAdmission, Capability::ParentLease] + .into_iter() + .collect(), + }, + ) + .unwrap(); + client + .authenticate(fixture.authority.consumer().unwrap()) + .unwrap(); + client.register("lease-owner".into()).unwrap(); + client +} + +fn holder(fixture: &Fixture) -> Client { + Client::connect( + &fixture.authority.socket(), + fixture.authority.uid(), + Compatibility { + minimum_protocol: 1, + maximum_protocol: 1, + required: [Capability::ParentLease].into_iter().collect(), + }, + ) + .unwrap() +} + +#[test] +fn lease_children_are_admitted_only_within_an_active_lease_with_its_token() { + let fixture = Fixture::start(); + let work = fixture.work("work"); + let lease = AttemptKey { + consumer_id: "dev-cli".into(), + consumer_generation: fixture.authority.generation(), + attempt_id: "lease-1".into(), + }; + let budget = Budget { + cpu_milli: 100, + memory_bytes: 64 * MIB, + tasks: 8, + }; + let token = owner(&fixture) + .admit_lease(lease.clone(), budget, None) + .unwrap() + .token + .unwrap(); + let run = |name: &str, token: &Secret, extra: &[&str], probe: Value| -> (i32, Value) { + let handoff = CredentialHandoff::new(token).unwrap(); + let receipt = work.join(format!("{name}.json")); + let mut options = strings(&[ + "--lease", + &key_text(&lease), + "--lease-token-fd", + &handoff.descriptor().to_string(), + "--receipt", + receipt.to_str().unwrap(), + ]); + options.extend(strings(extra)); + let options: Vec<&str> = options.iter().map(String::as_str).collect(); + let mut child = cli(&fixture, &exec_payload(&options), |command| { + quiet(command); + command.env(PROBE, probe.to_string()); + handoff.attach(command); + }); + let status = wait_exit(&mut child, EXIT_LIMIT); + (status.code().unwrap(), read_json(&receipt)) + }; + let out = work.join("within.out"); + let (code, within) = run("within", &token, &SMALL, json!({"out": out})); + assert_eq!(code, 0, "{within:#}"); + assert_eq!(within["lease"], serde_json::to_value(&lease).unwrap()); + assert_eq!(within["observed_after_reap"]["phase"], "released"); + // The token's descriptor was the CLI's alone. + assert_eq!(read_json(&out)["descriptors"], json!([0, 1, 2])); + // A child larger than the lease is denied, and no wait can admit it. + let larger = ["--cpu", "150", "--memory", "16MiB", "--tasks", "2"]; + let (code, beyond) = run("beyond", &token, &larger, json!({})); + assert_eq!(code, 125); + assert_eq!( + beyond["attempts"][0]["record"]["denial"], + "resource_unavailable" + ); + let mut waiting = vec!["--wait", "5s"]; + waiting.extend(larger); + let (code, waited) = run("beyond-wait", &token, &waiting, json!({})); + assert_eq!(code, 125); + assert_eq!(waited["wait"]["admissions"], 0); + assert_eq!( + waited["wait"]["lease_budget"], + serde_json::to_value(budget).unwrap() + ); + assert!(waited["reason"].as_str().unwrap().contains("parent lease")); + // A forged token admits nothing. + let forged = Secret::new("e".repeat(64)).unwrap(); + let (code, refused) = run("forged", &forged, &SMALL, json!({})); + assert_eq!(code, 125); + assert!( + refused["reason"].as_str().unwrap().contains("Unauthorized"), + "{refused:#}" + ); + // Once the lease has ended, a child that starts late is refused. + let ended = owner(&fixture).end_lease(lease.clone()).unwrap(); + assert_ne!(ended.lease.phase, LeasePhase::Active); + let (code, late) = run("late", &token, &SMALL, json!({})); + assert_eq!(code, 125); + assert_eq!( + late["attempts"][0]["record"]["denial"], + "invalid_transition" + ); + for (path, text) in texts(&work) { + assert!(!text.contains(token.expose()), "{path:?}"); + assert!(!text.contains(&fixture.secret()), "{path:?}"); + } + // Every child is settled, so the lease is released and returns its budget. + wait_for(Duration::from_secs(10), || { + holder(&fixture) + .lease_status(lease.clone(), token.clone()) + .unwrap() + .lease + .phase + == LeasePhase::Released + }); + assert_eq!(fixture.authority.committed().unwrap(), Budget::ZERO); + record( + "lease-children", + json!({"within": within, "beyond": beyond, "beyond_wait": waited, + "forged": refused, "late": late}), + &fixture.secret(), + ); +} + +#[test] +fn a_candidate_that_dies_fails_the_run_and_its_lease_is_still_released() { + let fixture = Fixture::start(); + let work = fixture.work("work"); + let crashing: DaemonWorkload = { + let work = work.clone(); + Box::new(move |_, _| Workload { + name: "candidate".into(), + ..payload_workload("candidate", &work, json!({"signal": libc::SIGKILL})) + }) + }; + let plan = plan(&work, Vec::new(), crashing); + let (exit, report) = run_plan(&fixture, &plan); + assert_eq!(exit, Exit::Code(1)); + assert_eq!(report["verdict"], "failed"); + let failures = report["failures"].to_string(); + assert!(failures.contains("never served its endpoint"), "{failures}"); + let candidate = &report["children"][0]; + assert_eq!(candidate["passed"], false); + assert_eq!( + candidate["result"]["exit"], + json!({"signal": libc::SIGKILL}) + ); + // The parent observed the dead candidate's scope and released it. + assert_eq!(candidate["attempt"]["phase"], "released"); + assert_eq!(report["lease"]["released"]["lease"]["phase"], "released"); + assert_eq!(fixture.authority.committed().unwrap(), Budget::ZERO); + record("test-candidate-crash", report, &fixture.secret()); +} diff --git a/crates/cli/tests/entrypoint.rs b/crates/cli/tests/entrypoint.rs index 2ee7628..8dcb8b0 100644 --- a/crates/cli/tests/entrypoint.rs +++ b/crates/cli/tests/entrypoint.rs @@ -18,6 +18,7 @@ fn help_and_version_describe_the_managed_contract() { for needle in [ "devguard exec", "devguard doctor", + "devguard test-candidate", "never run unmanaged", "no path or authority override", ] { @@ -41,6 +42,24 @@ fn malformed_invocations_start_nothing_and_exit_125() { &["exec", "--wait", "forever", "--", "true"], &["doctor", "--require", "kernel"], &["serve"], + &["exec", "--lease", "dev-cli/g/lease-1", "--", "true"], + &[ + "exec", + "--lease", + "dev-cli/g/lease-1", + "--lease-token-fd", + "200", + "--", + "true", + ], + &["test-candidate"], + &[ + "test-candidate", + "--candidate", + "/nonexistent", + "--report", + "/nonexistent/r", + ], ] { let output = devguard(args); assert_eq!(output.status.code(), Some(125), "{args:?}"); diff --git a/crates/cli/tests/support/mod.rs b/crates/cli/tests/support/mod.rs index f2d7835..9c796cb 100644 --- a/crates/cli/tests/support/mod.rs +++ b/crates/cli/tests/support/mod.rs @@ -21,6 +21,8 @@ use std::time::{Duration, Instant}; pub const CLI_CHILD: &str = "DEVGUARD_TEST_CLI"; /// Configuration for the workload role. pub const PROBE: &str = "DEVGUARD_TEST_PROBE"; +/// Configuration for the candidate authority role. +pub const CANDIDATE: &str = "DEVGUARD_TEST_CANDIDATE"; pub const MIB: u64 = 1024 * 1024; pub const EXIT_LIMIT: Duration = Duration::from_secs(30); @@ -130,14 +132,50 @@ pub fn cli_with_helper( args: &[String], configure: impl FnOnce(&mut Command), ) -> Child { + let mut command = cli_command_with_helper(base, helper, args); + configure(&mut command); + command.spawn().unwrap() +} + +/// The CLI role's command for `args` against `base`'s fixture authority, +/// not yet started. +pub fn cli_command(base: &Path, args: &[String]) -> Command { + cli_command_with_helper(base, &helper(), args) +} + +fn cli_command_with_helper(base: &Path, helper: &Path, args: &[String]) -> Command { let mut command = Command::new(exe()); command.args(probe_args("cli_child")); command.env( CLI_CHILD, json!({"base": base, "helper": helper, "args": args}).to_string(), ); - configure(&mut command); - command.spawn().unwrap() + command +} + +/// The candidate authority role: serve a candidate of the fixture authority +/// under `base`, as `devguardd candidate` serves one of the canonical +/// authority, and end as it would. +pub fn candidate_child() { + let Some(config) = std::env::var_os(CANDIDATE) else { + return; + }; + let config: Value = serde_json::from_str(&config.to_string_lossy()).unwrap(); + let base = PathBuf::from(config["base"].as_str().unwrap()); + let spec = devguard_daemon::candidate::CandidateSpec { + id: config["id"].as_str().unwrap().into(), + lease: devguard_daemon::candidate::parse_key(config["lease"].as_str().unwrap()).unwrap(), + capacity: devguard_daemon::candidate::parse_budget(config["capacity"].as_str().unwrap()) + .unwrap(), + }; + let fd = config["token_fd"].as_i64().unwrap() as i32; + match devguard_daemon::fixture::candidate_until_terminated(&base, spec, fd) { + Ok(devguard_daemon::candidate::Closure::Unconfirmed { error }) | Err(error) => { + eprintln!("candidate: {error}"); + std::process::exit(1) + } + Ok(_) => std::process::exit(0), + } } pub fn cli(fixture: &Fixture, args: &[String], configure: impl FnOnce(&mut Command)) -> Child { diff --git a/crates/client/src/credential.rs b/crates/client/src/credential.rs index 094b0df..4233480 100644 --- a/crates/client/src/credential.rs +++ b/crates/client/src/credential.rs @@ -33,6 +33,11 @@ impl CredentialHandoff { Ok(Self(unsafe { OwnedFd::from_raw_fd(fd) })) } + /// The descriptor number the child will inherit, for its arguments. + pub fn descriptor(&self) -> RawFd { + self.0.as_raw_fd() + } + /// Command owns the carrier until dropped. Only its child clears close-on-exec. pub fn attach(self, command: &mut Command) -> RawFd { let raw = self.0.as_raw_fd(); diff --git a/crates/client/src/lib.rs b/crates/client/src/lib.rs index 17b25bd..83794d3 100644 --- a/crates/client/src/lib.rs +++ b/crates/client/src/lib.rs @@ -7,8 +7,8 @@ pub mod peer; pub mod protocol; use devguard_contract::{ - AdmissionRequest, AttemptKey, AttemptRecord, Compatibility, Error, ErrorCode, InstanceIdentity, - Result, Secret, + AdmissionRequest, AttemptKey, AttemptRecord, Budget, Compatibility, Error, ErrorCode, + InstanceIdentity, LeaseView, Result, Secret, }; use protocol::*; use std::os::unix::net::UnixStream; @@ -133,6 +133,49 @@ impl Client { _ => Err(unavailable()), } } + /// Reserve a parent lease for this registered owner. Only the first + /// response carries the token; a replay returns the lease without it. + pub fn admit_lease( + &mut self, + key: AttemptKey, + budget: Budget, + ttl_ms: Option, + ) -> Result { + match self.call(Request::AdmitLease { + key, + budget, + ttl_ms, + })? { + Response::LeaseGranted(granted) => Ok(granted), + _ => Err(unavailable()), + } + } + /// Admit a child under a lease, presenting its token. + pub fn admit_child( + &mut self, + lease: AttemptKey, + token: Secret, + request: AdmissionRequest, + ) -> Result { + self.attempt(Request::AdmitChild { + lease, + token, + request, + }) + } + pub fn end_lease(&mut self, key: AttemptKey) -> Result { + match self.call(Request::EndLease { key })? { + Response::Lease(view) => Ok(view), + _ => Err(unavailable()), + } + } + /// A lease holder's view, in a session that makes no other request. + pub fn lease_status(&mut self, key: AttemptKey, token: Secret) -> Result { + match self.call(Request::LeaseStatus { key, token })? { + Response::Lease(view) => Ok(view), + _ => Err(unavailable()), + } + } fn attempt(&mut self, request: Request) -> Result { match self.call(request)? { Response::Attempt(attempt) => Ok(attempt), diff --git a/crates/client/src/protocol.rs b/crates/client/src/protocol.rs index 5b9d07e..f3fd69e 100644 --- a/crates/client/src/protocol.rs +++ b/crates/client/src/protocol.rs @@ -1,6 +1,6 @@ use devguard_contract::{ - AdmissionRequest, AttemptKey, AttemptRecord, Capability, Compatibility, Error, ErrorCode, - InstanceIdentity, Secret, + AdmissionRequest, AttemptKey, AttemptRecord, Budget, Capability, Compatibility, Error, + ErrorCode, InstanceIdentity, LeaseRecord, LeaseView, Secret, }; use serde::{Deserialize, Serialize}; use std::collections::BTreeSet; @@ -108,6 +108,39 @@ pub enum Request { instance_id: String, permit: Secret, }, + /// Reserve a parent lease of `budget` from host capacity for the + /// registered owner, fenced after `ttl_ms` if given. Sent only after the + /// service echoed the `parent_lease` capability. + AdmitLease { + key: AttemptKey, + budget: Budget, + ttl_ms: Option, + }, + /// Admit a child execution under a lease, presenting its token. The child + /// is then an ordinary attempt of the registered instance. + AdmitChild { + lease: AttemptKey, + token: Secret, + request: AdmissionRequest, + }, + /// End an owned lease: no new child, released once every child settles. + EndLease { + key: AttemptKey, + }, + /// A lease holder's view, authorized by the token alone. Like a helper, + /// a lease holder is not a caller: the session can make no other request. + LeaseStatus { + key: AttemptKey, + token: Secret, + }, +} + +/// A new lease with its token, returned once. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct LeaseGranted { + pub lease: LeaseRecord, + pub token: Option, } #[derive(Debug, Clone, Serialize, Deserialize)] @@ -231,5 +264,7 @@ pub enum Response { LaunchGranted(LaunchGrant), LaunchAuthorized(LaunchAuthorization), Terminated(Termination), + LeaseGranted(LeaseGranted), + Lease(LeaseView), Error(WireError), } diff --git a/crates/client/tests/wire_compatibility.rs b/crates/client/tests/wire_compatibility.rs index d7fd675..048bbda 100644 --- a/crates/client/tests/wire_compatibility.rs +++ b/crates/client/tests/wire_compatibility.rs @@ -269,3 +269,99 @@ fn reconcile_requests_decode_strictly_and_carry_no_process_identity() { future["body"]["value"]["future_field"] = json!(true); assert!(serde_json::from_value::>(future).is_err()); } + +fn lease() -> devguard_contract::LeaseRecord { + use devguard_contract::*; + let attempt = attempt(); + LeaseRecord { + key: attempt.key, + owner: attempt.owner, + policy_revision: "policy".into(), + budget: Budget { + cpu_milli: 2_000, + memory_bytes: 1, + tasks: 1, + }, + phase: LeasePhase::Active, + created_at: ObservationTime { + boot_id: "boot".into(), + monotonic_ms: 1, + }, + deadline_ms: Some(60_001), + end_reason: None, + } +} + +#[test] +fn lease_requests_decode_strictly_and_never_print_the_token() { + use devguard_client::protocol::LeaseGranted; + let intent = json!({"profile": "interactive", + "requested": {"cpu_milli": 1000, "memory_bytes": 1, "tasks": 1}, + "minimum": {"cpu": "cooperative", "memory": "accounted", "pids": "accounted"}}); + let budget = json!({"cpu_milli": 2000, "memory_bytes": 1, "tasks": 1}); + let child = json!({"key": key(), "execution_digest": devguard_contract::digest_bytes(b"child"), + "intent": intent}); + let requests = [ + json!({"method": "admit_lease", "params": {"key": key(), "budget": budget, "ttl_ms": 60000}}), + json!({"method": "admit_lease", "params": {"key": key(), "budget": budget, "ttl_ms": null}}), + json!({"method": "admit_child", "params": {"lease": key(), "token": PERMIT, "request": child}}), + json!({"method": "end_lease", "params": {"key": key()}}), + json!({"method": "lease_status", "params": {"key": key(), "token": PERMIT}}), + ]; + for body in requests { + let frame = json!({"version": 1, "request_id": 10, "body": body}); + assert!( + serde_json::from_value::>(frame.clone()).is_ok(), + "{frame}" + ); + // No holder can declare an identity, a phase or a larger lease. + for field in ["pid", "owner", "phase", "remaining", "future_field"] { + let mut changed = frame.clone(); + changed["body"]["params"][field] = json!(1); + assert!( + serde_json::from_value::>(changed).is_err(), + "accepted {field} in {frame}" + ); + } + } + for method in ["admit_child", "lease_status"] { + let params = if method == "admit_child" { + json!({"lease": key(), "token": "abc", "request": child}) + } else { + json!({"key": key(), "token": "abc"}) + }; + let short = + json!({"version": 1, "request_id": 11, "body": {"method": method, "params": params}}); + assert!(serde_json::from_value::>(short).is_err()); + } + let record = serde_json::to_value(lease()).unwrap(); + let responses = [ + json!({"result": "lease_granted", "value": {"lease": record, "token": PERMIT}}), + json!({"result": "lease_granted", "value": {"lease": record, "token": null}}), + json!({"result": "lease", "value": {"lease": record, + "remaining": {"cpu_milli": 1000, "memory_bytes": 1, "tasks": 1}, "children": [key()]}}), + ]; + for body in responses { + let frame = json!({"version": 1, "request_id": 12, "body": body}); + assert!( + serde_json::from_value::>(frame.clone()).is_ok(), + "{frame}" + ); + let mut future = frame.clone(); + future["body"]["value"]["future_field"] = json!(true); + assert!(serde_json::from_value::>(future).is_err()); + let mut inner = frame; + inner["body"]["value"]["lease"]["future_field"] = json!(true); + assert!(serde_json::from_value::>(inner).is_err()); + } + let granted = Response::LeaseGranted(LeaseGranted { + lease: lease(), + token: Some(devguard_contract::Secret::new(PERMIT.into()).unwrap()), + }); + assert!(!format!("{granted:?}").contains(PERMIT)); + // The capability added after protocol 1 has a stable name. + assert_eq!( + serde_json::to_value(devguard_contract::Capability::ParentLease).unwrap(), + json!("parent_lease") + ); +} diff --git a/crates/contract/src/lib.rs b/crates/contract/src/lib.rs index 488b57f..1cbcb03 100644 --- a/crates/contract/src/lib.rs +++ b/crates/contract/src/lib.rs @@ -526,6 +526,65 @@ pub enum Capability { StaticControlReservations, MacosCooperative, LinuxCgroupV2, + /// Bounded parent leases whose children are admitted against the lease, + /// not the host. A service states it only to a client that requires it, + /// so a client that cannot decode it never receives it. + ParentLease, +} + +/// A parent lease's phase. An Ending lease admits no new child and is released +/// once every child is settled. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum LeasePhase { + Active, + Ending, + Released, +} + +/// Why a lease stopped admitting children. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum LeaseEndReason { + /// Its owner ended it. + Ended, + /// Its owner process ended, or belonged to an earlier boot. + OwnerGone, + /// Its deadline passed. + Expired, +} + +/// A bounded budget reserved from the host, from which child executions are +/// admitted. The budget stays charged until the lease is released. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct LeaseRecord { + pub key: AttemptKey, + pub owner: InstanceIdentity, + pub policy_revision: String, + pub budget: Budget, + pub phase: LeasePhase, + pub created_at: ObservationTime, + /// Monotonic milliseconds in `created_at`'s boot after which no child is admitted. + pub deadline_ms: Option, + pub end_reason: Option, +} + +impl LeaseRecord { + /// A lease holds its whole budget until it is released. + pub fn charged(&self) -> bool { + self.phase != LeasePhase::Released + } +} + +/// A lease as its holders see it: its record, the budget its charged children +/// leave, and every child admitted under it. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct LeaseView { + pub lease: LeaseRecord, + pub remaining: Budget, + pub children: Vec, } #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] diff --git a/crates/core/src/authority.rs b/crates/core/src/authority.rs index 07b1c45..a6296b8 100644 --- a/crates/core/src/authority.rs +++ b/crates/core/src/authority.rs @@ -63,6 +63,14 @@ pub struct LaunchDecision { pub permit: Option, } +/// A newly admitted lease and its token, returned once; a replay of the same +/// admission returns the lease without the token. +#[derive(Debug, Clone)] +pub struct LeaseGrant { + pub lease: LeaseRecord, + pub token: Option, +} + #[derive(Debug)] pub struct RunDecision { pub attempt: AttemptRecord, @@ -173,6 +181,8 @@ impl Authority { // Storage may have waited for host readiness. Recheck atomically with // recovery so intervening corruption cannot hide a charged attempt. journal::validate_index(tx)?; + journal::ensure_lease_tables(tx)?; + journal::validate_leases(tx)?; validate_registration_policies(tx, &policy)?; journal::expire_prepared(tx, &now)?; for mut record in journal::active(tx)? { @@ -739,7 +749,8 @@ impl Authority { self.journal.transaction(|tx| { let instances: u64 = tx.query_row("SELECT COUNT(*) FROM instances WHERE consumer=?1 AND generation=?2 AND state!='retired'", params![consumer, generation], |r| r.get(0)).map_err(db_error)?; - let charged = journal::active(tx)?.iter().any(|r| r.key.consumer_id == consumer && r.key.consumer_generation == generation); + let charged = journal::active(tx)?.iter().any(|r| r.key.consumer_id == consumer && r.key.consumer_generation == generation) + || journal::active_leases(tx)?.iter().any(|l| l.key.consumer_id == consumer && l.key.consumer_generation == generation); if instances != 0 || charged { return Err(Error::new(ErrorCode::ReconciliationRequired, "generation still has live or suspect ownership")); } tx.execute("INSERT OR IGNORE INTO retired_generations(consumer,generation) VALUES (?1,?2)", params![consumer, generation]).map_err(db_error)?; tx.execute("DELETE FROM attempts WHERE consumer=?1 AND generation=?2 AND charged=0", params![consumer, generation]).map_err(db_error)?; @@ -748,6 +759,325 @@ impl Authority { } } +/// Parent leases: a bounded budget reserved from the host once, from which the +/// lease's children are admitted. Children never draw on the host again, and +/// their sum never exceeds the lease. +impl Authority { + /// Reserve `budget` from host capacity as a lease owned by `principal`, + /// fenced after `ttl_ms` if given. The token is returned once. + pub fn admit_lease( + &mut self, + principal: &Principal, + key: &AttemptKey, + budget: Budget, + ttl_ms: Option, + ) -> Result { + check_key(principal, key)?; + budget.validate_workload()?; + let now = self.clock.now(); + let target = self + .pressure + .current(&now) + .target(self.policy.work_capacity()?); + let policy_revision = self.policy.revision.clone(); + self.journal.transaction(|tx| { + validate_principal(tx, principal)?; + journal::expire_prepared(tx, &now)?; + if let Some((existing, _)) = journal::load_lease(tx, key)? { + if existing.owner != principal.instance || existing.budget != budget { + return Err(Error::new( + ErrorCode::AttemptConflict, + "lease identity has a different meaning or owner", + )); + } + return Ok(LeaseGrant { + lease: existing, + token: None, + }); + } + if !budget.fits(target.remaining_after(journal::committed(tx)?)) { + return Err(Error::new( + ErrorCode::ResourceUnavailable, + "the lease does not fit the host's capacity at the current pressure", + )); + } + let deadline_ms = match ttl_ms { + Some(ttl) => Some(now.monotonic_ms.checked_add(ttl).ok_or_else(|| { + Error::new(ErrorCode::InvalidRequest, "lease deadline overflow") + })?), + None => None, + }; + let token = Secret::new(format!( + "{}{}", + Uuid::new_v4().simple(), + Uuid::new_v4().simple() + ))?; + let lease = LeaseRecord { + key: key.clone(), + owner: principal.instance.clone(), + policy_revision, + budget, + phase: LeasePhase::Active, + created_at: now.clone(), + deadline_ms, + end_reason: None, + }; + journal::save_lease(tx, &lease, &token.digest())?; + Ok(LeaseGrant { + lease, + token: Some(token), + }) + }) + } + + /// Admit a child execution under `lease`, presented with its token, against + /// the lease's remainder. The child is then an ordinary attempt of + /// `principal`, launched and released by the usual evidence. + pub fn admit_child( + &mut self, + principal: &Principal, + lease: &AttemptKey, + token: &Secret, + request: AdmissionRequest, + ) -> Result { + let fingerprint = request.fingerprint()?; + check_key(principal, &request.key)?; + if lease.consumer_id != request.key.consumer_id + || lease.consumer_generation != request.key.consumer_generation + { + return Err(unauthorized()); + } + let now = self.clock.now(); + let policy = &self.policy; + let backend = &self.backend; + self.journal.transaction(|tx| { + validate_principal(tx, principal)?; + journal::expire_prepared(tx, &now)?; + let (mut parent, token_hash) = journal::load_lease(tx, lease)?.ok_or_else(not_found)?; + if !same_digest(&token_hash, &token.digest()) { + return Err(unauthorized()); + } + if let Some(record) = journal::load(tx, &request.key)? { + if record.request_fingerprint != fingerprint + || record.owner != principal.instance + || journal::child_lease(tx, &request.key)?.as_ref() != Some(lease) + { + return Err(Error::new( + ErrorCode::AttemptConflict, + "attempt identity has different meaning, owner or lease", + )); + } + return Ok(record); + } + if fence_if_due(&mut parent, &now) { + journal::save_lease(tx, &parent, &token_hash)?; + } + let remaining = lease_remaining(tx, &parent)?; + let plan = backend.plan(&request.intent); + let denial = if parent.phase != LeasePhase::Active { + // A fenced lease admits nothing; waiting cannot change that. + Some(ErrorCode::InvalidTransition) + } else { + match &plan { + Ok(plan) + if request.intent.profile == "interactive" + && plan.validate().is_ok() + && plan.levels().satisfies(request.intent.minimum) => + { + (!request.intent.requested.fits(remaining)) + .then_some(ErrorCode::ResourceUnavailable) + } + Err(error) if error.code == ErrorCode::ResourceControlUnavailable => { + Some(error.code) + } + _ => Some(ErrorCode::ResourcePolicyUnsupported), + } + }; + let record = AttemptRecord { + key: request.key, + request_fingerprint: fingerprint, + owner: principal.instance.clone(), + policy_revision: policy.revision.clone(), + phase: if denial.is_some() { + AttemptPhase::Denied + } else { + AttemptPhase::Prepared + }, + reservation: if denial.is_some() { + None + } else { + Some(ResourceReservation { + lease_id: Uuid::new_v4().to_string(), + quantities: request.intent.requested, + prepared_at: now.clone(), + prepare_deadline_ms: now + .monotonic_ms + .checked_add(PREPARED_TTL_MS) + .ok_or_else(|| { + Error::new(ErrorCode::InvalidRequest, "monotonic deadline overflow") + })?, + }) + }, + plan: if denial.is_some() { None } else { Some(plan?) }, + scope: None, + applied: None, + denial, + tracking_lost: false, + release_reason: None, + }; + journal::save(tx, &record)?; + journal::link_child(tx, &record.key, lease)?; + Ok(record) + }) + } + + /// End an owned lease: it admits no new child and is released once every + /// child is settled. + pub fn end_lease(&mut self, principal: &Principal, lease: &AttemptKey) -> Result { + let now = self.clock.now(); + self.journal.transaction(|tx| { + validate_principal(tx, principal)?; + journal::expire_prepared(tx, &now)?; + let (mut record, token_hash) = journal::load_lease(tx, lease)?.ok_or_else(not_found)?; + if record.owner != principal.instance { + return Err(unauthorized()); + } + let before = record.clone(); + if record.phase == LeasePhase::Active { + record.phase = LeasePhase::Ending; + record.end_reason = Some(LeaseEndReason::Ended); + } + settle(tx, &mut record)?; + if record != before { + journal::save_lease(tx, &record, &token_hash)?; + } + lease_view(tx, record) + }) + } + + /// A lease holder's view, authorized by the lease token alone. + pub fn lease_status(&mut self, lease: &AttemptKey, token: &Secret) -> Result { + lease.validate()?; + let now = self.clock.now(); + self.journal.transaction(|tx| { + journal::expire_prepared(tx, &now)?; + let (mut record, token_hash) = + journal::load_lease(tx, lease)?.ok_or_else(unauthorized)?; + if !same_digest(&token_hash, &token.digest()) { + return Err(unauthorized()); + } + if fence_if_due(&mut record, &now) { + journal::save_lease(tx, &record, &token_hash)?; + } + lease_view(tx, record) + }) + } + + /// Trusted reconciliation: fence a lease whose owner has ended, which + /// belongs to an earlier boot or whose deadline passed, and release it once + /// every child is settled. Suspect children keep it charged. + pub fn reconcile_lease( + &mut self, + lease: &AttemptKey, + owner_running: bool, + ) -> Result { + let now = self.clock.now(); + self.journal.transaction(|tx| { + journal::expire_prepared(tx, &now)?; + let (mut record, token_hash) = journal::load_lease(tx, lease)?.ok_or_else(not_found)?; + let before = record.clone(); + if record.phase == LeasePhase::Active && !owner_running { + record.phase = LeasePhase::Ending; + record.end_reason = Some(LeaseEndReason::OwnerGone); + } + fence_if_due(&mut record, &now); + settle(tx, &mut record)?; + if record != before { + journal::save_lease(tx, &record, &token_hash)?; + } + Ok(record) + }) + } + + /// Leases that still hold their budget, for the reconciler. + pub fn leases(&mut self) -> Result> { + self.journal.transaction(journal::active_leases) + } + + /// A lease and its children, for trusted diagnostics and tests. + pub fn lease(&mut self, lease: &AttemptKey) -> Result { + self.journal.transaction(|tx| { + let (record, _) = journal::load_lease(tx, lease)?.ok_or_else(not_found)?; + lease_view(tx, record) + }) + } +} + +/// Fence an Active lease of an earlier boot or past its deadline. +fn fence_if_due(lease: &mut LeaseRecord, now: &ObservationTime) -> bool { + if lease.phase != LeasePhase::Active { + return false; + } + let reason = if lease.created_at.boot_id != now.boot_id { + LeaseEndReason::OwnerGone + } else if lease + .deadline_ms + .is_some_and(|deadline| now.monotonic_ms >= deadline) + { + LeaseEndReason::Expired + } else { + return false; + }; + lease.phase = LeasePhase::Ending; + lease.end_reason = Some(reason); + true +} + +/// The budget of a lease's charged children. +fn children_charged(tx: &Transaction<'_>, lease: &LeaseRecord) -> Result { + let mut sum = Budget::ZERO; + for child in journal::lease_children(tx, &lease.key)? { + if let Some(record) = journal::load(tx, &child)? { + if let (true, Some(reservation)) = (record.phase.charged(), &record.reservation) { + sum = sum.checked_add(reservation.quantities)?; + } + } + } + Ok(sum) +} + +fn lease_remaining(tx: &Transaction<'_>, lease: &LeaseRecord) -> Result { + Ok(lease.budget.remaining_after(children_charged(tx, lease)?)) +} + +/// Release an Ending lease once none of its children is charged. +fn settle(tx: &Transaction<'_>, lease: &mut LeaseRecord) -> Result<()> { + if lease.phase != LeasePhase::Ending { + return Ok(()); + } + for child in journal::lease_children(tx, &lease.key)? { + if journal::load(tx, &child)?.is_some_and(|record| record.phase.charged()) { + return Ok(()); + } + } + lease.phase = LeasePhase::Released; + Ok(()) +} + +fn lease_view(tx: &Transaction<'_>, lease: LeaseRecord) -> Result { + let remaining = if lease.charged() { + lease_remaining(tx, &lease)? + } else { + Budget::ZERO + }; + let children = journal::lease_children(tx, &lease.key)?; + Ok(LeaseView { + lease, + remaining, + children, + }) +} + fn same_digest(left: &str, right: &str) -> bool { left.len() == right.len() && left diff --git a/crates/core/src/journal.rs b/crates/core/src/journal.rs index 11f4add..8ac8f96 100644 --- a/crates/core/src/journal.rs +++ b/crates/core/src/journal.rs @@ -11,6 +11,20 @@ use std::time::Duration; /// states it, so an incompatible downgrade can be refused before a switch. pub const JOURNAL_SCHEMA: &str = "1"; +/// Parent leases and their children. They are additive to schema 1: a reader +/// without them sees every child as an ordinary charged attempt, which only +/// over-counts, and does not hold a lease's unused remainder. +const LEASE_TABLES: &str = " + CREATE TABLE IF NOT EXISTS leases ( + consumer TEXT NOT NULL, generation TEXT NOT NULL, lease TEXT NOT NULL, + charged INTEGER NOT NULL CHECK (charged IN (0,1)), record TEXT NOT NULL, + token_hash TEXT NOT NULL, + PRIMARY KEY (consumer, generation, lease)); + CREATE TABLE IF NOT EXISTS lease_children ( + consumer TEXT NOT NULL, generation TEXT NOT NULL, attempt TEXT NOT NULL, + lease TEXT NOT NULL, + PRIMARY KEY (consumer, generation, attempt));"; + pub(crate) struct Journal { connection: Connection, } @@ -62,6 +76,7 @@ impl Journal { CREATE TABLE retired_generations ( consumer TEXT NOT NULL, generation TEXT NOT NULL, PRIMARY KEY (consumer, generation)); + {LEASE_TABLES} COMMIT;")).map_err(db_error)?; Ok(()) } @@ -280,14 +295,207 @@ pub(crate) fn expire_prepared(tx: &Transaction<'_>, now: &ObservationTime) -> Re Ok(()) } +/// Budget charged against the host: every lease's whole budget, and every +/// charged attempt that is not a lease child, whose budget its lease holds. pub(crate) fn committed(tx: &Transaction<'_>) -> Result { - active(tx)?.iter().try_fold(Budget::ZERO, |sum, record| { - sum.checked_add( - record - .reservation - .as_ref() - .expect("validated reservation") - .quantities, + let children = child_keys(tx)?; + let attempts = active(tx)? + .iter() + .filter(|record| !children.contains(&record.key)) + .try_fold(Budget::ZERO, |sum, record| { + sum.checked_add( + record + .reservation + .as_ref() + .expect("validated reservation") + .quantities, + ) + })?; + active_leases(tx)? + .iter() + .try_fold(attempts, |sum, lease| sum.checked_add(lease.budget)) +} + +/// Create the lease tables in a journal written before them. +pub(crate) fn ensure_lease_tables(tx: &Transaction<'_>) -> Result<()> { + tx.execute_batch(LEASE_TABLES).map_err(db_error) +} + +fn lease_invalid() -> Error { + Error::new( + ErrorCode::JournalInvalid, + "journal lease record is inconsistent", + ) +} + +fn validate_lease(record: &LeaseRecord) -> Result<()> { + record.key.validate().map_err(|_| lease_invalid())?; + record + .budget + .validate_workload() + .map_err(|_| lease_invalid())?; + if (record.phase == LeasePhase::Active) != record.end_reason.is_none() { + return Err(lease_invalid()); + } + Ok(()) +} + +/// A lease and the digest of its token. +pub(crate) fn load_lease( + tx: &Transaction<'_>, + key: &AttemptKey, +) -> Result> { + let row: Option<(String, bool, String)> = tx.query_row( + "SELECT record, charged, token_hash FROM leases WHERE consumer=?1 AND generation=?2 AND lease=?3", + params![key.consumer_id, key.consumer_generation, key.attempt_id], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?))) + .optional().map_err(db_error)?; + row.map(|(raw, charged, token_hash)| { + let record: LeaseRecord = decode(&raw)?; + if record.key != *key || record.charged() != charged { + return Err(lease_invalid()); + } + validate_lease(&record)?; + Ok((record, token_hash)) + }) + .transpose() +} + +/// Insert a new lease with its token digest, or update an existing lease's record. +pub(crate) fn save_lease( + tx: &Transaction<'_>, + record: &LeaseRecord, + token_hash: &str, +) -> Result<()> { + validate_lease(record)?; + tx.execute("INSERT INTO leases(consumer,generation,lease,charged,record,token_hash) VALUES (?1,?2,?3,?4,?5,?6) + ON CONFLICT(consumer,generation,lease) DO UPDATE SET charged=excluded.charged,record=excluded.record", + params![record.key.consumer_id, record.key.consumer_generation, record.key.attempt_id, record.charged(), encode(record)?, token_hash]) + .map_err(db_error)?; + Ok(()) +} + +/// Leases that still hold their budget. +pub(crate) fn active_leases(tx: &Transaction<'_>) -> Result> { + let mut statement = tx + .prepare("SELECT consumer,generation,lease,record FROM leases WHERE charged=1") + .map_err(db_error)?; + let mut rows = statement.query([]).map_err(db_error)?; + let mut records = Vec::new(); + while let Some(row) = rows.next().map_err(db_error)? { + let record: LeaseRecord = decode(&row.get::<_, String>(3).map_err(db_error)?)?; + let key = AttemptKey { + consumer_id: row.get(0).map_err(db_error)?, + consumer_generation: row.get(1).map_err(db_error)?, + attempt_id: row.get(2).map_err(db_error)?, + }; + if record.key != key || !record.charged() { + return Err(lease_invalid()); + } + validate_lease(&record)?; + records.push(record); + } + Ok(records) +} + +/// Check every lease record, as `validate_index` checks attempts. +pub(crate) fn validate_leases(tx: &Transaction<'_>) -> Result<()> { + let mut statement = tx + .prepare("SELECT consumer,generation,lease,record,charged FROM leases") + .map_err(db_error)?; + let mut rows = statement.query([]).map_err(db_error)?; + while let Some(row) = rows.next().map_err(db_error)? { + let record: LeaseRecord = decode(&row.get::<_, String>(3).map_err(db_error)?)?; + let key = AttemptKey { + consumer_id: row.get(0).map_err(db_error)?, + consumer_generation: row.get(1).map_err(db_error)?, + attempt_id: row.get(2).map_err(db_error)?, + }; + if record.key != key || record.charged() != row.get::<_, bool>(4).map_err(db_error)? { + return Err(lease_invalid()); + } + validate_lease(&record)?; + } + Ok(()) +} + +/// Record `child` as admitted under `lease`; both share one consumer generation. +pub(crate) fn link_child( + tx: &Transaction<'_>, + child: &AttemptKey, + lease: &AttemptKey, +) -> Result<()> { + tx.execute( + "INSERT INTO lease_children(consumer,generation,attempt,lease) VALUES (?1,?2,?3,?4)", + params![ + child.consumer_id, + child.consumer_generation, + child.attempt_id, + lease.attempt_id + ], + ) + .map_err(db_error)?; + Ok(()) +} + +/// The lease a child was admitted under, if it is a child. +pub(crate) fn child_lease(tx: &Transaction<'_>, child: &AttemptKey) -> Result> { + let lease: Option = tx + .query_row( + "SELECT lease FROM lease_children WHERE consumer=?1 AND generation=?2 AND attempt=?3", + params![ + child.consumer_id, + child.consumer_generation, + child.attempt_id + ], + |r| r.get(0), + ) + .optional() + .map_err(db_error)?; + Ok(lease.map(|attempt_id| AttemptKey { + consumer_id: child.consumer_id.clone(), + consumer_generation: child.consumer_generation.clone(), + attempt_id, + })) +} + +/// Every child admitted under `lease`, charged or not. +pub(crate) fn lease_children(tx: &Transaction<'_>, lease: &AttemptKey) -> Result> { + let mut statement = tx + .prepare("SELECT attempt FROM lease_children WHERE consumer=?1 AND generation=?2 AND lease=?3 ORDER BY attempt") + .map_err(db_error)?; + let rows = statement + .query_map( + params![ + lease.consumer_id, + lease.consumer_generation, + lease.attempt_id + ], + |r| r.get::<_, String>(0), ) + .map_err(db_error)?; + rows.map(|attempt_id| { + Ok(AttemptKey { + consumer_id: lease.consumer_id.clone(), + consumer_generation: lease.consumer_generation.clone(), + attempt_id: attempt_id.map_err(db_error)?, + }) }) + .collect() +} + +fn child_keys(tx: &Transaction<'_>) -> Result> { + let mut statement = tx + .prepare("SELECT consumer,generation,attempt FROM lease_children") + .map_err(db_error)?; + let rows = statement + .query_map([], |r| { + Ok(AttemptKey { + consumer_id: r.get(0)?, + consumer_generation: r.get(1)?, + attempt_id: r.get(2)?, + }) + }) + .map_err(db_error)?; + rows.collect::>() + .map_err(db_error) } diff --git a/crates/core/src/lib.rs b/crates/core/src/lib.rs index 87954e8..f118d70 100644 --- a/crates/core/src/lib.rs +++ b/crates/core/src/lib.rs @@ -7,8 +7,8 @@ mod policy; mod pressure; pub use authority::{ - Authority, AuthorityStorage, InstanceRecord, LaunchDecision, Principal, Registration, - RunDecision, TrustedPeer, + Authority, AuthorityStorage, InstanceRecord, LaunchDecision, LeaseGrant, Principal, + Registration, RunDecision, TrustedPeer, }; pub use journal::JOURNAL_SCHEMA; pub use policy::{ConsumerDefinition, ConsumerRole, Policy}; diff --git a/crates/core/tests/lease_contract.rs b/crates/core/tests/lease_contract.rs new file mode 100644 index 0000000..b14a83d --- /dev/null +++ b/crates/core/tests/lease_contract.rs @@ -0,0 +1,337 @@ +//! DG1-C10 parent leases: a budget reserved from the host once, from which the +//! lease's children are admitted. Children never draw on the host again, their +//! sum never exceeds the lease, and a lease is released only after every child +//! is settled. + +// The shared harness also serves the authority contract, whose steps this file does not all use. +#[allow(dead_code)] +mod support; +use devguard_contract::*; +use devguard_core::*; +use rusqlite::Connection; +use support::*; + +fn lease_key(id: &str) -> AttemptKey { + AttemptKey { + consumer_id: "codespace-runtime".into(), + consumer_generation: "generation-1".into(), + attempt_id: id.into(), + } +} + +fn budget(cpu_milli: u64) -> Budget { + Budget { + cpu_milli, + memory_bytes: 8 * GIB, + tasks: 256, + } +} + +fn granted(a: &mut TestAuthority, p: &Principal, id: &str, cpu_milli: u64) -> (AttemptKey, Secret) { + let key = lease_key(id); + let grant = a.admit_lease(p, &key, budget(cpu_milli), None).unwrap(); + assert_eq!(grant.lease.phase, LeasePhase::Active); + (key, grant.token.unwrap()) +} + +fn bound_child( + h: &Harness, + a: &mut TestAuthority, + p: &Principal, + lease: &AttemptKey, + token: &Secret, + id: &str, +) -> (AttemptRecord, ScopeIdentity) { + let record = a.admit_child(p, lease, token, request(id)).unwrap(); + assert_eq!(record.phase, AttemptPhase::Prepared); + let permit = a.begin_launch(p, &record.key).unwrap().permit.unwrap(); + let scope = h.provide_binding(&record); + let bound = a.bind_scope(p, &record.key, &permit, &scope).unwrap(); + (bound, scope) +} + +#[test] +fn a_lease_is_charged_once_and_its_children_draw_only_on_it() { + let h = Harness::new(); + let (mut a, p) = h.ready(); + // The host's work capacity is 7,500 mCPU; the lease takes 7,000. + let (lease, token) = granted(&mut a, &p, "lease-1", 7_000); + assert_eq!(a.committed_budget().unwrap(), budget(7_000)); + // Only 500 mCPU is left for work outside the lease. + let outside = a.admit(&p, request("outside")).unwrap(); + assert_eq!(outside.phase, AttemptPhase::Denied); + assert_eq!(outside.denial, Some(ErrorCode::ResourceUnavailable)); + // Children of 3,000 mCPU each fit the lease, not the host's remainder, + // and are not counted against the host a second time. + for id in ["child-1", "child-2"] { + let child = a.admit_child(&p, &lease, &token, request(id)).unwrap(); + assert_eq!(child.phase, AttemptPhase::Prepared, "{id}"); + assert_eq!(a.committed_budget().unwrap(), budget(7_000), "{id}"); + } + let view = a.lease(&lease).unwrap(); + assert_eq!(view.remaining.cpu_milli, 1_000); + assert_eq!(view.children.len(), 2); + // A third child would exceed the lease: its sum never passes the budget. + let third = a + .admit_child(&p, &lease, &token, request("child-3")) + .unwrap(); + assert_eq!(third.phase, AttemptPhase::Denied); + assert_eq!(third.denial, Some(ErrorCode::ResourceUnavailable)); + assert_eq!(a.lease(&lease).unwrap().remaining.cpu_milli, 1_000); + // A lease that does not fit the host is refused and never recorded. + assert_eq!( + a.admit_lease(&p, &lease_key("lease-2"), budget(1_000), None) + .unwrap_err() + .code, + ErrorCode::ResourceUnavailable + ); +} + +#[test] +fn a_child_needs_the_lease_token_its_generation_and_a_consistent_replay() { + let h = Harness::new(); + let (mut a, p) = h.ready(); + let (lease, token) = granted(&mut a, &p, "lease-1", 6_000); + let wrong = Secret::new("b".repeat(64)).unwrap(); + assert_eq!( + a.admit_child(&p, &lease, &wrong, request("child-1")) + .unwrap_err() + .code, + ErrorCode::Unauthorized + ); + let mut foreign = request("child-1"); + foreign.key.consumer_generation = "generation-2".into(); + assert_eq!( + a.admit_child(&p, &lease, &token, foreign).unwrap_err().code, + ErrorCode::Unauthorized + ); + assert_eq!( + a.admit_child(&p, &lease_key("unknown"), &token, request("child-1")) + .unwrap_err() + .code, + ErrorCode::NotFound + ); + let first = a + .admit_child(&p, &lease, &token, request("child-1")) + .unwrap(); + // A replay returns the same child; the same key as an ordinary attempt + // or under another lease conflicts. + assert_eq!( + a.admit_child(&p, &lease, &token, request("child-1")) + .unwrap(), + first + ); + let (other, other_token) = granted(&mut a, &p, "lease-2", 1_000); + assert_eq!( + a.admit_child(&p, &other, &other_token, request("child-1")) + .unwrap_err() + .code, + ErrorCode::AttemptConflict + ); + let mut changed = request("child-1"); + changed.execution_digest = digest_bytes(b"another command"); + assert_eq!( + a.admit_child(&p, &lease, &token, changed).unwrap_err().code, + ErrorCode::AttemptConflict + ); +} + +#[test] +fn replaying_a_lease_admission_returns_it_without_its_token() { + let h = Harness::new(); + let (mut a, p) = h.ready(); + let key = lease_key("lease-1"); + let first = a + .admit_lease(&p, &key, budget(4_000), Some(60_000)) + .unwrap(); + assert!(first.token.is_some()); + let replay = a + .admit_lease(&p, &key, budget(4_000), Some(60_000)) + .unwrap(); + assert_eq!(replay.lease, first.lease); + assert!(replay.token.is_none(), "the token is returned once"); + assert_eq!( + a.admit_lease(&p, &key, budget(5_000), None) + .unwrap_err() + .code, + ErrorCode::AttemptConflict + ); + assert_eq!(a.committed_budget().unwrap(), budget(4_000)); +} + +#[test] +fn lease_status_is_authorized_by_the_token_alone_and_reports_the_remainder() { + let h = Harness::new(); + let (mut a, p) = h.ready(); + let (lease, token) = granted(&mut a, &p, "lease-1", 6_000); + a.admit_child(&p, &lease, &token, request("child-1")) + .unwrap(); + let view = a.lease_status(&lease, &token).unwrap(); + assert_eq!(view.lease.phase, LeasePhase::Active); + assert_eq!(view.remaining.cpu_milli, 3_000); + assert_eq!(view.children, vec![request("child-1").key]); + let wrong = Secret::new("c".repeat(64)).unwrap(); + assert_eq!( + a.lease_status(&lease, &wrong).unwrap_err().code, + ErrorCode::Unauthorized + ); + // An unknown lease is indistinguishable from a wrong token. + assert_eq!( + a.lease_status(&lease_key("unknown"), &token) + .unwrap_err() + .code, + ErrorCode::Unauthorized + ); +} + +#[test] +fn ending_a_lease_fences_new_children_and_releases_it_after_they_settle() { + let h = Harness::new(); + let (mut a, p) = h.ready(); + let (lease, token) = granted(&mut a, &p, "lease-1", 6_000); + let (child, scope) = bound_child(&h, &mut a, &p, &lease, &token, "child-1"); + let ended = a.end_lease(&p, &lease).unwrap(); + assert_eq!(ended.lease.phase, LeasePhase::Ending); + assert_eq!(ended.lease.end_reason, Some(LeaseEndReason::Ended)); + // A fenced lease admits nothing, and waiting cannot change that. + let late = a + .admit_child(&p, &lease, &token, request("child-2")) + .unwrap(); + assert_eq!(late.phase, AttemptPhase::Denied); + assert_eq!(late.denial, Some(ErrorCode::InvalidTransition)); + // The running child keeps the lease charged. + assert_eq!( + a.reconcile_lease(&lease, true).unwrap().phase, + LeasePhase::Ending + ); + assert_eq!(a.committed_budget().unwrap(), budget(6_000)); + h.closed(&scope); + assert_eq!( + a.reconcile(&child.key).unwrap().phase, + AttemptPhase::Released + ); + let released = a.reconcile_lease(&lease, true).unwrap(); + assert_eq!(released.phase, LeasePhase::Released); + assert_eq!(a.committed_budget().unwrap(), Budget::ZERO); + assert!(a.leases().unwrap().is_empty()); +} + +#[test] +fn an_owner_that_ended_or_a_passed_deadline_fences_the_lease() { + let h = Harness::new(); + let (mut a, p) = h.ready(); + let key = lease_key("lease-1"); + let token = a + .admit_lease(&p, &key, budget(3_000), Some(1_000)) + .unwrap() + .token + .unwrap(); + h.clock.advance(1_000); + let late = a.admit_child(&p, &key, &token, request("child-1")).unwrap(); + assert_eq!(late.denial, Some(ErrorCode::InvalidTransition)); + let view = a.lease_status(&key, &token).unwrap(); + assert_eq!(view.lease.end_reason, Some(LeaseEndReason::Expired)); + // Without charged children an expired lease is released at once. + assert_eq!( + a.reconcile_lease(&key, true).unwrap().phase, + LeasePhase::Released + ); + // A lease whose owner is gone is fenced and, with no child, released. + let (orphan, _) = granted(&mut a, &p, "lease-2", 3_000); + let reconciled = a.reconcile_lease(&orphan, false).unwrap(); + assert_eq!(reconciled.phase, LeasePhase::Released); + assert_eq!(reconciled.end_reason, Some(LeaseEndReason::OwnerGone)); + assert_eq!(a.committed_budget().unwrap(), Budget::ZERO); +} + +#[test] +fn a_suspect_child_keeps_its_lease_charged() { + let h = Harness::new(); + let (mut a, p) = h.ready(); + let (lease, token) = granted(&mut a, &p, "lease-1", 6_000); + let (child, scope) = bound_child(&h, &mut a, &p, &lease, &token, "child-1"); + a.end_lease(&p, &lease).unwrap(); + h.closed(&scope); + h.backend + .0 + .lock() + .unwrap() + .observations + .get_mut(&scope.scope_id) + .unwrap() + .known_escape = true; + let suspect = a.reconcile(&child.key).unwrap(); + assert_eq!(suspect.phase, AttemptPhase::Suspect); + assert_eq!( + a.reconcile_lease(&lease, false).unwrap().phase, + LeasePhase::Ending + ); + assert_eq!(a.committed_budget().unwrap(), budget(6_000)); +} + +#[test] +fn leases_survive_a_restart_and_an_earlier_boot_fences_and_releases_them() { + let h = Harness::new(); + let (mut a, p) = h.ready(); + let (lease, token) = granted(&mut a, &p, "lease-1", 6_000); + let (child, _) = bound_child(&h, &mut a, &p, &lease, &token, "child-1"); + drop(a); + let (mut a, p) = h.ready(); + assert_eq!(a.lease(&lease).unwrap().lease.phase, LeasePhase::Active); + assert_eq!(a.committed_budget().unwrap(), budget(6_000)); + drop((a, p)); + // After a reboot the owner and the child's scope have ended. + h.clock.reboot(); + let mut a = h.open(); + assert_eq!( + a.reconcile(&child.key).unwrap().release_reason, + Some(ReleaseReason::PreviousBoot) + ); + let released = a.reconcile_lease(&lease, true).unwrap(); + assert_eq!(released.phase, LeasePhase::Released); + assert_eq!(released.end_reason, Some(LeaseEndReason::OwnerGone)); + assert_eq!(a.committed_budget().unwrap(), Budget::ZERO); +} + +#[test] +fn a_generation_with_a_lease_cannot_be_retired() { + let h = Harness::new(); + let (mut a, p) = h.ready(); + let (lease, _) = granted(&mut a, &p, "lease-1", 3_000); + // The owner has ended and holds no attempt, so its instance is retired; + // only the lease still holds the generation. + h.backend.0.lock().unwrap().processes.clear(); + let instance = &h.registration.instance.instance_id; + assert!(a + .reconcile_instance("codespace-runtime", "generation-1", instance) + .unwrap()); + assert_eq!( + a.retire_generation("codespace-runtime", "generation-1") + .unwrap_err() + .code, + ErrorCode::ReconciliationRequired + ); + assert_eq!( + a.reconcile_lease(&lease, false).unwrap().phase, + LeasePhase::Released + ); + drop(p); + a.retire_generation("codespace-runtime", "generation-1") + .unwrap(); +} + +#[test] +fn a_journal_written_before_leases_gains_their_tables_on_activation() { + let h = Harness::new(); + let journal = Connection::open(&h.path).unwrap(); + journal + .execute_batch("DROP TABLE leases; DROP TABLE lease_children;") + .unwrap(); + drop(journal); + let (mut a, p) = h.ready(); + let (lease, token) = granted(&mut a, &p, "lease-1", 3_000); + assert_eq!( + a.lease_status(&lease, &token).unwrap().lease.phase, + LeasePhase::Active + ); +} diff --git a/crates/daemon/src/candidate.rs b/crates/daemon/src/candidate.rs new file mode 100644 index 0000000..f7bb244 --- /dev/null +++ b/crates/daemon/src/candidate.rs @@ -0,0 +1,773 @@ +//! Candidate authorities: an isolated authority whose whole capacity is a +//! parent lease, used to verify a candidate build before it becomes a parent. +//! +//! A candidate runs as a child workload of its parent's lease, under the +//! parent's launcher. It reads the lease token from a private descriptor, +//! confirms with the parent that the lease is active and holds its capacity, +//! initializes disposable state in its own area and serves admission within +//! that capacity. It never observes the host's capacity for its policy, +//! launches nothing and holds no parent leases, and it never opens the normal +//! state, credentials, releases or recovery copies. It closes as soon as its +//! lease is no longer active, and fails closed when its parent cannot confirm +//! the lease. + +use crate::config; +use crate::paths::AuthorityPaths; +use crate::server::{receipt, CandidateMode, Options, Server}; +use devguard_client::Client; +use devguard_contract::{ + AttemptKey, Budget, Capability, Compatibility, Error, ErrorCode, LeasePhase, LeaseView, Result, + Secret, +}; +use devguard_macos::HostProbe; +use serde::Serialize; +use serde_json::json; +use std::collections::BTreeSet; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::Arc; +use std::time::{Duration, Instant}; + +/// How often a candidate confirms its lease with its parent. +pub const LEASE_CHECK_INTERVAL: Duration = Duration::from_secs(1); +/// How long a candidate keeps serving while its parent cannot confirm the +/// lease. A candidate never outlives its lease by more than this. +pub const LEASE_CONFIRMATION_LIMIT: Duration = Duration::from_secs(5); + +/// What a candidate is started with. The lease token is passed separately, +/// on a private descriptor. +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +pub struct CandidateSpec { + pub id: String, + pub lease: AttemptKey, + pub capacity: Budget, +} + +impl CandidateSpec { + /// `--id ID --lease CONSUMER/GENERATION/ATTEMPT --capacity MILLICPU,BYTES,TASKS`, + /// each exactly once and in any order. + pub fn parse(args: &[String]) -> Result<(Self, i32)> { + let usage = || { + Error::new( + ErrorCode::InvalidRequest, + "candidate takes --id ID --lease CONSUMER/GENERATION/ATTEMPT --capacity MILLICPU,BYTES,TASKS --token-fd N", + ) + }; + let (mut id, mut lease, mut capacity, mut fd) = (None, None, None, None); + let mut values = args.iter(); + while let Some(option) = values.next() { + let value = values.next().ok_or_else(usage)?; + let slot_taken = match option.as_str() { + "--id" => id.replace(value.clone()).is_some(), + "--lease" => lease.replace(parse_key(value)?).is_some(), + "--capacity" => capacity.replace(parse_budget(value)?).is_some(), + "--token-fd" => fd + .replace(value.parse::().map_err(|_| usage())?) + .is_some(), + _ => return Err(usage()), + }; + if slot_taken { + return Err(usage()); + } + } + let spec = Self { + id: id.ok_or_else(usage)?, + lease: lease.ok_or_else(usage)?, + capacity: capacity.ok_or_else(usage)?, + }; + spec.validate()?; + let fd = fd.filter(|fd| *fd > 2).ok_or_else(usage)?; + Ok((spec, fd)) + } + + pub fn validate(&self) -> Result<()> { + self.lease.validate()?; + self.capacity.validate_workload()?; + // The id is checked where the candidate's paths are derived from it. + Ok(()) + } + + /// The candidate's status reason names it and its parent lease. + pub fn reason(&self) -> String { + format!( + "candidate {} within parent lease {}; admission only, bounded by the leased capacity; nothing is launched here", + self.id, + key_text(&self.lease) + ) + } +} + +/// `CONSUMER/GENERATION/ATTEMPT`. Identifiers never contain `/`. +pub fn parse_key(value: &str) -> Result { + let parts: Vec<&str> = value.split('/').collect(); + let [consumer, generation, attempt] = parts[..] else { + return Err(Error::new( + ErrorCode::InvalidRequest, + "a lease is CONSUMER/GENERATION/ATTEMPT", + )); + }; + let key = AttemptKey { + consumer_id: consumer.into(), + consumer_generation: generation.into(), + attempt_id: attempt.into(), + }; + key.validate()?; + Ok(key) +} + +pub fn key_text(key: &AttemptKey) -> String { + format!( + "{}/{}/{}", + key.consumer_id, key.consumer_generation, key.attempt_id + ) +} + +/// `MILLICPU,BYTES,TASKS`, each a positive whole number. +pub fn parse_budget(value: &str) -> Result { + let invalid = || { + Error::new( + ErrorCode::InvalidRequest, + "a capacity is MILLICPU,BYTES,TASKS in whole numbers", + ) + }; + let parts: Vec = value + .split(',') + .map(|part| { + if part.is_empty() || part.len() > 20 || !part.bytes().all(|b| b.is_ascii_digit()) { + return Err(invalid()); + } + part.parse().map_err(|_| invalid()) + }) + .collect::>()?; + let [cpu_milli, memory_bytes, tasks] = parts[..] else { + return Err(invalid()); + }; + let budget = Budget { + cpu_milli, + memory_bytes, + tasks, + }; + budget.validate_workload()?; + Ok(budget) +} + +pub fn budget_text(budget: Budget) -> String { + format!( + "{},{},{}", + budget.cpu_milli, budget.memory_bytes, budget.tasks + ) +} + +/// The handshake a lease holder makes: the parent must state parent leases. +pub fn parent_compatibility() -> Compatibility { + Compatibility { + minimum_protocol: 1, + maximum_protocol: 1, + required: BTreeSet::from([Capability::ParentLease]), + } +} + +/// The lease as its parent reports it to a token holder. +pub fn lease_status( + parent: &AuthorityPaths, + lease: &AttemptKey, + token: &Secret, +) -> Result { + let mut client = Client::connect(&parent.socket(), parent.uid(), parent_compatibility())?; + client.lease_status(lease.clone(), token.clone()) +} + +/// A candidate ready to serve. +pub struct Candidate { + server: Server, + parent: AuthorityPaths, + paths: AuthorityPaths, + spec: CandidateSpec, + token: Secret, +} + +/// Why a candidate stopped serving. +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +#[serde(rename_all = "snake_case", tag = "closure")] +pub enum Closure { + /// It was asked to stop. + Stopped, + /// Its lease is no longer active. + LeaseEnded { phase: LeasePhase }, + /// Its parent could not confirm the lease within the limit. + Unconfirmed { error: Error }, +} + +impl Candidate { + /// Confirm the lease with the parent at `parent`, then initialize the + /// candidate's own state and open its service. The capacity must fit the + /// lease; the parent charges it when it admits the candidate as a child. + pub fn open( + parent: &AuthorityPaths, + spec: CandidateSpec, + token: Secret, + probe: Option>, + ) -> Result { + spec.validate()?; + let paths = parent.candidate(&spec.id)?; + // Checked before anything is created: the candidate's own daemon + // keeps a reservation out of the capacity. + spec.capacity + .remaining_after(config::candidate_reservation( + config::BOOTSTRAP_SYSTEM_TASKS, + )) + .validate_workload() + .map_err(|_| config::too_small_for_a_candidate())?; + let view = lease_status(parent, &spec.lease, &token)?; + if view.lease.phase != LeasePhase::Active { + return Err(Error::new( + ErrorCode::InvalidTransition, + "the parent lease is not active", + )); + } + if !spec.capacity.fits(view.lease.budget) { + return Err(Error::new( + ErrorCode::ResourceUnavailable, + "the candidate capacity exceeds its parent lease", + )); + } + // Each candidate starts from new state; an earlier candidate's area is + // never reused, whatever it holds. + if std::fs::symlink_metadata(paths.root()).is_ok() { + return Err(Error::new( + ErrorCode::InvalidRequest, + "the candidate area already exists; each candidate uses a new id", + )); + } + config::initialize(&paths)?; + let server = Server::open_with( + &paths, + Options { + probe, + reconcile_paused: None, + candidate: Some(CandidateMode { + capacity: spec.capacity, + reason: spec.reason(), + }), + }, + )?; + receipt(json!({"event": "candidate_opened", "candidate": spec, + "parent_lease": view.lease, "remaining": view.remaining, + "state": paths.root(), "socket": paths.socket()})); + Ok(Self { + server, + parent: parent.clone(), + paths, + spec, + token, + }) + } + + pub fn paths(&self) -> &AuthorityPaths { + &self.paths + } + + /// Serve until `stop` is set or the lease is no longer confirmed active. + pub fn serve(self, stop: Arc) -> Result { + let Self { + server, + parent, + spec, + token, + .. + } = self; + let watched = stop.clone(); + let monitor = std::thread::Builder::new() + .name("devguard-lease".into()) + .spawn(move || { + let closure = watch_lease(&parent, &spec.lease, &token, &watched); + // Whatever ended the watch ends the service. + watched.store(true, Ordering::Relaxed); + closure + }) + .map_err(|_| { + Error::new( + ErrorCode::ResourceControlUnavailable, + "cannot start the lease monitor", + ) + })?; + let served = server.run(stop.clone()); + stop.store(true, Ordering::Relaxed); + let closure = monitor.join().unwrap_or_else(|_| Closure::Unconfirmed { + error: Error::new( + ErrorCode::ResourceControlUnavailable, + "the lease monitor failed", + ), + }); + receipt(json!({"event": "candidate_closed", "closure": closure})); + served.map(|()| closure) + } +} + +/// Confirm the lease every interval until it ends, the parent cannot confirm +/// it for the limit, or `stop` is set. +fn watch_lease( + parent: &AuthorityPaths, + lease: &AttemptKey, + token: &Secret, + stop: &AtomicBool, +) -> Closure { + let mut confirmed = Instant::now(); + loop { + let next = Instant::now() + LEASE_CHECK_INTERVAL; + while Instant::now() < next { + if stop.load(Ordering::Relaxed) { + return Closure::Stopped; + } + std::thread::sleep(Duration::from_millis(20)); + } + match lease_status(parent, lease, token) { + Ok(view) if view.lease.phase == LeasePhase::Active => confirmed = Instant::now(), + Ok(view) => { + return Closure::LeaseEnded { + phase: view.lease.phase, + } + } + // An unknown lease or a refused token is final. + Err(error) if error.code == ErrorCode::Unauthorized => { + return Closure::Unconfirmed { error } + } + Err(error) if confirmed.elapsed() >= LEASE_CONFIRMATION_LIMIT => { + return Closure::Unconfirmed { error } + } + Err(_) => {} + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn args(values: &[&str]) -> Vec { + values.iter().map(|value| value.to_string()).collect() + } + + #[test] + fn a_candidate_is_named_by_its_id_lease_capacity_and_token_descriptor() { + let (spec, fd) = CandidateSpec::parse(&args(&[ + "--lease", + "dev-cli/g-1/lease-1", + "--id", + "c-1", + "--capacity", + "1000,1073741824,64", + "--token-fd", + "5", + ])) + .unwrap(); + assert_eq!(fd, 5); + assert_eq!(spec.id, "c-1"); + assert_eq!(key_text(&spec.lease), "dev-cli/g-1/lease-1"); + assert_eq!(budget_text(spec.capacity), "1000,1073741824,64"); + assert!(spec.reason().contains("c-1") && spec.reason().contains("dev-cli/g-1/lease-1")); + for invalid in [ + &[ + "--id", + "c-1", + "--lease", + "dev-cli/g-1/lease-1", + "--capacity", + "1,1,1", + ][..], + &[ + "--id", + "c-1", + "--lease", + "dev-cli/g-1", + "--capacity", + "1,1,1", + "--token-fd", + "5", + ], + &[ + "--id", + "c-1", + "--lease", + "a/b/c", + "--capacity", + "1,1", + "--token-fd", + "5", + ], + &[ + "--id", + "c-1", + "--lease", + "a/b/c", + "--capacity", + "0,1,1", + "--token-fd", + "5", + ], + &[ + "--id", + "c-1", + "--lease", + "a/b/c", + "--capacity", + "1,1,1", + "--token-fd", + "2", + ], + &[ + "--id", + "c-1", + "--id", + "c-2", + "--lease", + "a/b/c", + "--capacity", + "1,1,1", + "--token-fd", + "5", + ], + &[ + "--id", + "c-1", + "--lease", + "a/b/c", + "--capacity", + "1,1,1", + "--token-fd", + "5", + "--socket", + "/tmp/x", + ], + ] { + assert!(CandidateSpec::parse(&args(invalid)).is_err(), "{invalid:?}"); + } + } + + #[test] + fn candidate_paths_stay_inside_the_candidate_area() { + let directory = tempfile::tempdir().unwrap(); + let parent = AuthorityPaths::fixture(directory.path()); + let candidate = parent.candidate("c-1").unwrap(); + assert!(candidate.root().starts_with(parent.candidates())); + assert!(candidate.socket().starts_with(parent.candidate_runtime())); + for path in [ + candidate.journal(), + candidate.config(), + candidate.cli_credential(), + candidate.admin_credential(), + ] { + assert!( + path.starts_with(parent.candidates().join("c-1")), + "{path:?}" + ); + } + assert_ne!(candidate.journal(), parent.journal()); + assert_ne!(candidate.socket(), parent.socket()); + for invalid in ["", "C-1", "../state", "a/b", &"c".repeat(33)] { + assert!(parent.candidate(invalid).is_err(), "{invalid:?}"); + } + } + + /// Candidates of a fixture parent, served in this process. + #[cfg(target_os = "macos")] + mod served { + use super::*; + use crate::fixture::{healthy_probe, TestAuthority, CONSUMER}; + use crate::paths::read_private; + use devguard_client::protocol::CallerCredential; + use devguard_contract::{AdmissionRequest, AttemptPhase, ResourceIntent, ResourceLevels}; + + const MIB: u64 = 1024 * 1024; + + struct Parent { + authority: TestAuthority, + _directory: tempfile::TempDir, + } + + fn parent() -> Parent { + let directory = tempfile::Builder::new() + .prefix("dg-cd-") + .tempdir_in("/private/tmp") + .unwrap(); + let authority = TestAuthority::start(directory.path()).unwrap(); + authority + .wait_until_admitting(Duration::from_secs(10)) + .unwrap(); + Parent { + authority, + _directory: directory, + } + } + + fn compatibility(required: &[Capability]) -> Compatibility { + Compatibility { + minimum_protocol: 1, + maximum_protocol: 1, + required: required.iter().copied().collect(), + } + } + + fn owner(parent: &Parent) -> Client { + let mut client = Client::connect( + &parent.authority.socket(), + parent.authority.uid(), + compatibility(&[Capability::DurableAdmission, Capability::ParentLease]), + ) + .unwrap(); + client + .authenticate(parent.authority.consumer().unwrap()) + .unwrap(); + client.register("candidate-owner".into()).unwrap(); + client + } + + fn key(parent: &Parent, id: &str) -> AttemptKey { + AttemptKey { + consumer_id: CONSUMER.into(), + consumer_generation: parent.authority.generation(), + attempt_id: id.into(), + } + } + + /// Fits a 3-CPU host's 500 mCPU of work capacity. + fn lease_budget() -> Budget { + Budget { + cpu_milli: 450, + memory_bytes: 512 * MIB, + tasks: 96, + } + } + + /// Leaves 150 mCPU, 128 MiB and 16 tasks of work after the + /// candidate's own reservation. + fn capacity() -> Budget { + Budget { + cpu_milli: 400, + memory_bytes: 256 * MIB, + tasks: 64, + } + } + + fn lease(parent: &Parent, owner: &mut Client, id: &str) -> (AttemptKey, Secret) { + let lease = key(parent, id); + let token = owner + .admit_lease(lease.clone(), lease_budget(), None) + .unwrap() + .token + .unwrap(); + (lease, token) + } + + fn request(key: AttemptKey, requested: Budget) -> AdmissionRequest { + AdmissionRequest { + key, + execution_digest: devguard_contract::digest_bytes(b"candidate smoke"), + intent: ResourceIntent { + profile: "interactive".into(), + requested, + minimum: ResourceLevels::MACOS, + }, + } + } + + fn candidate_credential(paths: &AuthorityPaths) -> CallerCredential { + let config = crate::config::HostConfig::load(paths).unwrap(); + let secret = Secret::new( + String::from_utf8(read_private(&paths.cli_credential(), paths.uid(), 64).unwrap()) + .unwrap(), + ) + .unwrap(); + CallerCredential::Consumer { + consumer_id: CONSUMER.into(), + generation: config.consumers[CONSUMER].generation.clone(), + secret, + } + } + + #[test] + fn a_candidate_admits_within_its_lease_launches_nothing_and_closes_with_the_lease() { + let parent = parent(); + let (lease, token) = lease(&parent, &mut owner(&parent), "lease-1"); + let spec = CandidateSpec { + id: "c-1".into(), + lease: lease.clone(), + capacity: capacity(), + }; + let candidate = + Candidate::open(parent.authority.paths(), spec, token, Some(healthy_probe())) + .unwrap(); + let paths = candidate.paths().clone(); + assert!(paths + .root() + .starts_with(parent.authority.paths().candidates())); + let stop = Arc::new(AtomicBool::new(false)); + let serving = { + let stop = stop.clone(); + std::thread::spawn(move || candidate.serve(stop)) + }; + // Admission is stated; launch and parent leases are not. + let mut client = Client::connect( + &paths.socket(), + paths.uid(), + compatibility(&[Capability::DurableAdmission, Capability::MacosCooperative]), + ) + .unwrap(); + assert!(!client + .hello + .capabilities + .contains(&Capability::FencedLaunch)); + assert!(!client.hello.capabilities.contains(&Capability::ParentLease)); + for required in [Capability::FencedLaunch, Capability::ParentLease] { + assert!( + Client::connect(&paths.socket(), paths.uid(), compatibility(&[required])) + .is_err() + ); + } + // The parent's caller credential is not the candidate's. + let mut stranger = + Client::connect(&paths.socket(), paths.uid(), compatibility(&[])).unwrap(); + assert_eq!( + stranger + .authenticate(parent.authority.consumer().unwrap()) + .unwrap_err() + .code, + ErrorCode::Unauthorized + ); + client.authenticate(candidate_credential(&paths)).unwrap(); + let status = client.status().unwrap(); + assert!(status.registration_ready && !status.execution_ready); + assert!(status.reason.contains("c-1") && status.reason.contains(&key_text(&lease))); + // Every frame has a short deadline, so each step uses a new session. + let smoke = || { + let mut client = + Client::connect(&paths.socket(), paths.uid(), compatibility(&[])).unwrap(); + client.authenticate(candidate_credential(&paths)).unwrap(); + client.register("smoke".into()).unwrap(); + client + }; + let within = Budget { + cpu_milli: 100, + memory_bytes: 64 * MIB, + tasks: 4, + }; + let generation = crate::config::HostConfig::load(&paths).unwrap().consumers[CONSUMER] + .generation + .clone(); + let deadline = Instant::now() + Duration::from_secs(20); + let mut attempt = 0; + let admitted = loop { + attempt += 1; + let key = AttemptKey { + consumer_id: CONSUMER.into(), + consumer_generation: generation.clone(), + attempt_id: format!("within-{attempt}"), + }; + let record = smoke().admit(request(key, within)).unwrap(); + if record.phase == AttemptPhase::Prepared { + break record; + } + // A new authority starts closed until its pressure is normal. + assert!(Instant::now() < deadline, "{record:?}"); + std::thread::sleep(Duration::from_millis(250)); + }; + let mut client = smoke(); + assert_eq!( + client.begin_launch(admitted.key.clone()).unwrap_err().code, + ErrorCode::ResourcePolicyUnsupported + ); + let beyond = AttemptKey { + attempt_id: "beyond".into(), + ..admitted.key.clone() + }; + let denied = client.admit(request(beyond, capacity())).unwrap(); + assert_eq!(denied.denial, Some(ErrorCode::ResourceUnavailable)); + assert_eq!( + client + .admit_lease( + AttemptKey { + attempt_id: "nested".into(), + ..admitted.key.clone() + }, + within, + None + ) + .unwrap_err() + .code, + ErrorCode::ResourcePolicyUnsupported + ); + // The candidate charged nothing at the parent. + assert_eq!(parent.authority.committed().unwrap(), lease_budget()); + // Ending the lease closes the candidate. + owner(&parent).end_lease(lease).unwrap(); + let closed = Instant::now(); + let closure = serving.join().unwrap().unwrap(); + assert!(closed.elapsed() < LEASE_CHECK_INTERVAL * 3); + assert!( + matches!(closure, Closure::LeaseEnded { phase } if phase != LeasePhase::Active), + "{closure:?}" + ); + assert!(!paths.socket().exists()); + } + + #[test] + fn a_candidate_needs_an_active_lease_its_token_a_fitting_capacity_and_a_new_area() { + let parent = parent(); + let (lease, token) = lease(&parent, &mut owner(&parent), "lease-1"); + let open = |id: &str, lease: &AttemptKey, token: &Secret, capacity: Budget| { + Candidate::open( + parent.authority.paths(), + CandidateSpec { + id: id.into(), + lease: lease.clone(), + capacity, + }, + token.clone(), + Some(healthy_probe()), + ) + .map(|_| ()) + .unwrap_err() + .code + }; + let forged = Secret::new("f".repeat(64)).unwrap(); + assert_eq!( + open("c-1", &lease, &forged, capacity()), + ErrorCode::Unauthorized + ); + let unknown = key(&parent, "no-such-lease"); + assert_eq!( + open("c-1", &unknown, &token, capacity()), + ErrorCode::Unauthorized + ); + let larger = Budget { + cpu_milli: lease_budget().cpu_milli + 1, + ..capacity() + }; + assert_eq!( + open("c-1", &lease, &token, larger), + ErrorCode::ResourceUnavailable + ); + let too_small = Budget { + cpu_milli: 250, + ..capacity() + }; + assert_eq!( + open("c-1", &lease, &token, too_small), + ErrorCode::ResourceUnavailable + ); + // Nothing was created by a refused candidate. + assert!(!parent.authority.paths().candidates().join("c-1").exists()); + // An earlier candidate's area is never reused. + std::fs::create_dir_all(parent.authority.paths().candidates().join("c-2")).unwrap(); + assert_eq!( + open("c-2", &lease, &token, capacity()), + ErrorCode::InvalidRequest + ); + // An ended lease starts no candidate. + owner(&parent).end_lease(lease.clone()).unwrap(); + assert_eq!( + open("c-3", &lease, &token, capacity()), + ErrorCode::InvalidTransition + ); + } + } +} diff --git a/crates/daemon/src/config.rs b/crates/daemon/src/config.rs index ad4003b..97248e2 100644 --- a/crates/daemon/src/config.rs +++ b/crates/daemon/src/config.rs @@ -14,6 +14,22 @@ const GIB: u64 = 1024 * 1024 * 1024; /// plus the aggregate CLI control pool (0.25 CPU, 128 MiB for up to eight CLIs). const SYSTEM_CPU_MILLI: u64 = 250 + 250; const SYSTEM_MEMORY_BYTES: u64 = (128 + 128) * 1024 * 1024; +/// A candidate reserves its own daemon only: it launches nothing, so it has +/// no CLI pool. +const CANDIDATE_SYSTEM_CPU_MILLI: u64 = 250; +const CANDIDATE_SYSTEM_MEMORY_BYTES: u64 = 128 * 1024 * 1024; +/// The system tasks bootstrap accounts: bounded sessions and CLI control. +pub const BOOTSTRAP_SYSTEM_TASKS: u64 = devguard_client::protocol::MAX_SESSIONS as u64 + 16; + +/// What a candidate with `system_tasks` keeps for its own daemon; its +/// capacity must exceed this in every quantity. +pub fn candidate_reservation(system_tasks: u64) -> Budget { + Budget { + cpu_milli: CANDIDATE_SYSTEM_CPU_MILLI, + memory_bytes: CANDIDATE_SYSTEM_MEMORY_BYTES, + tasks: system_tasks, + } +} #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(deny_unknown_fields)] @@ -172,8 +188,47 @@ impl HostConfig { tasks: self.task_headroom, } .checked_add(self.additional_headroom)?; - let consumers = self - .consumers + let policy = Policy { + revision: self.policy_revision.clone(), + effective_capacity, + host_headroom, + system_reservation: self.system_reservation(), + consumers: self.consumer_definitions(uid), + }; + policy.validate()?; + Ok(policy) + } + + /// Policy for a candidate authority whose whole capacity is a parent + /// lease. The parent already kept the host's headroom, so there is none + /// here, and the host's capacity is not used. + pub fn candidate_policy(&self, capacity: Budget, uid: u32) -> Result { + capacity.validate_workload()?; + let policy = Policy { + revision: format!("{}-candidate", self.policy_revision), + effective_capacity: capacity, + host_headroom: Budget::ZERO, + system_reservation: candidate_reservation(self.system_tasks), + consumers: self.consumer_definitions(uid), + }; + policy.validate()?; + policy + .work_capacity()? + .validate_workload() + .map_err(|_| too_small_for_a_candidate())?; + Ok(policy) + } + + fn system_reservation(&self) -> Budget { + Budget { + cpu_milli: SYSTEM_CPU_MILLI, + memory_bytes: SYSTEM_MEMORY_BYTES, + tasks: self.system_tasks, + } + } + + fn consumer_definitions(&self, uid: u32) -> BTreeMap { + self.consumers .iter() .map(|(id, consumer)| { ( @@ -188,20 +243,7 @@ impl HostConfig { }, ) }) - .collect(); - let policy = Policy { - revision: self.policy_revision.clone(), - effective_capacity, - host_headroom, - system_reservation: Budget { - cpu_milli: SYSTEM_CPU_MILLI, - memory_bytes: SYSTEM_MEMORY_BYTES, - tasks: self.system_tasks, - }, - consumers, - }; - policy.validate()?; - Ok(policy) + .collect() } } @@ -258,6 +300,13 @@ impl ProjectSettings { } } +pub fn too_small_for_a_candidate() -> Error { + Error::new( + ErrorCode::ResourceUnavailable, + "a candidate capacity must exceed its own daemon's reservation of 250 mCPU, 128 MiB and the system tasks", + ) +} + fn new_secret() -> Result { Secret::new(format!( "{}{}", @@ -292,7 +341,7 @@ pub fn initialize(paths: &AuthorityPaths) -> Result { admin_credential_sha256: admin.digest(), task_capacity: 256, task_headroom: 64, - system_tasks: 48, + system_tasks: BOOTSTRAP_SYSTEM_TASKS, additional_headroom: Budget::ZERO, consumers: BTreeMap::from([( "dev-cli".into(), diff --git a/crates/daemon/src/fixture.rs b/crates/daemon/src/fixture.rs index 59c1081..01ef6de 100644 --- a/crates/daemon/src/fixture.rs +++ b/crates/daemon/src/fixture.rs @@ -3,6 +3,7 @@ //! test cannot assume the real host is free of pressure. Nothing here is //! reachable from the `devguardd` command. +use crate::candidate::{Candidate, CandidateSpec, Closure}; use crate::config::{self, HostConfig}; use crate::paths::{read_private, write_new_private, AuthorityPaths}; use crate::server::{NativeAuthority, Options, Server}; @@ -38,6 +39,11 @@ impl HostProbe for HealthyProbe { } } +/// A synthetic probe reporting a healthy host. +pub fn healthy_probe() -> Box { + Box::new(HealthyProbe) +} + fn unavailable(message: &'static str) -> Error { Error::new(ErrorCode::ResourceControlUnavailable, message) } @@ -90,6 +96,7 @@ impl TestAuthority { Options { probe: Some(Box::new(HealthyProbe)), reconcile_paused: Some(reconcile_paused.clone()), + candidate: None, }, )?; let authority = server.native_authority().ok_or_else(|| { @@ -246,17 +253,40 @@ impl TestAuthority { /// SIGINT, as `devguardd serve` serves the canonical one. Tests run it as the /// program of a service, under launchd or a fake manager. pub fn serve_until_terminated(base: &Path) -> Result<()> { - static STOP: AtomicBool = AtomicBool::new(false); - extern "C" fn stop(_: libc::c_int) { - STOP.store(true, Ordering::Relaxed); - } let server = Server::open_with( &AuthorityPaths::fixture(base), Options { probe: Some(Box::new(HealthyProbe)), - reconcile_paused: None, + ..Options::default() }, )?; + until_terminated(|stop| server.run(stop)) +} + +/// Serve a candidate of the fixture authority under `base` in this process, +/// as `devguardd candidate` serves one of the canonical authority: the token +/// is read from the inherited descriptor `token_fd`. Returns why it closed. +pub fn candidate_until_terminated( + base: &Path, + spec: CandidateSpec, + token_fd: i32, +) -> Result { + // SAFETY: the test passes this descriptor to the process for the token only. + let token = unsafe { devguard_client::credential::take_inherited(token_fd) }?; + let candidate = Candidate::open( + &AuthorityPaths::fixture(base), + spec, + token, + Some(Box::new(HealthyProbe)), + )?; + until_terminated(|stop| candidate.serve(stop)) +} + +fn until_terminated(serve: impl FnOnce(Arc) -> Result) -> Result { + static STOP: AtomicBool = AtomicBool::new(false); + extern "C" fn stop(_: libc::c_int) { + STOP.store(true, Ordering::Relaxed); + } // SAFETY: the handler only stores to a lock-free atomic. unsafe { libc::signal(libc::SIGTERM, stop as *const () as libc::sighandler_t); @@ -273,7 +303,7 @@ pub fn serve_until_terminated(base: &Path) -> Result<()> { std::thread::sleep(Duration::from_millis(10)); } }); - let result = server.run(flag.clone()); + let result = serve(flag.clone()); flag.store(true, Ordering::Relaxed); let _ = watcher.join(); result diff --git a/crates/daemon/src/install.rs b/crates/daemon/src/install.rs index c106217..fc306de 100644 --- a/crates/daemon/src/install.rs +++ b/crates/daemon/src/install.rs @@ -63,7 +63,10 @@ pub fn compiled() -> BuildCompatibility { package_version: env!("CARGO_PKG_VERSION").into(), wire_version: WIRE_VERSION, protocol: PROTOCOL_VERSION, - capabilities: crate::server::launch_capabilities(), + capabilities: crate::server::launch_capabilities() + .union(&crate::server::echoed_capabilities()) + .copied() + .collect(), journal_schema: devguard_core::JOURNAL_SCHEMA.into(), config_schema: crate::config::CONFIG_SCHEMA, } diff --git a/crates/daemon/src/lib.rs b/crates/daemon/src/lib.rs index 5ed2a91..669839b 100644 --- a/crates/daemon/src/lib.rs +++ b/crates/daemon/src/lib.rs @@ -1,4 +1,5 @@ //! Service ownership and configuration. No invented host evidence or hidden fallback. +pub mod candidate; pub mod config; #[cfg(any(test, feature = "test-fixtures"))] pub mod fixture; diff --git a/crates/daemon/src/main.rs b/crates/daemon/src/main.rs index 83a105d..cea2682 100644 --- a/crates/daemon/src/main.rs +++ b/crates/daemon/src/main.rs @@ -1,6 +1,7 @@ use devguard_contract::{Error, ErrorCode, Result}; use devguard_core::AuthorityStorage; use devguard_daemon::{ + candidate::{Candidate, CandidateSpec, Closure}, config::{self, HostConfig}, install::{self, InstallOptions, Launchctl}, paths::AuthorityPaths, @@ -16,23 +17,51 @@ extern "C" fn stop_signal(_: libc::c_int) { STOP.store(true, Ordering::Relaxed); } +/// Run `serve` with a stop flag that SIGINT and SIGTERM set. +fn until_signalled(serve: impl FnOnce(Arc) -> Result) -> Result { + // SAFETY: handlers only store to a lock-free atomic; no allocation or I/O. + unsafe { + libc::signal(libc::SIGINT, stop_signal as *const () as libc::sighandler_t); + libc::signal( + libc::SIGTERM, + stop_signal as *const () as libc::sighandler_t, + ); + } + let stop = Arc::new(AtomicBool::new(false)); + let watched = stop.clone(); + let watcher = std::thread::spawn(move || { + while !watched.load(Ordering::Relaxed) { + if STOP.load(Ordering::Relaxed) { + watched.store(true, Ordering::Relaxed); + break; + } + std::thread::sleep(std::time::Duration::from_millis(10)); + } + }); + let result = serve(stop.clone()); + stop.store(true, Ordering::Relaxed); + let _ = watcher.join(); + result +} + fn run() -> Result<()> { let args: Vec<_> = std::env::args().skip(1).collect(); if args == ["--help"] || args == ["help"] { - println!("devguardd paths | init | check | serve | status | install --package DIR | version --json\nNormal authority paths come from the OS account. No path or test-budget override is accepted.\nserve observes native boot, process and host pressure evidence and, with it, opens registration, fenced launch through devguard-launch and reconciliation; `devguard` runs commands through it. install copies a package into a protected release and starts it as the current user's LaunchAgent; it must run from that package and refuses while an authority or the agent exists. status reports the installed service. init and check never start workloads."); + println!("devguardd paths | init | check | serve | status | install --package DIR | version --json\n candidate --id ID --lease CONSUMER/GENERATION/ATTEMPT --capacity MILLICPU,BYTES,TASKS --token-fd N\nNormal authority paths come from the OS account. No path or test-budget override is accepted.\nserve observes native boot, process and host pressure evidence and, with it, opens registration, fenced launch through devguard-launch and reconciliation; `devguard` runs commands through it. install copies a package into a protected release and starts it as the current user's LaunchAgent; it must run from that package and refuses while an authority or the agent exists. status reports the installed service. init and check never start workloads.\ncandidate serves an isolated candidate authority whose capacity is a parent lease of the account's authority; it reads the lease token from the private descriptor N, launches nothing, and closes when the lease ends. `devguard test-candidate` starts it as a lease child."); return Ok(()); } let command = args.first().map(String::as_str).unwrap_or_default(); let valid = match command { "paths" | "init" | "check" | "serve" | "status" => args.len() == 1, "install" => args.len() == 3 && args[1] == "--package", + "candidate" => args.len() == 9, "version" => args.len() == 2 && args[1] == "--json", _ => false, }; if !valid { return Err(Error::new( ErrorCode::InvalidRequest, - "expected paths, init, check, serve, status, install --package DIR or version --json; no alternate authority arguments are supported", + "expected paths, init, check, serve, status, install --package DIR, candidate or version --json; no alternate authority arguments are supported", )); } if command == "version" { @@ -93,29 +122,19 @@ fn run() -> Result<()> { } "serve" => { let server = Server::open(&paths)?; - // SAFETY: handlers only store to a lock-free atomic; no allocation or I/O. - unsafe { - libc::signal(libc::SIGINT, stop_signal as *const () as libc::sighandler_t); - libc::signal( - libc::SIGTERM, - stop_signal as *const () as libc::sighandler_t, - ); + until_signalled(|stop| server.run(stop))?; + } + "candidate" => { + let (spec, fd) = CandidateSpec::parse(&args[1..])?; + // SAFETY: the descriptor was passed to this process for the + // lease token only; it is consumed and closed here. + let token = unsafe { devguard_client::credential::take_inherited(fd) }?; + let candidate = Candidate::open(&paths, spec, token, None)?; + // An unconfirmed lease fails closed; an ended lease or a stop + // signal is an ordinary end. + if let Closure::Unconfirmed { error } = until_signalled(|stop| candidate.serve(stop))? { + return Err(error); } - let stop = Arc::new(AtomicBool::new(false)); - let watched = stop.clone(); - let watcher = std::thread::spawn(move || { - while !watched.load(Ordering::Relaxed) { - if STOP.load(Ordering::Relaxed) { - watched.store(true, Ordering::Relaxed); - break; - } - std::thread::sleep(std::time::Duration::from_millis(10)); - } - }); - let result = server.run(stop.clone()); - stop.store(true, Ordering::Relaxed); - let _ = watcher.join(); - result?; } _ => unreachable!(), } diff --git a/crates/daemon/src/paths.rs b/crates/daemon/src/paths.rs index 11cc3b1..a79d799 100644 --- a/crates/daemon/src/paths.rs +++ b/crates/daemon/src/paths.rs @@ -107,6 +107,40 @@ impl AuthorityPaths { pub fn logs(&self) -> PathBuf { self.home.join("Library/Logs/DevGuard") } + /// The area of candidate authorities, apart from the normal state, + /// releases and recovery copies. + pub fn candidates(&self) -> PathBuf { + self.root.join("candidates") + } + pub fn candidate_runtime(&self) -> PathBuf { + self.runtime.join("candidates") + } + + /// Isolated paths for candidate `id` under this authority: its own state, + /// configuration, credentials, socket and cache. `id` is short and plain, + /// so the socket path stays within the Unix-domain socket limit. + pub fn candidate(&self, id: &str) -> Result { + if id.is_empty() + || id.len() > 32 + || !id + .bytes() + .all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'-') + { + return Err(Error::new( + ErrorCode::InvalidRequest, + "a candidate id is 1 to 32 lowercase letters, digits or '-'", + )); + } + let root = self.candidates().join(id); + Ok(Self { + uid: self.uid, + home: self.home.clone(), + config_directory: root.join("config"), + root, + runtime: self.candidate_runtime().join(id), + cache: self.cache.join("candidates").join(id), + }) + } pub fn runtime(&self) -> &Path { &self.runtime } diff --git a/crates/daemon/src/server.rs b/crates/daemon/src/server.rs index 9119e26..1c5cdcc 100644 --- a/crates/daemon/src/server.rs +++ b/crates/daemon/src/server.rs @@ -1,8 +1,9 @@ use crate::{config::HostConfig, paths::AuthorityPaths}; use devguard_client::{connect::connect_timeout, framing, peer, protocol::*}; use devguard_contract::{ - validate_id, AppliedResources, AttemptKey, AttemptPhase, AttemptRecord, Capability, Error, - ErrorCode, InstanceIdentity, ProcessIdentity, Result, ScopeIdentity, Secret, PROTOCOL_VERSION, + validate_id, AppliedResources, AttemptKey, AttemptPhase, AttemptRecord, Budget, Capability, + Error, ErrorCode, InstanceIdentity, ProcessIdentity, Result, ScopeIdentity, Secret, + PROTOCOL_VERSION, }; use devguard_core::{ Authority, AuthorityStorage, Backend as _, Clock, ConsumerRole, PressureState, Principal, @@ -44,6 +45,16 @@ pub(crate) struct Options { /// Lets an isolated fixture pause reconciler passes, so a test can tell an /// owner's own observation apart from the background pass. pub reconcile_paused: Option>, + /// A candidate authority bounded by a parent lease. + pub candidate: Option, +} + +/// A candidate's leased capacity and its status reason. A candidate states +/// no fenced launch and no parent leases: its workloads run as children of +/// its parent lease, through the parent's launcher. +pub(crate) struct CandidateMode { + pub capacity: Budget, + pub reason: String, } /// Exclusive storage, activated with actual host evidence when it is available. @@ -68,19 +79,21 @@ pub struct Server { status: ServiceStatus, evidence: Evidence, launcher: Option, + /// A candidate states no fenced launch or parent lease and refuses both. + candidate: bool, } /// Service receipts are JSON lines on stderr; they never include credentials. /// A failing stderr is ignored rather than allowed to panic a service thread. #[cfg(not(test))] -fn receipt(value: serde_json::Value) { +pub(crate) fn receipt(value: serde_json::Value) { use std::io::Write; let _ = writeln!(std::io::stderr().lock(), "{value}"); } /// Unit tests keep receipts in the harness's captured output. #[cfg(test)] -fn receipt(value: serde_json::Value) { +pub(crate) fn receipt(value: serde_json::Value) { eprintln!("{value}"); } @@ -101,7 +114,8 @@ fn activate( config: &HostConfig, paths: &AuthorityPaths, substitute: Option>, -) -> Result<(Evidence, &'static str)> { + candidate: Option<&CandidateMode>, +) -> Result<(Evidence, String)> { let host = match NativeHost::open() { Ok(host) => host, Err(error) => { @@ -111,7 +125,7 @@ fn activate( } else { FAILED_REASON }; - return Ok((Evidence::Closed { _storage: storage }, reason)); + return Ok((Evidence::Closed { _storage: storage }, reason.into())); } }; let mut volumes = vec![paths.state()]; @@ -122,19 +136,27 @@ fn activate( Ok(probe) => Box::new(probe), Err(error) => { receipt(json!({"event": "native_host_unavailable", "error": error})); - return Ok((Evidence::Closed { _storage: storage }, FAILED_REASON)); + return Ok((Evidence::Closed { _storage: storage }, FAILED_REASON.into())); } }, }; - let policy = config.policy(host.capacity(), paths.uid())?; + // A candidate's policy capacity is its parent lease, never the host's. + let policy = match candidate { + Some(candidate) => config.candidate_policy(candidate.capacity, paths.uid())?, + None => config.policy(host.capacity(), paths.uid())?, + }; let work_capacity = policy.work_capacity()?; let authority = Authority::from_storage(storage, policy, host.backend(), host.clock())?; + let capacity = match candidate { + Some(candidate) => json!({"source": "parent_lease", "budget": candidate.capacity}), + None => json!({"source": "host", "observed": host.capacity()}), + }; receipt(json!({ "event": "native_host", "boot_id": host.clock().boot_id(), "clock": "CLOCK_MONOTONIC_RAW milliseconds since boot", "started_at": host.clock().now(), - "capacity": host.capacity(), + "capacity": capacity, "work_capacity": work_capacity, "volumes": volumes, "sample_interval_ms": SAMPLE_INTERVAL_MS, @@ -146,7 +168,10 @@ fn activate( backend: host.backend(), probe: Some(probe), }, - LAUNCH_REASON, + candidate.map_or_else( + || LAUNCH_REASON.to_string(), + |candidate| candidate.reason.clone(), + ), )) } @@ -268,6 +293,11 @@ fn poisoned() -> Error { unavailable("authority state is unavailable") } +/// Capabilities stated only to clients that require them. +pub(crate) fn echoed_capabilities() -> BTreeSet { + BTreeSet::from([Capability::ParentLease]) +} + /// The capabilities of a service with registration and fenced launch open. pub(crate) fn launch_capabilities() -> BTreeSet { BTreeSet::from([ @@ -279,6 +309,20 @@ pub(crate) fn launch_capabilities() -> BTreeSet { ]) } +/// The capabilities of a candidate: admission without launch. +pub(crate) fn candidate_capabilities() -> BTreeSet { + let mut capabilities = launch_capabilities(); + capabilities.remove(&Capability::FencedLaunch); + capabilities +} + +fn candidate_refusal() -> Error { + Error::new( + ErrorCode::ResourcePolicyUnsupported, + "a candidate authority launches nothing and holds no parent leases; its workloads run as children of its parent lease", + ) +} + /// A consumer's authenticated credential, kept only to register its instance. struct ConsumerCredential { id: String, @@ -558,9 +602,28 @@ impl Launcher { receipt(json!({"event": "reconcile_failed", "key": record.key, "error": error})); } } + self.reconcile_leases()?; self.reconcile_instances() } + /// Fence leases whose owner has ended or whose deadline passed, and + /// release them once their children have settled. + fn reconcile_leases(&self) -> Result<()> { + let leases = self.authority()?.leases()?; + for lease in leases { + let owner_running = self.running(&lease.owner.process); + let reconciled = self + .authority()? + .reconcile_lease(&lease.key, owner_running)?; + if reconciled.phase != lease.phase { + receipt(json!({"event": "lease_reconciled", "lease": reconciled.key, + "from": lease.phase, "phase": reconciled.phase, + "end_reason": reconciled.end_reason})); + } + } + Ok(()) + } + /// Settle every registered instance whose process is gone: retired when it /// owns no charged work, otherwise Suspect until that work is settled. fn reconcile_instances(&self) -> Result<()> { @@ -712,7 +775,14 @@ impl Server { paths.validate_existing()?; let config = HostConfig::load(paths)?; let storage = AuthorityStorage::open(&paths.journal())?; - let (evidence, reason) = activate(storage, &config, paths, options.probe)?; + let (evidence, reason) = activate( + storage, + &config, + paths, + options.probe, + options.candidate.as_ref(), + )?; + let candidate = options.candidate.is_some(); // Registration and launch open only with native evidence, together // with the reconciler that run() starts. let launcher = match &evidence { @@ -778,8 +848,8 @@ impl Server { let status = ServiceStatus { storage_validated: true, registration_ready: launcher.is_some(), - execution_ready: launcher.is_some(), - reason: reason.into(), + execution_ready: launcher.is_some() && !candidate, + reason, configuration_fingerprint: config.fingerprint()?, }; Ok(Self { @@ -791,6 +861,7 @@ impl Server { status, evidence, launcher, + candidate, }) } @@ -868,10 +939,12 @@ impl Server { let status = self.status.clone(); let launcher = self.launcher.clone(); let stop = stop.clone(); + let candidate = self.candidate; if let Ok(worker) = std::thread::Builder::new() .name("devguard-session".into()) .spawn(move || { - let _ = session(stream, uid, &config, &status, launcher, &stop); + let _ = + session(stream, uid, &config, &status, launcher, candidate, &stop); }) { workers.push(worker); @@ -982,6 +1055,8 @@ struct Session { principal: Option, /// A helper session presents one grant and makes no other request. helper: bool, + /// A lease holder presents a lease token and asks only for its status. + lease_holder: bool, } struct Context<'a> { @@ -990,6 +1065,7 @@ struct Context<'a> { config: &'a HostConfig, status: &'a ServiceStatus, launcher: Option<&'a Launcher>, + candidate: bool, } fn session( @@ -998,6 +1074,7 @@ fn session( config: &HostConfig, status: &ServiceStatus, launcher: Option, + candidate: bool, stop: &AtomicBool, ) -> Result<()> { // Framing uses poll and per-call nonblocking I/O with an absolute deadline; @@ -1012,6 +1089,7 @@ fn session( config, status, launcher: launcher.as_ref(), + candidate, }; let timeout = Duration::from_millis(FRAME_DEADLINE_MS); let mut state = Session::default(); @@ -1072,6 +1150,25 @@ fn handle(session: &mut Session, request: Request, context: &Context<'_>) -> Res "a helper session makes no other request", )); } + if session.lease_holder && !matches!(request, Request::LeaseStatus { .. }) { + return Err(Error::new( + ErrorCode::Unauthorized, + "a lease-holder session only asks for its lease's status", + )); + } + if context.candidate + && matches!( + request, + Request::BeginLaunch { .. } + | Request::Launch { .. } + | Request::AdmitLease { .. } + | Request::AdmitChild { .. } + | Request::EndLease { .. } + | Request::LeaseStatus { .. } + ) + { + return Err(candidate_refusal()); + } match request { Request::Hello { compatibility } => { if session.greeted { @@ -1080,10 +1177,21 @@ fn handle(session: &mut Session, request: Request, context: &Context<'_>) -> Res "session already negotiated", )); } - let capabilities = if context.launcher.is_some() { - launch_capabilities() - } else { + // A capability added after protocol 1 is stated only to a client + // that requires it, so an older client never receives a value it + // cannot decode. + let capabilities = if context.launcher.is_none() { BTreeSet::new() + } else if context.candidate { + candidate_capabilities() + } else { + let mut capabilities = launch_capabilities(); + capabilities.extend( + echoed_capabilities() + .intersection(&compatibility.required) + .copied(), + ); + capabilities }; compatibility.check(PROTOCOL_VERSION, &capabilities)?; session.greeted = true; @@ -1212,6 +1320,59 @@ fn handle(session: &mut Session, request: Request, context: &Context<'_>) -> Res permit, )?)) } + Request::AdmitLease { + key, + budget, + ttl_ms, + } => { + let (launcher, principal) = registered(session, context)?; + let grant = launcher + .authority()? + .admit_lease(principal, &key, budget, ttl_ms)?; + // The token is returned to its owner only; receipts never hold it. + receipt(json!({"event": "lease_admitted", "lease": grant.lease, + "token_issued": grant.token.is_some()})); + Ok(Response::LeaseGranted(LeaseGranted { + lease: grant.lease, + token: grant.token, + })) + } + Request::AdmitChild { + lease, + token, + request, + } => { + let (launcher, principal) = registered(session, context)?; + let record = launcher + .authority()? + .admit_child(principal, &lease, &token, request)?; + receipt( + json!({"event": "admission", "key": record.key, "lease": lease, + "phase": record.phase, "denial": record.denial, + "quantities": record.reservation.as_ref().map(|r| r.quantities)}), + ); + Ok(Response::Attempt(record)) + } + Request::EndLease { key } => { + let (launcher, principal) = registered(session, context)?; + let view = launcher.authority()?.end_lease(principal, &key)?; + receipt(json!({"event": "lease_ended", "lease": view.lease, + "remaining": view.remaining, "children": view.children})); + Ok(Response::Lease(view)) + } + Request::LeaseStatus { key, token } => { + // A lease holder is not a caller: it presents only the token. + if !session.greeted || session.role.is_some() { + return Err(unauthorized()); + } + session.lease_holder = true; + let launcher = context + .launcher + .ok_or_else(|| unavailable("parent leases are not open"))?; + Ok(Response::Lease( + launcher.authority()?.lease_status(&key, &token)?, + )) + } } } @@ -1549,6 +1710,217 @@ mod tests { )))); } + /// Parent leases over the wire: a capability stated only to clients that + /// require it, children admitted against the lease, token-only holder + /// sessions and release by the reconciler. + #[cfg(target_os = "macos")] + mod lease { + use super::*; + use crate::fixture::{TestAuthority, CONSUMER}; + use devguard_contract::{ + AdmissionRequest, AttemptKey, AttemptPhase, Budget, Capability, Compatibility, + LeasePhase, ResourceIntent, ResourceLevels, Secret, + }; + + struct Open { + authority: TestAuthority, + _directory: tempfile::TempDir, + } + + fn open() -> Open { + let directory = tempfile::Builder::new() + .prefix("dg-l-") + .tempdir_in("/private/tmp") + .unwrap(); + let authority = TestAuthority::start(directory.path()).unwrap(); + authority + .wait_until_admitting(Duration::from_secs(10)) + .unwrap(); + Open { + authority, + _directory: directory, + } + } + + fn connect(open: &Open, required: &[Capability]) -> Client { + Client::connect( + &open.authority.socket(), + open.authority.uid(), + Compatibility { + minimum_protocol: 1, + maximum_protocol: 1, + required: required.iter().copied().collect(), + }, + ) + .unwrap() + } + + fn owner(open: &Open, instance: &str) -> Client { + let mut client = connect( + open, + &[Capability::DurableAdmission, Capability::ParentLease], + ); + client + .authenticate(open.authority.consumer().unwrap()) + .unwrap(); + client.register(instance.into()).unwrap(); + client + } + + fn holder(open: &Open) -> Client { + connect(open, &[Capability::ParentLease]) + } + + fn key(open: &Open, id: &str) -> AttemptKey { + AttemptKey { + consumer_id: CONSUMER.into(), + consumer_generation: open.authority.generation(), + attempt_id: id.into(), + } + } + + /// Fits a 3-CPU host's 500 mCPU of work capacity, even halved. + fn lease_budget() -> Budget { + Budget { + cpu_milli: 200, + memory_bytes: 128 * 1024 * 1024, + tasks: 8, + } + } + + fn child(open: &Open, id: &str) -> AdmissionRequest { + AdmissionRequest { + key: key(open, id), + execution_digest: devguard_contract::digest_bytes(b"lease child"), + intent: ResourceIntent { + profile: "interactive".into(), + requested: Budget { + cpu_milli: 100, + memory_bytes: 64 * 1024 * 1024, + tasks: 4, + }, + minimum: ResourceLevels::MACOS, + }, + } + } + + #[test] + fn lease_parent_lease_is_stated_only_to_clients_that_require_it() { + let open = open(); + let plain = connect( + &open, + &[Capability::DurableAdmission, Capability::FencedLaunch], + ); + assert!(!plain.hello.capabilities.contains(&Capability::ParentLease)); + assert!(plain.hello.capabilities.contains(&Capability::FencedLaunch)); + let asking = connect(&open, &[Capability::ParentLease]); + assert!(asking.hello.capabilities.contains(&Capability::ParentLease)); + } + + #[test] + fn lease_children_draw_on_the_lease_and_the_reconciler_releases_it() { + let open = open(); + let mut owner = owner(&open, "lease-owner"); + let lease = key(&open, "lease-1"); + let granted = owner + .admit_lease(lease.clone(), lease_budget(), None) + .unwrap(); + assert_eq!(granted.lease.phase, LeasePhase::Active); + let token = granted.token.unwrap(); + assert!(owner + .admit_lease(lease.clone(), lease_budget(), None) + .unwrap() + .token + .is_none()); + for id in ["child-1", "child-2"] { + let record = owner + .admit_child(lease.clone(), token.clone(), child(&open, id)) + .unwrap(); + assert_eq!(record.phase, AttemptPhase::Prepared, "{id}"); + } + // The two children fill the lease; a third exceeds it. + let third = owner + .admit_child(lease.clone(), token.clone(), child(&open, "child-3")) + .unwrap(); + assert_eq!(third.denial, Some(ErrorCode::ResourceUnavailable)); + assert_eq!(open.authority.committed().unwrap(), lease_budget()); + let view = holder(&open) + .lease_status(lease.clone(), token.clone()) + .unwrap(); + assert_eq!(view.remaining, Budget::ZERO); + assert_eq!(view.children.len(), 3); + // Ended: fenced, and released by the reconciler once the + // children are settled. + let ended = owner.end_lease(lease.clone()).unwrap(); + assert_eq!(ended.lease.phase, LeasePhase::Ending); + let fenced = owner + .admit_child(lease.clone(), token.clone(), child(&open, "child-4")) + .unwrap(); + assert_eq!(fenced.denial, Some(ErrorCode::InvalidTransition)); + for id in ["child-1", "child-2"] { + assert_eq!( + owner.cancel(key(&open, id)).unwrap().phase, + AttemptPhase::Cancelled + ); + } + let deadline = std::time::Instant::now() + Duration::from_secs(10); + loop { + let view = holder(&open) + .lease_status(lease.clone(), token.clone()) + .unwrap(); + if view.lease.phase == LeasePhase::Released { + break; + } + assert!(std::time::Instant::now() < deadline, "{view:?}"); + std::thread::sleep(Duration::from_millis(100)); + } + assert_eq!(open.authority.committed().unwrap(), Budget::ZERO); + } + + #[test] + fn lease_a_holder_presents_only_the_token_and_makes_no_other_request() { + let open = open(); + let mut owner = owner(&open, "lease-owner"); + let lease = key(&open, "lease-1"); + let token = owner + .admit_lease(lease.clone(), lease_budget(), None) + .unwrap() + .token + .unwrap(); + let wrong = Secret::new("d".repeat(64)).unwrap(); + assert_eq!( + holder(&open) + .lease_status(lease.clone(), wrong) + .unwrap_err() + .code, + ErrorCode::Unauthorized + ); + // A caller cannot turn its session into a lease holder's. + let mut caller = connect(&open, &[Capability::ParentLease]); + caller + .authenticate(open.authority.consumer().unwrap()) + .unwrap(); + assert_eq!( + caller + .lease_status(lease.clone(), token.clone()) + .unwrap_err() + .code, + ErrorCode::Unauthorized + ); + // A holder session asks only for its lease's status. + let mut holder = holder(&open); + holder.lease_status(lease.clone(), token.clone()).unwrap(); + assert_eq!( + holder + .authenticate(open.authority.consumer().unwrap()) + .unwrap_err() + .code, + ErrorCode::Unauthorized + ); + owner.end_lease(lease).unwrap(); + } + } + /// Sessions of a service with registration and fenced launch open. #[cfg(target_os = "macos")] mod launch { @@ -1874,7 +2246,7 @@ mod tests { let paths = AuthorityPaths::fixture(directory.path()); let config = config::initialize(&paths).unwrap(); let storage = AuthorityStorage::open(&paths.journal()).unwrap(); - let (evidence, reason) = activate(storage, &config, &paths, None).unwrap(); + let (evidence, reason) = activate(storage, &config, &paths, None, None).unwrap(); assert_eq!(reason, LAUNCH_REASON); let Evidence::Native { authority, clock, .. diff --git a/crates/daemon/tests/entrypoint.rs b/crates/daemon/tests/entrypoint.rs index d7957e1..f3dc278 100644 --- a/crates/daemon/tests/entrypoint.rs +++ b/crates/daemon/tests/entrypoint.rs @@ -19,6 +19,42 @@ fn alternate_authority_and_unparented_candidate_arguments_are_rejected() { } } +#[test] +fn a_candidate_needs_its_lease_token_descriptor_and_names_no_other_authority() { + let run = |extra: &[&str]| { + let mut arguments = vec![ + "candidate", + "--id", + "c-entrypoint", + "--lease", + "dev-cli/g/lease-1", + "--capacity", + "1000,1073741824,64", + // Far above anything a test harness holds open. + "--token-fd", + "200", + ]; + arguments.extend_from_slice(extra); + Command::new(env!("CARGO_BIN_EXE_devguardd")) + .args(arguments) + .output() + .unwrap() + }; + let named = run(&["--socket", "/tmp/second.sock"]); + assert!(!named.status.success()); + assert!(String::from_utf8(named.stderr) + .unwrap() + .contains("no alternate authority arguments")); + // Without the inherited token nothing is contacted or created. + let untokened = run(&[]); + assert!(!untokened.status.success()); + if cfg!(target_os = "macos") { + assert!(String::from_utf8(untokened.stderr) + .unwrap() + .contains("private credential FD")); + } +} + #[test] fn help_states_what_serve_opens_and_that_bootstrap_starts_no_workload() { let result = Command::new(env!("CARGO_BIN_EXE_devguardd")) @@ -30,6 +66,7 @@ fn help_states_what_serve_opens_and_that_bootstrap_starts_no_workload() { assert!(help.contains("with it, opens registration, fenced launch")); assert!(help.contains("install copies a package into a protected release")); assert!(help.contains("init and check never start workloads.")); + assert!(help.contains("candidate serves an isolated candidate authority")); } #[cfg(target_os = "macos")] diff --git a/docs/contracts.md b/docs/contracts.md index c2175a3..3741619 100644 --- a/docs/contracts.md +++ b/docs/contracts.md @@ -1,6 +1,6 @@ # Implemented authority contract -This document describes the implemented DG-0 authority, C01/C02 service, storage and transport behavior, C03 native macOS host evidence, C04 cooperative policy and scope evidence, the C05 fenced launch helper, C06 reconciliation, the C07 command-line owner, the C08 Cargo adapters and C09 installation as the current user's LaunchAgent. PR and post-merge main delivery evidence is tracked separately from implementation. [Operations](operations.md) lists actual command availability. With native host evidence the service opens registration, launch and reconciliation; CodeSpace integration remains unimplemented. [Korean translation](ko/contracts.md). +This document describes the implemented DG-0 authority, C01/C02 service, storage and transport behavior, C03 native macOS host evidence, C04 cooperative policy and scope evidence, the C05 fenced launch helper, C06 reconciliation, the C07 command-line owner, the C08 Cargo adapters, C09 installation as the current user's LaunchAgent and C10 parent leases with candidate authorities. PR and post-merge main delivery evidence is tracked separately from implementation. [Operations](operations.md) lists actual command availability. With native host evidence the service opens registration, launch and reconciliation; CodeSpace integration remains unimplemented. [Korean translation](ko/contracts.md). ## Authority and transport boundaries @@ -8,9 +8,9 @@ The core is a Rust library. `Authority::register` accepts a `TrustedPeer` and ve DG-0 proves the library registration boundary with a fake peer and backend. C02 supplies real local UDS authentication and a small client, but does not activate native registration. The server observes peer UID/PID through OS socket credentials; the client independently corroborates the authority UID/PID and its own identity in the handshake. Neither accepts caller-declared peer identity. C03 adds native boot/start identity, so an OS-observed peer UID/PID joined with the kernel's start identity forms a complete trusted registration observation; native tests exercise this path in-process. With native host evidence the service opens registration over the wire and the fenced launch helper described below, together with reconciliation. Without it, on another platform or after a failed macOS observation, workload/control-service registration returns `ResourceControlUnavailable`. Authentication alone issues no `Principal`, instance slot, lease or host budget. Administrative credentials cannot perform registration. -The handshake negotiates contract compatibility and wire version 1. A service with native evidence advertises durable admission, fenced launch, per-resource evidence, static control reservations and macOS cooperative control; without it, the service advertises none. Consumer generation and credential digest determine workload/control-service roles; an independent administrative digest grants only the explicitly exposed administrative role. All consumer and administrative digests must differ. Caller credentials, one-time helper permits and administrative operations are separate boundaries: a helper credential variant is not accepted as caller authentication. This remains a cooperative operating-account model, not isolation from a malicious same-UID process. +The handshake negotiates contract compatibility and wire version 1. A service with native evidence advertises durable admission, fenced launch, per-resource evidence, static control reservations and macOS cooperative control; without it, the service advertises none. Parent leases (C10) are stated only to a client that requires them. Consumer generation and credential digest determine workload/control-service roles; an independent administrative digest grants only the explicitly exposed administrative role. All consumer and administrative digests must differ. Caller credentials, one-time helper permits and administrative operations are separate boundaries: a helper credential variant is not accepted as caller authentication. This remains a cooperative operating-account model, not isolation from a malicious same-UID process. -The authority holds an exclusive no-follow lock in the journal's parent directory. All journals in that authority directory share the lock. The lock is opened with `O_NONBLOCK`, and its opened descriptor must identify a private, current-UID regular file with exactly one link, including during explicit initialization. A FIFO or linked file cannot stand in for the lock. C01 derives canonical normal-service paths from the OS account rather than caller HOME/XDG values and refuses state/socket overrides. Project configuration cannot carry authority credentials or capacity. Production candidate paths remain unavailable until C10 supplies a parent-lease boundary. +The authority holds an exclusive no-follow lock in the journal's parent directory. All journals in that authority directory share the lock. The lock is opened with `O_NONBLOCK`, and its opened descriptor must identify a private, current-UID regular file with exactly one link, including during explicit initialization. A FIFO or linked file cannot stand in for the lock. C01 derives canonical normal-service paths from the OS account rather than caller HOME/XDG values and refuses state/socket overrides. Project configuration cannot carry authority credentials or capacity. Candidate paths exist only under a parent lease (C10): `devguardd candidate` derives them from the account's own authority and accepts no other. `AuthorityStorage` exclusively opens and validates a journal without inventing a boot clock, recovering attempts or granting capabilities. `Authority::from_storage` activates it with an actual Backend/Clock and revalidates the accounting index inside the recovery transaction. `Authority::open` preserves that behavior through the same path. Explicit bootstrap remains separate from ordinary open; missing/corrupt/future-schema state is not repaired automatically. @@ -131,7 +131,7 @@ DG1-C07 adds `devguard`, the command-line owner of managed execution. It runs a - **Exit.** The CLI exits with the workload's status, or ends by the same signal; before it ends itself by a signal it sets its own core-file limit to zero, so only the workload may leave a core dump. It exits 125 when nothing was started, as `env` and `timeout` do, and passes the helper's 126 and 127 through. - **Receipts.** `--receipt PATH` creates a new private file before anything is admitted, and writes `devguard-exec-receipt/v1`: the result (`completed`, `exec_failed`, `not_started` or `uncertain`), the command, the adapter's report, the budget and its source, the project, the authority, every attempt with its reservation and lease ID, the wait with the work capacity it was checked against, the helper's phases, the exit, the observations before and after reap, and the signals received, forwarded and inherited as ignored. It names the variables an adapter set or removed, never their inherited values, and never contains a permit or a caller credential. - **Doctor.** `devguard doctor` reports the paths, the configuration, the helper, the service's identity, capabilities and status, and optionally registration and a project. It states whether managed execution is available, and that there is no unmanaged fallback. `--require admission,registration,macos-cooperative` makes it exit 1 unless each requirement holds. -- **Nested execution.** A workload that runs `devguard exec` again starts a separate managed execution under its own reservation and scope. It is not charged to, or contained by, the outer scope. Bounding nested work within a parent's budget is C10's parent-budget capability. +- **Nested execution.** A workload that runs `devguard exec` again starts a separate managed execution under its own reservation and scope. It is not charged to, or contained by, the outer scope. Bounding nested work within a parent's budget takes an explicit parent lease (`--lease`, C10). - **Adapters.** C07 defines the adapter interface, separate from admission. An adapter may change arguments and environment for the reservation it will run under, and it reports what it did. The generic adapter changes nothing. The Cargo adapters are described below. ## Cargo adapter @@ -188,6 +188,41 @@ DG1-C09 installs a packaged release as the current user's LaunchAgent. The servi - **Restart.** A restarted service reopens and reconciles the existing journal, as any start does. A missing or corrupt journal fails closed and stays down. - **Limits.** Replacing the installed release is an upgrade (C11); C09 refuses it. There is no uninstall command. `launchctl bootout gui//io.github.novelkr.devguard` and removing the plist stop the service and keep every release, recovery copy, the selection and the journal. +## Parent leases and candidate authorities + +DG1-C10 lets the stable authority lend one bounded budget, a parent lease, to the verification of a candidate build, instead of issuing the host's budget a second time. + +- **Capability.** The service states `parent_lease` only to a client whose handshake requires it, so an earlier client never receives a capability it cannot decode. Clients send lease requests only after that statement. +- **Lease.** A registered workload owner requests `AdmitLease {key, budget, ttl}`. + - The lease is admitted against the host's capacity at the current pressure, like any request. Its whole budget stays charged until it is released. + - Only the first reply carries a one-time 64-character token; a replay returns the lease without it. The journal keeps only the token's digest, and receipts never contain the token. +- **Children.** `AdmitChild {lease, token, request}` from a registered instance of the lease's consumer and generation admits an ordinary attempt against the lease's remainder: its budget less the reservations of its charged children. + - The host's capacity and pressure are not consulted again, so a live lease is never shrunk. + - A wrong token or an unknown lease is `Unauthorized`. A child larger than the remainder is denied `ResourceUnavailable`, and a lease that is no longer active denies every child `InvalidTransition`. + - An admitted child is launched, reconciled and released like any attempt, through the stable `devguard-launch`. Its release returns its budget to the lease, not to the host. +- **End.** A lease is Active, then Ending, then Released. + - It becomes Ending when its owner ends it (`EndLease`), when its owner's process ends or belongs to an earlier boot, or when its deadline passes. An Ending lease admits no child. + - It is Released once none of its children is charged, and its budget returns to the host. A Suspect child keeps the lease charged. The reconciler settles leases on every pass. +- **Holders.** `LeaseStatus {key, token}` is a session of its own: it presents only the token, is never authenticated as a caller and can make no other request. It reports the lease's phase, budget, remainder, deadline and children. +- **Journal.** The tables `leases` and `lease_children` are added when absent, and the journal schema stays 1. Committed capacity is the budget of every unreleased lease plus the reservations of charged attempts that are not lease children. A generation holding an unreleased lease cannot be retired. A C09 artifact that opens such a journal ignores the new tables: it counts the children as ordinary attempts and does not hold the unused remainder of a lease. +- **Candidate authority.** `devguardd candidate --id ID --lease CONSUMER/GENERATION/ATTEMPT --capacity MILLICPU,BYTES,TASKS --token-fd N` serves an isolated authority whose whole capacity is a parent lease of the account's authority. + - It reads the lease token from the inherited descriptor N and closes it. Without the token it contacts and creates nothing. + - It asks the parent for the lease's status as a holder. The lease must be active and hold the capacity. A capacity that does not exceed the candidate daemon's own reservation (250 mCPU, 128 MiB and the bootstrap system tasks) is refused before anything is created. + - Its state, configuration, credentials, socket and cache live under `candidates/` in the authority root and runtime directories, and must not exist yet: each candidate starts from new state. It never opens the normal state, credentials, releases or recovery copies. + - Its policy has the leased capacity and no host headroom; the host's capacity is not used. It samples host pressure like any authority. + - It advertises durable admission, per-resource evidence, static control reservations and macOS cooperative control, but neither fenced launch nor parent leases. It refuses launch, helper and lease requests with `ResourcePolicyUnsupported`. Its status reports registration ready and execution not ready, with a reason that names the candidate and its lease. + - It confirms the lease every second and closes as soon as the lease is no longer active. It fails closed when the parent refuses the token, or cannot confirm the lease for five seconds. + - The parent charges the capacity when it admits the candidate as a lease child, so the candidate's admissions can never exceed the lease. +- **Lease children from the CLI.** `devguard exec --lease CONSUMER/GENERATION/ATTEMPT --lease-token-fd N` admits the command as a child of that lease. It reads the token from descriptor N first and closes it, so the workload never inherits it. It requires the parent-lease capability, records the lease in the receipt, and checks a wait against the lease's budget instead of the host's capacity. +- **Test-candidate.** `devguard test-candidate --candidate DIR --report DIR [--cpu MILLICPU] [--memory SIZE] [--tasks N] [--ttl DURATION] [--wait DURATION]` verifies a candidate tree under one lease of the stable authority: + 1. It admits the lease: by default 2 CPU, 4 GiB and 96 tasks, with a two-hour deadline. `--wait` asks again while capacity or pressure refuses it. + 2. It runs each workload as a lease child through `devguard exec --lease`: the tree's `cargo build` of the daemon, CLI and helper, then `cargo test` of the crates whose tests create no process group or session and observe no host capacity (contract, core, client and the Cargo adapter), then the tree's own `devguardd candidate` with 1 CPU, 1 GiB and 64 tasks. + 3. From outside, it checks the candidate's endpoint: its capabilities and status, an admission within its capacity, a refused launch, a cancellation, a denial beyond its capacity and a refused nested lease. + 4. It ends the lease, waits for the candidate to close and the lease to be released, and removes the candidate's area. It writes `report.json` with the lease, each child's receipt and reservation and every check, and exits 0 only when all of them passed. + + A failure ends the lease first. Children that are running finish, and the lease is released once they are settled. If the command itself is killed, its lease ends by the owner rule. +- **Limits.** Workloads that create their own process groups or sessions would leave a lease child's scope and stay Suspect under the cooperative macOS model. The native launch, CLI, terminal and scope suites therefore remain bootstrap and CI qualification runs, not lease children. A candidate verifies admission only; its workloads never run through it. Real self-use under the installed parent needs a release that states parent leases. While host memory pressure keeps a service Critical, it admits no lease. + ## Durable admission and launch The journal schema is version 1. Initialization is explicit and only creates a new file. Opening a missing, corrupt, unknown-schema or inconsistent journal fails closed. SQLite uses WAL, FULL synchronous durability and immediate transactions. A startup scan validates the accounting index against every stored record. @@ -220,7 +255,7 @@ Every resource carries its own level and method. Accounting is not an OS memory ## Boundaries deliberately left to later milestones -- Remaining DG-1: bounded self-use, update/repair and measured macOS SLOs. +- Remaining DG-1: real self-use under an installed C10 parent, update/repair and measured macOS SLOs. - CS-RG: Runner slots and transport lanes, approval migration, pinned client, process status integration and regression qualification. - DG-LINUX: actual cgroup hierarchy, controllers, ancestor constraints and sandbox/proxy inclusion. - DG-CACHE / DG-ADAPTERS: registered cache reclamation and additional tool-specific controls. diff --git a/docs/ko/contracts.md b/docs/ko/contracts.md index c769fc4..f1bc0b8 100644 --- a/docs/ko/contracts.md +++ b/docs/ko/contracts.md @@ -1,6 +1,6 @@ # 구현된 authority 계약 -이 문서는 구현된 DG-0 authority, C01/C02 서비스·저장소·transport 동작, C03 native macOS 호스트 증거, C04 협조적 정책·scope 증거, C05 fenced launch helper, C06 대조, C07 명령행 owner, C08 Cargo adapter, 현재 사용자 LaunchAgent로의 C09 설치를 설명한다. PR과 병합 후 main 전달 증거는 구현과 별도로 추적한다. 실제 제공 명령은 [운영 문서](operations.md)에 있다. Native 호스트 증거가 있으면 서비스는 등록·launch·대조를 열며, CodeSpace 결합은 아직 구현하지 않았다. [영문 정본](../contracts.md). +이 문서는 구현된 DG-0 authority, C01/C02 서비스·저장소·transport 동작, C03 native macOS 호스트 증거, C04 협조적 정책·scope 증거, C05 fenced launch helper, C06 대조, C07 명령행 owner, C08 Cargo adapter, 현재 사용자 LaunchAgent로의 C09 설치, C10 부모 lease와 후보 authority를 설명한다. PR과 병합 후 main 전달 증거는 구현과 별도로 추적한다. 실제 제공 명령은 [운영 문서](operations.md)에 있다. Native 호스트 증거가 있으면 서비스는 등록·launch·대조를 열며, CodeSpace 결합은 아직 구현하지 않았다. [영문 정본](../contracts.md). ## Authority와 transport 경계 @@ -8,9 +8,9 @@ Core는 Rust 라이브러리다. Authority::register는 TrustedPeer를 받고, DG-0는 가짜 peer·backend로 라이브러리 등록 경계를 검증한다. C02는 실제 로컬 UDS 인증과 작은 client를 제공하지만 native 등록은 활성화하지 않는다. 서버는 OS socket 자격으로 peer UID/PID를 관측하고, client는 handshake의 authority UID/PID와 자기 정체성을 독립적으로 대조한다. 호출자가 선언한 peer 정체성을 신뢰하지 않는다. C03은 native boot/start 정체성을 추가하므로, OS가 관측한 peer UID/PID와 kernel start 정체성을 결합하면 완전한 신뢰 등록 관측이 된다. Native 시험은 이 경로를 프로세스 안에서 검증한다. Native 호스트 증거가 있으면 서비스는 wire 등록과 아래의 fenced launch helper를 대조와 함께 연다. 다른 플랫폼이거나 macOS 관측이 실패해 그 증거가 없으면 workload/control-service 등록은 ResourceControlUnavailable을 반환한다. 인증만으로 Principal·instance 슬롯·lease·호스트 예산을 발급하지 않는다. 관리 자격으로는 등록할 수 없다. -Handshake는 contract 호환성과 wire version 1을 확인한다. Native 증거가 있는 서비스는 내구성 admission, fenced launch, 자원별 증거, 정적 제어 예약, macOS 협조적 제어를 광고하며, 증거가 없으면 아무것도 광고하지 않는다. Consumer generation과 자격 digest가 workload/control-service 역할을 결정하며 별도 관리 digest는 명시적으로 공개한 관리 역할에만 사용한다. 모든 consumer·관리 digest는 서로 달라야 한다. Caller 자격, 일회성 helper permit과 관리 작업은 별도 경계이며 helper 자격 variant를 caller 인증으로 받지 않는다. 이는 협조적인 운영 계정 모델이며 악의적인 동일 UID 프로세스를 격리하지 않는다. +Handshake는 contract 호환성과 wire version 1을 확인한다. Native 증거가 있는 서비스는 내구성 admission, fenced launch, 자원별 증거, 정적 제어 예약, macOS 협조적 제어를 광고하며, 증거가 없으면 아무것도 광고하지 않는다. 부모 lease(C10)는 이를 요구한 client에게만 알린다. Consumer generation과 자격 digest가 workload/control-service 역할을 결정하며 별도 관리 digest는 명시적으로 공개한 관리 역할에만 사용한다. 모든 consumer·관리 digest는 서로 달라야 한다. Caller 자격, 일회성 helper permit과 관리 작업은 별도 경계이며 helper 자격 variant를 caller 인증으로 받지 않는다. 이는 협조적인 운영 계정 모델이며 악의적인 동일 UID 프로세스를 격리하지 않는다. -Authority는 journal 부모 디렉터리에 no-follow 배타 잠금을 보유한다. 같은 디렉터리의 모든 journal은 그 잠금을 공유한다. 잠금은 O_NONBLOCK으로 열고 열린 descriptor가 현재 UID 소유의 private 일반 파일이며 링크가 정확히 하나인지 명시 초기화 때도 확인한다. FIFO·링크된 파일은 잠금 대신 사용할 수 없다. C01은 caller HOME·XDG가 아니라 OS 계정으로 정상 경로를 결정하고 state·socket override를 거절한다. 프로젝트 설정에는 authority 자격이나 호스트 용량을 둘 수 없다. 운영 후보 경로는 C10의 부모 lease 경계가 제공될 때까지 사용할 수 없다. +Authority는 journal 부모 디렉터리에 no-follow 배타 잠금을 보유한다. 같은 디렉터리의 모든 journal은 그 잠금을 공유한다. 잠금은 O_NONBLOCK으로 열고 열린 descriptor가 현재 UID 소유의 private 일반 파일이며 링크가 정확히 하나인지 명시 초기화 때도 확인한다. FIFO·링크된 파일은 잠금 대신 사용할 수 없다. C01은 caller HOME·XDG가 아니라 OS 계정으로 정상 경로를 결정하고 state·socket override를 거절한다. 프로젝트 설정에는 authority 자격이나 호스트 용량을 둘 수 없다. 후보 경로는 부모 lease 아래에만 있다(C10). `devguardd candidate`는 이를 계정 자신의 authority에서 정하며 다른 경로는 받지 않는다. AuthorityStorage는 boot clock을 만들거나 attempt를 복구하거나 capability를 부여하지 않고 journal의 배타 열기·검사만 수행한다. Authority::from_storage는 실제 Backend·Clock으로 활성화하며 복구 transaction 안에서 회계 인덱스를 다시 검증한다. 기존 Authority::open도 같은 경로로 기존 복구 동작을 유지한다. 명시 bootstrap과 일반 open은 별개이며 누락·손상·미래 schema를 자동 수리하지 않는다. @@ -131,7 +131,7 @@ DG1-C07은 관리 실행의 명령행 owner인 `devguard`를 추가한다. 명 - **종료값.** CLI는 workload의 종료값으로 끝나거나 같은 signal로 끝난다. Signal로 스스로 끝나기 전에 자신의 core file 상한을 0으로 설정하므로 core dump는 workload만 남길 수 있다. 아무것도 시작하지 않았으면 `env`·`timeout`처럼 125로 끝나며, helper의 126·127은 그대로 전달한다. - **Receipt.** `--receipt PATH`는 admission 전에 새 private 파일을 만들고 `devguard-exec-receipt/v1`을 기록한다. 결과(`completed`·`exec_failed`·`not_started`·`uncertain`), 명령, adapter 보고, 예산과 그 출처, 프로젝트, authority, 예약과 lease ID를 포함한 모든 attempt, 비교에 쓴 작업 용량을 포함한 대기, helper phase, 종료값, reap 전후의 관측, 받은 signal·전달한 signal·무시 상태로 상속한 signal을 담는다. Adapter가 설정하거나 제거한 변수는 이름만 기록하고 상속된 값은 기록하지 않으며, permit과 호출자 자격은 담지 않는다. - **Doctor.** `devguard doctor`는 경로, 설정, helper, 서비스의 정체성·capability·상태를 보고하고, 선택적으로 등록과 프로젝트도 확인한다. 관리 실행을 사용할 수 있는지와 관리되지 않는 대체 실행이 없다는 점을 밝힌다. `--require admission,registration,macos-cooperative`를 주면 요구 사항이 하나라도 충족되지 않을 때 1로 끝난다. -- **중첩 실행.** Workload가 `devguard exec`를 다시 실행하면 자체 예약과 scope를 가진 별도의 관리 실행이 시작된다. 이 실행은 바깥 scope에 과금되지 않고 바깥 scope에 포함되지도 않는다. 중첩 작업을 부모 예산 안으로 제한하는 것은 C10의 부모 예산 기능이다. +- **중첩 실행.** Workload가 `devguard exec`를 다시 실행하면 자체 예약과 scope를 가진 별도의 관리 실행이 시작된다. 이 실행은 바깥 scope에 과금되지 않고 바깥 scope에 포함되지도 않는다. 중첩 작업을 부모 예산 안으로 제한하려면 명시적인 부모 lease(`--lease`, C10)가 필요하다. - **Adapter.** C07은 admission과 분리된 adapter 인터페이스를 정의한다. Adapter는 실행될 예약에 맞춰 인자와 환경을 바꿀 수 있으며 무엇을 했는지 보고한다. Generic adapter는 아무것도 바꾸지 않는다. Cargo adapter는 아래에서 설명한다. ## Cargo adapter @@ -188,6 +188,41 @@ DG1-C09는 패키지로 만든 release를 현재 사용자의 LaunchAgent로 설 - **재시작.** 다시 시작한 서비스는 다른 시작과 마찬가지로 기존 journal을 다시 열어 대조한다. Journal이 없거나 손상되었으면 닫힌 채 멈춰 있다. - **범위.** 설치된 release를 교체하는 것은 upgrade(C11)이며 C09는 이를 거절한다. 제거 명령은 없다. `launchctl bootout gui//io.github.novelkr.devguard`를 실행하고 plist를 지우면 서비스가 멈추며, 모든 release·복구 사본·선택·journal은 보존된다. +## 부모 lease와 후보 authority + +DG1-C10은 안정 authority가 호스트 예산을 두 번 발행하지 않고, 제한된 예산 하나인 부모 lease를 후보 build 검증에 빌려주게 한다. + +- **Capability.** 서비스는 handshake에서 `parent_lease`를 요구한 client에게만 이를 알린다. 따라서 이전 client는 decode할 수 없는 capability를 받지 않는다. Client는 이 capability를 확인한 뒤에만 lease 요청을 보낸다. +- **Lease.** 등록된 workload owner가 `AdmitLease {key, budget, ttl}`를 요청한다. + - Lease는 다른 요청과 마찬가지로 현재 압력에서의 호스트 용량에 대해 admission된다. 전체 예산은 해제될 때까지 과금된다. + - 첫 응답에만 일회성 64자 token이 들어 있다. 재요청은 token 없이 lease만 돌려준다. Journal은 token의 digest만 보관하며 receipt에는 token이 들어가지 않는다. +- **자식.** Lease와 같은 consumer·generation의 등록된 instance가 `AdmitChild {lease, token, request}`를 보내면, lease의 남은 예산에 대해 일반 attempt를 admission한다. 남은 예산은 lease 예산에서 과금 중인 자식들의 예약을 뺀 값이다. + - 호스트 용량과 압력은 다시 확인하지 않으므로 살아 있는 lease가 줄어들지 않는다. + - Token이 틀리거나 lease를 모르면 `Unauthorized`이다. 남은 예산보다 큰 자식은 `ResourceUnavailable`로 거절하고, 더 이상 active가 아닌 lease는 모든 자식을 `InvalidTransition`으로 거절한다. + - Admission된 자식은 안정 `devguard-launch`를 거쳐 다른 attempt와 똑같이 launch·대조·해제된다. 해제된 예산은 호스트가 아니라 lease로 돌아간다. +- **종료.** Lease는 Active, Ending, Released 순으로 진행한다. + - Owner가 끝내거나(`EndLease`), owner 프로세스가 끝났거나 이전 boot의 것이거나, 기한이 지나면 Ending이 된다. Ending lease는 자식을 admission하지 않는다. + - 과금 중인 자식이 하나도 없으면 Released가 되고 예산은 호스트로 돌아간다. Suspect 자식은 lease를 과금 상태로 유지한다. Reconciler는 매 pass마다 lease를 정리한다. +- **보유자.** `LeaseStatus {key, token}`은 별도 session이다. Token만 제시하고 caller로 인증되지 않으며 다른 요청을 할 수 없다. Lease의 phase·예산·남은 예산·기한·자식을 보고한다. +- **Journal.** `leases`와 `lease_children` table은 없을 때 추가하며 journal schema는 1로 유지한다. 과금 용량은 해제되지 않은 모든 lease의 예산과, lease 자식이 아닌 과금 attempt의 예약을 더한 값이다. 해제되지 않은 lease를 가진 generation은 폐기할 수 없다. 이런 journal을 여는 C09 artifact는 새 table을 무시한다. 자식은 일반 attempt로 계산하고, lease의 쓰지 않은 남은 예산은 보유하지 않는다. +- **후보 authority.** `devguardd candidate --id ID --lease CONSUMER/GENERATION/ATTEMPT --capacity MILLICPU,BYTES,TASKS --token-fd N`은 전체 용량이 계정 authority의 부모 lease인 격리 authority를 제공한다. + - 상속한 descriptor N에서 lease token을 읽고 닫는다. Token이 없으면 아무것에도 연결하지 않고 아무것도 만들지 않는다. + - 보유자로서 부모에게 lease 상태를 묻는다. Lease는 active이고 용량을 담을 수 있어야 한다. 후보 daemon 자신의 예약(250 mCPU, 128 MiB와 bootstrap system task)을 넘지 않는 용량은 아무것도 만들기 전에 거절한다. + - 상태·설정·자격·socket·cache는 authority root와 runtime 디렉터리의 `candidates/` 아래에 두며, 아직 없어야 한다. 모든 후보는 새 상태에서 시작한다. 정상 상태·자격·release·복구 사본은 열지 않는다. + - 정책은 lease 용량을 쓰며 호스트 headroom은 없다. 호스트 용량은 쓰지 않는다. 호스트 압력은 다른 authority와 같이 sample한다. + - 내구성 admission, 자원별 증거, 정적 제어 예약, macOS 협조적 제어를 광고하지만 fenced launch와 부모 lease는 광고하지 않는다. Launch·helper·lease 요청은 `ResourcePolicyUnsupported`로 거절한다. Status는 등록 준비, 실행 미준비를 보고하며, 이유에 후보와 그 lease를 적는다. + - 매초 lease를 확인하고 lease가 더 이상 active가 아니면 즉시 닫는다. 부모가 token을 거절하거나 5초 동안 lease를 확인하지 못하면 닫힌 채 끝난다. + - 부모는 후보를 lease 자식으로 admission할 때 그 용량을 과금하므로, 후보의 admission은 lease를 넘을 수 없다. +- **CLI의 lease 자식.** `devguard exec --lease CONSUMER/GENERATION/ATTEMPT --lease-token-fd N`은 명령을 그 lease의 자식으로 admission한다. Descriptor N에서 token을 먼저 읽고 닫으므로 workload는 이를 상속하지 않는다. 부모 lease capability를 요구하고, receipt에 lease를 기록하며, 대기는 호스트 용량 대신 lease 예산과 비교한다. +- **Test-candidate.** `devguard test-candidate --candidate DIR --report DIR [--cpu MILLICPU] [--memory SIZE] [--tasks N] [--ttl DURATION] [--wait DURATION]`는 안정 authority의 lease 하나 안에서 후보 tree를 검증한다. + 1. Lease를 admission한다. 기본값은 CPU 2개, 4 GiB, task 96개이며 기한은 2시간이다. `--wait`를 주면 용량이나 압력으로 거절되는 동안 다시 요청한다. + 2. 각 workload를 `devguard exec --lease`로 lease 자식으로 실행한다. 순서는 tree의 daemon·CLI·helper `cargo build`, 시험이 process group이나 session을 만들지 않고 호스트 용량을 관측하지 않는 crate(contract, core, client, Cargo adapter)의 `cargo test`, 그리고 CPU 1개·1 GiB·task 64개를 쓰는 tree 자신의 `devguardd candidate`이다. + 3. 바깥에서 후보 endpoint를 점검한다. Capability와 status, 용량 안의 admission, 거절되는 launch, 취소, 용량을 넘는 요청의 거절, 거절되는 중첩 lease를 확인한다. + 4. Lease를 끝내고 후보가 닫히고 lease가 해제될 때까지 기다린 뒤 후보 영역을 지운다. Lease, 각 자식의 receipt와 예약, 모든 점검을 담은 `report.json`을 쓰며, 모두 통과했을 때만 0으로 끝난다. + + 실패하면 먼저 lease를 끝낸다. 실행 중인 자식은 끝까지 실행되고, 모두 정리되면 lease가 해제된다. 명령 자체가 kill되면 owner 규칙에 따라 lease가 끝난다. +- **범위.** 자체 process group이나 session을 만드는 workload는 lease 자식의 scope를 벗어나 협조적 macOS 모델에서 Suspect로 남는다. 따라서 native launch·CLI·terminal·scope suite는 lease 자식이 아니라 bootstrap과 CI qualification 실행으로 남는다. 후보는 admission만 검증하며 후보의 workload는 후보를 거쳐 실행되지 않는다. 설치된 부모 아래의 실제 자기 적용에는 부모 lease를 광고하는 release가 필요하다. 호스트 메모리 압력으로 서비스가 Critical인 동안에는 lease를 admission하지 않는다. + ## 내구성 admission과 launch Journal schema는 1이다. 초기화는 명시적으로 새 파일만 만든다. 누락·손상·미지원 schema·불일치 journal은 fail-closed다. SQLite는 WAL, FULL synchronous와 immediate transaction을 사용한다. 시작 시 모든 저장 record와 회계 index를 대조한다. @@ -220,7 +255,7 @@ Journal은 instance의 등록 정책을 기록한다. Active/suspect instance가 ## 후속 마일스톤의 책임 -- 남은 DG-1: bounded 자기 적용, update·repair, 측정한 macOS SLO. +- 남은 DG-1: 설치된 C10 부모 아래의 실제 자기 적용, update·repair, 측정한 macOS SLO. - CS-RG: Runner 슬롯·전송 lane, 승인 migration, client pin, 상태 결합과 회귀 qualification. - DG-LINUX: 실제 cgroup 계층·controller·ancestor와 sandbox·proxy 포함. - DG-CACHE·DG-ADAPTERS: 등록 cache 회수와 추가 도구 제어. diff --git a/docs/ko/operations.md b/docs/ko/operations.md index 5f6020a..8a3f84f 100644 --- a/docs/ko/operations.md +++ b/docs/ko/operations.md @@ -2,7 +2,7 @@ [English](../operations.md) | [한국어](operations.md) -C01은 명시 bootstrap·고정 저장소를, C02는 인증된 로컬 transport를, C03은 native macOS boot·프로세스·호스트 압력 증거를, C04는 협조적 정책 readback과 scope 증거를, C05는 fenced launch helper를, C06은 대조를, C07은 `devguard` 명령행 owner를, C08은 Cargo adapter를, C09는 패키지로 만든 release를 현재 사용자 LaunchAgent로 설치하는 기능을 제공한다. PR과 병합 후 main 전달 증거는 구현과 별도로 추적한다. devguardd serve는 이 증거로 journal을 활성화하는 foreground 서비스를 실행하고 wire 등록·fenced launch·대조를 연다. `devguard exec`는 이 서비스를 통해 명령을 실행하고, `devguard doctor`는 이를 진단한다. 정상 authority는 현재 macOS만 지원하며 Linux CI는 이식 가능한 계약과 제한된 fixture를 검사한다. DG-LINUX 제어 자격을 부여하지 않는다. +C01은 명시 bootstrap·고정 저장소를, C02는 인증된 로컬 transport를, C03은 native macOS boot·프로세스·호스트 압력 증거를, C04는 협조적 정책 readback과 scope 증거를, C05는 fenced launch helper를, C06은 대조를, C07은 `devguard` 명령행 owner를, C08은 Cargo adapter를, C09는 패키지로 만든 release를 현재 사용자 LaunchAgent로 설치하는 기능을, C10은 부모 lease와 후보 authority를 제공한다. PR과 병합 후 main 전달 증거는 구현과 별도로 추적한다. devguardd serve는 이 증거로 journal을 활성화하는 foreground 서비스를 실행하고 wire 등록·fenced launch·대조를 연다. `devguard exec`는 이 서비스를 통해 명령을 실행하고, `devguard doctor`는 이를 진단한다. 정상 authority는 현재 macOS만 지원하며 Linux CI는 이식 가능한 계약과 제한된 fixture를 검사한다. DG-LINUX 제어 자격을 부여하지 않는다. ## 제공 명령 @@ -51,7 +51,15 @@ devguardd status `package.py`는 깨끗한 tree에서 release 패키지를 만든다. `install`은 그 패키지에서 실행해야 하며, authority가 제공 중이거나 agent가 있거나 기존 authority 상태가 없으면 거절한다. Release를 변경 불가능한 `releases/`로 복사하고, 현재 사용자 LaunchAgent `io.github.novelkr.devguard`로 시작한다. 이 agent는 `releases//bin/devguardd serve`를 실행한다. launchd가 정확히 그 바이너리를 실행한다고 검증한 뒤에만 release를 선택한다. `devguardd status`는 설치된 서비스를 보고하며, 검증된 current release를 실행하지 않으면 1로 끝난다. Release 교체는 upgrade이며 repair와 함께 후속 작업(C11)이다. [계약 문서](contracts.md#설치와-현재-사용자-서비스)를 참조한다. -`devguard exec [--project ID] [--adapter auto|generic|cargo|cargo-pipeline] [--wait DURATION] [--cpu MILLICPU] [--memory SIZE] [--tasks N] [--receipt PATH] -- PROGRAM [ARGS...]`는 명령을 admission하고 `devguard-launch`로 시작한 뒤 끝날 때까지 기다린다. 프로그램과 인자는 `--` 뒤에 두며 shell은 사용하지 않는다. 명령의 종료값으로 끝나거나 그 signal로 끝나며, 아무것도 시작하지 않았으면 125로 끝난다. `--wait`가 없으면 거절 즉시 끝나고, `--wait`가 있으면 용량·압력 거절을 기한까지 다시 시도하며, 호스트의 작업 용량보다 큰 요청은 즉시 거절한다. `--receipt`는 private JSON receipt를 기록한다. `devguard doctor`는 JSON 진단을 출력하며, `--require admission,registration,macos-cooperative`를 주면 요구 사항이 하나라도 충족되지 않을 때 실패한다. [계약 문서](contracts.md#명령행-owner)를 참조한다. +후보 build는 실행 중인 서비스의 부모 lease 안에서 검증한다. + +```sh +devguard test-candidate --candidate /path/to/candidate/tree --report /path/to/new/report --wait 5m +``` + +`test-candidate`는 lease 하나(기본값 CPU 2개, 4 GiB, task 96개, 기한 2시간)를 admission하고, tree의 build, 해당하는 시험, tree 자신의 `devguardd candidate`를 `devguard exec --lease`로 lease 자식으로 실행한다. 후보의 admission을 바깥에서 점검하고 lease를 끝낸 뒤 새 보고 디렉터리에 `report.json`을 쓰며, 모든 단계가 통과했을 때만 0으로 끝난다. 서비스는 부모 lease를 광고해야 하므로 C10 이후 release가 필요하며, 압력이 Critical인 동안에는 lease를 admission하지 않는다. [계약 문서](contracts.md#부모-lease와-후보-authority)를 참조한다. + +`devguard exec [--project ID] [--adapter auto|generic|cargo|cargo-pipeline] [--wait DURATION] [--cpu MILLICPU] [--memory SIZE] [--tasks N] [--receipt PATH] [--lease CONSUMER/GENERATION/ATTEMPT --lease-token-fd N] -- PROGRAM [ARGS...]`는 명령을 admission하고 `devguard-launch`로 시작한 뒤 끝날 때까지 기다린다. 프로그램과 인자는 `--` 뒤에 두며 shell은 사용하지 않는다. 명령의 종료값으로 끝나거나 그 signal로 끝나며, 아무것도 시작하지 않았으면 125로 끝난다. `--wait`가 없으면 거절 즉시 끝나고, `--wait`가 있으면 용량·압력 거절을 기한까지 다시 시도하며, 호스트의 작업 용량보다 큰 요청은 즉시 거절한다. `--receipt`는 private JSON receipt를 기록한다. `--lease`는 명령을 그 부모 lease의 자식으로 만들며, descriptor N에서 읽은 token으로 lease의 남은 예산에 대해 admission한다. `devguard doctor`는 JSON 진단을 출력하며, `--require admission,registration,macos-cooperative`를 주면 요구 사항이 하나라도 충족되지 않을 때 실패한다. [계약 문서](contracts.md#명령행-owner)를 참조한다. `--adapter cargo`는 `cargo`의 compiler job을 예약에 맞추고, `--adapter cargo-pipeline`은 검증 script 같은 프로그램이 실행하는 Cargo들이 jobserver 하나를 공유하게 한다. 기본값인 `auto`는 compile하는 `cargo` 명령에 Cargo adapter를 고른다. Cargo job 하나(CPU 1개와 2 GiB)도 수용할 수 없는 예약은 거절한다. [계약 문서](contracts.md#cargo-adapter)를 참조한다. @@ -110,7 +118,7 @@ SDK 조회 예제는 CARGO_BUILD_JOBS=1 cargo build --locked -p devguard-client ## 호환성·검사·복귀 -Core의 AuthorityStorage는 기존 배타 잠금을 보유하고 schema 1 journal을 검사하되 attempt 상태를 바꾸지 않는다. 실제 Backend·Clock으로 활성화할 때 boot 기반 복구와 같은 transaction 안에서 회계를 다시 검증한다. 기존 Authority::open의 복구 동작과 DG-0 시험을 유지하며 C01–C06은 journal schema와 contract 직렬화를 바꾸지 않는다. C05와 C06은 서비스가 fenced launch를 광고한 뒤에만 client가 사용하는 wire 요청과 응답을 추가한다. 새 로컬 protocol은 미지 필드·버전·필수 capability 미지원을 엄격히 거절하며 필드 추가도 명시적 호환성 시험이 필요하다. +Core의 AuthorityStorage는 기존 배타 잠금을 보유하고 schema 1 journal을 검사하되 attempt 상태를 바꾸지 않는다. 실제 Backend·Clock으로 활성화할 때 boot 기반 복구와 같은 transaction 안에서 회계를 다시 검증한다. 기존 Authority::open의 복구 동작과 DG-0 시험을 유지하며 C01–C06은 journal schema와 contract 직렬화를 바꾸지 않는다. C05와 C06은 서비스가 fenced launch를 광고한 뒤에만 client가 사용하는 wire 요청과 응답을 추가한다. C10은 서비스가 부모 lease를 알린 뒤에만 client가 사용하는 lease 요청과 응답, 그리고 없을 때 만드는 `leases`·`lease_children` table을 추가하며 schema는 바꾸지 않는다. 새 로컬 protocol은 미지 필드·버전·필수 capability 미지원을 엄격히 거절하며 필드 추가도 명시적 호환성 시험이 필요하다. ```sh python3 scripts/qualify.py dg1-authority --offline @@ -122,6 +130,7 @@ python3 scripts/qualify.py dg1-reconcile --offline python3 scripts/qualify.py dg1-cli --offline python3 scripts/qualify.py dg1-cargo --offline python3 scripts/qualify.py dg1-bootstrap --offline +python3 scripts/qualify.py dg1-self-use --offline python3 scripts/validate.py --offline ``` @@ -222,6 +231,14 @@ Core·launcher 증거·서비스·wire 시험은 이전 boot 회수, 옛 PID를 launchd gui domain이 없는 session에서는 launchd 경우를 `not_run`으로 기록하고 suite는 `incomplete`를 보고한다. +`dg1-self-use`도 macOS에서만 실행하며 먼저 `devguard-launch`를 빌드한다. 격리 authority에 대해 실제 lease 자식과 실제 후보 authority 프로세스를 실행한다. 검사 항목은 다음과 같다. +- 호스트에 한 번 과금되는 lease, 남은 예산에 대해서만 admission되는 자식, lease를 넘지 않는 자식 합계 +- 요구한 client에게만 알리는 부모 lease capability, token만 쓰는 보유자 session, 틀린 token이나 모르는 lease의 거절 +- Lease가 끝났을 때, owner가 사라졌거나 이전 boot의 것일 때, 기한이 지났을 때의 fence, lease를 과금 상태로 유지하는 Suspect 자식, 모든 자식이 정리된 뒤의 해제와 재시작 뒤의 같은 동작 +- workload를 lease 자식으로 실행하며 표준 descriptor만 상속시키는 `devguard exec --lease`, lease보다 큰 자식(대기 포함), 위조 token, lease가 끝난 뒤 시작하는 자식 +- 용량 안에서 admission하고, 아무것도 launch하지 않고, lease를 갖지 않고, lease와 함께 닫히며, 부모 상태를 열지 않는 후보 authority, 그리고 위조 token, 모르거나 끝난 lease, lease보다 크거나 자기 예약 이하인 용량, 이미 있는 영역 때문에 거절되는 후보 +- workload와 후보를 lease 하나의 자식으로 실행한 뒤 그 lease를 해제하고 후보 영역을 지우는 `test-candidate`, 그리고 제공 전에 죽어 실행은 실패하지만 scope와 lease는 해제되는 후보 + 이 suite들은 Linux 강제, 자기 적용, foreground SLO를 not_run으로 남긴다. 전체 검증은 기존 44개 시험과 workspace crate 8개의 명시적 전체 의존 그래프를 검사한다. Daemon은 자기 시험에서 fixture를 켜기 위해서만 자신에게 의존한다. Launch crate는 시험의 격리 authority를 위해서만 daemon에 의존한다. CLI는 daemon의 경로·설정과 Cargo adapter에 의존하며, daemon fixture에는 시험에서만 의존한다. Cargo adapter는 contract에만 의존한다. Core·contract는 daemon 설정과 native adapter에 독립적이며 client는 core에 의존하지 않는다. -복귀할 때 작업 소유 foreground 프로세스를 중지하고 보존한 설정과 schema 1 journal에 호환되는 source/artifact를 선택하며 영속 상태·자격을 유지한다. C03 활성화는 새 record 종류를 추가하지 않고 기존 복구만 수행하므로 C02 artifact로 같은 상태를 다시 열 수 있다. C06부터는 정상 서비스를 통해 workload가 시작될 수 있다. 대조가 없는 artifact로 복귀하기 전에 새 작업 시작을 멈추고 과금 중인 attempt가 종결 phase에 이를 때까지 기다린다. C05 artifact는 같은 journal을 읽지만 launch를 닫아 두고 대조하지 않으므로 남은 attempt는 과금된 Suspect로 남는다. Scope가 실행 중일 때 서비스가 멈추면 다시 시작한다. 그 attempt들은 owner가 claim되지 않은 grant를 보고하거나 reboot가 종료를 입증할 때까지 과금된 Suspect로 남는다. C07은 서비스 상태를 바꾸지 않는다. CLI를 되돌리려면 CLI로 명령을 시작하는 것을 멈춘다. 이미 시작한 명령은 scope가 끝날 때까지 과금되며, 관리되지 않는 실행으로 대체하지 않는다. C08도 서비스 상태를 바꾸지 않는다. Cargo job 조정을 멈추려면 `--adapter generic`을 명시하며, 기존 target과 cache는 유지된다. C09 installer는 release가 검증되기 전에는 job을 bootout하고 plist를 제거하며 아무것도 선택하지 않는다. 설치된 서비스를 멈추려면 `launchctl bootout gui//io.github.novelkr.devguard`를 실행하고 plist를 지운다. Release·복구 사본·선택·journal은 보존되며, 보존한 artifact의 foreground `devguardd serve`가 같은 상태를 읽는다. 복귀나 재시작을 성공시키려고 journal이나 tombstone을 삭제하지 않는다. +복귀할 때 작업 소유 foreground 프로세스를 중지하고 보존한 설정과 schema 1 journal에 호환되는 source/artifact를 선택하며 영속 상태·자격을 유지한다. C03 활성화는 새 record 종류를 추가하지 않고 기존 복구만 수행하므로 C02 artifact로 같은 상태를 다시 열 수 있다. C06부터는 정상 서비스를 통해 workload가 시작될 수 있다. 대조가 없는 artifact로 복귀하기 전에 새 작업 시작을 멈추고 과금 중인 attempt가 종결 phase에 이를 때까지 기다린다. C05 artifact는 같은 journal을 읽지만 launch를 닫아 두고 대조하지 않으므로 남은 attempt는 과금된 Suspect로 남는다. Scope가 실행 중일 때 서비스가 멈추면 다시 시작한다. 그 attempt들은 owner가 claim되지 않은 grant를 보고하거나 reboot가 종료를 입증할 때까지 과금된 Suspect로 남는다. C07은 서비스 상태를 바꾸지 않는다. CLI를 되돌리려면 CLI로 명령을 시작하는 것을 멈춘다. 이미 시작한 명령은 scope가 끝날 때까지 과금되며, 관리되지 않는 실행으로 대체하지 않는다. C08도 서비스 상태를 바꾸지 않는다. Cargo job 조정을 멈추려면 `--adapter generic`을 명시하며, 기존 target과 cache는 유지된다. C09 installer는 release가 검증되기 전에는 job을 bootout하고 plist를 제거하며 아무것도 선택하지 않는다. 설치된 서비스를 멈추려면 `launchctl bootout gui//io.github.novelkr.devguard`를 실행하고 plist를 지운다. Release·복구 사본·선택·journal은 보존되며, 보존한 artifact의 foreground `devguardd serve`가 같은 상태를 읽는다. C10은 lease table 말고는 서비스 상태를 추가하지 않는다. C09 artifact로 복귀하기 전에 모든 lease를 끝내고 해제되기를 기다린다. C09 서비스는 lease table을 무시하므로 lease의 쓰지 않은 남은 예산을 보유하지 않는다. `candidates/` 아래의 후보 영역은 버려도 되는 상태이며 정상 서비스는 읽지 않는다. 복귀나 재시작을 성공시키려고 journal이나 tombstone을 삭제하지 않는다. diff --git a/docs/ko/planning/milestones/DG-1.md b/docs/ko/planning/milestones/DG-1.md index 9f015e2..2e0e98e 100644 --- a/docs/ko/planning/milestones/DG-1.md +++ b/docs/ko/planning/milestones/DG-1.md @@ -4,7 +4,7 @@ 아래 ID·제목·PR 묶음은 **예정 값**이다. 실제 SHA나 GitHub PR 번호가 아니다. 모듈 경로는 구현 전까지 예정 책임을 나타낸다. 현재 daemon/client crate는 존재하며 launcher·platform-macos·cli·adapters는 후속 책임이다. crate 추가는 해당 PR에서 명시적 의존 allowlist와 전체 그래프 검증을 함께 확장하고 검사를 제거하지 않는다. 실제 상태는 ledger가 소유한다. -구현된 C01은 정상 경로·명시 bootstrap·배타 journal 검사를 제공한다. C02는 foreground devguardd serve, 제한된 인증 UDS 통신, OS UID/PID 관측, 엄격한 client 호환성과 private 자격 FD 전달을 추가한다. C03은 `devguard-macos`의 boot 시계, native PID/start 정체성, 호스트 용량과 serve에서 journal을 활성화하는 2초 압력 sampler를 추가한다. C04는 협조적 QoS/nice 적용과 readback, 이탈·추적 상실이 고정되는 관측 process group scope, 정체성을 확인한 종료를 추가한다. C05는 wire 등록과 fenced `devguard-launch` helper를 추가한다. Grant마다 claim된 helper는 하나이며, READY와 exec 전에 scope binding과 authorization을 거치고, transcript와 exec 실패 보고를 분리하며, payload descriptor를 정리한다. C06은 서비스 reconciler를 추가한다. 관측된 scope 종료, helper가 없다는 owner 보고, 이전 boot에서만 회수하며, reap 전 owner 관측, scope 종료, instance 폐기, 재시작 전후의 Suspect 회계를 제공한다. Native 증거가 있으면 서비스는 등록·launch·대조를 연다. C07은 `devguard` 명령행 owner를 추가한다. 명시적이고 제한된 대기, terminal·signal 전달, reap 전 관측, receipt, doctor 진단을 갖춘 관리 실행을 제공하며, 관리되지 않는 대체 실행은 없다. C08은 Cargo adapter를 추가한다. Compiler job을 예약에 맞춰 조정하거나 거절하며, pipeline과 중첩 Cargo 실행이 jobserver 하나를 공유하게 한다. C09는 패키지로 만든 release를 현재 사용자 LaunchAgent로 설치한다. hash와 컴파일된 호환성을 담은 manifest, 변경 불가능한 release·복구 사본을 두며, launchd가 그 release의 바이너리를 실행한다고 검증한 뒤에만 선택한다. PR과 병합 후 main 전달 증거는 별도로 추적한다. [운영 문서](../../operations.md)를 참조한다. +구현된 C01은 정상 경로·명시 bootstrap·배타 journal 검사를 제공한다. C02는 foreground devguardd serve, 제한된 인증 UDS 통신, OS UID/PID 관측, 엄격한 client 호환성과 private 자격 FD 전달을 추가한다. C03은 `devguard-macos`의 boot 시계, native PID/start 정체성, 호스트 용량과 serve에서 journal을 활성화하는 2초 압력 sampler를 추가한다. C04는 협조적 QoS/nice 적용과 readback, 이탈·추적 상실이 고정되는 관측 process group scope, 정체성을 확인한 종료를 추가한다. C05는 wire 등록과 fenced `devguard-launch` helper를 추가한다. Grant마다 claim된 helper는 하나이며, READY와 exec 전에 scope binding과 authorization을 거치고, transcript와 exec 실패 보고를 분리하며, payload descriptor를 정리한다. C06은 서비스 reconciler를 추가한다. 관측된 scope 종료, helper가 없다는 owner 보고, 이전 boot에서만 회수하며, reap 전 owner 관측, scope 종료, instance 폐기, 재시작 전후의 Suspect 회계를 제공한다. Native 증거가 있으면 서비스는 등록·launch·대조를 연다. C07은 `devguard` 명령행 owner를 추가한다. 명시적이고 제한된 대기, terminal·signal 전달, reap 전 관측, receipt, doctor 진단을 갖춘 관리 실행을 제공하며, 관리되지 않는 대체 실행은 없다. C08은 Cargo adapter를 추가한다. Compiler job을 예약에 맞춰 조정하거나 거절하며, pipeline과 중첩 Cargo 실행이 jobserver 하나를 공유하게 한다. C09는 패키지로 만든 release를 현재 사용자 LaunchAgent로 설치한다. hash와 컴파일된 호환성을 담은 manifest, 변경 불가능한 release·복구 사본을 두며, launchd가 그 release의 바이너리를 실행한다고 검증한 뒤에만 선택한다. C10은 부모 lease와 후보 authority를 추가한다. Lease는 호스트에 한 번 과금되고, 자식은 lease의 남은 예산에 대해서만 admission되며 lease가 끝나면 fence된다. 후보 authority는 lease로 제한되며 launch 없이 admission만 한다. `devguard test-candidate`는 후보 tree의 build·시험·authority를 lease 하나의 자식으로 실행한다. PR과 병합 후 main 전달 증거는 별도로 추적한다. [운영 문서](../../operations.md)를 참조한다. ## PR 순서와 활성화 경계 @@ -17,7 +17,7 @@ | DG1-P5 | DG1-C09, DG1-C10, DG1-C11 | DG1-P4 | 설치·후보·독립 복구를 묶어 자기 적용 활성화 | | DG1-P6 | DG1-C12 | DG1-P5 | 기능 기준 artifact와 SLO 안정 artifact를 구분해 승격 | -공통 현재 명령 python3 scripts/validate.py --offline은 Rust 1.95.0 계약 회귀를 확인하며 fake backend로 native 동작을 입증하지 않는다. C01 authority, C02 인증·transport, C03 native probe, C04 native scope, C05 native launch, C06 native 대조, C07 CLI, C08 Cargo, C09 설치 suite는 현재 제공한다. 아래에서 예정이라고 명시한 나머지 명령은 미제공이며 각 PR에서 fixture·실행 case 수·log·정리를 함께 구현하고 제공 상태를 갱신한다. 이름만 있는 테스트나 0개 실행을 통과로 처리하지 않는다. 공통 toolchain·증거·SLO 규칙은 상위 검증 문서에 있다. +공통 현재 명령 python3 scripts/validate.py --offline은 Rust 1.95.0 계약 회귀를 확인하며 fake backend로 native 동작을 입증하지 않는다. C01 authority, C02 인증·transport, C03 native probe, C04 native scope, C05 native launch, C06 native 대조, C07 CLI, C08 Cargo, C09 설치, C10 부모 lease suite는 현재 제공한다. 아래에서 예정이라고 명시한 나머지 명령은 미제공이며 각 PR에서 fixture·실행 case 수·log·정리를 함께 구현하고 제공 상태를 갱신한다. 이름만 있는 테스트나 0개 실행을 통과로 처리하지 않는다. 공통 toolchain·증거·SLO 규칙은 상위 검증 문서에 있다. P1은 runtime을 닫아 두고 P2는 실제 probe, P3는 launch·안전 정리, P4는 개발 진입점, P5는 설치·부모 예산·repair, P6는 측정·승격을 제공한다. C08까지 foreground daemon과 최소 단일 Cargo job·test thread bootstrap을 사용한다. P4 bundle은 삭제할 build 경로 밖에 보존한다. C10에서 부모 예산을 포함한 artifact를 먼저 기능 시험·동결한 직후 제한된 실제 자기 적용을 시작하며 SLO qualification은 C12에서 확립한다. @@ -146,7 +146,7 @@ P1은 runtime을 닫아 두고 P2는 실제 probe, P3는 launch·안전 정리, - 대상/산출물: 예정 bounded-test mode, 부모 capability 검증, 후보/자식 workload 합산과 fixture runner. - 불변 조건: 후보 CPU·memory·tasks 합계는 상위 budget 이하; 정상 journal 접근 불가; parent 소실/만료 시 신규 grant 금지. - 시험: 정상 후보 build/test; 초과 요청·가짜 parent·후보 정책 오류; 후보 crash와 부모 cancel·늦은 자식 시작 경쟁. -- 검증 명령: 현재 공통 회귀 + 예정 `python3 scripts/qualify.py dg1-self-use`. +- 검증 명령: 현재 공통 회귀 + **제공** `python3 scripts/qualify.py dg1-self-use --offline` (macOS 전용이며 다른 플랫폼은 `not_run`으로 기록한다). 격리 fixture 부모로 실행하며, 동결한 설치 부모 아래의 실제 자기 적용은 완료 증거로 따로 기록한다. - 완료 증거: 상위/하위 attempt 관계와 합계, 두 번째 full-host authority 거절, 실제 자식 scope 관측과 기준 artifact 보존. - rollback: 부모가 후보 scope를 정리·대조; 후보 자신의 admission이나 정상 journal 재초기화에 의존하지 않는다. - 인계: C10 기능 checkpoint 직후 실제 후보를 부모로 실행하고 이후 해당 build/test를 부모로 관리하며 admission/launch/대조 receipt를 보존한다. DG1-C11에 부모 경유 drain·독립 repair와 실패 증거를 전달한다. diff --git a/docs/ko/planning/verification.md b/docs/ko/planning/verification.md index 2b07194..6ce9d50 100644 --- a/docs/ko/planning/verification.md +++ b/docs/ko/planning/verification.md @@ -35,6 +35,7 @@ python3 scripts/qualify.py dg1-reconcile --offline python3 scripts/qualify.py dg1-cli --offline python3 scripts/qualify.py dg1-cargo --offline python3 scripts/qualify.py dg1-bootstrap --offline +python3 scripts/qualify.py dg1-self-use --offline git diff --check ``` @@ -76,7 +77,7 @@ V-DOC-DG는 scripts/check_docs.py와 수동 의미 검토로 영어/한국어 ha ## 향후 suite와 장애 주입 계약 -scripts/qualify.py dg1-authority --offline은 C01 설정·저장소, scripts/qualify.py dg1-auth --offline은 C02 인증·transport, scripts/qualify.py dg1-probes --offline은 C03 native 증거, scripts/qualify.py dg1-scopes --offline은 C04 정책·scope 증거, scripts/qualify.py dg1-launch --offline은 C05 launch helper, scripts/qualify.py dg1-reconcile --offline은 C06 대조, scripts/qualify.py dg1-cli --offline은 C07 명령행 owner, scripts/qualify.py dg1-cargo --offline은 C08 Cargo adapter, scripts/qualify.py dg1-bootstrap --offline은 C09 설치 검증을 제공한다. 그 밖의 DevGuard suite와 CodeSpace scripts/qualify-devguard.py는 해당 작업에서 제공할 예정 인터페이스다. 각 구현 PR이 실제 CLI·case inventory·nonzero case assertion·timeout·log 수집·격리 cleanup을 구현하고 제공 명령을 갱신해야 한다. macOS/Ubuntu CI는 전체 validator와 이식 가능한 두 기능 suite를 실행하고 각각의 report·log를 보존한다. Native suite는 macOS CI에서만 실행하고 그 밖의 환경에서는 `not_run`으로 기록한다. 증거를 제공할 수 없는 플랫폼에서 통과로 처리하지 않는다. 각 native 단계는 만들어야 할 raw receipt를 선언한다. 어떤 경우를 `not_run`으로 기록한 receipt가 있으면 suite는 `passed`가 아니라 `incomplete`가 된다. +scripts/qualify.py dg1-authority --offline은 C01 설정·저장소, scripts/qualify.py dg1-auth --offline은 C02 인증·transport, scripts/qualify.py dg1-probes --offline은 C03 native 증거, scripts/qualify.py dg1-scopes --offline은 C04 정책·scope 증거, scripts/qualify.py dg1-launch --offline은 C05 launch helper, scripts/qualify.py dg1-reconcile --offline은 C06 대조, scripts/qualify.py dg1-cli --offline은 C07 명령행 owner, scripts/qualify.py dg1-cargo --offline은 C08 Cargo adapter, scripts/qualify.py dg1-bootstrap --offline은 C09 설치, scripts/qualify.py dg1-self-use --offline은 C10 부모 lease와 후보 authority 검증을 제공한다. 그 밖의 DevGuard suite와 CodeSpace scripts/qualify-devguard.py는 해당 작업에서 제공할 예정 인터페이스다. 각 구현 PR이 실제 CLI·case inventory·nonzero case assertion·timeout·log 수집·격리 cleanup을 구현하고 제공 명령을 갱신해야 한다. macOS/Ubuntu CI는 전체 validator와 이식 가능한 두 기능 suite를 실행하고 각각의 report·log를 보존한다. Native suite는 macOS CI에서만 실행하고 그 밖의 환경에서는 `not_run`으로 기록한다. 증거를 제공할 수 없는 플랫폼에서 통과로 처리하지 않는다. 각 native 단계는 만들어야 할 raw receipt를 선언한다. 어떤 경우를 `not_run`으로 기록한 receipt가 있으면 suite는 `passed`가 아니라 `incomplete`가 된다. C03 증거는 다음 파일에 기록한다. - **Raw receipt** (보고서의 `raw/` 디렉터리): 단위를 포함한 boot ID·시계 읽기, 호스트 용량, zombie·reap·거부 관측을 포함한 반복 프로세스 정체성, 계산된 비율을 포함한 native 압력 읽기, 마지막 sample 이후 admission이 닫히기까지 측정한 시간, 주입한 실패부터 Critical까지 걸린 서비스 loop 시간과 멈춘 probe에 대한 서비스 loop의 동작. @@ -124,6 +125,12 @@ C09 증거는 다음을 포함한다. - **단위 시험**: `launchctl print` parsing, escape와 crash 전용 재시작을 포함한 plist 생성, release id 규칙, 이 build에 컴파일된 호환성. - **실제 설치**: 사용자 확인이 필요한 qualification 호스트의 설치는 패키지 manifest, status 보고, 실행 중인 바이너리 hash와 함께 C09 완료 증거로 기록한다. +C10 증거는 다음을 포함한다. +- **Raw receipt**: workload와 후보 authority를 자식으로 실행하고 해제로 예산을 돌려준 lease, 그리고 제공 전에 죽은 후보에 대한 `test-candidate` 보고, lease 안·밖·종료 뒤의 lease 자식과 위조 token을 쓴 자식. 실행이 남긴 파일에 caller 자격이 없고, lease 자식의 receipt에 lease token이 없는지 확인한다. +- **실제 프로세스**: 시험 바이너리를 `devguard exec --lease` owner, 실제 `devguard-launch` 아래의 workload, 격리 fixture 부모의 lease를 쓰는 후보 authority로 다시 실행한다. `test-candidate` orchestrator는 lease owner로서 시험 프로세스 안에서 실행한다. +- **단위·서비스 시험**: core lease 계약(과금, 남은 예산, token, 재요청, 종료·owner 소실·기한에 의한 fence, Suspect 자식, 재시작, 폐기, 새 table이 없는 journal), 요구한 client에게만 알리는 capability, token만 쓰는 보유자 session, 후보 session과 그 거절, 엄격한 lease wire fixture, 인자 parsing, 후보 tree로 만드는 plan. +- **실제 자기 적용**: 동결해 설치한 C10 부모 아래에서 작업 tree로 실행한 `test-candidate`는 그 release에 대한 사용자 확인이 필요하며, 보고·receipt·부모 hash와 함께 C10 완료 증거로 기록한다. 호스트 메모리 압력이 admission을 허용해야 한다. + C02 증거는 양쪽 실제 OS socket UID/PID 관측, 분리된 consumer·관리 자격, helper-role 인증 거절, 현재·미래 wire fixture의 엄격한 decoding, 64 KiB frame, 세션 32개 제한, idle 대기를 포함한 frame별 절대 250 ms 기한, 부분·느린·마지막 응답과 private 자격 FD 전달을 포함한다. 전용 subprocess helper는 후속 exec 전 FD 닫기를 관측하고 argv·환경·debug·출력의 secret 누출을 확인한다. Parent test가 helper를 실행하며 별도의 ignored test를 독립 qualification 성공으로 세지 않는다. 0개가 아닌 parent case 수와 프로세스 정리를 함께 기록한다. Framing은 poll·descriptor O_NONBLOCK·호출별 nonblocking I/O로 Darwin timeout 옵션 변경 없이 peer 종료 뒤 버퍼 데이터를 보존한다. 느린 writer와 느린 reader를 모두 시험한다. 인증과 닫힌 등록 응답은 boot/start 정체성, native 등록/Principal, lease, OS 정책, helper 권한·launch를 입증하지 않으며 해당 범위는 P2/P3까지 미검증이다. system_tasks >= 48도 검증된 회계 추정치이지 kernel task 제한이나 측정된 충분성이 아니다. 같은 schema 1에서 이전 값 16을 거절하는 시험을 명시하며 자동 migration을 의미하지 않는다. diff --git a/docs/milestones.md b/docs/milestones.md index d3f159a..feeb5e1 100644 --- a/docs/milestones.md +++ b/docs/milestones.md @@ -7,14 +7,14 @@ The [detailed planning index](planning/README.md) owns the 46 proposed commit un | Milestone | Deliverable and gate | Current implementation | |---|---|---| | DG-0 | Independent repository, approved design, resource and compatibility contracts, durable state transitions, fake-backend fault tests, reproducible 1.95.0 validation | Implemented; use the exact-source qualification report for validation status | -| DG-1 | Real macOS host probes, daemon and CLI, launch gate, Cargo/generic adapters, parent-budget candidate tests, safe service update/repair, three measured SLO runs per target backend | In progress: C01 configuration/storage, C02 authenticated foreground transport, C03 native boot/process/pressure evidence, C04 cooperative policy/scope evidence, C05 fenced launch helper, C06 reconciliation, C07 command-line owner, C08 Cargo adapters and C09 installation as the current user's LaunchAgent; self-use and SLO unqualified | +| DG-1 | Real macOS host probes, daemon and CLI, launch gate, Cargo/generic adapters, parent-budget candidate tests, safe service update/repair, three measured SLO runs per target backend | In progress: C01 configuration/storage, C02 authenticated foreground transport, C03 native boot/process/pressure evidence, C04 cooperative policy/scope evidence, C05 fenced launch helper, C06 reconciliation, C07 command-line owner, C08 Cargo adapters, C09 installation as the current user's LaunchAgent and C10 parent leases with candidate authorities; real self-use and SLO unqualified | | CS-RG | Full-SHA consumer pin, pre-spawn slots, PrepareExec/ExecPrepared, approval preservation, bounded control/data lanes and replay, InProcess/UDS parity and upstream regression qualification | Not started | | P1-RECOVERY | Opt-in Gateway restart/reconnection while an independent Runner retains processes and I/O; reconcile workspace, approvals and DevGuard leases | Not started; depends on CS-RG; InProcess and Runner-loss I/O restoration excluded | | DG-LINUX | Actual Linux controller, ancestor capacity, complete sandbox/proxy scope, control protection and reclaim evidence | Not started; required for product completion | | DG-CACHE | Registered-root lease/reclaim exclusion, protected artifacts/evidence, interrupted trash sweep and measured physical recovery | Not started; not a P1 prerequisite | | DG-ADAPTERS | Tool-specific Python, Node/Bun, make/ninja and container/VM verification, without language branches in core | Not started; not a P1 prerequisite | -DG-0's SQLite and fake-backend tests do not qualify a running resource governor. The `devguardd` bootstrap/path/check commands and foreground `serve` with authenticated status and native C03 host evidence are available. With native evidence the service opens registration over the wire, fenced launch through `devguard-launch` and reconciliation. The `devguard` command-line owner (C07) runs commands through that service, with Cargo adapters (C08). A packaged release can be installed as the current user's LaunchAgent (C09). CodeSpace resources settings and Runner wire changes remain future work; see [operations](operations.md). The approved design's examples must not be represented as currently runnable commands. +DG-0's SQLite and fake-backend tests do not qualify a running resource governor. The `devguardd` bootstrap/path/check commands and foreground `serve` with authenticated status and native C03 host evidence are available. With native evidence the service opens registration over the wire, fenced launch through `devguard-launch` and reconciliation. The `devguard` command-line owner (C07) runs commands through that service, with Cargo adapters (C08). A packaged release can be installed as the current user's LaunchAgent (C09), and a candidate build verified under a parent lease with `devguard test-candidate` (C10). CodeSpace resources settings and Runner wire changes remain future work; see [operations](operations.md). The approved design's examples must not be represented as currently runnable commands. Record design acceptance, implementation and platform qualification separately. Do not change Linux `not_run` to passed because a fake cgroup test passed on macOS, or call normal Cargo bootstrap self-governed development. The reports from `scripts/validate.py` include those distinctions explicitly. diff --git a/docs/operations.md b/docs/operations.md index 7d89321..24d62df 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -2,7 +2,7 @@ [English](operations.md) | [한국어](ko/operations.md) -C01 provides explicit bootstrap/canonical storage, C02 authenticated local transport, C03 native macOS boot, process and host pressure evidence, C04 cooperative policy readback and scope evidence, C05 the fenced launch helper, C06 reconciliation, C07 the `devguard` command-line owner, C08 the Cargo adapters and C09 installation of a packaged release as the current user's LaunchAgent. PR and post-merge main delivery evidence is tracked separately from implementation. `devguardd serve` runs a foreground service that activates the journal with that evidence and then opens registration over the wire, fenced launch and reconciliation. `devguard exec` runs commands through that service, and `devguard doctor` diagnoses it. The normal authority is currently macOS-only; Linux CI checks portable contracts and bounded fixtures, not DG-LINUX controls. +C01 provides explicit bootstrap/canonical storage, C02 authenticated local transport, C03 native macOS boot, process and host pressure evidence, C04 cooperative policy readback and scope evidence, C05 the fenced launch helper, C06 reconciliation, C07 the `devguard` command-line owner, C08 the Cargo adapters, C09 installation of a packaged release as the current user's LaunchAgent and C10 parent leases with candidate authorities. PR and post-merge main delivery evidence is tracked separately from implementation. `devguardd serve` runs a foreground service that activates the journal with that evidence and then opens registration over the wire, fenced launch and reconciliation. `devguard exec` runs commands through that service, and `devguard doctor` diagnoses it. The normal authority is currently macOS-only; Linux CI checks portable contracts and bounded fixtures, not DG-LINUX controls. ## Available commands @@ -51,7 +51,15 @@ devguardd status `package.py` builds a release package from a clean tree. `install` must run from that package. It refuses while an authority serves, while the agent exists, or without the existing authority state. It copies the release to an immutable `releases/` and starts it as the current user's LaunchAgent `io.github.novelkr.devguard`, which runs `releases//bin/devguardd serve`. It selects the release only after verifying that launchd runs exactly that binary. `devguardd status` reports the installed service and exits 1 unless it runs the verified current release. Replacing a release is an upgrade, and repair is future work (C11). See [contracts](contracts.md#installation-and-the-current-user-service). -`devguard exec [--project ID] [--adapter auto|generic|cargo|cargo-pipeline] [--wait DURATION] [--cpu MILLICPU] [--memory SIZE] [--tasks N] [--receipt PATH] -- PROGRAM [ARGS...]` admits the command, starts it through `devguard-launch` and waits for it. The program and its arguments follow `--`, and no shell is used. It exits with the command's status or ends by its signal, and exits 125 when nothing was started. Without `--wait` a denial ends it at once; `--wait` retries capacity and pressure denials until the deadline, and refuses at once a request larger than the host's work capacity. `--receipt` writes a private JSON receipt. `devguard doctor` prints a JSON diagnosis, and `--require admission,registration,macos-cooperative` makes it fail unless each requirement holds. See [contracts](contracts.md#command-line-owner). +A candidate build is verified under a parent lease of the running service: + +```sh +devguard test-candidate --candidate /path/to/candidate/tree --report /path/to/new/report --wait 5m +``` + +`test-candidate` admits one lease (by default 2 CPU, 4 GiB and 96 tasks, with a two-hour deadline) and runs the tree's build, its applicable tests and its own `devguardd candidate` as lease children through `devguard exec --lease`. It checks the candidate's admission from outside, ends the lease and writes `report.json` into the new report directory, exiting 0 only when every step passed. The service must state parent leases, which needs a C10 or later release, and admits no lease while its pressure is Critical. See [contracts](contracts.md#parent-leases-and-candidate-authorities). + +`devguard exec [--project ID] [--adapter auto|generic|cargo|cargo-pipeline] [--wait DURATION] [--cpu MILLICPU] [--memory SIZE] [--tasks N] [--receipt PATH] [--lease CONSUMER/GENERATION/ATTEMPT --lease-token-fd N] -- PROGRAM [ARGS...]` admits the command, starts it through `devguard-launch` and waits for it. The program and its arguments follow `--`, and no shell is used. It exits with the command's status or ends by its signal, and exits 125 when nothing was started. Without `--wait` a denial ends it at once; `--wait` retries capacity and pressure denials until the deadline, and refuses at once a request larger than the host's work capacity. `--receipt` writes a private JSON receipt. `--lease` makes the command a child of that parent lease, admitted against its remainder with the token read from descriptor N. `devguard doctor` prints a JSON diagnosis, and `--require admission,registration,macos-cooperative` makes it fail unless each requirement holds. See [contracts](contracts.md#command-line-owner). `--adapter cargo` fits `cargo`'s compiler jobs to the reservation, and `--adapter cargo-pipeline` shares one jobserver across the Cargo runs of a program such as a validation script. `auto`, the default, selects the Cargo adapter for `cargo` commands that compile. A reservation that cannot fit one Cargo job (1 CPU and 2 GiB) is refused. See [contracts](contracts.md#cargo-adapter). @@ -110,7 +118,7 @@ The SDK inspection example can be built with `CARGO_BUILD_JOBS=1 cargo build --l ## Compatibility, checks and rollback -Core `AuthorityStorage` holds the original exclusive lock and validates the existing schema-1 journal without changing attempt state. Activation with a real `Backend`/`Clock` revalidates accounting in the same transaction as boot-aware recovery. Existing `Authority::open` preserves its recovery behavior and the DG-0 tests. C01–C06 do not change the journal schema or contract serialization. C05 and C06 add wire requests and responses that clients use only after the service advertises fenced launch. The new local protocol strictly rejects unknown fields, versions and unsupported required capabilities; added fields need explicit compatibility tests. +Core `AuthorityStorage` holds the original exclusive lock and validates the existing schema-1 journal without changing attempt state. Activation with a real `Backend`/`Clock` revalidates accounting in the same transaction as boot-aware recovery. Existing `Authority::open` preserves its recovery behavior and the DG-0 tests. C01–C06 do not change the journal schema or contract serialization. C05 and C06 add wire requests and responses that clients use only after the service advertises fenced launch. C10 adds lease requests and responses that clients use only after the service states parent leases, and the tables `leases` and `lease_children`, created when absent, without changing the schema. The new local protocol strictly rejects unknown fields, versions and unsupported required capabilities; added fields need explicit compatibility tests. Available checks: @@ -124,6 +132,7 @@ python3 scripts/qualify.py dg1-reconcile --offline python3 scripts/qualify.py dg1-cli --offline python3 scripts/qualify.py dg1-cargo --offline python3 scripts/qualify.py dg1-bootstrap --offline +python3 scripts/qualify.py dg1-self-use --offline python3 scripts/validate.py --offline ``` @@ -224,6 +233,14 @@ These suites leave Linux enforcement, self-use and foreground SLO `not_run`. `dg A session without a launchd gui domain records the launchd case as `not_run`, and the suite reports `incomplete`. +`dg1-self-use` also runs only on macOS and builds `devguard-launch` first. It runs real lease children and a real candidate authority process against isolated authorities. It checks: +- a lease charged once against the host, children admitted only against its remainder, and their sum never above the lease +- the parent-lease capability stated only to clients that require it, a token-only holder session, and a refused wrong token or unknown lease +- fencing when the lease ends, when its owner goes or belongs to an earlier boot and when its deadline passes, a Suspect child that keeps the lease charged, and release once every child is settled, also across a restart +- `devguard exec --lease` running a workload as a lease child, which inherits only its standard descriptors; a child larger than the lease, also with a wait; a forged token; and a child that starts after the lease ended +- a candidate authority that admits within its capacity, launches nothing, holds no lease, closes with its lease and never opens the parent's state, and one refused for a forged token, an unknown or ended lease, a capacity larger than the lease or below its own reservation, or an existing area +- `test-candidate` running a workload and a candidate as children of one lease that is then released, with the candidate's area removed, and a candidate that dies before serving, which fails the run while its scope and the lease are still released + The full validator retains the 44 original tests and validates all eight workspace crates and their explicit dependency graph. The daemon depends on itself only to enable its fixtures in its own tests. The launch crate depends on the daemon only for its tests' isolated authorities. The CLI depends on the daemon's paths and configuration and on the Cargo adapter, and on the daemon's fixtures only in tests. The Cargo adapter depends only on the contract. Core/contract remain independent of daemon configuration and native adapters, and client does not depend on core. -To roll back this boundary, stop its task-owned foreground process and select a source/artifact compatible with the preserved configuration and schema-1 journal. Keep persistent state and credentials. A C02 artifact can reopen the same state: C03 activation adds no record type and only runs the existing recovery. From C06, workloads can start through the normal service. Before rolling back to an artifact without reconciliation, stop starting new work and let charged attempts reach a terminal phase. A C05 artifact reads the same journal but keeps launch closed and does not reconcile, so any remaining attempt stays charged and Suspect. If the service stops while scopes run, start it again: their attempts stay Suspect and charged until their owners report unclaimed grants or a reboot proves termination. C07 changes no service state: to roll back the CLI, stop starting commands through it. Commands it already started stay charged until their scopes end, and it never falls back to unmanaged execution. C08 changes no service state either: to stop fitting Cargo's jobs, select `--adapter generic` explicitly; existing targets and caches are kept. Before a release is verified, the C09 installer boots its job out and removes the plist, and nothing is selected. To stop the installed service, run `launchctl bootout gui//io.github.novelkr.devguard` and remove the plist. Releases, recovery copies, the selection and the journal are kept, and a preserved artifact's foreground `devguardd serve` reads the same state. Never delete the journal or its tombstones to make a rollback or restart succeed. +To roll back this boundary, stop its task-owned foreground process and select a source/artifact compatible with the preserved configuration and schema-1 journal. Keep persistent state and credentials. A C02 artifact can reopen the same state: C03 activation adds no record type and only runs the existing recovery. From C06, workloads can start through the normal service. Before rolling back to an artifact without reconciliation, stop starting new work and let charged attempts reach a terminal phase. A C05 artifact reads the same journal but keeps launch closed and does not reconcile, so any remaining attempt stays charged and Suspect. If the service stops while scopes run, start it again: their attempts stay Suspect and charged until their owners report unclaimed grants or a reboot proves termination. C07 changes no service state: to roll back the CLI, stop starting commands through it. Commands it already started stay charged until their scopes end, and it never falls back to unmanaged execution. C08 changes no service state either: to stop fitting Cargo's jobs, select `--adapter generic` explicitly; existing targets and caches are kept. Before a release is verified, the C09 installer boots its job out and removes the plist, and nothing is selected. To stop the installed service, run `launchctl bootout gui//io.github.novelkr.devguard` and remove the plist. Releases, recovery copies, the selection and the journal are kept, and a preserved artifact's foreground `devguardd serve` reads the same state. C10 adds no service state beyond its lease tables. Before rolling back to a C09 artifact, end every lease and let it be released: a C09 service ignores the lease tables, so it would not hold a lease's unused remainder. A candidate's area under `candidates/` is disposable and never read by the normal service. Never delete the journal or its tombstones to make a rollback or restart succeed. diff --git a/docs/planning/milestones/DG-1.md b/docs/planning/milestones/DG-1.md index 28294cf..2e6425a 100644 --- a/docs/planning/milestones/DG-1.md +++ b/docs/planning/milestones/DG-1.md @@ -4,7 +4,7 @@ Owner: DevGuard. Current implementation: `in-progress`; qualification: `not-run` All work IDs, commit titles and logical PR labels below are **proposed values**, not future SHAs or GitHub numbers. Module paths describe planned responsibilities until implemented. Each added workspace crate updates the explicit dependency allowlist in the same PR without removing full-graph validation. The ledger owns actual status. -Implemented behavior: C01 provides canonical paths, explicit bootstrap and locked journal validation. C02 adds foreground `devguardd serve`, authenticated bounded UDS communication, OS-observed UID/PID, strict client compatibility and private credential-FD transfer. C03 adds the `devguard-macos` boot clock, native PID/start identity, host capacity and a two-second pressure sampler that activates the journal in `serve`. C04 adds cooperative QoS/nice application with readback, observed process-group scopes with sticky escape and tracking loss, and identity-checked termination. C05 adds registration over the wire and the fenced `devguard-launch` helper: one claimed helper per grant, scope binding and authorization before READY and exec, a separate transcript and exec-failure report, and payload descriptor hygiene. C06 adds the service reconciler: releases only on observed scope termination, an owner's report that no helper exists or a previous boot; owner observation before reap; scope termination; instance retirement; and Suspect accounting across a restart. With native evidence the service opens registration, launch and reconciliation. C07 adds the `devguard` command-line owner: managed execution with explicit bounded waits, terminal and signal forwarding, observation before reap, receipts and doctor diagnostics, and never an unmanaged fallback. C08 adds the Cargo adapters: compiler jobs fitted to the reservation, with clamping and refusals, and one jobserver shared across a pipeline's and nested Cargo runs. C09 installs a packaged release as the current user's LaunchAgent: a manifest of hashes and compiled compatibility, immutable release and recovery copies, and selection only after launchd is verified to run that release's binary. PR and post-merge main delivery evidence is tracked separately. See [operations](../../operations.md). +Implemented behavior: C01 provides canonical paths, explicit bootstrap and locked journal validation. C02 adds foreground `devguardd serve`, authenticated bounded UDS communication, OS-observed UID/PID, strict client compatibility and private credential-FD transfer. C03 adds the `devguard-macos` boot clock, native PID/start identity, host capacity and a two-second pressure sampler that activates the journal in `serve`. C04 adds cooperative QoS/nice application with readback, observed process-group scopes with sticky escape and tracking loss, and identity-checked termination. C05 adds registration over the wire and the fenced `devguard-launch` helper: one claimed helper per grant, scope binding and authorization before READY and exec, a separate transcript and exec-failure report, and payload descriptor hygiene. C06 adds the service reconciler: releases only on observed scope termination, an owner's report that no helper exists or a previous boot; owner observation before reap; scope termination; instance retirement; and Suspect accounting across a restart. With native evidence the service opens registration, launch and reconciliation. C07 adds the `devguard` command-line owner: managed execution with explicit bounded waits, terminal and signal forwarding, observation before reap, receipts and doctor diagnostics, and never an unmanaged fallback. C08 adds the Cargo adapters: compiler jobs fitted to the reservation, with clamping and refusals, and one jobserver shared across a pipeline's and nested Cargo runs. C09 installs a packaged release as the current user's LaunchAgent: a manifest of hashes and compiled compatibility, immutable release and recovery copies, and selection only after launchd is verified to run that release's binary. C10 adds parent leases and candidate authorities: a lease charged once against the host, children admitted only against its remainder and fenced when it ends, a candidate authority bounded by its lease that admits without launching, and `devguard test-candidate`, which runs a candidate tree's build, tests and authority as children of one lease. PR and post-merge main delivery evidence is tracked separately. See [operations](../../operations.md). ## PR sequence and activation @@ -17,7 +17,7 @@ Implemented behavior: C01 provides canonical paths, explicit bootstrap and locke | DG1-P5 | DG1-C09, DG1-C10, DG1-C11 | DG1-P4 | | DG1-P6 | DG1-C12 | DG1-P5 | -Available regression: `python3 scripts/validate.py --offline` (Rust 1.95.0 contract regression; fake backends do not prove native behavior). C01 authority, C02 authentication/transport, C03 native probe, C04 native scope, C05 native launch, C06 native reconciliation, C07 CLI, C08 Cargo and C09 installation suites are now available; commands explicitly labelled planned below remain unavailable until implemented. Each PR must supply real fixtures, nonzero case counts, logs and cleanup, then update command availability. See [verification](../verification.md). +Available regression: `python3 scripts/validate.py --offline` (Rust 1.95.0 contract regression; fake backends do not prove native behavior). C01 authority, C02 authentication/transport, C03 native probe, C04 native scope, C05 native launch, C06 native reconciliation, C07 CLI, C08 Cargo, C09 installation and C10 parent-lease suites are now available; commands explicitly labelled planned below remain unavailable until implemented. Each PR must supply real fixtures, nonzero case counts, logs and cleanup, then update command availability. See [verification](../verification.md). DG1-P1 keeps runtime readiness closed; P2 provides actual probes; P3 ships launch with safe cleanup; P4 provides development entrypoints; P5 installation/parent-budget/repair; P6 measures and promotes. Through C08 use foreground daemons and minimum one-job/one-thread bootstrap. Preserve the P4 bundle outside disposable output. At C10, first test and freeze a parent containing parent-budget support, then immediately begin bounded real self-use; C12 alone establishes SLO qualification. @@ -149,7 +149,7 @@ DG1-P1 keeps runtime readiness closed; P2 provides actual probes; P3 ships launc - Completion evidence: Frozen parent hashes/capabilities, real admission/launch/reconciliation receipts, parent-child sums and scopes, second full-host authority rejection and protected recovery artifacts. - Rollback: Parent independently cleans/reconciles candidate scopes; repair does not depend on candidate admission or normal-journal reinitialization. - Handoff: Govern subsequent applicable builds/tests from this checkpoint. DG1-C11 receives parent-mediated drain/repair and preserved failure evidence; C12 later determines SLO promotion. -- Verification command: available regression above plus **planned, not yet provided** `python3 scripts/qualify.py dg1-self-use`. +- Verification command: available regression above plus **available** `python3 scripts/qualify.py dg1-self-use --offline` (macOS only; other platforms record `not_run`). It runs isolated fixture parents; real self-use under the frozen installed parent is recorded separately as completion evidence. ### DG1-C11 — recover upgrades without candidate admission diff --git a/docs/planning/verification.md b/docs/planning/verification.md index 50a1268..ef5bc2a 100644 --- a/docs/planning/verification.md +++ b/docs/planning/verification.md @@ -35,6 +35,7 @@ python3 scripts/qualify.py dg1-reconcile --offline python3 scripts/qualify.py dg1-cli --offline python3 scripts/qualify.py dg1-cargo --offline python3 scripts/qualify.py dg1-bootstrap --offline +python3 scripts/qualify.py dg1-self-use --offline git diff --check ``` @@ -72,7 +73,7 @@ For the preparation PRs, preserve runtime/Cargo/journal state. Documentation che ## Planned suites and fault injection -`scripts/qualify.py dg1-authority --offline` is available for C01 configuration/storage, `scripts/qualify.py dg1-auth --offline` for C02 authentication/transport, `scripts/qualify.py dg1-probes --offline` for C03 native evidence, `scripts/qualify.py dg1-scopes --offline` for C04 policy and scope evidence, `scripts/qualify.py dg1-launch --offline` for the C05 launch helper, `scripts/qualify.py dg1-reconcile --offline` for C06 reconciliation, `scripts/qualify.py dg1-cli --offline` for the C07 command-line owner, `scripts/qualify.py dg1-cargo --offline` for the C08 Cargo adapters, and `scripts/qualify.py dg1-bootstrap --offline` for C09 installation. Other DevGuard suites and CodeSpace `scripts/qualify-devguard.py ` remain planned interfaces until supplied by their work units. Each implementation PR supplies the actual interface, nonzero case inventory, timeouts, logs, isolation and cleanup, then updates its task command documentation. macOS/Ubuntu CI retains the full validator and both portable functional suites, preserving their separate reports and logs. Native suites run on macOS CI only and record `not_run` elsewhere; they never pass on a platform that cannot supply the evidence. Each native stage declares the raw receipts it must produce. A receipt that records a case as `not_run` makes the suite `incomplete`, not `passed`. +`scripts/qualify.py dg1-authority --offline` is available for C01 configuration/storage, `scripts/qualify.py dg1-auth --offline` for C02 authentication/transport, `scripts/qualify.py dg1-probes --offline` for C03 native evidence, `scripts/qualify.py dg1-scopes --offline` for C04 policy and scope evidence, `scripts/qualify.py dg1-launch --offline` for the C05 launch helper, `scripts/qualify.py dg1-reconcile --offline` for C06 reconciliation, `scripts/qualify.py dg1-cli --offline` for the C07 command-line owner, `scripts/qualify.py dg1-cargo --offline` for the C08 Cargo adapters, `scripts/qualify.py dg1-bootstrap --offline` for C09 installation, and `scripts/qualify.py dg1-self-use --offline` for C10 parent leases and candidate authorities. Other DevGuard suites and CodeSpace `scripts/qualify-devguard.py ` remain planned interfaces until supplied by their work units. Each implementation PR supplies the actual interface, nonzero case inventory, timeouts, logs, isolation and cleanup, then updates its task command documentation. macOS/Ubuntu CI retains the full validator and both portable functional suites, preserving their separate reports and logs. Native suites run on macOS CI only and record `not_run` elsewhere; they never pass on a platform that cannot supply the evidence. Each native stage declares the raw receipts it must produce. A receipt that records a case as `not_run` makes the suite `incomplete`, not `passed`. C03 evidence is recorded in these files: - **Raw receipts** (in the report's `raw/` directory): boot ID and clock readings with units, host capacity, repeated process identities, including zombie, reaped and refused observations, native pressure readings with their derived rates, the measured time from the last sample to closed admission, and the service loop's time from an injected failure to Critical and its behavior with a stuck probe. @@ -120,6 +121,12 @@ C09 evidence covers: - **Unit tests**: `launchctl print` parsing, plist rendering with escaping and the crash-only restart, release id rules, and this build's compiled compatibility. - **Real installation**: the installation on the qualification host, which needs the user's confirmation, is recorded with the package manifest, the status report and the running binary's hash as C09 completion evidence. +C10 evidence covers: +- **Raw receipts**: `test-candidate` reports for a lease whose workload and candidate authority ran as its children and whose release returned its budget, and for a candidate that died before serving; lease children within, beyond and after the end of a lease, and with a forged token. The files the runs leave are checked to hold no caller credential, and lease children's receipts to hold no lease token. +- **Real processes**: the test binary re-executed as `devguard exec --lease` owners, as workloads under the real `devguard-launch`, and as a candidate authority serving an isolated fixture parent's lease; the `test-candidate` orchestrator runs in the test process as the lease's owner. +- **Unit and service tests**: the core lease contract (charging, remainders, tokens, replays, fencing by end, owner loss and deadline, Suspect children, restart, retirement and journals without the new tables), the echo-only capability, token-only holder sessions, candidate sessions and their refusals, strict lease wire fixtures, argument parsing and the plan made from a candidate tree. +- **Real self-use**: `test-candidate` on the working tree under the frozen, installed C10 parent, which needs the user's confirmation of that release, is recorded with its report, receipts and the parent's hashes as C10 completion evidence. It needs the host's memory pressure to allow admission. + C02 evidence covers actual OS socket UID/PID observations at both ends, distinct consumer/admin credentials, rejected helper-role authentication, strict current/future wire fixtures, 64 KiB frames, a 32-session limit, absolute 250 ms per-frame deadlines including idle waits, partial/slow/final responses and private credential-FD transport. Dedicated subprocess helpers verify FD closure before a subsequent exec and inspect argv/environment/debug/output for secret leakage. They are executed by parent tests and are not independent ignored qualification successes. Record process cleanup as well as the nonzero parent-case inventory. The framing implementation uses `poll` with descriptor `O_NONBLOCK` and per-call nonblocking I/O, preserving buffered data after peer closure without Darwin timeout-option mutation. Test slow readers as well as slow writers. Authentication and a closed registration response do not prove boot/start identity, native registration/principals, leases, OS policy application, helper authorization or launch. Those remain unqualified until P2/P3. Likewise, `system_tasks >= 48` is a validated accounting estimate, not a kernel task cap or a measured sufficiency claim. Explicitly test rejection of the previous value 16 under unchanged schema 1; no automatic migration is implied. diff --git a/docs/translations.json b/docs/translations.json index cd3e8fa..a5cbd32 100644 --- a/docs/translations.json +++ b/docs/translations.json @@ -56,8 +56,8 @@ "id": "planning-milestones-dg-1", "source": "docs/planning/milestones/DG-1.md", "translation": "docs/ko/planning/milestones/DG-1.md", - "reviewed_source_sha256": "65ad40fe3febc141551f4e893365e60f3be54bd0a1243a7e4a5ad999255ee0a3", - "reviewed_translation_sha256": "af00fecfb3c3f701782584d00bf1655c2085c0df9e7e89bbcb17759f6481b9df" + "reviewed_source_sha256": "cbf9cd47a33cfb04e7ba095634c96af4f387d90caa230eecfebe40c181f5b631", + "reviewed_translation_sha256": "2681ebad9c718af5aa19399b1e117b42ac194520bead1be51c47bb26a4ea70f9" }, { "id": "planning-milestones-dg-adapters", @@ -98,22 +98,22 @@ "id": "planning-verification", "source": "docs/planning/verification.md", "translation": "docs/ko/planning/verification.md", - "reviewed_source_sha256": "d95b7391badceadc7cffc824a18e5016086f31c817114b5fd2208695f4ab30d6", - "reviewed_translation_sha256": "cba8d5fd716105abcb31c6ef93cdae82993e91719f7255d5c2d8bd065eedbca4" + "reviewed_source_sha256": "96644d0396c0e433ef201276d366d1eae4fa2fc6ffff4af5b3e4d2bee85798fa", + "reviewed_translation_sha256": "bfe5abc62695900938cf46f9177cf11ea97d1e64d8155678852e55bf0e104683" }, { "id": "operations", "source": "docs/operations.md", "translation": "docs/ko/operations.md", - "reviewed_source_sha256": "b83172718e3e16969aa06e967afba6b6e02cd5040fac2e3b6b7a6667ebb142de", - "reviewed_translation_sha256": "dd0b3611f0f82913689be628dc86a356812bc577da6391af37fb16bf209ddf15" + "reviewed_source_sha256": "25fea6879b444a6217cbe214c5a06e108a39518d4a42ee118f64bce8767efe06", + "reviewed_translation_sha256": "17dba9a2584b931b2d705c7d8f38f867962dbfc08cdb4e43f621ca8437fdc771" }, { "id": "contracts", "source": "docs/contracts.md", "translation": "docs/ko/contracts.md", - "reviewed_source_sha256": "004fb8e882980b9b0f2cfff9e6a0ed63f0b87dae6cd67d3b434f06ec683a59e4", - "reviewed_translation_sha256": "bd5ea1ba73c4b0e2e5aa5d27ccc34cea9d1ff376b535005596390be9262c0fa1" + "reviewed_source_sha256": "3ac22af28c6d94f16cad727e03ed3bf3594e426dd02b9d4404abf779f99059ee", + "reviewed_translation_sha256": "65adf165eb9d169a98f6a78d8f7590fd53a8d38162863c5859b132ebf54a480e" } ] } diff --git a/scripts/qualify.py b/scripts/qualify.py index e74e90d..cb92bc0 100644 --- a/scripts/qualify.py +++ b/scripts/qualify.py @@ -30,7 +30,8 @@ ("wire-compatibility", ["-p", "devguard-client", "--test", "wire_compatibility"]), # Native sampling tests belong to dg1-probes, not the transport suite. ("service", ["-p", "devguard-daemon", "--lib", "server::tests", "--", - "--skip", "server::tests::native", "--skip", "server::tests::launch"]), + "--skip", "server::tests::native", "--skip", "server::tests::launch", + "--skip", "server::tests::lease"]), ], "dg1-probes": [ ("pressure-controller", ["-p", "devguard-core", "--test", "pressure_contract"]), @@ -101,11 +102,23 @@ "cancel-after-authorization", "dead-owner", "retired-instance", "unresponsive-helper", "daemon-crash-restart", "journal-failure"]), ], + "dg1-self-use": [ + ("lease-contract", ["-p", "devguard-core", "--test", "lease_contract"]), + ("wire-lease", ["-p", "devguard-client", "--test", "wire_compatibility", "lease_"]), + ("service-leases", ["-p", "devguard-daemon", "--lib", "server::tests::lease"]), + ("candidate-service", ["-p", "devguard-daemon", "--lib", "candidate::tests"]), + ("candidate-entrypoint", ["-p", "devguard-daemon", "--test", "entrypoint", "a_candidate_"]), + ("lease-arguments", ["-p", "devguard-cli", "--lib", "args::tests::lease_"]), + ("candidate-plan", ["-p", "devguard-cli", "--lib", "candidate::tests"]), + ("native-self-use", ["-p", "devguard-cli", "--test", "candidate"], + ["test-candidate", "lease-children", "test-candidate-crash"]), + ], } # Binaries a suite's tests start but whose package the suite does not test. PREBUILD = { "dg1-cli": [["-p", "devguard-launch", "--bin", "devguard-launch"]], "dg1-cargo": [["-p", "devguard-launch", "--bin", "devguard-launch"]], + "dg1-self-use": [["-p", "devguard-launch", "--bin", "devguard-launch"]], } STAGE_TIMEOUT_SECONDS = 1800 SCOPES = { @@ -118,11 +131,13 @@ "dg1-cargo": "the Cargo adapters through the devguard owner against isolated authorities with real Cargo builds: jobs fitted to the reservation, clamping and refusals, one jobserver shared by a pipeline and by nested Cargo, inherited and stale jobservers, concurrent consumers and cancellation; Cargo jobs are compilation parallelism, not a cap on test threads or measured memory", "dg1-bootstrap": "installation of a release as the current user's LaunchAgent against isolated authorities: package validation, immutable release and recovery copies, verification that launchd runs the release's own binary before it is selected, refusals, concurrent installers, and under launchd itself a crash restart, a SIGTERM stop and a fail-closed start that is not restarted; functional artifacts only, not SLO qualification", "dg1-reconcile": "native macOS reconciliation through isolated authorities: prepared cancellation and expiry, owner reports that no helper exists, releases only on scope termination, sticky escape and tracking loss, scope termination signals, dead owners, and a daemon crash with restart", + "dg1-self-use": "parent leases and candidate authorities against isolated authorities: a lease charged once against the host, children admitted only against its remainder with its token, fencing when it ends, its owner goes or its deadline passes, release once every child is settled, and a candidate authority run as a lease child through the real launch helper that admits within its leased capacity, launches nothing and closes with its lease; functional fixtures only, not real self-use under the installed parent", } # Suites that start real workloads through the launch helper (in fixtures). -LAUNCHING = {"dg1-launch", "dg1-reconcile", "dg1-cli", "dg1-cargo"} +LAUNCHING = {"dg1-launch", "dg1-reconcile", "dg1-cli", "dg1-cargo", "dg1-self-use"} # Native suites observe the actual host; elsewhere they are not run, never passed. -NATIVE = {"dg1-probes", "dg1-scopes", "dg1-launch", "dg1-reconcile", "dg1-cli", "dg1-cargo", "dg1-bootstrap"} +NATIVE = {"dg1-probes", "dg1-scopes", "dg1-launch", "dg1-reconcile", "dg1-cli", "dg1-cargo", "dg1-bootstrap", + "dg1-self-use"} def host_facts(): @@ -172,7 +187,8 @@ def main(): "scope":SCOPES[args.suite] + ("; no kernel resource-control enforcement or SLO qualification" if args.suite in LAUNCHING else "; no workload launch, resource-control enforcement or SLO qualification"), "runtime_qualification":{"macos_launch":"functional_fixture" if args.suite in LAUNCHING else "not_run", - "linux_cgroups":"not_run","foreground_slo":"not_run","candidate_self_use":"not_run"},"stages":[]} + "linux_cgroups":"not_run","foreground_slo":"not_run", + "candidate_self_use":"functional_fixture" if args.suite=="dg1-self-use" else "not_run"},"stages":[]} environment=os.environ.copy() environment.update(CARGO_BUILD_JOBS="1",RUST_TEST_THREADS="1") try: From 4081e1a3b13d6c65c5528ea8a8df563686421776 Mon Sep 17 00:00:00 2001 From: Seongjae Date: Fri, 25 Sep 2026 00:48:24 +0900 Subject: [PATCH 3/7] fix(self-use): resolve a relative report directory before children run (DG1-C10) The real self-use run under the installed C10 parent showed that `devguard test-candidate --report` with a relative path would hand the same relative path to each `devguard exec --lease` child, which runs in the candidate tree, so the children's receipts would land in the tree and the report would miss them. The plan now resolves the report directory to an absolute path before anything runs. The recorded run used an absolute path with release 0.1.0-11acc80-a22cbbae. Co-Authored-By: Claude Opus 5.5 --- crates/cli/src/candidate.rs | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/crates/cli/src/candidate.rs b/crates/cli/src/candidate.rs index 3a9251a..d7c64d1 100644 --- a/crates/cli/src/candidate.rs +++ b/crates/cli/src/candidate.rs @@ -779,6 +779,9 @@ pub fn plan(args: &CandidateArgs, environment: &BTreeMap) -> "the candidate tree must be a Cargo workspace with a lock file", )); } + // Children run in the tree, so every path handed to them is absolute. + let report = std::path::absolute(&args.report) + .map_err(|_| invalid("the report directory cannot be resolved"))?; let lease = Budget { cpu_milli: args.cpu_milli.unwrap_or(DEFAULT_LEASE.cpu_milli), memory_bytes: args.memory_bytes.unwrap_or(DEFAULT_LEASE.memory_bytes), @@ -870,7 +873,7 @@ pub fn plan(args: &CandidateArgs, environment: &BTreeMap) -> ], env: Vec::new(), }), - report: args.report.clone(), + report, tree: Some(tree), }) } @@ -925,6 +928,18 @@ mod tests { std::fs::write(tree.path().join("Cargo.lock"), "").unwrap(); let planned = plan(&args(tree.path()), &BTreeMap::new()).unwrap(); assert_eq!(planned.lease, DEFAULT_LEASE); + // A relative report directory is resolved before any child runs in + // the tree, where it would name another place. + let relative = CandidateArgs { + report: "reports/run-1".into(), + ..args(tree.path()) + }; + let resolved = plan(&relative, &BTreeMap::new()).unwrap().report; + assert!(resolved.is_absolute()); + assert_eq!( + resolved, + std::env::current_dir().unwrap().join("reports/run-1") + ); assert_eq!(planned.capacity, DEFAULT_CAPACITY); assert!(planned.capacity.fits(planned.lease)); let names: Vec<&str> = planned.children.iter().map(|w| w.name.as_str()).collect(); From e4da3583d272eade12b415b4be944d456b78b55d Mon Sep 17 00:00:00 2001 From: Seongjae Date: Fri, 25 Sep 2026 01:14:20 +0900 Subject: [PATCH 4/7] feat(operations): recover upgrades without candidate admission (DG1-C11) Replace and repair the installed service without losing or duplicating charged work, and without depending on a candidate's admission. - Drains: an administrator closes admission (CloseAdmission), reopens it and asks what is charged (Quiescence), stated only to clients that require upgrade_drain. The closure is written to state/admission.json before it takes effect, so a restart or the next release starts closed. Closing cancels every Prepared attempt (known not started); admissions, launch commits, leases and lease children are refused with ResourceUnavailable while queries, stops, owner reports and reconciliation continue. Core gains cancel_prepared, and storage can report an idle journal's charges without activation and back it up with VACUUM INTO. - `devguardd stage --package DIR` copies a package into an immutable release without touching the service. - `devguard upgrade --release ID [--drain-timeout 60s] [--stopped]` runs from the staged release's own devguard. It refuses another wire version, protocol, journal or configuration schema, or a release without what consumers require (an incompatible downgrade), closes admission and drains, stops the service, takes a quiescent backup of the journal, selection and manifest under backups/, starts the new release closed and verifies its binary, handshake and idle state, then records it and reopens admission. A drain timeout reopens admission on the current release with every charge kept; a new release that cannot be verified gives way to the previous one on the same journal. A release before C11 is replaced with --stopped only when its journal then charges nothing. - `devguard repair --use last-known-good` refuses while any authority serves, keeps a journal that cannot be opened closed, runs the recovery copy of a damaged release (status accepts it) and reopens an admission left closed by an interrupted upgrade. Installer helpers are shared with the upgrade and the installer tests' fixtures move to tests/support. Tests: the core quiescence contract, service drains across a restart, strict drain wire fixtures, compatibility rules, arguments and entry points, and seven real-process upgrade and repair cases under a fake manager. qualify.py gains dg1-upgrade; CI runs it on macOS. Docs: contracts, operations, DG-1 ledger and verification (English authority, reviewed Korean), README and milestones. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 7 +- README.md | 5 +- crates/cli/src/args.rs | 88 ++- crates/cli/src/lib.rs | 67 +- crates/cli/tests/entrypoint.rs | 6 + crates/client/src/lib.rs | 18 + crates/client/src/protocol.rs | 38 + crates/client/tests/wire_compatibility.rs | 34 + crates/contract/src/lib.rs | 3 + crates/core/src/authority.rs | 41 + crates/core/src/journal.rs | 25 + crates/core/tests/quiescence_contract.rs | 158 ++++ crates/daemon/src/candidate.rs | 2 +- crates/daemon/src/fixture.rs | 13 +- crates/daemon/src/install.rs | 133 ++-- crates/daemon/src/lib.rs | 1 + crates/daemon/src/main.rs | 14 +- crates/daemon/src/paths.rs | 44 ++ crates/daemon/src/server.rs | 397 +++++++++- crates/daemon/src/upgrade.rs | 862 ++++++++++++++++++++++ crates/daemon/tests/entrypoint.rs | 2 + crates/daemon/tests/install.rs | 263 +------ crates/daemon/tests/support/mod.rs | 309 ++++++++ crates/daemon/tests/upgrade.rs | 537 ++++++++++++++ docs/contracts.md | 31 +- docs/ko/contracts.md | 31 +- docs/ko/operations.md | 28 +- docs/ko/planning/milestones/DG-1.md | 6 +- docs/ko/planning/verification.md | 9 +- docs/milestones.md | 4 +- docs/operations.md | 28 +- docs/planning/milestones/DG-1.md | 6 +- docs/planning/verification.md | 9 +- docs/translations.json | 16 +- scripts/qualify.py | 15 +- 35 files changed, 2877 insertions(+), 373 deletions(-) create mode 100644 crates/core/tests/quiescence_contract.rs create mode 100644 crates/daemon/src/upgrade.rs create mode 100644 crates/daemon/tests/support/mod.rs create mode 100644 crates/daemon/tests/upgrade.rs diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 028feb6..74b2689 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -61,6 +61,9 @@ jobs: - name: Native parent lease and candidate functional checks if: runner.os == 'macOS' run: python3 scripts/qualify.py dg1-self-use --output target/qualification/dg1-self-use + - name: Native upgrade and repair functional checks + if: runner.os == 'macOS' + run: python3 scripts/qualify.py dg1-upgrade --output target/qualification/dg1-upgrade - name: Preserve qualification evidence if: always() uses: actions/upload-artifact@v6 @@ -75,9 +78,9 @@ jobs: import json, os from pathlib import Path with open(os.environ['GITHUB_STEP_SUMMARY'], 'a') as out: - for name in ['ci', 'dg1-authority', 'dg1-auth', 'dg1-probes', 'dg1-scopes', 'dg1-launch', 'dg1-reconcile', 'dg1-cli', 'dg1-cargo', 'dg1-bootstrap', 'dg1-self-use']: + for name in ['ci', 'dg1-authority', 'dg1-auth', 'dg1-probes', 'dg1-scopes', 'dg1-launch', 'dg1-reconcile', 'dg1-cli', 'dg1-cargo', 'dg1-bootstrap', 'dg1-self-use', 'dg1-upgrade']: path = Path('target/qualification') / name / 'report.json' status = json.loads(path.read_text())['status'] if path.exists() else 'not_run' out.write(name + ': ' + status + '\n\n') - out.write('Contract regression, service/UDS, native host evidence, native scope, launch helper, reconciliation, command-line owner, Cargo adapter, installation and parent lease functional checks only; native suites run on macOS. Real self-use under an installed parent and foreground SLO qualification are not run.\n') + out.write('Contract regression, service/UDS, native host evidence, native scope, launch helper, reconciliation, command-line owner, Cargo adapter, installation, parent lease, upgrade and repair functional checks only; native suites run on macOS. Real self-use under an installed parent and foreground SLO qualification are not run.\n') PY diff --git a/README.md b/README.md index 544f2be..cbb0524 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ DevGuard centralizes resource admission for development workloads while preserving the resources needed to inspect and stop them. -The repository implements **DG-0 contracts and a durable authority core**, with DG-1 now in progress. C01 supplies canonical paths/bootstrap/storage checks; C02 supplies authenticated, bounded local communication with a small client and private credential-FD transfer; C03 supplies the native macOS boot clock, process identity and host pressure evidence that activate the service's journal; C04 supplies cooperative QoS/nice application with readback and observed process-group scope evidence; C05 supplies registration over the wire and the fenced `devguard-launch` helper; C06 supplies reconciliation, and with native evidence the service opens all three; C07 supplies the `devguard` command-line owner with doctor diagnostics, receipts and explicit bounded waits; C08 supplies the Cargo adapters, which fit compiler jobs to the reservation and share one jobserver; C09 installs a packaged release as the current user's LaunchAgent, selected only after launchd is verified to run it; C10 supplies parent leases, whose children are admitted only against the lease, and candidate authorities bounded by them. PR and post-merge main delivery evidence is tracked separately from implementation. Linux cgroups are not yet available. Use the operating guide for actual command availability; the design also contains future interfaces. +The repository implements **DG-0 contracts and a durable authority core**, with DG-1 now in progress. C01 supplies canonical paths/bootstrap/storage checks; C02 supplies authenticated, bounded local communication with a small client and private credential-FD transfer; C03 supplies the native macOS boot clock, process identity and host pressure evidence that activate the service's journal; C04 supplies cooperative QoS/nice application with readback and observed process-group scope evidence; C05 supplies registration over the wire and the fenced `devguard-launch` helper; C06 supplies reconciliation, and with native evidence the service opens all three; C07 supplies the `devguard` command-line owner with doctor diagnostics, receipts and explicit bounded waits; C08 supplies the Cargo adapters, which fit compiler jobs to the reservation and share one jobserver; C09 installs a packaged release as the current user's LaunchAgent, selected only after launchd is verified to run it; C10 supplies parent leases, whose children are admitted only against the lease, and candidate authorities bounded by them; C11 supplies upgrade and repair that drain charged work, back up the journal and verify the new release before admission reopens. PR and post-merge main delivery evidence is tracked separately from implementation. Linux cgroups are not yet available. Use the operating guide for actual command availability; the design also contains future interfaces. - [Authoritative design reference](docs/design.md) · [Korean translation](docs/ko/design.md) - [Historical approved design (Korean, immutable)](docs/design.ko.md) @@ -28,11 +28,12 @@ python3 scripts/qualify.py dg1-cli --offline # macOS only python3 scripts/qualify.py dg1-cargo --offline # macOS only python3 scripts/qualify.py dg1-bootstrap --offline # macOS only python3 scripts/qualify.py dg1-self-use --offline # macOS only +python3 scripts/qualify.py dg1-upgrade --offline # macOS only ``` Omit `--offline` when the locked crates have not been downloaded. Builds and tests use one Cargo job and one test thread by default. Results, source fingerprints and logs are written under `target/qualification/`. A newer compiler can be used with `--allow-toolchain-mismatch` for a supplemental check, which never counts as qualification for 1.95.0. -DG-0 tests use an explicitly fake OS backend. Their passing reports establish accounting, persistence and state-transition contracts. The additional C01–C10 suites check actual local storage, peer observations, credential transport, native macOS host evidence, cooperative scope evidence, the launch helper, reconciliation, the command-line owner, the Cargo adapters, installation and parent leases within bounded fixtures. They do not qualify Linux enforcement, browser responsiveness or real self-use under an installed parent; these remain `not_run`. +DG-0 tests use an explicitly fake OS backend. Their passing reports establish accounting, persistence and state-transition contracts. The additional C01–C11 suites check actual local storage, peer observations, credential transport, native macOS host evidence, cooperative scope evidence, the launch helper, reconciliation, the command-line owner, the Cargo adapters, installation, parent leases, upgrade and repair within bounded fixtures. They do not qualify Linux enforcement, browser responsiveness or real self-use under an installed parent; these remain `not_run`. ## Repository boundaries diff --git a/crates/cli/src/args.rs b/crates/cli/src/args.rs index e8b22f3..5b5097b 100644 --- a/crates/cli/src/args.rs +++ b/crates/cli/src/args.rs @@ -18,10 +18,21 @@ pub enum Command { Exec(ExecArgs), Doctor(DoctorArgs), TestCandidate(CandidateArgs), + Upgrade(UpgradeArgs), + /// `repair --use last-known-good`, the only repair. + Repair, Help, Version, } +/// `devguard upgrade`: replace the current release with a staged one. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct UpgradeArgs { + pub release: String, + pub drain_timeout: Option, + pub stopped: bool, +} + #[derive(Debug, Clone, PartialEq, Eq)] pub struct ExecArgs { pub project: Option, @@ -194,10 +205,12 @@ pub fn parse(args: impl IntoIterator) -> Result { Some("exec") => parse_exec(raw).map(Command::Exec), Some("doctor") => parse_doctor(raw).map(Command::Doctor), Some("test-candidate") => parse_candidate(raw).map(Command::TestCandidate), + Some("upgrade") => parse_upgrade(raw).map(Command::Upgrade), + Some("repair") => parse_repair(raw).map(|()| Command::Repair), Some("--help" | "-h" | "help") if raw.next().is_none() => Ok(Command::Help), Some("--version" | "-V") if raw.next().is_none() => Ok(Command::Version), _ => Err(usage( - "expected exec, doctor, test-candidate, --help or --version", + "expected exec, doctor, test-candidate, upgrade, repair, --help or --version", )), } } @@ -266,6 +279,40 @@ fn parse_exec(mut raw: impl Iterator) -> Result { }) } +fn parse_upgrade(mut raw: impl Iterator) -> Result { + let (mut release, mut drain_timeout, mut stopped) = (None, None, false); + while let Some(option) = raw.next() { + match option.to_str() { + Some("--release") => once(&mut release, project_id(value(&mut raw)?)?)?, + Some("--drain-timeout") => { + once(&mut drain_timeout, parse_duration(&value(&mut raw)?)?)? + } + Some("--stopped") if !stopped => stopped = true, + _ => { + return Err(usage( + "upgrade accepts --release ID, --drain-timeout DURATION and --stopped", + )) + } + } + } + Ok(UpgradeArgs { + release: release.ok_or_else(|| usage("upgrade needs --release ID"))?, + drain_timeout, + stopped, + }) +} + +fn parse_repair(mut raw: impl Iterator) -> Result<()> { + match ( + raw.next().as_deref().and_then(|option| option.to_str()), + raw.next().as_deref().and_then(|value| value.to_str()), + raw.next(), + ) { + (Some("--use"), Some("last-known-good"), None) => Ok(()), + _ => Err(usage("repair takes exactly --use last-known-good")), + } +} + fn parse_candidate(mut raw: impl Iterator) -> Result { let (mut candidate, mut report, mut cpu, mut memory, mut tasks, mut ttl, mut wait) = (None, None, None, None, None, None, None); @@ -428,6 +475,45 @@ mod tests { assert_eq!(candidate.wait, None); } + #[test] + fn upgrade_and_repair_name_a_release_or_the_last_known_good() { + let Command::Upgrade(upgrade) = parse(args(&[ + "upgrade", + "--release", + "0.1.0-abc-1234", + "--drain-timeout", + "90s", + ])) + .unwrap() else { + panic!("expected upgrade") + }; + assert_eq!(upgrade.release, "0.1.0-abc-1234"); + assert_eq!(upgrade.drain_timeout, Some(Duration::from_secs(90))); + assert!(!upgrade.stopped); + let Command::Upgrade(stopped) = + parse(args(&["upgrade", "--stopped", "--release", "r1"])).unwrap() + else { + panic!("expected upgrade") + }; + assert!(stopped.stopped); + assert_eq!( + parse(args(&["repair", "--use", "last-known-good"])).unwrap(), + Command::Repair + ); + for invalid in [ + &["upgrade"][..], + &["upgrade", "--release", "../x"], + &["upgrade", "--release", "a", "--release", "b"], + &["upgrade", "--release", "a", "--stopped", "--stopped"], + &["upgrade", "--release", "a", "--drain-timeout", "60"], + &["repair"], + &["repair", "--use", "current"], + &["repair", "--use", "last-known-good", "--force"], + ] { + assert!(parse(args(invalid)).is_err(), "{invalid:?}"); + } + } + #[test] fn exec_refuses_ambiguous_or_shell_like_invocations() { for invalid in [ diff --git a/crates/cli/src/lib.rs b/crates/cli/src/lib.rs index 252e787..b1b89a5 100644 --- a/crates/cli/src/lib.rs +++ b/crates/cli/src/lib.rs @@ -19,7 +19,9 @@ pub mod signals; pub mod terminal; use devguard_contract::Result; +use devguard_daemon::install::{InstallOptions, Launchctl}; use devguard_daemon::paths::AuthorityPaths; +use devguard_daemon::upgrade::UpgradeOptions; pub use receipt::Exit; use std::ffi::OsString; use std::path::PathBuf; @@ -31,6 +33,8 @@ devguard exec [--project ID] [--adapter auto|generic|cargo|cargo-pipeline] [--wa devguard doctor [--require admission,registration,macos-cooperative] [--project ID] devguard test-candidate --candidate DIR --report DIR [--cpu MILLICPU] [--memory SIZE] [--tasks N] [--ttl DURATION] [--wait DURATION] +devguard upgrade --release ID [--drain-timeout DURATION] [--stopped] +devguard repair --use last-known-good devguard --help | --version exec admits the command at the central authority, starts it through devguard-launch under @@ -43,7 +47,15 @@ authority comes from the operating account; there is no path or authority overri test-candidate admits one parent lease and runs a candidate tree's build, its applicable tests and its own candidate authority as children of it, checks the candidate's admission, -ends the lease and writes report.json in the new report directory."; +ends the lease and writes report.json in the new report directory. + +upgrade replaces the installed release with a staged one (`devguardd stage --package DIR`) +and must run from that release's own devguard: it closes admission, waits up to the drain +timeout (60s by default) for charged work to end, backs up the journal, starts the new +release closed, verifies it and reopens admission; otherwise the current release keeps +serving. --stopped replaces a release that cannot close admission by stopping it first, +only if nothing is then charged. repair starts the last known good release, or its +recovery copy, while no authority serves; it never reinitializes the journal."; /// Run the command line `args` (without the program name). `locate` supplies /// the authority paths and the launch helper, and is consulted only by @@ -82,6 +94,39 @@ pub fn run( Exit::Code(1) } }, + args::Command::Upgrade(upgrade) => match locate() { + Ok((paths, _)) => operate(|| { + devguard_daemon::upgrade::upgrade( + &paths, + &upgrade.release, + &Launchctl::new(paths.uid()), + &InstallOptions::canonical(&paths), + &UpgradeOptions { + drain_timeout: upgrade + .drain_timeout + .unwrap_or(devguard_daemon::upgrade::DEFAULT_DRAIN_TIMEOUT), + stopped: upgrade.stopped, + }, + ) + }), + Err(error) => { + eprintln!("devguard: {}", error.message); + Exit::Code(1) + } + }, + args::Command::Repair => match locate() { + Ok((paths, _)) => operate(|| { + devguard_daemon::upgrade::repair( + &paths, + &Launchctl::new(paths.uid()), + &InstallOptions::canonical(&paths), + ) + }), + Err(error) => { + eprintln!("devguard: {}", error.message); + Exit::Code(1) + } + }, args::Command::TestCandidate(candidate) => match locate() { Ok((paths, _)) => candidate::run_args(&candidate, &paths), Err(error) => { @@ -92,6 +137,26 @@ pub fn run( } } +/// Run an operation on the installed service and print its JSON report. +fn operate(operation: impl FnOnce() -> Result) -> Exit { + match operation() { + Ok(report) => match serde_json::to_string_pretty(&report) { + Ok(text) => { + println!("{text}"); + Exit::Code(0) + } + Err(_) => { + eprintln!("devguard: the report could not be encoded"); + Exit::Code(1) + } + }, + Err(error) => { + eprintln!("devguard: {:?}: {}", error.code, error.message); + Exit::Code(1) + } + } +} + /// End the process as the command ended: with its exit code, or by the same /// signal with its default action. The workload's core dump, if any, is its /// own; the CLI's core limit is set to zero first so it never writes one. diff --git a/crates/cli/tests/entrypoint.rs b/crates/cli/tests/entrypoint.rs index 8dcb8b0..add72ed 100644 --- a/crates/cli/tests/entrypoint.rs +++ b/crates/cli/tests/entrypoint.rs @@ -19,6 +19,8 @@ fn help_and_version_describe_the_managed_contract() { "devguard exec", "devguard doctor", "devguard test-candidate", + "devguard upgrade", + "devguard repair", "never run unmanaged", "no path or authority override", ] { @@ -53,6 +55,10 @@ fn malformed_invocations_start_nothing_and_exit_125() { "true", ], &["test-candidate"], + &["upgrade"], + &["upgrade", "--release", "../escape"], + &["repair"], + &["repair", "--use", "current"], &[ "test-candidate", "--candidate", diff --git a/crates/client/src/lib.rs b/crates/client/src/lib.rs index 83794d3..44d1162 100644 --- a/crates/client/src/lib.rs +++ b/crates/client/src/lib.rs @@ -176,6 +176,24 @@ impl Client { _ => Err(unavailable()), } } + /// Administrator only: close admission and cancel Prepared attempts. + pub fn close_admission(&mut self, reason: String) -> Result { + self.quiescence_call(Request::CloseAdmission { reason }) + } + /// Administrator only: reopen admission. + pub fn open_admission(&mut self) -> Result { + self.quiescence_call(Request::OpenAdmission) + } + /// Administrator only: whether admission is open and what is charged. + pub fn quiescence(&mut self) -> Result { + self.quiescence_call(Request::Quiescence) + } + fn quiescence_call(&mut self, request: Request) -> Result { + match self.call(request)? { + Response::Quiescence(quiescence) => Ok(quiescence), + _ => Err(unavailable()), + } + } fn attempt(&mut self, request: Request) -> Result { match self.call(request)? { Response::Attempt(attempt) => Ok(attempt), diff --git a/crates/client/src/protocol.rs b/crates/client/src/protocol.rs index f3fd69e..fd43b51 100644 --- a/crates/client/src/protocol.rs +++ b/crates/client/src/protocol.rs @@ -133,6 +133,43 @@ pub enum Request { key: AttemptKey, token: Secret, }, + /// Administrator only: close admission, cancelling every Prepared + /// attempt, while queries, stops and reconciliation continue. The closure + /// survives a restart. Sent only after the service echoed `upgrade_drain`. + CloseAdmission { + reason: String, + }, + /// Administrator only: reopen admission. + OpenAdmission, + /// Administrator only: whether admission is open and what is still charged. + Quiescence, +} + +/// The longest reason an administrator may give for closing admission. +pub const MAX_CLOSURE_REASON: usize = 256; + +/// Admission closed by an administrator, for example for an upgrade. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct AdmissionClosure { + pub reason: String, + pub since_unix_ms: u64, +} + +/// Whether admission is open, and every attempt and lease still charged. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct Quiescence { + pub closure: Option, + pub attempts: Vec, + pub leases: Vec, +} + +impl Quiescence { + /// Nothing is charged, so the service can be stopped without disturbing work. + pub fn quiet(&self) -> bool { + self.attempts.is_empty() && self.leases.is_empty() + } } /// A new lease with its token, returned once. @@ -266,5 +303,6 @@ pub enum Response { Terminated(Termination), LeaseGranted(LeaseGranted), Lease(LeaseView), + Quiescence(Quiescence), Error(WireError), } diff --git a/crates/client/tests/wire_compatibility.rs b/crates/client/tests/wire_compatibility.rs index 048bbda..4111629 100644 --- a/crates/client/tests/wire_compatibility.rs +++ b/crates/client/tests/wire_compatibility.rs @@ -365,3 +365,37 @@ fn lease_requests_decode_strictly_and_never_print_the_token() { json!("parent_lease") ); } + +#[test] +fn drain_requests_decode_strictly_and_report_what_is_charged() { + for body in [ + json!({"method": "close_admission", "params": {"reason": "upgrade"}}), + json!({"method": "open_admission"}), + json!({"method": "quiescence"}), + ] { + let frame = json!({"version": 1, "request_id": 13, "body": body}); + assert!( + serde_json::from_value::>(frame.clone()).is_ok(), + "{frame}" + ); + } + let mut extended = json!({"version": 1, "request_id": 14, "body": {"method": "close_admission", "params": {"reason": "upgrade"}}}); + extended["body"]["params"]["drain_timeout_ms"] = json!(1); + assert!(serde_json::from_value::>(extended).is_err()); + let report = json!({"version": 1, "request_id": 15, "body": {"result": "quiescence", "value": { + "closure": {"reason": "upgrade", "since_unix_ms": 1}, + "attempts": [serde_json::to_value(attempt()).unwrap()], + "leases": [serde_json::to_value(lease()).unwrap()]}}}); + let decoded = serde_json::from_value::>(report.clone()).unwrap(); + let Response::Quiescence(quiescence) = decoded.body else { + panic!("expected a quiescence report") + }; + assert!(!quiescence.quiet()); + let mut future = report; + future["body"]["value"]["closure"]["future_field"] = json!(true); + assert!(serde_json::from_value::>(future).is_err()); + assert_eq!( + serde_json::to_value(devguard_contract::Capability::UpgradeDrain).unwrap(), + json!("upgrade_drain") + ); +} diff --git a/crates/contract/src/lib.rs b/crates/contract/src/lib.rs index 1cbcb03..aaef100 100644 --- a/crates/contract/src/lib.rs +++ b/crates/contract/src/lib.rs @@ -530,6 +530,9 @@ pub enum Capability { /// not the host. A service states it only to a client that requires it, /// so a client that cannot decode it never receives it. ParentLease, + /// An administrator can close admission for an upgrade, reopen it and + /// ask what is still charged. Stated only to a client that requires it. + UpgradeDrain, } /// A parent lease's phase. An Ending lease admits no new child and is released diff --git a/crates/core/src/authority.rs b/crates/core/src/authority.rs index a6296b8..5e1ff19 100644 --- a/crates/core/src/authority.rs +++ b/crates/core/src/authority.rs @@ -152,6 +152,28 @@ impl AuthorityStorage { authority_lock, }) } + + /// What the idle journal still charges: attempts not yet settled and + /// leases not yet released. It is read without activating the journal + /// and changes nothing. + pub fn charged(&mut self) -> Result<(Vec, Vec)> { + self.journal.transaction(|tx| { + let attempts = journal::active(tx)?; + let leases = if journal::lease_tables_exist(tx)? { + journal::validate_leases(tx)?; + journal::active_leases(tx)? + } else { + Vec::new() + }; + Ok((attempts, leases)) + }) + } + + /// Copy the journal, complete and consistent, into the new empty private + /// file at `path`, while this storage holds the authority lock. + pub fn backup(&mut self, path: &Path) -> Result<()> { + self.journal.backup(path) + } } impl Authority { @@ -578,6 +600,25 @@ impl Authority { }) } + /// Trusted path for closing admission before an upgrade: cancel every + /// Prepared attempt, which is known not to have started. Launched work is + /// left to finish and be reconciled. + pub fn cancel_prepared(&mut self) -> Result> { + let now = self.clock.now(); + self.journal.transaction(|tx| { + journal::expire_prepared(tx, &now)?; + let mut cancelled = Vec::new(); + for mut record in journal::active(tx)? { + if record.phase == AttemptPhase::Prepared { + record.phase = AttemptPhase::Cancelled; + journal::save(tx, &record)?; + cancelled.push(record); + } + } + Ok(cancelled) + }) + } + /// Trusted reconciliation path; it is not an unauthenticated client RPC. /// There is deliberately no `release(lease_id)` operation without evidence. pub fn reconcile(&mut self, key: &AttemptKey) -> Result { diff --git a/crates/core/src/journal.rs b/crates/core/src/journal.rs index 8ac8f96..06f575e 100644 --- a/crates/core/src/journal.rs +++ b/crates/core/src/journal.rs @@ -144,6 +144,18 @@ impl Journal { tx.commit().map_err(db_error)?; Ok(value) } + + /// Write a complete, consistent copy of the journal to `path`, which must + /// be a new empty file. The copy is itself a valid journal. + pub(crate) fn backup(&mut self, path: &Path) -> Result<()> { + let path = path + .to_str() + .ok_or_else(|| Error::new(ErrorCode::InvalidRequest, "a backup path must be UTF-8"))?; + self.connection + .execute("VACUUM INTO ?1", params![path]) + .map_err(db_error)?; + Ok(()) + } } pub(crate) fn load(tx: &Transaction<'_>, key: &AttemptKey) -> Result> { @@ -321,6 +333,19 @@ pub(crate) fn ensure_lease_tables(tx: &Transaction<'_>) -> Result<()> { tx.execute_batch(LEASE_TABLES).map_err(db_error) } +/// Whether a journal already has the lease tables; one written before C10 +/// has none until an authority activates it. +pub(crate) fn lease_tables_exist(tx: &Transaction<'_>) -> Result { + let count: u64 = tx + .query_row( + "SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name IN ('leases','lease_children')", + [], + |row| row.get(0), + ) + .map_err(db_error)?; + Ok(count == 2) +} + fn lease_invalid() -> Error { Error::new( ErrorCode::JournalInvalid, diff --git a/crates/core/tests/quiescence_contract.rs b/crates/core/tests/quiescence_contract.rs new file mode 100644 index 0000000..c11b2b8 --- /dev/null +++ b/crates/core/tests/quiescence_contract.rs @@ -0,0 +1,158 @@ +//! DG1-C11 quiescence: what an upgrade needs from the authority core. Closing +//! admission cancels only Prepared attempts; an idle journal reports what it +//! still charges without being activated; and a backup of it is itself a +//! complete journal. + +// The shared harness also serves the authority contract, whose steps this file does not all use. +#[allow(dead_code)] +mod support; +use devguard_contract::*; +use devguard_core::*; +use rusqlite::Connection; +use std::os::unix::fs::OpenOptionsExt; +use support::*; + +fn lease_key(id: &str) -> AttemptKey { + AttemptKey { + consumer_id: "codespace-runtime".into(), + consumer_generation: "generation-1".into(), + attempt_id: id.into(), + } +} + +/// A request smaller than the harness's, so several fit the host. +fn small(id: &str) -> AdmissionRequest { + let mut request = request(id); + request.intent.requested.cpu_milli = 1_000; + request +} + +fn empty_private_file(path: &std::path::Path) { + std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .open(path) + .unwrap(); +} + +#[test] +fn closing_admission_cancels_only_prepared_attempts_and_returns_their_budget() { + let h = Harness::new(); + let (mut a, p) = h.ready(); + let prepared = a.admit(&p, small("prepared")).unwrap(); + assert_eq!(prepared.phase, AttemptPhase::Prepared); + let committed = a.admit(&p, small("committed")).unwrap(); + a.begin_launch(&p, &committed.key).unwrap(); + // A Prepared child of a lease returns its budget to the lease. + let lease = lease_key("lease-1"); + let token = a + .admit_lease( + &p, + &lease, + Budget { + cpu_milli: 3_000, + memory_bytes: 8 * GIB, + tasks: 256, + }, + None, + ) + .unwrap() + .token + .unwrap(); + let child = a.admit_child(&p, &lease, &token, request("child")).unwrap(); + assert_eq!(child.phase, AttemptPhase::Prepared); + let before = a.committed_budget().unwrap(); + let cancelled = a.cancel_prepared().unwrap(); + let keys: Vec<&str> = cancelled + .iter() + .map(|r| r.key.attempt_id.as_str()) + .collect(); + assert_eq!(keys.len(), 2); + assert!(keys.contains(&"prepared") && keys.contains(&"child")); + for record in &cancelled { + assert_eq!(record.phase, AttemptPhase::Cancelled); + assert!(record.known_not_started()); + } + // The committed launch is untouched and still charged; the lease keeps + // its whole budget, now with nothing drawn on it. + let remaining: Vec = a.attempts().unwrap(); + assert_eq!(remaining.len(), 1); + assert_eq!(remaining[0].key, committed.key); + assert_eq!(remaining[0].phase, AttemptPhase::LaunchCommitted); + assert_eq!( + a.committed_budget().unwrap(), + before.remaining_after(small("prepared").intent.requested) + ); + assert_eq!(a.lease(&lease).unwrap().remaining.cpu_milli, 3_000); + // Nothing is left to cancel. + assert!(a.cancel_prepared().unwrap().is_empty()); +} + +#[test] +fn an_idle_journal_reports_its_charges_without_activation_and_backs_up_whole() { + let h = Harness::new(); + let (mut a, p) = h.ready(); + let committed = a.admit(&p, small("committed")).unwrap(); + a.begin_launch(&p, &committed.key).unwrap(); + let lease = lease_key("lease-1"); + a.admit_lease( + &p, + &lease, + Budget { + cpu_milli: 2_000, + memory_bytes: GIB, + tasks: 16, + }, + None, + ) + .unwrap(); + drop(a); + let mut storage = AuthorityStorage::open(&h.path).unwrap(); + let (attempts, leases) = storage.charged().unwrap(); + assert_eq!(attempts.len(), 1); + assert_eq!(attempts[0].key, committed.key); + assert_eq!(leases.len(), 1); + assert_eq!(leases[0].key, lease); + // A second storage cannot open while this one holds the lock. + assert!(AuthorityStorage::open(&h.path).is_err()); + let directory = tempfile::tempdir().unwrap(); + let copy = directory.path().join("journal.sqlite"); + empty_private_file(©); + storage.backup(©).unwrap(); + // A backup never overwrites a file that already holds data. + assert!(storage.backup(©).is_err()); + drop(storage); + let mut restored = AuthorityStorage::open(©).unwrap(); + let (copied_attempts, copied_leases) = restored.charged().unwrap(); + assert_eq!(copied_attempts, attempts); + assert_eq!(copied_leases, leases); + drop(restored); + // Reading and backing up changed nothing. + let mut again = AuthorityStorage::open(&h.path).unwrap(); + assert_eq!(again.charged().unwrap(), (attempts, leases)); +} + +#[test] +fn a_journal_written_before_leases_reports_none() { + let h = Harness::new(); + let journal = Connection::open(&h.path).unwrap(); + journal + .execute_batch("DROP TABLE leases; DROP TABLE lease_children;") + .unwrap(); + drop(journal); + let mut storage = AuthorityStorage::open(&h.path).unwrap(); + let (attempts, leases) = storage.charged().unwrap(); + assert!(attempts.is_empty() && leases.is_empty()); + drop(storage); + // Reporting did not add the tables; activation still does. + let journal = Connection::open(&h.path).unwrap(); + let tables: u64 = journal + .query_row( + "SELECT COUNT(*) FROM sqlite_master WHERE name IN ('leases','lease_children')", + [], + |row| row.get(0), + ) + .unwrap(); + assert_eq!(tables, 0); +} diff --git a/crates/daemon/src/candidate.rs b/crates/daemon/src/candidate.rs index f7bb244..262329b 100644 --- a/crates/daemon/src/candidate.rs +++ b/crates/daemon/src/candidate.rs @@ -243,11 +243,11 @@ impl Candidate { &paths, Options { probe, - reconcile_paused: None, candidate: Some(CandidateMode { capacity: spec.capacity, reason: spec.reason(), }), + ..Options::default() }, )?; receipt(json!({"event": "candidate_opened", "candidate": spec, diff --git a/crates/daemon/src/fixture.rs b/crates/daemon/src/fixture.rs index 01ef6de..e12a3a2 100644 --- a/crates/daemon/src/fixture.rs +++ b/crates/daemon/src/fixture.rs @@ -96,7 +96,7 @@ impl TestAuthority { Options { probe: Some(Box::new(HealthyProbe)), reconcile_paused: Some(reconcile_paused.clone()), - candidate: None, + ..Options::default() }, )?; let authority = server.native_authority().ok_or_else(|| { @@ -253,10 +253,21 @@ impl TestAuthority { /// SIGINT, as `devguardd serve` serves the canonical one. Tests run it as the /// program of a service, under launchd or a fake manager. pub fn serve_until_terminated(base: &Path) -> Result<()> { + serve_fixture(base, false) +} + +/// As [`serve_until_terminated`], stating no upgrade drain, as a release +/// before C11 does. +pub fn serve_without_upgrade_drain_until_terminated(base: &Path) -> Result<()> { + serve_fixture(base, true) +} + +fn serve_fixture(base: &Path, without_upgrade_drain: bool) -> Result<()> { let server = Server::open_with( &AuthorityPaths::fixture(base), Options { probe: Some(Box::new(HealthyProbe)), + without_upgrade_drain, ..Options::default() }, )?; diff --git a/crates/daemon/src/install.rs b/crates/daemon/src/install.rs index fc306de..86d4667 100644 --- a/crates/daemon/src/install.rs +++ b/crates/daemon/src/install.rs @@ -10,7 +10,7 @@ //! service that cannot open or reconcile its state fails closed, and launchd //! restarts it only after a crash. -use crate::paths::{secure_directory, AuthorityPaths}; +use crate::paths::{replace_private, secure_directory, AuthorityPaths}; use devguard_client::protocol::{Hello, WIRE_VERSION}; use devguard_client::Client; use devguard_contract::{ @@ -140,7 +140,7 @@ fn read_regular(path: &Path, limit: u64) -> Result> { Ok(bytes) } -fn digest_file(path: &Path) -> Result { +pub(crate) fn digest_file(path: &Path) -> Result { Ok(digest_bytes(&read_regular(path, ARTIFACT_BYTES)?)) } @@ -166,6 +166,23 @@ fn directory_entries(path: &Path) -> Result> { /// binaries, each a regular executable file whose size and hash match, and a /// manifest this build can install. pub fn validate_package(dir: &Path) -> Result { + let package = read_release(dir)?; + let expected = compiled(); + if package.manifest.compatibility != expected + || package.manifest.version != expected.package_version + { + return Err(Error::new( + ErrorCode::ResourcePolicyUnsupported, + "the package's compatibility differs from this installer's build", + )); + } + Ok(package) +} + +/// Check an intact release of any build: exactly the manifest and the three +/// binaries, each a regular executable file whose size and hash match. Its +/// compatibility may differ from this build's, as an earlier release's does. +pub fn read_release(dir: &Path) -> Result { let entries = directory_entries(dir)?; if entries != BTreeSet::from([MANIFEST_FILE.to_string(), "bin".to_string()]) { return Err(invalid("a package holds exactly MANIFEST.json and bin/")); @@ -181,13 +198,6 @@ pub fn validate_package(dir: &Path) -> Result { "the release id may use only letters, digits, '.', '_' and '-'", )); } - let expected = compiled(); - if manifest.compatibility != expected || manifest.version != expected.package_version { - return Err(Error::new( - ErrorCode::ResourcePolicyUnsupported, - "the package's compatibility differs from this installer's build", - )); - } if !matches!( (manifest.scope.as_str(), manifest.slo_qualified), ("functional", false) | ("slo-qualified", true) @@ -453,7 +463,7 @@ pub struct SelectionEvent { pub unix_ms: u128, } -fn unix_ms() -> u128 { +pub(crate) fn unix_ms() -> u128 { std::time::SystemTime::now() .duration_since(std::time::UNIX_EPOCH) .map(|since| since.as_millis()) @@ -481,36 +491,7 @@ pub fn read_selection(paths: &AuthorityPaths) -> Result> { Ok(Some(selection)) } -/// Replace a private file atomically: write a new sibling, sync it, rename it. -fn replace_private(path: &Path, bytes: &[u8]) -> Result<()> { - let parent = path - .parent() - .ok_or_else(|| invalid("a private file needs a directory"))?; - let temporary = parent.join(format!( - ".{}.{}", - path.file_name().and_then(|n| n.to_str()).unwrap_or("file"), - uuid::Uuid::new_v4().simple() - )); - let written = (|| -> std::io::Result<()> { - let mut file = OpenOptions::new() - .write(true) - .create_new(true) - .mode(0o600) - .custom_flags(libc::O_NOFOLLOW | libc::O_CLOEXEC) - .open(&temporary)?; - file.write_all(bytes)?; - file.sync_all()?; - fs::rename(&temporary, path)?; - File::open(parent)?.sync_all() - })(); - if written.is_err() { - let _ = fs::remove_file(&temporary); - return Err(unavailable(format!("cannot write {}", path.display()))); - } - Ok(()) -} - -fn write_selection(paths: &AuthorityPaths, selection: &Selection) -> Result<()> { +pub(crate) fn write_selection(paths: &AuthorityPaths, selection: &Selection) -> Result<()> { let bytes = serde_json::to_vec_pretty(selection).map_err(|_| invalid("selection encoding failed"))?; replace_private(&paths.selection(), &bytes) @@ -520,7 +501,7 @@ fn write_selection(paths: &AuthorityPaths, selection: &Selection) -> Result<()> /// immutable release: the copy is made in a private sibling, synced, made /// read-only and renamed into place, so a partial copy never has the final /// name. An existing destination is reused only when its manifest is identical. -fn freeze_copy(from: &Package, destination: &Path) -> Result { +pub(crate) fn freeze_copy(from: &Package, destination: &Path) -> Result { if fs::symlink_metadata(destination).is_ok() { let existing = validate_package(destination)?; if existing.manifest_sha256 != from.manifest_sha256 { @@ -596,7 +577,7 @@ fn remove_incoming(incoming: &Path) { } /// Remove incoming copies left by installers that are no longer running. -fn sweep_incoming(parent: &Path) { +pub(crate) fn sweep_incoming(parent: &Path) { let Ok(entries) = fs::read_dir(parent) else { return; }; @@ -615,7 +596,7 @@ fn sweep_incoming(parent: &Path) { } /// A handshake that demands nothing, to learn who serves the socket. -fn handshake(paths: &AuthorityPaths) -> Result { +pub(crate) fn handshake(paths: &AuthorityPaths) -> Result { let client = Client::connect( &paths.socket(), paths.uid(), @@ -640,7 +621,7 @@ pub struct RunningEvidence { /// Prove that the service launchd runs is `release`'s `devguardd`, serving /// the canonical endpoint. -fn observe_running( +pub(crate) fn observe_running( paths: &AuthorityPaths, manager: &dyn ServiceManager, label: &str, @@ -676,7 +657,7 @@ fn observe_running( }) } -fn wait_running( +pub(crate) fn wait_running( paths: &AuthorityPaths, manager: &dyn ServiceManager, options: &InstallOptions, @@ -706,7 +687,7 @@ pub struct InstallReport { pub selection: Selection, } -fn ensure_plain_directory(path: &Path, mode: u32) -> Result<()> { +pub(crate) fn ensure_plain_directory(path: &Path, mode: u32) -> Result<()> { match fs::symlink_metadata(path) { Ok(meta) if meta.is_dir() && !meta.file_type().is_symlink() => Ok(()), Ok(_) => Err(invalid(format!("{} is not a directory", path.display()))), @@ -900,28 +881,48 @@ pub fn status( report.running_error = Some("no release has been installed".into()); return Ok(report); }; - let release_dir = paths.releases().join(&selection.current); - let release = match validate_package(&release_dir) { - Ok(release) => release, - Err(error) => { - report.running_error = Some(format!( - "the current release is not intact: {}", - error.message - )); - return Ok(report); + // The service runs the current release or, after a repair, its recovery + // copy; either must be intact, and the plist must name the one that runs. + let installed = fs::read_to_string(&options.plist).ok(); + let mut chosen = None; + let mut damage = Vec::new(); + for dir in [ + paths.releases().join(&selection.current), + paths.recovery().join(&selection.current), + ] { + let release = match read_release(&dir) { + Ok(release) => release, + Err(error) => { + damage.push(format!("{}: {}", dir.display(), error.message)); + continue; + } + }; + let spec = ServiceSpec { + label: options.label.clone(), + plist: options.plist.clone(), + program: dir.join("bin/devguardd"), + args: options.program_args.clone(), + environment: options.environment.clone(), + }; + let matches = render_plist(&spec, &options.log, options.throttle_seconds) + .ok() + .zip(installed.as_ref()) + .is_some_and(|(rendered, actual)| rendered == *actual); + if matches || chosen.is_none() { + chosen = Some((release, matches)); } + if matches { + break; + } + } + let Some((release, matches)) = chosen else { + report.running_error = Some(format!( + "the current release is not intact: {}", + damage.join("; ") + )); + return Ok(report); }; - let spec = ServiceSpec { - label: options.label.clone(), - plist: options.plist.clone(), - program: release_dir.join("bin/devguardd"), - args: options.program_args.clone(), - environment: options.environment.clone(), - }; - report.plist_matches_selection = render_plist(&spec, &options.log, options.throttle_seconds) - .ok() - .zip(fs::read_to_string(&options.plist).ok()) - .is_some_and(|(rendered, actual)| rendered == actual); + report.plist_matches_selection = matches; match observe_running(paths, manager, &options.label, &release) { Ok(evidence) => report.running = Some(evidence), Err(error) => report.running_error = Some(error.message), diff --git a/crates/daemon/src/lib.rs b/crates/daemon/src/lib.rs index 669839b..347a2f3 100644 --- a/crates/daemon/src/lib.rs +++ b/crates/daemon/src/lib.rs @@ -6,3 +6,4 @@ pub mod fixture; pub mod install; pub mod paths; pub mod server; +pub mod upgrade; diff --git a/crates/daemon/src/main.rs b/crates/daemon/src/main.rs index cea2682..7a00582 100644 --- a/crates/daemon/src/main.rs +++ b/crates/daemon/src/main.rs @@ -47,13 +47,13 @@ fn until_signalled(serve: impl FnOnce(Arc) -> Result) -> Resul fn run() -> Result<()> { let args: Vec<_> = std::env::args().skip(1).collect(); if args == ["--help"] || args == ["help"] { - println!("devguardd paths | init | check | serve | status | install --package DIR | version --json\n candidate --id ID --lease CONSUMER/GENERATION/ATTEMPT --capacity MILLICPU,BYTES,TASKS --token-fd N\nNormal authority paths come from the OS account. No path or test-budget override is accepted.\nserve observes native boot, process and host pressure evidence and, with it, opens registration, fenced launch through devguard-launch and reconciliation; `devguard` runs commands through it. install copies a package into a protected release and starts it as the current user's LaunchAgent; it must run from that package and refuses while an authority or the agent exists. status reports the installed service. init and check never start workloads.\ncandidate serves an isolated candidate authority whose capacity is a parent lease of the account's authority; it reads the lease token from the private descriptor N, launches nothing, and closes when the lease ends. `devguard test-candidate` starts it as a lease child."); + println!("devguardd paths | init | check | serve | status | install --package DIR | stage --package DIR | version --json\n candidate --id ID --lease CONSUMER/GENERATION/ATTEMPT --capacity MILLICPU,BYTES,TASKS --token-fd N\nNormal authority paths come from the OS account. No path or test-budget override is accepted.\nserve observes native boot, process and host pressure evidence and, with it, opens registration, fenced launch through devguard-launch and reconciliation; `devguard` runs commands through it. install copies a package into a protected release and starts it as the current user's LaunchAgent; it must run from that package and refuses while an authority or the agent exists. status reports the installed service. stage copies a package into a protected release for `devguard upgrade`, without touching the service; it too must run from that package. init and check never start workloads.\ncandidate serves an isolated candidate authority whose capacity is a parent lease of the account's authority; it reads the lease token from the private descriptor N, launches nothing, and closes when the lease ends. `devguard test-candidate` starts it as a lease child."); return Ok(()); } let command = args.first().map(String::as_str).unwrap_or_default(); let valid = match command { "paths" | "init" | "check" | "serve" | "status" => args.len() == 1, - "install" => args.len() == 3 && args[1] == "--package", + "install" | "stage" => args.len() == 3 && args[1] == "--package", "candidate" => args.len() == 9, "version" => args.len() == 2 && args[1] == "--json", _ => false, @@ -61,7 +61,7 @@ fn run() -> Result<()> { if !valid { return Err(Error::new( ErrorCode::InvalidRequest, - "expected paths, init, check, serve, status, install --package DIR, candidate or version --json; no alternate authority arguments are supported", + "expected paths, init, check, serve, status, install --package DIR, stage --package DIR, candidate or version --json; no alternate authority arguments are supported", )); } if command == "version" { @@ -87,6 +87,14 @@ fn run() -> Result<()> { .map_err(|_| Error::new(ErrorCode::InvalidRequest, "report encoding failed"))? ); } + "stage" => { + let report = devguard_daemon::upgrade::stage(&paths, std::path::Path::new(&args[2]))?; + println!( + "{}", + serde_json::to_string_pretty(&report) + .map_err(|_| Error::new(ErrorCode::InvalidRequest, "report encoding failed"))? + ); + } "status" => { let report = install::status( &paths, diff --git a/crates/daemon/src/paths.rs b/crates/daemon/src/paths.rs index a79d799..12de3b2 100644 --- a/crates/daemon/src/paths.rs +++ b/crates/daemon/src/paths.rs @@ -153,6 +153,15 @@ impl AuthorityPaths { pub fn lock(&self) -> PathBuf { self.state().join("authority.lock") } + /// Present while an administrator has closed admission, for example for + /// an upgrade; a service that starts finds admission still closed. + pub fn admission_marker(&self) -> PathBuf { + self.state().join("admission.json") + } + /// Quiescent journal backups taken before a release is replaced. + pub fn backups(&self) -> PathBuf { + self.root.join("backups") + } pub fn socket(&self) -> PathBuf { self.runtime.join("authority.sock") } @@ -302,6 +311,41 @@ pub fn read_private(path: &Path, uid: u32, limit: usize) -> Result> { Ok(bytes) } +/// Replace a private file atomically: write a new sibling, sync it, rename it. +pub fn replace_private(path: &Path, bytes: &[u8]) -> Result<()> { + let parent = path.parent().ok_or_else(|| { + Error::new( + ErrorCode::InvalidRequest, + "a private file needs a directory", + ) + })?; + let temporary = parent.join(format!( + ".{}.{}", + path.file_name().and_then(|n| n.to_str()).unwrap_or("file"), + uuid::Uuid::new_v4().simple() + )); + let written = (|| -> std::io::Result<()> { + let mut file = OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .custom_flags(libc::O_NOFOLLOW | libc::O_CLOEXEC) + .open(&temporary)?; + file.write_all(bytes)?; + file.sync_all()?; + fs::rename(&temporary, path)?; + File::open(parent)?.sync_all() + })(); + if written.is_err() { + let _ = fs::remove_file(&temporary); + return Err(Error::new( + ErrorCode::ResourceControlUnavailable, + format!("cannot write {}", path.display()), + )); + } + Ok(()) +} + pub fn write_new_private(path: &Path, bytes: &[u8]) -> Result<()> { let mut file = OpenOptions::new() .write(true) diff --git a/crates/daemon/src/server.rs b/crates/daemon/src/server.rs index 1c5cdcc..22e09f1 100644 --- a/crates/daemon/src/server.rs +++ b/crates/daemon/src/server.rs @@ -1,4 +1,7 @@ -use crate::{config::HostConfig, paths::AuthorityPaths}; +use crate::{ + config::HostConfig, + paths::{read_private, replace_private, AuthorityPaths}, +}; use devguard_client::{connect::connect_timeout, framing, peer, protocol::*}; use devguard_contract::{ validate_id, AppliedResources, AttemptKey, AttemptPhase, AttemptRecord, Budget, Capability, @@ -47,6 +50,8 @@ pub(crate) struct Options { pub reconcile_paused: Option>, /// A candidate authority bounded by a parent lease. pub candidate: Option, + /// Fixtures only: state no upgrade drain, as a release before C11 does. + pub without_upgrade_drain: bool, } /// A candidate's leased capacity and its status reason. A candidate states @@ -81,6 +86,8 @@ pub struct Server { launcher: Option, /// A candidate states no fenced launch or parent lease and refuses both. candidate: bool, + /// Whether the upgrade drain is stated to clients that require it. + upgrade_drain: bool, } /// Service receipts are JSON lines on stderr; they never include credentials. @@ -295,7 +302,69 @@ fn poisoned() -> Error { /// Capabilities stated only to clients that require them. pub(crate) fn echoed_capabilities() -> BTreeSet { - BTreeSet::from([Capability::ParentLease]) + BTreeSet::from([Capability::ParentLease, Capability::UpgradeDrain]) +} + +/// Admission closed by an administrator. The closure is kept in a private +/// marker beside the journal, written before it takes effect, so a restarted +/// service, or the next release, still finds admission closed. +#[derive(Clone)] +struct AdmissionGate { + marker: PathBuf, + closure: Arc>>, +} + +impl AdmissionGate { + fn open(paths: &AuthorityPaths) -> Self { + let marker = paths.admission_marker(); + let closure = fs::symlink_metadata(&marker).is_ok().then(|| { + read_private(&marker, paths.uid(), 4096) + .ok() + .and_then(|bytes| serde_json::from_slice::(&bytes).ok()) + // A marker that cannot be read still keeps admission closed. + .unwrap_or(AdmissionClosure { + reason: "the admission marker is unreadable".into(), + since_unix_ms: 0, + }) + }); + if let Some(closure) = &closure { + receipt(json!({"event": "admission_closed_at_start", "closure": closure})); + } + Self { + marker, + closure: Arc::new(Mutex::new(closure)), + } + } + + fn lock(&self) -> Result>> { + self.closure.lock().map_err(|_| poisoned()) + } + + /// Held for the whole of an admission, so a closure cannot interleave. + fn admitting(&self) -> Result>> { + let gate = self.lock()?; + if let Some(closure) = gate.as_ref() { + return Err(closed_admission(closure)); + } + Ok(gate) + } +} + +fn closed_admission(closure: &AdmissionClosure) -> Error { + Error::new( + ErrorCode::ResourceUnavailable, + format!( + "admission is closed ({}); queries, stops and reconciliation continue", + closure.reason + ), + ) +} + +fn unix_ms() -> u64 { + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|since| since.as_millis() as u64) + .unwrap_or(0) } /// The capabilities of a service with registration and fenced launch open. @@ -343,6 +412,7 @@ struct Launcher { principals: Arc>, clock: BootClock, paused: Arc, + gate: AdmissionGate, } /// Whether `identity` still names a running process. A process of an earlier @@ -363,6 +433,67 @@ impl Launcher { self.authority.lock().map_err(|_| poisoned()) } + /// Whether admission is open, and every charged attempt and lease. + fn quiescence(&self) -> Result { + let closure = self.gate.lock()?.clone(); + let mut authority = self.authority()?; + Ok(Quiescence { + closure, + attempts: authority.attempts()?, + leases: authority.leases()?, + }) + } + + /// Close admission: the closure is written before it takes effect, and + /// Prepared attempts are cancelled while the gate is held, so none can be + /// admitted meanwhile. Closing again keeps the first closure. + fn close_admission(&self, reason: String) -> Result { + if reason.is_empty() + || reason.len() > MAX_CLOSURE_REASON + || reason.chars().any(char::is_control) + { + return Err(Error::new( + ErrorCode::InvalidRequest, + "a closure reason is 1 to 256 bytes without control characters", + )); + } + let mut gate = self.gate.lock()?; + if gate.is_none() { + let closure = AdmissionClosure { + reason, + since_unix_ms: unix_ms(), + }; + let bytes = serde_json::to_vec(&closure) + .map_err(|_| unavailable("cannot encode the admission closure"))?; + replace_private(&self.gate.marker, &bytes)?; + receipt(json!({"event": "admission_closed", "closure": closure})); + *gate = Some(closure); + } + let cancelled = self.authority()?.cancel_prepared()?; + if !cancelled.is_empty() { + receipt(json!({"event": "prepared_cancelled", + "keys": cancelled.iter().map(|record| &record.key).collect::>()})); + } + drop(gate); + self.quiescence() + } + + /// Reopen admission; the marker is removed before it takes effect. + fn open_admission(&self) -> Result { + let mut gate = self.gate.lock()?; + if gate.is_some() { + match fs::remove_file(&self.gate.marker) { + Ok(()) => {} + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(_) => return Err(unavailable("cannot remove the admission marker")), + } + receipt(json!({"event": "admission_opened", "closure": *gate})); + *gate = None; + } + drop(gate); + self.quiescence() + } + /// Register the OS-observed peer with its native start identity. fn register( &self, @@ -797,6 +928,7 @@ impl Server { principals: Arc::default(), clock: clock.clone(), paused: options.reconcile_paused.unwrap_or_default(), + gate: AdmissionGate::open(paths), }), Evidence::Closed { .. } => None, }; @@ -862,6 +994,7 @@ impl Server { evidence, launcher, candidate, + upgrade_drain: !options.without_upgrade_drain, }) } @@ -939,12 +1072,14 @@ impl Server { let status = self.status.clone(); let launcher = self.launcher.clone(); let stop = stop.clone(); - let candidate = self.candidate; + let flags = Flags { + candidate: self.candidate, + upgrade_drain: self.upgrade_drain, + }; if let Ok(worker) = std::thread::Builder::new() .name("devguard-session".into()) .spawn(move || { - let _ = - session(stream, uid, &config, &status, launcher, candidate, &stop); + let _ = session(stream, uid, &config, &status, launcher, flags, &stop); }) { workers.push(worker); @@ -1059,6 +1194,13 @@ struct Session { lease_holder: bool, } +/// What a session's service states and refuses. +#[derive(Clone, Copy)] +struct Flags { + candidate: bool, + upgrade_drain: bool, +} + struct Context<'a> { uid: u32, caller: PeerIdentity, @@ -1066,6 +1208,7 @@ struct Context<'a> { status: &'a ServiceStatus, launcher: Option<&'a Launcher>, candidate: bool, + upgrade_drain: bool, } fn session( @@ -1074,7 +1217,7 @@ fn session( config: &HostConfig, status: &ServiceStatus, launcher: Option, - candidate: bool, + flags: Flags, stop: &AtomicBool, ) -> Result<()> { // Framing uses poll and per-call nonblocking I/O with an absolute deadline; @@ -1089,7 +1232,8 @@ fn session( config, status, launcher: launcher.as_ref(), - candidate, + candidate: flags.candidate, + upgrade_drain: flags.upgrade_drain, }; let timeout = Duration::from_millis(FRAME_DEADLINE_MS); let mut state = Session::default(); @@ -1129,6 +1273,16 @@ fn session( Ok(()) } +/// The launcher, for a session authenticated as the administrator. +fn administrator<'a>(session: &Session, context: &Context<'a>) -> Result<&'a Launcher> { + if session.role != Some(SessionRole::Administrator) { + return Err(unauthorized()); + } + context + .launcher + .ok_or_else(|| unavailable("the authority is not open; there is no admission to close")) +} + /// The launcher and the instance registered in this session. fn registered<'a>( session: &'a Session, @@ -1165,6 +1319,9 @@ fn handle(session: &mut Session, request: Request, context: &Context<'_>) -> Res | Request::AdmitChild { .. } | Request::EndLease { .. } | Request::LeaseStatus { .. } + | Request::CloseAdmission { .. } + | Request::OpenAdmission + | Request::Quiescence ) { return Err(candidate_refusal()); @@ -1189,7 +1346,10 @@ fn handle(session: &mut Session, request: Request, context: &Context<'_>) -> Res capabilities.extend( echoed_capabilities() .intersection(&compatibility.required) - .copied(), + .copied() + .filter(|capability| { + context.upgrade_drain || *capability != Capability::UpgradeDrain + }), ); capabilities }; @@ -1227,7 +1387,14 @@ fn handle(session: &mut Session, request: Request, context: &Context<'_>) -> Res if session.role.is_none() { return Err(unauthorized()); } - Ok(Response::Status(context.status.clone())) + let mut status = context.status.clone(); + if let Some(launcher) = context.launcher { + if let Some(closure) = launcher.gate.lock()?.as_ref() { + status.execution_ready = false; + status.reason = closed_admission(closure).message; + } + } + Ok(Response::Status(status)) } Request::Register { instance_id } => { validate_id(&instance_id)?; @@ -1252,6 +1419,7 @@ fn handle(session: &mut Session, request: Request, context: &Context<'_>) -> Res } Request::Admit { request } => { let (launcher, principal) = registered(session, context)?; + let _gate = launcher.gate.admitting()?; let record = launcher.authority()?.admit(principal, request)?; receipt( json!({"event": "admission", "key": record.key, "phase": record.phase, @@ -1262,6 +1430,7 @@ fn handle(session: &mut Session, request: Request, context: &Context<'_>) -> Res } Request::BeginLaunch { key } => { let (launcher, principal) = registered(session, context)?; + let _gate = launcher.gate.admitting()?; let decision = launcher.authority()?.begin_launch(principal, &key)?; receipt( json!({"event": "launch_committed", "key": decision.attempt.key, @@ -1326,6 +1495,7 @@ fn handle(session: &mut Session, request: Request, context: &Context<'_>) -> Res ttl_ms, } => { let (launcher, principal) = registered(session, context)?; + let _gate = launcher.gate.admitting()?; let grant = launcher .authority()? .admit_lease(principal, &key, budget, ttl_ms)?; @@ -1343,6 +1513,7 @@ fn handle(session: &mut Session, request: Request, context: &Context<'_>) -> Res request, } => { let (launcher, principal) = registered(session, context)?; + let _gate = launcher.gate.admitting()?; let record = launcher .authority()? .admit_child(principal, &lease, &token, request)?; @@ -1373,6 +1544,15 @@ fn handle(session: &mut Session, request: Request, context: &Context<'_>) -> Res launcher.authority()?.lease_status(&key, &token)?, )) } + Request::CloseAdmission { reason } => Ok(Response::Quiescence( + administrator(session, context)?.close_admission(reason)?, + )), + Request::OpenAdmission => Ok(Response::Quiescence( + administrator(session, context)?.open_admission()?, + )), + Request::Quiescence => Ok(Response::Quiescence( + administrator(session, context)?.quiescence()?, + )), } } @@ -1921,6 +2101,205 @@ mod tests { } } + /// Closing admission for an upgrade: an administrator's request, echoed + /// only to clients that require it, kept across a restart, and leaving + /// queries and reconciliation open. + #[cfg(target_os = "macos")] + mod drain { + use super::*; + use crate::fixture::{TestAuthority, CONSUMER}; + use devguard_contract::{ + AdmissionRequest, AttemptKey, Budget, Capability, Compatibility, ResourceIntent, + ResourceLevels, Secret, + }; + + fn base() -> tempfile::TempDir { + tempfile::Builder::new() + .prefix("dg-d-") + .tempdir_in("/private/tmp") + .unwrap() + } + + fn started(base: &std::path::Path, restart: bool) -> TestAuthority { + let authority = if restart { + TestAuthority::restart(base).unwrap() + } else { + TestAuthority::start(base).unwrap() + }; + authority + .wait_until_admitting(Duration::from_secs(10)) + .unwrap(); + authority + } + + fn connect(authority: &TestAuthority, required: &[Capability]) -> Client { + Client::connect( + &authority.socket(), + authority.uid(), + Compatibility { + minimum_protocol: 1, + maximum_protocol: 1, + required: required.iter().copied().collect(), + }, + ) + .unwrap() + } + + fn administrator(authority: &TestAuthority) -> Client { + let secret = Secret::new( + String::from_utf8( + read_private(&authority.paths().admin_credential(), authority.uid(), 64) + .unwrap(), + ) + .unwrap(), + ) + .unwrap(); + let mut client = connect(authority, &[Capability::UpgradeDrain]); + client + .authenticate(CallerCredential::Administrator { secret }) + .unwrap(); + client + } + + fn workload(authority: &TestAuthority) -> Client { + let mut client = connect( + authority, + &[Capability::DurableAdmission, Capability::ParentLease], + ); + client.authenticate(authority.consumer().unwrap()).unwrap(); + client.register("drain-owner".into()).unwrap(); + client + } + + fn request(authority: &TestAuthority, id: &str) -> AdmissionRequest { + AdmissionRequest { + key: AttemptKey { + consumer_id: CONSUMER.into(), + consumer_generation: authority.generation(), + attempt_id: id.into(), + }, + execution_digest: devguard_contract::digest_bytes(b"drain"), + intent: ResourceIntent { + profile: "interactive".into(), + requested: Budget { + cpu_milli: 100, + memory_bytes: 64 * 1024 * 1024, + tasks: 4, + }, + minimum: ResourceLevels::MACOS, + }, + } + } + + #[test] + fn drain_closes_admission_for_the_administrator_only_and_keeps_it_across_a_restart() { + let directory = base(); + let authority = started(directory.path(), false); + let plain = connect(&authority, &[Capability::DurableAdmission]); + assert!(!plain.hello.capabilities.contains(&Capability::UpgradeDrain)); + // A workload cannot close admission. + assert_eq!( + workload(&authority) + .close_admission("not mine".into()) + .unwrap_err() + .code, + ErrorCode::Unauthorized + ); + let prepared = workload(&authority) + .admit(request(&authority, "prepared")) + .unwrap(); + assert_eq!(prepared.phase, AttemptPhase::Prepared); + let staged = workload(&authority) + .admit(request(&authority, "staged")) + .unwrap(); + assert_eq!( + administrator(&authority) + .close_admission(String::new()) + .unwrap_err() + .code, + ErrorCode::InvalidRequest + ); + let closed = administrator(&authority) + .close_admission("upgrade to test".into()) + .unwrap(); + let since = closed.closure.clone().unwrap().since_unix_ms; + // The Prepared attempt was cancelled, so nothing is charged. + assert!(closed.quiet(), "{closed:?}"); + assert_eq!( + workload(&authority) + .lookup(prepared.key.clone()) + .unwrap() + .phase, + AttemptPhase::Cancelled + ); + for refused in [ + workload(&authority) + .admit(request(&authority, "late")) + .err(), + workload(&authority) + .admit_lease( + request(&authority, "lease").key, + request(&authority, "lease").intent.requested, + None, + ) + .err(), + ] { + let error = refused.expect("admission is closed"); + assert_eq!(error.code, ErrorCode::ResourceUnavailable); + assert!(error.message.contains("admission is closed"), "{error:?}"); + } + // No launch is committed while admission is closed, so no helper + // can arrive for an attempt the drain cancelled. + assert_eq!( + workload(&authority) + .begin_launch(staged.key.clone()) + .unwrap_err() + .code, + ErrorCode::ResourceUnavailable + ); + let status = workload(&authority).status().unwrap(); + assert!(!status.execution_ready && status.registration_ready); + assert!(status.reason.contains("upgrade to test")); + // Closing again keeps the first closure. + let again = administrator(&authority) + .close_admission("another reason".into()) + .unwrap(); + assert_eq!(again.closure.unwrap().since_unix_ms, since); + let marker = authority.paths().admission_marker(); + assert!(read_private(&marker, authority.uid(), 4096).is_ok()); + // A restarted service starts with admission still closed. + drop(authority); + let authority = started(directory.path(), true); + assert_eq!( + workload(&authority) + .admit(request(&authority, "after-restart")) + .unwrap_err() + .code, + ErrorCode::ResourceUnavailable + ); + let opened = administrator(&authority).open_admission().unwrap(); + assert!(opened.closure.is_none()); + assert!(!marker.exists()); + assert_eq!( + workload(&authority) + .admit(request(&authority, "reopened")) + .unwrap() + .phase, + AttemptPhase::Prepared + ); + // Quiescence reports what is charged again. + let charged = administrator(&authority).quiescence().unwrap(); + assert_eq!(charged.attempts.len(), 1); + assert!(!charged.quiet()); + // Opening twice changes nothing. + assert!(administrator(&authority) + .open_admission() + .unwrap() + .closure + .is_none()); + } + } + /// Sessions of a service with registration and fenced launch open. #[cfg(target_os = "macos")] mod launch { diff --git a/crates/daemon/src/upgrade.rs b/crates/daemon/src/upgrade.rs new file mode 100644 index 0000000..2335971 --- /dev/null +++ b/crates/daemon/src/upgrade.rs @@ -0,0 +1,862 @@ +//! Replacing and repairing the installed service without losing or +//! duplicating charged work (DG1-C11). +//! +//! An upgrade replaces the current release with a staged one. It checks that +//! the new release can serve the same consumers and read the same journal, +//! closes admission at the running service (cancelling Prepared attempts) +//! and waits until nothing is charged. It then stops the service, takes a +//! quiescent backup of the journal, the selection and the current manifest, +//! starts the new release with admission still closed, verifies the running +//! binary, the handshake and the state, and only then reopens admission. A +//! drain that does not finish in time is abandoned: admission reopens on the +//! current release and every charge is kept. If the new release cannot be +//! verified, the previous one is started again on the same journal; a backup +//! is never restored over state a release may have admitted from. +//! +//! Repair starts the last known good release while no authority serves. It +//! never starts a second authority, keeps a journal that cannot be opened +//! closed, and uses the recovery copy when the installed release is damaged. + +use crate::install::{ + self, compiled, digest_file, freeze_copy, handshake, observe_running, read_release, + read_selection, render_plist, unix_ms, validate_package, wait_running, write_selection, + BuildCompatibility, InstallOptions, Package, RunningEvidence, Selection, SelectionEvent, + ServiceManager, ServiceSpec, +}; +use crate::paths::{read_private, replace_private, secure_directory, AuthorityPaths}; +use devguard_client::protocol::{AdmissionClosure, CallerCredential, Quiescence}; +use devguard_client::Client; +use devguard_contract::{Capability, Compatibility, Error, ErrorCode, Result, Secret}; +use devguard_core::AuthorityStorage; +use serde::Serialize; +use std::collections::{BTreeMap, BTreeSet}; +use std::fs::{self, File, OpenOptions}; +use std::io::Write; +use std::os::unix::fs::{DirBuilderExt, OpenOptionsExt}; +use std::path::{Path, PathBuf}; +use std::time::{Duration, Instant}; + +/// The default time a drain may take before the upgrade is abandoned. +pub const DEFAULT_DRAIN_TIMEOUT: Duration = Duration::from_secs(60); +/// How long a stopped service may take to release the endpoint and the lock. +const STOP_LIMIT: Duration = Duration::from_secs(15); +const POLL: Duration = Duration::from_millis(200); +/// What every consumer of the service requires; a release without it cannot +/// replace the current one. +pub const CONSUMER_REQUIREMENTS: [Capability; 2] = + [Capability::DurableAdmission, Capability::FencedLaunch]; + +fn invalid(message: impl Into) -> Error { + Error::new(ErrorCode::InvalidRequest, message) +} + +fn unavailable(message: impl Into) -> Error { + Error::new(ErrorCode::ResourceControlUnavailable, message) +} + +fn busy(message: impl Into) -> Error { + Error::new(ErrorCode::ResourceUnavailable, message) +} + +fn unsupported(message: impl Into) -> Error { + Error::new(ErrorCode::ResourcePolicyUnsupported, message) +} + +fn macos_only() -> Result<()> { + if cfg!(target_os = "macos") { + Ok(()) + } else { + Err(unsupported("the service is managed as a macOS LaunchAgent")) + } +} + +/// The running program must be `artifact` of `release`, so a release's +/// binaries are never mixed with another build's. +fn running_from(release: &Package, artifact: &str) -> Result<()> { + let running = + std::env::current_exe().map_err(|_| unavailable("cannot locate the running program"))?; + if digest_file(&running)? != release.manifest.artifacts[artifact].sha256 { + return Err(invalid(format!( + "run this from the release it selects: its {artifact} must be this program" + ))); + } + Ok(()) +} + +/// The release under `releases/`, by a plain id. +fn release_dir(paths: &AuthorityPaths, id: &str) -> Result { + if id.is_empty() + || id.starts_with('.') + || !id + .bytes() + .all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'_' | b'-')) + { + return Err(invalid("a release id is a plain name")); + } + Ok(paths.releases().join(id)) +} + +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +pub struct StageReport { + pub release_id: String, + pub release: PathBuf, + pub manifest_sha256: String, + pub reused_release: bool, +} + +/// Copy `package` into the releases as an immutable release, ready for an +/// upgrade. The service, the selection and the journal are untouched. It must +/// run from the package, as installation does. +pub fn stage(paths: &AuthorityPaths, package: &Path) -> Result { + macos_only()?; + let package = validate_package(package)?; + running_from(&package, "devguardd")?; + secure_directory(&paths.releases(), paths.uid(), true)?; + install::sweep_incoming(&paths.releases()); + let id = package.manifest.release_id.clone(); + let release = paths.releases().join(&id); + let reused = freeze_copy(&package, &release)?; + let staged = validate_package(&release)?; + Ok(StageReport { + release_id: id, + release, + manifest_sha256: staged.manifest_sha256, + reused_release: reused, + }) +} + +/// What changes between the current release and its replacement. +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +pub struct CompatibilityReport { + pub from: BuildCompatibility, + pub to: BuildCompatibility, + /// Capabilities the current release states that the replacement lacks. + pub dropped: BTreeSet, + /// The replacement states less than the current release. It is allowed + /// only when it reads the same journal and serves the same consumers. + pub downgrade: bool, +} + +/// Whether `to` can replace `from`: the same wire version, protocol, journal +/// schema and configuration schema, and everything consumers require. A +/// replacement that states less is a downgrade; an incompatible one is refused. +pub fn check_compatibility( + from: &BuildCompatibility, + to: &BuildCompatibility, +) -> Result { + if from.wire_version != to.wire_version || from.protocol != to.protocol { + return Err(unsupported( + "the releases speak different wire versions or protocols; the replacement is refused", + )); + } + if from.journal_schema != to.journal_schema { + return Err(unsupported(format!( + "the replacement reads journal schema {}, not {}; an incompatible downgrade or an unsupported migration is refused", + to.journal_schema, from.journal_schema + ))); + } + if from.config_schema != to.config_schema { + return Err(unsupported( + "the replacement reads another configuration schema; the replacement is refused", + )); + } + let missing: Vec = CONSUMER_REQUIREMENTS + .into_iter() + .filter(|capability| !to.capabilities.contains(capability)) + .collect(); + if !missing.is_empty() { + return Err(unsupported(format!( + "the replacement lacks {missing:?}, which consumers require; the replacement is refused" + ))); + } + let dropped: BTreeSet = from + .capabilities + .difference(&to.capabilities) + .copied() + .collect(); + Ok(CompatibilityReport { + from: from.clone(), + to: to.clone(), + downgrade: !dropped.is_empty(), + dropped, + }) +} + +/// Options of one upgrade. +#[derive(Debug, Clone)] +pub struct UpgradeOptions { + pub drain_timeout: Duration, + /// Replace a running release that cannot close admission by stopping it + /// first; the replacement proceeds only if nothing is then charged. + pub stopped: bool, +} + +impl Default for UpgradeOptions { + fn default() -> Self { + Self { + drain_timeout: DEFAULT_DRAIN_TIMEOUT, + stopped: false, + } + } +} + +/// How the running service was brought to rest. +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +#[serde(rename_all = "snake_case")] +pub enum DrainMode { + /// It closed admission and drained while serving. + ClosedAdmission, + /// It could not close admission, so it was stopped and its journal checked. + Stopped, +} + +#[derive(Debug, Clone, Serialize)] +pub struct DrainReport { + pub mode: DrainMode, + pub waited_ms: u64, + /// What the service reported when admission closed. + pub at_close: Option, +} + +#[derive(Debug, Clone, Serialize)] +pub struct BackupReport { + pub directory: PathBuf, + /// File name to SHA-256. + pub files: BTreeMap, +} + +#[derive(Debug, Clone, Serialize)] +pub struct UpgradeReport { + pub from: String, + pub to: String, + pub compatibility: CompatibilityReport, + pub drain: DrainReport, + pub backup: BackupReport, + pub running: RunningEvidence, + /// What the new release reported before admission reopened. + pub before_reopening: Quiescence, + pub recovery: PathBuf, + pub selection: Selection, +} + +fn admin_secret(paths: &AuthorityPaths) -> Result { + let bytes = read_private(&paths.admin_credential(), paths.uid(), 1024)?; + Secret::new( + String::from_utf8(bytes) + .map_err(|_| Error::new(ErrorCode::Unauthorized, "the admin credential is invalid"))?, + ) +} + +/// A new administrator session that requires the drain capability. +fn administrator(paths: &AuthorityPaths, secret: &Secret) -> Result { + let mut client = Client::connect( + &paths.socket(), + paths.uid(), + Compatibility { + minimum_protocol: 1, + maximum_protocol: 1, + required: BTreeSet::from([Capability::UpgradeDrain]), + }, + )?; + client.authenticate(CallerCredential::Administrator { + secret: secret.clone(), + })?; + Ok(client) +} + +/// A call that is retried once when its reply was lost; every drain call is +/// idempotent. +fn admin_call( + paths: &AuthorityPaths, + secret: &Secret, + call: impl Fn(&mut Client) -> Result, +) -> Result { + let attempt = || administrator(paths, secret).and_then(|mut client| call(&mut client)); + match attempt() { + Err(error) if error.code == ErrorCode::ResourceControlUnavailable => attempt(), + other => other, + } +} + +/// The handshake every consumer makes must succeed. +fn consumer_handshake(paths: &AuthorityPaths) -> Result<()> { + Client::connect( + &paths.socket(), + paths.uid(), + Compatibility { + minimum_protocol: 1, + maximum_protocol: 1, + required: CONSUMER_REQUIREMENTS.into_iter().collect(), + }, + ) + .map(drop) +} + +fn spec_for(release: &Package, options: &InstallOptions) -> ServiceSpec { + ServiceSpec { + label: options.label.clone(), + plist: options.plist.clone(), + program: release.dir.join("bin/devguardd"), + args: options.program_args.clone(), + environment: options.environment.clone(), + } +} + +/// Replace the plist, which is not private, atomically. +fn write_plist(spec: &ServiceSpec, options: &InstallOptions) -> Result<()> { + let text = render_plist(spec, &options.log, options.throttle_seconds)?; + let parent = options + .plist + .parent() + .ok_or_else(|| invalid("the plist needs a directory"))?; + install::ensure_plain_directory(parent, 0o755)?; + let temporary = parent.join(format!( + ".{}.{}", + options.label, + uuid::Uuid::new_v4().simple() + )); + let written = (|| -> std::io::Result<()> { + let mut file = OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o644) + .custom_flags(libc::O_NOFOLLOW | libc::O_CLOEXEC) + .open(&temporary)?; + file.write_all(text.as_bytes())?; + file.sync_all()?; + fs::rename(&temporary, &options.plist)?; + File::open(parent)?.sync_all() + })(); + if written.is_err() { + let _ = fs::remove_file(&temporary); + return Err(unavailable("cannot write the service plist")); + } + Ok(()) +} + +/// Stop the service and wait until it has released the endpoint and the +/// authority lock; the returned storage holds the lock. +fn stop( + paths: &AuthorityPaths, + manager: &dyn ServiceManager, + options: &InstallOptions, +) -> Result { + manager.bootout(&options.label)?; + let deadline = Instant::now() + STOP_LIMIT; + loop { + let last = if handshake(paths).is_ok() { + busy("an authority still serves after the service was stopped") + } else { + match AuthorityStorage::open(&paths.journal()) { + Ok(storage) => return Ok(storage), + Err(error) => error, + } + }; + if Instant::now() >= deadline { + return Err(last); + } + std::thread::sleep(POLL); + } +} + +/// Start `release` as the service and verify that launchd runs it. +fn start( + paths: &AuthorityPaths, + manager: &dyn ServiceManager, + options: &InstallOptions, + release: &Package, +) -> Result { + let spec = spec_for(release, options); + write_plist(&spec, options)?; + let started = manager + .bootstrap(&spec) + .and_then(|()| wait_running(paths, manager, options, release)); + if started.is_err() { + let _ = manager.bootout(&options.label); + } + started +} + +fn write_marker(paths: &AuthorityPaths, reason: &str) -> Result<()> { + let closure = AdmissionClosure { + reason: reason.into(), + since_unix_ms: unix_ms() as u64, + }; + let bytes = + serde_json::to_vec(&closure).map_err(|_| unavailable("cannot encode the closure"))?; + replace_private(&paths.admission_marker(), &bytes) +} + +fn remove_marker(paths: &AuthorityPaths) -> Result<()> { + match fs::remove_file(paths.admission_marker()) { + Ok(()) => Ok(()), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()), + Err(_) => Err(unavailable("cannot remove the admission marker")), + } +} + +/// Reopen admission on whatever release now serves: through the service when +/// it can, or by removing the marker when it cannot read one. +fn reopen(paths: &AuthorityPaths, secret: &Secret) -> Result<()> { + match admin_call(paths, secret, |client| client.open_admission()) { + Ok(_) => Ok(()), + Err(error) if error.code == ErrorCode::ResourcePolicyUnsupported => remove_marker(paths), + Err(error) => Err(error), + } +} + +/// Take a quiescent backup while `storage` holds the lock: the journal, the +/// selection and the current release's manifest, each hashed. +fn backup( + paths: &AuthorityPaths, + storage: &mut AuthorityStorage, + current: &Package, + to: &str, +) -> Result { + secure_directory(&paths.backups(), paths.uid(), true)?; + let directory = paths.backups().join(format!( + "{}-{}-to-{}", + unix_ms(), + current.manifest.release_id, + to + )); + fs::DirBuilder::new() + .mode(0o700) + .create(&directory) + .map_err(|_| unavailable("cannot create the backup directory"))?; + let journal = directory.join("authority.sqlite"); + OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .custom_flags(libc::O_NOFOLLOW | libc::O_CLOEXEC) + .open(&journal) + .map_err(|_| unavailable("cannot create the journal backup"))?; + storage.backup(&journal)?; + let mut copies = vec![( + "MANIFEST.json".to_string(), + current.dir.join("MANIFEST.json"), + )]; + if fs::symlink_metadata(paths.selection()).is_ok() { + copies.push(("selection.json".into(), paths.selection())); + } + for (name, from) in copies { + let bytes = fs::read(&from).map_err(|_| unavailable("cannot read a backed-up file"))?; + crate::paths::write_new_private(&directory.join(&name), &bytes)?; + } + let mut files = BTreeMap::new(); + for entry in fs::read_dir(&directory) + .map_err(|_| unavailable("cannot list the backup"))? + .flatten() + { + let name = entry.file_name().to_string_lossy().into_owned(); + files.insert(name, digest_file(&entry.path())?); + } + File::open(&directory) + .and_then(|dir| dir.sync_all()) + .map_err(|_| unavailable("cannot sync the backup"))?; + Ok(BackupReport { directory, files }) +} + +/// Close admission at the running service and wait until nothing is charged. +/// On a timeout admission reopens and the charges are kept. +fn drain( + paths: &AuthorityPaths, + secret: &Secret, + reason: &str, + timeout: Duration, +) -> Result<(Quiescence, Duration)> { + let started = Instant::now(); + // Whatever interrupts the drain, admission does not stay closed. + let reopened = |error: Error| match reopen(paths, secret) { + Ok(()) => error, + Err(reopening) => Error::new( + error.code, + format!( + "{}; reopening admission also failed ({})", + error.message, reopening.message + ), + ), + }; + let at_close = admin_call(paths, secret, |client| { + client.close_admission(reason.into()) + }) + .map_err(reopened)?; + let deadline = started + timeout; + loop { + let now = admin_call(paths, secret, |client| client.quiescence()).map_err(reopened)?; + if now.quiet() { + return Ok((at_close, started.elapsed())); + } + if Instant::now() >= deadline { + reopen(paths, secret)?; + return Err(busy(format!( + "the drain did not finish in {} s: {} attempts and {} leases are still charged; admission reopened on the current release, which keeps them", + timeout.as_secs(), + now.attempts.len(), + now.leases.len() + ))); + } + std::thread::sleep(POLL); + } +} + +/// Start the current release again after a failed replacement, on the same +/// journal, and reopen admission. +fn restore( + paths: &AuthorityPaths, + manager: &dyn ServiceManager, + options: &InstallOptions, + current: &Package, + secret: &Secret, + cause: Error, +) -> Error { + let restored = start(paths, manager, options, current).and_then(|_| reopen(paths, secret)); + match restored { + Ok(()) => Error::new( + cause.code, + format!( + "{}; release {} serves again with admission open", + cause.message, current.manifest.release_id + ), + ), + Err(error) => Error::new( + ErrorCode::ReconciliationRequired, + format!( + "{}; restarting release {} also failed ({}): run `devguard repair --use last-known-good`", + cause.message, current.manifest.release_id, error.message + ), + ), + } +} + +/// Replace the current release with the staged release `to`, run from that +/// release's own `devguard`. +pub fn upgrade( + paths: &AuthorityPaths, + to: &str, + manager: &dyn ServiceManager, + options: &InstallOptions, + upgrade: &UpgradeOptions, +) -> Result { + macos_only()?; + paths.validate_existing()?; + let mut selection = + read_selection(paths)?.ok_or_else(|| invalid("no release is installed; install one"))?; + let from = selection.current.clone(); + if from == to { + return Err(invalid(format!("release {to} is already current"))); + } + let target = validate_package(&release_dir(paths, to)?) + .map_err(|error| Error::new(error.code, format!("release {to}: {}", error.message)))?; + running_from(&target, "devguard")?; + let current = read_release(&release_dir(paths, &from)?).map_err(|error| { + Error::new( + error.code, + format!( + "the current release {from} is not intact ({}); use repair", + error.message + ), + ) + })?; + let compatibility = check_compatibility( + ¤t.manifest.compatibility, + &target.manifest.compatibility, + )?; + observe_running(paths, manager, &options.label, ¤t).map_err(|error| { + busy(format!( + "the current release {from} is not running verified ({}); use repair", + error.message + )) + })?; + // The recovery copy is made before anything changes; repair uses it only + // once the release has been verified running. + secure_directory(&paths.recovery(), paths.uid(), true)?; + let recovery = paths.recovery().join(to); + freeze_copy(&target, &recovery)?; + let secret = admin_secret(paths)?; + let reason = format!("upgrade from {from} to {to}"); + // A release that cannot close admission refuses the capability itself. + let drainable = match administrator(paths, &secret) { + Ok(_) => true, + Err(error) if error.code == ErrorCode::ResourcePolicyUnsupported => false, + Err(error) => return Err(error), + }; + let (mode, at_close, waited) = if drainable { + let (at_close, waited) = drain(paths, &secret, &reason, upgrade.drain_timeout)?; + (DrainMode::ClosedAdmission, Some(at_close), waited) + } else if upgrade.stopped { + (DrainMode::Stopped, None, Duration::ZERO) + } else { + return Err(unsupported(format!( + "release {from} cannot close admission; rerun with --stopped to stop it first, which proceeds only if nothing is then charged" + ))); + }; + let mut storage = match stop(paths, manager, options) { + Ok(storage) => storage, + Err(error) => { + return Err(restore(paths, manager, options, ¤t, &secret, error)); + } + }; + let (attempts, leases) = storage.charged()?; + if !attempts.is_empty() || !leases.is_empty() { + drop(storage); + let cause = busy(format!( + "{} attempts and {} leases are still charged; nothing was replaced", + attempts.len(), + leases.len() + )); + return Err(restore(paths, manager, options, ¤t, &secret, cause)); + } + let backup = match backup(paths, &mut storage, ¤t, to) { + Ok(backup) => backup, + Err(error) => { + drop(storage); + return Err(restore(paths, manager, options, ¤t, &secret, error)); + } + }; + // The new release starts with admission closed and opens it only once + // it has been verified. + let marked = write_marker(paths, &reason); + drop(storage); + if let Err(error) = marked { + return Err(restore(paths, manager, options, ¤t, &secret, error)); + } + let verified = start(paths, manager, options, &target).and_then(|running| { + consumer_handshake(paths)?; + let before = admin_call(paths, &secret, |client| client.quiescence())?; + if before.closure.is_none() || !before.quiet() { + return Err(unavailable( + "the new release did not start closed and idle on the journal", + )); + } + Ok((running, before)) + }); + let (running, before_reopening) = match verified { + Ok(verified) => verified, + Err(error) => { + let _ = manager.bootout(&options.label); + return Err(restore(paths, manager, options, ¤t, &secret, error)); + } + }; + // Verified: select it, then reopen admission. + selection.current = to.into(); + selection.last_known_good = Some(to.into()); + selection.history.push(SelectionEvent { + release_id: to.into(), + event: format!("upgraded from {from} and verified running"), + unix_ms: unix_ms(), + }); + write_selection(paths, &selection).map_err(|error| { + Error::new( + error.code, + format!( + "release {to} serves with admission closed, but the selection could not be recorded ({}); run the upgrade again", + error.message + ), + ) + })?; + admin_call(paths, &secret, |client| client.open_admission()).map_err(|error| { + Error::new( + error.code, + format!( + "release {to} serves and is selected, but admission could not be reopened: {}", + error.message + ), + ) + })?; + Ok(UpgradeReport { + from, + to: to.into(), + compatibility, + drain: DrainReport { + mode, + waited_ms: waited.as_millis() as u64, + at_close, + }, + backup, + running, + before_reopening, + recovery, + selection, + }) +} + +#[derive(Debug, Clone, Serialize)] +pub struct RepairReport { + pub release_id: String, + /// The installed release, or its recovery copy when that was damaged. + pub artifact: PathBuf, + pub from_recovery: bool, + /// Why the installed release was not used, when it was not. + pub damaged: Option, + pub running: RunningEvidence, + /// An admission closure left by an interrupted upgrade, now reopened. + pub reopened_closure: bool, + pub selection: Selection, +} + +/// Start the last known good release while no authority serves, run from +/// that release's own `devguard` or its recovery copy's. +pub fn repair( + paths: &AuthorityPaths, + manager: &dyn ServiceManager, + options: &InstallOptions, +) -> Result { + macos_only()?; + paths.validate_existing()?; + let mut selection = + read_selection(paths)?.ok_or_else(|| invalid("no release is installed; install one"))?; + let id = selection + .last_known_good + .clone() + .ok_or_else(|| invalid("no release has been verified running yet"))?; + if handshake(paths).is_ok() { + return Err(busy( + "an authority is serving; repair never starts a second one, and a serving release is replaced by an upgrade", + )); + } + // The journal must open: repair never creates, resets or restores it. + let storage = AuthorityStorage::open(&paths.journal()).map_err(|error| { + Error::new( + error.code, + format!( + "the journal cannot be opened ({}); admission stays closed and repair never reinitializes it", + error.message + ), + ) + })?; + drop(storage); + let installed = release_dir(paths, &id)?; + let (release, damaged) = match validate_package(&installed) { + Ok(release) => (release, None), + Err(error) => { + let copy = validate_package(&paths.recovery().join(&id)).map_err(|recovery| { + Error::new( + ErrorCode::ReconciliationRequired, + format!( + "release {id} is damaged ({}) and so is its recovery copy ({})", + error.message, recovery.message + ), + ) + })?; + (copy, Some(error.message)) + } + }; + running_from(&release, "devguard")?; + let closure = fs::symlink_metadata(paths.admission_marker()).is_ok(); + // A job left loaded, for example one that keeps failing, is replaced. + manager.bootout(&options.label)?; + let running = start(paths, manager, options, &release)?; + let secret = admin_secret(paths)?; + if closure { + reopen(paths, &secret)?; + } + let from_recovery = damaged.is_some(); + selection.current = id.clone(); + selection.history.push(SelectionEvent { + release_id: id.clone(), + event: if from_recovery { + "repaired to the last known good release from its recovery copy".into() + } else { + "repaired to the last known good release".into() + }, + unix_ms: unix_ms(), + }); + write_selection(paths, &selection)?; + Ok(RepairReport { + release_id: id, + artifact: release.dir.clone(), + from_recovery, + damaged, + running, + reopened_closure: closure, + selection, + }) +} + +/// This build's compatibility, for checks that compare releases. +pub fn this_build() -> BuildCompatibility { + compiled() +} + +#[cfg(test)] +mod tests { + use super::*; + + fn build(capabilities: &[Capability]) -> BuildCompatibility { + BuildCompatibility { + capabilities: capabilities.iter().copied().collect(), + ..compiled() + } + } + + #[test] + fn a_replacement_must_read_the_journal_and_serve_every_consumer() { + let current = compiled(); + let report = check_compatibility(¤t, ¤t).unwrap(); + assert!(!report.downgrade && report.dropped.is_empty()); + // A release that states less is a downgrade, allowed while it reads + // the same journal and serves the same consumers. + let older = build(&[ + Capability::DurableAdmission, + Capability::FencedLaunch, + Capability::PerResourceEvidence, + Capability::StaticControlReservations, + Capability::MacosCooperative, + ]); + let report = check_compatibility(¤t, &older).unwrap(); + assert!(report.downgrade); + assert_eq!( + report.dropped, + BTreeSet::from([Capability::ParentLease, Capability::UpgradeDrain]) + ); + assert!(!check_compatibility(&older, ¤t).unwrap().downgrade); + // Incompatible replacements are refused. + for (to, needle) in [ + ( + BuildCompatibility { + journal_schema: "2".into(), + ..current.clone() + }, + "journal schema", + ), + ( + BuildCompatibility { + protocol: current.protocol + 1, + ..current.clone() + }, + "protocols", + ), + ( + BuildCompatibility { + wire_version: current.wire_version + 1, + ..current.clone() + }, + "protocols", + ), + ( + BuildCompatibility { + config_schema: current.config_schema + 1, + ..current.clone() + }, + "configuration", + ), + (build(&[Capability::DurableAdmission]), "consumers require"), + ] { + let error = check_compatibility(¤t, &to).unwrap_err(); + assert_eq!(error.code, ErrorCode::ResourcePolicyUnsupported); + assert!(error.message.contains(needle), "{needle}: {error:?}"); + } + } + + #[test] + fn release_ids_name_plain_directories() { + let directory = tempfile::tempdir().unwrap(); + let paths = AuthorityPaths::fixture(directory.path()); + assert!(release_dir(&paths, "0.1.0-abc-12345678").is_ok()); + for bad in ["", ".x", "../state", "a/b", "a b"] { + assert!(release_dir(&paths, bad).is_err(), "{bad}"); + } + } +} diff --git a/crates/daemon/tests/entrypoint.rs b/crates/daemon/tests/entrypoint.rs index f3dc278..52e2919 100644 --- a/crates/daemon/tests/entrypoint.rs +++ b/crates/daemon/tests/entrypoint.rs @@ -7,6 +7,8 @@ fn alternate_authority_and_unparented_candidate_arguments_are_rejected() { vec!["check", "--socket", "/tmp/second.sock"], vec!["init", "--test-capacity", "8000"], vec!["candidate"], + vec!["stage"], + vec!["stage", "--package"], ] { let result = Command::new(env!("CARGO_BIN_EXE_devguardd")) .args(arguments) diff --git a/crates/daemon/tests/install.rs b/crates/daemon/tests/install.rs index 38375ea..f1a0377 100644 --- a/crates/daemon/tests/install.rs +++ b/crates/daemon/tests/install.rs @@ -8,268 +8,25 @@ //! test uses launchd itself, with a unique label and a plist under the test's //! own directory, and always boots the job out. -use devguard_contract::{digest_bytes, ErrorCode}; -use devguard_daemon::fixture::{serve_until_terminated, TestAuthority}; -use devguard_daemon::install::{ - self, Artifact, InstallOptions, Launchctl, Manifest, ServiceManager, ServiceSpec, ServiceState, - ARTIFACTS, MANIFEST_SCHEMA, -}; +mod support; +use devguard_contract::ErrorCode; +use devguard_daemon::fixture::TestAuthority; +use devguard_daemon::install::{self, Launchctl, Manifest, ServiceManager, ServiceSpec, ARTIFACTS}; use devguard_daemon::paths::AuthorityPaths; -use serde_json::{json, Value}; -use std::collections::BTreeMap; +use serde_json::json; use std::fs; -use std::os::unix::fs::PermissionsExt; -use std::os::unix::process::CommandExt; -use std::path::{Path, PathBuf}; -use std::process::{Child, Command, Stdio}; -use std::sync::{Arc, Mutex}; +use std::path::Path; +use std::process::{Command, Stdio}; +use std::sync::Arc; use std::time::{Duration, Instant}; - -const DAEMON: &str = "DEVGUARD_TEST_DAEMON"; +use support::*; #[test] #[ignore = "the service program of the installer tests"] fn daemon_child() { - let Some(base) = std::env::var_os(DAEMON) else { - return; - }; - if let Err(error) = serve_until_terminated(Path::new(&base)) { - eprintln!("fixture daemon failed: {error:?}"); - std::process::exit(1); - } -} - -fn harness_args() -> Vec { - [ - "--exact", - "daemon_child", - "--ignored", - "--nocapture", - "--test-threads=1", - ] - .map(String::from) - .to_vec() -} - -/// A private base directory. Installed releases are read-only, so their -/// directories are made writable again before the directory is removed. -struct Base(tempfile::TempDir); - -impl Base { - fn new() -> Self { - Self( - tempfile::Builder::new() - .prefix("dg-i-") - .tempdir_in("/private/tmp") - .unwrap(), - ) - } - - fn path(&self) -> &Path { - self.0.path() - } -} - -fn make_writable(path: &Path) { - let Ok(meta) = fs::symlink_metadata(path) else { - return; - }; - if meta.is_dir() && !meta.file_type().is_symlink() { - let _ = fs::set_permissions(path, fs::Permissions::from_mode(0o700)); - if let Ok(entries) = fs::read_dir(path) { - for entry in entries.flatten() { - make_writable(&entry.path()); - } - } - } -} - -impl Drop for Base { - fn drop(&mut self) { - make_writable(self.0.path()); - } -} - -/// Bootstrap an authority under `base`, then stop serving it. -fn initialized(base: &Path) -> AuthorityPaths { - drop(TestAuthority::start(base).unwrap()); - AuthorityPaths::fixture(base) -} - -fn copy_executable(from: &Path, to: &Path) { - fs::copy(from, to).unwrap(); - fs::set_permissions(to, fs::Permissions::from_mode(0o755)).unwrap(); -} - -fn manifest_for(dir: &Path, release_id: &str, created_at: &str) -> Manifest { - let artifacts = ARTIFACTS - .iter() - .map(|name| { - let bytes = fs::read(dir.join("bin").join(name)).unwrap(); - ( - name.to_string(), - Artifact { - sha256: digest_bytes(&bytes), - bytes: bytes.len() as u64, - }, - ) - }) - .collect(); - Manifest { - schema: MANIFEST_SCHEMA.into(), - release_id: release_id.into(), - version: install::compiled().package_version, - source: json!({"test": true}), - build: json!({"profile": "test"}), - artifacts, - compatibility: install::compiled(), - scope: "functional".into(), - slo_qualified: false, - created_at: created_at.into(), - } -} - -fn write_manifest(dir: &Path, manifest: &Manifest) { - fs::write( - dir.join("MANIFEST.json"), - serde_json::to_vec_pretty(manifest).unwrap(), - ) - .unwrap(); -} - -/// A package whose `devguardd` is this test binary. -fn package(parent: &Path, name: &str, release_id: &str, created_at: &str) -> PathBuf { - let dir = parent.join(name); - fs::create_dir_all(dir.join("bin")).unwrap(); - copy_executable( - &std::env::current_exe().unwrap(), - &dir.join("bin/devguardd"), - ); - copy_executable(Path::new("/usr/bin/true"), &dir.join("bin/devguard")); - copy_executable(Path::new("/usr/bin/true"), &dir.join("bin/devguard-launch")); - let manifest = manifest_for(&dir, release_id, created_at); - write_manifest(&dir, &manifest); - dir -} - -fn options(paths: &AuthorityPaths, base: &Path, label: &str) -> InstallOptions { - InstallOptions { - label: label.into(), - plist: paths.launch_agents().join(format!("{label}.plist")), - log: paths.logs().join("devguardd.log"), - program_args: harness_args(), - environment: BTreeMap::from([(DAEMON.to_string(), base.to_string_lossy().into_owned())]), - start_deadline: Duration::from_secs(20), - throttle_seconds: 1, - } -} - -/// Starts the job's program as launchd would: its own process group, the job's -/// environment added, and a report while it runs. -#[derive(Default)] -struct FakeLaunchd { - job: Mutex>, - bootstraps: Mutex, -} - -impl ServiceManager for FakeLaunchd { - fn bootstrap(&self, spec: &ServiceSpec) -> devguard_contract::Result<()> { - let mut job = self.job.lock().unwrap(); - if job.is_some() { - return Err(devguard_contract::Error::new( - ErrorCode::ResourceUnavailable, - "a job with this label is already loaded", - )); - } - *self.bootstraps.lock().unwrap() += 1; - let child = Command::new(&spec.program) - .args(&spec.args) - .envs(&spec.environment) - .process_group(0) - .stdin(Stdio::null()) - .stdout(Stdio::null()) - .stderr(Stdio::null()) - .spawn() - .unwrap(); - *job = Some((spec.label.clone(), child)); - Ok(()) - } - - fn bootout(&self, label: &str) -> devguard_contract::Result<()> { - let mut job = self.job.lock().unwrap(); - if job.as_ref().is_some_and(|(loaded, _)| loaded == label) { - let (_, mut child) = job.take().unwrap(); - // SAFETY: signalling the child this fake started. - unsafe { libc::kill(child.id() as i32, libc::SIGTERM) }; - let _ = child.wait(); - } - Ok(()) - } - - fn state(&self, label: &str) -> devguard_contract::Result> { - let mut job = self.job.lock().unwrap(); - Ok(match job.as_mut() { - Some((loaded, child)) if loaded == label => Some(match child.try_wait().unwrap() { - None => ServiceState { - running: true, - pid: Some(child.id()), - runs: Some(1), - last_exit: None, - }, - Some(status) => ServiceState { - running: false, - pid: None, - runs: Some(1), - last_exit: Some(format!("last exit code = {:?}", status.code())), - }, - }), - _ => None, - }) - } + support::daemon_child(); } -impl Drop for FakeLaunchd { - fn drop(&mut self) { - // Release the lock before booting out, which takes it again. - let label = self - .job - .lock() - .unwrap() - .as_ref() - .map(|(label, _)| label.clone()); - if let Some(label) = label { - let _ = ServiceManager::bootout(self, &label); - } - } -} - -fn wait_for(limit: Duration, mut done: impl FnMut() -> bool) { - let deadline = Instant::now() + limit; - while !done() { - assert!(Instant::now() < deadline, "condition not met in {limit:?}"); - std::thread::sleep(Duration::from_millis(100)); - } -} - -fn mode(path: &Path) -> u32 { - fs::symlink_metadata(path).unwrap().permissions().mode() & 0o777 -} - -/// Qualification runs collect raw receipts; ordinary runs write nothing. -fn record(name: &str, value: Value) { - if let Some(directory) = std::env::var_os("DEVGUARD_EVIDENCE_DIR") { - let directory = PathBuf::from(directory); - fs::create_dir_all(&directory).unwrap(); - fs::write( - directory.join(format!("{name}.json")), - serde_json::to_string_pretty(&value).unwrap(), - ) - .unwrap(); - } -} - -const LABEL: &str = "io.github.novelkr.devguard.test"; - #[test] fn an_installed_release_is_verified_running_before_it_is_selected() { let base = Base::new(); diff --git a/crates/daemon/tests/support/mod.rs b/crates/daemon/tests/support/mod.rs new file mode 100644 index 0000000..790ab84 --- /dev/null +++ b/crates/daemon/tests/support/mod.rs @@ -0,0 +1,309 @@ +//! Installed-service fixtures shared by the installer and upgrade tests. The +//! test binary is copied into each package as its `devguardd`, re-executed as +//! `daemon_child` to serve an isolated fixture authority, and started by a +//! fake manager as launchd would. + +#![allow(dead_code)] + +use devguard_contract::{digest_bytes, ErrorCode}; +use devguard_daemon::fixture::{ + serve_until_terminated, serve_without_upgrade_drain_until_terminated, TestAuthority, +}; +use devguard_daemon::install::{ + self, Artifact, InstallOptions, Manifest, ServiceManager, ServiceSpec, ServiceState, ARTIFACTS, + MANIFEST_SCHEMA, +}; +use devguard_daemon::paths::AuthorityPaths; +use serde_json::{json, Value}; +use std::collections::BTreeMap; +use std::fs; +use std::os::unix::fs::PermissionsExt; +use std::os::unix::process::CommandExt; +use std::path::{Path, PathBuf}; +use std::process::{Child, Command, Stdio}; +use std::sync::Mutex; +use std::time::{Duration, Instant}; + +/// Asks the service program to act as a release before C11. +pub const WITHOUT_UPGRADE_DRAIN: &str = "DEVGUARD_TEST_WITHOUT_UPGRADE_DRAIN"; + +pub const DAEMON: &str = "DEVGUARD_TEST_DAEMON"; + +/// The service program's role: serve the fixture authority under the base +/// named in the environment until SIGTERM. A release whose id ends in +/// `-nodrain` states no upgrade drain, as a release before C11; one whose id +/// ends in `-broken` exits at once, as a release that cannot start. +pub fn daemon_child() { + let Some(base) = std::env::var_os(DAEMON) else { + return; + }; + let program = std::env::current_exe().unwrap_or_default(); + let release = program + .parent() + .and_then(Path::parent) + .and_then(Path::file_name) + .map(|name| name.to_string_lossy().into_owned()) + .unwrap_or_default(); + if release.ends_with("-broken") { + std::process::exit(1); + } + let result = + if std::env::var_os(WITHOUT_UPGRADE_DRAIN).is_some() || release.ends_with("-nodrain") { + serve_without_upgrade_drain_until_terminated(Path::new(&base)) + } else { + serve_until_terminated(Path::new(&base)) + }; + if let Err(error) = result { + eprintln!("fixture daemon failed: {error:?}"); + std::process::exit(1); + } +} + +pub fn harness_args() -> Vec { + [ + "--exact", + "daemon_child", + "--ignored", + "--nocapture", + "--test-threads=1", + ] + .map(String::from) + .to_vec() +} + +/// A private base directory. Installed releases are read-only, so their +/// directories are made writable again before the directory is removed. +pub struct Base(pub tempfile::TempDir); + +impl Base { + pub fn new() -> Self { + Self( + tempfile::Builder::new() + .prefix("dg-i-") + .tempdir_in("/private/tmp") + .unwrap(), + ) + } + + pub fn path(&self) -> &Path { + self.0.path() + } +} + +pub fn make_writable(path: &Path) { + let Ok(meta) = fs::symlink_metadata(path) else { + return; + }; + if meta.is_dir() && !meta.file_type().is_symlink() { + let _ = fs::set_permissions(path, fs::Permissions::from_mode(0o700)); + if let Ok(entries) = fs::read_dir(path) { + for entry in entries.flatten() { + make_writable(&entry.path()); + } + } + } +} + +impl Drop for Base { + fn drop(&mut self) { + make_writable(self.0.path()); + } +} + +/// Bootstrap an authority under `base`, then stop serving it. +pub fn initialized(base: &Path) -> AuthorityPaths { + drop(TestAuthority::start(base).unwrap()); + AuthorityPaths::fixture(base) +} + +pub fn copy_executable(from: &Path, to: &Path) { + fs::copy(from, to).unwrap(); + fs::set_permissions(to, fs::Permissions::from_mode(0o755)).unwrap(); +} + +pub fn manifest_for(dir: &Path, release_id: &str, created_at: &str) -> Manifest { + let artifacts = ARTIFACTS + .iter() + .map(|name| { + let bytes = fs::read(dir.join("bin").join(name)).unwrap(); + ( + name.to_string(), + Artifact { + sha256: digest_bytes(&bytes), + bytes: bytes.len() as u64, + }, + ) + }) + .collect(); + Manifest { + schema: MANIFEST_SCHEMA.into(), + release_id: release_id.into(), + version: install::compiled().package_version, + source: json!({"test": true}), + build: json!({"profile": "test"}), + artifacts, + compatibility: install::compiled(), + scope: "functional".into(), + slo_qualified: false, + created_at: created_at.into(), + } +} + +pub fn write_manifest(dir: &Path, manifest: &Manifest) { + fs::write( + dir.join("MANIFEST.json"), + serde_json::to_vec_pretty(manifest).unwrap(), + ) + .unwrap(); +} + +/// A package whose `devguardd` is this test binary. +pub fn package(parent: &Path, name: &str, release_id: &str, created_at: &str) -> PathBuf { + package_with( + parent, + name, + release_id, + created_at, + Path::new("/usr/bin/true"), + ) +} + +/// A package whose `devguardd` is this test binary and whose `devguard` is +/// `devguard`; an upgrade or a repair runs from a release's own `devguard`. +pub fn package_with( + parent: &Path, + name: &str, + release_id: &str, + created_at: &str, + devguard: &Path, +) -> PathBuf { + let dir = parent.join(name); + fs::create_dir_all(dir.join("bin")).unwrap(); + copy_executable( + &std::env::current_exe().unwrap(), + &dir.join("bin/devguardd"), + ); + copy_executable(devguard, &dir.join("bin/devguard")); + copy_executable(Path::new("/usr/bin/true"), &dir.join("bin/devguard-launch")); + let manifest = manifest_for(&dir, release_id, created_at); + write_manifest(&dir, &manifest); + dir +} + +pub fn options(paths: &AuthorityPaths, base: &Path, label: &str) -> InstallOptions { + InstallOptions { + label: label.into(), + plist: paths.launch_agents().join(format!("{label}.plist")), + log: paths.logs().join("devguardd.log"), + program_args: harness_args(), + environment: BTreeMap::from([(DAEMON.to_string(), base.to_string_lossy().into_owned())]), + start_deadline: Duration::from_secs(20), + throttle_seconds: 1, + } +} + +/// Starts the job's program as launchd would: its own process group, the job's +/// environment added, and a report while it runs. +#[derive(Default)] +pub struct FakeLaunchd { + pub job: Mutex>, + pub bootstraps: Mutex, +} + +impl ServiceManager for FakeLaunchd { + fn bootstrap(&self, spec: &ServiceSpec) -> devguard_contract::Result<()> { + let mut job = self.job.lock().unwrap(); + if job.is_some() { + return Err(devguard_contract::Error::new( + ErrorCode::ResourceUnavailable, + "a job with this label is already loaded", + )); + } + *self.bootstraps.lock().unwrap() += 1; + let child = Command::new(&spec.program) + .args(&spec.args) + .envs(&spec.environment) + .process_group(0) + .stdin(Stdio::null()) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .spawn() + .unwrap(); + *job = Some((spec.label.clone(), child)); + Ok(()) + } + + fn bootout(&self, label: &str) -> devguard_contract::Result<()> { + let mut job = self.job.lock().unwrap(); + if job.as_ref().is_some_and(|(loaded, _)| loaded == label) { + let (_, mut child) = job.take().unwrap(); + // SAFETY: signalling the child this fake started. + unsafe { libc::kill(child.id() as i32, libc::SIGTERM) }; + let _ = child.wait(); + } + Ok(()) + } + + fn state(&self, label: &str) -> devguard_contract::Result> { + let mut job = self.job.lock().unwrap(); + Ok(match job.as_mut() { + Some((loaded, child)) if loaded == label => Some(match child.try_wait().unwrap() { + None => ServiceState { + running: true, + pid: Some(child.id()), + runs: Some(1), + last_exit: None, + }, + Some(status) => ServiceState { + running: false, + pid: None, + runs: Some(1), + last_exit: Some(format!("last exit code = {:?}", status.code())), + }, + }), + _ => None, + }) + } +} + +impl Drop for FakeLaunchd { + fn drop(&mut self) { + // Release the lock before booting out, which takes it again. + let label = self + .job + .lock() + .unwrap() + .as_ref() + .map(|(label, _)| label.clone()); + if let Some(label) = label { + let _ = ServiceManager::bootout(self, &label); + } + } +} + +pub fn wait_for(limit: Duration, mut done: impl FnMut() -> bool) { + let deadline = Instant::now() + limit; + while !done() { + assert!(Instant::now() < deadline, "condition not met in {limit:?}"); + std::thread::sleep(Duration::from_millis(100)); + } +} + +pub fn mode(path: &Path) -> u32 { + fs::symlink_metadata(path).unwrap().permissions().mode() & 0o777 +} + +/// Qualification runs collect raw receipts; ordinary runs write nothing. +pub fn record(name: &str, value: Value) { + if let Some(directory) = std::env::var_os("DEVGUARD_EVIDENCE_DIR") { + let directory = PathBuf::from(directory); + fs::create_dir_all(&directory).unwrap(); + fs::write( + directory.join(format!("{name}.json")), + serde_json::to_string_pretty(&value).unwrap(), + ) + .unwrap(); + } +} + +pub const LABEL: &str = "io.github.novelkr.devguard.test"; diff --git a/crates/daemon/tests/upgrade.rs b/crates/daemon/tests/upgrade.rs new file mode 100644 index 0000000..394b0b9 --- /dev/null +++ b/crates/daemon/tests/upgrade.rs @@ -0,0 +1,537 @@ +#![cfg(target_os = "macos")] +//! DG1-C11: replacing and repairing the installed service without losing or +//! duplicating charged work. +//! +//! As in the installer tests, the test binary is copied into each package as +//! its `devguardd` and `devguard`, so an upgrade or a repair really runs from +//! the release it selects, and each service is that copy serving an isolated +//! fixture authority, started by a fake manager as launchd would. A release +//! whose id ends in `-nodrain` acts as one before C11; one that ends in +//! `-broken` cannot start. + +mod support; +use devguard_client::protocol::{AbandonReason, CallerCredential}; +use devguard_client::Client; +use devguard_contract::{ + AdmissionRequest, AttemptKey, AttemptPhase, AttemptRecord, Budget, Capability, Compatibility, + ErrorCode, ReleaseReason, ResourceIntent, ResourceLevels, Secret, +}; +use devguard_core::AuthorityStorage; +use devguard_daemon::config::HostConfig; +use devguard_daemon::install::{self, InstallOptions, Manifest, ServiceManager}; +use devguard_daemon::paths::{read_private, AuthorityPaths}; +use devguard_daemon::upgrade::{self, DrainMode, UpgradeOptions}; +use serde_json::json; +use std::fs; +use std::io::{Seek, SeekFrom, Write}; +use std::os::unix::fs::PermissionsExt; +use std::path::PathBuf; +use std::time::{Duration, Instant}; +use support::*; + +#[test] +#[ignore = "the service program of the upgrade tests"] +fn daemon_child() { + support::daemon_child(); +} + +const MIB: u64 = 1024 * 1024; + +/// An installed fixture service. The manager is dropped first, stopping the +/// service before the base directory is removed. +struct Installed { + manager: FakeLaunchd, + options: InstallOptions, + paths: AuthorityPaths, + packages: PathBuf, + _base: Base, +} + +impl Installed { + fn new(release_id: &str) -> Self { + let base = Base::new(); + let paths = initialized(base.path()); + let packages = base.path().join("packages"); + let package = package_with( + &packages, + "installed", + release_id, + "2026-09-25T00:00:00Z", + &std::env::current_exe().unwrap(), + ); + let manager = FakeLaunchd::default(); + let mut options = options(&paths, base.path(), LABEL); + options.start_deadline = Duration::from_secs(10); + install::install(&paths, &package, &manager, &options).unwrap(); + Self { + manager, + options, + paths, + packages, + _base: base, + } + } + + /// Stage a release from a new package of this test binary. + fn stage(&self, name: &str, release_id: &str) { + let package = package_with( + &self.packages, + name, + release_id, + "2026-09-25T01:00:00Z", + &std::env::current_exe().unwrap(), + ); + let staged = upgrade::stage(&self.paths, &package).unwrap(); + assert_eq!(staged.release_id, release_id); + } + + fn upgrade( + &self, + to: &str, + drain: Duration, + stopped: bool, + ) -> devguard_contract::Result { + upgrade::upgrade( + &self.paths, + to, + &self.manager, + &self.options, + &UpgradeOptions { + drain_timeout: drain, + stopped, + }, + ) + } + + fn pid(&self) -> u32 { + self.manager.state(LABEL).unwrap().unwrap().pid.unwrap() + } + + fn executable(&self) -> PathBuf { + let pid = self.pid(); + fs::canonicalize(devguard_macos::executable_path(pid).unwrap().unwrap()).unwrap() + } + + fn release(&self, id: &str) -> PathBuf { + fs::canonicalize(self.paths.releases().join(id).join("bin/devguardd")).unwrap() + } + + fn backups(&self) -> Vec { + fs::read_dir(self.paths.backups()) + .map(|entries| entries.flatten().map(|entry| entry.path()).collect()) + .unwrap_or_default() + } + + fn current(&self) -> String { + install::read_selection(&self.paths) + .unwrap() + .unwrap() + .current + } +} + +fn connect(paths: &AuthorityPaths, required: &[Capability]) -> devguard_contract::Result { + Client::connect( + &paths.socket(), + paths.uid(), + Compatibility { + minimum_protocol: 1, + maximum_protocol: 1, + required: required.iter().copied().collect(), + }, + ) +} + +fn consumer(paths: &AuthorityPaths) -> CallerCredential { + let config = HostConfig::load(paths).unwrap(); + let secret = Secret::new( + String::from_utf8(read_private(&paths.cli_credential(), paths.uid(), 64).unwrap()).unwrap(), + ) + .unwrap(); + CallerCredential::Consumer { + consumer_id: "dev-cli".into(), + generation: config.consumers["dev-cli"].generation.clone(), + secret, + } +} + +/// A new registered workload session; every frame has a short deadline. +fn workload(paths: &AuthorityPaths) -> Client { + let mut client = connect( + paths, + &[Capability::DurableAdmission, Capability::FencedLaunch], + ) + .unwrap(); + client.authenticate(consumer(paths)).unwrap(); + client.register("upgrade-owner".into()).unwrap(); + client +} + +fn administrator(paths: &AuthorityPaths) -> Client { + let secret = Secret::new( + String::from_utf8(read_private(&paths.admin_credential(), paths.uid(), 64).unwrap()) + .unwrap(), + ) + .unwrap(); + let mut client = connect(paths, &[Capability::UpgradeDrain]).unwrap(); + client + .authenticate(CallerCredential::Administrator { secret }) + .unwrap(); + client +} + +fn request(paths: &AuthorityPaths, id: &str) -> AdmissionRequest { + AdmissionRequest { + key: AttemptKey { + consumer_id: "dev-cli".into(), + consumer_generation: HostConfig::load(paths).unwrap().consumers["dev-cli"] + .generation + .clone(), + attempt_id: id.into(), + }, + execution_digest: devguard_contract::digest_bytes(b"upgrade"), + intent: ResourceIntent { + profile: "interactive".into(), + requested: Budget { + cpu_milli: 50, + memory_bytes: 16 * MIB, + tasks: 2, + }, + minimum: ResourceLevels::MACOS, + }, + } +} + +/// Admit `id`, waiting while a new service's pressure is still closed. +fn admitted(paths: &AuthorityPaths, id: &str) -> AttemptRecord { + let deadline = Instant::now() + Duration::from_secs(20); + let mut attempt = 0; + loop { + attempt += 1; + let key = format!("{id}-{attempt}"); + let result = workload(paths).admit(request(paths, &key)); + match result { + Ok(record) if record.phase == AttemptPhase::Prepared => return record, + other => assert!(Instant::now() < deadline, "{other:?}"), + } + std::thread::sleep(Duration::from_millis(250)); + } +} + +/// A committed launch whose grant no helper claims: charged until its owner +/// reports that no helper exists. +fn committed(paths: &AuthorityPaths, id: &str) -> AttemptRecord { + let record = admitted(paths, id); + let grant = workload(paths).begin_launch(record.key.clone()).unwrap(); + assert!(grant.permit.is_some()); + grant.attempt +} + +fn settle(paths: &AuthorityPaths, record: &AttemptRecord) { + let settled = workload(paths) + .abandon_launch(record.key.clone(), AbandonReason::GrantNotReceived) + .unwrap(); + assert_eq!(settled.release_reason, Some(ReleaseReason::NoHelperCreated)); +} + +#[test] +fn an_upgrade_drains_backs_up_starts_the_release_closed_and_then_reopens_admission() { + let installed = Installed::new("0.1.0-test-a"); + let paths = &installed.paths; + // A tombstone and a Prepared attempt that the drain cancels. + let finished = admitted(paths, "finished"); + workload(paths).cancel(finished.key.clone()).unwrap(); + installed.stage("b", "0.1.0-test-b"); + let prepared = admitted(paths, "prepared"); + let before = installed.pid(); + let report = installed + .upgrade("0.1.0-test-b", Duration::from_secs(20), false) + .unwrap(); + assert_eq!( + (report.from.as_str(), report.to.as_str()), + ("0.1.0-test-a", "0.1.0-test-b") + ); + assert_eq!(report.drain.mode, DrainMode::ClosedAdmission); + assert!(report.drain.at_close.as_ref().unwrap().quiet()); + assert!(!report.compatibility.downgrade); + // The quiescent backup holds the journal, the selection and the manifest. + let files: Vec<&str> = report.backup.files.keys().map(String::as_str).collect(); + assert_eq!( + files, + ["MANIFEST.json", "authority.sqlite", "selection.json"] + ); + let mut backed_up = + AuthorityStorage::open(&report.backup.directory.join("authority.sqlite")).unwrap(); + let (attempts, leases) = backed_up.charged().unwrap(); + assert!(attempts.is_empty() && leases.is_empty()); + // The new release started closed and idle, and serves now. + assert!(report.before_reopening.closure.is_some()); + assert!(report.before_reopening.quiet()); + assert_ne!(installed.pid(), before); + assert_eq!(installed.executable(), installed.release("0.1.0-test-b")); + assert_eq!(report.selection.current, "0.1.0-test-b"); + assert_eq!( + report.selection.last_known_good.as_deref(), + Some("0.1.0-test-b") + ); + assert!(report.recovery.join("bin/devguardd").is_file()); + assert!(!paths.admission_marker().exists()); + // Tombstones survive. The Prepared attempt was cancelled by the drain, or + // expired first; either way it is known not to have started. + assert_eq!( + workload(paths).lookup(finished.key.clone()).unwrap().phase, + AttemptPhase::Cancelled + ); + let settled = workload(paths).lookup(prepared.key.clone()).unwrap(); + assert!( + matches!( + settled.phase, + AttemptPhase::Cancelled | AttemptPhase::Expired + ), + "{settled:?}" + ); + assert!(settled.known_not_started()); + admitted(paths, "after"); + // Old and new clients are served alike. + let plain = connect(paths, &[]).unwrap(); + assert!(!plain.hello.capabilities.contains(&Capability::UpgradeDrain)); + assert!(connect(paths, &[Capability::UpgradeDrain, Capability::ParentLease]).is_ok()); + let status = install::status(paths, &installed.manager, &installed.options).unwrap(); + assert!(status.healthy, "{status:?}"); + record( + "upgrade-normal", + json!({"report": report, "status": status}), + ); +} + +#[test] +fn a_drain_that_does_not_finish_in_time_keeps_the_current_release_and_its_charges() { + let installed = Installed::new("0.1.0-test-a"); + let paths = &installed.paths; + let running = committed(paths, "running"); + installed.stage("b", "0.1.0-test-b"); + let before = installed.pid(); + let started = Instant::now(); + let error = installed + .upgrade("0.1.0-test-b", Duration::from_secs(1), false) + .unwrap_err(); + assert_eq!(error.code, ErrorCode::ResourceUnavailable, "{error:?}"); + assert!(error.message.contains("did not finish"), "{error:?}"); + assert!(started.elapsed() < Duration::from_secs(10)); + // Nothing was replaced or backed up; the charge is kept and admission reopened. + assert_eq!(installed.pid(), before); + assert_eq!(installed.current(), "0.1.0-test-a"); + assert!(installed.backups().is_empty()); + assert!(!paths.admission_marker().exists()); + let quiescence = administrator(paths).quiescence().unwrap(); + assert!(quiescence.closure.is_none()); + assert_eq!(quiescence.attempts.len(), 1); + assert_eq!(quiescence.attempts[0].key, running.key); + let during = admitted(paths, "during"); + workload(paths).cancel(during.key).unwrap(); + // Once the work is settled, the same upgrade proceeds. + settle(paths, &running); + let report = installed + .upgrade("0.1.0-test-b", Duration::from_secs(20), false) + .unwrap(); + assert_eq!(report.selection.current, "0.1.0-test-b"); + record( + "drain-timeout", + json!({"error": error, "kept": quiescence, "retried": report}), + ); +} + +#[test] +fn a_release_that_cannot_be_verified_gives_way_to_the_previous_one_on_the_same_journal() { + let installed = Installed::new("0.1.0-test-a"); + let paths = &installed.paths; + let finished = admitted(paths, "finished"); + workload(paths).cancel(finished.key.clone()).unwrap(); + installed.stage("c", "0.1.0-test-c-broken"); + let error = installed + .upgrade("0.1.0-test-c-broken", Duration::from_secs(20), false) + .unwrap_err(); + assert!(error.message.contains("serves again"), "{error:?}"); + // The previous release serves the same journal with admission open. + assert_eq!(installed.executable(), installed.release("0.1.0-test-a")); + assert_eq!(installed.current(), "0.1.0-test-a"); + assert!(!paths.admission_marker().exists()); + assert_eq!( + workload(paths).lookup(finished.key.clone()).unwrap().phase, + AttemptPhase::Cancelled + ); + admitted(paths, "after"); + // The backup was taken before the switch; it is kept, never restored. + assert_eq!(installed.backups().len(), 1); + let selection = install::read_selection(paths).unwrap().unwrap(); + assert!(selection + .history + .iter() + .all(|event| event.release_id != "0.1.0-test-c-broken")); + record( + "upgrade-unverified", + json!({"error": error, "selection": selection}), + ); +} + +#[test] +fn an_incompatible_downgrade_is_refused_before_anything_changes() { + let installed = Installed::new("0.1.0-test-a"); + let paths = &installed.paths; + // The current release reads a newer journal schema than the staged one. + let future = package_with( + &installed.packages, + "future", + "0.1.0-test-future", + "2026-09-25T02:00:00Z", + &std::env::current_exe().unwrap(), + ); + let mut manifest: Manifest = + serde_json::from_slice(&fs::read(future.join("MANIFEST.json")).unwrap()).unwrap(); + manifest.compatibility.journal_schema = "2".into(); + write_manifest(&future, &manifest); + let release = paths.releases().join("0.1.0-test-future"); + fs::create_dir_all(release.join("bin")).unwrap(); + for name in ["devguardd", "devguard", "devguard-launch"] { + copy_executable( + &future.join("bin").join(name), + &release.join("bin").join(name), + ); + } + fs::copy(future.join("MANIFEST.json"), release.join("MANIFEST.json")).unwrap(); + let mut selection = install::read_selection(paths).unwrap().unwrap(); + selection.current = "0.1.0-test-future".into(); + fs::write(paths.selection(), serde_json::to_vec(&selection).unwrap()).unwrap(); + installed.stage("b", "0.1.0-test-b"); + let before = installed.pid(); + let error = installed + .upgrade("0.1.0-test-b", Duration::from_secs(20), false) + .unwrap_err(); + assert_eq!( + error.code, + ErrorCode::ResourcePolicyUnsupported, + "{error:?}" + ); + assert!(error.message.contains("journal schema"), "{error:?}"); + assert_eq!(installed.pid(), before); + assert!(installed.backups().is_empty()); + assert!(!paths.admission_marker().exists()); + admitted(paths, "after"); + record("downgrade-refused", json!({"error": error})); +} + +#[test] +fn a_release_that_cannot_drain_is_replaced_only_when_stopped_and_nothing_is_charged() { + let installed = Installed::new("0.1.0-test-a-nodrain"); + let paths = &installed.paths; + assert_eq!( + connect(paths, &[Capability::UpgradeDrain]) + .err() + .unwrap() + .code, + ErrorCode::ResourcePolicyUnsupported + ); + let running = committed(paths, "running"); + installed.stage("b", "0.1.0-test-b"); + let before = installed.pid(); + let refused = installed + .upgrade("0.1.0-test-b", Duration::from_secs(20), false) + .unwrap_err(); + assert_eq!(refused.code, ErrorCode::ResourcePolicyUnsupported); + assert!(refused.message.contains("--stopped"), "{refused:?}"); + assert_eq!(installed.pid(), before); + // Stopped, it finds work still charged and starts the release again. + let charged = installed + .upgrade("0.1.0-test-b", Duration::from_secs(20), true) + .unwrap_err(); + assert_eq!(charged.code, ErrorCode::ResourceUnavailable, "{charged:?}"); + assert!(charged.message.contains("still charged"), "{charged:?}"); + assert!(charged.message.contains("serves again"), "{charged:?}"); + assert_eq!( + installed.executable(), + installed.release("0.1.0-test-a-nodrain") + ); + assert_eq!(installed.current(), "0.1.0-test-a-nodrain"); + assert!(installed.backups().is_empty()); + // The restarted service kept the charge; once settled, the upgrade proceeds. + settle(paths, &running); + let report = installed + .upgrade("0.1.0-test-b", Duration::from_secs(20), true) + .unwrap(); + assert_eq!(report.drain.mode, DrainMode::Stopped); + assert!(report.compatibility.dropped.is_empty()); + assert_eq!(installed.executable(), installed.release("0.1.0-test-b")); + assert!(connect(paths, &[Capability::UpgradeDrain]).is_ok()); + assert!(!paths.admission_marker().exists()); + admitted(paths, "after"); + record( + "upgrade-stopped", + json!({"refused": refused, "charged": charged, "report": report}), + ); +} + +#[test] +fn repair_never_starts_a_second_authority_and_uses_the_recovery_copy_of_a_damaged_release() { + let installed = Installed::new("0.1.0-test-a"); + let paths = &installed.paths; + let serving = upgrade::repair(paths, &installed.manager, &installed.options).unwrap_err(); + assert_eq!(serving.code, ErrorCode::ResourceUnavailable, "{serving:?}"); + installed.manager.bootout(LABEL).unwrap(); + // Damage the installed release. + let release = paths.releases().join("0.1.0-test-a"); + for directory in [release.clone(), release.join("bin")] { + fs::set_permissions(&directory, fs::Permissions::from_mode(0o700)).unwrap(); + } + let launcher = release.join("bin/devguard-launch"); + fs::set_permissions(&launcher, fs::Permissions::from_mode(0o700)).unwrap(); + fs::OpenOptions::new() + .append(true) + .open(&launcher) + .unwrap() + .write_all(b"damage") + .unwrap(); + let report = upgrade::repair(paths, &installed.manager, &installed.options).unwrap(); + assert!(report.from_recovery); + assert!(report.damaged.is_some()); + assert_eq!( + installed.executable(), + fs::canonicalize(paths.recovery().join("0.1.0-test-a/bin/devguardd")).unwrap() + ); + assert!(report + .selection + .history + .last() + .unwrap() + .event + .contains("recovery copy")); + admitted(paths, "after"); + let status = install::status(paths, &installed.manager, &installed.options).unwrap(); + assert!(status.healthy, "{status:?}"); + record( + "repair-recovery", + json!({"refused_while_serving": serving, "report": report, "status": status}), + ); +} + +#[test] +fn repair_keeps_a_journal_that_cannot_be_opened_closed() { + let installed = Installed::new("0.1.0-test-a"); + let paths = &installed.paths; + installed.manager.bootout(LABEL).unwrap(); + let mut journal = fs::OpenOptions::new() + .write(true) + .open(paths.journal()) + .unwrap(); + journal.seek(SeekFrom::Start(0)).unwrap(); + journal.write_all(&[0xa5; 128]).unwrap(); + drop(journal); + let error = upgrade::repair(paths, &installed.manager, &installed.options).unwrap_err(); + assert!( + error.message.contains("journal cannot be opened"), + "{error:?}" + ); + // Nothing was started; the installation's own start is the only one. + assert!(installed.manager.state(LABEL).unwrap().is_none()); + assert_eq!(*installed.manager.bootstraps.lock().unwrap(), 1); + record("repair-corrupt-journal", json!({"error": error})); +} diff --git a/docs/contracts.md b/docs/contracts.md index 3741619..63dd45e 100644 --- a/docs/contracts.md +++ b/docs/contracts.md @@ -1,6 +1,6 @@ # Implemented authority contract -This document describes the implemented DG-0 authority, C01/C02 service, storage and transport behavior, C03 native macOS host evidence, C04 cooperative policy and scope evidence, the C05 fenced launch helper, C06 reconciliation, the C07 command-line owner, the C08 Cargo adapters, C09 installation as the current user's LaunchAgent and C10 parent leases with candidate authorities. PR and post-merge main delivery evidence is tracked separately from implementation. [Operations](operations.md) lists actual command availability. With native host evidence the service opens registration, launch and reconciliation; CodeSpace integration remains unimplemented. [Korean translation](ko/contracts.md). +This document describes the implemented DG-0 authority, C01/C02 service, storage and transport behavior, C03 native macOS host evidence, C04 cooperative policy and scope evidence, the C05 fenced launch helper, C06 reconciliation, the C07 command-line owner, the C08 Cargo adapters, C09 installation as the current user's LaunchAgent, C10 parent leases with candidate authorities and C11 upgrade and repair. PR and post-merge main delivery evidence is tracked separately from implementation. [Operations](operations.md) lists actual command availability. With native host evidence the service opens registration, launch and reconciliation; CodeSpace integration remains unimplemented. [Korean translation](ko/contracts.md). ## Authority and transport boundaries @@ -186,7 +186,7 @@ DG1-C09 installs a packaged release as the current user's LaunchAgent. The servi It exits 0 only when the service runs the verified current release. - **Restart.** A restarted service reopens and reconciles the existing journal, as any start does. A missing or corrupt journal fails closed and stays down. -- **Limits.** Replacing the installed release is an upgrade (C11); C09 refuses it. There is no uninstall command. `launchctl bootout gui//io.github.novelkr.devguard` and removing the plist stop the service and keep every release, recovery copy, the selection and the journal. +- **Limits.** Replacing the installed release is an upgrade (C11, below); installation refuses it. There is no uninstall command. `launchctl bootout gui//io.github.novelkr.devguard` and removing the plist stop the service and keep every release, recovery copy, the selection and the journal. ## Parent leases and candidate authorities @@ -223,6 +223,31 @@ DG1-C10 lets the stable authority lend one bounded budget, a parent lease, to th A failure ends the lease first. Children that are running finish, and the lease is released once they are settled. If the command itself is killed, its lease ends by the owner rule. - **Limits.** Workloads that create their own process groups or sessions would leave a lease child's scope and stay Suspect under the cooperative macOS model. The native launch, CLI, terminal and scope suites therefore remain bootstrap and CI qualification runs, not lease children. A candidate verifies admission only; its workloads never run through it. Real self-use under the installed parent needs a release that states parent leases. While host memory pressure keeps a service Critical, it admits no lease. +## Upgrade and repair + +DG1-C11 replaces and repairs the installed service without losing or duplicating charged work, and without depending on a candidate's admission. + +- **Staging.** `devguardd stage --package DIR` validates a package and copies it into an immutable `releases/`, as installation does. It must run from the package, and it leaves the service, the selection and the journal untouched. +- **Closing admission.** An administrator session can close admission (`CloseAdmission {reason}`), reopen it (`OpenAdmission`) and ask what is still charged (`Quiescence`). The service states `upgrade_drain` only to a client that requires it. + - Closing writes a private marker, `state/admission.json`, before it takes effect, so a restarted service or the next release starts with admission still closed. Every Prepared attempt is cancelled and is known not to have started. + - While admission is closed, admissions, launch commits, parent leases and lease children are refused with `ResourceUnavailable`, which a waiting CLI retries. Queries, cancellations, stops, owner reports and reconciliation continue. The status reports execution not ready, with the closure's reason. + - Closing again keeps the first closure; reopening removes the marker before it takes effect. +- **Upgrade.** `devguard upgrade --release ID [--drain-timeout DURATION] [--stopped]` must run from the staged release's own `devguard`. + 1. It refuses a release that speaks another wire version or protocol, reads another journal or configuration schema, or lacks durable admission or fenced launch, which consumers require. A release that states less than the current one is reported as a downgrade and allowed only within those limits. An incompatible downgrade is refused before anything changes. + 2. The current release must be running verified. The staged release's recovery copy is made before anything changes. + 3. It closes admission at the running service and waits until no attempt or lease is charged. If the drain does not finish within the timeout (60 s by default), admission reopens on the current release, which keeps every charge, and nothing is replaced. A release before C11 cannot close admission: `--stopped` stops it first and proceeds only if its journal then charges nothing; otherwise that release starts again. + 4. It stops the service. While holding the authority lock, it takes a quiescent backup under `backups/