fix: a devpod up no longer outlives the dl that holds its lock

CHANGELOG.md

@@ -7,6 +7,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.59.3] - 2026-10-02
+
+### Fixed
+
+- **A `devpod up` no longer outlives the `dl` or `aid` that started it** (#668). Two `devpod up`
+  processes were found on a host reparented to init after their `aid --boot-up` parent died,
+  holding devpod's workspace lock for four hours, and every later `dl <ws>`, `rm` and
+  `devpod delete` waited on them with no end. On Linux each `devpod up` now takes a SIGKILL
+  from the kernel when its `dl` dies, however it dies (`PR_SET_PDEATHSIG`), and `aid`'s
+  background boot takes a SIGINT when `aid` dies, so it cancels as a Ctrl-C would. The
+  interrupt handler also gives the `devpod up` group two seconds to act on its SIGTERM and
+  then SIGKILLs it, where it used to send the SIGTERM and exit at once.
+- **`rm`, `rme` and `--rm` clear an orphan holding devpod's lock, as a launch does** (#668). A
+  delete blocked on the lock used to print advice to run `dl <ws> kill` in another
+  terminal and then wait. It now runs the launch's sweep: orphans holding the workspace are
+  signalled, a holder somebody is still waiting on is spared, and the line says what was
+  found. Both a launch and a delete also sweep again about once a minute while the wait
+  goes on, because a holder spared once can lose its parent later.
+
 ## [0.59.2] - 2026-10-01
 
 ### Fixed

README.md

@@ -19,7 +19,7 @@ one argument instead of a clone, a config file and a build command.
 [![GitHub pull-requests merged](https://badgen.net/github/merged-prs/blooop/devlaunch)](https://github.com/blooop/devlaunch/pulls?q=is%3Amerged)
 [![GitHub release](https://img.shields.io/github/release/blooop/devlaunch.svg)](https://GitHub.com/blooop/devlaunch/releases/)
 [![PyPI](https://img.shields.io/pypi/v/devlaunch)](https://pypi.org/project/devlaunch/)
-[![Conda](https://img.shields.io/badge/conda-v0.59.2-brightgreen?logo=anaconda)](https://prefix.dev/channels/blooop/packages/devlaunch)
+[![Conda](https://img.shields.io/badge/conda-v0.59.3-brightgreen?logo=anaconda)](https://prefix.dev/channels/blooop/packages/devlaunch)
 [![License](https://img.shields.io/github/license/blooop/devlaunch)](https://opensource.org/license/mit/)
 [![Platform](https://img.shields.io/badge/platform-linux--64-blue)](https://github.com/blooop/devlaunch/releases)
 [![Pixi Badge](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/prefix-dev/pixi/main/assets/badge/v0.json)](https://pixi.sh)
@@ -294,10 +294,11 @@ outlived SIGKILL, and it says which.
 
 `rm` is the happy path and keeps its guard and its `--force`: use it whenever the workspace
 might still be wanted, and `kill` when it is stuck and finished with. An `rm` that devpod cannot
-get the workspace's lock for now says so while it waits, and names the `kill` that clears it.
-So does a launch: `dl <ws>`, `up`, `restart`, `recreate`, `reset`, `code` and `dotfiles` all
-say the same thing while their `devpod up` sits behind the lock, and add that `kill` deletes
-the workspace, so the launch is typed again once it has.
+get the workspace's lock for now says so while it waits, then clears every holder that nothing
+is waiting on and carries on. So does a launch: `dl <ws>`, `up`, `restart`, `recreate`, `reset`,
+`code` and `dotfiles` all sweep the lock while their `devpod up` sits behind it. Only a holder
+somebody is still waiting on is left, and the line says what is left to do. See
+[docs/cli.md](docs/cli.md) for the details.
 
 [docs/cli.md](docs/cli.md) has the rest: what the delete asks of devpod, what stands it down,
 and why `kill` is the one command in dl with a deadline on it.
@@ -341,7 +342,7 @@ clone, and [docs/cleanup.md](docs/cleanup.md) says what it carries one past and
 
 ```bash
 $ dl --version
-dl 0.59.2
+dl 0.59.3
 ```
 
 `--devcontainer <variant|path>` picks a non-default `devcontainer.json`. A bare name means

docs/cli.md

@@ -388,10 +388,10 @@ removal (a signal handler may not allocate or lock, and this one `_exit`s):
 
 What all three *do* run is the cleanup the removal is not: the staged plaintext
 `GH_TOKEN` file is unlinked and the `devpod up` child is killed, so none of these three
-leaves a credential on disk or a build running behind you. The one exception is a run
-whose SIGTERM was disarmed before it started. The drain fells the build with a
-`killpg(…, SIGTERM)`, so disarming that signal disarms its own reach into the child too.
-Ctrl-\ (SIGQUIT) is not one of them and still does mean "die now and dump core", where
+leaves a credential on disk or a build running behind you. The drain sends the build's
+process group a SIGTERM, gives the `devpod up` up to two seconds to unwind, and then sends
+the group a SIGKILL. So a run whose SIGTERM was disarmed before it started, and whose child
+inherits that, still loses the build, two seconds later. Ctrl-\ (SIGQUIT) is not one of them and still does mean "die now and dump core", where
 tidying up first is not what it asks for. The workspace is what stays, still there
 under its name, and `dl <ws> rm` is how it goes.
 
@@ -930,9 +930,19 @@ waits for as long as whatever holds the lock lives. The usual holder is a `devpo
 up` that outlived the `dl` that started it: reparented to init, sleeping, no
 children, and nothing on the machine is ever going to reap it.
 
-dl watches for that line. An `rm` behind the lock says so while it waits and names
-the `kill` that clears it; the terminal it is printed in is busy holding the command
-the advice is about, so the advice names another one on purpose.
+On Linux dl makes that orphan hard to create. Each `devpod up` it starts is set to
+take a SIGKILL from the kernel when its `dl` dies (`PR_SET_PDEATHSIG`), so a `dl`
+that is SIGKILLed, or whose interrupt handler signals an `up` that does not stop,
+takes the `up` with it. `aid`'s background boot gets a SIGINT the same way, so an
+`aid` that dies cancels its boot as a Ctrl-C would. A holder started some other way,
+or on another host, can still wedge the workspace, and the sweep below is for that.
+
+dl watches for that line. **An `rm`, `rme` or `--rm` behind the lock does not wait
+for you either.** It says devpod is waiting, then runs the same sweep a launch runs,
+described next, and the delete goes on once the holder lets go. A holder that
+somebody is still waiting on is spared. When that is a live build, which `kill`
+spares too, the line says to stop it in its own terminal, and the delete goes on
+once it lets go.
 
 **A launch behind the lock does not wait for you.** It says devpod is waiting and
 that the wait has no deadline, and then it clears the lock itself: the same sweep
@@ -942,7 +952,10 @@ restarted and not abandoned. devpod's acquire polls behind that five second line
 the `up` that was blocked takes the freed flock itself and goes on to build, about
 a second later, measured. Every verb that brings a workspace up is covered, `dl
 <ws>` itself, `up`, `restart`, `recreate`, `reset`, `code` and `dotfiles`, because
-they all run the same `devpod up`.
+they all run the same `devpod up`. The sweep runs on devpod's first lock line and
+again once a minute for as long as the wait goes on, because a holder that a live
+`dl` was behind can lose that `dl` later. A repeat sweep prints a line only when it
+signalled something.
 
 Three things it will not do, and they are the reason a launch may do this at all.
 It never signals a holder somebody is waiting on: a `devpod up` with a live `dl`
@@ -1064,10 +1077,13 @@ on screen above it.
 A `dl <ws> rm` that devpod cannot get the lock for is the harder half, because it
 never refuses: devpod waits on that lock with no deadline, logging the five second
 line at the top of this section for as long as the holder lives, so there is no
-exit code for anything downstream to read. dl reads devpod's stderr as it arrives
-instead, and answers the first of those lines while the command is still blocked,
-naming the `dl <ws> kill` to run in another terminal. It says it once, however many
-times devpod says it.
+exit code for anything downstream to read. dl reads devpod's output as it arrives
+instead, and answers the first of those lines while the command is still blocked:
+it prints the launch's notice, the one that ends "Looking for what is holding
+it...", and runs the sweep described above. It sweeps again on every twelfth line
+after that, about once a minute, and prints a repeat sweep only when it signalled
+something. `dl <ws> kill` ends in this same delete, so a holder that arrives after
+its own sweep is swept here too.
 
 
 ## When devpod is missing or will not answer

rust/Cargo.toml

@@ -11,7 +11,7 @@ members = [
 # The single source of the version (docs/rust-rewrite-plan.md: cutover ships
 # 0.1.0, version read from Cargo.toml).
 [workspace.package]
-version = "0.59.2"
+version = "0.59.3"
 edition = "2024"
 license = "MIT"
 repository = "https://github.com/blooop/devlaunch"

rust/aid/src/interactive.rs

@@ -77,13 +77,18 @@
                 return None;
             }
         };
-        let spawned = Command::new(me)
+        let mut command = Command::new(me);
+        command
             .arg(BOOT_WORD)
             .args(boot_args)
             .stdin(Stdio::null())
             .stdout(Stdio::from(out))
-            .stderr(Stdio::from(err))
-            .spawn();
+            .stderr(Stdio::from(err));
+        // The boot must not outlive this aid. An aid that is SIGKILLed never gets
+        // to `cancel`, and its boot then goes on holding devpod's workspace lock
+        // with nobody left to wait on it.
+        dl::interrupted_with_this_process(&mut command);
+        let spawned = command.spawn();
         let Ok(child) = spawned else {
             // The fallback path must not litter: the log was created for a boot
             // that never started.

rust/aid/tests/interactive.rs

@@ -102,6 +102,29 @@ struct PtyAid {
 
 impl PtyAid {
     fn spawn(world: &World, args: &[&str], extra: &[(&str, &str)]) -> Self {
+        Self::spawn_behind(world, &[env!("CARGO_BIN_EXE_aid")], args, extra)
+    }
+
+    /// `aid` as a child of a shell that leads the pty's session and stays for a
+    /// minute after aid ends. Neither aid's death nor the shell's is then a
+    /// session leader's exit within that minute, so the kernel sends the
+    /// foreground group no SIGHUP, and only what aid itself arranged reaches its
+    /// children.
+    fn spawn_under_a_shell(world: &World, args: &[&str], extra: &[(&str, &str)]) -> Self {
+        Self::spawn_behind(
+            world,
+            &[
+                "sh",
+                "-c",
+                "\"$0\" \"$@\"; exec sleep 60",
+                env!("CARGO_BIN_EXE_aid"),
+            ],
+            args,
+            extra,
+        )
+    }
+
+    fn spawn_behind(world: &World, lead: &[&str], args: &[&str], extra: &[(&str, &str)]) -> Self {
         let pty = native_pty_system()
             .openpty(PtySize {
                 rows: 24,
@@ -111,7 +134,8 @@ impl PtyAid {
             })
             .expect("a pty");
         let root = world.root.display().to_string();
-        let mut command = CommandBuilder::new(env!("CARGO_BIN_EXE_aid"));
+        let mut command = CommandBuilder::new(lead[0]);
+        command.args(&lead[1..]);
         command.args(args);
         command.env_clear();
         // `KeepingCoverage` by hand: the trait extends `std::process::Command`,
@@ -761,6 +785,110 @@ fn a_ctrl_c_at_the_editor_tears_the_whole_boot_down() {
     );
 }
 
+/// An aid that is SIGKILLed at the editor takes its boot with it, and the boot
+/// still unlinks its staged token.
+///
+/// A SIGKILL runs no handler, so aid never reaches the `cancel` a Ctrl-C does.
+/// The kernel's parent-death signal is what reaches the boot instead, and it is a
+/// SIGINT so the boot's own handler runs: it kills the `devpod up` and unlinks the
+/// token file, as the Ctrl-C above has it do.
+#[cfg(target_os = "linux")]
+#[test]
+fn a_sigkilled_aid_takes_its_boot_down_and_the_token_with_it() {
+    let world = World::with(&["--gh"]);
+    let devpod = world.root.join("bin/devpod");
+    let original = std::fs::read_to_string(&devpod).expect("the scenario's devpod");
+    let delegate = original
+        .lines()
+        .find(|line| line.starts_with("exec "))
+        .expect("the delegate exec line");
+    let script = format!(
+        "#!/bin/sh\n\
+         if [ \"$1\" = \"up\" ]; then\n\
+         \x20 echo \"$$\" > \"$DL_UP_PID\"\n\
+         \x20 : > \"$DL_UP_STARTED\"\n\
+         \x20 exec sleep 120\n\
+         fi\n\
+         {delegate}\n"
+    );
+    std::fs::write(&devpod, script).expect("rewrite devpod");
+    use std::os::unix::fs::PermissionsExt as _;
+    std::fs::set_permissions(&devpod, std::fs::Permissions::from_mode(0o755))
+        .expect("keep devpod executable");
+    let tmpdir = world.root.join("tmp");
+    std::fs::create_dir_all(&tmpdir).expect("a scratch TMPDIR");
+    let up_pid = world.root.join("up.pid");
+    let up_started = world.root.join("up.started");
+
+    let session = PtyAid::spawn_under_a_shell(
+        &world,
+        &["blooop/devlaunch@cold"],
+        &[
+            ("TMPDIR", &tmpdir.display().to_string()),
+            ("DL_UP_PID", &up_pid.display().to_string()),
+            ("DL_UP_STARTED", &up_started.display().to_string()),
+        ],
+    );
+    session.reach_the_editor();
+    assert!(
+        wait_for(|| up_started.exists() && token_file(&tmpdir).is_some()),
+        "devpod up never blocked with a token staged"
+    );
+    let up = std::fs::read_to_string(&up_pid).expect("the up pid");
+    let up = up.trim().to_owned();
+    let only_child = |parent: &str| {
+        let children = Command::new("ps")
+            .args(["-o", "pid=", "--ppid", parent])
+            .output()
+            .expect("ps is installed");
+        String::from_utf8_lossy(&children.stdout)
+            .split_whitespace()
+            .next()
+            .expect("a child")
+            .to_owned()
+    };
+    let shell = session
+        .child
+        .process_id()
+        .expect("the shell's pid")
+        .to_string();
+    let aid = only_child(&shell);
+    let boot = only_child(&aid);
+    let alive = |pid: &str| {
+        Command::new("kill")
+            .args(["-0", pid])
+            .output()
+            .expect("kill is installed")
+            .status
+            .success()
+    };
+
+    assert!(
+        Command::new("kill")
+            .args(["-KILL", &aid])
+            .status()
+            .expect("kill is installed")
+            .success(),
+        "sending SIGKILL to aid"
+    );
+    let aid_gone = wait_for(|| !alive(&aid));
+
+    let boot_gone = wait_for(|| !alive(&boot));
+    let token_gone = wait_for(|| token_file(&tmpdir).is_none());
+    let up_gone = wait_for(|| !alive(&up));
+    for pid in [&boot, &up, &shell] {
+        let _ = Command::new("kill").args(["-KILL", pid]).output();
+    }
+    let _ = session.wait();
+    assert!(aid_gone, "aid (pid {aid}) outlived its SIGKILL");
+    assert!(
+        boot_gone,
+        "the boot (pid {boot}) outlived the SIGKILLed aid"
+    );
+    assert!(token_gone, "the boot left its token file behind");
+    assert!(up_gone, "the boot left its devpod up (pid {up}) behind");
+}
+
 /// The one staged GitHub-token file under `dir`, if any.
 fn token_file(dir: &Path) -> Option<PathBuf> {
     std::fs::read_dir(dir).ok()?.flatten().find_map(|entry| {

rust/devlaunch-core/public-api.api.txt

@@ -12,14 +12,14 @@ pub fn devlaunch_core::flows::launch::ColdRefused::fmt(&self, &mut core::fmt::Fo
 impl core::marker::StructuralPartialEq for devlaunch_core::flows::launch::ColdRefused
 pub enum devlaunch_core::api::DeleteStalled
 pub devlaunch_core::api::DeleteStalled::OnTheLock
+pub devlaunch_core::api::DeleteStalled::Swept(devlaunch_core::flows::kill::Released)
 impl core::clone::Clone for devlaunch_core::flows::lifecycle::DeleteStalled
 pub fn devlaunch_core::flows::lifecycle::DeleteStalled::clone(&self) -> devlaunch_core::flows::lifecycle::DeleteStalled
 impl core::cmp::Eq for devlaunch_core::flows::lifecycle::DeleteStalled
 impl core::cmp::PartialEq for devlaunch_core::flows::lifecycle::DeleteStalled
 pub fn devlaunch_core::flows::lifecycle::DeleteStalled::eq(&self, &devlaunch_core::flows::lifecycle::DeleteStalled) -> bool
 impl core::fmt::Debug for devlaunch_core::flows::lifecycle::DeleteStalled
 pub fn devlaunch_core::flows::lifecycle::DeleteStalled::fmt(&self, &mut core::fmt::Formatter<'_>) -> core::fmt::Result
-impl core::marker::Copy for devlaunch_core::flows::lifecycle::DeleteStalled
 impl core::marker::StructuralPartialEq for devlaunch_core::flows::lifecycle::DeleteStalled
 pub enum devlaunch_core::api::Insistence
 pub devlaunch_core::api::Insistence::Insisted
@@ -749,14 +749,14 @@ pub fn devlaunch_core::flows::launch::ToolProvisioning<'_>::remembered_claude(&s
 pub fn devlaunch_core::flows::launch::ToolProvisioning<'_>::stage_missing_for_this_launch(&self, &str) -> bool
 pub enum devlaunch_core::flows::lifecycle::DeleteStalled
 pub devlaunch_core::flows::lifecycle::DeleteStalled::OnTheLock
+pub devlaunch_core::flows::lifecycle::DeleteStalled::Swept(devlaunch_core::flows::kill::Released)
 impl core::clone::Clone for devlaunch_core::flows::lifecycle::DeleteStalled
 pub fn devlaunch_core::flows::lifecycle::DeleteStalled::clone(&self) -> devlaunch_core::flows::lifecycle::DeleteStalled
 impl core::cmp::Eq for devlaunch_core::flows::lifecycle::DeleteStalled
 impl core::cmp::PartialEq for devlaunch_core::flows::lifecycle::DeleteStalled
 pub fn devlaunch_core::flows::lifecycle::DeleteStalled::eq(&self, &devlaunch_core::flows::lifecycle::DeleteStalled) -> bool
 impl core::fmt::Debug for devlaunch_core::flows::lifecycle::DeleteStalled
 pub fn devlaunch_core::flows::lifecycle::DeleteStalled::fmt(&self, &mut core::fmt::Formatter<'_>) -> core::fmt::Result
-impl core::marker::Copy for devlaunch_core::flows::lifecycle::DeleteStalled
 impl core::marker::StructuralPartialEq for devlaunch_core::flows::lifecycle::DeleteStalled
 pub enum devlaunch_core::flows::lifecycle::Insistence
 pub devlaunch_core::flows::lifecycle::Insistence::Insisted