fix: remove the compose containers a forgotten workspace left running

CHANGELOG.md

@@ -7,6 +7,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Fixed
+
+- **`rm`, `kill` and a launch remove the containers a forgotten workspace left running.**
+  `devpod delete` finds a container only by the uid of the record it deletes, so a forced
+  delete whose `compose down` failed, or a record made again under a new uid, left the old
+  compose project up with nothing that owned it. Its fixed `container_name`s then failed every
+  later `up` of that workspace with `Conflict. The container name ... is already in use`.
+  Once devpod holds no record of the workspace, dl now removes every container whose compose
+  labels name devpod's directory for that id or the clone dl made, and names each one. A
+  workspace devpod still holds (or whose record dl cannot read), a local folder's own compose
+  projects, and any container that names another workspace's devpod directory or a folder
+  devpod holds as another workspace's source, are left alone.
+
 ## [0.62.0] - 2026-10-16
 
 ### Added

docs/cleanup.md

@@ -97,6 +97,47 @@ records died before any of this existed, no route reaches them that is not the
 pattern above, and they stay. See [the disk neither command
 frees](#the-disk-neither-command-frees).
 
+### Containers a forgotten workspace left running
+
+`devpod delete` finds a workspace's container by one label,
+`dev.containers.id=<uid>`, and runs `compose down` only when it finds it. Two
+ordinary events leave a compose project running after devpod has dropped its
+record:
+
+- A forced delete (`devpod delete --force`, which `dl <ws> kill` runs) whose
+  `compose down` failed. A compose file that no longer interpolates is enough, for
+  example a required variable from a `.env` that the `initializeCommand` wrote and
+  that is gone from the clone. devpod logs the error and drops the record anyway.
+- A record made again under a new uid. A `devpod up` over the leftovers makes a
+  new record, then fails because a compose file that pins `container_name:` cannot
+  make a second container with that name. The next `devpod delete` looks for the
+  new uid, finds no container, and reports success. The leftovers stay up.
+
+So `dl <ws> rm` (and `kill`), once devpod holds no record of the workspace in any
+context, removes every container whose compose labels
+(`com.docker.compose.project.working_dir`, `com.docker.compose.project.config_files`)
+name a path inside devpod's own directory for that workspace id
+(`<devpod home>/agent/contexts/<context>/workspaces/<id>`, where devpod writes its
+compose override) or inside the clone dl made for it. A launch does the same
+before its `devpod up` for a workspace devpod has no record of, so the new compose
+project does not trip over the old names. Each removal is named on stderr:
+
+```
+Removed containers <id> left behind: <name>, <name>
+```
+
+A workspace devpod still holds is left alone (a record dl cannot read counts as
+held), and so is a container that names
+only a local folder opened with `dl ./folder`: that folder is yours, and a compose
+project you started in it by hand is not a leftover. Names are never matched.
+
+When any doubt remains, the container stays. A container is left when one of
+its labels names devpod's directory for a *different* workspace id, or a path
+inside a folder that devpod holds as another workspace's source and that sits
+in the clone (a local folder opened with `dl ./folder` can sit inside a clone).
+When devpod's records cannot be read, a container that names only the clone
+stays too.
+
 ### What a delete says while it does it
 
 `dl <ws> rm` names the workspace going in and again once it has gone, both on

docs/development.md

@@ -69,7 +69,7 @@ once under the `api` section and once under the module that owns them.
 
 **What it still does not reach, and it is not one type.** A type `api` never re-exports but a
 promised signature hands back is reachable from outside and classified as binary surface. Counted
-on the checked-in files rather than guessed at, that is **41 types owning close to six hundred rows** in
+on the checked-in files rather than guessed at, that is **42 types owning close to six hundred rows** in
 `public-api.rest.txt`, and the command that lists them needs no toolchain:
 
 **A falling count is not automatically a win, and it is worth knowing which kind you are looking

rust/devlaunch-core/public-api.api.txt

@@ -93,6 +93,16 @@ pub devlaunch_core::api::LaunchNotice::HerdrTab(devlaunch_core::flows::launch::H
 pub devlaunch_core::api::LaunchNotice::LaunchLockUnavailable
 pub devlaunch_core::api::LaunchNotice::LaunchLockUnavailable::reason: alloc::string::String
 pub devlaunch_core::api::LaunchNotice::LaunchLockUnavailable::workspace_id: alloc::string::String
+pub devlaunch_core::api::LaunchNotice::LeftoverContainersNotListed
+pub devlaunch_core::api::LaunchNotice::LeftoverContainersNotListed::reason: alloc::string::String
+pub devlaunch_core::api::LaunchNotice::LeftoverContainersNotListed::workspace_id: alloc::string::String
+pub devlaunch_core::api::LaunchNotice::LeftoverContainersNotRemoved
+pub devlaunch_core::api::LaunchNotice::LeftoverContainersNotRemoved::names: devlaunch_core::domain::workspace_state::NonEmpty<alloc::string::String>
+pub devlaunch_core::api::LaunchNotice::LeftoverContainersNotRemoved::reason: alloc::string::String
+pub devlaunch_core::api::LaunchNotice::LeftoverContainersNotRemoved::workspace_id: alloc::string::String
+pub devlaunch_core::api::LaunchNotice::LeftoverContainersRemoved
+pub devlaunch_core::api::LaunchNotice::LeftoverContainersRemoved::names: devlaunch_core::domain::workspace_state::NonEmpty<alloc::string::String>
+pub devlaunch_core::api::LaunchNotice::LeftoverContainersRemoved::workspace_id: alloc::string::String
 pub devlaunch_core::api::LaunchNotice::Lifecycle(devlaunch_core::flows::lifecycle::LifecycleNotice)
 pub devlaunch_core::api::LaunchNotice::MountSourceEmpty
 pub devlaunch_core::api::LaunchNotice::MountSourceMissing
@@ -177,6 +187,16 @@ pub devlaunch_core::api::LifecycleNotice::CloneNotRemoved::refusal: devlaunch_co
 pub devlaunch_core::api::LifecycleNotice::CloneNotRemoved::workspace_id: alloc::string::String
 pub devlaunch_core::api::LifecycleNotice::CloneRemoved
 pub devlaunch_core::api::LifecycleNotice::CloneRemoved::workspace_id: alloc::string::String
+pub devlaunch_core::api::LifecycleNotice::LeftoverContainersNotListed
+pub devlaunch_core::api::LifecycleNotice::LeftoverContainersNotListed::reason: alloc::string::String
+pub devlaunch_core::api::LifecycleNotice::LeftoverContainersNotListed::workspace_id: alloc::string::String
+pub devlaunch_core::api::LifecycleNotice::LeftoverContainersNotRemoved
+pub devlaunch_core::api::LifecycleNotice::LeftoverContainersNotRemoved::names: devlaunch_core::domain::workspace_state::NonEmpty<alloc::string::String>
+pub devlaunch_core::api::LifecycleNotice::LeftoverContainersNotRemoved::reason: alloc::string::String
+pub devlaunch_core::api::LifecycleNotice::LeftoverContainersNotRemoved::workspace_id: alloc::string::String
+pub devlaunch_core::api::LifecycleNotice::LeftoverContainersRemoved
+pub devlaunch_core::api::LifecycleNotice::LeftoverContainersRemoved::names: devlaunch_core::domain::workspace_state::NonEmpty<alloc::string::String>
+pub devlaunch_core::api::LifecycleNotice::LeftoverContainersRemoved::workspace_id: alloc::string::String
 pub devlaunch_core::api::LifecycleNotice::RecordNotDropped
 pub devlaunch_core::api::LifecycleNotice::RecordNotDropped::path: std::path::PathBuf
 pub devlaunch_core::api::LifecycleNotice::RecordNotDropped::refusal: devlaunch_core::domain::metadata::MetadataError
@@ -677,6 +697,16 @@ pub devlaunch_core::flows::launch::LaunchNotice::HerdrTab(devlaunch_core::flows:
 pub devlaunch_core::flows::launch::LaunchNotice::LaunchLockUnavailable
 pub devlaunch_core::flows::launch::LaunchNotice::LaunchLockUnavailable::reason: alloc::string::String
 pub devlaunch_core::flows::launch::LaunchNotice::LaunchLockUnavailable::workspace_id: alloc::string::String
+pub devlaunch_core::flows::launch::LaunchNotice::LeftoverContainersNotListed
+pub devlaunch_core::flows::launch::LaunchNotice::LeftoverContainersNotListed::reason: alloc::string::String
+pub devlaunch_core::flows::launch::LaunchNotice::LeftoverContainersNotListed::workspace_id: alloc::string::String
+pub devlaunch_core::flows::launch::LaunchNotice::LeftoverContainersNotRemoved
+pub devlaunch_core::flows::launch::LaunchNotice::LeftoverContainersNotRemoved::names: devlaunch_core::domain::workspace_state::NonEmpty<alloc::string::String>
+pub devlaunch_core::flows::launch::LaunchNotice::LeftoverContainersNotRemoved::reason: alloc::string::String
+pub devlaunch_core::flows::launch::LaunchNotice::LeftoverContainersNotRemoved::workspace_id: alloc::string::String
+pub devlaunch_core::flows::launch::LaunchNotice::LeftoverContainersRemoved
+pub devlaunch_core::flows::launch::LaunchNotice::LeftoverContainersRemoved::names: devlaunch_core::domain::workspace_state::NonEmpty<alloc::string::String>
+pub devlaunch_core::flows::launch::LaunchNotice::LeftoverContainersRemoved::workspace_id: alloc::string::String
 pub devlaunch_core::flows::launch::LaunchNotice::Lifecycle(devlaunch_core::flows::lifecycle::LifecycleNotice)
 pub devlaunch_core::flows::launch::LaunchNotice::MountSourceEmpty
 pub devlaunch_core::flows::launch::LaunchNotice::MountSourceMissing
@@ -839,6 +869,16 @@ pub devlaunch_core::flows::lifecycle::LifecycleNotice::CloneNotRemoved::refusal:
 pub devlaunch_core::flows::lifecycle::LifecycleNotice::CloneNotRemoved::workspace_id: alloc::string::String
 pub devlaunch_core::flows::lifecycle::LifecycleNotice::CloneRemoved
 pub devlaunch_core::flows::lifecycle::LifecycleNotice::CloneRemoved::workspace_id: alloc::string::String
+pub devlaunch_core::flows::lifecycle::LifecycleNotice::LeftoverContainersNotListed
+pub devlaunch_core::flows::lifecycle::LifecycleNotice::LeftoverContainersNotListed::reason: alloc::string::String
+pub devlaunch_core::flows::lifecycle::LifecycleNotice::LeftoverContainersNotListed::workspace_id: alloc::string::String
+pub devlaunch_core::flows::lifecycle::LifecycleNotice::LeftoverContainersNotRemoved
+pub devlaunch_core::flows::lifecycle::LifecycleNotice::LeftoverContainersNotRemoved::names: devlaunch_core::domain::workspace_state::NonEmpty<alloc::string::String>
+pub devlaunch_core::flows::lifecycle::LifecycleNotice::LeftoverContainersNotRemoved::reason: alloc::string::String
+pub devlaunch_core::flows::lifecycle::LifecycleNotice::LeftoverContainersNotRemoved::workspace_id: alloc::string::String
+pub devlaunch_core::flows::lifecycle::LifecycleNotice::LeftoverContainersRemoved
+pub devlaunch_core::flows::lifecycle::LifecycleNotice::LeftoverContainersRemoved::names: devlaunch_core::domain::workspace_state::NonEmpty<alloc::string::String>
+pub devlaunch_core::flows::lifecycle::LifecycleNotice::LeftoverContainersRemoved::workspace_id: alloc::string::String
 pub devlaunch_core::flows::lifecycle::LifecycleNotice::RecordNotDropped
 pub devlaunch_core::flows::lifecycle::LifecycleNotice::RecordNotDropped::path: std::path::PathBuf
 pub devlaunch_core::flows::lifecycle::LifecycleNotice::RecordNotDropped::refusal: devlaunch_core::domain::metadata::MetadataError

rust/devlaunch-core/src/clients/devpod_home.rs

@@ -134,6 +134,116 @@
             .join("workspace.lock")
     }
 
+    /// devpod's agent-side directories, one per context, each holding one
+    /// directory per workspace.
+    fn agent_contexts(&self) -> PathBuf {
+        self.root.join("agent").join("contexts")
+    }
+
+    /// Whether any context still holds a record for this workspace.
+    ///
+    /// Every context rather than [`Self::sole_context_holding`]'s single answer,
+    /// because the caller asks the opposite question: whether devpod has *no*
+    /// workspace by this id anywhere, so that nothing on the machine can still
+    /// claim the containers that name it. Two contexts holding it is
+    /// [`RecordPresence::Held`].
+    ///
+    /// Only "not found", for the `contexts` directory or for every record path
+    /// under it, is [`RecordPresence::Absent`]. Anything else that would not
+    /// read (a directory dl may not search, an entry the listing could not
+    /// return) is [`RecordPresence::Unknown`]: a record may be there. A context
+    /// name that is not UTF-8 is read like any other, by its bytes.
+    pub(crate) fn record_presence(&self, workspace_id: &str) -> RecordPresence {
+        let contexts = match std::fs::read_dir(self.contexts()) {
+            Ok(contexts) => contexts,
+            Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
+                return RecordPresence::Absent;
+            }
+            Err(_) => return RecordPresence::Unknown,
+        };
+        let mut unknown = false;
+        for context in contexts {
+            let Ok(context) = context else {
+                unknown = true;
+                continue;
+            };
+            let record = context
+                .path()
+                .join("workspaces")
+                .join(workspace_id)
+                .join("workspace.json");
+            match std::fs::metadata(&record) {
+                Ok(_) => return RecordPresence::Held,
+                Err(error) if error.kind() == std::io::ErrorKind::NotFound => {}
+                Err(_) => unknown = true,
+            }
+        }
+        if unknown {
+            RecordPresence::Unknown
+        } else {
+            RecordPresence::Absent
+        }
+    }
+
+    /// The workspace id whose devpod agent-side directory `path` is inside,
+    /// under any context, or nothing when it is inside none.
+    ///
+    /// devpod writes its compose override for a workspace there
+    /// (`.docker-compose/docker-compose.devcontainer.containerFeatures-*.yml`), and
+    /// compose records that file in the `config_files` label of every container
+    /// the project makes. So the label names the workspace *by id*, whatever uid
+    /// devpod gave the project, which is the one handle a container keeps after
+    /// devpod has forgotten it.
+    pub(crate) fn agent_dir_owner<'p>(&self, path: &'p Path) -> Option<&'p std::ffi::OsStr> {
+        let rest = path.strip_prefix(self.agent_contexts()).ok()?;
+        let mut components = rest.components().skip(1);
+        if components.next()?.as_os_str() != "workspaces" {
+            return None;
+        }
+        match components.next()? {
+            std::path::Component::Normal(id) => Some(id),
+            _ => None,
+        }
+    }
+
+    /// The local folders of every workspace devpod holds a record for, or
+    /// nothing when any record, or the walk to it, would not read.
+    ///
+    /// A record whose source is not a local folder (a git source devpod cloned
+    /// itself) names no folder on this machine and is skipped. A record that is
+    /// there and would not read is not: it could name any folder, so the answer
+    /// is "unknown" rather than a list missing one.
+    pub(crate) fn held_local_folders(&self) -> Option<Vec<PathBuf>> {
+        let mut folders = Vec::new();
+        let Some(contexts) = read_dir_if_any(&self.contexts())? else {
+            return Some(folders);
+        };
+        for context in contexts {
+            let Some(workspaces) = read_dir_if_any(&context.ok()?.path().join("workspaces"))?
+            else {
+                continue;
+            };
+            for workspace in workspaces {
+                let record = workspace.ok()?.path().join("workspace.json");
+                let text = match std::fs::read_to_string(&record) {
+                    Ok(text) => text,
+                    Err(error) if error.kind() == std::io::ErrorKind::NotFound => continue,
+                    Err(_) => return None,
+                };
+                let record: serde_json::Value = serde_json::from_str(&text).ok()?;
+                if let Some(folder) = record
+                    .get("source")
+                    .and_then(|source| source.get("localFolder"))
+                    .and_then(serde_json::Value::as_str)
+                    .filter(|folder| !folder.is_empty())
+                {
+                    folders.push(PathBuf::from(folder));
+                }
+            }
+        }
+        Some(folders)
+    }
+
     /// devpod's record of a *completed* create for one workspace.
     ///
     /// devpod writes this beside [`Self::record`] on its way out of a successful
@@ -247,6 +357,30 @@
     }
 }
 
+/// Whether devpod holds a record for one workspace id, in any context.
+///
+/// Three arms, because "could not tell" must not read as "no": the one caller
+/// removes containers on [`Self::Absent`] alone.
+#[derive(Clone, Copy, Debug, PartialEq, Eq)]
+pub(crate) enum RecordPresence {
+    /// A record is there, in at least one context.
+    Held,
+    /// No context holds one, and every place one could be said so.
+    Absent,
+    /// Some place a record could be would not read.
+    Unknown,
+}
+
+/// A directory's entries; `Some(None)` when it is not there, and nothing when
+/// it is there and would not read.
+fn read_dir_if_any(dir: &Path) -> Option<Option<std::fs::ReadDir>> {
+    match std::fs::read_dir(dir) {
+        Ok(entries) => Some(Some(entries)),
+        Err(error) if error.kind() == std::io::ErrorKind::NotFound => Some(None),
+        Err(_) => None,
+    }
+}
+
 /// Whether devpod finished creating a workspace, as far as its own records say.
 ///
 /// Three arms rather than a bool, for the reason [`crate::domain::workspace_state`]
@@ -428,6 +562,36 @@
     ScratchHome { _dir: dir, home }
 }
 
+/// An empty record for one workspace under a context named by raw bytes, for
+/// the test that needs a context name that is not UTF-8.
+#[cfg(test)]
+pub(crate) fn record_under_context(
+    home: &DevpodHome,
+    context: &std::ffi::OsStr,
+    workspace_id: &str,
+) {
+    let dir = home
+        .contexts()
+        .join(context)
+        .join("workspaces")
+        .join(workspace_id);
+    std::fs::create_dir_all(&dir).expect("a record directory");
+    std::fs::write(dir.join("workspace.json"), "{}").expect("a record");
+}
+
+/// A file in devpod's agent-side directory for one workspace, where devpod
+/// writes its compose override. Not created: a container label names it whether
+/// or not it is still on disk.
+#[cfg(test)]
+pub(crate) fn agent_compose_file(home: &DevpodHome, context: &str, workspace_id: &str) -> PathBuf {
+    home.agent_contexts()
+        .join(context)
+        .join("workspaces")
+        .join(workspace_id)
+        .join(".docker-compose")
+        .join("docker-compose.devcontainer.containerFeatures-0.yml")
+}
+
 /// The `flock` devpod blocks on, created empty, for the one test that has to
 /// assert it is still standing afterwards.
 ///