Collect SSH logs directly into the support bundle

CHANGELOG.md

@@ -6,8 +6,9 @@
 
 - include a Coder support bundle for the selected workspace and agent only when using the environment's _Collect logs_
   action in Toolbox
-- include active and recent local CLI SSH logs in workspace log collection, using the configured or default CLI log
-  directory
+- include active and recent local CLI SSH logs in workspace log collection. For CLI 2.4.0 and newer the plugin now
+  passes an explicit `--log-dir` to the SSH ProxyCommand (the configured directory, `CODER_SSH_LOG_DIR`, or
+  `ssh-logs/` in the deployment's plugin data directory) and collects and cleans up logs from that same directory
 
 ### Changed
 

README.md

@@ -409,33 +409,40 @@ The plugin runs `coder support bundle` using the deployment's existing CLI login
 and its agent, when available. It places `coder-support.zip` inside the environment's diagnostic directory in the
 Toolbox log archive. Collection does not start a stopped workspace.
 
-The same diagnostic directory also includes recent local CLI SSH logs under `coder-ssh/`, even if support bundle
-generation fails. The plugin uses the configured SSH log directory, then `CODER_SSH_LOG_DIR`, or otherwise the CLI's
-default state directory: `$XDG_STATE_HOME/coder/logs` when set to an absolute path, falling back to
-`~/.local/state/coder/logs` on Linux, `~/Library/Application Support/coder/logs` on macOS, or
-`%LOCALAPPDATA%/coder/logs` on Windows.
-
-Only regular files directly inside the SSH log directory with these names are eligible:
-
-- `coder-ssh*.log`: CLI SSH session logs.
-- `coder-ssh*.log.<number>`: rotated CLI SSH logs, where `<number>` contains only digits, such as `.log.1`.
-- `<PID>.log`: legacy SSH logs with a numeric filename, such as `12345.log`.
-
-Collection skips subdirectories, symbolic links, and other filenames, including rotated legacy files such as
-`12345.log.1`. These local logs can include sessions for other workspaces that share the log directory.
+Recent local CLI SSH logs are added in place to `coder-support.zip` under `toolbox-logs/ssh/`. If the bundle could
+not be generated or is not a valid archive, the logs are written to a separate `coder-ssh.zip` in the diagnostic
+directory instead, using the same layout.
+
+For CLI versions **2.4.0 and newer**, the plugin passes `--log-dir <directory> -v` to the SSH ProxyCommand, enabling
+verbose file logging even on CLIs that do not log by default. The directory is the configured SSH log directory,
+then `CODER_SSH_LOG_DIR`, then `ssh-logs/` inside the deployment's plugin data directory. The plugin does not infer
+the CLI's default state directory, so logs written there by earlier setups are not collected. CLI versions older than
+2.4.0, or whose version is not recognized, receive neither logging flag and skip cleanup. Their SSH logs go wherever
+the CLI writes them by default, so collection only finds logs if you point the configured SSH log directory or
+`CODER_SSH_LOG_DIR` at that location.
+
+Only regular files directly inside the SSH log directory named `coder-ssh*-<pid>.log` are eligible, where `<pid>`
+contains only digits. Collection skips subdirectories, symbolic links, and other filenames, including bare PID names
+such as `12345.log` and rotated `.log.<number>` files. These local logs can include sessions for other workspaces
+that share the log directory.
 
 The following fixed limits apply and are not configurable:
 
 - **Age:** include files modified within the last **3 days (72 hours)**, based on the file's last-modified time.
   The identified active SSH session's log for the selected workspace/agent is included regardless of age.
-  Rotated logs remain subject to the age cutoff. If the active log cannot be identified, the cutoff applies to
-  every file.
+  If the active log cannot be identified, the cutoff applies to every file.
 - **Size:** copy only the last **50 MiB (52,428,800 bytes) per file**, including the active log. This is a limit on
   each file's uncompressed contents, not on the total archive size.
 
-These limits apply only to the collected copies; the original log files are unchanged. Missing or unreadable SSH
+These limits apply only to the archived copies; the original log files are unchanged. Missing or unreadable SSH
 logs do not prevent collecting the remaining diagnostics.
 
+On each connection attempt with a supported CLI, background cleanup keeps the newest **20** `coder-ssh*-<pid>.log`
+files by modification time and deletes other matching regular files only if they are older than **7 days**, so more
+than 20 files can remain when the extra ones are newer. Cleanup does not recurse or delete symlinks. Collection itself
+never deletes source logs.
+CLI 2.38+ also applies its own 50-file pruning in this directory.
+
 The bundle can contain deployment health and configuration, network diagnostics, workspace build logs, template
 source, and agent diagnostics. Available information depends on your permissions and workspace connectivity.
 Review the bundle before sharing, following

build.gradle.kts

@@ -38,6 +38,7 @@ dependencies {
     compileOnly(libs.bundles.serialization)
     compileOnly(libs.coroutines.core)
     implementation(libs.okhttp)
+    implementation(libs.zip4j)
     implementation(libs.moshi)
     ksp(libs.moshi.codegen)
     implementation(libs.retrofit)

gradle/libs.versions.toml

@@ -17,6 +17,7 @@ mockk = "1.14.11"
 detekt = "1.23.8"
 bouncycastle = "1.86"
 jackson = "2.22.3"
+zip4j = "2.11.5"
 
 [libraries]
 toolbox-core-api = { module = "com.jetbrains.toolbox:core-api", version.ref = "toolbox-plugin-api" }
@@ -37,6 +38,7 @@ mokk = { module = "io.mockk:mockk", version.ref = "mockk" }
 marketplace-client = { module = "org.jetbrains.intellij:plugin-repository-rest-client", version.ref = "marketplace-client" }
 bouncycastle-bcpg = { module = "org.bouncycastle:bcpg-jdk18on", version.ref = "bouncycastle" }
 bouncycastle-bcprov = { module = "org.bouncycastle:bcprov-jdk18on", version.ref = "bouncycastle" }
+zip4j = { module = "net.lingala.zip4j:zip4j", version.ref = "zip4j" }
 jackson-module-kotlin = { module = "com.fasterxml.jackson.module:jackson-module-kotlin", version.ref = "jackson" }
 
 [bundles]

src/main/kotlin/com/coder/toolbox/CoderRemoteEnvironment.kt

@@ -115,6 +115,7 @@ class CoderRemoteEnvironment(
     private val networkMetricsMarshaller = Moshi.Builder().build().adapter(NetworkMetrics::class.java)
     private val proxyCommandHandle = SshCommandProcessHandle(context)
     private var pollJob: Job? = null
+
     @Volatile
     private var sshPid: Long? = null
 
@@ -326,6 +327,7 @@ class CoderRemoteEnvironment(
 
     private fun pollNetworkMetrics(): Job =
         context.cs.launch(CoroutineName("Network Metrics Poller")) {
+            cli.cleanupSshLogs(currentSessionId())
             context.logger.info(currentSessionId(), "Starting the network metrics poll job for $id")
             while (isActive) {
                 val currentAgent = agent ?: break

src/main/kotlin/com/coder/toolbox/cli/CoderCLIManager.kt

@@ -12,6 +12,7 @@ import com.coder.toolbox.cli.gpg.GPGVerifier
 import com.coder.toolbox.cli.gpg.VerificationResult
 import com.coder.toolbox.cli.gpg.VerificationResult.Failed
 import com.coder.toolbox.cli.gpg.VerificationResult.Invalid
+import com.coder.toolbox.diagnostics.CoderSshLogCollector
 import com.coder.toolbox.sdk.CoderHttpClientBuilder
 import com.coder.toolbox.session.SessionId
 import com.coder.toolbox.settings.SignatureFallbackStrategy.ALLOW
@@ -26,6 +27,7 @@ import com.squareup.moshi.Json
 import com.squareup.moshi.JsonClass
 import com.squareup.moshi.JsonDataException
 import com.squareup.moshi.Moshi
+import kotlinx.coroutines.CancellationException
 import kotlinx.coroutines.Dispatchers
 import kotlinx.coroutines.runInterruptible
 import kotlinx.coroutines.withContext
@@ -101,6 +103,7 @@ suspend fun ensureCLI(
  * The supported features of the CLI.
  */
 data class Features(
+    val proxyLogDirectory: Boolean = false,
     val disableAutostart: Boolean = false,
     val reportWorkspaceUsage: Boolean = false,
     val wildcardSsh: Boolean = false,
@@ -132,9 +135,22 @@ class CoderCLIManager(
     /** Returns the effective SSH log directory for this CLI's current settings. */
     internal fun getSshLogDirectory(): Path = getSshLogDirectory(
         context.settingsStore.sshLogDirectory,
-        os = currentOs,
+        defaultDirectory = context.settingsStore.dataDir(deploymentURL).resolve("ssh-logs"),
     )
 
+    @Suppress("TooGenericExceptionCaught")
+    internal suspend fun cleanupSshLogs(sessionId: SessionId?) = withContext(Dispatchers.IO) {
+        try {
+            if (features.proxyLogDirectory) {
+                CoderSshLogCollector(context.logger, getSshLogDirectory()).cleanup(sessionId)
+            }
+        } catch (ex: CancellationException) {
+            throw ex
+        } catch (ex: Exception) {
+            context.logger.warn(sessionId, ex, "Could not prepare SSH log cleanup")
+        }
+    }
+
     private fun createDownloadService(): CoderDownloadService {
 
         val okHttpClient = CoderHttpClientBuilder.default(context)
@@ -341,6 +357,9 @@ class CoderCLIManager(
         sshConfigPath: String = context.settingsStore.sshConfigPath,
     ) {
         context.logger.info(sessionIds, "Configuring SSH config at $sshConfigPath")
+        if (workspaceAddresses.isNotEmpty() && feats.proxyLogDirectory) {
+            Files.createDirectories(getSshLogDirectory())
+        }
         writeSSHConfig(
             modifySSHConfig(workspaceAddresses, sessionIds, readSSHConfig(sshConfigPath), feats),
             sshConfigPath,
@@ -389,9 +408,9 @@ class CoderCLIManager(
             )
         val proxyArgs = buildList {
             addAll(baseArgs)
-            context.settingsStore.sshLogDirectory?.takeIf { it.isNotBlank() }?.let {
+            if (feats.proxyLogDirectory) {
                 add("--log-dir")
-                add(it)
+                add(getSshLogDirectory().toString())
                 add("-v")
             }
             if (feats.reportWorkspaceUsage) add("--usage-app=jetbrains")
@@ -530,10 +549,6 @@ class CoderCLIManager(
                 sshConfPath.parent?.toFile()?.mkdirs()
                 sshConfPath.toFile().writeText(contents)
             }
-            // The Coder cli will *not* create the log directory.
-            if (!context.settingsStore.sshLogDirectory.isNullOrBlank()) {
-                Path.of(context.settingsStore.sshLogDirectory).toFile().mkdirs()
-            }
         }
     }
 
@@ -661,6 +676,7 @@ class CoderCLIManager(
                 Features()
             } else {
                 Features(
+                    proxyLogDirectory = version >= SemVer(2, 4, 0),
                     disableAutostart = version >= SemVer(2, 5, 0),
                     reportWorkspaceUsage = version >= SemVer(2, 13, 0),
                     wildcardSsh = version >= SemVer(2, 19, 0),
@@ -686,44 +702,14 @@ class CoderCLIManager(
 
         internal fun supportsKeyringStorage(os: OS?): Boolean = os == OS.MAC || os == OS.WINDOWS
 
-        /** Resolves the log directory used by `coder ssh`, including its XDG state directory default. */
+        /** Resolve the same explicit directory for ProxyCommand logging, cleanup, and support bundles. */
         internal fun getSshLogDirectory(
             configuredDirectory: String?,
+            defaultDirectory: Path,
             env: Map<String, String> = System.getenv(),
-            os: OS? = getOS(),
-            home: Path = Path.of(
-                env[if (os == OS.WINDOWS) "USERPROFILE" else "HOME"].orEmpty().ifEmpty {
-                    System.getProperty("user.home")
-                },
-            ),
-        ): Path {
-            val directoryOverride = configuredDirectory?.takeIf { it.isNotBlank() }
-                ?: env["CODER_SSH_LOG_DIR"]?.takeIf { it.isNotEmpty() }
-            if (directoryOverride != null) return Path.of(directoryOverride)
-
-            // The CLI uses github.com/adrg/xdg: XDG_STATE_HOME must be absolute after home expansion.
-            val stateHome = absoluteStatePath(env["XDG_STATE_HOME"], home, os) ?: when (os) {
-                OS.MAC -> home.resolve("Library/Application Support")
-                OS.WINDOWS -> absoluteStatePath(env["LOCALAPPDATA"], home, os) ?: home.resolve("AppData/Local")
-                else -> home.resolve(".local/state")
-            }
-            return stateHome.resolve("coder/logs")
-        }
-
-        private fun absoluteStatePath(value: String?, home: Path, os: OS?): Path? {
-            if (value.isNullOrEmpty()) return null
-            val prefix = if (os == OS.WINDOWS) {
-                "%USERPROFILE%".takeIf { value.startsWith(it) }
-            } else {
-                listOf("~", "\$HOME").firstOrNull { value.startsWith(it) }
-            }
-            val path = if (prefix == null) {
-                Path.of(value)
-            } else {
-                home.resolve(value.removePrefix(prefix).trimStart('/', '\\'))
-            }
-            return path.takeIf { it.isAbsolute }
-        }
+        ): Path = (configuredDirectory?.takeIf { it.isNotBlank() }
+            ?: env["CODER_SSH_LOG_DIR"]?.takeIf { it.isNotBlank() })
+            ?.let { Path.of(it) } ?: defaultDirectory
     }
 
     private fun globalConfigArgs(): List<String> = listOf("--global-config", coderConfigPath.toString())