[DEV-99] View devintern worker logs directly in the dashboard via a Logs tab

docs/code/dashboard.md

@@ -1,14 +1,14 @@
 ---
 title: "Observability Dashboard"
-description: "A local web dashboard for worker run history: per-task timelines, stage-by-stage outcomes, and aggregate stats"
+description: "A local web dashboard for worker run history: per-task timelines, stage-by-stage outcomes, aggregate stats, and worker logs"
 section: "Server Automation"
 order: 2
-dateModified: 2026-08-24
+dateModified: 2026-08-27
 ---
 
 # Observability Dashboard
 
-`devintern dashboard` serves a local web dashboard over the worker's run history: every task, PR mention, and scheduled automation the worker handled, the stages each run went through (feasibility, implementation, self-review, change requests, outcome), and aggregate stats like success rate and runs per week.
+`devintern dashboard` serves a local web dashboard over the worker's run history: every task, PR mention, and scheduled automation the worker handled, the stages each run went through (feasibility, implementation, self-review, change requests, outcome), aggregate stats like success rate and runs per week, and the worker's own log output.
 
 All data is read from the worker's local database (`.devintern-code/queue.db`). Nothing is uploaded anywhere: the dashboard runs on your machine and binds to localhost by default.
 
@@ -31,10 +31,22 @@ The standalone command reads the database in read-only mode, so it is safe to ru
 - **Run list**: every run with its status, task key or automation id, origin (tracker task, PR mention, or scheduled), agent harness, PR link, and duration. Filter by status or origin (`origin=scheduled` isolates automation runs).
 - **Run detail**: a stage-by-stage timeline for one run: the feasibility verdict, the implementation summary, each self-review iteration, each human change request and how it was handled, and the final outcome.
 - **Stats**: runs per week, success and escalation rates, median run duration, and a per-harness breakdown over a selectable window (7, 30, or 90 days, or all time).
+- **Logs**: the most recent worker log lines (timestamp, severity, message), filterable by level (`everything` / `warnings` / `errors`) with a search box over the loaded window. Lines that mention a task key link straight to that task's latest run in the Runs view.
 - **Worker status**: whether the daemon is running, queued and failed events, open agent PRs, and per-source poll cursors.
 
 Success and escalation rates are computed over finished runs only. Run duration is measured from pickup to PR creation and is a proxy for ticket-to-PR time. Merge rate is not shown yet: the worker records PRs as open or closed but does not track merges separately.
 
+## Where the logs come from
+
+The worker daemon logs to stdout/stderr. When it runs as a background service, that output is captured: the macOS launchd agent written by `worker init` redirects it into `worker.stdout.log` and `worker.stderr.log` in the workspace directory (on Linux systemd, output goes to the journal instead). The Logs tab tails those capture files — whether or not the worker is currently running — and only reads a bounded tail (the most recent ~256 KiB per file, at most 500 lines), so huge logs never slow down or bloat the dashboard.
+
+A few behaviors worth knowing:
+
+- **No timestamps?** Capture files written by launchd have no timestamps; those entries show `–` for time and keep their relative order.
+- **Secrets**: log lines are scanned for credential-shaped content (`TOKEN=…` style assignments, common token formats) and masked before being served. Logs stay on this machine either way.
+- **ANSI codes**: terminal colors and control characters are stripped for clean rendering.
+- **If nothing shows up**: when no capture file exists (fresh install, or the worker has only ever run in the foreground), the tab explains where logs would be instead of showing an error.
+
 ## Options
 
 | Option          | Description                                           |
@@ -56,6 +68,7 @@ The dashboard is backed by a small read-only JSON API you can use directly, for
 | `GET /api/runs/:id`         | One run with its stage timeline                                       |
 | `GET /api/stats?window=30d` | Aggregate stats (`7d`, `30d`, `90d`, or `all`)                        |
 | `GET /api/worker`           | Worker liveness, queue counts, agent PRs, poll cursors                |
+| `GET /api/logs`             | Recent worker log entries (`limit` 1–1000, default 500; `level` all/info/warn/error) |
 | `GET /api/health`           | Health check                                                          |
 
 ## License

docs/code/worker.md

@@ -106,6 +106,7 @@ On shutdown the scheduler stops its timer, terminates active automation subproce
 | `occurrence skipped: repository is busy` | Another task holds the repo run lock; the next occurrence will retry. |
 | Scheduled runs missing from the dashboard | Filter the run list by origin `scheduled`; check the worker has an automation license (startup log). |
 | Task files pile up under `~/.devintern/automations/` | They are small and safe to delete — they are only run inputs; the durable record is the run history in `queue.db`. |
+| A run failed and you need to know why | Open the dashboard's Logs tab to read recent worker output without a shell on the machine ([details](./dashboard.md)). |
 
 ## Polling mode
 

packages/code/src/dashboard-server.ts

@@ -16,6 +16,7 @@ import { join, normalize, resolve } from "path";
 
 import {
   DashboardData,
+  handleLogs,
   handleRuns,
   handleRunDetail,
   handleStats,
@@ -30,6 +31,8 @@ export interface DashboardServerOptions {
   dbPath?: string;
   /** Project root used to locate the worker lock file. */
   workingDir?: string;
+  /** Directories to search for worker capture files (primary first). */
+  logDirs?: string[];
 }
 
 /** Resolve the built dashboard UI directory, or null when not shipped/built. */
@@ -87,7 +90,11 @@ export function startDashboardServer(
   const port =
     options.port ?? parseInt(process.env.DASHBOARD_PORT || String(DEFAULT_DASHBOARD_PORT), 10);
   const host = options.host ?? "127.0.0.1";
-  const data = new DashboardData({ dbPath: options.dbPath, workingDir: options.workingDir });
+  const data = new DashboardData({
+    dbPath: options.dbPath,
+    workingDir: options.workingDir,
+    logDirs: options.logDirs,
+  });
   const uiDir = resolveUiDir();
 
   if (host !== "127.0.0.1" && host !== "localhost") {
@@ -126,6 +133,9 @@ export function startDashboardServer(
         if (pathname === "/api/worker") {
           return json(handleWorkerStatus(data));
         }
+        if (pathname === "/api/logs") {
+          return json(handleLogs(data, url.searchParams));
+        }
         return json({ status: 404, body: { error: "not found" } });
       }
 

packages/code/src/lib/dashboard-api.ts

@@ -1,10 +1,10 @@
 /**
  * Dashboard API
  *
- * Read-only JSON handlers over the worker's SQLite state (run records, agent
- * PRs, cursors, queue counts). Pure functions over a lazily opened read-only
- * database so they are testable without HTTP; `dashboard-server.ts` maps them
- * to routes.
+ * Read-only JSON handlers over the worker's local state (SQLite run records,
+ * agent PRs, cursors, queue counts, plus the tailed worker capture files).
+ * Pure functions over a lazily opened read-only database so they are testable
+ * without HTTP; `dashboard-server.ts` maps them to routes.
  *
  * The database may not exist yet (fresh install, worker never run) or may
  * predate some tables (older versions). Every handler degrades to an empty
@@ -14,7 +14,10 @@
 import { LockManager } from "./lock-manager";
 import { RunStore } from "./run-recorder";
 import type { RunOrigin, RunRecord, RunStageRecord, RunStats, RunStatus } from "./run-recorder";
+import { readWorkerLogs } from "./worker-logs";
+import type { LogEntry, WorkerLogLevel, WorkerLogsResult } from "./worker-logs";
 import { resolveQueueDbPath, WebhookQueue } from "./webhook-queue";
+import { resolveWorkspaceDir } from "./workspace/paths";
 import { WorkerState } from "./worker-state";
 import type { Cursor } from "./worker-state";
 
@@ -38,6 +41,10 @@ const STATS_WINDOWS: Record<string, number | null> = {
 const DEFAULT_LIMIT = 50;
 const MAX_LIMIT = 200;
 
+const LOG_LEVELS: (WorkerLogLevel | "all")[] = ["all", "info", "warn", "error"];
+const DEFAULT_LOG_LIMIT = 500;
+const MAX_LOG_LIMIT = 1000;
+
 /** Name of the worker daemon's lock file (see `startWorker`). */
 const WORKER_LOCK_FILE = ".worker.lock";
 
@@ -50,6 +57,10 @@ export interface DashboardDataOptions {
   dbPath?: string;
   /** Project root used to locate the worker lock file. */
   workingDir?: string;
+  /** Directories to search for worker capture files (primary first). */
+  logDirs?: string[];
+  /** Tail window per capture file; tests shrink this for truncation cases. */
+  maxLogBytesPerFile?: number;
 }
 
 interface Stores {
@@ -58,6 +69,12 @@ interface Stores {
   queue: WebhookQueue;
 }
 
+/** A log entry extended with the latest matching run, when one exists. */
+export interface EnrichedLogEntry extends LogEntry {
+  runId?: number;
+  runStatus?: RunStatus;
+}
+
 /**
  * Lazily opened read-only view over the worker's SQLite database.
  *
@@ -68,11 +85,27 @@ interface Stores {
 export class DashboardData {
   readonly dbPath: string;
   readonly workingDir: string;
+  private readonly logDirs: string[];
+  private readonly maxLogBytesPerFile: number | undefined;
   private stores: Stores | null = null;
 
   constructor(options: DashboardDataOptions = {}) {
     this.dbPath = options.dbPath ?? resolveQueueDbPath();
     this.workingDir = options.workingDir ?? process.cwd();
+    if (options.logDirs !== undefined) {
+      // Explicit dirs keep tests hermetic; the workspace home is not probed.
+      this.logDirs = options.logDirs;
+    } else {
+      const dirs = [this.workingDir];
+      const workspaceDir = resolveWorkspaceDir();
+      if (!dirs.includes(workspaceDir)) {
+        // Standalone `devintern dashboard` often runs outside the workspace
+        // home where the daemon's service definition drops its capture files.
+        dirs.push(workspaceDir);
+      }
+      this.logDirs = dirs;
+    }
+    this.maxLogBytesPerFile = options.maxLogBytesPerFile;
   }
 
   /** Open (or reuse) the read-only stores; null while the DB file is missing. */
@@ -150,6 +183,53 @@ export class DashboardData {
     return this.read([], (stores) => stores.state.listCursors());
   }
 
+  /**
+   * Tail the worker's capture files and link entries to their latest run.
+   * File reads are bounded (see `readWorkerLogs`); a missing DB only skips
+   * the run enrichment, never breaks the response.
+   */
+  getWorkerLogs(
+    filter: { limit?: number; level?: WorkerLogLevel | "all" } = {},
+  ): WorkerLogsResult & {
+    entries: EnrichedLogEntry[];
+  } {
+    const result = readWorkerLogs({
+      dirs: this.logDirs,
+      limit: filter.limit,
+      level: filter.level,
+      maxBytesPerFile: this.maxLogBytesPerFile,
+    });
+    const keys = [
+      ...new Set(result.entries.flatMap((entry) => (entry.taskKey ? [entry.taskKey] : []))),
+    ];
+    // One batched lookup for all keys (see RunStore.latestRunByTaskKey) —
+    // per-key queries here would mean up to MAX_LOG_LIMIT queries per poll.
+    const runsByKey = this.read<Map<string, { id: number; status: RunStatus | undefined }>>(
+      new Map(),
+      (stores) => {
+        const latest = stores.runs.latestRunByTaskKey(keys);
+        const trimmed = new Map<string, { id: number; status: RunStatus | undefined }>();
+        for (const [key, run] of latest) {
+          trimmed.set(key, { id: run.id, status: run.status });
+        }
+        return trimmed;
+      },
+    );
+    if (runsByKey.size === 0) {
+      return { ...result, entries: result.entries };
+    }
+    return {
+      ...result,
+      entries: result.entries.map((entry): EnrichedLogEntry => {
+        const linkedRun = entry.taskKey ? runsByKey.get(entry.taskKey) : undefined;
+        if (!linkedRun) {
+          return entry;
+        }
+        return { ...entry, runId: linkedRun.id, runStatus: linkedRun.status };
+      }),
+    };
+  }
+
   /** Close the underlying SQLite connections (tests, shutdown). */
   close(): void {
     if (this.stores) {
@@ -259,3 +339,27 @@ export function handleWorkerStatus(data: DashboardData): ApiResponse {
     },
   };
 }
+
+/**
+ * `GET /api/logs` — the most recent worker log entries, tailed from the
+ * capture files with an entry-count bound.
+ *
+ * @param data - Dashboard data source
+ * @param params - Query params: `limit` (1..1000, default 500) and
+ *                 `level` (all | info | warn | error, default all)
+ */
+export function handleLogs(data: DashboardData, params: URLSearchParams): ApiResponse {
+  const rawLimit = params.get("limit");
+  const limit = rawLimit === null ? DEFAULT_LOG_LIMIT : parseInt(rawLimit, 10);
+  if (!Number.isFinite(limit) || limit < 1 || limit > MAX_LOG_LIMIT) {
+    return badRequest(`limit must be between 1 and ${MAX_LOG_LIMIT}`);
+  }
+  const rawLevel = params.get("level") ?? "all";
+  if (!LOG_LEVELS.includes(rawLevel as WorkerLogLevel | "all")) {
+    return badRequest(`level must be one of: ${LOG_LEVELS.join(", ")}`);
+  }
+  return {
+    status: 200,
+    body: data.getWorkerLogs({ limit, level: rawLevel as WorkerLogLevel | "all" }),
+  };
+}

packages/code/src/lib/run-recorder.ts

@@ -401,6 +401,38 @@ export class RunStore {
     return rows.map((row) => this.rowToRun(row));
   }
 
+  /**
+   * Latest run per task key, using the same ordering as {@link listRuns}
+   * (started_at DESC, id DESC). One query for all keys instead of one per key.
+   *
+   * @param taskKeys - Task keys to look up; duplicates are ignored
+   */
+  latestRunByTaskKey(taskKeys: string[]): Map<string, RunRecord> {
+    const map = new Map<string, RunRecord>();
+    const keys = [...new Set(taskKeys)];
+    if (keys.length === 0) {
+      return map;
+    }
+    const placeholders = keys.map(() => "?").join(", ");
+    const rows = this.db
+      .query(
+        `SELECT * FROM (
+           SELECT *, ROW_NUMBER() OVER (
+             PARTITION BY task_key ORDER BY started_at DESC, id DESC
+           ) AS rn
+           FROM runs WHERE task_key IN (${placeholders})
+         ) WHERE rn = 1`,
+      )
+      .all(...keys) as Record<string, unknown>[];
+    for (const row of rows) {
+      const run = this.rowToRun(row);
+      if (run.taskKey) {
+        map.set(run.taskKey, run);
+      }
+    }
+    return map;
+  }
+
   /**
    * Count runs matching a filter (pagination totals).
    *