diff --git a/docs/code/dashboard.md b/docs/code/dashboard.md index 4e884f4..c8b0580 100644 --- a/docs/code/dashboard.md +++ b/docs/code/dashboard.md @@ -1,14 +1,14 @@ --- title: "Observability Dashboard" -description: "A local web dashboard for worker run history: per-task timelines, stage-by-stage outcomes, and aggregate stats" +description: "A local web dashboard for worker run history: per-task timelines, stage-by-stage outcomes, aggregate stats, and worker logs" section: "Server Automation" order: 2 -dateModified: 2026-08-24 +dateModified: 2026-08-27 --- # Observability Dashboard -`devintern dashboard` serves a local web dashboard over the worker's run history: every task, PR mention, and scheduled automation the worker handled, the stages each run went through (feasibility, implementation, self-review, change requests, outcome), and aggregate stats like success rate and runs per week. +`devintern dashboard` serves a local web dashboard over the worker's run history: every task, PR mention, and scheduled automation the worker handled, the stages each run went through (feasibility, implementation, self-review, change requests, outcome), aggregate stats like success rate and runs per week, and the worker's own log output. All data is read from the worker's local database (`.devintern-code/queue.db`). Nothing is uploaded anywhere: the dashboard runs on your machine and binds to localhost by default. @@ -31,10 +31,30 @@ The standalone command reads the database in read-only mode, so it is safe to ru - **Run list**: every run with its status, task key or automation id, origin (tracker task, PR mention, or scheduled), agent harness, PR link, and duration. Filter by status or origin (`origin=scheduled` isolates automation runs). - **Run detail**: a stage-by-stage timeline for one run: the feasibility verdict, the implementation summary, each self-review iteration, each human change request and how it was handled, and the final outcome. - **Stats**: runs per week, success and escalation rates, median run duration, and a per-harness breakdown over a selectable window (7, 30, or 90 days, or all time). +- **Logs**: the most recent worker log lines (timestamp, severity, message), filterable by level (`everything` / `warnings` / `errors`) with a search box over the loaded window. Lines that mention a task key link straight to that task's latest run in the Runs view. - **Worker status**: whether the daemon is running, queued and failed events, open agent PRs, and per-source poll cursors. Success and escalation rates are computed over finished runs only. Run duration is measured from pickup to PR creation and is a proxy for ticket-to-PR time. Merge rate is not shown yet: the worker records PRs as open or closed but does not track merges separately. +## Where the logs come from + +The worker daemon tees its own console output into `worker.stdout.log` and `worker.stderr.log` in the workspace home (`~/.devintern`), so the Logs tab works no matter how the daemon is launched — a systemd unit, launchd agent, cron wrapper, or a plain terminal — without the service definition having to redirect stdout/stderr. Each file rotates to `.1` past 8 MiB. The Logs tab tails those capture files — whether or not the worker is currently running — and only reads a bounded tail (the most recent ~256 KiB per file, at most 500 lines), so huge logs never slow down or bloat the dashboard. + +To follow the same output in a terminal: + +```bash +tail -f ~/.devintern/worker.stdout.log ~/.devintern/worker.stderr.log +``` + +If `DEVINTERN_WORKSPACE_DIR` is set, the capture files live in that directory instead. When the files stay empty, your service definition still redirects output elsewhere — for example a `>> file` shell wrapper in the unit. Remove the redirect (or re-run `worker init` and reinstall the unit), or fall back to the journal: `journalctl --user -u devintern-worker -f`. + +A few behaviors worth knowing: + +- **No timestamps?** Capture files written by a service redirect instead of the worker's own capture have no timestamps; those entries show `–` for time and keep their relative order. The worker's own capture timestamps every line. +- **Secrets**: log lines are scanned for credential-shaped content (`TOKEN=…` style assignments, common token formats) and masked before being served. Logs stay on this machine either way. +- **ANSI codes**: terminal colors and control characters are stripped for clean rendering. +- **If nothing shows up**: when no capture file exists (fresh install, or the worker has only ever run in the foreground), the tab explains where logs would be instead of showing an error. + ## Options | Option | Description | @@ -56,6 +76,7 @@ The dashboard is backed by a small read-only JSON API you can use directly, for | `GET /api/runs/:id` | One run with its stage timeline | | `GET /api/stats?window=30d` | Aggregate stats (`7d`, `30d`, `90d`, or `all`) | | `GET /api/worker` | Worker liveness, queue counts, agent PRs, poll cursors | +| `GET /api/logs` | Recent worker log entries (`limit` 1–1000, default 500; `level` all/info/warn/error) | | `GET /api/health` | Health check | ## License diff --git a/docs/code/worker.md b/docs/code/worker.md index 0e8bfb6..fb17207 100644 --- a/docs/code/worker.md +++ b/docs/code/worker.md @@ -106,6 +106,8 @@ On shutdown the scheduler stops its timer, terminates active automation subproce | `occurrence skipped: repository is busy` | Another task holds the repo run lock; the next occurrence will retry. | | Scheduled runs missing from the dashboard | Filter the run list by origin `scheduled`; check the worker has an automation license (startup log). | | Task files pile up under `~/.devintern/automations/` | They are small and safe to delete — they are only run inputs; the durable record is the run history in `queue.db`. | +| A run failed and you need to know why | Open the dashboard's Logs tab to read recent worker output without a shell on the machine ([details](./dashboard.md)). | +| The dashboard Logs tab is empty | The daemon tees its output to `worker.stdout.log` / `worker.stderr.log` in the workspace home — check those files (or `journalctl --user -u devintern-worker`) and see [where the logs come from](./dashboard.md#where-the-logs-come-from). | ## Polling mode diff --git a/packages/code/CHANGELOG.md b/packages/code/CHANGELOG.md index 03cad5e..4d0552c 100644 --- a/packages/code/CHANGELOG.md +++ b/packages/code/CHANGELOG.md @@ -6,6 +6,7 @@ - **High-signal worker analytics**: a worker emits one anonymous `worker_started` event after its sources start (`polling`, `relay`, `hybrid`, or `scheduled`) and one `worker_task_run` event per terminal task outcome. Worker task subprocesses no longer duplicate the ordinary `cli_run` event; no polling heartbeats, task keys, repository names, prompts, or error text are sent - **Workspace worker probes push access at startup**: each configured GitHub HTTPS remote gets a side-effect-free `git push --dry-run` against its bare clone, so an under-scoped token (fine-grained PAT without `Contents: Read and write`, or a `$GITHUB_TOKEN` that silently overrides the keyring login via `gh auth git-credential`) is reported as a clear startup warning instead of burning task pickups on 403 pushes +- **Worker logs just work**: the daemon tees its own console output into `worker.stdout.log` / `worker.stderr.log` in the workspace home (`~/.devintern`; one log set per workspace, not per repo, rotating at 8 MiB), so the dashboard Logs tab is populated under any launch method — systemd unit, launchd agent, cron wrapper, or terminal — with no service-definition redirection required ### Changed diff --git a/packages/code/src/dashboard-server.ts b/packages/code/src/dashboard-server.ts index e946e81..cb9b192 100644 --- a/packages/code/src/dashboard-server.ts +++ b/packages/code/src/dashboard-server.ts @@ -16,6 +16,7 @@ import { join, normalize, resolve } from "path"; import { DashboardData, + handleLogs, handleRuns, handleRunDetail, handleStats, @@ -30,6 +31,8 @@ export interface DashboardServerOptions { dbPath?: string; /** Project root used to locate the worker lock file. */ workingDir?: string; + /** Directories to search for worker capture files (primary first). */ + logDirs?: string[]; } /** Resolve the built dashboard UI directory, or null when not shipped/built. */ @@ -87,7 +90,11 @@ export function startDashboardServer( const port = options.port ?? parseInt(process.env.DASHBOARD_PORT || String(DEFAULT_DASHBOARD_PORT), 10); const host = options.host ?? "127.0.0.1"; - const data = new DashboardData({ dbPath: options.dbPath, workingDir: options.workingDir }); + const data = new DashboardData({ + dbPath: options.dbPath, + workingDir: options.workingDir, + logDirs: options.logDirs, + }); const uiDir = resolveUiDir(); if (host !== "127.0.0.1" && host !== "localhost") { @@ -126,6 +133,9 @@ export function startDashboardServer( if (pathname === "/api/worker") { return json(handleWorkerStatus(data)); } + if (pathname === "/api/logs") { + return json(handleLogs(data, url.searchParams)); + } return json({ status: 404, body: { error: "not found" } }); } diff --git a/packages/code/src/lib/dashboard-api.ts b/packages/code/src/lib/dashboard-api.ts index ef0b45c..b2c5070 100644 --- a/packages/code/src/lib/dashboard-api.ts +++ b/packages/code/src/lib/dashboard-api.ts @@ -1,10 +1,10 @@ /** * Dashboard API * - * Read-only JSON handlers over the worker's SQLite state (run records, agent - * PRs, cursors, queue counts). Pure functions over a lazily opened read-only - * database so they are testable without HTTP; `dashboard-server.ts` maps them - * to routes. + * Read-only JSON handlers over the worker's local state (SQLite run records, + * agent PRs, cursors, queue counts, plus the tailed worker capture files). + * Pure functions over a lazily opened read-only database so they are testable + * without HTTP; `dashboard-server.ts` maps them to routes. * * The database may not exist yet (fresh install, worker never run) or may * predate some tables (older versions). Every handler degrades to an empty @@ -14,7 +14,10 @@ import { LockManager } from "./lock-manager"; import { RunStore } from "./run-recorder"; import type { RunOrigin, RunRecord, RunStageRecord, RunStats, RunStatus } from "./run-recorder"; +import { readWorkerLogs } from "./worker-logs"; +import type { LogEntry, WorkerLogLevel, WorkerLogsResult } from "./worker-logs"; import { resolveQueueDbPath, WebhookQueue } from "./webhook-queue"; +import { resolveWorkspaceDir } from "./workspace/paths"; import { WorkerState } from "./worker-state"; import type { Cursor } from "./worker-state"; @@ -38,6 +41,10 @@ const STATS_WINDOWS: Record = { const DEFAULT_LIMIT = 50; const MAX_LIMIT = 200; +const LOG_LEVELS: (WorkerLogLevel | "all")[] = ["all", "info", "warn", "error"]; +const DEFAULT_LOG_LIMIT = 500; +const MAX_LOG_LIMIT = 1000; + /** Name of the worker daemon's lock file (see `startWorker`). */ const WORKER_LOCK_FILE = ".worker.lock"; @@ -50,6 +57,10 @@ export interface DashboardDataOptions { dbPath?: string; /** Project root used to locate the worker lock file. */ workingDir?: string; + /** Directories to search for worker capture files (primary first). */ + logDirs?: string[]; + /** Tail window per capture file; tests shrink this for truncation cases. */ + maxLogBytesPerFile?: number; } interface Stores { @@ -58,6 +69,12 @@ interface Stores { queue: WebhookQueue; } +/** A log entry extended with the latest matching run, when one exists. */ +export interface EnrichedLogEntry extends LogEntry { + runId?: number; + runStatus?: RunStatus; +} + /** * Lazily opened read-only view over the worker's SQLite database. * @@ -68,11 +85,27 @@ interface Stores { export class DashboardData { readonly dbPath: string; readonly workingDir: string; + private readonly logDirs: string[]; + private readonly maxLogBytesPerFile: number | undefined; private stores: Stores | null = null; constructor(options: DashboardDataOptions = {}) { this.dbPath = options.dbPath ?? resolveQueueDbPath(); this.workingDir = options.workingDir ?? process.cwd(); + if (options.logDirs !== undefined) { + // Explicit dirs keep tests hermetic; the workspace home is not probed. + this.logDirs = options.logDirs; + } else { + const dirs = [this.workingDir]; + const workspaceDir = resolveWorkspaceDir(); + if (!dirs.includes(workspaceDir)) { + // Standalone `devintern dashboard` often runs outside the workspace + // home where the daemon's service definition drops its capture files. + dirs.push(workspaceDir); + } + this.logDirs = dirs; + } + this.maxLogBytesPerFile = options.maxLogBytesPerFile; } /** Open (or reuse) the read-only stores; null while the DB file is missing. */ @@ -150,6 +183,53 @@ export class DashboardData { return this.read([], (stores) => stores.state.listCursors()); } + /** + * Tail the worker's capture files and link entries to their latest run. + * File reads are bounded (see `readWorkerLogs`); a missing DB only skips + * the run enrichment, never breaks the response. + */ + getWorkerLogs( + filter: { limit?: number; level?: WorkerLogLevel | "all" } = {}, + ): WorkerLogsResult & { + entries: EnrichedLogEntry[]; + } { + const result = readWorkerLogs({ + dirs: this.logDirs, + limit: filter.limit, + level: filter.level, + maxBytesPerFile: this.maxLogBytesPerFile, + }); + const keys = [ + ...new Set(result.entries.flatMap((entry) => (entry.taskKey ? [entry.taskKey] : []))), + ]; + // One batched lookup for all keys (see RunStore.latestRunByTaskKey) — + // per-key queries here would mean up to MAX_LOG_LIMIT queries per poll. + const runsByKey = this.read>( + new Map(), + (stores) => { + const latest = stores.runs.latestRunByTaskKey(keys); + const trimmed = new Map(); + for (const [key, run] of latest) { + trimmed.set(key, { id: run.id, status: run.status }); + } + return trimmed; + }, + ); + if (runsByKey.size === 0) { + return { ...result, entries: result.entries }; + } + return { + ...result, + entries: result.entries.map((entry): EnrichedLogEntry => { + const linkedRun = entry.taskKey ? runsByKey.get(entry.taskKey) : undefined; + if (!linkedRun) { + return entry; + } + return { ...entry, runId: linkedRun.id, runStatus: linkedRun.status }; + }), + }; + } + /** Close the underlying SQLite connections (tests, shutdown). */ close(): void { if (this.stores) { @@ -259,3 +339,27 @@ export function handleWorkerStatus(data: DashboardData): ApiResponse { }, }; } + +/** + * `GET /api/logs` — the most recent worker log entries, tailed from the + * capture files with an entry-count bound. + * + * @param data - Dashboard data source + * @param params - Query params: `limit` (1..1000, default 500) and + * `level` (all | info | warn | error, default all) + */ +export function handleLogs(data: DashboardData, params: URLSearchParams): ApiResponse { + const rawLimit = params.get("limit"); + const limit = rawLimit === null ? DEFAULT_LOG_LIMIT : parseInt(rawLimit, 10); + if (!Number.isFinite(limit) || limit < 1 || limit > MAX_LOG_LIMIT) { + return badRequest(`limit must be between 1 and ${MAX_LOG_LIMIT}`); + } + const rawLevel = params.get("level") ?? "all"; + if (!LOG_LEVELS.includes(rawLevel as WorkerLogLevel | "all")) { + return badRequest(`level must be one of: ${LOG_LEVELS.join(", ")}`); + } + return { + status: 200, + body: data.getWorkerLogs({ limit, level: rawLevel as WorkerLogLevel | "all" }), + }; +} diff --git a/packages/code/src/lib/run-recorder.ts b/packages/code/src/lib/run-recorder.ts index 45c6a17..020eceb 100644 --- a/packages/code/src/lib/run-recorder.ts +++ b/packages/code/src/lib/run-recorder.ts @@ -401,6 +401,38 @@ export class RunStore { return rows.map((row) => this.rowToRun(row)); } + /** + * Latest run per task key, using the same ordering as {@link listRuns} + * (started_at DESC, id DESC). One query for all keys instead of one per key. + * + * @param taskKeys - Task keys to look up; duplicates are ignored + */ + latestRunByTaskKey(taskKeys: string[]): Map { + const map = new Map(); + const keys = [...new Set(taskKeys)]; + if (keys.length === 0) { + return map; + } + const placeholders = keys.map(() => "?").join(", "); + const rows = this.db + .query( + `SELECT * FROM ( + SELECT *, ROW_NUMBER() OVER ( + PARTITION BY task_key ORDER BY started_at DESC, id DESC + ) AS rn + FROM runs WHERE task_key IN (${placeholders}) + ) WHERE rn = 1`, + ) + .all(...keys) as Record[]; + for (const row of rows) { + const run = this.rowToRun(row); + if (run.taskKey) { + map.set(run.taskKey, run); + } + } + return map; + } + /** * Count runs matching a filter (pagination totals). * diff --git a/packages/code/src/lib/worker-capture.ts b/packages/code/src/lib/worker-capture.ts new file mode 100644 index 0000000..9d82aba --- /dev/null +++ b/packages/code/src/lib/worker-capture.ts @@ -0,0 +1,158 @@ +/** + * Worker self-capture: the daemon tees its own console output into + * `worker.stdout.log` / `worker.stderr.log` under the workspace home. + * + * The dashboard Logs tab tails those capture files. By writing them from + * inside the worker, logs work no matter how the daemon is launched — a + * generated systemd unit or launchd agent, a hand-written unit, a shell + * wrapper, or a plain terminal — without every service definition having to + * redirect stdout/stderr to the right files. + * + * Writes are synchronous (`writeSync`) so a crash or `process.exit` mid-line + * cannot lose buffered output. A failure to open or write the files is never + * allowed to stop the worker. + */ + +import { closeSync, existsSync, openSync, renameSync, statSync, writeSync } from "fs"; +import { join, resolve } from "path"; +import { format } from "util"; + +export type WorkerCaptureStream = "out" | "err"; + +export interface WorkerCaptureHandle { + /** Absolute paths of the files being appended to. */ + paths: Record; + /** Restore the original console methods and close the file descriptors. */ + stop(): void; +} + +export interface WorkerCaptureOptions { + /** Rotation threshold per file; defaults to {@link WORKER_CAPTURE_ROTATE_BYTES}. */ + maxBytes?: number; +} + +/** Capture file names (also tailed by the dashboard; see `worker-logs.ts`). */ +const FILE_NAMES: Record = { + out: "worker.stdout.log", + err: "worker.stderr.log", +}; + +/** Rename a capture file to `.1` once it grows past this size. */ +export const WORKER_CAPTURE_ROTATE_BYTES = 8 * 1024 * 1024; + +type ConsoleKeys = "log" | "info" | "debug" | "warn" | "error"; + +interface Originals { + log: typeof console.log; + info: typeof console.info; + debug: typeof console.debug; + warn: typeof console.warn; + error: typeof console.error; +} + +/** Rename `path` to `path.1` (replacing any previous rotation) when oversized. */ +function rotateIfNeeded(path: string, maxBytes: number): void { + if (!existsSync(path)) { + return; + } + try { + if (statSync(path).size > maxBytes) { + renameSync(path, `${path}.1`); + } + } catch { + // Rotation is best-effort; appending to an oversized file beats crashing. + } +} + +/** ISO timestamp prefix, added per line so the dashboard can order entries. */ +function timestamp(): string { + return `${new Date().toISOString()} `; +} + +function writeLines(fd: number, chunk: string): void { + // console methods never end their formatted output in a newline; add one. + const lines = chunk.split("\n"); + const text = lines.map((line) => `${timestamp()}${line}`).join("\n"); + try { + writeSync(fd, `${text}\n`); + } catch { + // Disk full, file rotated away, …: the console still got the output. + } +} + +/** + * Start teeing console output into capture files under `dir`. + * + * console.log/info/debug go to `worker.stdout.log`, console.warn/error to + * `worker.stderr.log`; the original methods still run, so foreground + * terminals keep streaming output. Returns a handle whose `stop()` restores + * the original console. + * + * @param dir - Directory for the capture files (the workspace home) + * @param options - Rotation threshold override + */ +export function startWorkerCapture( + dir: string, + options: WorkerCaptureOptions = {}, +): WorkerCaptureHandle { + const maxBytes = options.maxBytes ?? WORKER_CAPTURE_ROTATE_BYTES; + const paths: Record = { + out: resolve(join(resolve(dir), FILE_NAMES.out)), + err: resolve(join(resolve(dir), FILE_NAMES.err)), + }; + const fds: Record = { out: -1, err: -1 }; + + for (const stream of ["out", "err"] as const) { + try { + rotateIfNeeded(paths[stream], maxBytes); + fds[stream] = openSync(paths[stream], "a"); + } catch { + fds[stream] = -1; + } + } + + const originals: Originals = { + log: console.log, + info: console.info, + debug: console.debug, + warn: console.warn, + error: console.error, + }; + + const tee = (stream: WorkerCaptureStream, original: (...args: unknown[]) => void) => { + return (...args: unknown[]): void => { + const fd = fds[stream]; + if (fd !== -1) { + writeLines(fd, format(...args)); + } + original(...args); + }; + }; + + console.log = tee("out", originals.log); + console.info = tee("out", originals.info); + console.debug = tee("out", originals.debug); + console.warn = tee("err", originals.warn); + console.error = tee("err", originals.error); + + return { + paths, + stop(): void { + console.log = originals.log; + console.info = originals.info; + console.debug = originals.debug; + console.warn = originals.warn; + console.error = originals.error; + for (const stream of ["out", "err"] as const) { + if (fds[stream] !== -1) { + try { + closeSync(fds[stream]); + } catch { + // Already closed. + } + fds[stream] = -1; + } + } + }, + }; +} diff --git a/packages/code/src/lib/worker-init.ts b/packages/code/src/lib/worker-init.ts index bf4e509..bad78a5 100644 --- a/packages/code/src/lib/worker-init.ts +++ b/packages/code/src/lib/worker-init.ts @@ -96,6 +96,11 @@ export function upsertEnvVars(content: string, vars: Record): st /** * Render a systemd service unit for the worker. * + * No stdout/stderr redirection here on purpose: the daemon tees its own + * console output into the dashboard's capture files (see `worker-capture.ts`), + * so custom units and shell wrappers need no redirect either — adding one + * would hide the output from the dashboard. + * * @param options - Binary path, working directory, and whether to run the direct webhook service */ export function renderSystemdUnit(options: { @@ -134,7 +139,12 @@ function escapeXml(value: string): string { .replace(/'/g, "'"); } -/** Render a per-user macOS launchd agent for the workspace worker. */ +/** + * Render a per-user macOS launchd agent for the workspace worker. + * + * Like the systemd unit, this does not redirect stdout/stderr: the worker + * self-captures into the dashboard's log files (see `worker-capture.ts`). + */ export function renderLaunchdPlist(options: { execPath: string; workingDir: string; @@ -161,10 +171,6 @@ export function renderLaunchdPlist(options: { SuccessfulExit - StandardOutPath - ${escapeXml(join(options.workingDir, "worker.stdout.log"))} - StandardErrorPath - ${escapeXml(join(options.workingDir, "worker.stderr.log"))} `; diff --git a/packages/code/src/lib/worker-logs.ts b/packages/code/src/lib/worker-logs.ts new file mode 100644 index 0000000..bdfcb9d --- /dev/null +++ b/packages/code/src/lib/worker-logs.ts @@ -0,0 +1,393 @@ +/** + * Worker log tailing for the dashboard. + * + * The worker daemon logs to stdout/stderr and tees its own console output + * into `worker.stdout.log` / `worker.stderr.log` in the workspace home (see + * `worker-capture.ts`), so the files exist no matter how the daemon is + * launched — systemd, launchd, cron, or a plain terminal. Legacy setups that + * redirect stdout/stderr to those same files keep working; this module tails + * the capture files so the dashboard can surface recent worker log entries + * without SSH access — the same local-data philosophy as the SQLite-backed + * endpoints in `dashboard-api.ts`. + * + * Reads are strictly bounded: at most `maxBytesPerFile` bytes from the end of + * each file are touched, and at most `limit` entries are returned. Lines are + * stripped of ANSI codes, classified by severity, and obvious credentials are + * masked before anything leaves this machine. + */ + +import { closeSync, existsSync, openSync, readSync, statSync } from "fs"; +import { join, resolve } from "path"; + +export type WorkerLogLevel = "info" | "warn" | "error"; + +export type LogStream = "out" | "err"; + +export interface LogEntry { + /** 0-based position within the returned window (oldest first). */ + index: number; + /** Epoch ms parsed from a leading timestamp; null when the line has none. */ + timestamp: number | null; + level: WorkerLogLevel; + stream: LogStream; + message: string; + /** Best-effort task key extracted from the message (`PROJ-123`). */ + taskKey: string | null; +} + +export interface LogSourceInfo { + path: string; + stream: LogStream; + exists: boolean; + /** Total file size when readable; -1 otherwise. */ + totalBytes: number; + /** Bytes actually parsed (the tail window). */ + readBytes: number; + /** True when older content existed before the read window. */ + truncated: boolean; + /** Read failure (permissions, transient rewrite); empty otherwise. */ + error: string; +} + +export interface ReadWorkerLogsOptions { + /** Directories holding candidate capture files (primary first). */ + dirs?: string[]; + /** Maximum entries returned (the most recent ones). Default 500. */ + limit?: number; + /** Only return entries of this level. Default "all". */ + level?: WorkerLogLevel | "all"; + /** Tail window per file. Default 256 KiB. */ + maxBytesPerFile?: number; +} + +export interface WorkerLogsResult { + /** Whether any capture file was found (false on a fresh install). */ + available: boolean; + entries: LogEntry[]; + sources: LogSourceInfo[]; + /** True when entries exist beyond the byte window or the entry limit. */ + truncated: boolean; +} + +const DEFAULT_LIMIT = 500; +const DEFAULT_MAX_BYTES_PER_FILE = 256 * 1024; +const MAX_MESSAGE_CHARS = 4000; + +/** Capture file names from the launchd service definition (see worker-init.ts). */ +const LOG_FILES: { name: string; stream: LogStream }[] = [ + { name: "worker.stdout.log", stream: "out" }, + { name: "worker.stderr.log", stream: "err" }, +]; + +/** CSI escape sequences (colors, cursor moves) emitted by colorizers. */ +// oxlint-disable-next-line no-control-regex -- these are control characters by definition +const ANSI_PATTERN = /\x1B\[[0-9;?]*[A-Za-z]/g; + +/** Strip ANSI escapes and carriage returns so lines render cleanly. */ +export function stripAnsi(line: string): string { + return line.replace(ANSI_PATTERN, "").replace(/\r/g, ""); +} + +/** + * Parse an optional leading ISO-like timestamp prefix (`2026-08-27T12:34:56Z`, + * `2026-08-27 12:34:56.123`, journalctl short-iso with offset, …) and return + * epoch ms plus the rest of the line. A date/time without zone counts as + * local time. Returns null for bare dates and everything else so we never + * guess timestamps we cannot support. + */ +export function parseTimestampPrefix(line: string): { timestampMs: number; rest: string } | null { + const match = line.match( + /^(\d{4}-\d{2}-\d{2})[Tt ](\d{2}:\d{2}:\d{2})(?:[.,](\d{1,6}))?(?:\s*(Z|z|[+-]\d{2}:?\d{2}))?\s*/, + ); + if (!match) { + return null; + } + const [, date, time, fraction, zone] = match; + const normalized = `${date}T${time}${fraction ? `.${fraction}` : ""}${zone ?? ""}`; + const timestampMs = Date.parse(normalized); + if (Number.isNaN(timestampMs)) { + return null; + } + return { timestampMs, rest: line.slice(match[0].length) }; +} + +/** + * Classify a line's severity. Explicit markers win over the stream heuristic, + * because the daemon writes warnings through console.error too. The line must + * already be ANSI-stripped (see {@link stripAnsi}); callers pass the same + * stripped line they keep as the message, so levels always match what renders. + */ +export function classifyLevel(stream: LogStream, line: string): WorkerLogLevel { + if (line.includes("❌") || /\b(?:FATAL|ERROR)\b/i.test(line)) { + return "error"; + } + if (line.includes("⚠") || /\bWARN(?:ING)?\b/i.test(line)) { + return "warn"; + } + return stream === "err" ? "error" : "info"; +} + +/** Uppercase prefixes that look like ticket keys but never are. */ +const TASK_KEY_STOP_LIST = new Set([ + "SHA", + "HTTP", + "HTTPS", + "UTF", + "DNS", + "AWS", + "UUID", + "SSL", + "TLS", + "CVE", + "ISO", +]); + +/** First Jira-style task key in a message (`PROJ-123`), or null. */ +export function extractTaskKey(message: string): string | null { + for (const match of message.matchAll(/\b([A-Z][A-Z0-9]{1,19}-\d{1,10})\b/g)) { + const key = match[1]; + if (!key) { + continue; + } + const prefix = key.split("-")[0]; + if (prefix && TASK_KEY_STOP_LIST.has(prefix)) { + continue; + } + return key; + } + return null; +} + +/** + * Mask obvious credentials before serving a line: KEY=value assignments whose + * key smells like a secret, plus common token shapes that can appear echoed + * without a variable name. + */ +export function redactSecrets(message: string): string { + let redacted = message.replace( + /\b([A-Za-z0-9_]*(?:token|secret|password|passwd|api_?key|private[_-]?key|access[_-]?key|pat)[a-z0-9_]*)\s*=\s*(\S+)/gi, + (_match, key: string) => `${key}=[redacted]`, + ); + redacted = redacted + .replace(/\bgh[pousr]_[A-Za-z0-9]{16,}\b/g, "[redacted]") + .replace(/\bxox[baprs]-[A-Za-z0-9-]{10,}\b/g, "[redacted]") + .replace(/\bAKIA[0-9A-Z]{16}\b/g, "[redacted]") + .replace(/\bBearer\s+[A-Za-z0-9._~-]{8,}/gi, "Bearer [redacted]") + .replace(/\bsk-[A-Za-z0-9_-]{20,}\b/g, "[redacted]"); + return redacted; +} + +interface RawEntry { + timestamp: number | null; + level: WorkerLogLevel; + stream: LogStream; + message: string; + seq: number; +} + +function parseSlice(content: string, stream: LogStream): RawEntry[] { + const lines = content.split("\n"); + if (lines.length > 0 && lines[lines.length - 1] === "") { + lines.pop(); + } + const entries: RawEntry[] = []; + for (const [seq, rawLine] of lines.entries()) { + const line = stripAnsi(rawLine); + if (!line.trim()) { + continue; + } + const prefix = parseTimestampPrefix(line); + const body = redactSecrets(prefix ? prefix.rest : line); + entries.push({ + timestamp: prefix?.timestampMs ?? null, + level: classifyLevel(stream, line), + stream, + message: body.length > MAX_MESSAGE_CHARS ? `${body.slice(0, MAX_MESSAGE_CHARS)}…` : body, + seq, + }); + } + return entries; +} + +/** + * Positional read of `[start, fileSize)` without ever holding more than the + * window itself (plus one scan buffer) in memory. + */ +function readWindow(fd: number, start: number, size: number): Buffer { + const windowBytes = Buffer.alloc(Math.max(0, size - start)); + let filled = 0; + while (filled < windowBytes.length) { + const read = readSync(fd, windowBytes, filled, windowBytes.length - filled, start + filled); + if (read <= 0) { + break; + } + filled += read; + } + return windowBytes.subarray(0, filled); +} + +/** + * Advance `rawStart` just past the first newline at or after it, so the + * window opens on a complete line. Falls back to `rawStart` when the rest of + * the file has no newline (one giant trailing line). + */ +function snapToLineStart(fd: number, rawStart: number, size: number): number { + const probeBuffer = Buffer.alloc(8192); + let cursor = rawStart; + while (cursor < size) { + const probeLength = Math.min(probeBuffer.length, size - cursor); + const read = readSync(fd, probeBuffer, 0, probeLength, cursor); + if (read <= 0) { + break; + } + const newlineOffset = probeBuffer.subarray(0, read).indexOf(0x0a); + if (newlineOffset !== -1) { + return cursor + newlineOffset + 1; + } + cursor += read; + } + return rawStart; +} + +function readSource( + filePath: string, + stream: LogStream, + maxBytes: number, +): { + source: LogSourceInfo; + entries: RawEntry[]; +} { + const source: LogSourceInfo = { + path: filePath, + stream, + exists: false, + totalBytes: -1, + readBytes: 0, + truncated: false, + error: "", + }; + if (!existsSync(filePath)) { + return { source, entries: [] }; + } + source.exists = true; + + let fd: number | undefined; + try { + source.totalBytes = statSync(filePath).size; + const size = source.totalBytes; + fd = openSync(filePath, "r"); + let start = 0; + if (size > maxBytes) { + start = snapToLineStart(fd, Math.max(0, size - maxBytes), size); + source.truncated = true; + } + const content = readWindow(fd, start, size).toString("utf8"); + source.readBytes = size - start; + return { source, entries: parseSlice(content, stream) }; + } catch (cause) { + source.error = cause instanceof Error ? cause.message : String(cause); + return { source, entries: [] }; + } finally { + if (fd !== undefined) { + try { + closeSync(fd); + } catch { + // Closing a partially-opened file must not mask the read error. + } + } + } +} + +/** Weave slices together while preserving each slice's internal order. */ +function mergeSlices(slices: RawEntry[][]): RawEntry[] { + if (slices.length === 0) { + return []; + } + const allTimestamped = slices.every((slice) => slice.every((entry) => entry.timestamp !== null)); + if (allTimestamped) { + return slices.flat().sort((a, b) => { + const timeDiff = (a.timestamp ?? 0) - (b.timestamp ?? 0); + if (timeDiff !== 0) { + return timeDiff; + } + if (a.stream !== b.stream) { + return a.stream === "out" ? -1 : 1; + } + return a.seq - b.seq; + }); + } + // Without trustworthy timestamps the true interleave of stdout/stderr is + // unknowable; round-robin keeps recency at the bottom and each stream's + // internal order intact. + const merged: RawEntry[] = []; + const cursors = Array.from({ length: slices.length }, () => 0); + for (;;) { + let tookAny = false; + for (let s = 0; s < slices.length; s++) { + const slice = slices[s]; + const cursor = cursors[s]; + if (slice !== undefined && cursor !== undefined && slice.length > cursor) { + merged.push(slice[cursor]); + cursors[s] = cursor + 1; + tookAny = true; + } + } + if (!tookAny) { + break; + } + } + return merged; +} + +/** + * Read the tail of each worker capture file across `dirs` and return the most + * recent `limit` entries after level filtering, oldest first. + */ +export function readWorkerLogs(options: ReadWorkerLogsOptions = {}): WorkerLogsResult { + const limit = options.limit ?? DEFAULT_LIMIT; + const maxBytes = options.maxBytesPerFile ?? DEFAULT_MAX_BYTES_PER_FILE; + const level = options.level ?? "all"; + const dirs = options.dirs ?? [process.cwd()]; + + const seenPaths = new Set(); + const sources: LogSourceInfo[] = []; + const slices: RawEntry[][] = []; + + for (const dir of dirs) { + for (const candidate of LOG_FILES) { + const filePath = resolve(join(resolve(dir), candidate.name)); + if (seenPaths.has(filePath)) { + continue; + } + seenPaths.add(filePath); + const result = readSource(filePath, candidate.stream, maxBytes); + sources.push(result.source); + if (result.entries.length > 0) { + slices.push(result.entries); + } + } + } + + const available = sources.some((source) => source.exists); + const merged = mergeSlices(slices); + + const windowStart = Math.max(0, merged.length - limit); + const bounded = merged.slice(windowStart); + const filtered = level === "all" ? bounded : bounded.filter((entry) => entry.level === level); + + const entries: LogEntry[] = filtered.map((entry, index) => ({ + index, + timestamp: entry.timestamp, + level: entry.level, + stream: entry.stream, + message: entry.message, + taskKey: extractTaskKey(entry.message), + })); + + return { + available, + entries, + sources, + truncated: sources.some((source) => source.truncated) || windowStart > 0, + }; +} diff --git a/packages/code/src/lib/workspace/workspace-worker.ts b/packages/code/src/lib/workspace/workspace-worker.ts index 7899b96..22e38d5 100644 --- a/packages/code/src/lib/workspace/workspace-worker.ts +++ b/packages/code/src/lib/workspace/workspace-worker.ts @@ -463,6 +463,9 @@ export async function runWorkspaceWorker(options: RunWorkspaceWorkerOptions): Pr { lock: createWorkspaceLock(workspaceDir), label: workspaceDir, + // Capture logs in the workspace home: one daemon serves many repos, and + // the dashboard's log tailer already searches this directory. + logDir: workspaceDir, onStarted: async (acquirerNames) => { trackWorkerStarted({ cliVersion: options.cliVersion ?? "0.0.0", diff --git a/packages/code/src/worker.ts b/packages/code/src/worker.ts index e6d43f8..68b3696 100644 --- a/packages/code/src/worker.ts +++ b/packages/code/src/worker.ts @@ -11,6 +11,8 @@ */ import { LockManager } from "./lib/lock-manager"; +import { startWorkerCapture } from "./lib/worker-capture"; +import type { WorkerCaptureHandle } from "./lib/worker-capture"; export interface WorkerOptions { /** Single-instance lock override (workspace mode locks the workspace home @@ -18,6 +20,9 @@ export interface WorkerOptions { lock?: LockManager; /** What this worker serves, for the startup banner (defaults to cwd). */ label?: string; + /** Directory for the dashboard's `worker.stdout.log` / `worker.stderr.log` + * capture files (the workspace home). Omit to skip self-capture. */ + logDir?: string; /** Called once after every configured event source starts successfully. */ onStarted?: (acquirerNames: string[]) => Promise | void; } @@ -43,6 +48,17 @@ export async function startWorker( options: WorkerOptions, acquirers: Acquirer[] = [], ): Promise { + // Self-capture first so the startup banner lands in the log files the + // dashboard tails; a capture failure never blocks the daemon. + let capture: WorkerCaptureHandle | null = null; + if (options.logDir) { + try { + capture = startWorkerCapture(options.logDir); + } catch { + capture = null; + } + } + console.log("👷 Starting devintern worker"); console.log(` Workspace: ${options.label ?? process.cwd()}`); console.log( @@ -93,6 +109,7 @@ export async function startWorker( // In-flight queue events stay marked in SQLite and are recovered on the // next start; shutdown never loses accepted work. lock.release(); + capture?.stop(); console.log("👋 Worker stopped"); process.exit(0); }; diff --git a/packages/code/tests/dashboard-api.test.ts b/packages/code/tests/dashboard-api.test.ts index a17825d..ad5d7b9 100644 --- a/packages/code/tests/dashboard-api.test.ts +++ b/packages/code/tests/dashboard-api.test.ts @@ -5,6 +5,7 @@ import { tmpdir } from "os"; import { DashboardData, + handleLogs, handleRuns, handleRunDetail, handleStats, @@ -255,6 +256,94 @@ describe("dashboard API", () => { }); }); +describe("logs endpoint", () => { + let dir: string; + let logDir: string; + let dbPath: string; + let data: DashboardData; + + beforeEach(() => { + dir = join(tmpdir(), `dash-logs-${Date.now()}-${Math.random().toString(36).slice(2)}`); + mkdirSync(dir, { recursive: true }); + logDir = join(dir, "capture"); + mkdirSync(logDir, { recursive: true }); + dbPath = join(dir, "queue.db"); + data = new DashboardData({ dbPath, workingDir: dir, logDirs: [logDir] }); + }); + + afterEach(() => { + data.close(); + rmSync(dir, { recursive: true, force: true }); + }); + + test("handleLogs rejects invalid params", () => { + expect(handleLogs(data, new URLSearchParams("limit=0")).status).toBe(400); + expect(handleLogs(data, new URLSearchParams("limit=9999")).status).toBe(400); + expect(handleLogs(data, new URLSearchParams("level=trace")).status).toBe(400); + }); + + test("returns an empty state when no capture files exist", () => { + const response = handleLogs(data, new URLSearchParams()); + expect(response.status).toBe(200); + const body = response.body as { available: boolean; entries: unknown[]; truncated: boolean }; + expect(body.available).toBe(false); + expect(body.entries).toEqual([]); + expect(body.truncated).toBe(false); + }); + + test("tails entries, redacts secrets, and links runs by task key", () => { + const store = new RunStore(dbPath); + const runId = store.createRun({ origin: "task", taskKey: "DEV-42", harness: "claude-code" }); + store.finishRun(runId, "failed"); + store.close(); + + writeFileSync( + join(logDir, "worker.stdout.log"), + "\x1b[31m❌ DEV-42 run failed\x1b[0m\nplain info line\n", + "utf8", + ); + writeFileSync( + join(logDir, "worker.stderr.log"), + "retry with WEBHOOK_SECRET=hunter2 super-secret ignored\n", + "utf8", + ); + + const response = handleLogs(data, new URLSearchParams()); + expect(response.status).toBe(200); + const body = response.body as { + available: boolean; + entries: { message: string; level: string; taskKey?: string | null; runId?: number }[]; + sources: { exists: boolean }[]; + }; + expect(body.available).toBe(true); + expect(body.entries.length).toBe(3); + + const failed = body.entries.find((entry) => entry.message === "❌ DEV-42 run failed"); + expect(failed?.level).toBe("error"); + expect(failed?.taskKey).toBe("DEV-42"); + expect(failed?.runId).toBe(runId); + + const secretLine = body.entries.find((entry) => entry.message.includes("WEBHOOK_SECRET=")); + expect(secretLine).toBeDefined(); + expect(secretLine?.message.includes("hunter2")).toBe(false); + + expect(body.sources.every((source) => source.exists)).toBe(true); + }); + + test("serves /api/logs end-to-end and degrades without files", async () => { + const server = startDashboardServer({ port: 0, dbPath, workingDir: dir, logDirs: [logDir] }); + try { + const base = `http://127.0.0.1:${server.port}`; + const logs = (await (await fetch(`${base}/api/logs`)).json()) as { available: boolean }; + expect(logs.available).toBe(false); + const bad = await fetch(`${base}/api/logs?level=nope`); + expect(bad.status).toBe(400); + } finally { + server.stop(true); + } + }); +}); + describe("dashboard server", () => { let dir: string; let dbPath: string; diff --git a/packages/code/tests/run-recorder.test.ts b/packages/code/tests/run-recorder.test.ts index aec3932..8dfe1ac 100644 --- a/packages/code/tests/run-recorder.test.ts +++ b/packages/code/tests/run-recorder.test.ts @@ -151,6 +151,21 @@ describe("RunStore", () => { expect(runs.map((r) => r.id).sort()).toEqual([first, second].sort()); }); + test("latestRunByTaskKey returns the newest run per key in one query", () => { + const firstProj = store.createRun({ origin: "task", taskKey: "PROJ-10" }); + store.finishRun(firstProj, "failed"); + const secondProj = store.createRun({ origin: "task", taskKey: "PROJ-10" }); + const otherRun = store.createRun({ origin: "task", taskKey: "OTHER-3" }); + store.createRun({ origin: "pr_mention", repo: "acme/widgets" }); + + const latest = store.latestRunByTaskKey(["PROJ-10", "OTHER-3", "PROJ-10", "UNKNOWN-1"]); + expect(latest.size).toBe(2); + expect(latest.get("PROJ-10")?.id).toBe(secondProj); + expect(latest.get("PROJ-10")?.status).toBe("in_progress"); + expect(latest.get("OTHER-3")?.id).toBe(otherRun); + expect(latest.has("UNKNOWN-1")).toBe(false); + }); + test("run records survive a restart", () => { const id = store.createRun({ origin: "task", taskKey: "PROJ-8" }); store.finishRun(id, "succeeded"); diff --git a/packages/code/tests/worker-capture.test.ts b/packages/code/tests/worker-capture.test.ts new file mode 100644 index 0000000..054bb76 --- /dev/null +++ b/packages/code/tests/worker-capture.test.ts @@ -0,0 +1,119 @@ +import { mkdirSync, readFileSync, rmSync, statSync, writeFileSync } from "fs"; +import { join } from "path"; +import { afterAll, afterEach, describe, expect, test } from "bun:test"; + +import { parseTimestampPrefix } from "../src/lib/worker-logs"; +import { startWorkerCapture, WORKER_CAPTURE_ROTATE_BYTES } from "../src/lib/worker-capture"; + +describe("startWorkerCapture", () => { + let tempDir: string; + const savedConsole = { + log: console.log, + info: console.info, + debug: console.debug, + warn: console.warn, + error: console.error, + }; + + const makeTempDir = (): string => { + const dir = join( + "/tmp", + `devintern-worker-capture-${process.pid}-${Math.random().toString(36).slice(2)}`, + ); + mkdirSync(dir, { recursive: true }); + return dir; + }; + + afterAll(() => { + Object.assign(console, savedConsole); + if (tempDir) { + rmSync(tempDir, { recursive: true, force: true }); + } + }); + + afterEach(() => { + Object.assign(console, savedConsole); + }); + + test("tees console output into the capture files", () => { + tempDir = makeTempDir(); + const capture = startWorkerCapture(tempDir); + try { + console.log("hello out"); + console.info("info out"); + console.error("hello err"); + console.warn("warn err"); + const out = readFileSync(join(tempDir, "worker.stdout.log"), "utf8"); + const err = readFileSync(join(tempDir, "worker.stderr.log"), "utf8"); + expect(out).toContain("hello out"); + expect(out).toContain("info out"); + expect(err).toContain("hello err"); + expect(err).toContain("warn err"); + expect(out).not.toContain("hello err"); + } finally { + capture.stop(); + } + }); + + test("prefixes timestamped lines the dashboard can parse and order", () => { + tempDir = makeTempDir(); + const capture = startWorkerCapture(tempDir); + try { + console.log("with timestamp"); + const first = readFileSync(join(tempDir, "worker.stdout.log"), "utf8").trimEnd(); + for (const line of first.split("\n")) { + expect(parseTimestampPrefix(line)).not.toBeNull(); + } + console.log("second line"); + const second = readFileSync(join(tempDir, "worker.stdout.log"), "utf8").trimEnd(); + const a = parseTimestampPrefix(first.split("\n").at(-1)!)!.timestampMs; + const b = parseTimestampPrefix(second.split("\n").at(-1)!)!.timestampMs; + expect(a <= b).toBe(true); + } finally { + capture.stop(); + } + }); + + test("formats multiple arguments like console does", () => { + tempDir = makeTempDir(); + const capture = startWorkerCapture(tempDir); + try { + console.log("key:", 42, { a: 1 }); + const out = readFileSync(join(tempDir, "worker.stdout.log"), "utf8"); + expect(out).toContain("key: 42 { a: 1 }"); + } finally { + capture.stop(); + } + }); + + test("stop() restores the original console and closes the files", () => { + tempDir = makeTempDir(); + const originalLog = console.log; + const capture = startWorkerCapture(tempDir); + expect(console.log).not.toBe(originalLog); + capture.stop(); + expect(console.log).toBe(originalLog); + + // Writes after stop() must not land anywhere. + const before = statSync(join(tempDir, "worker.stdout.log")).size; + savedConsole.log("not captured"); + expect(statSync(join(tempDir, "worker.stdout.log")).size).toBe(before); + }); + + test("rotates oversized files to .1 on start", () => { + tempDir = makeTempDir(); + writeFileSync(join(tempDir, "worker.stdout.log"), "x".repeat(WORKER_CAPTURE_ROTATE_BYTES + 1)); + const capture = startWorkerCapture(tempDir); + try { + expect(statSync(join(tempDir, "worker.stdout.log.1")).size).toBe( + WORKER_CAPTURE_ROTATE_BYTES + 1, + ); + console.log("fresh"); + const out = readFileSync(join(tempDir, "worker.stdout.log"), "utf8"); + expect(out).toContain("fresh"); + expect(out).not.toContain("xxxxxxxx"); + } finally { + capture.stop(); + } + }); +}); diff --git a/packages/code/tests/worker-init.test.ts b/packages/code/tests/worker-init.test.ts index 48261e7..bcf4697 100644 --- a/packages/code/tests/worker-init.test.ts +++ b/packages/code/tests/worker-init.test.ts @@ -49,6 +49,15 @@ describe("renderSystemdUnit", () => { expect(unit).toContain("Restart=on-failure"); }); + test("does not redirect stdout/stderr — the worker self-captures", () => { + const unit = renderSystemdUnit({ + execPath: "/usr/local/bin/devintern", + projectDir: "/srv/app", + }); + expect(unit).not.toContain("StandardOutput="); + expect(unit).not.toContain("StandardError="); + }); + test("uses the canonical webhook command when webhook mode is chosen", () => { const unit = renderSystemdUnit({ execPath: "devintern", projectDir: "/srv/app", listen: true }); expect(unit).toContain("ExecStart=devintern webhook serve"); diff --git a/packages/code/tests/worker-logs.test.ts b/packages/code/tests/worker-logs.test.ts new file mode 100644 index 0000000..a7a5ea8 --- /dev/null +++ b/packages/code/tests/worker-logs.test.ts @@ -0,0 +1,210 @@ +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { appendFileSync, mkdirSync, rmSync, writeFileSync } from "fs"; +import { join } from "path"; +import { tmpdir } from "os"; + +import { + classifyLevel, + extractTaskKey, + parseTimestampPrefix, + readWorkerLogs, + redactSecrets, + stripAnsi, +} from "../src/lib/worker-logs"; + +describe("worker log parsing", () => { + test("stripAnsi removes color/cursor codes and carriage returns", () => { + expect(stripAnsi("\x1b[31m❌ Failed\x1b[0m\r\nnext")).toBe("❌ Failed\nnext"); + // Bracketed text without a real escape prefix must survive. + expect(stripAnsi("[plain] [0m [info]")).toBe("[plain] [0m [info]"); + }); + + test("parseTimestampPrefix reads ISO prefixes and rejects bare dates/none", () => { + const zulu = parseTimestampPrefix("2026-08-27T12:34:56.789Z hello world"); + expect(zulu?.timestampMs).toBe(Date.parse("2026-08-27T12:34:56.789Z")); + expect(zulu?.rest).toBe("hello world"); + + const offset = parseTimestampPrefix("2026-08-27 12:34:56 +0200 boot"); + expect(offset?.rest).toBe("boot"); + expect(offset?.timestampMs).toBe(Date.parse("2026-08-27T12:34:56+02:00")); + + // Bare dates are not trusted as timestamps. + expect(parseTimestampPrefix("2026-08-27 something happened")).toBeNull(); + expect(parseTimestampPrefix("plain line")).toBeNull(); + }); + + test("classifyLevel prefers explicit markers over the stream default", () => { + expect(classifyLevel("out", "❌ boom")).toBe("error"); + expect(classifyLevel("err", "⚠️ careful")).toBe("warn"); + expect(classifyLevel("out", "ALL CAPS ERROR TEXT")).toBe("error"); + expect(classifyLevel("err", "console.error output")).toBe("error"); + expect(classifyLevel("out", "polling jira")).toBe("info"); + }); + + test("extractTaskKey skips stop-listed prefixes", () => { + expect(extractTaskKey("processed SHA-256 then DEV-99 ok")).toBe("DEV-99"); + expect(extractTaskKey("CVE-2024-1234 alone")).toBeNull(); + expect(extractTaskKey("nothing to see")).toBeNull(); + }); + + test("redactSecrets masks credential assignments and token shapes", () => { + expect(redactSecrets("starting with GITHUB_TOKEN=ghp_abcdef0123456789 done")).toBe( + "starting with GITHUB_TOKEN=[redacted] done", + ); + expect(redactSecrets("raw github ghp_abcdef0123456789012 stays masked")).toBe( + "raw github [redacted] stays masked", + ); + expect(redactSecrets("Bearer eyJhbGciOiJIUzI1Ni.x.y")).toBe("Bearer [redacted]"); + // Unrelated assignments survive untouched. + expect(redactSecrets("WORKER_POLL_INTERVAL=30")).toBe("WORKER_POLL_INTERVAL=30"); + }); +}); + +describe("worker log tailing", () => { + let dir: string; + + beforeEach(() => { + dir = join(tmpdir(), `wlogs-${Date.now()}-${Math.random().toString(36).slice(2)}`); + mkdirSync(dir, { recursive: true }); + }); + + afterEach(() => { + rmSync(dir, { recursive: true, force: true }); + }); + + test("missing capture files degrade to an empty, unavailable result", () => { + const result = readWorkerLogs({ dirs: [dir], limit: 10 }); + expect(result.available).toBe(false); + expect(result.entries).toEqual([]); + expect(result.truncated).toBe(false); + expect(result.sources.every((source) => !source.exists)).toBe(true); + }); + + test("reads both streams, strips ANSI, classifies levels, redacts secrets", () => { + writeFileSync( + join(dir, "worker.stdout.log"), + "✅ Connected\n\x1b[36mDEV-1 started\x1b[0m\n", + "utf8", + ); + writeFileSync( + join(dir, "worker.stderr.log"), + "review failed GITHUB_TOKEN=ghp_secretsecretsecret123\n", + "utf8", + ); + + const result = readWorkerLogs({ dirs: [dir], limit: 100 }); + expect(result.available).toBe(true); + + const messages = result.entries.map((entry) => entry.message); + expect(messages).toContain("✅ Connected"); + expect(messages).toContain("DEV-1 started"); + expect(messages.some((message) => message.includes("ghp_"))).toBe(false); + expect(messages.some((message) => message.includes("[redacted]"))).toBe(true); + + expect(result.entries.find((entry) => entry.message === "DEV-1 started")?.level).toBe("info"); + expect( + result.entries.find((entry) => entry.stream === "err" && entry.level === "error"), + ).toBeDefined(); + + const keyEntry = result.entries.find((entry) => entry.taskKey !== null); + expect(keyEntry?.taskKey).toBe("DEV-1"); + }); + + test("reads are bounded by a byte window and flagged truncated", () => { + const padded = Array.from({ length: 40 }, (_, i) => `${"y".repeat(72)} row-${i}`) + .join("\n") + .concat("\n"); + writeFileSync(join(dir, "worker.stdout.log"), `${padded}tail marker\n`, "utf8"); + + const result = readWorkerLogs({ dirs: [dir], limit: 500, maxBytesPerFile: 256 }); + expect(result.truncated).toBe(true); + + const stdoutSource = result.sources.find((source) => source.stream === "out"); + expect(stdoutSource?.readBytes ?? Number.POSITIVE_INFINITY).toBeLessThanOrEqual(257); + expect(result.entries.length).toBeLessThan(5); + expect(result.entries.at(-1)?.message).toBe("tail marker"); + }); + + test("limit keeps only the newest entries while marking truncation", () => { + writeFileSync(join(dir, "worker.stdout.log"), "first\nsecond\nthird\n", "utf8"); + const result = readWorkerLogs({ dirs: [dir], limit: 2 }); + expect(result.entries.map((entry) => entry.message)).toEqual(["second", "third"]); + expect(result.entries.map((entry) => entry.index)).toEqual([0, 1]); + expect(result.truncated).toBe(true); + }); + + test("level filter narrows the returned window server-side", () => { + writeFileSync( + join(dir, "worker.stdout.log"), + "ok line\n⚠️ warn one\nregular\n❌ fatal one\n⚠️ warn two\n", + "utf8", + ); + const errors = readWorkerLogs({ dirs: [dir], limit: 100, level: "error" }); + expect(errors.entries.map((entry) => entry.message)).toEqual(["❌ fatal one"]); + const warns = readWorkerLogs({ dirs: [dir], limit: 100, level: "warn" }); + expect(warns.entries.map((entry) => entry.message)).toEqual(["⚠️ warn one", "⚠️ warn two"]); + const all = readWorkerLogs({ dirs: [dir], limit: 100 }); + expect(all.entries.length).toBe(5); + }); + + test("interleaves streams round-robin when timestamps are absent", () => { + writeFileSync(join(dir, "worker.stdout.log"), "out a\nout b\n", "utf8"); + writeFileSync(join(dir, "worker.stderr.log"), "err a\nerr b\n", "utf8"); + const result = readWorkerLogs({ dirs: [dir], limit: 100 }); + expect(result.entries.map((entry) => entry.message)).toEqual([ + "out a", + "err a", + "out b", + "err b", + ]); + }); + + test("sorts truly by timestamp when every line carries one", () => { + writeFileSync( + join(dir, "worker.stdout.log"), + "2026-01-01T00:00:01Z out later\n2026-01-01T00:00:03Z out latest\n", + "utf8", + ); + writeFileSync(join(dir, "worker.stderr.log"), "2026-01-01T00:00:02Z err between\n", "utf8"); + const result = readWorkerLogs({ dirs: [dir], limit: 100 }); + expect(result.entries.map((entry) => entry.timestamp)).toEqual([ + Date.parse("2026-01-01T00:00:01Z"), + Date.parse("2026-01-01T00:00:02Z"), + Date.parse("2026-01-01T00:00:03Z"), + ]); + }); + + test("appends are picked up on the next read (tailing)", () => { + const path = join(dir, "worker.stdout.log"); + writeFileSync(path, "before append\n", "utf8"); + expect(readWorkerLogs({ dirs: [dir] }).entries.map((entry) => entry.message)).toEqual([ + "before append", + ]); + appendFileSync(path, "after append\n", "utf8"); + expect(readWorkerLogs({ dirs: [dir] }).entries.map((entry) => entry.message)).toEqual([ + "before append", + "after append", + ]); + }); + + test("deduplicates capture files shared by primary and fallback dirs", () => { + writeFileSync(join(dir, "worker.stdout.log"), "shared line\n", "utf8"); + const result = readWorkerLogs({ dirs: [dir, dir], limit: 100 }); + expect(result.entries.length).toBe(1); + expect(result.sources.length).toBe(2); + }); + + test("unreadable sources report the error without failing the read", () => { + // A directory standing in for the capture file fails on read everywhere + // (EISDIR locally, or on macOS at open), exercising the guarded path. + mkdirSync(join(dir, "worker.stderr.log")); + writeFileSync(join(dir, "worker.stdout.log"), "fine\n", "utf8"); + + const result = readWorkerLogs({ dirs: [dir], limit: 100 }); + expect(result.available).toBe(true); + expect(result.entries.map((entry) => entry.message)).toEqual(["fine"]); + const broken = result.sources.find((source) => source.stream === "err"); + expect(broken?.exists).toBe(true); + expect(broken?.error).not.toBe(""); + }); +}); diff --git a/packages/dashboard-ui/src/App.tsx b/packages/dashboard-ui/src/App.tsx index 2c2710c..fe8e364 100644 --- a/packages/dashboard-ui/src/App.tsx +++ b/packages/dashboard-ui/src/App.tsx @@ -2,14 +2,15 @@ import { useEffect, useState } from "react"; import { StatusStrip } from "@/components/StatusStrip"; import { buttonVariants } from "@/components/ui/button"; +import { LogsView } from "@/views/LogsView"; import { RunDetailView } from "@/views/RunDetailView"; import { RunsView } from "@/views/RunsView"; import { StatsView } from "@/views/StatsView"; import { cn } from "@/lib/utils"; -type Route = { view: "runs" } | { view: "run"; id: number } | { view: "stats" }; +type Route = { view: "runs" } | { view: "run"; id: number } | { view: "stats" } | { view: "logs" }; -/** Parse the location hash (#/, #/runs/:id, #/stats) into a route. */ +/** Parse the location hash (#/, #/runs/:id, #/stats, #/logs) into a route. */ function parseHash(): Route { const hash = window.location.hash; const runMatch = hash.match(/^#\/runs\/(\d+)$/); @@ -19,6 +20,9 @@ function parseHash(): Route { if (hash === "#/stats") { return { view: "stats" }; } + if (hash === "#/logs") { + return { view: "logs" }; + } return { view: "runs" }; } @@ -32,8 +36,9 @@ export function App() { }, []); const tabs = [ - { label: "Runs", hash: "#/", active: route.view !== "stats" }, + { label: "Runs", hash: "#/", active: route.view === "runs" || route.view === "run" }, { label: "Stats", hash: "#/stats", active: route.view === "stats" }, + { label: "Logs", hash: "#/logs", active: route.view === "logs" }, ]; return ( @@ -70,6 +75,7 @@ export function App() { (window.location.hash = "#/")} /> ) : null} {route.view === "stats" ? : null} + {route.view === "logs" ? : null} ); } diff --git a/packages/dashboard-ui/src/lib/api.ts b/packages/dashboard-ui/src/lib/api.ts index 9600f11..f6c8ab1 100644 --- a/packages/dashboard-ui/src/lib/api.ts +++ b/packages/dashboard-ui/src/lib/api.ts @@ -92,6 +92,38 @@ export interface WorkerResponse { dbMissing: boolean; } +export type WorkerLogLevel = "info" | "warn" | "error"; + +export type LogStream = "out" | "err"; + +export interface LogEntry { + index: number; + timestamp: number | null; + level: WorkerLogLevel; + stream: LogStream; + message: string; + taskKey?: string | null; + runId?: number; + runStatus?: RunStatus; +} + +export interface LogSourceInfo { + path: string; + stream: LogStream; + exists: boolean; + totalBytes: number; + readBytes: number; + truncated: boolean; + error: string; +} + +export interface LogsResponse { + available: boolean; + entries: LogEntry[]; + sources: LogSourceInfo[]; + truncated: boolean; +} + export interface PollState { data: T | null; error: string | null; diff --git a/packages/dashboard-ui/src/lib/log-view.test.ts b/packages/dashboard-ui/src/lib/log-view.test.ts new file mode 100644 index 0000000..c60946a --- /dev/null +++ b/packages/dashboard-ui/src/lib/log-view.test.ts @@ -0,0 +1,49 @@ +import { expect, test } from "bun:test"; + +import type { LogEntry } from "@/lib/api"; +import { levelDotClass, matchesQuery, severityCounts } from "@/lib/log-view"; + +function entry(partial: Partial): LogEntry { + return { + index: 0, + timestamp: null, + level: "info", + stream: "out", + message: "", + taskKey: null, + ...partial, + }; +} + +test("matchesQuery is a case-insensitive substring match", () => { + const line = entry({ message: "❌ Run PROJ-1 failed: git push rejected" }); + expect(matchesQuery(line, "PROJ-1")).toBe(true); + expect(matchesQuery(line, "proj-1")).toBe(true); + expect(matchesQuery(line, "rejected")).toBe(true); + expect(matchesQuery(line, "approved")).toBe(false); +}); + +test("matchesQuery treats blank/whitespace queries as match-all", () => { + const line = entry({ message: "anything" }); + expect(matchesQuery(line, "")).toBe(true); + expect(matchesQuery(line, " ")).toBe(true); +}); + +test("severityCounts tallies errors and warnings only", () => { + const counts = severityCounts([ + entry({ level: "error" }), + entry({ level: "warn" }), + entry({ level: "info" }), + entry({ level: "error" }), + ]); + expect(counts).toEqual({ error: 2, warn: 1 }); +}); + +test("severityCounts of an empty window is zeroed", () => { + expect(severityCounts([])).toEqual({ error: 0, warn: 0 }); +}); + +test("levelDotClass distinguishes severities", () => { + expect(levelDotClass("error")).not.toBe(levelDotClass("warn")); + expect(levelDotClass("info")).not.toBe(levelDotClass("error")); +}); diff --git a/packages/dashboard-ui/src/lib/log-view.ts b/packages/dashboard-ui/src/lib/log-view.ts new file mode 100644 index 0000000..04b9886 --- /dev/null +++ b/packages/dashboard-ui/src/lib/log-view.ts @@ -0,0 +1,48 @@ +/** + * Client-side helpers for the Logs view: search matching and severity counts + * over the entries already served by `/api/logs` (the server bounds the + * window; filtering here never refetches). Pure, for unit testing. + */ + +import type { LogEntry, WorkerLogLevel } from "@/lib/api"; + +/** Case-insensitive substring match over the rendered line; empty query matches all. */ +export function matchesQuery(entry: Pick, rawQuery: string): boolean { + const query = rawQuery.trim().toLowerCase(); + if (!query) { + return true; + } + return entry.message.toLowerCase().includes(query); +} + +export interface SeverityCounts { + error: number; + warn: number; +} + +/** Errors/warnings in a window of entries — triage summary for the header. */ +export function severityCounts(entries: Pick[]): SeverityCounts { + return entries.reduce( + (counts, entry) => { + if (entry.level === "error") { + counts.error += 1; + } else if (entry.level === "warn") { + counts.warn += 1; + } + return counts; + }, + { error: 0, warn: 0 }, + ); +} + +/** Tailwind classes for the severity marker next to each line. */ +export function levelDotClass(level: WorkerLogLevel): string { + switch (level) { + case "error": + return "bg-destructive"; + case "warn": + return "bg-chart-5"; + default: + return "bg-muted-foreground/40"; + } +} diff --git a/packages/dashboard-ui/src/views/LogsView.tsx b/packages/dashboard-ui/src/views/LogsView.tsx new file mode 100644 index 0000000..d1cd9a4 --- /dev/null +++ b/packages/dashboard-ui/src/views/LogsView.tsx @@ -0,0 +1,231 @@ +import { useEffect, useMemo, useRef, useState } from "react"; + +import { EmptyState, FilterGroup } from "@/components/shared"; +import { Badge } from "@/components/ui/badge"; +import { Card } from "@/components/ui/card"; +import { usePoll } from "@/lib/api"; +import type { LogsResponse, WorkerLogLevel } from "@/lib/api"; +import { levelDotClass, matchesQuery, severityCounts } from "@/lib/log-view"; +import { cn, formatTime } from "@/lib/utils"; + +type LevelFilter = WorkerLogLevel | "all"; + +const LEVEL_FILTERS = ["all", "warn", "error"] as const; + +/** + * Rows rendered initially and added per "Show older" click. The server window + * is bounded by the API limit, but at 1000 entries every 5s poll would + * re-render/reconcile the full list; the cap keeps the DOM small and grows + * only when the user asks for older history. + */ +const RENDER_STEP = 400; + +function levelFilterLabel(level: LevelFilter): string { + if (level === "all") { + return "everything"; + } + return level === "warn" ? "warnings" : "errors"; +} + +/** Global worker log stream, tailed from the machine running the worker. */ +export function LogsView() { + const [level, setLevel] = useState("all"); + const [query, setQuery] = useState(""); + const [following, setFollowing] = useState(true); + + const { data, error } = usePoll(`/api/logs?level=${encodeURIComponent(level)}`); + + const scrollRef = useRef(null); + const followRef = useRef(following); + followRef.current = following; + + const visible = useMemo( + () => (data?.entries ?? []).filter((entry) => matchesQuery(entry, query)), + [data, query], + ); + const counts = useMemo(() => severityCounts(data?.entries ?? []), [data]); + + const [renderCap, setRenderCap] = useState(RENDER_STEP); + // Newest entries stay pinned to the bottom (follow mode), so the cap hides + // the oldest ones. Filters change what matters, so restart the window there. + const rendered = useMemo( + () => (visible.length > renderCap ? visible.slice(visible.length - renderCap) : visible), + [visible, renderCap], + ); + const hiddenCount = visible.length - rendered.length; + + useEffect(() => { + setRenderCap(RENDER_STEP); + }, [query, level]); + + useEffect(() => { + if (!followRef.current || !scrollRef.current) { + return; + } + scrollRef.current.scrollTop = scrollRef.current.scrollHeight; + }, [rendered]); + + const onScroll = (): void => { + const el = scrollRef.current; + if (!el) { + return; + } + setFollowing(el.scrollHeight - el.scrollTop - el.clientHeight < 48); + }; + + const jumpToEnd = (): void => { + setFollowing(true); + const el = scrollRef.current; + if (el) { + el.scrollTop = el.scrollHeight; + } + }; + + return ( +
+
+
+ + setQuery(event.target.value)} + placeholder="Search this window…" + aria-label="Search log entries" + className="w-56 rounded-md border bg-background px-2.5 py-1 text-sm outline-none placeholder:text-muted-foreground focus-visible:border-ring" + /> +
+
+ + {counts.error} error{counts.error === 1 ? "" : "s"} · {counts.warn} warning + {counts.warn === 1 ? "" : "s"} + + {!following ? ( + + ) : null} +
+
+ + {error ? : null} + + {data && !data.available ? ( + + ) : null} + + {data && data.available && visible.length === 0 ? ( + + ) : null} + + {data && data.available && visible.length > 0 ? ( + + {hiddenCount > 0 ? ( + + ) : null} +
+ {rendered.map((entry) => ( +
+ + + {entry.timestamp === null ? "–" : formatTime(entry.timestamp)} + + + {entry.message} + + {entry.runId ? ( + + {entry.taskKey ? ( + {entry.taskKey} + ) : ( + `run ${entry.runId}` + )} + + ) : entry.taskKey ? ( + + {entry.taskKey} + + ) : null} +
+ ))} +
+
+ ) : null} + + {data && data.available ? ( +
+ {query ? `Matching ${visible.length} of ` : ""} + {data.entries.length} + {data.entries.length === 1 ? " entry" : " entries"} read from disk. + {hiddenCount > 0 ? ` Showing the newest ${rendered.length}.` : ""} + {data.truncated ? " Older history beyond this tail window is not loaded." : null} + {data.sources.some((source) => source.error) ? ( + <> + {" "} + Unreadable:{" "} + {data.sources + .filter((source) => source.error) + .map((source) => `${source.path} (${source.error})`) + .join(", ")} + . + + ) : null} +
+ ) : null} +
+ ); +}