Fix task-state availability and production diagnostics

.papercuts/troubleshooting.md

@@ -1,5 +1,8 @@
 # Troubleshooting
 
+- OpenCode API message dumps can exceed the CLI's output limit and become truncated JSON; use `GET /api/session/{id}/message?limit=1` for the latest completed review, or bounded pagination, rather than dumping every tool result. Verify the returned assistant model metadata when exact-model reviews are required.
+- Provider diagnostics must classify `finalized` inside the Pi harness before `closedFailureMessage` replaces the raw error. Reclassifying `runtimeOutcome.finalMessage` in `llm-client` loses model-unavailable/authentication evidence; assert the emitted production event with a real faux-provider harness test.
+
 - `.papercuts/` is ignored even when its troubleshooting file is present in the PR branch, so persisting a required update needs an explicit `git add -f`.
 - Layout stabilization must race `animation.finished` against a short timeout because paused or infinite document animations never settle; keep geometry polling as the authoritative E2E readiness check.
 - Pi 0.80.10 can choose the oldest oversized user turn as `firstKeptEntryId`, leaving both summary inputs empty and producing a no-op checkpoint. When the journal has a newer turn, retry `prepareCompaction` with a minimal retained-tail budget; still refuse the checkpoint if both summary inputs remain empty.

docs/plans/logging-and-diagnostics-upgrade-plan.md

@@ -27,6 +27,28 @@ failure vocabulary. No upload path was added.
 
 ## Verification evidence
 
+### September 2026 production-cause hardening
+
+- Main diagnostics retain only closed error/cause categories, validated structural
+  HTTP status, and bounded fingerprints; they never persist raw request/response
+  text. Cause traversal is bounded, cycle-aware, and excludes proxies/accessors.
+  Fingerprints group the closed error/cause/status tuple rather than call sites:
+  V8 stack accessors may execute custom formatting, so stack materialization is
+  deliberately excluded. Provider terminal failures count as failed health;
+  they previously contributed to the degraded bucket through the legacy logger.
+- Renderer exceptions use `renderer-exception`, with script/promise/React/route
+  phases; actual renderer process death retains `renderer-crashed`. Abort errors
+  are counted as cancellation rather than failure.
+- Provider failure classification runs in the harness before outcome redaction.
+  `model_unavailable` is diagnostic-only; the existing portable provider-failure
+  DTO and native client presentation remain compatible. MCP discovery projects
+  the caught error structurally instead of interpolating it into a log string.
+- Task reads and generations distinguish neutral `todo-storage-disabled` evidence
+  from `todo-snapshot-invalid`; policy-disabled tracking does not inflate degraded
+  health counts. Events contain neither chat identifiers nor task content.
+- Historical opaque errors remain undiagnosed where the old journal discarded
+  their causes. These changes improve future evidence, not retrospective certainty.
+
 The implementation has passed the repository's full desktop test command, the
 focused diagnostic contract/policy suite, TypeScript and E2E type checks, ESLint,
 the standard Electron E2E matrix, the isolated production-profile diagnostics

docs/plans/rpiv-todo-integration-plan.md

@@ -14,10 +14,10 @@ The extension is deliberately excluded from Assistant mode, Bots, Telegram/mobil
 
 1. `main/services/rpiv-todo/contract.ts` defines the closed version-1 snapshot and parameter contract, terminal/control-character sanitization, descriptor-safe plain-JSON checks, hard byte/count/depth limits, dependency DAG validation, and the one-`in_progress` invariant.
 2. `main/services/rpiv-todo/reducer.ts` owns create, update, list, get, tombstone delete, and clear. Validation failures are successful in-band tool results with an unchanged complete snapshot, so the journal remains replayable. Completed tasks cannot reopen; deleted tasks remain tombstones until clear; dependency references are preserved.
-3. `main/services/rpiv-todo/replay.ts` scans the current Pi branch for the newest todo tool result. A malformed newest result fails closed and disables todo for the chat; it never regresses to an older valid state. Compaction entries do not become a second state authority.
+3. `main/services/rpiv-todo/replay.ts` scans the entire current Pi branch in oldest-to-newest order, selects the newest non-`isError` todo tool result, and validates only that authoritative full snapshot. Older malformed snapshots are superseded; a malformed newest checkpoint fails closed and disables todo for the chat, never regressing to an older valid state. Dispatch/schema error results are skipped, and branch read or iteration failures propagate unchanged. Compaction entries do not become a second state authority.
 4. `main/services/rpiv-todo/extension.ts` contributes a generation-local native Pi tool and guidance. Replay policy is `safe`: mutation exists only in the generation closure until the full result is durably journaled, so a crash retry cannot duplicate durable state. Renderer publication waits for a durable `toolResult` `message_end` runtime event.
-5. `main/services/llm-client.ts` opens the private chat session before freezing runtime contributions, replays todo state, requires an explicitly classified chat usage source, publishes the initial projection, and sends later projections only after journal durability. Verified corrupt replay immediately publishes the content-free unavailable projection.
-6. `main/handlers/chats.ts` exposes an owner-fenced `chats:todoSnapshot` read. It rechecks the exact renderer document after asynchronous work. Corrupt todo journals return a content-free unavailable state; other storage errors remain errors.
+5. `main/services/llm-client.ts` opens the private chat session before freezing runtime contributions and shares `loadDurableTodoSnapshot` with the chat-open read. Todo requires an explicitly classified chat usage source and a durable session: journalless generations omit the tool and publish `storage_not_enabled`, never an ephemeral ready list. Durable generations publish the initial projection and later projections only after journal durability. Verified corrupt replay publishes the content-free `invalid_snapshot` projection. Tool results are validated against the reader contract before generation-local mutation.
+6. `main/handlers/chats.ts` exposes an owner-fenced `chats:todoSnapshot` read. It rechecks the exact renderer document after asynchronous work. Storage-disabled chats and invalid snapshots have distinct, closed reasons; other storage errors remain errors. The renderer explains disabled storage without implying corruption, and older unavailable DTOs retain the verification-failure presentation.
 7. `renderer/shared/todo.ts` is the only renderer projection. Its strict versioned allowlist contains `id`, `subject`, `status`, `activeForm`, and `blockedBy`. Tool arguments/results, descriptions, ownership, metadata, and journal structure remain private.
 8. `renderer/lib/ipc.ts` validates both snapshot reads and stream notifications, and fences notifications by generation stream and chat id. A local live-snapshot revision fence prevents a slow initial read from replacing newer generation state. `renderer/components/todo-panel.tsx` renders a zero-layout-height elevated chip anchored above the footer and a portal-backed, headerless hover/focus task list with semantic Aiden tokens, per-task screen-reader status, bounded polite progress/unavailable announcements, and reduced-motion behavior. `ScrollArea` raises its centered scroll-to-bottom control only while this overlay is visible, so the two controls never share a hit target. Fully completed plans retain only the live-region completion announcement.
 9. `main/services/generation-timeline.ts` exposes only the content-free activity label “Update task list.”
@@ -37,7 +37,7 @@ The extension is deliberately excluded from Assistant mode, Bots, Telegram/mobil
 `npm run test:todo` is registered in `pretest` and covers:
 
 - strict contract parsing, sanitization, size limits, graph invariants, transitions, tombstones, and unchanged in-band error snapshots;
-- branch replay, compaction survival, no-snapshot initialization, and fail-closed newest-result corruption;
+- branch replay, compaction survival, no-snapshot initialization, superseded corruption, fail-closed newest-checkpoint corruption, skipped dispatch errors, and propagated branch read/iteration failures;
 - admission fencing, per-generation state isolation, cancellation, replay policy, and real harness coverage proving publication follows successful durable append and never follows append failure;
 - closed renderer projection and unavailable-state parsing;
 - slow-initial-read versus live-snapshot ordering and immediate corrupt-replay unavailability;

docs/testing/pi-compaction-phase7-rollout-gates.md

@@ -3,7 +3,7 @@
 Status: automated evaluation and signed development-package acceptance pass;
 installed production and credentialed-provider evidence remains **Pending** until
 the release owner runs the steps below against the installed candidate.
-Generation is never blocked by the rollout gates at any stage:
+Valid rollout-stage ineligibility does not block generation:
 rollout-ineligible chats generate **journalless** over an in-memory session
 instead of failing (see "Journalless generation" below), so advancing the
 stage is a durability decision, not an availability one.
@@ -48,6 +48,25 @@ npm run pi-upgrade:advance -- migrated_low_risk_chats
 
 Repeat only after observing the current stage and completing the next cohort's acceptance. The command cannot skip or regress a stage, validates the evaluation receipt against the complete signed `.app` digest, and requires the installed receipt for `v4_only`.
 
+Every policy read reloads and validates the device document, so an operator's
+successful CLI advance is visible to subsequent eligibility checks in the
+running app without a restart. A generation already running keeps its selected
+session and per-run behavior flags; send a new turn to exercise the new stage.
+Reads never advance the stage. A missing policy is initialized only before that
+store has observed a valid policy, using atomic, no-overwrite publication;
+concurrent creators use the winning document. Malformed or schema-invalid
+replacements reject rather than falling back to a cached permissive policy, and
+a policy removed after a successful read also rejects. These failures are not
+ordinary cohort ineligibility and can block the operation. Restoring a valid
+document lets the next read recover without restarting or rewriting that data.
+
+`activatedAt` is the new-chat cohort cutoff: first policy creation establishes
+it for the production `new_chats` default, or an explicit advance into
+`new_chats` establishes it when starting from an earlier stage. Later advances
+preserve it so a previously eligible new chat does not lose compaction/memory
+eligibility merely because it has grown beyond 100 messages. Existing policy
+timestamps are used as stored; no historical cutoff is inferred or rewritten.
+
 ## Rollback
 
 Set `AIDEN_PI_UPGRADE_BEHAVIOR_ENABLED=0` before app startup and restart Aiden. This disables new v4 journal creation, legacy migration, automatic/manual Pi checkpoint generation, and durable-memory retrieval or writes. Existing v4 journals remain readable and are not downgraded or rewritten. With the journalless safety net, chats that have no existing v4 journal continue to generate in the rollback environment — journalless over an in-memory session — rather than failing; durable compaction, memory, and history recall stay disabled for them until the override is removed and the persisted rollout stage resumes. Remove the override and restart to resume the persisted rollout stage.
@@ -73,8 +92,9 @@ It engages exactly when a chat cannot yet hold a durable journal:
 
 Semantics of a journalless run: the request path is identical, visible turns
 persist through the chat store exactly as with journaled runs, but VCC history
-recall is omitted (nothing durable to recall), todo replay reads the empty
-in-memory journal (the todo panel reports the snapshot unavailable),
+recall is omitted (nothing durable to recall), the durable todo tool is omitted
+and both chat-open and live snapshots report `storage_not_enabled` (the panel
+explains that saved task tracking is not enabled for this chat),
 automatic and manual Pi checkpoints stay cohort-disabled, effect-recovery
 boundaries are written only in-process and never acknowledged as durable, and
 the durable store can never be quarantined by an in-memory failure.

main/handlers/chats.test.ts

@@ -2,7 +2,7 @@ import assert from "node:assert/strict";
 import { readFileSync } from "node:fs";
 import test from "node:test";
 
-test("the todo snapshot handler probes rollout eligibility and never mints journals", () => {
+test("the todo snapshot handler shares durable admission and preserves the owner fence", () => {
   const handlers = readFileSync(new URL("./chats.ts", import.meta.url), "utf8");
   const snapshot = handlers.slice(
     handlers.indexOf('ipcMain.handle("chats:todoSnapshot"'),
@@ -11,6 +11,6 @@ test("the todo snapshot handler probes rollout eligibility and never mints journ
   assert.ok(snapshot.includes('ipcMain.handle("chats:todoSnapshot"'));
   assert.match(snapshot, /openChatIfEligible/u);
   assert.doesNotMatch(snapshot, /openChat\(/u);
-  assert.match(snapshot, /unavailableTodoSnapshot\(chatId\)/u);
-  assert.match(snapshot, /!opened\.session/u);
-});
+  assert.match(snapshot, /loadDurableTodoSnapshot\(chatId, opened\.session\)/u);
+  assert.match(snapshot, /await loadDurableTodoSnapshot[\s\S]*owner\.isDestroyed\(\)[\s\S]*return snapshot/u);
+});

main/handlers/chats.ts

@@ -62,11 +62,9 @@ import {
 import { botApplicationService } from "../services/bot-application-service-main.js";
 import { piCompactionSessionStore } from "../services/pi-compaction-session-store.js";
 import { memoryStore } from "../services/memory-store-main.js";
-import { isTodoSnapshotFailure, replayTodoState } from "../services/rpiv-todo/replay.js";
-import {
-  todoSnapshotForRenderer,
-  unavailableTodoSnapshot,
-} from "../../renderer/shared/todo.js";
+import { loadDurableTodoSnapshot } from "../services/rpiv-todo/snapshot.js";
+import { todoSnapshotDiagnostic } from "../services/rpiv-todo/diagnostics.js";
+import { writeDiagnosticEvent } from "../services/diagnostic-journal.js";
 
 function asString(value: unknown, name: string): string {
   if (typeof value !== "string" || value.length === 0) {
@@ -156,20 +154,11 @@ export function registerChatHistoryHandlers(): void {
     }
     if (owner.isDestroyed()) throw new Error("The renderer document is no longer active.");
     const opened = await piCompactionSessionStore.openChatIfEligible(chatId, chat);
-    if (!opened.session) {
-      // Rollout-ineligible chats have no durable journal to replay, so todo is
-      // unavailable exactly like a corrupt journal. Never mint a journal here.
-      return unavailableTodoSnapshot(chatId);
-    }
-    try {
-      const snapshot = todoSnapshotForRenderer(chatId, await replayTodoState(opened.session));
-      if (owner.isDestroyed()) throw new Error("The renderer document is no longer active.");
-      return snapshot;
-    } catch (error) {
-      if (owner.isDestroyed()) throw new Error("The renderer document is no longer active.");
-      if (!isTodoSnapshotFailure(error)) throw error;
-      return unavailableTodoSnapshot(chatId);
-    }
+    const { snapshot } = await loadDurableTodoSnapshot(chatId, opened.session);
+    if (owner.isDestroyed()) throw new Error("The renderer document is no longer active.");
+    const diagnostic = todoSnapshotDiagnostic(snapshot);
+    if (diagnostic) writeDiagnosticEvent(diagnostic);
+    return snapshot;
   });
 
   ipcMain.handle("chats:waitUntilIdle", async (_event, id: unknown) =>

main/handlers/diagnostics.ts

@@ -12,7 +12,7 @@ import {
 import { app, BrowserWindow, dialog, ipcMain, shell } from "../platform.js";
 import { currentRuntimeProfile } from "../runtime-profile.js";
 import { writeDiagnosticEvent } from "../services/diagnostic-journal.js";
-import type { DiagnosticEventName } from "../services/diagnostics-contract.js";
+import { rendererDiagnosticClassification, type DiagnosticEventName } from "../services/diagnostics-contract.js";
 import { rendererDocumentOwner } from "../services/renderer-document-owner.js";
 import { createRendererDiagnosticRateLimiter } from "../services/renderer-diagnostic-rate.js";
 import {
@@ -110,10 +110,12 @@ export function registerDiagnosticHandlers(): void {
       level: report.suppressed ? "warn" : "error",
       area: "renderer",
       event: RENDERER_EVENT_NAMES[report.kind],
-      outcome: report.suppressed ? "degraded" : "failed",
-      code: "renderer-crashed",
+      ...rendererDiagnosticClassification(report.errorType, report.suppressed),
       fields: {
         errorType: report.errorType,
+        failurePhase: report.kind === "global-error" ? "renderer-script"
+          : report.kind === "unhandled-rejection" ? "renderer-promise"
+            : report.kind === "route-error" ? "renderer-route" : "renderer-react",
         rendererContext: report.context,
         referenceId: durableReferenceId,
         suppressed: report.suppressed ?? 0,

main/services/diagnostic-journal.test.ts

@@ -75,6 +75,40 @@ test("production journal strips development-only fields from typed callers", asy
   });
 });
 
+test("legacy logging safely classifies cancellation and ignores hostile proxy errors", async () => {
+  for (const profile of ["production", "development"] as const) await withJournal(profile, async () => {
+    const revoked = Proxy.revocable({}, {});
+    revoked.revoke();
+    assert.doesNotThrow(() => writeLegacyDiagnostic("error", "pi", [revoked.proxy]));
+    const inheritsProxy = Object.create(revoked.proxy);
+    assert.doesNotThrow(() => writeLegacyDiagnostic("error", "pi", [inheritsProxy]));
+    assert.doesNotThrow(() => writeLegacyDiagnostic("error", "pi", [{
+      toJSON() { throw new Error("private"); },
+      toString() { throw new Error("private"); },
+    }]));
+    const event = writeLegacyDiagnostic("error", "pi", [new DOMException("private", "AbortError")]);
+    assert.equal(event.code, "cancelled");
+    assert.equal(event.outcome, "cancelled");
+    assert.equal(writeLegacyDiagnostic("warn", "pi", [{ status: 503 }]).code, undefined);
+    assert.equal(writeLegacyDiagnostic("warn", "pi", [{ name: "AbortError" }]).code, undefined);
+  });
+});
+
+test("production legacy adapter retains structural SDK causes without serializing envelopes", async () => {
+  await withJournal("production", async (target) => {
+    const event = writeLegacyDiagnostic("warn", "mcp", [
+      "Skipping private server", { cause: { code: "ECONNREFUSED", status: 503 },
+        message: "private-prompt", headers: { authorization: "private-auth" }, url: "https://private-endpoint", task: "private-task" },
+    ]);
+    assert.equal(event.event, "mcp-degraded");
+    assert.equal(event.code, "network-failed");
+    assert.equal(event.fields?.causeCode, "network-failed");
+    assert.equal(event.fields?.httpStatus, 503);
+    await flushDiagnosticJournal();
+    assert.doesNotMatch(await fs.readFile(target, "utf8"), /private-/);
+  });
+});
+
 test("journal enforces owner-only modes", async () => {
   await withJournal("production", async (target, dir) => {
     await flushDiagnosticJournal();