Add attempt-scoped Experiment readouts

[Cedric Vidal (cedricvidal)]

Summary

Adds the built-in aggregate readout for canonical Experiments, carrying forward the useful #1053/#1495 implementation while replacing obsolete Submission identity and contracts.

• Processes the immutable request/run vector for an exact experimentId + experimentExecutionId and preserves append-only JSON, derived Markdown, blob artifacts, stale/in-flight checks, cooperative supersession, and final atomic CAS publication.
• Treats failed coding outcomes and passed:false observations as valid evidence; blocks only failed/dependency-blocked required observations while ignoring unrelated handler failures.
• Adds exact-execution API, CLI, and Portal parity for report listing, structured artifacts, logs, retries, attempt selection, browser history, and deep links.
• Organizes Experiment Detail into compact, URL-addressable Runs / Reports / Processing tabs directly below the global attempt selector. Runs uses the inset shared DataTable, Reports renders one persisted report type/version at a time with latest-safe defaults and compact history controls, and Processing owns handler progress, diagnostics, and retry/recovery. Completed, processing, blocked, failed, and no-artifact reports share one stable panel shell so identity, selectors, badges, and actions do not reflow when only the body state changes.
• Keeps public terminology generic (Reports) and retains pp-readout only as the internal handler identifier.
• Updates OpenAPI, architecture docs, deployment wiring, Storybook, and all affected static prompt inventory/adapters/datasets/contracts/rubrics.
• Deliberately excludes Experiment report-template authoring, configuration, and triggering; those remain follow-up scope.

Demos

Before

N/A — this adds the first built-in Experiment report surface.

After

Two-take, attention-cued walkthrough of the real host Vite /experiments/:id route with deterministic browser-level API mocks: normal Scope shell/navigation, compact global attempt selector, above-the-fold Runs / Reports / Processing tabs, stable completed/failed report controls with body-only state changes, disabled unavailable artifact actions, inset exact-runs DataTable, persisted report types/history, and retry through processing to recovered publication.

experiment-report-stable-shell.mp4

Testing

• pnpm build — passed across all 26 workspace projects.
• pnpm test — 281 files / 3446 tests passed; one unrelated scripts/build-windows-worker.test.ts PowerShell subprocess timed out. The single targeted retry, pnpm exec vitest run scripts/build-windows-worker.test.ts, passed all 8 tests.
• pnpm vitest run apps/portal/src/components/ExperimentReportCard.test.tsx apps/portal/src/pages/ExperimentDetail.test.tsx apps/portal/src/pages/ExperimentDetail.play.test.tsx apps/portal/src/components/ui/stories.play.test.tsx --reporter=dot — passed, 60/60 focused Experiment Detail/report shell unit/play/Storybook tests.
• pnpm --filter portal build — passed.
• pnpm --filter portal build-storybook --quiet — passed with stable completed, processing, blocked, failed, no-artifact, retry/error, report-history, multiple-report-type, stale-link, and exact-attempt stories.
• pnpm eval:static-prompts:validate-data — passed; 223 approved cases and 16 quality targets validated.
• pnpm --filter static-prompt-evals typecheck — passed.
• pnpm --filter static-prompt-evals test — passed; 33 TypeScript and 94 Python tests.
• pnpm eval:static-prompts:smoke — passed in offline smoke mode (execution check only, not a production quality acceptance).
• pnpm --dir website test — passed.
• cd website && pnpm run build — passed; documentation production build completed.
• Focused API OpenAPI and CLI command snapshot tests — passed; generated website OpenAPI was refreshed.
• Terminology/scope audit found no Submission identity or Experiment report-template authoring/configuration surface in the readout implementation.
• Replacement walkthrough was recorded twice from the real host Vite route on exact pushed head 0e1a4d77ecd036d960117899e80159cd241831b5; take 2 validated as H.264/yuv420p, 1440×900, 25 fps, 39.16 s, fast-start. Full decode passed, the MP4 was opened, and representative completed, failed, processing, and recovered frames were inspected.
• pnpm lint — unavailable because this branch/base has no installed or declared eslint executable (eslint: command not found); TypeScript builds passed.
• Container validation — not completed. The full-stack build and one targeted reduced-stack retry both failed before services launched while the shared container VM committed image layers (no space left on device). Per the bounded retry rule, no further attempt was made. Project-scoped teardown then verified zero containers, volumes, and networks for scope-mt-app-8-cedricvidal-refactored-system; therefore there is no Compose five-run E2E evidence for this PR.

Documentation and compatibility

Updated docs/architecture/experiments.md, docs/architecture/post-processing.md, docs/architecture/prompt-evaluations.md, docs/architecture/overview.md, website/src/content/docs/guides/creating-experiments.mdx, ENV_VARIABLES.md, generated OpenAPI, and Compose/deployment wiring.

The new report APIs and CLI commands use exact Experiment execution identity. Generic report list parsers continue to accept the legacy { items, nextCursor } envelope as a compatibility fallback while preferring { data, nextCursor }. Portal links preserve executionId, tab, reportType, and reportId, with safe fallback for stale selections. No database migration or report-template contract is introduced.

Checklist

• If Portal features changed, keep CLI capabilities in sync.
• If Portal components changed, update their Storybook stories.
• If database changes require a migration, include up() / down() and keep it CosmosDB-compatible. (N/A — no migration required.)
• If dependencies changed, update the lockfile and regenerate NOTICE / NOTICE-REVIEW.txt with pnpm notice as needed. (N/A — no new third-party dependency.)
• Video showing the behavior before the suggested change (N/A — new feature.)
• Video showing the behavior after the suggested change

[github-actions]

Test Results (Node.js 22)

test: Run #281

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
3478	3478	0	0	0	0	0	1m13s

🎉 All tests passed!

Github Test Reporter

[github-actions]

Test Results (Node.js 22)

test: Run #289

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
3483	3483	0	0	0	0	0	1m45s

🎉 All tests passed!

Github Test Reporter

[github-actions]

Test Results (Node.js 22)

test: Run #292

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
3485	3485	0	0	0	0	0	1m49s

🎉 All tests passed!

Github Test Reporter

[github-actions]

Test Results (Node.js 22)

test: Run #296

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
3483	3483	0	0	0	0	0	1m47s

🎉 All tests passed!

Github Test Reporter