Skip to content

feat: add Experiment lifecycle recovery - #1517

Draft
Cedric Vidal (cedricvidal) wants to merge 3 commits into
cedricvidal-experiment-report-templatesfrom
cedricvidal-experiment-lifecycle
Draft

Cedric Vidal (cedricvidal) wants to merge 3 commits into
cedricvidal-experiment-report-templatesfrom
cedricvidal-experiment-lifecycle

Conversation

@cedricvidal

Copy link
Copy Markdown
Collaborator

Summary

Adds recoverable Experiment deletion and restore as the fifth stacked Experiment successor PR.

  • Persists lifecycle identity, phase, CAS version, exact chunk-backed request/report target snapshots, progress, blockers, and deletion provenance.
  • Freezes Experiment mutation and publication before cascading; queue claims and worker final writes reject deleted or lifecycle-active records.
  • Restores only children owned by the exact operation, preserves independently deleted records, and rolls to a fresh execution when the restored request/run vector drifted.
  • Repairs interrupted operations from dispatcher polling, isolates per-Experiment repair failures, and rotates blocked work fairly.
  • Adds project-scoped REST, CLI, Portal, Storybook, migration, OpenAPI, and architecture/website documentation surfaces.

Stacked on #1516.

Demos

Before

N/A — Experiment lifecycle management did not previously exist.

After

Lifecycle list visibility, deletion progress, retained deleted state, child-first restore, and reconciled active state:

experiment-lifecycle.mp4

Deletion progress:

Experiment deletion progress

Restore progress:

Experiment restore progress

Testing

  • pnpm test — passed 294 files / 3,528 tests before final review fixes.
  • pnpm test — passed 293 files / 3,529 tests after final review fixes; one unrelated Windows PowerShell test timed out.
  • pnpm exec vitest run scripts/build-windows-worker.test.ts — passed 8/8 on immediate isolated rerun.
  • Consolidated lifecycle coverage — passed 130/130 tests before final review; focused lifecycle/coordinator/migration regressions passed 18/18 after review fixes.
  • Production builds passed for shared, telemetry, db-migrations, API, CLI, Portal, readout-handler, report-generator, Storybook, and documentation website.
  • OpenAPI generation/snapshot, Storybook play coverage, website tests/build, and git diff --check passed.
  • Compose smoke attempted once with the isolated worktree project; blocked before startup by host Podman ENOSPC at /var/tmp/libpod_builder…. No prune, delete, or retry was performed.
  • Walkthrough take 2 validated as H.264/yuv420p, 1440×900, 25 fps, faststart; full decode and representative encoded-frame inspection passed.

Documentation and compatibility

  • Updated docs/architecture/experiments.md and docs/architecture/app-design.md.
  • Updated the website Experiment guide and regenerated the committed OpenAPI document.
  • Migration 036 adds Cosmos-compatible lifecycle polling and deletion-provenance indexes.
  • Existing active Experiment behavior remains compatible; deleted records remain hidden unless includeDeleted / Show deleted is explicit.

Checklist

  • If Portal features changed, keep CLI capabilities in sync.
  • If Portal components changed, update their Storybook stories.
  • If database changes require a migration, include up() / down() and keep it CosmosDB-compatible.
  • If dependencies changed, update the lockfile and regenerate NOTICE / NOTICE-REVIEW.txt with pnpm notice as needed. (N/A — no dependency changes.)
  • Video showing the behavior before the suggested change (N/A — new capability.)
  • Video showing the behavior after the suggested change

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e767814e-3331-4a7e-ae96-15207b53ac9c
@github-actions github-actions Bot added type: documentation Documentation additions, corrections, and improvements. area: worker Coding agent worker runtime, task execution, and lifecycle. area: cli Scope command-line interface and terminal workflows. area: infrastructure Cloud resources, hosting, networking, and shared infrastructure. language: javascript Work involving JavaScript code, tooling, or dependencies. area: portal Scope web portal, pages, and user-facing components. area: reporting Generation and delivery of benchmark reports. topic: testing Test coverage, test infrastructure, and validation quality. topic: ui Portal, visual presentation, layout, and interface components. labels Oct 17, 2026
@cedricvidal
Cedric Vidal (cedricvidal) added this pull request to stack #1518 October 17, 2026 00:58
@github-actions

Copy link
Copy Markdown

Test Results (Node.js 22)

test: Run #283

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Pending ⏳ Other ❓ Flaky 🍂 Duration ⏱️
3530 3526 4 0 0 0 0 2m1s

❌ Some tests failed!

Name Failure Message
❌ ExperimentStore CAS contracts > plans chunked-to-chunked growth and chunked-to-inline shrink independently Test timed out in 5000ms. If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❌ ExperimentDetail > starts an asynchronous deletion from the confirmation dialog Test timed out in 5000ms. If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❌ ExperimentDetail > shows deleted lifecycle progress and restores the Experiment Test timed out in 5000ms. If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❌ ExperimentDetail > removes selected current members in one revision-guarded request Test timed out in 5000ms. If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".

Github Test Reporter

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dab2dcc-d695-4b4f-9858-0ec01f2b7b56
@github-actions

Copy link
Copy Markdown

Test Results (Node.js 22)

test: Run #294

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Pending ⏳ Other ❓ Flaky 🍂 Duration ⏱️
3535 3535 0 0 0 0 0 1m17s

🎉 All tests passed!

Github Test Reporter

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9dab2dcc-d695-4b4f-9858-0ec01f2b7b56
@github-actions

Copy link
Copy Markdown

Test Results (Node.js 22)

test: Run #299

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Pending ⏳ Other ❓ Flaky 🍂 Duration ⏱️
3535 3534 1 0 0 0 0 1m49s

❌ Some tests failed!

Name Failure Message
❌ ExperimentStore CAS contracts > plans chunked-to-chunked growth and chunked-to-inline shrink independently Test timed out in 5000ms. If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".

Github Test Reporter

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: cli Scope command-line interface and terminal workflows. area: infrastructure Cloud resources, hosting, networking, and shared infrastructure. area: portal Scope web portal, pages, and user-facing components. area: reporting Generation and delivery of benchmark reports. area: worker Coding agent worker runtime, task execution, and lifecycle. language: javascript Work involving JavaScript code, tooling, or dependencies. topic: testing Test coverage, test infrastructure, and validation quality. topic: ui Portal, visual presentation, layout, and interface components. type: documentation Documentation additions, corrections, and improvements.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant