Skip to content

feat: manage existing-run experiment membership - #1515

Draft
Cedric Vidal (cedricvidal) wants to merge 22 commits into
cedricvidal-experiment-readoutfrom
cedricvidal-experiment-membership-recovery
Draft

Cedric Vidal (cedricvidal) wants to merge 22 commits into
cedricvidal-experiment-readoutfrom
cedricvidal-experiment-membership-recovery

Conversation

@cedricvidal

Copy link
Copy Markdown
Collaborator

Summary

Adds complete-set membership revisions for creating Experiments from existing runs and atomically adding, moving, removing, or deleting members.

  • Plans exact current run vectors before mutation, publishes one immutable revision/execution with CAS ownership, and supports deterministic replay plus interruption repair.
  • Keeps Experiment history authoritative while maintaining request.experimentId as denormalized lookup state.
  • Adds REST/OpenAPI and CLI parity, coordinated deletion, Runs-list bulk Create/Move, and current-attempt member removal within the compact Experiment Detail Runs tab.
  • Covers inline/chunked storage transitions, stale/conflicting ownership, replay, retry/replan, deletion coordination, Portal interactions, and Storybook states.

Demos

Before

N/A — existing standalone runs could not be managed as Experiment membership.

After

Integrated Scope Portal walkthrough: bulk Create/Move conflict handling, Atom Experiment navigation, compact attempt history, completed/failed report versions, one-revision member removal, interrupted reconciliation, and retry recovery.

experiment-membership.mp4

Testing

  • pnpm vitest run packages/shared/src/experiments/experiment-store.test.ts apps/api/src/experiment-membership.test.ts apps/api/src/endpoints.test.ts apps/api/src/routes/experiments.test.ts apps/cli/src/commands/experiment.test.ts — passed, 5 files / 210 tests.
  • pnpm --filter portal test -- src/lib/experiment-membership.test.ts src/lib/api-experiment-membership.test.ts src/pages/ExperimentDetail.test.tsx src/pages/ExperimentDetail.play.test.tsx src/pages/ExperimentsList.test.tsx src/pages/ExperimentsList.play.test.tsx src/components/Layout.test.tsx src/components/ExperimentReportCard.test.tsx — passed, 8 files / 39 tests.
  • pnpm --filter portal build — passed.
  • pnpm --filter portal build-storybook — passed.
  • pnpm test -- --testTimeout=15000 — passed before the later Portal-only PR B reconciliation, 286 files / 3,473 tests; affected Portal tests and builds were rerun above after the final merge.
  • Website OpenAPI tests and production build — passed after restoring the website's frozen-lockfile dependencies; generated OpenAPI artifact matches the API snapshot.
  • Retained demo take — H.264/yuv420p, 1440x900, 25 fps, 47.28 seconds / 1,182 frames; full FFmpeg decode and encoded first/last/contact-sheet inspection passed; moov precedes mdat.
  • Compose smoke — environment-blocked, not a product failure. The normal attempt failed with shared Docker VM ENOSPC; the one permitted targeted retry failed at /var/tmp/libpod_builder with the same condition. Safe exact cleanup reclaimed 0 bytes because Docker metadata reported unable to open database file / retained references. Teardown verified zero project containers, networks, and volumes for scope-mt-app-9-cedricvidal-refactored-meme. No further Docker cleanup was attempted.
  • Real Cosmos — not run because credentials were unavailable.

Documentation and compatibility

Updated docs/architecture/app-design.md, docs/architecture/experiments.md, apps/cli/README.md, and the generated website OpenAPI artifact.

The REST API adds existing-run creation plus bulk membership POST/DELETE routes. CLI adds experiment create --request-id, experiment add/move, and experiment remove. Existing APIs remain compatible. Request documents gain optional denormalized membership fields; no data migration or dependency change is required.

Checklist

  • If Portal features changed, keep CLI capabilities in sync.
  • If Portal components changed, update their Storybook stories.
  • If database changes require a migration, include up() / down() and keep it CosmosDB-compatible. N/A — only optional schema fields were added.
  • If dependencies changed, update the lockfile and regenerate NOTICE / NOTICE-REVIEW.txt with pnpm notice as needed. N/A — no dependency changes.
  • Video showing the behavior before the suggested change. N/A — this adds a new membership workflow.
  • Video showing the behavior after the suggested change.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 0d6d0638-e765-4142-b984-9fa2b494bb7f
Ensure interrupted membership rollovers claim request ownership before publishing the next experiment revision, clean up failed unpublished claims, and include membership repair in coordinator polling.\n\nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>\nCopilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
Create Experiments from existing standalone requests and add or remove multiple members through a single complete-set revision replacement with exact conflict reporting.\n\nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>\nCopilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
Publish one membership revision before deleting selected members, surface the resulting revision identity, and reject bulk deletion across multiple Experiments to avoid partial success.\n\nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>\nCopilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
Support creating Experiments from existing requests and moving or removing multiple requests with expected revision CAS and resulting revision output.\n\nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>\nCopilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
Verify inline-to-inline, inline-to-chunked with immutable archive, chunked growth, and chunked-to-inline shrink while preserving exact vectors and fingerprints.\n\nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>\nCopilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
…ment-membership-recovery

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a

# Conflicts:
#	docs/architecture/experiments.md
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a

# Conflicts:
#	apps/portal/src/pages/ExperimentDetail.test.tsx
#	apps/portal/src/pages/ExperimentDetail.tsx
#	docs/architecture/experiments.md
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
@github-actions github-actions Bot added type: documentation Documentation additions, corrections, and improvements. area: cli Scope command-line interface and terminal workflows. language: javascript Work involving JavaScript code, tooling, or dependencies. area: portal Scope web portal, pages, and user-facing components. topic: testing Test coverage, test infrastructure, and validation quality. topic: ui Portal, visual presentation, layout, and interface components. labels Oct 16, 2026
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3d995fd8-10c6-477c-b1d4-27ba9b2d762a
@cedricvidal
Cedric Vidal (cedricvidal) added this pull request to stack #1518 October 17, 2026 00:58
@github-actions

Copy link
Copy Markdown

Test Results (Node.js 22)

test: Run #280

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Pending ⏳ Other ❓ Flaky 🍂 Duration ⏱️
3499 3498 1 0 0 0 0 1m56s

❌ Some tests failed!

Name Failure Message
❌ ExperimentStore CAS contracts > plans chunked-to-chunked growth and chunked-to-inline shrink independently Test timed out in 5000ms. If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".

Github Test Reporter

…into cedricvidal-experiment-membership-recovery
@github-actions

Copy link
Copy Markdown

Test Results (Node.js 22)

test: Run #290

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Pending ⏳ Other ❓ Flaky 🍂 Duration ⏱️
3504 3504 0 0 0 0 0 1m8s

🎉 All tests passed!

Github Test Reporter

…into cedricvidal-experiment-membership-recovery
@github-actions

Copy link
Copy Markdown

Test Results (Node.js 22)

test: Run #297

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Pending ⏳ Other ❓ Flaky 🍂 Duration ⏱️
3504 3503 1 0 0 0 0 1m56s

❌ Some tests failed!

Name Failure Message
❌ ExperimentStore CAS contracts > plans chunked-to-chunked growth and chunked-to-inline shrink independently Test timed out in 5000ms. If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".

Github Test Reporter

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: cli Scope command-line interface and terminal workflows. area: portal Scope web portal, pages, and user-facing components. language: javascript Work involving JavaScript code, tooling, or dependencies. topic: testing Test coverage, test infrastructure, and validation quality. topic: ui Portal, visual presentation, layout, and interface components. type: documentation Documentation additions, corrections, and improvements.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant