feat(biometrics): add retained backtest replay engine and isolated package experiments by ng · Pull Request #829 · sleepypod/core

ng · 2026-10-13T23:16:23Z

Part 2 of 4 splitting #824 (reference snapshot 7b9ac5d3) into reviewable stacked PRs. #824 stays open as the umbrella and is not merged; the four parts reproduce its tree exactly (the tree of feat/bio-cal-4-ui is identical to the tree of 7b9ac5d3).

Stack, merge in order:

feat/bio-cal-1-foundation → dev: versioned calibration profiles, references, DB schema and migrations, shared Python runtime
feat/bio-cal-2-backtest → part 1: retained backtest replay engine and isolated package experiments
feat/bio-cal-3-shadow-api → part 2: prospective shadow runtime, isolated coordinator host, calibration API and MCP, demo handlers
feat/bio-cal-4-ui → part 3: calibration UI, i18n catalogs, browser tests and docs

This part

Retained backtest execution (src/biometrics/backtest): the job runner, retained report artifacts, resource admission, paired-night uncertainty, replay traces and waveform replay of retained RAW copies, on top of the contracts and worker core from part 1.
Isolated algorithm package experiments (src/biometrics/packages, scripts/algorithm-experiments): manifest validation, Docker-restricted runs, determinism checks, resource evidence and managed trials.
The biometrics:experiment script. The coordinator and shadow host bundles are added to scripts/build/biometrics-worker.mjs in part 3 together with their entry points.
Review follow-ups on this slice: the package-experiment transport ceiling is derived from the frozen input so a passing freeze cannot fail mid-stream after holdout exposure is recorded, and submission pins the coordinator bundle beside the packaged worker so execution hashes the same bundle.
Coverage pass: unit tests for waveform execution, durable trace reads, runner lease and cancellation paths, waveform selection and the child-process helpers lift this slice's estimated patch coverage from 81% to about 98%.

Test plan

tsc --noEmit clean at this slice
eslint on the changed TS/TSX files clean
Full vitest run green at this slice (429 files, 6,458 tests) (the only failure under full-suite load was the pre-existing, unrelated scripts/tests/diagnosticCli.test.ts timing test, which passes in isolation)
CI: Typecheck, Lint, Unit Tests, Python modules, Build & Package
CodeRabbit review via the mirror proxy addressed

🤖 Generated with Claude Code

🛏️ Beam this PR onto a sleepypod ✨

Pick the snippet that matches your pod. Already-installed pods can't use the curl bootstraps below — sleepypod's egress firewall DROPs github.com, and the script that knows how to unblock WAN is the very file curl is trying to fetch. Use sp-update instead; it's on disk and opens WAN as its first step.

🔄 Already on a sleepypod (most common — review a PR on a running pod):

sudo sp-update feat/bio-cal-2-backtest

🚀 Fresh pod / first install — bootstraps from GitHub, rebuilds every push:

curl -fsSL https://raw.githubusercontent.com/sleepypod/core/feat/bio-cal-2-backtest/scripts/install \
  | sudo bash -s -- --branch feat/bio-cal-2-backtest

🎯 Pin to this exact build (run 38519463600 · b925d8f) — fresh-pod path, locked to one CI artifact for reproducible review:

curl -fsSL https://raw.githubusercontent.com/sleepypod/core/b925d8f6f6e2a7461925ff5f6b5c3bd42d0e0197/scripts/install \
  | sudo bash -s -- \
      --branch feat/bio-cal-2-backtest \
      --artifact-url 'https://nightly.link/sleepypod/core/actions/runs/38519463600/sleepypod-core.zip'

🧊 Artifact: sleepypod-core · self-destructs in 30 days 💥

vercel · 2026-10-13T23:16:28Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
sleepypod-demo	Ignored	Preview	Oct 14, 2026 5:58am UTC

coderabbitai · 2026-10-13T23:16:33Z

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: defaults
Review profile: CHILL
Plan: Advanced
Run ID: 9a0f8b23-7de0-4f46-b0a3-dd3e0031b44e

📥 Commits

Reviewing files that changed from the base of the PR and between 0e7e865 and b925d8f.

📒 Files selected for processing: 91

package.json
scripts/algorithm-experiments/README.md
scripts/algorithm-experiments/accuracy-policy.ts
scripts/algorithm-experiments/accuracy.test.ts
scripts/algorithm-experiments/accuracy.ts
scripts/algorithm-experiments/cli.ts
scripts/algorithm-experiments/comparison.test.ts
scripts/algorithm-experiments/comparison.ts
scripts/algorithm-experiments/container.test.ts
scripts/algorithm-experiments/container.ts
scripts/algorithm-experiments/determinism.ts
scripts/algorithm-experiments/environment.test.ts
scripts/algorithm-experiments/environment.ts
scripts/algorithm-experiments/files.test.ts
scripts/algorithm-experiments/files.ts
scripts/algorithm-experiments/fixtures.test.ts
scripts/algorithm-experiments/fixtures.ts
scripts/algorithm-experiments/input.test.ts
scripts/algorithm-experiments/input.ts
scripts/algorithm-experiments/ledger.test.ts
scripts/algorithm-experiments/ledger.ts
scripts/algorithm-experiments/managed-evidence.ts
scripts/algorithm-experiments/managed-files.test.ts
scripts/algorithm-experiments/managed-files.ts
scripts/algorithm-experiments/managed-protocol.ts
scripts/algorithm-experiments/managed.test.ts
scripts/algorithm-experiments/managed.ts
scripts/algorithm-experiments/policy.ts
scripts/algorithm-experiments/preflight-evidence.ts
scripts/algorithm-experiments/preflight.ts
scripts/algorithm-experiments/qualification.test.ts
scripts/algorithm-experiments/qualification.ts
scripts/algorithm-experiments/recovery.test.ts
scripts/algorithm-experiments/recovery.ts
scripts/algorithm-experiments/resource-assessment.test.ts
scripts/algorithm-experiments/resource-assessment.ts
scripts/algorithm-experiments/resource-test-policy.ts
scripts/algorithm-experiments/resources.test.ts
scripts/algorithm-experiments/resources.ts
scripts/algorithm-experiments/staging-recovery.test.ts
scripts/algorithm-experiments/staging-recovery.ts
scripts/algorithm-experiments/workspace.test.ts
scripts/algorithm-experiments/workspace.ts
src/biometrics/backtest/admission-service.ts
src/biometrics/backtest/admission.ts
src/biometrics/backtest/discovery.ts
src/biometrics/backtest/execution-budget.ts
src/biometrics/backtest/night-summary.ts
src/biometrics/backtest/numeric-trace-page.ts
src/biometrics/backtest/process-group-memory.ts
src/biometrics/backtest/raw-collection-child.ts
src/biometrics/backtest/raw-publication-child.ts
src/biometrics/backtest/reference-trace-page.ts
src/biometrics/backtest/report-summary.ts
src/biometrics/backtest/runner.ts
src/biometrics/backtest/submission.ts
src/biometrics/backtest/tests/admission-build.test.ts
src/biometrics/backtest/tests/admission-collector.test.ts
src/biometrics/backtest/tests/admission-service.test.ts
src/biometrics/backtest/tests/admission.test.ts
src/biometrics/backtest/tests/discovery.test.ts
src/biometrics/backtest/tests/execution-budget.test.ts
src/biometrics/backtest/tests/night-summary.test.ts
src/biometrics/backtest/tests/numeric-traces.test.ts
src/biometrics/backtest/tests/process-group-memory.test.ts
src/biometrics/backtest/tests/raw-collection-child.test.ts
src/biometrics/backtest/tests/raw-publication-child.test.ts
src/biometrics/backtest/tests/reference-traces.test.ts
src/biometrics/backtest/tests/report-summary.test.ts
src/biometrics/backtest/tests/runner-waveform.test.ts
src/biometrics/backtest/tests/runner.test.ts
src/biometrics/backtest/tests/submission-build.test.ts
src/biometrics/backtest/tests/traces.test.ts
src/biometrics/backtest/tests/uncertainty.test.ts
src/biometrics/backtest/tests/waveform-dataset.test.ts
src/biometrics/backtest/tests/waveform-evaluator.test.ts
src/biometrics/backtest/tests/waveform-execution.test.ts
src/biometrics/backtest/tests/waveform-selection.test.ts
src/biometrics/backtest/trace-contracts.ts
src/biometrics/backtest/trace-page.ts
src/biometrics/backtest/traces.ts
src/biometrics/backtest/uncertainty.ts
src/biometrics/backtest/waveform-dataset.ts
src/biometrics/backtest/waveform-execution.ts
src/biometrics/backtest/waveform-selection.ts
src/biometrics/packages/README.md
src/biometrics/packages/manifest.ts
src/biometrics/packages/protocol.ts
src/biometrics/packages/tests/fixture.ts
src/biometrics/packages/tests/manifest.test.ts
src/biometrics/packages/tests/protocol.test.ts

 ______________________________________________________
< Your bugs are no match for my GPU-powered intellect. >
 ------------------------------------------------------
  \
   \   \
        \ /\
        ( )
      .( o ).

✨ Finishing Touches

📝 Generate docstrings

Commit to this branch
Create a new PR

🧪 Generate unit tests (beta)

Commit to this branch
Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

codecov · 2026-10-13T23:22:07Z

Codecov Report

❌ Patch coverage is 98.10726% with 12 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/biometrics/backtest/runner.ts	96.51%	0 Missing and 3 partials ⚠️
src/biometrics/backtest/raw-collection-child.ts	94.87%	0 Missing and 2 partials ⚠️
src/biometrics/backtest/raw-publication-child.ts	95.23%	0 Missing and 2 partials ⚠️
src/biometrics/backtest/reference-trace-page.ts	93.10%	1 Missing and 1 partial ⚠️
src/biometrics/backtest/waveform-selection.ts	97.26%	0 Missing and 2 partials ⚠️
src/biometrics/backtest/process-group-memory.ts	98.30%	0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

…ckage experiments Part 2 of 4 splitting #824 (reference snapshot 7b9ac5d).

…g freeze never fails mid-stream

…ution hash one bundle

…es in backtest tests

This was referenced Oct 13, 2026

feat(biometrics): add calibration profiles, replay, and guarded algorithm development #824

Closed

CR: feat/bio-cal-2-backtest ng/core#19

Closed

ng force-pushed the feat/bio-cal-2-backtest branch from 7662a31 to e5d908e Compare October 13, 2026 23:35

ng force-pushed the feat/bio-cal-1-foundation branch from aef0dad to d5c16c8 Compare October 13, 2026 23:35

ng force-pushed the feat/bio-cal-2-backtest branch 2 times, most recently from 02b6167 to e4eff71 Compare October 14, 2026 00:58

ng force-pushed the feat/bio-cal-1-foundation branch from 4256406 to 00adc14 Compare October 14, 2026 00:58

ng force-pushed the feat/bio-cal-2-backtest branch 4 times, most recently from c920546 to 54e0381 Compare October 14, 2026 03:34

ng force-pushed the feat/bio-cal-1-foundation branch from 783cd9e to 67f674b Compare October 14, 2026 03:34

ng force-pushed the feat/bio-cal-2-backtest branch 2 times, most recently from b86305d to 19e8c1f Compare October 14, 2026 05:45

ng force-pushed the feat/bio-cal-1-foundation branch from bd39697 to f170f13 Compare October 14, 2026 05:45

Base automatically changed from feat/bio-cal-1-foundation to dev October 14, 2026 05:55

ng added 4 commits October 13, 2026 22:56

feat(biometrics): add retained backtest replay engine and isolated pa…

19867c7

…ckage experiments Part 2 of 4 splitting #824 (reference snapshot 7b9ac5d).

Derive the experiment transport ceiling from frozen input so a passin…

d342bdd

…g freeze never fails mid-stream

Pin the coordinator beside the packaged worker so submission and exec…

2c81bfd

…ution hash one bundle

Cover waveform execution, durable trace reads and runner lease failur…

b925d8f

…es in backtest tests

ng force-pushed the feat/bio-cal-2-backtest branch from 19e8c1f to b925d8f Compare October 14, 2026 05:57

ng enabled auto-merge October 14, 2026 05:57

ng added this pull request to the merge queue Oct 14, 2026

Merged via the queue into dev with commit 35e6141 Oct 14, 2026
8 of 9 checks passed

ng deleted the feat/bio-cal-2-backtest branch October 14, 2026 06:10

Navigation Menu

Repository navigation

feat(biometrics): add retained backtest replay engine and isolated package experiments - #829

feat(biometrics): add retained backtest replay engine and isolated package experiments#829
ng merged 4 commits into
devfrom
feat/bio-cal-2-backtest

ng commented Oct 13, 2026 •

edited by github-actions Bot

Loading

Uh oh!

vercel Bot commented Oct 13, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 13, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Oct 13, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ng commented Oct 13, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

This part

Test plan

Uh oh!

vercel Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ng commented Oct 13, 2026 •

edited by github-actions Bot

Loading

vercel Bot commented Oct 13, 2026 •

edited

Loading

coderabbitai Bot commented Oct 13, 2026 •

edited

Loading

codecov Bot commented Oct 13, 2026 •

edited

Loading