Skip to content

feat(biometrics): add versioned calibration profiles, references and shared Python runtime - #828

Merged
ng merged 7 commits into
devfrom
feat/bio-cal-1-foundation
Oct 14, 2026
Merged

ng merged 7 commits into
devfrom
feat/bio-cal-1-foundation

Conversation

@ng

@ng ng commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Part 1 of 4 splitting #824 (reference snapshot 7b9ac5d3) into reviewable stacked PRs. #824 stays open as the umbrella and is not merged; the four parts reproduce its tree exactly (the tree of feat/bio-cal-4-ui is identical to the tree of 7b9ac5d3).

Stack, merge in order:

  1. feat/bio-cal-1-foundation → dev: versioned calibration profiles, references, DB schema and migrations, shared Python runtime
  2. feat/bio-cal-2-backtest → part 1: retained backtest replay engine and isolated package experiments
  3. feat/bio-cal-3-shadow-api → part 2: prospective shadow runtime, isolated coordinator host, calibration API and MCP, demo handlers
  4. feat/bio-cal-4-ui → part 3: calibration UI, i18n catalogs, browser tests and docs

This part

  • Versioned calibration contracts and per-side profile storage (src/biometrics/profiles), with activation intent, runtime receipts and legacy stage-settings migration.
  • Typed reference normalization (src/biometrics/references), canonical JSON and the algorithm registry (src/biometrics/algorithms).
  • Backtest contracts, immutable datasets, fenced job records, objectives, the replay worker core (src/biometrics/backtest: contracts, datasets, jobs, submission, evaluator, worker entry) and the worker-only bundle script with the build hook, because the profile activation path and its tests depend on them.
  • Biometrics DB schema, migrations and package-experiment migrations, plus the schema snapshot and migration tests.
  • Shared parameterized Python processing split out of the piezo processor (modules/common/biometrics: frames, gates, signal, replay, raw records, shadow workers) with the runtime adapter and Python tests.
  • Prospective frame tap in the piezo stream, legacy settings writes routed through pod storage, deploy packaging that omits workstation Python environments, lint scope-analysis fix, and CI updates for the Python module matrix and mutation targets.
  • New dependencies: @noble/hashes, esbuild, @playwright/test (lockfile updated here so later parts do not touch it).
  • Review follow-ups on this slice: reference corrections continue after the latest revision is deleted (next revision numbers from the family maximum), prospective frame observation time is clamped monotonic across wall-clock steps, and a non-null stage-rule save over malformed legacy data resets the legacy value before activating. The manifest-hash scope finding (build fingerprints inside the profile-compatibility hash) is deferred to sleepypod-core-125 because it changes TypeScript and Python identity in lockstep and needs a pod migration plan.
  • Rebased onto dev at 4ea6aa5f (feat(mcp): build and save a sleep curve from pod data #825, feat(curves): save named curves on the pod #827, feat(schedule): optional notes on curves #832 landed after the split). The stack's migration moved from 0024 to 0026_strong_the_order because dev took 0024 and 0025; the SQL was regenerated with drizzle-kit from the merged schema and is byte-for-byte the original, and the schema snapshot test was refreshed. Codecov uploads now run only from sleepypod/core so the review mirror's CI cannot poison the patch status.
  • Coverage pass: new and extended unit tests for the backtest contracts, evaluator core, jobs, datasets, submission, worker entry, reference normalization/alignment/scoring and provenance helpers lift this slice's estimated patch coverage from 84% to about 94%.

Test plan

  • tsc --noEmit clean at this slice
  • eslint on the changed TS/TSX files clean
  • Full vitest run green at this slice (389 files, 6,236 tests) (the only failure under full-suite load was the pre-existing, unrelated scripts/tests/diagnosticCli.test.ts timing test, which passes in isolation)
  • CI: Typecheck, Lint, Unit Tests, Python modules, Build & Package
  • CodeRabbit review via the mirror proxy addressed

🤖 Generated with Claude Code

🛏️ Beam this PR onto a sleepypod ✨

Pick the snippet that matches your pod. Already-installed pods can't use the curl bootstraps below — sleepypod's egress firewall DROPs github.com, and the script that knows how to unblock WAN is the very file curl is trying to fetch. Use sp-update instead; it's on disk and opens WAN as its first step.

🔄 Already on a sleepypod (most common — review a PR on a running pod):

sudo sp-update feat/bio-cal-1-foundation

🚀 Fresh pod / first install — bootstraps from GitHub, rebuilds every push:

curl -fsSL https://raw.githubusercontent.com/sleepypod/core/feat/bio-cal-1-foundation/scripts/install \
  | sudo bash -s -- --branch feat/bio-cal-1-foundation

🎯 Pin to this exact build (run 38518366373 · f170f13) — fresh-pod path, locked to one CI artifact for reproducible review:

curl -fsSL https://raw.githubusercontent.com/sleepypod/core/f170f13baad42c43ba91ce1f2a7f647c8b6fcd92/scripts/install \
  | sudo bash -s -- \
      --branch feat/bio-cal-1-foundation \
      --artifact-url 'https://nightly.link/sleepypod/core/actions/runs/38518366373/sleepypod-core.zip'

🧊 Artifact: sleepypod-core · self-destructs in 30 days 💥

@vercel

vercel Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
sleepypod-demo Ignored Ignored Preview Oct 14, 2026 5:45am UTC

@coderabbitai

coderabbitai Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 234 files, which is 134 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 5d0f33f8-a791-41e7-a306-eac27a578f10
📥 Commits

Reviewing files that changed from the base of the PR and between b18e100 and f170f13.

⛔ Files ignored due to path filters: 2
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
  • src/db/tests/__snapshots__/schema.test.ts.snap is excluded by !**/*.snap
📒 Files selected for processing: 234
  • .github/workflows/mutation.yml
  • .github/workflows/python-modules.yml
  • .github/workflows/test.yml
  • docs/DEPLOYMENT.md
  • drizzle.package-experiments.config.ts
  • eslint.config.ts
  • modules/common/biometrics/__init__.py
  • modules/common/biometrics/build_identity.py
  • modules/common/biometrics/collection_worker.py
  • modules/common/biometrics/config.py
  • modules/common/biometrics/execution_lock.py
  • modules/common/biometrics/frames.py
  • modules/common/biometrics/gates.py
  • modules/common/biometrics/inspection_worker.py
  • modules/common/biometrics/manifest.v1.json
  • modules/common/biometrics/processing.py
  • modules/common/biometrics/publication_worker.py
  • modules/common/biometrics/raw_artifacts.py
  • modules/common/biometrics/raw_publication.py
  • modules/common/biometrics/raw_records.py
  • modules/common/biometrics/raw_sources.py
  • modules/common/biometrics/replay.py
  • modules/common/biometrics/replay_worker.py
  • modules/common/biometrics/retained_replay.py
  • modules/common/biometrics/routing.py
  • modules/common/biometrics/runtime.py
  • modules/common/biometrics/shadow.py
  • modules/common/biometrics/shadow_build.py
  • modules/common/biometrics/shadow_process.py
  • modules/common/biometrics/shadow_worker.py
  • modules/common/biometrics/signal.py
  • modules/common/test_biometric_config.py
  • modules/common/test_biometric_routing.py
  • modules/common/test_biometric_runtime.py
  • modules/common/test_build_identity.py
  • modules/common/test_execution_lock.py
  • modules/common/test_inspection_worker.py
  • modules/common/test_raw_artifacts.py
  • modules/common/test_raw_collection.py
  • modules/common/test_raw_records.py
  • modules/common/test_raw_sources.py
  • modules/common/test_replay_resources.py
  • modules/common/test_shadow_build.py
  • modules/common/test_shadow_process.py
  • modules/common/test_worker_limits.py
  • modules/piezo-processor/fixtures/default-processing-v1.json
  • modules/piezo-processor/main.py
  • modules/piezo-processor/manifest.json
  • modules/piezo-processor/runtime_adapter.py
  • modules/piezo-processor/test_default_golden.py
  • modules/piezo-processor/test_main.py
  • modules/piezo-processor/test_raw_publication.py
  • modules/piezo-processor/test_replay_worker.py
  • modules/piezo-processor/test_retained_replay.py
  • modules/piezo-processor/test_runtime_adapter.py
  • modules/piezo-processor/test_shadow.py
  • modules/piezo-processor/test_shadow_worker.py
  • modules/piezo-processor/test_shared_gates.py
  • modules/piezo-processor/test_shared_signal.py
  • modules/piezo-processor/test_stateful_processing.py
  • modules/piezo-processor/test_waveform_replay.py
  • next-env.d.ts
  • package.json
  • scripts/build/biometrics-worker.mjs
  • scripts/deploy
  • src/biometrics/algorithms/canonical-json.ts
  • src/biometrics/algorithms/canonical.ts
  • src/biometrics/algorithms/registry-core.ts
  • src/biometrics/algorithms/registry.ts
  • src/biometrics/algorithms/tests/data-hash.test.ts
  • src/biometrics/algorithms/tests/registry.test.ts
  • src/biometrics/background-admission.ts
  • src/biometrics/backtest/artifacts.ts
  • src/biometrics/backtest/child.ts
  • src/biometrics/backtest/common-support.ts
  • src/biometrics/backtest/contracts.ts
  • src/biometrics/backtest/coverage.ts
  • src/biometrics/backtest/datasets.ts
  • src/biometrics/backtest/evaluator-core.ts
  • src/biometrics/backtest/evaluator.ts
  • src/biometrics/backtest/job-operations.ts
  • src/biometrics/backtest/jobs.ts
  • src/biometrics/backtest/location.ts
  • src/biometrics/backtest/night-summary-core.ts
  • src/biometrics/backtest/numeric-comparison.ts
  • src/biometrics/backtest/numeric-night-summary.ts
  • src/biometrics/backtest/numeric-subgroups.ts
  • src/biometrics/backtest/objective-finalization.ts
  • src/biometrics/backtest/objective-policy.ts
  • src/biometrics/backtest/process-group-memory-contract.ts
  • src/biometrics/backtest/raw-catalog.ts
  • src/biometrics/backtest/raw-inspection-child.ts
  • src/biometrics/backtest/splits.ts
  • src/biometrics/backtest/stage-comparison.ts
  • src/biometrics/backtest/stage-objective.ts
  • src/biometrics/backtest/submission.ts
  • src/biometrics/backtest/tests/artifacts.test.ts
  • src/biometrics/backtest/tests/child.test.ts
  • src/biometrics/backtest/tests/common-support.test.ts
  • src/biometrics/backtest/tests/coverage.test.ts
  • src/biometrics/backtest/tests/datasets.test.ts
  • src/biometrics/backtest/tests/evaluator.test.ts
  • src/biometrics/backtest/tests/fixtures/backtest.ts
  • src/biometrics/backtest/tests/fixtures/stage-objective.ts
  • src/biometrics/backtest/tests/fixtures/waveform.ts
  • src/biometrics/backtest/tests/job-operations.test.ts
  • src/biometrics/backtest/tests/jobs.test.ts
  • src/biometrics/backtest/tests/location.test.ts
  • src/biometrics/backtest/tests/night-summary-core.test.ts
  • src/biometrics/backtest/tests/numeric-comparison.test.ts
  • src/biometrics/backtest/tests/numeric-night-summary.test.ts
  • src/biometrics/backtest/tests/numeric-subgroups.test.ts
  • src/biometrics/backtest/tests/objective-finalization.test.ts
  • src/biometrics/backtest/tests/objective-policy.test.ts
  • src/biometrics/backtest/tests/raw-artifact-references.test.ts
  • src/biometrics/backtest/tests/raw-catalog.test.ts
  • src/biometrics/backtest/tests/raw-inspection-child.test.ts
  • src/biometrics/backtest/tests/splits.test.ts
  • src/biometrics/backtest/tests/stage-comparison.test.ts
  • src/biometrics/backtest/tests/stage-objective.test.ts
  • src/biometrics/backtest/tests/submission.test.ts
  • src/biometrics/backtest/tests/uncertainty-core.test.ts
  • src/biometrics/backtest/tests/waveform-build.test.ts
  • src/biometrics/backtest/tests/waveform-child.test.ts
  • src/biometrics/backtest/tests/waveform-contracts.test.ts
  • src/biometrics/backtest/tests/waveform-resources.test.ts
  • src/biometrics/backtest/tests/worker-entry.test.ts
  • src/biometrics/backtest/uncertainty-core.ts
  • src/biometrics/backtest/uncertainty-policy.ts
  • src/biometrics/backtest/waveform-build.ts
  • src/biometrics/backtest/waveform-child.ts
  • src/biometrics/backtest/waveform-contracts.ts
  • src/biometrics/backtest/waveform-output.ts
  • src/biometrics/backtest/waveform-resources.ts
  • src/biometrics/backtest/worker-entry.ts
  • src/biometrics/profiles/activation.ts
  • src/biometrics/profiles/contracts.ts
  • src/biometrics/profiles/evaluation-assessment.ts
  • src/biometrics/profiles/evaluation-identity.ts
  • src/biometrics/profiles/evaluation-store.ts
  • src/biometrics/profiles/health.ts
  • src/biometrics/profiles/history.ts
  • src/biometrics/profiles/legacy-stage.ts
  • src/biometrics/profiles/promotion-build.ts
  • src/biometrics/profiles/qualification-registry.ts
  • src/biometrics/profiles/recovery.ts
  • src/biometrics/profiles/replay-baseline.ts
  • src/biometrics/profiles/replay-context.ts
  • src/biometrics/profiles/runtime-host.ts
  • src/biometrics/profiles/runtime.ts
  • src/biometrics/profiles/stage-view.ts
  • src/biometrics/profiles/store.ts
  • src/biometrics/profiles/tests/activation.test.ts
  • src/biometrics/profiles/tests/contracts.test.ts
  • src/biometrics/profiles/tests/evaluation-assessment.test.ts
  • src/biometrics/profiles/tests/evaluation-identity.test.ts
  • src/biometrics/profiles/tests/fixtures/evaluation.ts
  • src/biometrics/profiles/tests/fixtures/replay-context.ts
  • src/biometrics/profiles/tests/health.test.ts
  • src/biometrics/profiles/tests/history.test.ts
  • src/biometrics/profiles/tests/provenance-migration.test.ts
  • src/biometrics/profiles/tests/qualification-registry.test.ts
  • src/biometrics/profiles/tests/recovery.test.ts
  • src/biometrics/profiles/tests/replay-context.test.ts
  • src/biometrics/profiles/tests/runtime-host.test.ts
  • src/biometrics/profiles/tests/runtime.test.ts
  • src/biometrics/profiles/tests/stage-view.test.ts
  • src/biometrics/profiles/tests/store.test.ts
  • src/biometrics/profiles/tests/validated-activation.test.ts
  • src/biometrics/profiles/tests/vital-provenance.test.ts
  • src/biometrics/profiles/tests/vital-summary-provenance.test.ts
  • src/biometrics/profiles/vital-provenance.ts
  • src/biometrics/profiles/vital-summary-provenance.ts
  • src/biometrics/references/alignment.ts
  • src/biometrics/references/contracts.ts
  • src/biometrics/references/legacy.ts
  • src/biometrics/references/normalize-core.ts
  • src/biometrics/references/normalize.ts
  • src/biometrics/references/parse.ts
  • src/biometrics/references/stage-scoring.ts
  • src/biometrics/references/store.ts
  • src/biometrics/references/tests/alignment.test.ts
  • src/biometrics/references/tests/legacy.test.ts
  • src/biometrics/references/tests/normalize.test.ts
  • src/biometrics/references/tests/stage-scoring.test.ts
  • src/biometrics/references/tests/store.test.ts
  • src/biometrics/references/tests/time.test.ts
  • src/biometrics/references/time.ts
  • src/biometrics/tests/background-admission.test.ts
  • src/db/biometrics-migrations/0020_black_patch.sql
  • src/db/biometrics-migrations/0021_common_hedge_knight.sql
  • src/db/biometrics-migrations/0022_ordinary_amphibian.sql
  • src/db/biometrics-migrations/0023_demonic_the_call.sql
  • src/db/biometrics-migrations/0024_silky_bloodscream.sql
  • src/db/biometrics-migrations/0025_worthless_kat_farrell.sql
  • src/db/biometrics-migrations/0026_overjoyed_fabian_cortez.sql
  • src/db/biometrics-migrations/0027_release_backtest_reports.sql
  • src/db/biometrics-migrations/0028_retained_raw_artifacts.sql
  • src/db/biometrics-migrations/0029_thick_typhoid_mary.sql
  • src/db/biometrics-migrations/0030_icy_maginty.sql
  • src/db/biometrics-migrations/meta/0020_snapshot.json
  • src/db/biometrics-migrations/meta/0021_snapshot.json
  • src/db/biometrics-migrations/meta/0022_snapshot.json
  • src/db/biometrics-migrations/meta/0023_snapshot.json
  • src/db/biometrics-migrations/meta/0024_snapshot.json
  • src/db/biometrics-migrations/meta/0025_snapshot.json
  • src/db/biometrics-migrations/meta/0026_snapshot.json
  • src/db/biometrics-migrations/meta/0027_snapshot.json
  • src/db/biometrics-migrations/meta/0028_snapshot.json
  • src/db/biometrics-migrations/meta/0029_snapshot.json
  • src/db/biometrics-migrations/meta/0030_snapshot.json
  • src/db/biometrics-migrations/meta/_journal.json
  • src/db/biometrics-schema.ts
  • src/db/migrations/0026_strong_the_order.sql
  • src/db/migrations/meta/0026_snapshot.json
  • src/db/migrations/meta/_journal.json
  • src/db/package-experiment-migrations/0000_bent_overlord.sql
  • src/db/package-experiment-migrations/meta/0000_snapshot.json
  • src/db/package-experiment-migrations/meta/_journal.json
  • src/db/package-experiment-schema.ts
  • src/db/schema.ts
  • src/db/tests/migrations.test.ts
  • src/db/tests/package-experiment-schema.test.ts
  • src/db/tests/schema.test.ts
  • src/hardware/alarmPersistence.ts
  • src/homekit/storage.ts
  • src/lib/podStorage.ts
  • src/lib/sleep-stage-rules.ts
  • src/streaming/piezoStream.ts
  • src/streaming/prospectiveFrameTap.ts
  • src/streaming/tests/piezoStreamSource.test.ts
  • src/streaming/tests/prospectiveFrameTap.test.ts
  • stryker.config.mjs
  • tests/updateDeploy.test.ts

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ng
ng force-pushed the feat/bio-cal-1-foundation branch 2 times, most recently from 4256406 to 00adc14 Compare October 14, 2026 00:58
@ng ng closed this Oct 14, 2026
@ng ng reopened this Oct 14, 2026
@ng
ng force-pushed the feat/bio-cal-1-foundation branch from 783cd9e to 67f674b Compare October 14, 2026 03:34
@ng
ng force-pushed the feat/bio-cal-1-foundation branch from bd39697 to f170f13 Compare October 14, 2026 05:45
@ng
ng enabled auto-merge October 14, 2026 05:45
@ng
ng added this pull request to the merge queue Oct 14, 2026
Merged via the queue into dev with commit 0e7e865 Oct 14, 2026
16 checks passed
@ng
ng deleted the feat/bio-cal-1-foundation branch October 14, 2026 05:55
ng added a commit that referenced this pull request Oct 14, 2026
…ibration, saved curves and presence auto-on (#820)

Promotes `dev` to `main` with Apple Watch reference data, configurable sleep-stage classification, biometrics calibration, saved and annotated curves, experimental presence auto-on, and Host/Origin hardening for the pod API.

## Changes
- Import Apple Watch sleep stages and heart rate, overlay reference heart rate on the vitals chart, explore vitals, and build datasets for classifier calibration (#802, #803, #814, #821, #826).
- Add per-side sleep-stage rules, written as sentences with quick adjust, preview and a change log, and respect piezo calibration quality, including unknown calibration in the week view (#815, #816, #817, #822).
- Add biometrics calibration: versioned profiles and references, a shared Python runtime, a retained backtest replay engine, a prospective shadow runtime with a calibration API, and the calibration UI (#828–#831).
- MCP tools to inspect, preview and apply sleep-stage rules against the Watch reference, and to build and save a sleep curve from pod data (#823, #825).
- Save named curves on the pod and add optional notes to curves (#827, #832).
- Experimental per-side presence auto-on with person-vs-pet occupancy classification; auto-off now ignores light loads such as pets (#810).
- Reject unknown Host headers so DNS-rebinding pages cannot reach the unauthenticated pod API, and apply the same Host/Origin allowlist to the sensor WebSocket handshake (#833, #834).
- Improve session endings, movement scoring during Pod 5 pump operation, and vitals continuity for quiet sleepers (#804, #813).
- Add away-mode toggles to the side card and Stage panel, move Settings to per-section side tabs, improve the scheduled label contrast, and prevent premature stalled-pipeline banners (#805–#808).
- Update dependencies and pin Babel runtime to fix the Turbopack MQTT build (#811, #812, #818, #819).

## Migrations
- Configuration DB: side stage classifier, stage classifier changes, curve templates, schedule notes and presence auto-on columns (0022–0027).
- Biometrics DB: Watch reference stages and vitals, and calibration tables (0017–0030).

## Test plan
- [x] GitHub checks on the `dev` head: build, lint, typecheck, unit tests, browser tests and all Python test jobs.
- [ ] After merge, wait for "Analyze & Release" on the `main` merge commit, then `sudo sp-update main` once on a pod and confirm the service starts and migrations apply.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant