Run eb_jepa Two Rooms and draft the planning protocol

.github/workflows/ci.yml

@@ -16,7 +16,7 @@ jobs:
   python:
     name: Python ${{ matrix.python }}
     runs-on: ubuntu-latest
-    timeout-minutes: 20
+    timeout-minutes: 10
     strategy:
       fail-fast: false
       matrix:
@@ -40,10 +40,6 @@ jobs:
           uv venv
           uv pip install --torch-backend cpu -c "$RUNNER_TEMP/constraints.txt" -e . --group dev
 
-      # The rotation test writes rotation metadata with the ffmpeg CLI.
-      - name: Install ffmpeg
-        run: sudo apt-get update -q && sudo apt-get install -y -q ffmpeg
-
       - name: Lint
         run: |
           uv run --no-sync ruff check src tests

.gitignore

@@ -20,3 +20,7 @@ src/jepa_scope/static/
 # Claude Code: per-machine settings and agent worktrees
 .claude/settings.local.json
 .claude/worktrees/
+
+# eb_jepa checkpoints and planning videos stay in the lab directory, never in the repository.
+*.pth.tar
+*.gif

CLAUDE.md

@@ -81,7 +81,10 @@ and report formats live there.
 - Merging into the default branch.
 - Publishing a release: tags, GitHub releases, PyPI uploads.
 - Changing repository settings or visibility, or posting about the project anywhere public.
-- Paid compute, such as renting a GPU for embedding runs or `eb_jepa` training.
+- Paid compute outside the spending cap of a plan the owner approved. Within that cap, cloud
+  GPUs (RunPod or similar) need no further approval, provided each pod's GPU type and hourly
+  price from a live read go in the plan's log before it is created, a watchdog stops it if the
+  session is lost, and it is deleted when the job ends.
 
 ### Model routing
 

docs/decisions.md

@@ -27,17 +27,21 @@ anything the code already shows.
 
 ## Method · `docs/decisions/method.md`
 
+- 2026-11-06 — Planning protocol: a Markov planning state as one fixed-shape tensor, one-step stateless predict, model-owned energy, summed objective by default
 - 2026-11-05 — Surprise = mean L1 between predictor output and layer_norm(target), with the context frames encoded on their own; never the HF full-clip forward
 - 2026-11-05 — Windows of 16 frames at 8 fps, overlapping by half: context is the first half, every later tubelet is predicted exactly once
 - 2026-11-05 — Any output-changing edit bumps OUTPUT_VERSION, which keys the result cache
 
 ## Product & data · `docs/decisions/product.md`
 
+- 2026-11-06 — The planning protocol is drafted here (`src/jepa_scope/planning/`, `docs/planning-protocol.md`) and moves to latent-plan when that project starts
 - 2026-11-05 — Apache-2.0, public at ryangswe/jepa-scope; V-JEPA 2 weights are downloaded at run time, never redistributed
 - 2026-11-05 — `.latent` = safetensors plus a validated JSON header; the viewer refuses a video whose SHA-256 differs from the header
 - 2026-11-03 — jepa-scope is the first tool; the order of the other two is a proposal; tools target a model-agnostic protocol rather than V-JEPA 2
 
 ## Process & tooling · `docs/decisions/process.md`
 
+- 2026-11-05 — Cloud GPUs within the spending cap of an owner-approved plan need no further approval: log the price, run a watchdog, delete the pod
+- 2026-11-05 — Milestones follow the roadmap's weekly order; Week 2's scope landed in Week 1, so Weeks 3 and 4 come next
 - 2026-11-05 — Adopted the workflow kit at the full level, with its process files, the roadmap and this record published in the public repo
 - 2026-11-05 — CI installs uv.lock versions as constraints with CPU torch; the real-model test stays opt-in and runs locally

docs/decisions/method.md

@@ -5,6 +5,48 @@ index is `docs/decisions.md`.
 
 ---
 
+## 2026-11-06 — Planning protocol: a Markov planning state as one fixed-shape tensor, one-step stateless predict, model-owned energy, summed objective by default
+
+**Status:** live, indexed
+
+**Context.** The 2026-11-03 product decision said the tools target a model-agnostic protocol
+rather than V-JEPA 2. Week 1 drafted that protocol and checked it against `eb_jepa`'s Two Rooms
+model. latent-plan will build on it, with V-JEPA 2-AC as its second model, so its shape is costly
+to change later.
+
+**Decision.** A world model has `state_shape`, `action_dim`, `encode(obs)`, `predict(z, a)` and
+`energy(z, z_goal)`. `z` is a planning state: one fixed-shape tensor holding everything the next
+prediction needs, which may be more than the latent `energy` compares (for V-JEPA 2-AC, its token
+context and the arm's pose). `predict` takes one step and keeps no state. `encode` accepts the
+model's own observation type. `plan_cost` sums the energy over all `T + 1` rollout states by
+default, with `"last"` as a planner setting; `plan_cost` adds the energies step by step and keeps
+only the current state. Adapters treat inputs as read-only, keep batch rows independent, refuse
+to overflow a packed state's declared capacity, and promise inference only. The owner approved
+the signatures, then a Fable architect's changes (planning state, `encode(obs: Any)`, the
+objective setting), then an outside review's (renaming `latent_shape` to `state_shape`, the
+adapter rules, streaming cost, input validation). The interface stays experimental until a
+second adapter reproduces its own model's native rollouts.
+
+**Evidence.** Through an adapter, rollouts and costs match `eb_jepa`'s own, and the reference CEM
+returns `eb_jepa`'s plans given the same noise. In a closed-loop engineering check it solved 16
+of 20 episodes against 18 of 20 natively, which does not establish equivalence (details in
+`docs/planning-protocol.md`). vjepa2's planner (`notebooks/utils/
+mpc_utils.py`) starts from one image and grows its context from its own predictions, so its
+state is Markov and fits a stateless `predict`.
+
+**Rejected.**
+- *An explicit history argument or an `initial_state(obs_history)` hook.* Neither target model's
+  planner uses an observation history, and a fixed history length does not hold the pose.
+- *An opaque state object with a separate latent.* `cem_plan` expands `z0` with tensor
+  operations, so it would also need an `expand_state` operation; two new operations before any
+  adapter needs them.
+- *A summed objective hard-coded into `plan_cost`.* V-JEPA 2-AC's planner scores only the final
+  frame, and a parity check against it needs the same objective.
+- *Deferred, not rejected:* an optional `latent(z) -> [B, *latent]` for drawing latents along a plan, to be
+  decided when latent-plan is planned.
+
+---
+
 ## 2026-11-05 — Surprise = mean L1 between predictor output and layer_norm(target), with the context frames encoded on their own; never the HF full-clip forward
 
 **Status:** live, indexed

docs/decisions/process.md

@@ -5,6 +5,43 @@ Full rationale for this theme's decisions, newest first. The lean, auto-loaded i
 
 ---
 
+## 2026-11-05 — Milestones follow the roadmap's weekly order; Week 2's scope landed in Week 1, so Weeks 3 and 4 come next
+
+**Status:** live, indexed
+
+**Context.** After Week 1's scaffold, the next milestone was the owner's choice among the
+roadmap's candidates.
+
+**Decision.** Milestones follow the roadmap's planned weekly order, finishing a week's remaining
+steps before moving on. Week 2's scope (inference, the grid, the overlay, the file format and the
+CLI) landed in Week 1, so Weeks 3 and 4 come next.
+
+**Rejected.**
+- *Re-choosing the order at each milestone.* The weekly order already reflects the roadmap's
+  reasoning; changing it is a roadmap edit, not a per-milestone choice.
+
+---
+
+## 2026-11-05 — Cloud GPUs within the spending cap of an owner-approved plan need no further approval: log the price, run a watchdog, delete the pod
+
+**Status:** live, indexed
+
+**Context.** Training `eb_jepa`'s Two Rooms model on the laptop CPU projected to about 17 hours.
+The owner has a RunPod account with credit and said cloud GPUs are fine if the cost is not
+excessive.
+
+**Decision.** Paid compute outside an approved plan's spending cap stays owner-only. Within the
+cap, a session may create cloud GPUs without asking again, provided the GPU type and hourly
+price from a live read go in the plan's log before each pod is created, a watchdog stops the pod
+if the session is lost, and the pod is deleted when the job ends. `CLAUDE.md` carries the rule.
+The Week 1 plan's cap was $25; it spent about $1.65.
+
+**Rejected.**
+- *Asking before every pod.* The cap and the logging already bound the cost, and asking again
+  adds owner time for no control.
+
+---
+
 ## 2026-11-05 — Adopted the workflow kit at the full level, with its process files, the roadmap and this record published in the public repo
 
 **Status:** live, indexed

docs/decisions/product.md

@@ -5,6 +5,24 @@ lean, auto-loaded index is `docs/decisions.md`.
 
 ---
 
+## 2026-11-06 — The planning protocol is drafted here (`src/jepa_scope/planning/`, `docs/planning-protocol.md`) and moves to latent-plan when that project starts
+
+**Status:** live, indexed
+
+**Context.** The roadmap's Week 1 included running `eb_jepa`'s Two Rooms example and drafting
+the planning protocol, which belongs to latent-plan, the third tool, and does not exist yet.
+
+**Decision.** The draft lives in `src/jepa_scope/planning/` (protocol, reference CEM, `eb_jepa`
+adapter) with its spec in `docs/planning-protocol.md`. It depends only on torch; `eb_jepa` is
+imported only when the adapter is built and is never a dependency. The code moves to latent-plan
+when that project starts.
+
+**Rejected.**
+- *Starting the latent-plan repository now.* Its plan, UI and audience are not decided; a draft
+  in an existing tested package is cheaper to change.
+
+---
+
 ## 2026-11-05 — Apache-2.0, public at ryangswe/jepa-scope; V-JEPA 2 weights are downloaded at run time, never redistributed
 
 **Status:** live, indexed