[ryangswe]

This is the first pull request of the Weeks 3 and 4 plan (docs/plans/2026-11-06-weeks3-4-folder-search-trajectory.md), covering tasks T0 and T1. The plan itself, its review record and the decision about reviews are included.

T0: which vector search should use, decided by an experiment

Before any file format or user interface is built, an opt-in test (test_retrieval_candidates in tests/test_real_model.py, helpers in tests/retrieval_eval.py) measured four predeclared ways of turning a 16-frame window into one vector. It used the 50 Kinetics-mini validation clips (five actions, ten clips each) at frozen dataset and model revisions. The candidates, metrics, controls, pass bars and decision rule were fixed in the plan before anything was measured.

	Validation set (chooses)	Training set (confirms)
(a) mean of all final tokens, plain cosine: nearest other clip has the same action	42/50 (0.84)	44/50 (0.88)
Best colour-histogram baseline	15/50 (0.30)	15/50 (0.30)
A single repeated frame through the same model	35/50 (0.70)	35/50 (0.70)
Chance	about 0.18	about 0.18

• Every candidate passed a localisation check, in which a search from inside a clip must land on the right moment of a composite video, 16 of 16 times. A deliberately broken version that gives every window its clip's average passed 0 of 16.
• Three candidates tied at 42/50, so the rule chose the simplest, (a). Centring on the folder mean scored 39/50 and stays an explicit option rather than the default.
• The single-frame control shows that on these five classes, which differ by scene, search mostly reflects the scene, with motion adding 14 to 18 points. The result is exploratory and will be described that way in the README.

T1: .latent format version 2

• A new tensor, timeline_pooled ([tubelets, D] float32), holds the mean of each timeline tubelet's patches from the same tokens as pooled. The trajectory view in Week 4 will plot it, and folders indexed from now on will already carry it.
• A new header block, representation, says how a window vector was made: the model, an identity of its weights' content (Hugging Face blob hashes, or size and modification time for a local folder), the preprocessing hash, the token layer, the pooling, the window settings and the output version, with a hash over them. The folder index in later tasks will refuse to mix vectors whose hashes differ.
• The cache reuses a file only when its format version, output version and representation match what the current settings would produce, and otherwise re-embeds it in place with a message. save now always writes the current format version.
• OUTPUT_VERSION is unchanged, because no existing tensor changes. Version-1 files are refused with a message to re-embed.

Checks

• The fast suite passes (90 tests). New tests cover the scoring helpers, the format and cache rules, and the weights identity, including a fake Hugging Face cache and a retrained local folder. Each new rule's test was checked to fail when the rule is broken.
• On the real model, an existing version-1 cache file was refused, re-embedded at the same path, and reused on the next run.
• The plan had an adversarial review by a second model and an outside review before approval. The T0 code and the T1 change each had fresh-context reviews, and their findings are fixed. The execution log beside the plan records every deviation and result.

Reproduce T0

JEPA_SCOPE_MODEL_TESTS=1 uv run pytest tests/test_real_model.py -s -k "retrieval_candidates and val"
JEPA_SCOPE_MODEL_TESTS=1 JEPA_SCOPE_EVAL_DEFAULT=a uv run pytest tests/test_real_model.py -s -k "retrieval_candidates and train"