[ryangswe]

This is the second pull request of the Weeks 3 and 4 plan (docs/plans/2026-11-06-weeks3-4-folder-search-trajectory.md), covering tasks T2 to T5. It adds a folder index, search over it, and two commands, jepa-scope index and jepa-scope search. The server and viewer do not use the index yet; that is the next pull request.

T2: the folder index file

• A folder index, jepa-scope.index, is a safetensors file with a validated JSON header (src/jepa_scope/index_file.py, format version 1). Its tensors are each window's pooled vector, each window's start time, the clip each window belongs to, and the folder mean.
• The header records the representation hash from the .latent files, the run settings (maximum seconds, device and dtype), the folder root, the clips with their SHA-256, size and modification time, the clips that were skipped and why, and whether the folder was indexed recursively.
• Every clip in an index must share one representation hash, so vectors made by different models, preprocessing or settings are never mixed.

T3: search

• Search (src/jepa_scope/search.py) is a brute-force cosine search over the index, rescored in float64 so that ties and ordering are stable. It finds either the nearest moments or the nearest clips.
• By default a search from inside a clip leaves that clip out, so results are moments of other videos. Overlapping windows are decided by window number, not by floating-point times. k is capped at 1000.

T4: the indexer and the commands

• jepa-scope index <folder> embeds every video in the folder that is not already cached, then writes the index. A second run with nothing new reuses every cached .latent file and does not load the model.
• The cache rule moved from the embedding code into src/jepa_scope/cache.py, so the indexer, the single-clip command and later the server share one rule for when a cached file can be reused. Cache file names are unchanged, and a test pins them.
• A clip that fails to decode is skipped and remembered with its hash, size and modification time, so it is not retried until it changes or the package version changes. A failure that may be temporary, such as a full disk, a permission error or running out of memory, is retried on the next run.
• --centre (subtract the folder mean before comparing) and --recursive are remembered in the index, so a later refresh without the flag keeps the earlier choice, even when the index has to be rebuilt.
• The index is rebuilt when its representation, run settings or format version differ from the current ones, when it cannot be read, or when it belongs to another folder.
• jepa-scope search <folder or index> <query> accepts a clip in the index, a relative path, or a video outside the folder. An outside video is embedded only if the cache does not already hold it, and only then does it need the index's device.
• .latent files are now written to a temporary file and renamed, so an interrupted run cannot leave a half-written file.

T5: search reproduces the retrieval experiment

An opt-in test (test_search_reproduces_experiment in tests/test_real_model.py) indexes the same 50 Kinetics-mini validation clips that the first pull request's experiment used, through the real indexer and search code, and compares the results with the experiment's.

	Experiment (first pull request)	Folder index and search
Nearest other clip has the same action	42/50	42/50, with the same count for every action
Nearest window in another clip has the same action	0.7427	0.7427
A search from inside a clip lands on the right moment of a composite video	16/16	16/16

The test fails if any of the three results drifts. It was checked to fail when the moment search is deliberately broken.

When a search is allowed to return moments from the query's own clip (--same-clip), 9 of the 16 composite queries find a later moment of their own video first. Within one video, the most similar moments are mostly other moments of the same scene. The default search, which leaves the query's clip out, is not affected. The README will say this when the Week 4 documentation lands.

Timings on an Apple-silicon laptop (MPS)

• Indexing a folder of 10 Kinetics clips took 34 seconds the first time, about 0.32 seconds of embedding per second of video.
• Refreshing an index with nothing new takes 3 to 4 seconds and does not load the model. Most of that time is spent importing the model libraries to describe the model.
• A search takes about 0.7 seconds from the command line.

Checks

• The fast suite passes, and ruff, the workflow check and ./scripts/build.sh pass.
• New tests cover the index format and its validation, search ordering and masking, the indexer's refresh, rebuild and skip rules, the remembered choices, query resolution and the commands. Each new rule's test was checked to fail when the rule is broken.
• Each task had a fresh-context review, and the findings are fixed. The execution log beside the plan records every deviation and result.

Reproduce T5

JEPA_SCOPE_MODEL_TESTS=1 uv run pytest tests/test_real_model.py -s -k "search_reproduces_experiment"