[KartikP]

The experiment API for UMI v2 in core: a study reads like a lab protocol (describe the task, bring in a subject, attach recorders, run once), and every subject sees each stimulus in its own native form. This folds in what was #210. Paired with KartikP/brainscore-unified#54; the two are pinned to each other (umi-v2.yml here, install/peer-revisions.json there).

What changes

1. Behavior and activity from one presentation. A BrainScoreModel session may ask for behavior and neural:<region> together. The regions are recorded during the same forward pass that produced the answer, and each stimulus is shown once.

• Rule 4.5: the session's task_context decides what the subject does; the requested outputs decide what is measured.
  • task_type='passive' means plain viewing.
  • A session that asks only for neural:* under a real task records during that task.
  • A task configured by an earlier session no longer captures a later neural-only session. That used to return the behavioral assembly.
• TaskContext.readout_fit: 'intact' fits a readout with every active change undone, then tests the changed network with it. The default 'current' is unchanged.
• Labelling: recordings made during a task carry attrs['recording']. Plain neural sessions are unchanged.

2. The form of the answer: TaskContext.response (brainscore_core.responses):

Response	Channels
Choice(labels)	behavior:choice
Choice(..., probabilities=True)	+ behavior:probabilities (implied)
Choice(..., confidence=Scale(...))	+ behavior:confidence (reported)
Scale(low, high, step)	behavior:rating
Text()	behavior:text

• Reported or implied: a reported answer is a parsed reply; an implied one is read from a readout or from likelihoods. Each result says which in attrs['measure'].
• Benchmarks own wording and parsing: each response builds its own directive and parses replies. An unparseable reply is recorded as invalid.
• New parts: reply(prompt) and likelihood(prompt, candidates). RatingRegressor gives ratings for readout models.
• Compatibility from declarations: each part declares which behavior:<kind> channels it can give, so mismatches are N/A before any compute.
• Existing task_type values keep their path on plain behavior.

3. Native input forms and ordered trials.

• Rule 5.9: vision is addressable as vision:still (declared by a vision-keyed extractor) and vision:clip (declared by a video-keyed one). A subject with both takes each stimulus in its own form, so a clip goes to the clip extractor.
• Order: a trial is shown as a Prompt in this order: the instruction, the stimulus parts in TaskContext.present order, then the response's directive. A stimulus with several parts and no declared order fails before any compute.
• Clips keep their time steps when recorded during a task.

Rule 9.3 reason: TaskContext is a session setting. Separating the task from the form of its answer needed task_type to become optional, which adding a field alone couldn't do. Every existing call keeps its meaning.

Out of scope, each raising a clear error before any compute:

• recording during likelihood scoring;
• recording with probabilities alongside a generated answer;
• composite regions, streaming and more than one time bin during a task.

Verification

• Unit tests:
  • test_joint_behavior_neural.py (10 tests): one presentation, prompt-dependent activity, readout_fit, neural-only sessions under a task, plain sessions unchanged.
  • test_task_responses.py (12 tests): parsing, confidence, ratings from a reply and a readout, text, likelihood and readout probabilities, N/A cases, trial order.
  • test_visual_forms.py (6 tests): form declarations, dispatch by form, clip time steps during a task.
• Full core suite: 1172 passed. The 8 failures fail the same way on unchanged v2 in my environment: plugin and conda tests break on a space in the checkout path, and one test needs a database.
• Real weights: Qwen3.6-27B on 4×A10G.
  • One presentation: 8 images × 2 conditions made 16 generate calls.
  • Recording during the answer: IT matched passive viewing to within bf16 rounding.
  • Likelihood choice: 8/8 correct, with probability 0.97 to 0.99 on the right colour.
  • Reported confidence: valid, and recorded during the answer.
  • Rating: circles 7, squares 1.
  • Text: faithful descriptions.
• Clips:
  • stills come back as (presentation, neuroid), and clips as (presentation, time_bin, neuroid): three 1-second bins for a 3-second clip at 2 fps;
  • a moving ball changes the activity across time (0.132), and a static clip does not change it at all (0.000);
  • "Which way does the ball move?", asked with the text before the video: 8/8 correct by reply, and 8/8 by likelihood with probability 0.994 to 0.996 on the right direction;
  • IT recorded during the answer keeps its time bins and is identical to passive viewing (max abs diff 0.0); the language recording is one vector per clip.