[KartikP]

Stacked on #209: merge that first, then retarget this PR to unified-model-interface-v2.

TaskContext.task_type mixed what a subject is asked to do with the form its answer takes. Ratings, reported confidence and written answers had no place in it. This adds TaskContext.response, built from the types in the new brainscore_core.responses:

Response	Answer	Channels
task_type='passive'	nothing; plain viewing	(neural only)
Choice(labels)	one label	behavior:choice
Choice(labels, probabilities=True)	also the implied probability of each label	+ behavior:probabilities
Choice(labels, confidence=Scale(1, 5, 1))	also a reported confidence, in the same reply	+ behavior:confidence
Scale(low, high, step)	a rating	behavior:rating
Text()	a written answer, kept as written	behavior:text

Reported or implied. A reported answer is the model's reply, parsed. An implied measure is read from the model's internals: a readout, or a language model's likelihoods, the same kind of link Futrell2018 uses when it reads reading times from surprisal. Every result says which in attrs['measure'].

Prompt wording and parsing belong to the benchmark (rule 11.3). Each response builds its own directive and parses replies, and a benchmark can subclass it to change either. An unparseable reply is recorded as invalid, with answer_valid, raw_answer and parse_error (rule 6.2).

New BrainScoreModel parts:

• reply(stimulus_row, prompt) -> str: the raw reply.
• likelihood(stimulus_row, prompt, candidates) -> {candidate: log p}: scores each candidate reply without generating.

RatingRegressor, a ridge readout, gives ratings for feature models.

Paths: prefer_path accepts 'generation', 'likelihood', 'readout' or 'auto', tried in that order. One benchmark can still take both kinds of model for choices, ratings and implied probabilities. Reported confidence and written answers need a model that replies.

Compatibility. behavior is now an addressable catalog family. Each part declares which behavior:<kind> channels it can give:

• readout: choice, probabilities, rating;
• answer: choice;
• reply: choice, rating, confidence, text;
• likelihood: choice, probabilities.

So, for example, a written-answer benchmark is N/A for a readout-only model before any compute (rule 5.3).

Existing calls are unchanged. task_type='label' / 'classification' / 'probabilities' keep their path on plain behavior, so existing scores don't move. Two tests that pin out_channels now also list the addressed channels that readout models truthfully declare.

Rule 9.3 reason. TaskContext is a session setting in the contract. Adding a field alone couldn't separate the task from the form of the answer without also making task_type optional. Embodied tasks stay on their own path.

Recording during these tasks uses #209's single presentation:

• the prompt pass of a reply;
• the readout's own extraction.

Recording during likelihood scoring, or together with probabilities alongside a generated answer, needs extra forward passes. Both raise a clear error before any compute.

Verification

• Unit tests: tests/test_task_responses.py (10 tests) covers:
  • parsing and invalid replies;
  • reported confidence;
  • ratings from a reply and from the ridge readout;
  • text;
  • likelihood probabilities;
  • readout probabilities;
  • N/A for reported answers from readout-only models;
  • plain behavior requests;
  • TaskContext validation.
• Full core suite: 1164 passed. The 8 failures are the same environment failures as on the base branch.
• Real weights: Qwen3.6-27B on 4×A10G, through unified's chat_parts (feat(experiments): record every kind of answer; chat reply and likelihood parts KartikP/brainscore-unified#55):
  • likelihood choice: 8/8 correct, probability 0.97 to 0.99 on the right colour;
  • reported confidence: valid labels, confidence 5 throughout; IT recorded during the answer, labelled as the prompt pass;
  • rating ("How round?"): circles 7, squares 1;
  • text: "The image displays a solid red circle centered on a white background."

[KartikP]

Folded into #209, which now carries these commits.