A case's head is read from the held-out home, pinned by the case, and held to its tree (#611, record NNNN)

adapters/AGENTS.md

@@ -33,7 +33,7 @@ Not built yet, by the stories that hold their rules: contract.py, binding.py, is
 | An issue asks for a contract only by an Intent that asks or a person's `factory:contract` label; if none asks, `acceptance` is null and whole | `tests/test_adapters.py::test_an_issue_asks_for_a_contract_only_by_a_persons_label` |
 | Closing references read short, of another repo or named twice, are not whole | `tests/test_adapters.py::test_closing_references_are_whole_only_of_this_repo_and_every_one_counted` |
 | A pull request to another base than the default is not whole on its contract | `tests/test_adapters.py::test_a_pull_request_to_another_base_than_the_default_is_not_read_whole_on_its_contract` |
-| A held-out case is the bytes its manifest hashed, or none and a code; one that could name another file is not asked | `tests/test_adapters.py::test_a_held_out_case_is_its_hashed_bytes`, `tests/test_adapters.py::test_a_held_out_case_not_read_whole_says_why` |
+| A held-out case and its head are the bytes pinned, or none and a code | `tests/test_adapters.py::test_a_held_out_case_not_read_whole_says_why`, `tests/test_adapters.py::test_what_the_home_keeps_beside_a_case_is_read_raw_by_the_hash_the_case_pins` |
 | The title, and the head the snapshot was held to, are handed beside the facts, never in them | `tests/test_adapters.py::test_the_pull_requests_own_facts_and_the_title_beside_them`, `tests/test_adapters.py::test_a_snapshot_hands_back_its_head` |
 | A plan of writes not its pass record's sends nothing; a dispatch is sent once, never retried | `tests/test_effects.py::test_a_write_list_that_is_not_the_records_is_refused_whole`, `tests/test_effects.py::test_a_dispatch_is_sent_once_and_never_retried` |
 | Each write goes through its one credential, which sends only its own requests by path and body | `tests/test_effects.py::test_each_write_goes_through_its_one_credential_and_its_allowlist` |

adapters/heldout.py

@@ -1,6 +1,7 @@
 """The held-out home's cases, read for the eval job alone (design §3.13 and §4.9, issue 309): each case a manifest names,
 `GET /repos/{owner}/{repo}/contents/{role}/{id}.json`, through the one transport, with the Maintainer's read-only token for the
 home and nothing else (decision 0053). A case is handed back as the bytes its manifest hashed, or not at all, with a code that says why.
+So is what the home keeps beside it, its head's tarball and its commit's tree, each by the sha256 the case pins (issue 611).
 
 The home's owner and name are the job's environment's, never this repo's, and no refusal, code or message here names
 them, a case's path or a byte of a case: what the job prints is counts and codes. A case is read from GitHub's JSON
@@ -11,9 +12,13 @@
 import hashlib
 import re
 
-from adapters.github import Unanswered, request
+from adapters.github import TARRED, Unanswered, raw, request
 
 LARGEST = 1024 * 1024       # bytes a case may be: a facts snapshot and a diff; GitHub inlines a file's content to 1 MiB
+RAW = "application/vnd.github.raw"      # a file's bytes as they are, whatever its size (P-62)
+# What the home keeps beside a case, by its extension, and the bytes each may be: the head's tarball as the transport
+# bounds one, and its tree, which GitHub truncates past 7 MB, as `harness/head.tree` refuses one truncated.
+BESIDE = {"tar.gz": TARRED, "tree.json": 8 * 1024 * 1024}
 OWNER = re.compile(r"[A-Za-z0-9](?:[A-Za-z0-9-]{0,37}[A-Za-z0-9])?")
 NAME = re.compile(r"[A-Za-z0-9_.-]{1,100}")
 PART = re.compile(r"[a-z0-9][a-z0-9-]*")      # a Role or a case id, as eval-set/1 holds them: no dot, no slash
@@ -64,3 +69,33 @@ def case(transport, owner: str, name: str, role: str, entry) -> tuple:
         return None, "hash_differs"
     return data, None
 
+
+def beside(transport, owner: str, name: str, role: str, ident: str, kind: str, sha256) -> tuple:
+    """`(bytes, None)` for the file the home keeps beside a case, `{role}/{ident}.{kind}`, its head's tarball (`tar.gz`)
+    or its commit's tree (`tree.json`), read whole with the raw media type and hashing to the sha256 the case pins;
+    `(None, code)` otherwise, as `case` says them, and `too_large` over the kind's bound (issue 611). The raw media type
+    answers a file of more than 1 MiB whole, in one answer and no redirect (trials/P-62.json), which the JSON answer
+    `case` reads does not: it holds a file's content to 1 MiB alone."""
+    if not (isinstance(role, str) and PART.fullmatch(role) and isinstance(ident, str) and PART.fullmatch(ident)
+            and kind in BESIDE and isinstance(sha256, str) and SHA256.fullmatch(sha256)):
+        return None, "not_this_set"
+    if not home(owner, name):
+        return None, "unanswered"
+    try:
+        answer = transport.send(request("GET", "/repos/%s/%s/contents/%s/%s.%s" % (owner, name, role, ident, kind),
+                                        accept=RAW))
+    except Unanswered:
+        return None, "unanswered"
+    if answer["status"] == 404:
+        return None, "not_found"
+    body = answer["body"]
+    if answer["status"] != 200 or not isinstance(body, str) \
+            or not str(answer["headers"].get("content-type", "")).startswith(RAW + ";"):
+        return None, "unanswered"
+    data = raw(body)
+    if len(data) > BESIDE[kind]:
+        return None, "too_large"
+    if hashlib.sha256(data).hexdigest() != sha256:
+        return None, "hash_differs"
+    return data, None
+

docs/decisions/0138-the-review-reads-the-pull-requests-head-with-the-read-tools-alone-held-to-it-by-restricted.md

@@ -41,7 +41,7 @@ What is owed: what codeload's address holds for a private repo read with an App'
 | Each archive it refuses, it refuses whole | `tests/test_harness.py::test_each_archive_the_unpacker_refuses_it_refuses_whole` |
 | It holds its caps, writes into a new folder, and nothing outside it | `tests/test_harness.py::test_the_unpacker_holds_its_caps_and_writes_nothing_outside_its_folder` |
 | An eval case requires its head, as 40 lowercase hex characters | `tools/check_examples.py`, `examples/invalid/eval-case.no-head.json`, `examples/invalid/eval-case.a-head-as-an-object.json`, `examples/invalid/eval-case.a-short-head.json` |
-| Each case's head is read with no credential and removed once asked; one not read closes the gate | `tests/test_evals.py::test_each_cases_head_is_read_with_no_credential_and_one_not_read_closes_the_gate` |
+| Each case's head is read with no credential and removed once asked; one not read closes the gate | `tests/test_evals.py::test_each_cases_head_is_read_from_the_home_by_its_pins_and_one_not_read_closes_the_gate`, `harness/eval/__init__.py`, mutant: "the eval fetch reads no case's head" |
 | A case is reviewed with the read tools in its head, as a live Run is | `tests/test_evals.py::test_a_case_is_reviewed_by_the_task_and_the_engine_a_live_run_is` |
 | The read tools are held to the folder at the pin, and by which flag | `trials/P-59.json`, `tests/test_tools.py::test_the_read_tools_on_a_linux_runner_are_held_to_their_folder_only_with_both_flags` |
 | The tarball is one redirect to codeload, which redirects no further over HTTPS | `trials/P-60.json`, `tests/test_tools.py::test_a_commits_tarball_is_one_redirect_to_codeload_which_never_redirects_over_https` |

docs/decisions/NNNN-a-cases-head-is-read-from-the-held-out-home-pinned-by-the-case-and-held-to-its-tree.md

@@ -0,0 +1,29 @@
+# NNNN. A case's head is read from the held-out home, pinned by the case, and held to its tree
+
+Status: accepted. The Factory Builder's rulings of 2026-11-08 on issue 611, "A case's head is read from the private home, pinned by its hash, and the run is sharded", points (a) to (d): this is the first of its two pull requests, the heads; the second shards the run. It amends decision 0138's rows "A transport with no token" and "The eval case" where they say the eval job reads a case's head with no credential, and its Boundaries row "Each case's head is read with no credential and removed once asked", whose check is now the test of the read from the home; and decision 0142's row "An eval case's head", which this closes. Nothing else in either changes.
+
+## Decision
+
+| What | Decided | Because |
+|---|---|---|
+| Where a head is kept | Beside each case in the held-out home, `<role>/<id>.json`, the home keeps the case's head as two files: `<id>.tar.gz`, the commit's tarball as GitHub archived it, and `<id>.tree.json`, the commit's tree, every level of it, as the API answered it. `eval-case/1` requires `head_sha256` and `tree_sha256`, the sha256 of each, beside `head` | The issue's: a head read anonymously shares the anonymous cap, 60 an hour, so a pool of three runs, or a run again within the hour, can fault on `head_unreadable`; and a head that is not on GitHub cannot be read at all. Pinned by the case, which the manifest pins, the head is the bytes the Maintainer chose |
+| How it is read | `fetch` reads both files with the home's token, through the transport the case is read with, by `beside` in `adapters/heldout.py`: each with the raw media type, `RAW` in `adapters/heldout.py`, whose answer is a file's bytes whole however large, in one answer and no redirect (`trials/P-62.json`). An answer of another type, `+json` among them, which the transport would read as JSON, is no file. A file over its bound (the transport's tarball bound, `TARRED`, or 8 MiB for a tree) is `too_large`, and one that does not hash to its pin `hash_differs`. Nothing in the eval job reads with no credential, and the home's token is still sent nowhere else | The Factory Builder's ruling (c): no transition, and no case without its pins, since an anonymous read kept for one would keep the read the issue removes. Ruling (d): the raw media type, grounded by its trial, the git blobs API its fallback; and the ruling on P-62's verdict, that the raw media type is GitHub's `vnd.github.raw`, which the transport must ask for |
+| How it is held | The tarball is unpacked as before (decision 0138), and then held to the tree file by file, by git's blob hash, by the check a live head is held by (decision 0142): `tree_of` in `harness/head.py` reads the tree file with the checks `tree` reads GitHub's answer with, and `held` holds the head to it. A head not the one pinned is `hash_differs`, a fault of the run; one not read, unpacked or held whole is `head_unreadable`, the Engine's or the Factory's. Either closes the gate | The Factory Builder's ruling (b): pin the tree as well as the tarball, and hold every eval head to it. 0142 left an eval head unheld for want of room under the anonymous cap; read with the home's token, there is room. An eval case is now graded on the files a live review would be given |
+| Who puts them there | The Maintainer, at his terminal, in his checkout of the home: `GH_TOKEN="$(gh auth token)" python3 -m tools.heads_into_home <home> <role folder>`. For each case of the folder, it reads the head's tarball and tree from GitHub through the one transport, read-only, holds the one to the other, writes both beside the case, the tree compact as GitHub's answer held it, and writes the case anew with their sha256 after its `head`; a case it cannot pin, or whose head is over what the job reads, it leaves as it was. Each file is written whole and moved over its name, so a link there is replaced, never written through, and a case or a folder that is a link is not read. It prints counts, then each pinned case's id and the sha256 of its new bytes, never a byte of a case. It takes a role folder and touches nothing else, so a gate set made later is pinned as the review Intent's set is | The Factory Builder's ruling (c): the tool serves the coming gate set too, and prints counts and hashes alone. A session never sees a case |
+| The set changes with it | Pinning changes each case's bytes, so the set's manifest under `evals/` takes the hashes the tool prints, in the pull request that brings this change. It merges only once the home holds the files, and once `trials/P-62.json` is on main | The Factory Builder's ruling (c) and (d) |
+| The canary | `harness/eval/canary.py` still reads its case's head with no credential: its case is this repo's own example, of this public repo, never one of the home's, and pins no file the home keeps | The canary is no step of the eval job (decision 0106) |
+| Residual: a head only in the home | The tool reads a head from GitHub. A head that is not there, one built for a case, is written beside its case by hand, its tarball as `git archive` writes one with the commit in its pax header and one folder, and its tree in the API's shape; nothing builds that yet | No case of the set needs one yet. The eval reads it as it reads any other |
+| Residual: the home's answer | P-62 asked a public repo. Should the API answer a private repo's file otherwise, a redirect among them, the read fails closed: the transport follows no redirect for a file, and a head not read whole is `head_unreadable` | Nothing the eval reads can be given in its place; the first dispatch after the Maintainer's run shows it |
+
+## Why
+
+A case's head was read with no credential, from the public repo its change was made in, which ties every run of the set to an hourly cap it shares with nothing else of the Factory's, and to GitHub keeping the commit. Kept in the home and pinned by the case, the head is read with the token every case is already read with, and is the same bytes on every run. The tree beside it lets the eval hold the head to the commit as the live review does, so a case can no longer be graded on files the live review would refuse.
+
+## Boundaries
+
+| Boundary | Check |
+|---|---|
+| What the home keeps beside a case is asked with the raw media type and the home's token, and handed back only as the bytes the case pins: an answer of another type, `+json` among them, one not found, refused, over its bound or of another hash is none, and a name, kind, pin or home that could name another file is not asked | `tests/test_adapters.py::test_what_the_home_keeps_beside_a_case_is_read_raw_by_the_hash_the_case_pins`, `trials/P-62.json`, `adapters/heldout.py`, mutant: "the eval blob is asked for as raw+json", mutant: "a head beside a case of another hash is read", mutant: "what the home keeps beside a case is asked as JSON", mutant: "a head of any size is read", mutant: "a head answered under another type is read", mutant: "a head answered as raw JSON is read", mutant: "a head GitHub did not find is unanswered", mutant: "a head GitHub refused is read", mutant: "a file of any kind is asked beside a case", mutant: "a pin of any shape is asked by", mutant: "a file beside a case of any name is asked", mutant: "a home of any name is asked beside a case" |
+| The fetch reads each case's head from the home by its pins, with no other transport, unpacks it and holds it to its tree; a head not the one pinned is `hash_differs`, one not read, unpacked or held whole `head_unreadable` and removed, a pin that is no sha256 reads nothing and the case is `not_this_set`, and the gate closes | `tests/test_evals.py::test_each_cases_head_is_read_from_the_home_by_its_pins_and_one_not_read_closes_the_gate`, `harness/eval/__init__.py`, `harness/eval/fetch.py`, `harness/head.py`, mutant: "the eval fetch reads no case's head", mutant: "a head is unpacked and never held to its tree", mutant: "a head not the one pinned is read as unreadable", mutant: "the tree's pin is never read", mutant: "a head is read beside another Role", mutant: "a tree that cannot be read stops the fetch", mutant: "a head not held is left in the heads folder", mutant: "a case whose pin is no sha256 has its head read" |
+| An eval case requires both pins, each a whole sha256 | `tests/test_evals.py::test_a_case_dropped_under_evals_in_any_file_is_caught`, `tools/check_examples.py`, `schemas/eval-case.schema.json`, `examples/invalid/eval-case.no-head-pin.json`, `examples/invalid/eval-case.no-tree-pin.json`, `examples/invalid/eval-case.a-head-pin-cut-short.json`, `examples/invalid/eval-case.a-tree-pin-cut-short.json`, mutant: "a case need pin no head", mutant: "a case need pin no tree", mutant: "a head pin may be cut short", mutant: "a tree pin may be cut short" |
+| The Maintainer's tool writes a head beside its case only once it is held to its tree and within what the job reads, pins what it wrote, writes each file whole and through no link, reads no case or folder that is a link, leaves a case it cannot pin as it was, reads no tree file as a case, prints counts and hashes alone, and says so in its exit; what it writes is what the fetch reads | `tests/test_evals.py::test_the_maintainers_tool_pins_each_head_beside_its_case_and_prints_counts_and_hashes_alone`, `tools/heads_into_home.py`, mutant: "the Maintainer's tool writes a head not held to its tree", mutant: "the Maintainer's tool pins a case of another id", mutant: "the Maintainer's tool reads a tree answered otherwise", mutant: "the Maintainer's tool says every case was pinned", mutant: "the Maintainer's tool reads a tree it wrote as a case", mutant: "the Maintainer's tool pins a tree it did not write", mutant: "the Maintainer's tool reads with no token", mutant: "the Maintainer's tool writes through a link", mutant: "the Maintainer's tool reads a case that is a link", mutant: "the Maintainer's tool pins a head the job will refuse", mutant: "the Maintainer's tool reads a folder that is a link", mutant: "a write that fails stops the Maintainer's tool", mutant: "the Maintainer's tool leaves a file half written", mutant: "the Maintainer's tool writes a head beside a case it cannot pin" |

examples/eval-case.also-at.json

@@ -238,6 +238,8 @@
   },
   "diff": "--- a/kernel/decide.py\n+++ b/kernel/decide.py\n@@ -86,9 +86,9 @@\n-    if not items:\n-        return []\n     first = items[0]\n+    if not items:\n+        return []\n     return parse(first, items[1:])\n",
   "head": "c514ff021d93934192c9dbb14b6c31c2b421e6c7",
+  "head_sha256": "6bbe1f08b91c69085d33a368e265ba4f7ef7438228aa0047546ab018b61eeec2",
+  "tree_sha256": "c8037a8b9d2fca258427534558b219fa94635816faf136ab59684a97d2429010",
   "expected": {
     "verdict": "finding",
     "criterion": "PR-3",

examples/eval-case.json

@@ -238,6 +238,8 @@
   },
   "diff": "--- a/kernel/decide.py\n+++ b/kernel/decide.py\n@@ -86,9 +86,9 @@\n-    if not items:\n-        return []\n     first = items[0]\n+    if not items:\n+        return []\n     return parse(first, items[1:])\n",
   "head": "c514ff021d93934192c9dbb14b6c31c2b421e6c7",
+  "head_sha256": "6bbe1f08b91c69085d33a368e265ba4f7ef7438228aa0047546ab018b61eeec2",
+  "tree_sha256": "c8037a8b9d2fca258427534558b219fa94635816faf136ab59684a97d2429010",
   "expected": {
     "verdict": "finding",
     "criterion": "PR-3",

examples/eval-case.met.json

@@ -238,6 +238,8 @@
   },
   "diff": "--- a/kernel/decide.py\n+++ b/kernel/decide.py\n@@ -86,9 +86,9 @@\n-    if not items:\n-        return []\n     first = items[0]\n+    if not items:\n+        return []\n     return parse(first, items[1:])\n",
   "head": "c514ff021d93934192c9dbb14b6c31c2b421e6c7",
+  "head_sha256": "6bbe1f08b91c69085d33a368e265ba4f7ef7438228aa0047546ab018b61eeec2",
+  "tree_sha256": "c8037a8b9d2fca258427534558b219fa94635816faf136ab59684a97d2429010",
   "expected": {
     "verdict": "met",
     "criterion": "PR-1",

examples/eval-case.with-issue.json

@@ -238,6 +238,8 @@
   },
   "diff": "--- a/kernel/decide.py\n+++ b/kernel/decide.py\n@@ -86,3 +86,5 @@\n def parse(items):\n+    if not items:\n+        return []\n     first = items[0]\n     return parse(first, items[1:])\n",
   "head": "c514ff021d93934192c9dbb14b6c31c2b421e6c7",
+  "head_sha256": "6bbe1f08b91c69085d33a368e265ba4f7ef7438228aa0047546ab018b61eeec2",
+  "tree_sha256": "c8037a8b9d2fca258427534558b219fa94635816faf136ab59684a97d2429010",
   "expected": {
     "verdict": "met",
     "criterion": "PR-1",

examples/invalid/eval-case.a-category-nobody-owns.json

@@ -237,6 +237,8 @@
   },
   "diff": "--- a/kernel/decide.py\n+++ b/kernel/decide.py\n@@ -86,9 +86,9 @@\n-    if not items:\n-        return []\n     first = items[0]\n+    if not items:\n+        return []\n     return parse(first, items[1:])\n",
   "head": "c514ff021d93934192c9dbb14b6c31c2b421e6c7",
+  "head_sha256": "6bbe1f08b91c69085d33a368e265ba4f7ef7438228aa0047546ab018b61eeec2",
+  "tree_sha256": "c8037a8b9d2fca258427534558b219fa94635816faf136ab59684a97d2429010",
   "expected": {
     "verdict": "finding",
     "criterion": "PR-3",

examples/invalid/eval-case.a-head-as-an-object.json

@@ -240,6 +240,8 @@
   "head": {
     "sha": "c514ff021d93934192c9dbb14b6c31c2b421e6c7"
   },
+  "head_sha256": "6bbe1f08b91c69085d33a368e265ba4f7ef7438228aa0047546ab018b61eeec2",
+  "tree_sha256": "c8037a8b9d2fca258427534558b219fa94635816faf136ab59684a97d2429010",
   "expected": {
     "verdict": "finding",
     "criterion": "PR-3",