Repository navigation
lab L2: the t27b lab's decisions as .t27 specs, lab.py as the oracle - #11079
Merged
Merged
Conversation
…loses #11042) Five specs under specs/lab/ port the pure decisions of the t27b lab (contrib/railway/t27b-lab/lab.py and the checker it runs, scripts/tri_loop/t27b.py ratchet; both unchanged) as functions of text and numbers: how test-report and cargo output are read (parse_test_report, parse_test_verdicts, disagreements, parse_cargo_test, the verdict of a reference run, the summary block of a run), the reference cache (use_closure in pathlib's order, reference_key as a streamed sha256, reference_cacheable, the hit rule, the stamp), the per-spec ratchet of a run against the ledger, and the run's own rules (run_done, admitted with the judge as a function, the loop's schedule question, how the checker's process is read, the index after a publish). Every table was computed by running the Python itself: 1392 cases, 0 differences, 46 mutants killed. They name no host and assume no single machine. Reused, not copied: specs/lab/text.t27 (L1), receipt.t27, steward.t27, ledger_steward.t27, corpus_receipt.t27, sha256.t27. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
gHashTag
enabled auto-merge (squash)
October 19, 2026 15:01
Contributor
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #11042. Part of the lab epic #11040.
Five specs under
specs/lab/port the pure decisions of the t27b lab as functions of text and numbers, with the Python as the oracle until the cutover:contrib/railway/t27b-lab/lab.pyand the checker it runs,scripts/tri_loop/t27b.py ratchet, both unchanged. They name no host and assume no single machine: whoever runs a lab gate (one lab, or any node that takes a lab job, #11043) hands them the text and the facts and gets the decision back.Functions ported
parse_test_reportlr_report_verdict,lr_report_detail(the verdict as steward.t27's code)parse_test_verdictslr_verdicts(a table of three words a test),lr_table_textdisagreementslr_disagreementsparse_cargo_testlr_cargo_totalsreference_one, once the process has endedlr_ref_verdict,lr_ref_detail,lr_over_detail,lr_spawn_detail,lr_keeps_testslab_run(run-document shape)lr_summary_field,lr_put_summary,lr_put_reference_summary,lr_summary_crashreference_cacheablelc_cacheablereference_all, its log line andreusedtextlc_hit_usable,lc_log_line,lc_reusedreference_stamp,first_linelc_stamp,lc_first_lineuse_closurelc_closure(lc_reach,lc_path_cmp: pathlib's component order)reference_keylc_key_begin,lc_key_file,lc_key_hex(streaming sha256 overspecs/tri/crypto/sha256.t27)ratchet()of t27b.py: every ledger row against the run, the unlisted specs, the caplt_ratchet(reads the run and the ledger as written; counts, status, the lines and the summary linetri t27b ratchetprints)run_donelt_run_doneadmittedlt_admit(the judge isrequest_verdictandrequest_exitof corpus_receipt.t27, called as functions),lt_facts,lt_has_word,lt_age_u32schedulequestion and answer (main)lt_schedule_facts,lt_schedule_nextratchet(doc, log)of lab.py: how the checker's process is readlt_step_kind,lt_step_ok,lt_step_errorpublish: the index of runs, paths of a run's fileslt_index_publish,lt_run_path\d,\w, saturating counts (what LabText lacks)lx_*Left out, and why
Everything that starts a process, touches git or the disk or the network, or serves files:
run,run_group,checkout,lab_run's steps,reference_one's process half and the worker pool,read_ref_cache/write_ref_cache,remote_sha, the receipt and ddc steps, the HTTP server,main's loop. That is the runtime, #11040 L3. The run document's step records and the toolchain/image blocks are written by it; what is here is the summary block, the keys of the index entry and the paths.lab.pyhas noself_check()(the t27c lab's has one), so there are no self-check cases to pin.Cases, and the zero differences
Every table was computed by running the Python itself (lab.py imported unedited; the checker through its CLI;
admittedwith master's real judget27c corpus-receipt admitand git's answers replaced by text; the two statements ofmain()taken out of lab.py's own source and run;publish,ratchetandreference_allwith their processes replaced). The inputs are small texts written for the table: the real shapes oft27c test-report(a pass; a fail with a vacuous pass; a spec that does not compile; a machine with no zig), then cases that break each rule on its edge.parse_test_report(verdict and detail)parse_test_verdictsdisagreementsparse_cargo_testreference_oneafter the process ended (+ 10 spawn errors, 6 timeouts)use_closureandreference_keyreference_cacheablereference_allitself)reusedtextfirst_linereference_stampt27b.py ratchet(exit code, nine counts, finding lines; 12 full outputs)run_doneadmitted(directories of requests, real judge)schedulefacts / answer readpublishindex1392 cases, 5676 runtime asserts in 84 tests, 0 vacuous.
Mutants
46 mutants, each a one-expression change in a function, each killed (a test fails; none fails to compile). text 8: a decimal that saturates a digit early,
\wwithout the underscore, CR LF read as two breaks, a sum that wraps, a dropped sign, a pad that writes the byte at cap, a length compared with>, a digit run that ends late. report 10: a number that follows its key with no space, a later BLOCKED line winning, a fail called blocked, the first verdict of a repeated test kept, the reference-only line worded as the t27b-only one,ignorednot required, the cut taken over the whole line, a summary count inverted, the last verdict word unreachable, the crash condition ignored. cache 9: a killed t27c's blocked verdict cached, a one-word entry used, directory-after-file order, the wildcard list ignored, a missing file hashed as 7 bytes, the NUL after the stamp dropped, stderr read for blank stdout, the closure written in reverse, the first tab field taken. ratchet 10: surrogate pairs not combined, a string read as a cap, the cap judged against one more row,asserts_countedread backwards, the repr quote choice, paths sorted backwards, a repeated row kept, no fall back to the detail, a search that walks the wrong way, an unknown word read as missing. run 9:run_doneinverted, a dropped request counted as waiting, a negative age read as fresh, three words accepted from the judge, the exit code forgotten, any nonzero exit read as no ratchet, the old entry of a commit kept in the index, a word matched inside another, the known flag written backwards.Reuse, nothing copied
specs/lab/text.t27(LabText, L1 #11066, on master): lines, spaces, characters and spans, read there off Python 3.14.specs/tri/lab/receipt.t27(the run layout reader),specs/tri/t27b/steward.t27andledger_steward.t27(entry_code,unlisted,over_cap,ratchet_is_red,is_pass, the verdict and finding codes),specs/verified/corpus_receipt.t27(request_verdict,request_exit),specs/tri/crypto/sha256.t27.python3 tools/dupe_scan.py --likeon each new file: nothing in it is written elsewhere.Found in lab.py while porting (reproduced, not repaired; lab.py is unchanged)
lab_runreadsr["reference"]of every merged result before it builds the summary, so a run in which the reference step produced nothing for some file raisesKeyErrorand is never published (and is retried every poll):lr_summary_crashis that condition; 12 of the 62 result sets are such runs.run()drops the oldest 2000 lines whenever the list passes 4000, soparse_cargo_testsums a long cargo run from its tail.first_lineof a command that printed nothing is the wordsunavailable: list index out of range(an IndexError it catches).Limits, each pinned by a test
A count above 18446744073709551615 reads as that number (the verdict never differs; only a detail that prints such a count does);
\dand\ware the ASCII classes; a table wider than the caller's buffer is no table; the checker's JSON is read injson.dumps(indent=1)'s layout and a ledger row must stay on one line; the repr of a bad reason is exact for ASCII and the listed Unicode spaces. Details are in each file's header.Gates
t27c test-reporton each new spec: FAIL 0, vacuous passes 0 (text 8 tests, report 29, cache 16, ratchet 14, run 17).t27c seal --savethen--verifyon each: all MATCH (seals under.trinity/seals/lab_LabT27b*.json).tri types ratchet:RATCHET: CLEAN.docs/reports/t27b_expectations.json, specs/analog/, the AnalogSar seals, .claude/.🤖 Generated with Claude Code