Repository navigation
spec(port): port specs/port/trios/crates/trios-data/src/lib.t27 (Closes #9499) - #9630
Conversation
gHashTag#9499) Port gHashTag/trios:crates/trios-data/src/lib.rs (at 8b229e9489ee, 123 lines) as specs/port/trios/crates/trios-data/src/lib.t27 - Roadmap Stage 6, goal gHashTag#4548, target t27c gen-rust. The FineWeb stub loader's decisions are ported as pure functions over the loader state and tested (6 tests, 6 pass, 0 vacuous, 20 runtime asserts, 100% function coverage), including the original's own three tests: the initial state (offset 0 under the 1000-sample stub epoch limit, the data path the stub ignores), the exhaustion gate at offset >= max_samples, the batch shape (batch_size sequences of 128 tokens, every token the dummy ID 42), the offset advancing by batch_size, and the reset to offset 0. The growable Vec of Vecs is the plumbing; the outcome struct carries its decisions. No function here moves data in or out, so none carries an undefined; transport. Verified: parse OK, spec-status IMPLEMENTED, typecheck 0 errors, coverage 8/8, test-report 6/6 pass, 0 BLOCKED, gen 95 lines with 0 'not yet implemented', boundary exactly this one file, dupe_scan: all 8 names unique across 2039 specs. Dmitrii Fedorov ([@dmitrii-f-t27](https://github.com/dmitrii-f-t27))
|
Correction of my earlier comment on this PR. The earlier text asked for a merge and said the red checks were base failures only. That was wrong while CI is red, and it is withdrawn. This is not a new merge request. Current state (head ae88001): 26 checks success, 3 failure. The CI is not green. The port: specs/port/trios/crates/trios-data/src/lib.t27 only, Closes #9499. Its own tests: 6 of 6 pass, 20 runtime asserts, checked natively with t27b on the latest base and by the reference. The three failures, precisely:
Tracking: #9631 (STEP1/STEP2), #9632 (rust_convert), #9633 with PR #9636 (gen_w), #9635 with PR #9637 (mesh-node crypto), #9634 (CLI), #9600 (ledger). @gHashTag: please keep the golden-chain merge pending until every check on the last SHA is green. Nothing is requested now. Dmitrii Fedorov (@dmitrii-f-t27) |
Summary
Port
gHashTag/trios:crates/trios-data/src/lib.rs(at8b229e9489ee, 123 lines) asspecs/port/trios/crates/trios-data/src/lib.t27— Roadmap Stage 6 (goal #4548), target t27c gen-rust. Closes #9499.The original is the batched FineWeb data loader stub:
newbuilds a FineWebBatch over a dataset path,next_batchanswers batches of dummy token sequences until the stub epoch limit,offset/batch_sizeread the state,resetrewinds to the start.Per the issue's rule ("Port the decision, not the plumbing"), the five listed functions are ported under their names as pure functions over the loader state — none moves data in or out, so none carries an
undefined;transport body; the growableVec<Vec<u32>>is the plumbing, and aFineWebBatchOutcomestruct carries the batch decision instead (availability, sequence count, tokens per sequence, next offset):FineWebBatch_new— the initial state: offset 0, the 1000-sample stub epoch limit, the data path kept in the signature exactly as the original keeps its unused stub parameter (lines 24-30);FineWebBatch_next_batch— the core decision: exhausted onceoffset >= max_samples(None), otherwise batch_size sequences of 128 tokens advancing the offset by batch_size (lines 40-53);FineWebBatch_offset/FineWebBatch_batch_size/FineWebBatch_reset— the readers and the rewind to offset 0 with everything else kept (lines 56-68);sequence_length(128),dummy_token_id(42),stub_epoch_limit(1000) — the stub content constants (lines 28, 45-48).Rust's
Optionbecomes the availability verdict on the outcome; the&mut selfadvance becomes thenext_offsetthe caller feeds back through the state.Verification (all run live from the worktree root, zig 0.16.0)
t27c parse: OK.t27c spec-status: IMPLEMENTED.t27c typecheck: 0 errors, 0 warnings.t27c coverage: 8/8 tested (100%) — no transports to leave untested.t27c test-report: 6 tests / 6 pass / 0 FAIL / 0 BLOCKED, 0 vacuous (20 runtime asserts), including the original's own three tests (test_new_batch_loader,test_next_batch_returns_correct_size,test_multiple_batches— the exhaustion loop's boundary pinned directly at 996 + 4 = 1000).t27c gen | grep -c 'not yet implemented': 0;gen | wc -l: 95 (> 12).grep -cE '^\s*(pub )?fn (FineWebBatch_new|FineWebBatch_next_batch|FineWebBatch_offset|FineWebBatch_batch_size|FineWebBatch_reset)\(': 5.grep -cE '^\s*test\s+(["|[A-Za-z_])': 6 (≥ 5).git status --short— exactly this one file.python3 tools/dupe_scan.py --name <fn>for all 8 names on a cleanorigin/masterworktree: no function named … in 2039 spec(s) — all unique.Dmitrii Fedorov (@dmitrii-f-t27)