Skip to content

spec(port): port specs/port/trios/crates/trios-data/src/lib.t27 (Closes #9499) - #9630

Merged
gHashTag merged 1 commit into
gHashTag:masterfrom
dmitrii-f-t27:zcode-port-9499-data-lib
Oct 17, 2026
Merged

gHashTag merged 1 commit into
gHashTag:masterfrom
dmitrii-f-t27:zcode-port-9499-data-lib

Conversation

@dmitrii-f-t27

Copy link
Copy Markdown
Contributor

Summary

Port gHashTag/trios:crates/trios-data/src/lib.rs (at 8b229e9489ee, 123 lines) as specs/port/trios/crates/trios-data/src/lib.t27 — Roadmap Stage 6 (goal #4548), target t27c gen-rust. Closes #9499.

The original is the batched FineWeb data loader stub: new builds a FineWebBatch over a dataset path, next_batch answers batches of dummy token sequences until the stub epoch limit, offset / batch_size read the state, reset rewinds to the start.

Per the issue's rule ("Port the decision, not the plumbing"), the five listed functions are ported under their names as pure functions over the loader state — none moves data in or out, so none carries an undefined; transport body; the growable Vec<Vec<u32>> is the plumbing, and a FineWebBatchOutcome struct carries the batch decision instead (availability, sequence count, tokens per sequence, next offset):

  • FineWebBatch_new — the initial state: offset 0, the 1000-sample stub epoch limit, the data path kept in the signature exactly as the original keeps its unused stub parameter (lines 24-30);
  • FineWebBatch_next_batch — the core decision: exhausted once offset >= max_samples (None), otherwise batch_size sequences of 128 tokens advancing the offset by batch_size (lines 40-53);
  • FineWebBatch_offset / FineWebBatch_batch_size / FineWebBatch_reset — the readers and the rewind to offset 0 with everything else kept (lines 56-68);
  • sequence_length (128), dummy_token_id (42), stub_epoch_limit (1000) — the stub content constants (lines 28, 45-48).

Rust's Option becomes the availability verdict on the outcome; the &mut self advance becomes the next_offset the caller feeds back through the state.

Verification (all run live from the worktree root, zig 0.16.0)

  • t27c parse: OK. t27c spec-status: IMPLEMENTED. t27c typecheck: 0 errors, 0 warnings.
  • t27c coverage: 8/8 tested (100%) — no transports to leave untested.
  • t27c test-report: 6 tests / 6 pass / 0 FAIL / 0 BLOCKED, 0 vacuous (20 runtime asserts), including the original's own three tests (test_new_batch_loader, test_next_batch_returns_correct_size, test_multiple_batches — the exhaustion loop's boundary pinned directly at 996 + 4 = 1000).
  • t27c gen | grep -c 'not yet implemented': 0; gen | wc -l: 95 (> 12).
  • grep -cE '^\s*(pub )?fn (FineWebBatch_new|FineWebBatch_next_batch|FineWebBatch_offset|FineWebBatch_batch_size|FineWebBatch_reset)\(': 5. grep -cE '^\s*test\s+(["|[A-Za-z_])': 6 (≥ 5).
  • Boundary: git status --short — exactly this one file.
  • python3 tools/dupe_scan.py --name <fn> for all 8 names on a clean origin/master worktree: no function named … in 2039 spec(s) — all unique.

Dmitrii Fedorov (@dmitrii-f-t27)

gHashTag#9499)

Port gHashTag/trios:crates/trios-data/src/lib.rs (at 8b229e9489ee,
123 lines) as specs/port/trios/crates/trios-data/src/lib.t27 -
Roadmap Stage 6, goal gHashTag#4548, target t27c gen-rust.

The FineWeb stub loader's decisions are ported as pure functions over
the loader state and tested (6 tests, 6 pass, 0 vacuous, 20 runtime
asserts, 100% function coverage), including the original's own three
tests: the initial state (offset 0 under the 1000-sample stub epoch
limit, the data path the stub ignores), the exhaustion gate at
offset >= max_samples, the batch shape (batch_size sequences of 128
tokens, every token the dummy ID 42), the offset advancing by
batch_size, and the reset to offset 0. The growable Vec of Vecs is
the plumbing; the outcome struct carries its decisions. No function
here moves data in or out, so none carries an undefined; transport.

Verified: parse OK, spec-status IMPLEMENTED, typecheck 0 errors,
coverage 8/8, test-report 6/6 pass, 0 BLOCKED, gen 95 lines with 0
'not yet implemented', boundary exactly this one file, dupe_scan:
all 8 names unique across 2039 specs.

Dmitrii Fedorov ([@dmitrii-f-t27](https://github.com/dmitrii-f-t27))
@dmitrii-f-t27

dmitrii-f-t27 commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor Author

Correction of my earlier comment on this PR. The earlier text asked for a merge and said the red checks were base failures only. That was wrong while CI is red, and it is withdrawn. This is not a new merge request.

Current state (head ae88001): 26 checks success, 3 failure. The CI is not green.

The port: specs/port/trios/crates/trios-data/src/lib.t27 only, Closes #9499. Its own tests: 6 of 6 pass, 20 runtime asserts, checked natively with t27b on the latest base and by the reference.

The three failures, precisely:

Tracking: #9631 (STEP1/STEP2), #9632 (rust_convert), #9633 with PR #9636 (gen_w), #9635 with PR #9637 (mesh-node crypto), #9634 (CLI), #9600 (ledger).

@gHashTag: please keep the golden-chain merge pending until every check on the last SHA is green. Nothing is requested now.

Dmitrii Fedorov (@dmitrii-f-t27)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Port gHashTag/trios:crates/trios-data/src/lib.rs (Rust, 5 functions) to specs/port/trios/crates/trios-data/src/lib.t27

2 participants