Repository navigation
pgsum rows: build GenomeOS PGS evidence batches natively - #9
Merged
Merged
Conversation
`pgsum rows` merges the `evidence` rows with the terms of a GenomeOS catalog pack and writes one JSON line per batch (digest, exact checkpoint delta, optionally the document), byte for byte what GenomeOS's Python builder writes (src/rows.rs lists the functions it mirrors). A non-ASCII source line exits 3 so GenomeOS builds that run in Python. HG002, 126 models: 354 sample batches identical; 200,000 terms in 4.4 s against about 28 s in Python. `evidence` keeps its output: its rows now come from `for_each_row`, and a record met twice is recognised by its line (identical output on three HG002 models, about 40% faster). mimalloc is the allocator. Version 0.6.0-dev. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…thon The task's numbers are written back into the digested documents. serde_json's default parser can read a float one unit in the last place off, and turns an integer beyond 64 bits into a float, so such a number would print differently from Python (GenomeOS week review round 2, finding 3; latent: today's only float, 1.959963984540054, reads exactly). Task::parse now reads each number token back and requires it to print to the same text; otherwise it exits with status 3 and GenomeOS builds the run in Python. No crate-wide parser change, so nothing else pgsum reads is affected. HG002: the 354 sample batches are still identical to Python's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner
Author
|
Added 498358a for review finding 3 (round 2): |
CI's stable jobs failed at cargo fmt --check; clippy then flagged one collapsible if. No behaviour change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
New command,
pgsum rows: builds GenomeOS's per-term PGS evidence batches, byte for byte what GenomeOS's Python builder (pgs_pgsum_rows._build) writes:evidencerows with each term's source row, description and orientation from a GenomeOS catalog pack;src/rows.rslists every Python function it mirrors (canonical JSON with Python's float repr,decimal.Decimalarithmetic,describe,orientation,weight, the reducer).Non-ASCII source lines: exit with status 3, so GenomeOS builds that run in Python. Python's text rules differ from ASCII ones only there.
evidenceoutput unchanged: its rows now come fromfor_each_row, and a record met twice is recognised by its line instead of by re-serializing it. About 40% faster.Allocator: mimalloc. With glibc malloc, rows built in one thread and freed in the other were about 3× slower.
Version: 0.6.0-dev. Tagging and releasing 0.6.0 is a separate step.
Checks (HG002, run D's inputs)
evidenceoutput is byte-identical to this branch's base (main74ecb0a) on 100,000-term ranges of three models.cargo test --release: 134 passed, 0 failed.The GenomeOS side is DylanCosto/GenomeOS branch
pgs-rows-rust-20261022. It usesrowswhen the pinned pgsum has it, rebuilds each run's first batch in Python as a check, and falls back to Python for older pgsum.🤖 Generated with Claude Code