Repository navigation
docs(streaming): bring the task ledger and the checkpoint contract up to what landed (#205, #206, #207) - #400
Merged
Merged
Conversation
… to what landed (#205, #206, #207) The ledger stopped at the header and the R13-R2000 walk, and the README's checkpoint contract described the header checkpoint alone and said full object and decompression checkpoints remain open. Both now cover the ten checkpoint kinds on main and the walkers of #240, #246, #260, #276, #299, #312, #333, #340 and #363, the perf series #360 to #366, and cite the evidence files of Streaming E2E run 40532346151. What was not measured is marked NOT MEASURED.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
7 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings
docs/streaming/task-ledger.mdand the README's "Checkpoint contract" section up to what is onmain, from the code and from the evidence files of a green Streaming E2E run; anything without a run or file behind it is marked NOT MEASURED. Documentation only.Spec reference
None; no decoder changes. Refs #205, #206, #207 (they stay open: their remaining items are what the ledger now lists), and #209, #210, #211, #212 for their ledger rows.
Evidence: the required Streaming E2E job of run 40532346151 on
mainate7c53e1, success, artifactstreaming-evidence(one*-evidence.jsonper step). The same commit'sci.ymlrun is 40532346172.Test plan
walk-evidence.json(17 file cases, 5,835 records, each withlegacy_paritytrue and the oracle "independent framing parse"),lz77-evidence.json(25 files, 266 pages, 6,097,392 bytes),map-evidence.json,sections-evidence.json,file-evidence.json,objects-evidence.json,r2007-evidence.json,r2007-objects-evidence.json,source-evidence.json, the fourwasm*-evidence.json(allstatus: passed).restore(input, expected_source)signatures and size constants read fromsrc/streaming/checkpoint.rsandr2007_checkpoint.rs: ten kinds (DWGH,DWGW,DWG4,DWGZ,DWGM,DWGS,DWGF,DWGO,DWGR,DWGQ); everyrestorechecks marker, version, length and the CRC-32, and every one butDWGZtakes the expected source identity.tools/streaming/*-corpus.json(54 entries) name all 52 pinned drawings intools/corpus/pins.tsv..github/rulesets/main-protection.json) has no bypass actor and the same twelve required checks as the live one;git worktree list --porcelainshows no prunable worktree.python3 tools/ci/spec_coverage.py --checkandpython3 tools/maintainer/train/arch.py check: clean.What changed:
task-ledger.md: a "last reviewed" line (2026-10-27,e7c53e1, the run); the rows for Streaming 1: replace whole-file parsing with explicit byte-demand states #205, Streaming 2: transport-independent pausable byte sources #206, Streaming 3: portable bounded checkpoints with fresh-process resume #207, Streaming 5: bounded browser input and checkpoint API in native/WASM CI #209, Streaming 6: proptest chunk schedules and automatic streaming invariant gate #210, Streaming 7: real golden corpus through multi-gigabyte DWG and honest reader benchmarks #211 and Watch ODA specification revisions and open deduplicated grammar-review issues #212 rewritten from "header only" to what landed (feat(streaming): walk the R13-R2000 container with byte-demand states #240, feat(streaming): bounded, resumable LZ77 decoder for R2004-family pages #246, feat(streaming): asynchronous byte source driven by its host #255, feat(streaming): verify the R2004-family file header with byte-demand states #260, feat(streaming): walk the R2004 page map and section map with byte-demand states #276, feat(streaming): assemble R2004 sections from their data pages with byte-demand states #299, test(streaming): restore a native checkpoint in a wasm32 kernel and the reverse #301, feat(streaming): walk a whole R2004-family file behind one checkpoint #312, feat(streaming): stream the object records of an R2004-family file (#205) #333, feat(streaming): walk a whole R2007 file with every checksum checked before its bytes are handed out (#205) #340, feat(streaming): stream the object records of an R2007 file (§20.1, #205) #363, the perf series docs(streaming): measure the streaming walks against the legacy reader per release band (#205) #343, perf(streaming): fold eight bytes per step in the DWG CRC-8 (#205) #360, perf(streaming): add the §4.2 page checksum in eight lanes (#205) #362, perf(streaming): copy a record's first bytes as a slice (#205) #364 to docs(streaming): record where the streaming walks' time went (#205) #367) with the run's counts and a per-row list of what is open or NOT MEASURED; the stale "Discovered" notes resolved (ruleset drift: resolved by docs(maintaining): main is governed by the main-protection ruleset alone, with its ten required checks and no bypass (#117) #220 and docs(ci): Corpus and Spec coverage are required on main #292; the two real inputs are pinned and walked in CI; the foundation PR is merged); the "prunable worktree" row closed.README.md, Checkpoint contract: was theDWGHheader checkpoint alone, ending "Full object/decompression checkpoints remain open in Streaming 3: portable bounded checkpoints with fresh-process resume #207". It now states the contract for all ten kinds in one table (marker, what each holds, size bound as the section of each walker states it), the common restore rules, the caller's custody of emitted output, and what no checkpoint holds (a partly decoded object, because none is decoded). TheDWGHbyte table stays.README.md, other edits in the same vein: a paragraph about the R2007 object stream's wasm32 run, which sat at the end of the contract section, moves to the end of "Cross-runtime restore" where it belongs; "Current evidence" (which cited a 2026-10-26 header-only run) now cites run 40532346151 with a per-step table; the closing sentences of "Repeatable acceptance" no longer say the manifests record no object counts or that no independent oracle has passed (they say what each manifest records), and keep the R13, multi-gigabyte and browser gaps as NOT MEASURED.Left as it was, because it is still true: the per-walker sections of the README, the throughput table and the
audit.mdfile.Source-provenance declaration
By submitting this PR I certify that my contribution does not incorporate executable code from any source whose license is incompatible with Apache-2.0. Specifically:
CLEANROOM.md).dwg-rsis Apache-2.0 and must remain compatible.My contribution is based on:
CLEANROOM.md.See
CLEANROOM.mdfor the full allowed/forbidden source list and the honest scope of what "clean-room" means for this project.Developer Certificate of Origin
By submitting this PR, I certify that the contribution complies with the
Developer Certificate of Origin 1.1.
Specifically, I affirm that the contribution is my own work (or rightfully
licensed to me) and that it can be distributed under the project's Apache-2.0
license.
I will sign off commits with
git commit -swhen asked, which adds aSigned-off-bytrailer to the commit message asserting the DCO.