Skip to content

perf: store census data as single JSON arrays - #2

Merged
hyperpolymath merged 1 commit into
mainfrom
claude/new-session-th4e9w
Oct 26, 2026
Merged

hyperpolymath merged 1 commit into
mainfrom
claude/new-session-th4e9w

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

Summary

#1 was merged at 55aa899, just before this follow-up commit was pushed to the same branch, so it never reached main. This PR carries that one commit.

The proven-suite bench gate failed on #1: the ijson_jcs benchmark ran at 63× its baseline. scripts/check-ijson-jcs.sh starts a separate runtime process for every JSON Lines line (about 15 ms each), and the census added about 750 lines. repos, unreachable and catalogue-rows are now each one I-JSON/JCS array in a .json file, so the checker starts three processes instead of about 750. The census scripts, Justfile and README are updated to match.

Checks (local)

  • check-ijson-jcs.sh --enforce . drops from about 11.5 s to about 0.3 s, with all 16 JSON files canonical.
  • just catalogue-check passes; the catalogue renders byte-identically from the arrays.
  • check-root-shape, check-no-md-in-docs and check-no-vlang all pass.
  • An end-to-end census run on one real repository reproduces its earlier row.

Not in this PR

  • The per-line process spawning is a defect in the template's checker, and it needs fixing upstream in rsr-template-repo.
  • openssf-compliance and Canon lockstep stay red until the repo is minted with just repo-init. That's being handled separately.

Generated by Claude Code

The proven-suite bench gate failed: ijson_jcs ran 63x its baseline. The
checker spawns one runtime process per JSON Lines line (~15 ms each),
and this PR added ~750 lines. repos, unreachable and catalogue-rows are
now one JCS array per file (.json), so the checker spawns three times:
check-ijson-jcs --enforce over the whole repo drops from ~11.5 s to
~0.3 s locally, all 15 files canonical.

The per-line spawning itself is a template checker issue and is left for
a separate fix upstream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019a95kk88TZ1iegsBK3kox3
@coderabbitai

coderabbitai Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Caution

Review failed

The pull request is closed.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sonarqubecloud

Copy link
Copy Markdown

Copy link
Copy Markdown
Owner Author

Two checks are red on be9978b: Canon lockstep and openssf-compliance. Neither is caused by this PR's change, which only switches the census data from JSONL to single JSON arrays.

  • Canon lockstep requires mathesis_chora.deed, which the repo-init mint creates.
  • openssf-compliance fails on the template placeholders that the same mint fills in.

Both are red on main for the same reason. The mint (just repo-init, clade fv) is blocked in the agent session because the nickel binary can't be downloaded there. The owner can run it locally, or allow the download. A re-run won't help until the mint lands, so I haven't re-run either check.


Generated by Claude Code

@hyperpolymath
hyperpolymath marked this pull request as ready for review October 26, 2026 19:01
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-26T19:04:03.807731Z be9978b Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@hyperpolymath
hyperpolymath merged commit 6ae67c9 into main Oct 26, 2026
41 of 44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants