Skip to content

DOOP: add twelve pinned datasets and portable preparation - #91

Merged
StarGazerM merged 1 commit into
mainfrom
feat/doop-twelve-datasets
Sep 24, 2026
Merged

StarGazerM merged 1 commit into
mainfrom
feat/doop-twelve-datasets

Conversation

@StarGazerM

Copy link
Copy Markdown
Collaborator

Scope

Add a reproducible twelve-application DOOP corpus without committing datasets or research output trees.

  • Pin DaCapo 23.11-MR2-chopin fact archives to Hugging Face revision da9e91b3ff75d94604f57ba2b21ef3aa97e241ec, with exact archive bytes and SHA-256.
  • Local reference-VPT tiers: small (xalan, zxing, biojava, pmd, sunflow), medium (h2o, spring), large (batik, eclipse, fop, h2), xlarge (jython). Explicitly not input-size tiers or official DOOP editions.
  • Portable list/fetch/prepare commands; safe atomic extraction, raw input normalization, shared per-dataset interning, complete relation-set deduplication, per-dataset constants, provenance and entrypoint checks.
  • Prepare all 39 declared input files. The instantiated canonical program uses 37; documentation distinguishes prepared from consumed size.
  • Data/build/output paths remain external. No generated binaries, facts, profiles or large result dumps enter Git.

Verification

All 12 pinned archives downloaded/reused and SHA-256 verified; all 12 complete input preparations succeeded against real public facts. The archives total 1.736 GB and expand to 28.316 GB of raw facts. Added regression coverage for projections, IDs, roots, set semantics, numeric boundaries, unsupported normalization, overwrite refusal and unsafe/corrupt archives.

The complete integration worktree Python suite passed 1,110 tests (5 skipped); later stacked changes add CPU/GPU execution separately. Preparation status deliberately remains prepared_not_engine_validated.

Reproduce

python examples/doop_benchmark.py list
python examples/doop_benchmark.py prepare --all --root /path/to/doop-data

First PR in the DOOP expansion stack. Compiler optimization and native benchmark execution are intentionally separate reviews.

@StarGazerM
StarGazerM merged commit ea64f67 into main Sep 24, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant