Skip to content

Balance DOOP corpus across 21 measured applications - #94

Open
StarGazerM wants to merge 1 commit into
mainfrom
feat/doop-balanced-corpus
Open

StarGazerM wants to merge 1 commit into
mainfrom
feat/doop-balanced-corpus

Conversation

@StarGazerM

Copy link
Copy Markdown
Collaborator

Change

Expand the catalog from 12 to 21 distinct applications, with 6 small / 5 medium / 6 large / 4 xlarge instead of 5/2/4/1. The size thresholds are unchanged. Sixteen genuinely new application families were extracted and measured; nine were selected rather than padding the suite with more small cases.

Tier New applications and canonical VarPointsTo rows
small Bloat 11,227,250
medium Clojure 16,591,860; Chart 17,221,612; javac 24,163,325
large JRuby 60,512,570; PDFBox 69,315,812
xlarge Soot 412,802,921; Scala 612,741,889; Kotlin 1,076,040,872
  • Keep all original twelve archive URLs, byte sizes, checksums and tiers unchanged.
  • Use canonical CPU VPT cardinality uniformly across all 21 entries; retain old upstream counts separately for traceability.
  • Move source provenance to individual entries instead of assigning every application the Chopin/HuggingFace identity.
  • Preserve all raw extracted facts and genuine application roots. No duplicated application versions, synthetic roots, multiplied facts, sampling or threshold changes.
  • Leave the canonical query, normalizer and CPU/GPU implementations unchanged.

Published data and evidence

Corpus prerelease: nine pinned raw archives, per-application source/dependency/image/platform hashes and extraction commands, extraction logs, full CPU count reports/input fingerprints, candidate-selection audit, preparation-equivalence evidence and SHA256SUMS. Data and binaries stay outside Git. Total catalog downloads are approximately 2.96 GB; complete raw inputs approximately 52.3 GB.

Verification

  • Every selected application completed the unchanged canonical CPU fixedpoint: 74 relation counts and all 37 IDB exports. Original twelve results are reused only with the identical canonical query hash and their immutable validated inputs; all nine additions were newly executed.
  • Re-prepared all nine new ZIP archives through the real CLI into a fresh root: every one of the 39 normalized relation records, metadata hash, entrypoint set and query hash matches the CPU-validated snapshot. Original CPU report/manifest hashes were not rewritten.
  • After publishing, downloaded all nine new archives through the normal public fetch command into a fresh root, without the archive-cache option. All size/SHA256 checks passed.
  • Exercised list and list --tier xlarge; the latter selects Soot, Jython, Scala and Kotlin.
  • PYTHONPATH=src python -m pytest -q: 1111 passed, 5 skipped, one existing Jupyter deprecation warning.
  • ruff check examples/doop_benchmark.py: passed.

Limits

Zero warmups/one repeat establish cardinality and completion, not comparative performance. GPU equivalence/performance is not claimed here. Static extraction retains its reflection/phantom limitations: Kotlin has 23 phantom methods and 3 phantom-based methods after completing declared dependencies plus JNA and Java 8 tools; Scala also retains phantom-based methods. Exact diagnostics are preserved, not suppressed. Failed extractor runs, including runs that exited zero with Soot errors, were not promoted. Upstream software retains its own licenses.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant