Repository navigation
Research: five post-certificate structural improvements and executed benchmarks #33
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| name: Post-certificate structural improvements | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| replay-and-validate: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 30 | ||
| env: | ||
| PYTHONDONTWRITEBYTECODE: '1' | ||
| steps: | ||
| - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 | ||
| with: | ||
| fetch-depth: 0 | ||
| persist-credentials: false | ||
| - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 | ||
| with: | ||
| python-version: '3.12' | ||
| - name: Verify frozen protocol ancestry and bytes | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| protocol=8187cc4e4b98e40f3146ce787b71f262b62b3abd | ||
| git merge-base --is-ancestor "$protocol" HEAD | ||
| git show "$protocol:graph_synthesis/post_certificate/PROTOCOL.md" | cmp - graph_synthesis/post_certificate/PROTOCOL.md | ||
| python -B -m graph_synthesis.verify | ||
| - name: Focused regressions and frozen replay | ||
| run: | | ||
| python -B -m unittest discover -s graph_synthesis/post_certificate/tests -v | ||
| python -B -m graph_synthesis.post_certificate.run --check | ||
| python -B -m graph_synthesis.post_certificate.report --check --update-paper | ||
| - name: Install rendering dependencies | ||
| run: | | ||
| python -m pip install -r graph_synthesis/requirements-figures.txt | ||
| sudo apt-get update | ||
| sudo apt-get install -y poppler-utils | ||
| - name: Verify committed build hashes and deterministic manuscript | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| python - <<'PY' | ||
| import hashlib,json | ||
| from pathlib import Path | ||
| sha=lambda p:hashlib.sha256(Path(p).read_bytes()).hexdigest() | ||
| b=json.loads(Path('manuscript/paper-current.build.json').read_text(encoding='utf-8')) | ||
| assert sha('manuscript/paper-current.md')==b['source_sha256'] | ||
| assert sha('graph_synthesis/render_current_paper.py')==b['renderer_sha256'] | ||
| assert sha('manuscript/paper-current.pdf')==b['pdf_sha256'] | ||
| PY | ||
| python -B -m graph_synthesis.render_current_paper --output "$RUNNER_TEMP/post-certificate-paper.pdf" | ||
|
There was a problem hiding this comment.
In the Useful? React with 👍 / 👎. |
||
| git diff --exit-code -- manuscript/paper-current.md manuscript/paper-current.html | ||
| git diff --check | ||
| - name: PDF structural and raster smoke check | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| pdf="$RUNNER_TEMP/post-certificate-paper.pdf" | ||
| pdfinfo "$pdf" | tee "$RUNNER_TEMP/pdfinfo.txt" | ||
| pages=$(pdfinfo "$pdf" | awk '/^Pages:/ {print $2}') | ||
| test "${pages:-0}" -gt 0 | ||
| pdftoppm -f 1 -singlefile -png -r 120 "$pdf" "$RUNNER_TEMP/paper-first" | ||
| pdftoppm -f "$pages" -singlefile -png -r 120 "$pdf" "$RUNNER_TEMP/paper-last" | ||
| test -s "$RUNNER_TEMP/paper-first.png" | ||
| test -s "$RUNNER_TEMP/paper-last.png" | ||
| - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 | ||
| if: always() | ||
| with: | ||
| name: post-certificate-pdf-review-${{ github.run_id }} | ||
| path: | | ||
| ${{ runner.temp }}/post-certificate-paper.pdf | ||
| ${{ runner.temp }}/post-certificate-paper.build.json | ||
| ${{ runner.temp }}/pdfinfo.txt | ||
| ${{ runner.temp }}/paper-first.png | ||
| ${{ runner.temp }}/paper-last.png | ||
| if-no-files-found: warn | ||
| retention-days: 14 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| # Correction record: structural extensions in PR #33 | ||
|
|
||
| Reviewed September 28, 2026. This record supplements the unchanged | ||
| [frozen protocol](PROTOCOL.md) at commit | ||
| `8187cc4e4b98e40f3146ce787b71f262b62b3abd`. Original implementation and results | ||
| remain inspectable at PR head `26131e887164bc59f0cb29f3512f36f98f967356`. | ||
| No frozen fixture population, seed, threshold, comparator or cap was retuned. | ||
| No fresh model calls or new scientific documents were used. | ||
|
|
||
| ## H4: endpoint scenarios and the rejected broader interpretation | ||
|
|
||
| The frozen H4 Method explicitly evaluates the worst case across all interval | ||
| endpoint vectors. Its original heading and some publication prose described this | ||
| as interval minimax without making the finite scenario restriction clear. | ||
| Expected residual Bernoulli variance need not be largest at an endpoint, so that | ||
| broader continuous-interval claim is withdrawn. | ||
|
|
||
| The corrected implementation names the function `select_endpoint_minimax`. | ||
| `H4.primary_target_met` continues to mean the original endpoint-scenario | ||
| conjunction, with the original 128 fixtures and 16 point controls. Its measured | ||
| status is generated from the recorded booleans, separately from | ||
| `continuous_interval_claim_supported: false`. | ||
|
|
||
| A supplemental negative control has three singleton queries, review budget two, | ||
| and intervals `0=[0.1,0.9]`, `1=[0.3,0.4]`, `2=[0.5,0.5]`. Endpoint optimization | ||
| selects reviews `(1,2)` and reports worst residual variance `0.09`. The permitted | ||
| interior value `p0=0.5` yields `0.25`; reviewing `(0,2)` instead has continuous | ||
| worst variance `0.24`. For these singleton queries the continuous maxima follow | ||
| directly from `p*(1-p)`, maximized at the allowed point nearest `0.5`. | ||
| This falsifies continuous-interval minimax, even when the frozen endpoint target | ||
| passes. The control is labeled post-review evidence and does not enter the | ||
| original fixture count or threshold. | ||
|
|
||
| ## Implementation and validation repairs | ||
|
|
||
| - H1/H5 reject over-width scopes before allocating exponential tables and check | ||
| cumulative table allocation before construction. The original counted | ||
| elimination-evaluation metric is retained separately from that allocation | ||
| guard; it is not a memory or runtime measurement. | ||
| - All graph sizes use actual deterministic min-fill ordering. Numeric vertex | ||
| distance is no longer substituted for induced width. H1 large chains now use | ||
| the promised independent prefix dynamic-programming oracle, replacing the | ||
| earlier closed-form-only check. | ||
| - H2 validates graph dependencies, computes topological order, and rejects cycles. | ||
| Accepted transactions still compare their returned state with independent full | ||
| recomputation. Each rejection control additionally requires the expected | ||
| exception; unchanged input alone is insufficient. A mutation regression proves | ||
| the benchmark fails when these exceptions are suppressed. | ||
| - Report outcomes derive from each `primary_target_met` field. Failed targets | ||
| remain failed in CSV, abstract, outcome table and current-results snippet. | ||
| Standard CSV quoting preserves the five-column schema. | ||
| - Manual validation preserves protocol ancestry/bytes and rejects any changed | ||
| deterministic manuscript Markdown or HTML after rendering. It does not publish | ||
| commits or perform model calls. | ||
|
|
||
| The corrected [results](results.json), [report](RESULTS.md), figures and cumulative | ||
| manuscript are regenerated from the unchanged frozen seed. Inspect individual | ||
| outcomes and limitations; separate target passes are not a semantic-success rate. | ||
|
|
||
| ## Corrected measured outcome | ||
|
|
||
| The frozen H2 conjunction is **not met**: only 63 of 64 selected evaluation-failure | ||
| controls actually raise. One selected primitive has no affected derived node, so | ||
| the old control passed without injecting a failure. It remains in the original | ||
| denominator; no replacement fixture was selected. All 640 accepted transactions | ||
| match recomputation, stale/malformed controls each pass 64/64, and the local | ||
| evaluation reduction remains 90.1092529296875%. These favorable endpoints do not | ||
| rescue the failed conjunction. The exact failed control is retained in results. | ||
|
|
||
| H1, H3 and H5 retain their controlled target passes. H4 retains its finite endpoint | ||
| conjunction (45/128 strict endpoint gains); continuous interval minimax remains | ||
| unsupported and explicitly falsified by the supplemental control. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| <!-- POST_CERTIFICATE_RESEARCH_START --> | ||
|
|
||
| ## Five post-certificate structural extensions | ||
|
|
||
| [Executed report](graph_synthesis/post_certificate/RESULTS.md), [frozen protocol](graph_synthesis/post_certificate/PROTOCOL.md), [corrections](graph_synthesis/post_certificate/CORRECTIONS.md), and [five figures](graph_synthesis/post_certificate/figures/). Recorded target outcomes: **H1: met; H2: not met; H3: met; H4: met; H5: met.** H4 covers only endpoint scenarios; its broader continuous-interval minimax claim is unsupported, with an interior counterexample retained. Counted H1 reduction: 99.26%; local H2 reduction: 90.11%, connected-control saving: 0.00%. No fresh Jev calls, independent semantic-accuracy claim, external-system superiority or production-policy change. | ||
|
|
||
| <!-- POST_CERTIFICATE_RESEARCH_END --> |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This verification deterministically fails for the reviewed commit:
git merge-base --is-ancestor 8187cc4e4b98e40f3146ce787b71f262b62b3abd 7e79b3a54a68b822df0a27c1c7b6320d00d4680breturns 1 because the commit's sole parent is baseline9d2e4a6, not the frozen protocol commit. Although the copied protocol bytes match, the required pre-execution ancestry is absent, so every PR run exits at this step before reaching the tests; include the protocol commit in the history rather than only copying its file.Useful? React with 👍 / 👎.