[fraware]

Purpose

Second-stage cumulative review draft. Integrate the already-qualified, still-unmerged security foundation from #224 with the non-circular external-pilot metrics corrections of #217 and remaining release-truth corrections from #219.

Source heads pinned for this review:

• integration(security): qualify release installer and PR source binding together #224 e690b02e68b9866eed55b36bcf01c8ef59de958f, full CI + six-platform repro successful.
• fix(pilot): prevent circular false-positive metrics and unsafe strict promotion #217 652e158146a4711bf4f94422efe62c8e8df5772d (independent adjudication and pilots).
• docs: correct unsigned release-tag claims and unsafe pilot promotion guidance #219 795811a624e0db9889e975a1be51d5a6409f6fec (historical release integrity truth).

This branch derives from #224 and targets main only to exercise the actual existing full CI and six-platform reproducibility workflow. Neither source PR nor main is modified.

Verified source composition and deliberate deviations

Transferred #217 ingester, its original tests, and its report template as exact blob contents. Imported the #217 advisory playbook with its explicit reviewer/consent/coverage/no-strict framework, adding:

• an explicit warning that historical caller-supplied PR diffs are unbound input and older Action installations may fall back to checkout;
• disabled GitHub Check publishing and comments on both fork-adopter Action invocations;
• removal of unreviewed external_pilot_report.json from automated artifact upload.

Corrected an additional metric cohort contamination case: previously adjudicated error counts/rates are now invalidated for any explicitly newly supplied artifact directory, including a single evidence bundle or missing aggregate denominator. Prior #217 logic cleared them only when an aggregate contained a non-null prs_evaluated. An explicitly selected report may provide adjudications again, but the ingester does not authenticate reviewer authority or cohort correspondence. New adversarial regression cases test the unknown-denominator scenario and distinguish the no-new-artifacts path.

Transferred #219's docs/PILOT_CASE_STUDIES.md, docs/RELEASE.md, and docs/RELEASE_NOTES_v1.2.1.md verbatim; the overlapping README/release-status/integration docs had already received substantial fact-checking in #220/#224. Instead of replacing those newer documents with stale competing copies, this branch preserves the correct release-status API facts, makes the README's copyable Action example read-only, and adds #219's HMAC trust clarification, historical-example restrictions, and explicit future-only strict rollout scope to the integration guide.

The extra cross-PR contract test enforces copyable least-privilege examples, no automatic publication, null unmeasured metrics, and live-audited release-integrity language.

Scientific, product, and release nonclaims

A checker verdict is not independent ground truth. An explicit self-reported reviewer file is not independently authenticated adjudication. No natural-PR pilot, paid adopter, externally calibrated false-positive/false-negative performance, source-to-model correspondence, or production authorization has been established by these files.

Live GitHub release audit (2026-10-15): main protected=false, repository rulesets [], v1.2.1 annotated Git tag verification.verified=false (unsigned), GitHub Release immutable=false; v1.3.0-rc.1 tag returns 404. GitHub installation lacks the administration permission needed for branch/ruleset changes. Issue #25 remains an explicit public-release NO-GO.

Required before review promotion

1. Inspect final exact-head CI (all jobs) and six OS × Python reproducibility cells; no extrapolation from source PR green checks.
2. Independently review source-data provenance, duplicate/omitted natural PRs, reviewer authority, source-profile coverage, cohort-scoped error rates, and any missing ground truth. Treat artifact-only statistics as non-adjudicating.
3. Inspect entire cumulative diff with special attention to the inherited historical Action fallback and external-fork permissions.
4. Verify document truth against live GitHub API, preserve exact source/target/evidence identity, and prohibit promoting strict mode from pilot sample size alone.
5. Retain draft/unmerged status. No tag, release, branch-protection change, production VALID, or automatic evidence reuse is authorized.

Acceptance: bounded empirical-evidence integrity and release truth after review, not generalized verifier soundness or launch authority.

Additional independent dogfood-metrics audit

A further source-level review found that scripts/collect_pilot_metrics.py::_false_positive_rate itself classified not passed AND merge_recommendation=block as a false positive. This uses the same checker/fixture outcomes as both prediction and truth and has no independent error adjudication. Its renderer in scripts/render_pilot_metrics.py would also reproduce stale rates from historical reports. Both paths now produce false_positive_rate=null until a separately specified, reviewed adjudication mechanism exists.

The historical July 2026 docs/benchmarks/adoption-summary.json dogfood rate and the three maintained/fixture registry entries formerly showed 0.0 without independent ground truth. They are corrected to null/unknown in both the source registry and summary. Original numerical artifacts remain recoverable from Git history; provenance notes state that the 2026-10-15 modification is a data-interpretation correction, not a new experiment. PILOT_CASE_STUDIES.md removes the corresponding unqualified zero-error figures and the leftover <5% strict-promotion heuristic. Regression tests prohibit future reproduction of the misleading claims.

No independent adopter or source-to-model empirical calibration is thereby established. The legitimate remaining dogfood metrics are fixture pass/fail and timings under their stated controls. Unknown false-positive rates must not be silently treated as zero.

Updated exact integration head: 59fd9ab3d293204ab700de28dd6f8d58886b74f1. Prior branch checks and the earlier queued runs are non-controlling; full CI and six-platform reproducibility must pass on this exact new SHA. No merge or release authorized.

Historical per-pilot report consistency

A final audit identified three archival per-pilot Markdown reports that still described fixture 0/2 and 0/3 values as false-positive rates, and claimed zero false negatives without an independent evaluation population. The reports now retain their actual safe/unsafe fixture observations but state adjudicated false positives, false-positive rate and false negatives = unknown. Obsolete heuristic FP <5% promotion language and an unsupported immutable historical tag claim were corrected. Companion raw files remain in place as historical records; the normalized registry/summary are the explicitly corrected publication surface. A regression test covers all three reports.

Final proposed source head: 9a44ddd290ec5b220ead088a4a8173eaf0110005. Freeze this commit for exact-head CI and six-platform repro. No further changes are authorized by older successful jobs; source PRs remain draft/unmerged. Historical-release ref governance under #25 remains the hard blocker; there is no independent external pilot yet.

[fraware]

Fail-closed pilot source ingestion audit — 2026-10-15

Latest proposed head: 1976010a50cc3678d900acf7b77415222c5aab36 (draft/unmerged). Exact-head CI and six-platform reproducibility remain queued as of this audit; do not extrapolate success from historical/intermediate green runs.

Newly established defect in scripts/ingest_external_pilot_metrics.py:

1. --artifacts-dir supplied with a missing/not-a-directory path was silently ignored; --report supplied with a missing file was also silently ignored. Re-running an ingest with a mistyped path could produce a superficially fresh registry timestamp from old measurements.
2. Even for a real new artifact directory, previous prs_evaluated, prs_blocked, and median latency were retained if the new bundle contained only partial/nonaggregate observations. Existing false-positive labels were already cleared; the other cohort-specific metrics were not.
3. A fresh explicit report updating prs_evaluated without corresponding adjudicated errors could retain an obsolete historical zero-error label.

Corrections: both explicit source arguments now fail closed if missing; all five cohort-scoped metrics are cleared on new artifact-directory ingest and only reintroduced if supplied by the new material; new report counts invalidate old error labels. Regression cases cover missing selected paths, empty and partial new cohorts, and report-only count updates.

Qualification boundary: no independently authenticated reviewer, completeness of natural-PR enumeration, source-to-model proof, strict policy maturity, or historical-certificate applicability is claimed. The pilot source schema still lacks a cryptographically bound cohort identity and independent adjudication chain. Until both new exact-head workflows pass and a qualified reviewer audits the cumulative diff, retain draft and no release/merge.

Product evidence: independent pilot prospect screening has been recorded under #216; no maintainer was contacted and none has consented to installation. Live main branch protection remains disabled under blocker #25.

[fraware]

Final cohort-coherence hardening — 2026-10-15

Proposed frozen development head: 1ce6d5938e1bc4dc68cac4ffe3a03c041aa994cd; draft, unmerged. Previous head 1976010a has been superseded. Only CI 38827722220 and six-platform repro 38827722286 at this exact head will qualify this revision; both were queued at the time of this comment.

Review discovered one final consistency gap in explicitly selected reviewer reports: a newly supplied prs_evaluated might replace the cohort denominator while retaining old prs_blocked and median latency; a report supplying just one error statistic could also inherit an obsolete other statistic. The updated _apply_report reads the JSON file exactly once, checks repository identity before application, invalidates all prior cohort metrics on count-bearing reports, and clears stale paired error statistics for partial reports. Three regression scenarios cover stale block/latency and stale complementary adjudications.

Known residual: This v1 schema has no independently authenticated reviewer identity, unique natural-PR census, cohort evidence digest, signed report-to-artifact correspondence, or quantified false-negative rate. Reporter-supplied labels remain self-reported and non-controlling. All uses must remain advisory and no strict/source-to-model/transport/general-security guarantee follows.

No more source edits are planned in this bounded integration pass. Obtain exact-head tests and independent review; preserve #25 as a live release NO-GO and recruit a consenting design partner under #20/#216. Do not merge or publish on the basis of older CI, the 2026-07 fixture reports, or candidate repository activity.

[fraware]

Exact-head integration qualification complete — 2026-10-15

Frozen source 49a9e91026c3aee9c74154a749bff963d0165146, PR is still draft/open/unmerged.

• CI run 38829045421: completed SUCCESS on this exact commit.
• Six-platform reproducibility 38829045444: completed SUCCESS on this exact commit, after confirming that the earlier global failure had arisen from test_build_pilot_row_from_artifacts expecting a mixed-cohort latency 88.5. The corrected test now expects None because no artifact-to-explicit-report cohort link was authenticated; both original and cross-platform failure logs were inspected.

No historical failure is retroactively counted as passing. This qualifies the bounded combined development metrics/provenance contract; it does not independently attest reviewer identities, prove a source-to-IR refinement, authorize historic certificate transport, create an external adopter, or meet release/ref protection #25.

Additional independent hazard: draft #226 builds on this frozen head and guards the composite Action's unbound manifest and caller-supplied diff publication paths. #226 has its own exact-head CI/repro gate and remains unmerged. Do not merge #225 as if that independent publish-safety issue were resolved in this branch.