Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/workflows/quality.yml
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,17 @@ jobs:
done
done

# The front-door documents fail on drift rather than warn: the counts the
# README quotes are measured from the tree, and every local link in the
# README, EXPLAINME and the docs index must resolve.
- name: README counts and front-door links match the tree
run: |
# just counts the recipes the README quotes; the contracts compare it
# with the grep fallback the check uses where just is absent.
sudo apt-get update -q && sudo apt-get install -y -q just
bash tests/readme-claims-test.sh
bash scripts/check-readme-claims.sh

# ============================================================================
# Deploy to GitHub Pages
# ============================================================================
Expand Down
9 changes: 6 additions & 3 deletions AFFIRMATION.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,12 @@ state and commitments. Companion detail:

== We affirm (current normative state — MUST)

* *Guix-only builds.* Nix is retired estate-wide (owner ruling 2026-06-05);
`guix.scm` / guix channels are the sole reproducible-build manifests. No
`flake.nix` remains; all references reconciled.
* *Guix-only builds.* Nix is retired estate-wide (owner ruling 2026-06-05).
No `flake.nix` remains; all references reconciled. Guix is the only
reproducible-build manifest the estate accepts, but this repository does
not track a `guix.scm` yet; `deploy/Containerfile` builds the image and
`mise.toml` pins Erlang/OTP, Elixir and Zig (its other tools float), until
it does.
* *The formal-proof corpus checks in CI* across Idris 2 (v0.7.0),
Agda (2.8.0.2, one artefact) and TLA+ (bounded TLC
model checking, which is `tested-only`, not a proof). What each artefact
Expand Down
26 changes: 26 additions & 0 deletions CHANGELOG.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,32 @@ https://semver.org/[Semantic Versioning].

=== Changed

==== README claims match the tree, and a gate keeps them there (2026-11-09)

* Corrected counts in the README:
** rule modules 35 → 42;
** `lib/` files 133 → 154;
** Just recipes 39 → 45;
** CLI commands five → six (`secret-alerts`);
** Mix tasks "eleven more" → twelve.
* `just build-all` is described as what it builds, the Rust workspace and
Elixir, with `build-ffi` and `compile-abi` listed beside it.
* The `guix shell -m guix.scm` step is gone: no `guix.scm` is tracked. The
README and AFFIRMATION say so, and name `mise.toml` and
`deploy/Containerfile` as the stand-ins.
* The rule-distillation loop is presented as the design, not the running code.
* The `/api/*` access policy is stated as `api_router.ex` applies it.
* Dead links fixed:
** `topology.md` and `DEBT-REGISTER.md` (both `.adoc`), and
`CONTRIBUTING.md` (in `.github/`);
** 17 `.md` → `.adoc` links in `docs/README.adoc`.
* `scripts/check-readme-claims.sh` measures the counts the README quotes,
and resolves every local link in the README, EXPLAINME and the docs index.
Fragments are checked too: a link to `#id` needs that anchor in its target.
It runs in the Validate Documentation job and fails on drift, where the
older link step only warned. `tests/readme-claims-test.sh` plants each kind
of drift.

==== Proof-ledger statuses are derived from CI receipts (2026-11-09)

* Every ledger row names its `lane`: the CI job whose receipt decides it
Expand Down
78 changes: 42 additions & 36 deletions EXPLAINME.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -17,28 +17,30 @@ or `mix hypatia.watch`.

[quote, README.adoc]
____
*Symbolic reasoning* — 35 Elixir rule modules producing fast, deterministic findings
*Symbolic reasoning* — Elixir rules producing fast, deterministic findings, in
42 source files under `lib/rules/`
____

[quote, README.adoc]
____
The system learns complex patterns through neural analysis, then *distils* them into simple,
fast declarative rules that can be pre-emptively injected into new repositories, applied as
cures to existing ones, and executed without ML inference overhead.
That loop is the design, not yet the running code. Today the rules are hand-written, and the
neural layer can only lower a recipe's confidence or demote a finding.
____

How this is implemented::
Pure-Elixir modules in link:lib/rules/[] behind `Hypatia.Rules`
(link:lib/rules/rules.ex[]). The escript runs them with no OTP tree and no
neural process (`app: nil` in link:mix.exs[]).
neural process (`app: nil` in link:mix.exs[]). The count is
`find lib/rules -maxdepth 1 -name '*.ex' | wc -l`. link:scripts/check-readme-claims.sh[]
measures it, with the README's other structural counts, in the Validate Documentation
job, and fails on drift.

Caveat::
`find lib/rules -maxdepth 1 -name '*.ex' | wc -l` gives 42, not 35; the
generated link:docs/rules/catalogue.adoc[] says 41 and lacks
The generated link:docs/rules/catalogue.adoc[] says 41 and lacks
`structured_format`. No code path distils learned patterns into rules:
`learn_from_fix/3` and `promote_patterns/0` in link:lib/rules/learning.ex[]
have no outside callers, and `git grep RecipeGenerator` finds no caller.
The rules are hand-written.
The README states this beside the design sentence.

=== Neural learning

Expand Down Expand Up @@ -140,7 +142,7 @@ an action is capped at review, not report-only (line 67).

[quote, README.adoc]
____
133 Elixir modules — pipeline, rules, VCL, neural, safety, web, merge orchestration
154 Elixir source files — pipeline, rules, VCL, neural, safety, web, merge orchestration
____

How this is implemented::
Expand All @@ -149,53 +151,60 @@ Each README row, measured:
[cols="1,2", options="header"]
|===
| README row | Tree
| `lib/`: 133 | 154 `.ex` files (`+ls lib/**/*.ex \| wc -l+`), 180 `defmodule`
| `lib/rules/`: 35 | 42, as above
| `lib/`: 154 | 154 `.ex` files (`+find lib -name '*.ex' \| wc -l+`), 180 `defmodule`; gated
| `lib/rules/`: 42 | 42, as above; gated
| `lib/neural/`, `lib/vcl/` | Correct
| `lib/merge_orchestration/`: 12 | Correct (one file in `kin_gate/`)
| `lib/hypatia/web/` | SARIF is link:lib/hypatia/sarif.ex[], outside `web/`
| `src/Hypatia/ABI/` | Also `Verdict`, `Retraction`; `Gen.idr` is a generator
| `ffi/zig/`: 18, 16 | Correct; pinned by link:.github/workflows/zig.yml[]
| Rust workspace | 7 members in link:Cargo.toml[]
|===
+
"Gated" means link:scripts/check-readme-claims.sh[] measures the row in CI and
fails when the README disagrees. The same check fails on a dead local link in
the README, this file or link:docs/README.adoc[].

Caveat::
The README links `docs/architecture/topology.md`; the file is
link:docs/architecture/topology.adoc[].
The other rows are measured by hand, as of the date above.

=== Installation, usage and the operational surface

[quote, README.adoc]
____
`just --list` shows all 39 recipes; `CONTRIBUTING.md` treats `just` as the canonical entry point.
`just --list` shows all 45 recipes; link:.github/CONTRIBUTING.md[`CONTRIBUTING.md`] treats `just` as the canonical entry point.
____

[quote, README.adoc]
____
Running the supervision tree also serves an HTML dashboard, an SSE event stream, a Prometheus
`/metrics` endpoint and a bearer-authenticated `/api/*` on port 9090.
`/metrics` endpoint and an `/api/*` on port 9090. With
`HYPATIA_API_BEARER_TOKEN` set and not empty, every `/api/*` request needs that bearer
token. Without one, only loopback clients are answered unless
`HYPATIA_API_ALLOW_NONLOCAL=true`.
____

How this is implemented::
link:Justfile[] (with `contractile.just`), link:lib/hypatia/cli.ex[],
link:lib/mix/tasks/[]; link:mise.toml[] pins Erlang 27.3.4.4 and Elixir
1.17.3. link:lib/application.ex[] supervises 24 children (two more when
configured); link:lib/hypatia/web/router.ex[] serves `/`, `/metrics`, `/api`.
configured); link:lib/hypatia/web/router.ex[] serves `/`, `/metrics`, `/api`,
and `protect/2` in link:lib/hypatia/web/api_router.ex[] applies the bearer
and loopback policy. The recipe count is `just --summary | wc -w`; the six
CLI commands and the sixteen Mix tasks the README describes are gated as well.

Caveat::
The Justfiles hold 43 recipes. `just build-all` builds Rust and Elixir only.
The CLI has six commands, not five (`secret-alerts`); there are 16 Mix
tasks, not 15. No `guix.scm` is tracked. `CONTRIBUTING.md` is in `.github/`.
`/api/*` needs a bearer token when `HYPATIA_API_BEARER_TOKEN` is set and not
empty. Otherwise only loopback clients are answered, unless
`HYPATIA_API_ALLOW_NONLOCAL=true` lets non-local clients in without a token.
`just build-all` builds the Rust workspace and Elixir; the README lists
`just build-ffi` and `just compile-abi` beside it for the Zig FFI and the
Idris2 ABI. No `guix.scm` is tracked: the README and AFFIRMATION say so, and
name `mise.toml` and `deploy/Containerfile` as the stand-ins.

=== Project status and licence

[quote, README.adoc]
____
242 tests that are excluded from every run, the safety systems that are built but not yet invoked
by any dispatch path, and the CI blockage currently stopping the Rust lane.
That includes the `:verisim_data` tests excluded from every run and the safety systems that are
built but not yet invoked by any dispatch path.
____

[quote, README.adoc]
Expand All @@ -210,12 +219,11 @@ that tag. The safety modules are in link:lib/safety/[]. Every tracked
`+lib/**/*.ex+` file carries `MPL-2.0`; link:LICENSES/[] holds both texts.

Caveat::
The nine files hold 224 literal `test` blocks; 242 needs a `mix test` run
this audit did not make. The Rust blockage is stale: entry CI-1 of
link:docs/DEBT-REGISTER.adoc[] records the fix, visible in
link:integration/Cargo.toml[] lines 48–55. `docs/DEBT-REGISTER.md` is dead.
45 of 152 tracked `.adoc` files have no SPDX header (recorded for the owner,
not a licence change).
The nine files hold 224 literal `test` blocks; the README no longer quotes
a count. Entry CI-1 of link:docs/DEBT-REGISTER.adoc[] records the fix for
the former Rust-lane blockage, visible in link:integration/Cargo.toml[]
lines 48–55. 45 of 152 tracked `.adoc` files have no SPDX header (recorded
for the owner, not a licence change).

== Dogfooded across the account

Expand Down Expand Up @@ -251,12 +259,10 @@ have no callers, so its circuit breaker never opens. The moduledoc of

[CAUTION]
====
README overclaims, each detailed above: learned patterns are not distilled
into rules (the promotion and recipe-generation code is never invoked); the
counts of rule modules, `lib/` modules, Just recipes, CLI commands and Mix
tasks are wrong; `just build-all` is narrower than stated; `guix.scm` is
absent; two links are dead. Excluded `:verisim_data` tests run in no
workflow (debt register T-1).
Learned patterns are not distilled into rules: the promotion and
recipe-generation code is never invoked. The README now presents the loop as
the design and says the rules are hand-written. No `guix.scm` is tracked.
Excluded `:verisim_data` tests run in no workflow (debt register T-1).
====

== Evidence index
Expand Down
45 changes: 29 additions & 16 deletions README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ toc::[]
*Hypatia* is the CI/CD governance scanner for the hyperpolymath estate (~400 repositories).
It combines:

* *Symbolic reasoning* — 35 Elixir rule modules producing fast, deterministic findings
* *Symbolic reasoning* — Elixir rules producing fast, deterministic findings, in
42 source files under `lib/rules/`
(link:docs/rules/catalogue.adoc[full rule catalogue])
* *Neural learning* — 8 networks on a shared blackboard, learning which fixes actually stick
* *Bayesian confidence* — every recipe carries a Beta-distributed confidence updated from real
Expand All @@ -40,6 +41,10 @@ The system learns complex patterns through neural analysis, then *distils* them
fast declarative rules that can be pre-emptively injected into new repositories, applied as
cures to existing ones, and executed without ML inference overhead.

That loop is the design, not yet the running code. Today the rules are hand-written, and the
neural layer can only lower a recipe's confidence or demote a finding. The promotion code in
`lib/rules/learning.ex` has no caller (link:EXPLAINME.adoc[EXPLAINME]).

== The safety triangle

Every finding is routed *Eliminate > Substitute > Control*, in that order — remove the hazard
Expand Down Expand Up @@ -71,8 +76,8 @@ PatternRegistry ──> TriangleRouter ──> FleetDispatcher ──> DispatchM

[cols="1,3"]
|===
| `lib/` | 133 Elixir modules — pipeline, rules, VCL, neural, safety, web, merge orchestration
| `lib/rules/` | 35 rule modules; see link:docs/rules/catalogue.adoc[the catalogue]
| `lib/` | 154 Elixir source files — pipeline, rules, VCL, neural, safety, web, merge orchestration
| `lib/rules/` | 42 source files: rule modules and their helpers; see link:docs/rules/catalogue.adoc[the catalogue]
| `lib/neural/` | 8 networks + blackboard, coordinator, persistence, rebalancer
| `lib/vcl/` | VCL query layer over verisim-data: parser, file executor, cache, federation
| `lib/merge_orchestration/` | 12 modules — leases, ledger, ticker
Expand All @@ -83,24 +88,28 @@ PatternRegistry ──> TriangleRouter ──> FleetDispatcher ──> DispatchM
| `verification/` | Proof ledger and proofs: Idris2, Agda, TLA+
|===

See link:docs/architecture/topology.md[docs/architecture/topology.md] for the full map.
See link:docs/architecture/topology.adoc[docs/architecture/topology.adoc] for the full map.

== Installation

Guix is the reproducible-build path; Nix was retired estate-wide on 2026-06-05. Podman, not
Docker, is the container tool.
Guix is the estate's reproducible-build path; Nix was retired estate-wide on 2026-06-05.
This repository does not track a `guix.scm` yet. Until it does, `mise.toml` pins
Erlang/OTP, Elixir and Zig (its other tools float at `latest`), and
`deploy/Containerfile` builds the image. Podman, not Docker, is the
container tool.

[source,bash]
----
git clone https://github.com/hyperpolymath/hypatia.git
cd hypatia

guix shell -m guix.scm # reproducible environment (optional)
just doctor # toolchain preflight
just build-all # Elixir + Rust + Zig FFI + Idris2 ABI
just build-all # Rust workspace + Elixir
just build-ffi # Zig FFI
just compile-abi # Idris2 ABI
----

`just --list` shows all 39 recipes; `CONTRIBUTING.md` treats `just` as the canonical entry point.
`just --list` shows all 45 recipes; link:.github/CONTRIBUTING.md[`CONTRIBUTING.md`] treats `just` as the canonical entry point.

=== Elixir development

Expand All @@ -125,23 +134,27 @@ or clean test run.

== Usage

The escript exposes five commands:
The escript exposes six commands:

[source,bash]
----
hypatia scan <path> [--format json|text|github|sarif] [--exit-zero] [--advisory]
hypatia report <path>
hypatia pr-eligibility --owner <owner> --repo <repo> --pr <n>
hypatia secret-alerts --owner <owner> --repo <repo>
hypatia version
hypatia help
----

Mix tasks cover the operational surface — `mix hypatia.rsr_score`, `mix hypatia.recipe_health`,
`mix hypatia.watch`, `mix hypatia.triage_issues` and eleven more. See the
`mix hypatia.watch`, `mix hypatia.triage_issues` and twelve more. See the
https://github.com/hyperpolymath/hypatia/wiki/Operations[Operations wiki page].

Running the supervision tree also serves an HTML dashboard, an SSE event stream, a Prometheus
`/metrics` endpoint and a bearer-authenticated `/api/*` on port 9090.
`/metrics` endpoint and an `/api/*` on port 9090. With
`HYPATIA_API_BEARER_TOKEN` set and not empty, every `/api/*` request needs that bearer
token. Without one, only loopback clients are answered unless
`HYPATIA_API_ALLOW_NONLOCAL=true`.

=== Consuming the RSR oracle from another repository

Expand Down Expand Up @@ -177,10 +190,10 @@ Development state, milestones and known gaps live in machine-readable form:
* `.machine_readable/descriptiles/PLAYBOOK.a2ml` — operational runbook
* link:ROADMAP.adoc[ROADMAP.adoc] — milestone history

**Before trusting any metric or green board, read link:docs/DEBT-REGISTER.md[docs/DEBT-REGISTER.md].**
It is the authoritative, evidence-backed account of what is broken, ungated or stale — including
242 tests that are excluded from every run, the safety systems that are built but not yet invoked
by any dispatch path, and the CI blockage currently stopping the Rust lane.
**Before trusting any metric or green board, read link:docs/DEBT-REGISTER.adoc[docs/DEBT-REGISTER.adoc].**
It is the authoritative, evidence-backed account of what is broken, ungated or stale. That
includes the `:verisim_data` tests excluded from every run and the safety systems that are
built but not yet invoked by any dispatch path.

== Documentation

Expand Down
Loading
Loading