diff --git a/CHANGELOG.md b/CHANGELOG.md index 947a245..16cc924 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,11 @@ commit that set the version in `app.json`. The format follows ## [Unreleased] +### Changed +- The developer guide (code layout, checks, running the page without Kiro Crew, and proving each + reader against the real tool) moved from the README to `docs/development.md`. The README keeps a + short pointer and the two check commands. + ## [0.12.2] - 2026-11-09 ### Security diff --git a/README.md b/README.md index 09eed19..951ecc7 100644 --- a/README.md +++ b/README.md @@ -78,8 +78,9 @@ home region, and any agent whose model runs elsewhere becomes a finding, with th The Agents tab: four agents with what starts each, their model and whether it asks you first, and the Restock agent's detail, from its nightly job to its hand-off to the Buyer -Screenshots show a made-up bookshop, taken from `ui/dev`, the stand-in host described under -[Development](#development). In Kiro Crew the page sits inside the dashboard. +Screenshots show a made-up bookshop, taken from `ui/dev`, the stand-in host described in +[docs/development.md](docs/development.md#run-the-page-without-kiro-crew). In Kiro Crew the page +sits inside the dashboard. ## Install @@ -238,155 +239,14 @@ choose **Sync**, then reload the page. A new tool reaches new sessions only. ## Development -| Path | Job | -|---|---| -| `core/` | The model and extractors. Imports nothing from Kiro Crew. | -| `core/extractors/` | One module per stack. Each reads a repo into Facts. | -| `mcp_server/` | Tools that agents use. Stdlib only. | -| `backend/` | HTTP API for the dashboard pages. Stdlib only. | -| `ui/` | The app page and the chat side panel tab. | -| `skills/system-map/` | How agents should read and write the model. | -| `scripts/` | Replay migrations and try proposals on a throwaway Postgres, and hold the readers to the compiler and to `drizzle-kit`. | -| `docs/` | Context rules, platform notes and the backlog. | - -Host rules learned while building it are in `docs/platform-notes.md`, and what's left is in -`docs/backlog.md`. Releases are in `CHANGELOG.md`. - -### Checks +The code layout, the checks, running the page without Kiro Crew, and how to prove each reader +against the real tool are in [docs/development.md](docs/development.md). The checks in short: ```bash -python3 -m unittest discover -s tests python3 tests/strict.py (cd ui && npm test && npm run typecheck && npm run build) ``` -`tests/strict.py` runs the same tests but fails on any skip, and prints each skipped test with its -reason. CI runs every PR against the oldest supported and the newest Kiro Crew. - -The replay test needs Postgres server binaries. Set `PGBIN` if they are not in -`/usr/lib/postgresql/16/bin`. The compiler test needs `npm install` in `ui`. The host contract tests -need an installed Kiro Crew, found through the `kirocrew` launcher or `KIROCREW_PYTHON`. Each skips -itself when its tool is missing. - -### Run the page without Kiro Crew - -`ui/dev` stands in for the host, so you can see and screenshot the page on its own: - -```bash -SYSTEM_MAP_DATA=/tmp/model python3 -m core build -SYSTEM_MAP_DATA=/tmp/model PORT=9191 SYSTEM_MAP_UNSIGNED=1 python3 backend/server.py & -(cd ui && npm run dev) -``` - -Open . Add `?theme=dark` or `?view=panel`. - -The backend refuses any call the gateway didn't sign, except `/health`. `ui/dev` can't sign, so -`SYSTEM_MAP_UNSIGNED=1` lets it in. A backend that Kiro Crew starts ignores that flag. - -### Prove a model against real Postgres - -Replay a repo's migrations on a throwaway server, then compare it with the model: - -```bash -scripts/replay_catalog.sh origin/main supabase/migrations /tmp/replay [bootstrap.sql] -python3 -m core build -python3 -m core verify /tmp/replay/catalog.json -``` - -`bootstrap.sql` is a repo path for anything the migrations expect to exist, such as a stub of -Supabase's `auth` schema. The server listens on a socket only, and stops when the script ends. - -### Prove a SQLite model - -Load the schema into a separate real SQLite, then compare every table with its pragmas: - -```bash -python3 -m core verify-sqlite -``` - -### Prove a Drizzle model - -Have the tool write its SQL in a copy of the project, then compare that with the reader. Use -`generate`, because `export` ignores `casing`: - -```bash -(cd && npx drizzle-kit generate --name proof) -python3 -m core verify-drizzle /drizzle/0000_proof.sql -``` - -To try every form the reader knows against the real tool, give the sweep any `node_modules` with -`drizzle-kit` and `drizzle-orm` installed. The same seed gives the same schemas: - -```bash -scripts/sweep_drizzle.py /tmp/sweep [rounds] [tables] [seed] -``` - -### Prove routes against Next.js - -Run `next build` in a copy of the app, then compare the routes Next wrote down: - -```bash -python3 -m core verify-routes /.next/app-path-routes-manifest.json /.next/server/pages-manifest.json -``` - -### Prove a model against the TypeScript compiler - -Ask the real compiler what a checkout imports and which tables it calls, then compare. This also -checks routes against TanStack's generated route tree: - -```bash -node scripts/ts_truth.mjs /tmp/truth.json -python3 -m core verify-code /tmp/truth.json -``` - -### Prove the SDK agent readers - -Ask the real compiler which agents a project declares with an agent SDK, then compare. Install -the SDKs the project uses, and TypeScript, in a scratch folder above or beside it, so its -`node_modules` resolve: - -```bash -node scripts/sdk_truth.mjs > truth.json -python3 -m core verify-sdk truth.json -``` - -An agent the compiler finds in a test file, or outside the code root, is listed under `skipped` -and is not a difference. Any other difference exits non-zero. - -### Prove risk against real tools - -Compare each number with the tool that owns it: paths with the compiler, commits with git's log, -and coverage with the test runner's own summary: - -```bash -python3 -m core coverage [commit] -python3 -m core verify-risk /tmp/truth.json -``` - -Paths are counted from text, so screen copy can fool them a little. They must stay within 0.5% of -the compiler. - -### Prove limits against real Postgres - -The rules that say what Postgres will refuse are a floor. When one says refused, Postgres refuses. -When they stay silent, Postgres may still refuse for a reason the model does not hold, such as a -function that takes a table's row. - -Run one proposal on a throwaway server, with one file of changes as JSON: - -```bash -python3 -m core proposal-sql changes.json /tmp/changes -scripts/try_proposal.sh origin/main supabase/migrations /tmp/try /tmp/changes [bootstrap.sql] -python3 -m core verify-proposal changes.json /tmp/try/results.json -``` - -Or hold every rule to a whole schema. This tries a drop, three new types and both null rules on -every column, and a drop on every table. It fails only if a rule says refused and Postgres accepts: - -```bash -scripts/sweep_limits.py origin/main supabase/migrations /tmp/sweep [bootstrap.sql] -``` - ## License Apache-2.0 diff --git a/docs/backlog.md b/docs/backlog.md index 04c5f8b..d7cd2c3 100644 --- a/docs/backlog.md +++ b/docs/backlog.md @@ -4,11 +4,11 @@ What's left, written so another agent can pick it up cold. Nothing here is start ## Before you start -- Work from `main`, on version 0.12.1. Each change goes on its own branch, with a PR into - `main`. The remote, `origin`, is a private GitHub repo. The older slice branches are kept, and - `main` holds all of them. -- Run the checks in the README's Checks section. All pass on `main`: 987 Python tests and - 116 UI tests. +- Work from `main`, on version 0.12.2. Each change goes on its own branch, with a PR into + `main`. The remote, `origin`, is the public GitHub repo `mj-hahn/system-map`. `main` only + merges once CI's `checks` job passes. +- Run the checks in [development.md](development.md#checks). All pass on `main`: 1056 Python tests + and 119 UI tests, with none skipped. - Agents can't log in to the dashboard. `ui/dev` stands in for the host, so a person has to press Sync and look. Say what has and hasn't been seen for real. - Look at screenshots before you commit UI. Checks have passed while pictures showed faults. diff --git a/docs/development.md b/docs/development.md new file mode 100644 index 0000000..2a78cc6 --- /dev/null +++ b/docs/development.md @@ -0,0 +1,154 @@ +# Development + +How System Map is built, how to check it, and how to hold each reader to the tool that owns the +facts. Run every command from the repo root. For what the app does and how to install it, see the +[README](../README.md). + +| Path | Job | +|---|---| +| `core/` | The model and extractors. Imports nothing from Kiro Crew. | +| `core/extractors/` | One module per stack. Each reads a repo into Facts. | +| `mcp_server/` | Tools that agents use. Stdlib only. | +| `backend/` | HTTP API for the dashboard pages. Stdlib only. | +| `ui/` | The app page and the chat side panel tab. | +| `skills/system-map/` | How agents should read and write the model. | +| `scripts/` | Replay migrations and try proposals on a throwaway Postgres, and hold the readers to the compiler and to `drizzle-kit`. | +| `docs/` | Context rules, platform notes and the backlog. | + +Host rules learned while building it are in [platform-notes.md](platform-notes.md), and what's left +is in [backlog.md](backlog.md). Releases are in [CHANGELOG.md](../CHANGELOG.md). + +## Checks + +```bash +python3 -m unittest discover -s tests +python3 tests/strict.py +(cd ui && npm test && npm run typecheck && npm run build) +``` + +`tests/strict.py` runs the same tests but fails on any skip, and prints each skipped test with its +reason. CI runs every PR against the oldest supported and the newest Kiro Crew. + +The replay test needs Postgres server binaries. Set `PGBIN` if they are not in +`/usr/lib/postgresql/16/bin`. The compiler test needs `npm install` in `ui`. The host contract tests +need an installed Kiro Crew, found through the `kirocrew` launcher or `KIROCREW_PYTHON`. Each skips +itself when its tool is missing. + +## Run the page without Kiro Crew + +`ui/dev` stands in for the host, so you can see and screenshot the page on its own: + +```bash +SYSTEM_MAP_DATA=/tmp/model python3 -m core build +SYSTEM_MAP_DATA=/tmp/model PORT=9191 SYSTEM_MAP_UNSIGNED=1 python3 backend/server.py & +(cd ui && npm run dev) +``` + +Open . Add `?theme=dark` or `?view=panel`. + +The backend refuses any call the gateway didn't sign, except `/health`. `ui/dev` can't sign, so +`SYSTEM_MAP_UNSIGNED=1` lets it in. A backend that Kiro Crew starts ignores that flag. + +## Prove a model against real Postgres + +Replay a repo's migrations on a throwaway server, then compare it with the model: + +```bash +scripts/replay_catalog.sh origin/main supabase/migrations /tmp/replay [bootstrap.sql] +python3 -m core build +python3 -m core verify /tmp/replay/catalog.json +``` + +`bootstrap.sql` is a repo path for anything the migrations expect to exist, such as a stub of +Supabase's `auth` schema. The server listens on a socket only, and stops when the script ends. + +## Prove a SQLite model + +Load the schema into a separate real SQLite, then compare every table with its pragmas: + +```bash +python3 -m core verify-sqlite +``` + +## Prove a Drizzle model + +Have the tool write its SQL in a copy of the project, then compare that with the reader. Use +`generate`, because `export` ignores `casing`: + +```bash +(cd && npx drizzle-kit generate --name proof) +python3 -m core verify-drizzle /drizzle/0000_proof.sql +``` + +To try every form the reader knows against the real tool, give the sweep any `node_modules` with +`drizzle-kit` and `drizzle-orm` installed. The same seed gives the same schemas: + +```bash +scripts/sweep_drizzle.py /tmp/sweep [rounds] [tables] [seed] +``` + +## Prove routes against Next.js + +Run `next build` in a copy of the app, then compare the routes Next wrote down: + +```bash +python3 -m core verify-routes /.next/app-path-routes-manifest.json /.next/server/pages-manifest.json +``` + +## Prove a model against the TypeScript compiler + +Ask the real compiler what a checkout imports and which tables it calls, then compare. This also +checks routes against TanStack's generated route tree: + +```bash +node scripts/ts_truth.mjs /tmp/truth.json +python3 -m core verify-code /tmp/truth.json +``` + +## Prove the SDK agent readers + +Ask the real compiler which agents a project declares with an agent SDK, then compare. Install +the SDKs the project uses, and TypeScript, in a scratch folder above or beside it, so its +`node_modules` resolve: + +```bash +node scripts/sdk_truth.mjs > truth.json +python3 -m core verify-sdk truth.json +``` + +An agent the compiler finds in a test file, or outside the code root, is listed under `skipped` +and is not a difference. Any other difference exits non-zero. + +## Prove risk against real tools + +Compare each number with the tool that owns it: paths with the compiler, commits with git's log, +and coverage with the test runner's own summary: + +```bash +python3 -m core coverage [commit] +python3 -m core verify-risk /tmp/truth.json +``` + +Paths are counted from text, so screen copy can fool them a little. They must stay within 0.5% of +the compiler. + +## Prove limits against real Postgres + +The rules that say what Postgres will refuse are a floor. When one says refused, Postgres refuses. +When they stay silent, Postgres may still refuse for a reason the model does not hold, such as a +function that takes a table's row. + +Run one proposal on a throwaway server, with one file of changes as JSON: + +```bash +python3 -m core proposal-sql changes.json /tmp/changes +scripts/try_proposal.sh origin/main supabase/migrations /tmp/try /tmp/changes [bootstrap.sql] +python3 -m core verify-proposal changes.json /tmp/try/results.json +``` + +Or hold every rule to a whole schema. This tries a drop, three new types and both null rules on +every column, and a drop on every table. It fails only if a rule says refused and Postgres accepts: + +```bash +scripts/sweep_limits.py origin/main supabase/migrations /tmp/sweep [bootstrap.sql] +``` diff --git a/tests/test_contract.py b/tests/test_contract.py index 1cc949e..69f1ee2 100644 --- a/tests/test_contract.py +++ b/tests/test_contract.py @@ -178,13 +178,21 @@ def test_the_readmes_local_links_and_images_exist(self): def test_the_readmes_jump_links_land_on_a_heading(self): text = (ROOT / "README.md").read_text() - # GitHub's heading anchors: lower case, punctuation dropped, each space a hyphen. - slugs = {re.sub(r"[^\w\- ]", "", heading.strip().lower()).replace(" ", "-") for heading in re.findall(r"^#+ (.+)$", text, re.M)} anchors = re.findall(r"\]\(#([^)]+)\)", text) self.assertGreaterEqual(len(anchors), 4) for anchor in anchors: with self.subTest(anchor=anchor): - self.assertIn(anchor, slugs) + self.assertIn(anchor, heading_anchors(text)) + + def test_links_between_the_docs_land_on_a_file_and_a_heading(self): + for doc in ("README.md", "docs/development.md", "docs/backlog.md"): + text = (ROOT / doc).read_text() + for target, anchor in re.findall(r"\]\(([^)#\s:]*)(?:#([^)\s]+))?\)", text): + path = (ROOT / doc).parent / target if target else ROOT / doc + with self.subTest(doc=doc, target=target, anchor=anchor): + self.assertTrue(path.exists()) + if anchor: + self.assertIn(anchor, heading_anchors(path.read_text())) def test_the_readmes_live_badges_read_fields_the_manifest_has(self): text = (ROOT / "README.md").read_text() @@ -221,6 +229,11 @@ def test_the_ui_bundle_is_committed(self): self.fail(f"ui/dist is not committed: {error}") +def heading_anchors(text: str) -> set[str]: + """GitHub's anchor for each heading: lower case, punctuation dropped, each space a hyphen.""" + return {re.sub(r"[^\w\- ]", "", heading.strip().lower()).replace(" ", "-") for heading in re.findall(r"^#+ (.+)$", text, re.M)} + + def released() -> list[str]: """The released versions in CHANGELOG.md, top first.""" text = (ROOT / "CHANGELOG.md").read_text()