Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@ commit that set the version in `app.json`. The format follows

## [Unreleased]

### Changed
- The developer guide (code layout, checks, running the page without Kiro Crew, and proving each
reader against the real tool) moved from the README to `docs/development.md`. The README keeps a
short pointer and the two check commands.

## [0.12.2] - 2026-11-09

### Security
Expand Down
150 changes: 5 additions & 145 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,8 +78,9 @@ home region, and any agent whose model runs elsewhere becomes a finding, with th
<img src="assets/screenshots/agents-light.png" alt="The Agents tab: four agents with what starts each, their model and whether it asks you first, and the Restock agent's detail, from its nightly job to its hand-off to the Buyer">
</picture>

<sub>Screenshots show a made-up bookshop, taken from `ui/dev`, the stand-in host described under
[Development](#development). In Kiro Crew the page sits inside the dashboard.</sub>
<sub>Screenshots show a made-up bookshop, taken from `ui/dev`, the stand-in host described in
[docs/development.md](docs/development.md#run-the-page-without-kiro-crew). In Kiro Crew the page
sits inside the dashboard.</sub>

## Install

Expand Down Expand Up @@ -238,155 +239,14 @@ choose **Sync**, then reload the page. A new tool reaches new sessions only.

## Development

| Path | Job |
|---|---|
| `core/` | The model and extractors. Imports nothing from Kiro Crew. |
| `core/extractors/` | One module per stack. Each reads a repo into Facts. |
| `mcp_server/` | Tools that agents use. Stdlib only. |
| `backend/` | HTTP API for the dashboard pages. Stdlib only. |
| `ui/` | The app page and the chat side panel tab. |
| `skills/system-map/` | How agents should read and write the model. |
| `scripts/` | Replay migrations and try proposals on a throwaway Postgres, and hold the readers to the compiler and to `drizzle-kit`. |
| `docs/` | Context rules, platform notes and the backlog. |

Host rules learned while building it are in `docs/platform-notes.md`, and what's left is in
`docs/backlog.md`. Releases are in `CHANGELOG.md`.

### Checks
The code layout, the checks, running the page without Kiro Crew, and how to prove each reader
against the real tool are in [docs/development.md](docs/development.md). The checks in short:

```bash
python3 -m unittest discover -s tests
python3 tests/strict.py
(cd ui && npm test && npm run typecheck && npm run build)
```

`tests/strict.py` runs the same tests but fails on any skip, and prints each skipped test with its
reason. CI runs every PR against the oldest supported and the newest Kiro Crew.

The replay test needs Postgres server binaries. Set `PGBIN` if they are not in
`/usr/lib/postgresql/16/bin`. The compiler test needs `npm install` in `ui`. The host contract tests
need an installed Kiro Crew, found through the `kirocrew` launcher or `KIROCREW_PYTHON`. Each skips
itself when its tool is missing.

### Run the page without Kiro Crew

`ui/dev` stands in for the host, so you can see and screenshot the page on its own:

```bash
SYSTEM_MAP_DATA=/tmp/model python3 -m core build <repo>
SYSTEM_MAP_DATA=/tmp/model PORT=9191 SYSTEM_MAP_UNSIGNED=1 python3 backend/server.py &
(cd ui && npm run dev)
```

Open <http://127.0.0.1:5199>. Add `?theme=dark` or `?view=panel`.

The backend refuses any call the gateway didn't sign, except `/health`. `ui/dev` can't sign, so
`SYSTEM_MAP_UNSIGNED=1` lets it in. A backend that Kiro Crew starts ignores that flag.

### Prove a model against real Postgres

Replay a repo's migrations on a throwaway server, then compare it with the model:

```bash
scripts/replay_catalog.sh <repo> origin/main supabase/migrations /tmp/replay [bootstrap.sql]
python3 -m core build <repo>
python3 -m core verify <repo> /tmp/replay/catalog.json
```

`bootstrap.sql` is a repo path for anything the migrations expect to exist, such as a stub of
Supabase's `auth` schema. The server listens on a socket only, and stops when the script ends.

### Prove a SQLite model

Load the schema into a separate real SQLite, then compare every table with its pragmas:

```bash
python3 -m core verify-sqlite <repo>
```

### Prove a Drizzle model

Have the tool write its SQL in a copy of the project, then compare that with the reader. Use
`generate`, because `export` ignores `casing`:

```bash
(cd <copy> && npx drizzle-kit generate --name proof)
python3 -m core verify-drizzle <repo> <copy>/drizzle/0000_proof.sql
```

To try every form the reader knows against the real tool, give the sweep any `node_modules` with
`drizzle-kit` and `drizzle-orm` installed. The same seed gives the same schemas:

```bash
scripts/sweep_drizzle.py <node_modules> /tmp/sweep [rounds] [tables] [seed]
```

### Prove routes against Next.js

Run `next build` in a copy of the app, then compare the routes Next wrote down:

```bash
python3 -m core verify-routes <repo> <copy>/.next/app-path-routes-manifest.json <copy>/.next/server/pages-manifest.json
```

### Prove a model against the TypeScript compiler

Ask the real compiler what a checkout imports and which tables it calls, then compare. This also
checks routes against TanStack's generated route tree:

```bash
node scripts/ts_truth.mjs <checkout> /tmp/truth.json
python3 -m core verify-code <repo> /tmp/truth.json
```

### Prove the SDK agent readers

Ask the real compiler which agents a project declares with an agent SDK, then compare. Install
the SDKs the project uses, and TypeScript, in a scratch folder above or beside it, so its
`node_modules` resolve:

```bash
node scripts/sdk_truth.mjs <project> > truth.json
python3 -m core verify-sdk <project> truth.json
```

An agent the compiler finds in a test file, or outside the code root, is listed under `skipped`
and is not a difference. Any other difference exits non-zero.

### Prove risk against real tools

Compare each number with the tool that owns it: paths with the compiler, commits with git's log,
and coverage with the test runner's own summary:

```bash
python3 -m core coverage <repo> <report> [commit]
python3 -m core verify-risk <repo> /tmp/truth.json <coverage-summary.json>
```

Paths are counted from text, so screen copy can fool them a little. They must stay within 0.5% of
the compiler.

### Prove limits against real Postgres

The rules that say what Postgres will refuse are a floor. When one says refused, Postgres refuses.
When they stay silent, Postgres may still refuse for a reason the model does not hold, such as a
function that takes a table's row.

Run one proposal on a throwaway server, with one file of changes as JSON:

```bash
python3 -m core proposal-sql changes.json /tmp/changes
scripts/try_proposal.sh <repo> origin/main supabase/migrations /tmp/try /tmp/changes [bootstrap.sql]
python3 -m core verify-proposal <repo> changes.json /tmp/try/results.json
```

Or hold every rule to a whole schema. This tries a drop, three new types and both null rules on
every column, and a drop on every table. It fails only if a rule says refused and Postgres accepts:

```bash
scripts/sweep_limits.py <repo> origin/main supabase/migrations /tmp/sweep [bootstrap.sql]
```

## License

Apache-2.0
10 changes: 5 additions & 5 deletions docs/backlog.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ What's left, written so another agent can pick it up cold. Nothing here is start

## Before you start

- Work from `main`, on version 0.12.1. Each change goes on its own branch, with a PR into
`main`. The remote, `origin`, is a private GitHub repo. The older slice branches are kept, and
`main` holds all of them.
- Run the checks in the README's Checks section. All pass on `main`: 987 Python tests and
116 UI tests.
- Work from `main`, on version 0.12.2. Each change goes on its own branch, with a PR into
`main`. The remote, `origin`, is the public GitHub repo `mj-hahn/system-map`. `main` only
merges once CI's `checks` job passes.
- Run the checks in [development.md](development.md#checks). All pass on `main`: 1056 Python tests
and 119 UI tests, with none skipped.
- Agents can't log in to the dashboard. `ui/dev` stands in for the host, so a person has to
press Sync and look. Say what has and hasn't been seen for real.
- Look at screenshots before you commit UI. Checks have passed while pictures showed faults.
Expand Down
154 changes: 154 additions & 0 deletions docs/development.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
# Development

How System Map is built, how to check it, and how to hold each reader to the tool that owns the
facts. Run every command from the repo root. For what the app does and how to install it, see the
[README](../README.md).

| Path | Job |
|---|---|
| `core/` | The model and extractors. Imports nothing from Kiro Crew. |
| `core/extractors/` | One module per stack. Each reads a repo into Facts. |
| `mcp_server/` | Tools that agents use. Stdlib only. |
| `backend/` | HTTP API for the dashboard pages. Stdlib only. |
| `ui/` | The app page and the chat side panel tab. |
| `skills/system-map/` | How agents should read and write the model. |
| `scripts/` | Replay migrations and try proposals on a throwaway Postgres, and hold the readers to the compiler and to `drizzle-kit`. |
| `docs/` | Context rules, platform notes and the backlog. |

Host rules learned while building it are in [platform-notes.md](platform-notes.md), and what's left
is in [backlog.md](backlog.md). Releases are in [CHANGELOG.md](../CHANGELOG.md).

## Checks

```bash
python3 -m unittest discover -s tests
python3 tests/strict.py
(cd ui && npm test && npm run typecheck && npm run build)
```

`tests/strict.py` runs the same tests but fails on any skip, and prints each skipped test with its
reason. CI runs every PR against the oldest supported and the newest Kiro Crew.

The replay test needs Postgres server binaries. Set `PGBIN` if they are not in
`/usr/lib/postgresql/16/bin`. The compiler test needs `npm install` in `ui`. The host contract tests
need an installed Kiro Crew, found through the `kirocrew` launcher or `KIROCREW_PYTHON`. Each skips
itself when its tool is missing.

## Run the page without Kiro Crew

`ui/dev` stands in for the host, so you can see and screenshot the page on its own:

```bash
SYSTEM_MAP_DATA=/tmp/model python3 -m core build <repo>
SYSTEM_MAP_DATA=/tmp/model PORT=9191 SYSTEM_MAP_UNSIGNED=1 python3 backend/server.py &
(cd ui && npm run dev)
```

Open <http://127.0.0.1:5199>. Add `?theme=dark` or `?view=panel`.

The backend refuses any call the gateway didn't sign, except `/health`. `ui/dev` can't sign, so
`SYSTEM_MAP_UNSIGNED=1` lets it in. A backend that Kiro Crew starts ignores that flag.

## Prove a model against real Postgres

Replay a repo's migrations on a throwaway server, then compare it with the model:

```bash
scripts/replay_catalog.sh <repo> origin/main supabase/migrations /tmp/replay [bootstrap.sql]
python3 -m core build <repo>
python3 -m core verify <repo> /tmp/replay/catalog.json
```

`bootstrap.sql` is a repo path for anything the migrations expect to exist, such as a stub of
Supabase's `auth` schema. The server listens on a socket only, and stops when the script ends.

## Prove a SQLite model

Load the schema into a separate real SQLite, then compare every table with its pragmas:

```bash
python3 -m core verify-sqlite <repo>
```

## Prove a Drizzle model

Have the tool write its SQL in a copy of the project, then compare that with the reader. Use
`generate`, because `export` ignores `casing`:

```bash
(cd <copy> && npx drizzle-kit generate --name proof)
python3 -m core verify-drizzle <repo> <copy>/drizzle/0000_proof.sql
```

To try every form the reader knows against the real tool, give the sweep any `node_modules` with
`drizzle-kit` and `drizzle-orm` installed. The same seed gives the same schemas:

```bash
scripts/sweep_drizzle.py <node_modules> /tmp/sweep [rounds] [tables] [seed]
```

## Prove routes against Next.js

Run `next build` in a copy of the app, then compare the routes Next wrote down:

```bash
python3 -m core verify-routes <repo> <copy>/.next/app-path-routes-manifest.json <copy>/.next/server/pages-manifest.json
```

## Prove a model against the TypeScript compiler

Ask the real compiler what a checkout imports and which tables it calls, then compare. This also
checks routes against TanStack's generated route tree:

```bash
node scripts/ts_truth.mjs <checkout> /tmp/truth.json
python3 -m core verify-code <repo> /tmp/truth.json
```

## Prove the SDK agent readers

Ask the real compiler which agents a project declares with an agent SDK, then compare. Install
the SDKs the project uses, and TypeScript, in a scratch folder above or beside it, so its
`node_modules` resolve:

```bash
node scripts/sdk_truth.mjs <project> > truth.json
python3 -m core verify-sdk <project> truth.json
```

An agent the compiler finds in a test file, or outside the code root, is listed under `skipped`
and is not a difference. Any other difference exits non-zero.

## Prove risk against real tools

Compare each number with the tool that owns it: paths with the compiler, commits with git's log,
and coverage with the test runner's own summary:

```bash
python3 -m core coverage <repo> <report> [commit]
python3 -m core verify-risk <repo> /tmp/truth.json <coverage-summary.json>
```

Paths are counted from text, so screen copy can fool them a little. They must stay within 0.5% of
the compiler.

## Prove limits against real Postgres

The rules that say what Postgres will refuse are a floor. When one says refused, Postgres refuses.
When they stay silent, Postgres may still refuse for a reason the model does not hold, such as a
function that takes a table's row.

Run one proposal on a throwaway server, with one file of changes as JSON:

```bash
python3 -m core proposal-sql changes.json /tmp/changes
scripts/try_proposal.sh <repo> origin/main supabase/migrations /tmp/try /tmp/changes [bootstrap.sql]
python3 -m core verify-proposal <repo> changes.json /tmp/try/results.json
```

Or hold every rule to a whole schema. This tries a drop, three new types and both null rules on
every column, and a drop on every table. It fails only if a rule says refused and Postgres accepts:

```bash
scripts/sweep_limits.py <repo> origin/main supabase/migrations /tmp/sweep [bootstrap.sql]
```
19 changes: 16 additions & 3 deletions tests/test_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -178,13 +178,21 @@ def test_the_readmes_local_links_and_images_exist(self):

def test_the_readmes_jump_links_land_on_a_heading(self):
text = (ROOT / "README.md").read_text()
# GitHub's heading anchors: lower case, punctuation dropped, each space a hyphen.
slugs = {re.sub(r"[^\w\- ]", "", heading.strip().lower()).replace(" ", "-") for heading in re.findall(r"^#+ (.+)$", text, re.M)}
anchors = re.findall(r"\]\(#([^)]+)\)", text)
self.assertGreaterEqual(len(anchors), 4)
for anchor in anchors:
with self.subTest(anchor=anchor):
self.assertIn(anchor, slugs)
self.assertIn(anchor, heading_anchors(text))

def test_links_between_the_docs_land_on_a_file_and_a_heading(self):
for doc in ("README.md", "docs/development.md", "docs/backlog.md"):
text = (ROOT / doc).read_text()
for target, anchor in re.findall(r"\]\(([^)#\s:]*)(?:#([^)\s]+))?\)", text):
path = (ROOT / doc).parent / target if target else ROOT / doc
with self.subTest(doc=doc, target=target, anchor=anchor):
self.assertTrue(path.exists())
if anchor:
self.assertIn(anchor, heading_anchors(path.read_text()))

def test_the_readmes_live_badges_read_fields_the_manifest_has(self):
text = (ROOT / "README.md").read_text()
Expand Down Expand Up @@ -221,6 +229,11 @@ def test_the_ui_bundle_is_committed(self):
self.fail(f"ui/dist is not committed: {error}")


def heading_anchors(text: str) -> set[str]:
"""GitHub's anchor for each heading: lower case, punctuation dropped, each space a hyphen."""
return {re.sub(r"[^\w\- ]", "", heading.strip().lower()).replace(" ", "-") for heading in re.findall(r"^#+ (.+)$", text, re.M)}


def released() -> list[str]:
"""The released versions in CHANGELOG.md, top first."""
text = (ROOT / "CHANGELOG.md").read_text()
Expand Down