feat(api): build/rebuild the resource-state read model tables

[AlexAxthelm]

Second PR of splitting #324 into smaller PRs for ease of review. Note that this is the biggest of the batch, since it contains the core of the functionality, and tests for same checking against current behavior

Builds on #330, introducing mechanism to build rows in the table, and rebuild the full table. The individual-row builds will be used to keep resources up to date as they are operated on (merge, attach, etc), while rebuild is intended for initial population and maitenence. Importantly, because the git SHA is the cache key, they will need to be rebuilt on each re-deploy, but this will be automated in followup PRs. Using the code version as the cache key lets us lean on an automatable step, rather than tracking a separate invalidation key.

Uses the same query as the live coalescing route, but INSERTs the result into the table, rather than reporting the results.

Other notes: This PR depends on some Postgres-specific features, so the tests for them are Opt-in tests locally, since they require a real PG database to test against (all the other tests still work fine in SQLite), and there are changes to the CI python tests to run them on GH.

---

Summary

Second PR in the STIT-766 read-model stack (builds on #330). It adds the code that fills og_field_resource_state and the command to rebuild it. Nothing reads the table yet and API writes don't refresh it, so answers are unchanged.

Building rows

• Same winner query as the live list: each listable resource gets one row per permission profile, computed with coalesced_winner_rows, the same SQL ranking the live list uses, run once per profile with that profile's exact source set. There's no second ranking implementation, so stored rows can't drift from live answers.
• No validation while building: values are stored as the live path materializes them and aren't validated. Like the live list, they're validated only when a page is built from them, so one value that today's model rejects can't stop a rebuild.
• refresh_resource_states(session, ids) recomputes given resources inside the caller's transaction.
  • It locks their resource rows first, in id order.
  • It deletes their state and re-inserts rows for those still listable; a resource that stopped being listable is left with none.
  • Its queries are labeled read_model.refresh.
  • resource_universe() in queries.py (previously private) decides listability for both the live list and the read model, so they can't diverge.

Rebuilding

python -m stitch.api.db.read_model.rebuild

• Full rebuild:
  • Clears readiness, then rebuilds in committed batches (--batch-size, default 500).
  • Redoes any resource whose rows were since rewritten by other code, up to 5 passes.
  • Removes rows of resources that are no longer listable.
  • Checks the result (every listable resource has all four rows, all computed by this code, and nothing else has any), and only then marks it ready with this commit as code_version.
  • Never empties the table, so a reader that checked readiness just before the rebuild started still finds a row for every resource.
  • ~90 s at 50K resources.
• --if-stale (the post-deploy step): does nothing if the table is already built by this commit. Otherwise it recomputes only what isn't already current, so re-running after a failed or cancelled run picks up where it stopped.
• --ids 12 34: repairs specific resources and never changes readiness.
  • Unknown ids (likely typos) fail without changing anything.
  • It refuses to touch a table that is ready but was built by a different commit, so the table never mixes two versions.
• One rebuild at a time: a Postgres advisory lock ensures this. A second full rebuild fails at once; --if-stale waits up to 10 minutes, then re-checks. The lock's connection asks Postgres to drop vanished clients within about a minute, so a killed rebuild (e.g. a cancelled CI job) doesn't hold the lock for hours.
• Errors print as one ERROR line and exit with status 1.

CI

python-test.yml gets a throwaway Postgres service for the tests SQLite can't exercise (here: the rebuild lock). The workflow sets STITCH_REQUIRE_POSTGRES_TESTS, so a missing database URL fails the run instead of silently skipping those tests.

Docs

deployments/api/READ_MODEL.md gains "Running a rebuild" (commands, where to run it, what happens, failures, a stuck lock) and "Tests" (how to run the Postgres tests locally).

The stack

1. Schema (feat(api): add the resource-state read model tables #330)
2. Build and rebuild (this PR)
3. Writes keep the table current: each write locks the affected resources, then refreshes their rows
4. Read path: list and filter-options read from the table when it's ready, plus a per-process readiness cache, an operator switch-off, and its migration
5. CI: rebuild after each deploy, and wait for the old API revision to stop serving first

Related issues

Part of: STIT-766
Follow-up: STIT-809 (incremental --since repair)
Replaces: #324 (split for review)

Testing

• New tests:
  • tests/db/read_model/test_state.py covers:
    • stored rows match live coalescing for every profile, including restricted values, null shells, per-resource overrides, JSON and numeric values, and merged resources;
    • targeted refresh;
    • lock ordering;
    • commit versions (table and rows);
    • repeated passes when other code rewrites rows, and giving up after 5;
    • resuming with --if-stale;
    • repairing corrupted, missing and stray rows;
    • failed rebuilds leaving the table not ready;
    • the integrity check;
    • CLI arguments, output and guards.
  • tests/db/read_model/test_rebuild_lock_postgres.py (Postgres only): --if-stale waits for a running rebuild, then rebuilds or skips; gives up after its limit; a full rebuild fails at once; keepalive settings on the lock's connection.
• Results: the API suite passes on SQLite and against Postgres; make check passes.
• Deliberate breakage check: I broke each of these by hand and confirmed a test fails:
  • the --if-stale skip;
  • redoing rows from other code;
  • the --ids refusal and its "not ready" exception.
• Gap: the lock inside the refresh is only exercised under real concurrent writes, which arrive with Reconcile Dockerfile & compose for dev/prod targets #3's Postgres concurrency tests.
• End to end: ran the CLI against a copy of the 50K-resource benchmark database with GIT_SHA=pr2-check.
  • --if-stale rebuilt 52,017 resources (208,068 rows, all stamped with that commit) in 90 s.
  • A second --if-stale did nothing.
  • --ids with an unknown id refused and changed nothing.

Checklist

• PR is focused on a single concern
• Tests pass locally and in CI
• Docs updated for user-visible changes
• AI-assisted portions declared (see CONTRIBUTING.md)

AI use: Claude Code split this out of #324 and wrote most of the code, tests and docs. I reviewed it before committing. Verified by the tests, the deliberate breakage check and the 50K run above.

🤖 Generated with Claude Code

[github-actions]

CD summary ecbe7f5

Frontend: https://witty-mushroom-017a3dc1e-331.westus2.1.azurestaticapps.net

Deployments (3)

service	url	fqdn
api	open	pr-0331-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io
frontend	https://witty-mushroom-017a3dc1e-331.westus2.1.azurestaticapps.net
stitch-llm	open	pr-0331-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io

Database (1)

db_name	postgres_host	postgres_port	postgres_db
pr_0331	stitch-dev.postgres.database.azure.com	5432	pr_0331

Jobs (2)

job	image	postgres_db	api_url	auth_mode
db-migrations	ghcr.io/rmi/stitch-api:pr-0331@sha256:b21dda2d10ced3cecc8eb60384880443d742309d6825a6eaafffe48aa2cad07e	pr_0331
seed	ghcr.io/rmi/stitch-seed:pr-0331@sha256:540765ef12649182cc4f1e534b5c6ff517a3e9edc5e5a9469dd1183d4f48476f		https://pr-0331-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io/api/v1	stitch-client-bearer-token

Images (3)

build_time	commit_time	git_sha	image	image_digest
2026-10-15T10:49:17Z	2026-10-15T10:48:54Z	b05a99a	ghcr.io/rmi/stitch-api:pr-0331	ghcr.io/rmi/stitch-api:pr-0331@sha256:b21dda2d10ced3cecc8eb60384880443d742309d6825a6eaafffe48aa2cad07e
2026-10-15T10:49:12Z	2026-10-15T10:48:54Z	b05a99a	ghcr.io/rmi/stitch-seed:pr-0331	ghcr.io/rmi/stitch-seed:pr-0331@sha256:540765ef12649182cc4f1e534b5c6ff517a3e9edc5e5a9469dd1183d4f48476f
2026-10-15T10:49:11Z	2026-10-15T10:48:54Z	b05a99a	ghcr.io/rmi/stitch-stitch-llm:pr-0331	ghcr.io/rmi/stitch-stitch-llm:pr-0331@sha256:0459b1a2a674030b65e8539388292bb270ab2fbca252f991ed470e5c493823d5

[Copilot]

🟡 Changes recommended

Concurrent repairs can leave mixed-version rows marked ready, and CLI failures do not consistently follow the documented error handling.

2 open findings

• Prevent mixed-version readiness during concurrent state repair · New
• Normalize unexpected CLI errors to one-line failures · New

What changed in this PR

Adds read-model population and rebuild support on top of #330’s schema. API reads and write paths remain unchanged.

Changes:

• Builds permission-scoped rows using the existing live coalescer.
• Adds batched rebuilds, resumable execution, targeted repairs, and readiness checks.
• Adds tests, PostgreSQL CI support, and operational documentation.

File	Description
deployments/​api/​tests/​db/​read_model/​test_state.py	Tests parity, refreshes, rebuilds, and CLI guards.
deployments/​api/​tests/​db/​read_model/​test_rebuild_lock_postgres.py	Tests PostgreSQL rebuild locking and keepalives.
deployments/​api/​tests/​db/​read_model/​postgres.py	Controls PostgreSQL test requirements.
deployments/​api/​tests/​db/​read_model/​dataset.py	Provides representative resource fixtures.
deployments/​api/​tests/​db/​read_model/​conftest.py	Adds rebuild and isolated PostgreSQL fixtures.
deployments/​api/​src/​stitch/​api/​db/​read_model/​state.py	Implements row building, locking, and rebuild readiness.
deployments/​api/​src/​stitch/​api/​db/​read_model/​rebuild.py	Adds rebuild and repair CLI commands.
deployments/​api/​src/​stitch/​api/​db/​queries.py	Exposes shared resource-listability query.
deployments/​api/​READ_MODEL.md	Documents rebuild operations and testing.
.github/​workflows/​python-test.yml	Supplies PostgreSQL for CI tests.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.