feat(api): add the resource-state read model tables

[AlexAxthelm]

First PR of splitting #324 into smaller PRs for ease of review.

Adds the table that will eventually hold the read-state model, but no mechanisms to populate it (empty table). Introduces concept that read model will track the permissions and coalesced values.

---

Summary

First PR in a stack that adds a precomputed read model for the oil & gas field list and filter-options endpoints (STIT-766). Today those endpoints work out every field's winning value on each request (~2–3 s at ~50K resources); the read model lets them read precomputed rows instead (under 10 ms at the same size, measured on the original combined PR, #324).

This PR is schema only. The tables start empty and not ready, and nothing writes or reads them yet, so no answers change.

What's added

• og_field_resource_state: one row per resource per permission profile.
  • It holds each field's winning value, which source it came from (provenance), and the commit of the code that computed the row (code_version).
  • It is derived data: every row can be rebuilt from memberships, source values and priorities, so emptying it loses nothing.
  • Indexes cover the list's default sort and each filter-options field within a profile.
• og_field_resource_state_status: a single readiness row, created with ready = false. Only a complete full rebuild will set it true, recording the commit that built the table (code_version). Readers may use the table only when it is ready and was built by the same commit they are running, so every deploy of new code reads live until that code has rebuilt it.
• Four permission profiles (db/read_model/permissions.py): all public sources, plus each combination of the restricted wm / ccr.
  • A request is served from the table only when its licensed sources exactly match a profile; anything else gets the live answer.
  • That keeps the rule "you only see sources you're granted": a cached answer can never show more, or less, than the live one.
• Two drift checks:
  • The model fails at import if its value columns stop matching the field list.
  • A test fails if a new source key isn't assigned to public or restricted.
• deployments/api/READ_MODEL.md: the start of the runbook. Later PRs add their own sections.

The stack

1. Schema (this PR)
2. Build and rebuild: computing rows, the rebuild CLI (--if-stale, --ids), the rebuild lock, Postgres tests in CI
3. Writes keep the table current: each write locks the affected resources, then refreshes their rows
4. Read path: list and filter-options read from the table when it's ready, plus a per-process readiness cache, an operator switch-off, and its migration
5. CI: rebuild after each deploy, and wait for the old API revision to stop serving first

Every PR is safe to merge on its own. Nothing reads the table until #4, and readiness is tied to the commit, so a deploy without a matching rebuild reads live. #3 must land before (or with) #4.

Related issues

Part of: STIT-766
Replaces: #324 (split for review)

Testing

• New tests:
  • tests/db/read_model/test_resource_state_schema.py: columns follow the field list and types; primary key and cascade on delete; indexes; typed values round-trip; an unset JSON value is SQL NULL; masks outside 0–3 are rejected.
  • tests/db/read_model/test_permissions.py: exact profile matching; unscoped and partial grants fall back to live; every source key is assigned.
• Migration:
  • Applies cleanly on a fresh SQLite and a fresh Postgres database (single head).
  • On Postgres, alembic reports no differences between models and migrations.
  • The status row starts as ready = false.
• make check passes.

Checklist

• PR is focused on a single concern
• Tests pass locally and in CI
• Docs updated for user-visible changes
• AI-assisted portions declared (see CONTRIBUTING.md)

AI use: Claude Code split this out of #324 and wrote most of the code, tests and docs. I reviewed it before committing. Verified by the tests and migration checks above.

🤖 Generated with Claude Code

[Copilot]

🟢 Approval recommended

No blocking issues were identified in the additive schema-only changes, which leave existing API paths unchanged.

0 open findings

What changed in this PR

Adds the schema foundation for a permission-scoped read model for oil and gas field list and filter-options endpoints. The tables remain unused, so API behavior is unchanged.

Changes:

• Adds resource-state and readiness tables, indexes, and a migration.
• Defines four exact permission profiles with live fallback for unmatched grants.
• Adds schema and permission tests plus initial runbook documentation.

File	Description
deployments/​api/​tests/​db/​read_model/​test_resource_state_schema.py	Tests columns, types, constraints, indexes, and value storage.
deployments/​api/​tests/​db/​read_model/​test_permissions.py	Tests exact profile matching and source coverage.
deployments/​api/​tests/​db/​read_model/​__init__.py	Initializes the test package.
deployments/​api/​src/​stitch/​api/​db/​read_model/​permissions.py	Defines permission profiles and matching helpers.
deployments/​api/​src/​stitch/​api/​db/​read_model/​__init__.py	Documents the read-model package.
deployments/​api/​src/​stitch/​api/​db/​model/​og_field_resource_state.py	Defines state and readiness models with a column drift guard.
deployments/​api/​src/​stitch/​api/​db/​model/​__init__.py	Exports the new models.
deployments/​api/​READ_MODEL.md	Documents scope, readiness, and maintenance.
deployments/​api/​alembic/​versions/​48a49d197641_add_resource_state_read_model.py	Creates tables and indexes; seeds readiness as false.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

CD summary 3eaa743

Frontend: https://witty-mushroom-017a3dc1e-330.westus2.1.azurestaticapps.net

Deployments (3)

service	url	fqdn
api	open	pr-0330-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io
frontend	https://witty-mushroom-017a3dc1e-330.westus2.1.azurestaticapps.net
stitch-llm	open	pr-0330-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io

Database (1)

db_name	postgres_host	postgres_port	postgres_db
pr_0330	stitch-dev.postgres.database.azure.com	5432	pr_0330

Jobs (2)

job	image	postgres_db	api_url	auth_mode
db-migrations	ghcr.io/rmi/stitch-api:pr-0330@sha256:fed67f3d6b95dc0d35731b1f4767535f1dc06f257af3825b11a4ba1bc16c8d69	pr_0330
seed	ghcr.io/rmi/stitch-seed:pr-0330@sha256:a3ec1b6b431cffc0f8935d3598c48f716cb6813fc664b1fe5bd663bf6d78490a		https://pr-0330-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io/api/v1	stitch-client-bearer-token

Images (3)

build_time	commit_time	git_sha	image	image_digest
2026-10-15T10:25:22Z	2026-10-15T10:25:04Z	c0bc82c	ghcr.io/rmi/stitch-api:pr-0330	ghcr.io/rmi/stitch-api:pr-0330@sha256:fed67f3d6b95dc0d35731b1f4767535f1dc06f257af3825b11a4ba1bc16c8d69
2026-10-15T10:25:23Z	2026-10-15T10:25:04Z	c0bc82c	ghcr.io/rmi/stitch-seed:pr-0330	ghcr.io/rmi/stitch-seed:pr-0330@sha256:a3ec1b6b431cffc0f8935d3598c48f716cb6813fc664b1fe5bd663bf6d78490a
2026-10-15T10:25:26Z	2026-10-15T10:25:04Z	c0bc82c	ghcr.io/rmi/stitch-stitch-llm:pr-0330	ghcr.io/rmi/stitch-stitch-llm:pr-0330@sha256:297703536cc0c3f95fe656acf02a11ac6e76724e7e4e128107232bc842f34522

[mbarlow12]

2 big things: concurrency and "live" query fallbacks

I acknowledge that it's technically possible for transactions to overlap unpredictably, but the approach in these PRs leads to a lot of knock on effects and added complexity. The 4 stacked PRs add roughly ~5300 lines, ~1200 of which are in deployments/api/src (~750 if we strip comments, docstrings, and blank lines) which is roughly a 20% increase in the current api size. I know LOC is a poor metric, but it can serve as a quick, rough barometer on the suitability of a design or implementation or on development trends we may not be aware we're instituting.

A few of the standout concerns:

• There's commit sha tracking in the db? And it can apparently divert reads from the new table to the original SQL functions which led to the ticket in the first place.
• A later PR has tcp configuration related to the locking inside the db package.
• There's a separate script that appears to only be called from CI.
• There's an entire test suite checking postgres concurrency.
• If a future user—and we have 0 right now—is somehow missing source:read:bc in their permissions list, they're now subject to the same performance degradation as before despite the data being free and available, and we're erroneously and artificially restricting access.
• Maintaining the "live_query" path retains the coupling of our public API to the existing coalescing implementation, and going forward we have to maintain parity between the state table rebuilding and delivering the live querying

If we do all list reads from the state table, there's no parity to maintain and we can keep or change as much as we want around when and how we update the state table to ensure it's always representative. We fully solve the performance problem from the ticket in exchange for a mostly hypothetical, likely minor correctness risk (see below).

While it may sound like a big risk to possibly have the state table written from a transaction with a stale view of the data, there are a number of mitigating factors:

• One or more users (of which we have 0), would need to hit multiple/repeated mutating endpoints rather quickly. Even then, the final transaction with the "winning" rebuild will probably see most of the previous commits
• The vast majority of our resources are single source and will be until we add quite a lot more sources. The staleness risk only applies to a small proportion of our data.
• This is further narrowed down at the attribute level. A stale read doesn't mean that every attribute is also stale. That leaves a small proportion of the resources and what is probably a small proportion of the source values that could even meaningfully differ in the first place
• 1 reasonably isolated rebuild gets the table back in line. We could refresh the state in middleware every 15 requests or something—there are a lot of options here.

With a reasonably frequent update/sync cadence, I'd like to see if the eventual consistency we get is sufficient for our needs before adding the locking and concurrency. Let's at least try dropping the live_query functions and ignoring possibly stale state writes or at least considering other update options like syncing state from middleware or using sqalchemy events.