Skip to content

refactor(store): bun builders on existing models - #2134

Merged
tschellenbach merged 7 commits into
mainfrom
bun-builders-1
Nov 19, 2026
Merged

tschellenbach merged 7 commits into
mainfrom
bun-builders-1

Conversation

@tschellenbach

Copy link
Copy Markdown
Member

Why

The store mixes bun's query builders with hand-written SQL strings and positional ? lists, even for tables that already have a model. A raw statement repeats the table's column names, so it misses a renamed column or a new notnull default and is easy to get out of order. This is batch 1 of three: statements on tables that already have a model. Stacked on #2133, which fixes the duplicate migration version on main.

How it works

each raw statement in internal/store and usage/ledger
├─ an existing model fits ............ NewSelect / NewUpdate / NewDelete /
│                                      NewInsert ... On("CONFLICT ...")
├─ SQL text pinned by a test ......... kept raw (claim queries,
│                                      FOR NO KEY UPDATE lock, EXPLAIN reads)
└─ JSON must fail on \u0000 .......... kept raw (channel inbound queue)
Rule Why
Inserts list .Column(...) state, attempts, received_at, seq keep their database defaults
Explicit Returning(...) lists stay imageJobClaimed must not read the 10 MiB picture
One-row updates use a nil pointer model a slice model means a bulk update
A union ordered as a whole is wrapped in TableExpr("(?) AS member", union) bun puts a union's ORDER BY inside its first arm

Changes

  • Connector tokens, connection open attempts (attemptOpen shares one scope), OAuth clients, tool pins and the connector delete loop.
  • Chat and channel inbound queues (all but their claims), balance notices, event destination rotation.
  • Image job claims, Reship and Unshipped, campaign lease takeover and classification put-off.
  • Forget (store and ledger), user phones, prices and price changes, the store.go reads, grants, episodes, campaign analytics GROUP BYs and the ledger prune.
  • fix(store): keep the channel inbound queue insert raw. bun writes a JSON \u0000 as the text \\u0000 (schema/dialect.go, AppendJSON), so a delivery whose headers jsonb cannot hold was claimed instead of failing, and the provider's retry found it taken. That insert stays json.Marshal plus ?::jsonb. The balance notice insert does go through the builder: its headers are ones the router builds, so no NUL reaches it.

Validation

  • go test -tags integration passes for ./internal/store, ./internal/usage/ledger and ./cmd/router, plus the API channel and chat suites and go vet -tags integration ./....
  • TestEventForwardingSuite/TestAMessageTheStoreFailsToQueueIsNotClaimed caught the jsonb change above; it passes again.
  • New tests cover conversions nothing tested. Each passes on the raw SQL too, so they pin behaviour rather than the new code:
    • TestReshipQueuesTheShippedRowsStartedSinceAgain, TestReshipOfRejectedRowsQueuesTheRefusedOnesAndClearsWhy: a reship that misses rows or keeps ship_error.
    • TestUnshippedCountsTheRowsWithAnOrganizationStillToShip: a count that includes shipped rows or ones with no organization.
    • TestAnOrganizationsAppsAreTheOnesSeenOrRegisteredUnderItAndItsOwn: the union losing one of its arms or its order.

tschellenbach and others added 6 commits November 19, 2026 12:53
20270102090000_agent_config_tool_groups.sql shared its version with 20270102090000_ui_instructions.sql, and goose refuses to run with a duplicate, so the router could not migrate. It is 20270103090000 now.

TestARedactedCallStillReportsWhatItAndItsToolCost set redact_at on the global database, so under the sharded suite RedactToolOutput, which runs on the shard, found nothing.

Co-authored-by: Cursor <cursoragent@cursor.com>
…th bun

Config tokens, authorization attempts, OAuth clients, the tool re-approval read and the connector delete loop go through bun's builders. The attempt updates share openAttempt's conditions instead of repeating them.

Co-authored-by: Cursor <cursoragent@cursor.com>
The chat and channel inbound queues (all but their claims and the thread lock, whose SQL text tests match), balance notice deliveries and event destination rotation go through bun's builders. The models' jsonb types marshal themselves, so the hand-written json.Marshal and ?::jsonb casts go.

Co-authored-by: Cursor <cursoragent@cursor.com>
…th bun

ClaimImageJobs, Reship, Unshipped, TakeOverExpired and PutOffClassification go through bun's With, TableExpr and For. Reship and Unshipped had no test; they have now, and the tests pass on the raw SQL they replace too.

Co-authored-by: Cursor <cursoragent@cursor.com>
…ith bun

User organizations and settings, linked phones, active price entries, organization apps, tag keys, turns, grant locks, episode summaries, campaign counts and the ledger's forget and prune go through bun's builders. qualified() goes: bun names a model's columns itself.

reconcileGrantsQuery stays raw: a test EXPLAINs its text. OrganizationApps had no test; it has now, passing on the old statement too.

Co-authored-by: Cursor <cursoragent@cursor.com>
bun writes a JSON \u0000 as the text \\u0000, so a delivery whose headers jsonb cannot hold was queued and claimed instead of failing, and the provider's retry found it taken. The insert goes back to json.Marshal and ?::jsonb, which TestAMessageTheStoreFailsToQueueIsNotClaimed checks.

Co-authored-by: Cursor <cursoragent@cursor.com>
Base automatically changed from fix-tool-groups-migration to main November 19, 2026 20:52
@tschellenbach
tschellenbach marked this pull request as ready for review November 19, 2026 20:52
@tschellenbach
tschellenbach merged commit d123bcb into main Nov 19, 2026
5 checks passed
@tschellenbach
tschellenbach deleted the bun-builders-1 branch November 19, 2026 20:53
tschellenbach added a commit that referenced this pull request Nov 19, 2026
## Why

Batch 2 of 3. The tables here had no model, so every statement on them
was hand-written SQL with positional arguments. Each gets a small
unexported model, and its statements move to the builders. Stacked on
#2134.

## How it works

```
table                         model                    statements
channel_thread_messages,      channelThreadMessage,    claim, release,
  _people, _members,            Person, Member,        set / read person,
  _starters, _waiting           Starter,               member, starter, wait
                                waitingMessage
channel_deliveries            channelDelivery          claim, last at
chat_channel_turns            chatChannelTurn          take, release
connection_event_deliveries   connectionEventDelivery  claim
hook_deliveries               hookDelivery             record, forget
data_move_cursors             dataMoveCursor           read, save
limit_grant_uses,             grantUse,                counted,
  _reconciles                   grantReconcile         reconcile mark
```

A conflict clause that names the table goes by the model's alias: bun
writes `INSERT INTO chat_channel_turns AS cht`, and Postgres accepts
only the alias after that (`cht.turn_until < now()`,
`GREATEST(dmc.cursor, ...)`, `GREATEST(lgr.started_at, ...)`).

## Changes

- `channel_threads.go` has no raw SQL left. The waiting-message take is
a `NewDelete ... Returning(...)` with its column list kept.
- `NextConnectionEventSubscriptionAt` reads `min(next_attempt_at)` on
the existing model; the subscription claim stays raw, as its SQL text is
swapped in a test.
- `hook_deliveries` inserts list only `key`, so `seen_at` keeps its
`now()` default.

## Validation

- `go test -tags integration` passes for `./internal/store`,
`./internal/policy` (`TestJevSuite` excluded: it calls the hosted
classifier, which answered 401 locally), `./internal/channelbridge`,
`cmd/router`'s `TestReplicateRepeatSuite` (move cursors), and the API
Slack, WhatsApp, Linq, Telnyx, event forwarding and scheduled tasks
suites.
- New tests, both passing on the raw SQL as well:
- `TestGrantUsesCountedIsHowManyGrantsEachRowCountsToward`: a count that
drops the GROUP BY, or includes rows nobody asked about.
- `TestTheFirstStarterOfAThreadIsTheOneKept`: a starter upsert that
overwrites the first one.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant