Skip to content

fix(store): give the tool groups migration its own version - #2133

Merged
tschellenbach merged 1 commit into
mainfrom
fix-tool-groups-migration
Nov 19, 2026
Merged

tschellenbach merged 1 commit into
mainfrom
fix-tool-groups-migration

Conversation

@tschellenbach

Copy link
Copy Markdown
Member

Why

Main has two migrations at version 20270102090000: agent_config_tool_groups (from #2088) and ui_instructions. Goose refuses a duplicate version, so a fresh database cannot migrate, and a database that applied one of them skips the other. Locally that showed up as column "ui_instructions" of relation "agent_configs" does not exist in every API suite.

How it works

before                                after
20270101090000 caller_tools           20270101090000 caller_tools
20270102090000 ui_instructions        20270102090000 ui_instructions
20270102090000 tool_groups  <- dup    20270103090000 tool_groups

Changes

  • Renames the tool groups migration to 20270103090000, so it runs after ui_instructions.
  • tool_groups_migration_test.go migrates to the version before it and then to it, using the new numbers.
  • store/usage_test.go's redact test writes through ShardDB(), since the rows it changes live in the shard, and a comment in models.go names the new version.

Validation

  • go test -tags integration ./internal/store passes against a fresh Postgres.
  • TestToolGroupsMigrationSuite migrates from 20270102090000 to 20270103090000 and checks the configs it rewrites, a rerun after a half-finished run, and the down migration.
  • A database that already applied the old 20270102090000 as tool groups is missing ui_instructions. Local test databases in that state need dropping. Whether a hosted environment migrated while main had the duplicate is unverified.

20270102090000_agent_config_tool_groups.sql shared its version with 20270102090000_ui_instructions.sql, and goose refuses to run with a duplicate, so the router could not migrate. It is 20270103090000 now.

TestARedactedCallStillReportsWhatItAndItsToolCost set redact_at on the global database, so under the sharded suite RedactToolOutput, which runs on the shard, found nothing.

Co-authored-by: Cursor <cursoragent@cursor.com>
@tschellenbach
tschellenbach marked this pull request as ready for review November 19, 2026 20:52
@tschellenbach
tschellenbach merged commit ca3a6c1 into main Nov 19, 2026
12 of 14 checks passed
@tschellenbach
tschellenbach deleted the fix-tool-groups-migration branch November 19, 2026 20:52
tschellenbach added a commit that referenced this pull request Nov 19, 2026
## Why

The store mixes bun's query builders with hand-written SQL strings and
positional `?` lists, even for tables that already have a model. A raw
statement repeats the table's column names, so it misses a renamed
column or a new `notnull` default and is easy to get out of order. This
is batch 1 of three: statements on tables that already have a model.
Stacked on #2133, which fixes the duplicate migration version on main.

## How it works

```
each raw statement in internal/store and usage/ledger
├─ an existing model fits ............ NewSelect / NewUpdate / NewDelete /
│                                      NewInsert ... On("CONFLICT ...")
├─ SQL text pinned by a test ......... kept raw (claim queries,
│                                      FOR NO KEY UPDATE lock, EXPLAIN reads)
└─ JSON must fail on \u0000 .......... kept raw (channel inbound queue)
```

| Rule | Why |
| --- | --- |
| Inserts list `.Column(...)` | `state`, `attempts`, `received_at`,
`seq` keep their database defaults |
| Explicit `Returning(...)` lists stay | `imageJobClaimed` must not read
the 10 MiB picture |
| One-row updates use a nil pointer model | a slice model means a bulk
update |
| A union ordered as a whole is wrapped in `TableExpr("(?) AS member",
union)` | bun puts a union's ORDER BY inside its first arm |

## Changes

- Connector tokens, connection open attempts (`attemptOpen` shares one
scope), OAuth clients, tool pins and the connector delete loop.
- Chat and channel inbound queues (all but their claims), balance
notices, event destination rotation.
- Image job claims, `Reship` and `Unshipped`, campaign lease takeover
and classification put-off.
- Forget (store and ledger), user phones, prices and price changes, the
`store.go` reads, grants, episodes, campaign analytics GROUP BYs and the
ledger prune.
- `fix(store): keep the channel inbound queue insert raw`. bun writes a
JSON `\u0000` as the text `\\u0000` (`schema/dialect.go`, `AppendJSON`),
so a delivery whose headers jsonb cannot hold was claimed instead of
failing, and the provider's retry found it taken. That insert stays
`json.Marshal` plus `?::jsonb`. The balance notice insert does go
through the builder: its headers are ones the router builds, so no NUL
reaches it.

## Validation

- `go test -tags integration` passes for `./internal/store`,
`./internal/usage/ledger` and `./cmd/router`, plus the API channel and
chat suites and `go vet -tags integration ./...`.
-
`TestEventForwardingSuite/TestAMessageTheStoreFailsToQueueIsNotClaimed`
caught the jsonb change above; it passes again.
- New tests cover conversions nothing tested. Each passes on the raw SQL
too, so they pin behaviour rather than the new code:
- `TestReshipQueuesTheShippedRowsStartedSinceAgain`,
`TestReshipOfRejectedRowsQueuesTheRefusedOnesAndClearsWhy`: a reship
that misses rows or keeps `ship_error`.
- `TestUnshippedCountsTheRowsWithAnOrganizationStillToShip`: a count
that includes shipped rows or ones with no organization.
- `TestAnOrganizationsAppsAreTheOnesSeenOrRegisteredUnderItAndItsOwn`:
the union losing one of its arms or its order.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant