Skip to content

fix(db): make PostgreSQL migrations atomic and serialized - #921

Open
onmax wants to merge 1 commit into
nuxt-hub:mainfrom
onmax:fix/postgresql-atomic-migrations
Open

onmax wants to merge 1 commit into
nuxt-hub:mainfrom
onmax:fix/postgresql-atomic-migrations

Conversation

@onmax

@onmax onmax commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

on postgres, a migration that fails halfway leaves the db half-migrated, and two instances booting at once can apply the same migration twice.

this runs each postgres migration file and its _hub_migrations row in one transaction, under an advisory lock, so each file is applied once and fully or not at all. mysql and sqlite don't change. statements that can't run in a transaction (CREATE INDEX CONCURRENTLY, VACUUM, BEGIN/COMMIT) now fail, unless the file starts with -- nuxthub:no-transaction. this is documented, and I think it is worth a line in the release notes. a new enum value has to be used in the next file. migration names are now passed as parameters in the runner, mark-as-migrated and squash.

@vercel

vercel Bot commented Aug 12, 2026

Copy link
Copy Markdown

@onmax is attempting to deploy a commit to the NuxtLabs Team on Vercel.

A member of the Team first needs to authorize it.

@pkg-pr-new

pkg-pr-new Bot commented Aug 12, 2026 •

Copy link
Copy Markdown
npm i https://pkg.pr.new/@nuxthub/core@921

commit: 84444ee

@onmax onmax changed the title fix(db): make PostgreSQL migrations atomic feat(db): support PostgreSQL migration schemas Sep 7, 2026

atinux commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

Drafted by an AI agent (Amp) on behalf of @atinux, who reviewed it before posting.

Thanks @onmax, making PostgreSQL migrations atomic and serialized is a real fix. Today a statement that fails halfway leaves the database half-migrated, and two instances booting at once can run the same migration twice. The PGlite tests checking real database state are great too.

A few things before this can land:

1. Atomicity changes behavior for every PostgreSQL user, not only those setting migrationsSchema.
Each migration now runs in a single transaction, so some valid migrations will start failing:

  • CREATE INDEX CONCURRENTLY, VACUUM, or anything managing its own transaction
  • ALTER TYPE … ADD VALUE followed by a use of the new value in the same file ("unsafe use of new enum value")

I think this needs a changelog note and docs, and maybe a way to opt out.

2. The migrationsSchema: 'auth' example will fail on the first run with Drizzle-generated migrations.
The runner creates the schema before any migration runs, but drizzle-kit does not emit IF NOT EXISTS. With drizzle-kit 0.31.9 (the version in this repo), pgSchema('auth') generates:

CREATE SCHEMA "auth";
--> statement-breakpoint
CREATE TABLE "auth"."sessions" (
	"token" text PRIMARY KEY NOT NULL
);

So the app's own first migration would error with "schema already exists". Users don't hand-write these files, so asking for IF NOT EXISTS in the docs won't really help. Options:

  • require the schema to already exist instead of creating it
  • recommend a schema dedicated to the history table

3. Moving history implicitly feels risky.
Today ALTER TABLE public._hub_migrations SET SCHEMA … runs whenever the option is set, including from CLI commands. During a rolling deploy or a rollback, old code would recreate an empty public._hub_migrations and replay every migration. An explicit CLI step to move the history would be safer.

4. Scope.

Would you be open to splitting this into two PRs, rebased on main?

  1. Atomic + advisory-locked migrations, with the transaction limits documented
  2. migrationsSchema, without creating the schema automatically or moving existing history implicitly

Smaller notes:

  • MySQL/SQLite still insert into a hardcoded _hub_migrations, while PostgreSQL uses getMigrationsTableName, so the two paths are inconsistent.
  • mark-as-migrated interpolates args.name directly into SQL. That predates this PR, but it might be worth fixing while you're in there.

Each PostgreSQL migration file now runs in one transaction with its history row, under an advisory lock that also guards history table creation. Files starting with `-- nuxthub:no-transaction` run statement by statement on a reserved connection under a session-level lock, for statements such as CREATE INDEX CONCURRENTLY.

Migration names are passed as parameters when recording history, including in `db mark-as-migrated` and `db squash`.
@onmax
onmax force-pushed the fix/postgresql-atomic-migrations branch from 783861d to 84444ee Compare November 16, 2026 12:34
@onmax onmax changed the title feat(db): support PostgreSQL migration schemas fix(db): make PostgreSQL migrations atomic and serialized Nov 16, 2026
@onmax

onmax commented Nov 16, 2026

Copy link
Copy Markdown
Contributor Author

thanks for the review! split it: this PR is now only the atomic migrations, rebased on main, with the transaction limits documented and a -- nuxthub:no-transaction opt-out per file. migrationsSchema is on its own branch (no schema creation, history only moves with an explicit CLI command), I will open it once this one lands :)

atinux commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Drafted by an AI agent (Amp) on behalf of @atinux, who reviewed it before posting.

Thanks for splitting it, this reads much better. The -- nuxthub:no-transaction opt-out and the docs cover the transaction limits well, and using try-lock with a retry instead of a blocking lock to avoid deadlocking CREATE INDEX CONCURRENTLY is a nice touch. Good call fixing the SQL interpolation in mark-as-migrated/squash too.

A few notes:

1. This is a breaking change for some PostgreSQL users.
Already-applied migrations don't run again, but a fresh database (new environment, preview DB, CI, a teammate's local setup) replays every file. An existing migration that uses CREATE INDEX CONCURRENTLY, BEGIN/COMMIT, or adds an enum value and uses it in the same file will now fail there. I am wondering if we should ship this as a breaking change

2. Docs advice for existing files.
The docs recommend splitting an enum change into two migrations. That works for new migrations, but splitting an already-applied file creates migration names existing databases have never recorded. Adding -- nuxthub:no-transaction to the old file fixes every case (each statement autocommits, so the new enum value is usable right after), and the name doesn't change. Could the docs point to the directive for files that are already applied?

3. Question about retryWhileLocked.
It retries forever. If another instance holds the lock and gets stuck (not crashed, since a crash releases the session lock), boot hangs with no error. Would a timeout (a few minutes, maybe configurable) with a clear message make sense? Not blocking for me.

Looking forward to the migrationsSchema PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants