Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,7 @@ ever commits a change — every other command is read-only or fully offline.

| Command | Live database | What the connection is used for |
|---|---|---|
| `migrate` | required | Resolve the target table, preflight it (privileges, partitioning, size and catalog facts), classify and route the change, then **execute** the routed SQL under bounded budgets. A change routed to copy-and-swap also takes `--free-disk`, and optionally `--quiesced` (fence writes instead of streaming them) and `--restart SCHEMA.TABLE` (start a stopped run over); see [running a copy-and-swap](docs/copy-and-swap.md) |
| `migrate` | required | Resolve the target table, preflight it (privileges, partitioning, size and catalog facts), classify and route the change, then **execute** the routed SQL under bounded budgets. A change routed to copy-and-swap also takes `--free-disk`, and optionally `--block-writes SCHEMA.TABLE` (block the table's writes instead of streaming them, offline mode) and `--restart SCHEMA.TABLE` (start a stopped run over); see [running a copy-and-swap](docs/copy-and-swap.md) |
| `migrate --dry-run` | required | The same introspection as a real run — server version, target resolution, table facts — so the printed plan reflects the actual target; executes nothing |
| `diff` | required | Introspect the live table (read-only) and materialize the desired-state file on a scratch schema inside a transaction that is always rolled back; prints the plan, changes nothing |
| [`pull`](docs/pull.md) | required | Introspect each supported table in a schema and create one desired-state file per table; existing files are never overwritten, and a zero-change `diff` verifies the baseline |
Expand Down
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Aurora-only. Why that combination is the product is [vision.md](vision.md); star
| [lock-budgeted-passthrough.md](lock-budgeted-passthrough.md) | The **lock-budgeted passthrough contract** — the explicit `--accept-blocking` acknowledgement, typed refusal eligibility, required engine-owned lock and statement budgets, marked verdict and exit code, interruption semantics, and orchestrator boundary. |
| [plan-report.md](plan-report.md) | The **plan report contract** — the versioned JSON shape both front doors emit for dry-run plans: fields, closed vocabularies, the fingerprint identity, required consumer behavior for unknown versions/values, and one generated example per source (pinned by test). |
| [pull.md](pull.md) | **Export an existing schema** — use `pull` to create one desired-state file per table, then prove the baseline round-trips with a zero-change `diff`. |
| [copy-and-swap.md](copy-and-swap.md) | **Running a copy-and-swap** — the operator's guide: live or quiesced and how to choose, what each run needs, `--free-disk`, and how to stop, resume, and start over. |
| [copy-and-swap.md](copy-and-swap.md) | **Running a copy-and-swap** — the operator's guide: live or offline (`--block-writes`) and how to choose, what each run needs, `--free-disk`, and how to stop, resume, and start over. |
| [cli-output-examples.md](cli-output-examples.md) | **CLI output examples** — one real, captured JSON output per shape the CLI produces: the plan report for every dry-run disposition (execute, safer-sequence substitution, copy-and-swap, rewrite-required, backend-unavailable, refusal, destructive), the execution verdicts, exit codes, the linter, and diff. |
| [safer-sequences.md](safer-sequences.md) | **Safer-sequence substitution** (the improve path) — how the planner replaces a native-but-blocking form with the ordered online sequence that reaches the same end state: a worked `ADD CONSTRAINT … UNIQUE` comparison (locking, failure modes, transactionality, cost), what the engine adds over running the idiom by hand, the substitutions made today, and the typed caveats. |
| [execution-model.md](execution-model.md) | The **execution model** — why safer sequences run autocommit-each-step with no wrapping transaction (PostgreSQL forbids it for the online forms), the **committed prefix** a mid-sequence failure leaves, how the verdict reports the boundary, and the per-sequence partial-failure contracts with their retry paths. Read this to answer "if a multi-step change fails halfway, what state is my table in?" |
Expand Down
8 changes: 4 additions & 4 deletions docs/cli-output-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -391,10 +391,10 @@ $ pg-sprite migrate --alter 'ALTER TABLE users ALTER COLUMN id TYPE text' --free
}
```

With `--quiesced`, the run fences writes to the table instead of streaming them through a
replication slot, needs no logical decoding, and `detail` says the table was quiesced. When
to choose it, and an example of its output, are in
[copy-and-swap.md](copy-and-swap.md#live-or-quiesced).
With `--block-writes SCHEMA.TABLE`, the run fences writes to the table instead of streaming
them through a replication slot, needs no logical decoding, and `detail` says writes to the
table were blocked. When to choose it, and an example of its output, are in
[copy-and-swap.md](copy-and-swap.md#live-or-offline).

### Refused: the engine does not carry the operation yet (`backend-unavailable`) — exit 2

Expand Down
34 changes: 19 additions & 15 deletions docs/copy-and-swap.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,15 @@ and how to stop, resume, and start over. The design is in

## Contents

- [Live or quiesced](#live-or-quiesced)
- [Live or offline](#live-or-offline)
- [What each run needs](#what-each-run-needs)
- [Free disk space](#free-disk-space)
- [Run it](#run-it)
- [Stop, resume, and start over](#stop-resume-and-start-over)

## Live or quiesced
## Live or offline

| | Live (default) | Quiesced (`--quiesced`) |
| | Live (default) | Offline (`--block-writes SCHEMA.TABLE`) |
| --- | --- | --- |
| Application writes during the run | Keep working. pg-sprite reads them from the WAL through a logical replication slot and replays them onto the shadow. | Refused. pg-sprite fences the table with a trigger, and every write fails with SQLSTATE `55006` until the swap commits. |
| Server requirements | `wal_level = logical`, a free replication slot, and two free WAL senders | None beyond the copy itself |
Expand All @@ -30,17 +30,20 @@ and how to stop, resume, and start over. The design is in
**Choose live** for a table the application writes to in normal operation, on a server with
logical decoding enabled.

**Choose quiesced** when writes to the table are already stopped or can be stopped for the run:
**Choose offline** when writes to the table are already stopped or can be stopped for the run:

- archive, history, or reference tables nothing writes to;
- a maintenance window, or a feature flag that pauses the table's writer;
- a server without logical decoding. On Aurora and RDS that needs
`rds.logical_replication = 1`, a parameter that takes a reboot.

`--quiesced` is your declaration that nothing writes the table. pg-sprite does not take it on
trust: the fence refuses any write that does arrive, so nothing is lost, but the application
sees errors for the whole run. The fence and why it also covers replication apply workers are
described in [engine-role.md](engine-role.md#the-write-fence).
`--block-writes` is your declaration that nothing needs to write the table. Its value must name
the table the statement changes, exactly as `SCHEMA.TABLE`; a value naming any other table
stops the run before anything executes, so a flag left in a script cannot take the next
table offline. pg-sprite does not take the declaration on trust: the fence refuses any write
that does arrive, so nothing is lost, but the application sees errors for the whole run.
The fence and why it also covers replication apply workers are described in
[engine-role.md](engine-role.md#the-write-fence).

## What each run needs

Expand Down Expand Up @@ -69,7 +72,8 @@ $ pg-sprite migrate --free-disk 1GiB --json \
}
```

Either enable logical decoding, or run with `--quiesced` if the table can take no writes.
Either enable logical decoding, or run with `--block-writes SCHEMA.TABLE` if the table can go
without writes for the run.

## Free disk space

Expand Down Expand Up @@ -101,8 +105,8 @@ pg-sprite migrate --dry-run \
pg-sprite migrate --free-disk 50GiB \
--alter 'ALTER TABLE public.orders ALTER COLUMN amount TYPE numeric(12,2)'

# Quiesced: nothing writes the table for the length of the run
pg-sprite migrate --free-disk 50GiB --quiesced \
# Offline: the table's writes are blocked for the length of the run
pg-sprite migrate --free-disk 50GiB --block-writes public.order_archive \
--alter 'ALTER TABLE public.order_archive ALTER COLUMN amount TYPE numeric(12,2)'
```

Expand All @@ -115,7 +119,7 @@ A successful run prints one verdict, `executed-via-copy-and-swap`, and exits 0.
"outcome": "executed-via-copy-and-swap",
"statement": "ALTER TABLE public.order_archive ALTER COLUMN amount TYPE numeric(12,2)",
"table": "public.order_archive",
"detail": "verified 5000 rows in 3 chunks; the retained source is dropped; the table was quiesced: writes were fenced for the length of the run"
"detail": "verified 5000 rows in 3 chunks; the retained source is dropped; writes to the table were blocked for the length of the run"
}
```

Expand All @@ -126,13 +130,13 @@ name for a rollback window; you drop it yourself when the window ends.

| You want to | Do this |
| --- | --- |
| Stop a running change | Press Ctrl-C, or send `SIGTERM`, once. The run saves where its copy reached and exits. The table keeps its fence (quiesced) or its replication slot (live) until you resume or start over. |
| Resume it | Run the same command again. It continues from where it stopped; nothing is copied twice. |
| Stop a running change | Press Ctrl-C, or send `SIGTERM`, once. The run saves where its copy reached and exits. The table keeps its fence (offline) or its replication slot (live) until you resume or start over. |
| Resume it | Run the same command again, including `--block-writes SCHEMA.TABLE` if the run blocked writes. It continues from where it stopped; nothing is copied twice. |
| Start over | Run the same command with `--restart SCHEMA.TABLE`, naming the table exactly. pg-sprite discards the shadow, the fence or slot, and the saved progress, then runs from the beginning. |
| Recover from bad data | When the server rejects a row under the new type or a constraint, the run reports `code: data-fault` and tries to abandon itself. Follow the verdict's detail: if it says the run was abandoned, nothing is left behind, so fix the data or the statement and run it again. If it says the run could not be abandoned, the shadow and the fence or slot remain, so fix the data or the statement and run it with `--restart SCHEMA.TABLE`. |

```sh
pg-sprite migrate --free-disk 50GiB --quiesced --restart public.order_archive \
pg-sprite migrate --free-disk 50GiB --block-writes public.order_archive --restart public.order_archive \
--alter 'ALTER TABLE public.order_archive ALTER COLUMN amount TYPE numeric(12,2)'
```

Expand Down
9 changes: 5 additions & 4 deletions docs/engine-role.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,9 +95,10 @@ not every start.

### The write fence

A quiesced copy-and-swap run (`pkg/schemachange.Runner`) does not take the caller's word that
nothing writes the table: it installs a statement-level `BEFORE INSERT OR UPDATE OR DELETE OR
TRUNCATE` trigger, `pgsprite_write_fence`, on the source before its first checkpoint save, and
A copy-and-swap run that blocks writes (`--block-writes`; a quiesced target to
`pkg/schemachange.Runner`) does not take the caller's word that nothing writes the table: it
installs a statement-level `BEFORE INSERT OR UPDATE OR DELETE OR TRUNCATE` trigger,
`pgsprite_write_fence`, on the source before its first checkpoint save, and
every write the trigger sees is refused with SQLSTATE `55006` (`object_in_use`) and a message
naming the table. The trigger is enabled `ALWAYS`, so it fires for a session running under
`session_replication_role = replica` as well. The writers that run in that role are the apply
Expand Down Expand Up @@ -134,7 +135,7 @@ static parameter requiring a reboot), one free slot under `max_replication_slots
senders under `max_wal_senders` — the run creates the slot on one replication connection,
closes it once the checkpoint row naming the slot commits, and later decodes on a connection of
its own, so it holds one sender at a time and the second is headroom. Both apply only to a run
that decodes WAL; a quiesced run skips them. The same check compares the free disk the
that decodes WAL; a run that blocks writes skips them. The same check compares the free disk the
caller measured on the database volume — PostgreSQL has no function that reports it —
against the shadow copy's headroom, and
refuses an unmeasured volume rather than assuming it is large enough. The figure is the
Expand Down
Loading
Loading