Skip to content

fix(postgres): read TransactGetItems from one snapshot - #382

Open
robinnsc wants to merge 1 commit into
mainfrom
fix/pg-transact-get-snapshot
Open

robinnsc wants to merge 1 commit into
mainfrom
fix/pg-transact-get-snapshot

Conversation

@robinnsc

@robinnsc robinnsc commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator

What

transact_get_items_impl() in crates/storage-postgres/src/data/transactions.rs opens its read transaction with BEGIN ISOLATION LEVEL REPEATABLE READ READ ONLY instead of the pool default (READ COMMITTED). Every item in one TransactGetItems is now read from the snapshot taken at the first read.

No retry is added. On a primary, a REPEATABLE READ transaction that only reads cannot fail with a serialization error; PostgreSQL raises those only when such a transaction modifies a row changed since its snapshot. All writes in this backend go through the same data pool, so in a deployment that accepts writes that pool cannot be a hot standby.

SQLite and MongoDB are unchanged. SQLite's read transaction already holds one WAL snapshot. MongoDB reads with snapshot read concern.

Why

No issue filed. Found in the 1.0 readiness review (P0-3).

Under READ COMMITTED every SELECT takes a fresh snapshot, so a TransactWriteItems that commits between two reads of one TransactGetItems is seen half-applied. On a server built from main, with one writer moving two items to the same generation in a single TransactWriteItems and four readers calling TransactGetItems on both, 27 of 3,578 reads returned the two items at different generations. TransactGetItems promises an all-or-nothing view, so a client reading invariant-linked items gets inconsistent data with no error.

Testing done

New tests/test_transact_get_snapshot.py: the same one-writer, four-reader probe for five seconds. Any read with mismatched generations fails the test. A TransactionCanceledException from a conflicting write is a valid outcome and is not counted. Against a PostgreSQL server built from main it fails 10 runs out of 10 (17 to 36 torn reads per run); against this branch it passes 3 of 3, including alongside the 50-thread contention suite under xdist. It passes on SQLite.

The transaction suites (test_transaction_operations.py, test_transact_expression_validation.py, test_transaction_key_size_validation.py) pass: 55 passed.

Full PostgreSQL pytest run (tests/, import/export excluded as in CI) and the comprehensive suite (tests/python), against servers built from this branch and from main: no test fails on the branch that passes on main, and the comprehensive suite passes 331 of 331 on both. I ran pytest directly rather than through devtools/run-tests, so the CLI lifecycle and GSI queue suites, which need the runner's PostgreSQL connection string and server restarts, errored the same way on both builds.

cargo fmt --all -- --check
cargo +1.97.0 clippy --all-targets -- -D warnings   # the CI toolchain
cargo test --workspace                       # 1,222 passed
cargo +1.88.0 check --workspace --locked

Checklist

  • I have read CONTRIBUTING.md
  • All tests pass (cargo test --workspace)
  • Code is formatted (cargo fmt --check)
  • Clippy is clean (cargo clippy -- -W clippy::pedantic)
  • I have added or updated tests for new functionality
  • I have updated documentation if behavior changed
  • Breaking changes are noted below (if any)
  • If this changes the wire protocol, Storage trait, auth model, on-disk
    format, or public CLI surface, an RFC has been accepted or is linked
    below. Otherwise, an ADR captures the decision (link below).

ADR / RFC: n/a. Isolation level of one backend's read transaction; no wire, trait, auth, on-disk, or CLI change.

Breaking changes

None.


By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache License 2.0 and I agree to the Developer Certificate of
Origin (DCO). See CONTRIBUTING.md for details.

TransactGetItems ran its reads in a READ COMMITTED transaction, where each
SELECT takes a fresh snapshot. A TransactWriteItems that committed between
two of those reads was observed half-applied: with one writer moving two
items to the same generation and four readers, 27 of 3,578 reads returned
the items at different generations.

Begin the read transaction as REPEATABLE READ READ ONLY so every read is
served from the snapshot taken at the first one. On a primary, a
REPEATABLE READ transaction that only reads cannot fail with a
serialization error (PostgreSQL raises those only when such a transaction
modifies a row changed since its snapshot), so no retry path is added.

tests/test_transact_get_snapshot.py runs the writer-and-readers probe for
five seconds and fails on any torn read; it fails on main and passes with
this change.

Readiness assessment P0-3.

Signed-off-by: Scott Robinson <robinnsc@amazon.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant