Conversation
TransactGetItems ran its reads in a READ COMMITTED transaction, where each SELECT takes a fresh snapshot. A TransactWriteItems that committed between two of those reads was observed half-applied: with one writer moving two items to the same generation and four readers, 27 of 3,578 reads returned the items at different generations. Begin the read transaction as REPEATABLE READ READ ONLY so every read is served from the snapshot taken at the first one. On a primary, a REPEATABLE READ transaction that only reads cannot fail with a serialization error (PostgreSQL raises those only when such a transaction modifies a row changed since its snapshot), so no retry path is added. tests/test_transact_get_snapshot.py runs the writer-and-readers probe for five seconds and fails on any torn read; it fails on main and passes with this change. Readiness assessment P0-3. Signed-off-by: Scott Robinson <robinnsc@amazon.com>
robinnsc
requested review from
LeeroyHannigan,
amrith,
c33howard,
jcshepherd,
pdf-amzn and
yesyayen
as code owners
October 5, 2026 02:45
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
transact_get_items_impl()incrates/storage-postgres/src/data/transactions.rsopens its read transaction withBEGIN ISOLATION LEVEL REPEATABLE READ READ ONLYinstead of the pool default (READ COMMITTED). Every item in one TransactGetItems is now read from the snapshot taken at the first read.No retry is added. On a primary, a REPEATABLE READ transaction that only reads cannot fail with a serialization error; PostgreSQL raises those only when such a transaction modifies a row changed since its snapshot. All writes in this backend go through the same data pool, so in a deployment that accepts writes that pool cannot be a hot standby.
SQLite and MongoDB are unchanged. SQLite's read transaction already holds one WAL snapshot. MongoDB reads with snapshot read concern.
Why
No issue filed. Found in the 1.0 readiness review (P0-3).
Under READ COMMITTED every SELECT takes a fresh snapshot, so a TransactWriteItems that commits between two reads of one TransactGetItems is seen half-applied. On a server built from main, with one writer moving two items to the same generation in a single TransactWriteItems and four readers calling TransactGetItems on both, 27 of 3,578 reads returned the two items at different generations. TransactGetItems promises an all-or-nothing view, so a client reading invariant-linked items gets inconsistent data with no error.
Testing done
New
tests/test_transact_get_snapshot.py: the same one-writer, four-reader probe for five seconds. Any read with mismatched generations fails the test. A TransactionCanceledException from a conflicting write is a valid outcome and is not counted. Against a PostgreSQL server built from main it fails 10 runs out of 10 (17 to 36 torn reads per run); against this branch it passes 3 of 3, including alongside the 50-thread contention suite under xdist. It passes on SQLite.The transaction suites (
test_transaction_operations.py,test_transact_expression_validation.py,test_transaction_key_size_validation.py) pass: 55 passed.Full PostgreSQL pytest run (
tests/, import/export excluded as in CI) and the comprehensive suite (tests/python), against servers built from this branch and from main: no test fails on the branch that passes on main, and the comprehensive suite passes 331 of 331 on both. I ran pytest directly rather than throughdevtools/run-tests, so the CLI lifecycle and GSI queue suites, which need the runner's PostgreSQL connection string and server restarts, errored the same way on both builds.Checklist
cargo test --workspace)cargo fmt --check)cargo clippy -- -W clippy::pedantic)Storagetrait, auth model, on-diskformat, or public CLI surface, an RFC has been accepted or is linked
below. Otherwise, an ADR captures the decision (link below).
ADR / RFC: n/a. Isolation level of one backend's read transaction; no wire, trait, auth, on-disk, or CLI change.
Breaking changes
None.
By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache License 2.0 and I agree to the Developer Certificate of
Origin (DCO). See CONTRIBUTING.md for details.