Conversation
… the millisecond Shard ids were `shardId-<table name>-<n>`, while `stream_shards` is keyed by shard id alone. Any second stream-enabled table under one name collided: - PostgreSQL keeps a deleted table's shards, so DeleteTable followed by CreateTable with the same name and a stream failed with InternalServerError every time. - On PostgreSQL and SQLite, a second account creating a stream-enabled table with a name another account already uses failed the same way. - A table name over about 40 characters produced a ShardId longer than the 65 characters the AWS SDKs accept, so DescribeStream and GetShardIterator calls carrying it were rejected client-side. Use the table id (a fresh UUID per table) in place of the name on PostgreSQL and SQLite, as the MongoDB backend already does. New shard ids are 61 characters. Shard rows that already exist keep their ids and keep working; shards created from now on, including on a pre-existing table that enables a stream for the first time, get the new form. With recreate working, a second defect becomes reachable: stream labels had one-second resolution on all three backends, so a table deleted and recreated within the same second got the same stream ARN, and the old ARN resolved to the new table's stream. Labels now carry milliseconds (`2026-10-05T01:54:50.312`), the shape the service uses. Existing labels are unchanged. On MongoDB, where recreate already worked, this reproduced in one of three runs of the new test. PostgreSQL still never removes a deleted table's shard rows (four per stream). That is unchanged by this commit. tests/test_stream_table_reuse.py covers recreate under the same name (including that the old stream ARN no longer resolves), two accounts with one name, a long name, and disabling then re-enabling a stream. On main the first three fail on PostgreSQL and the second and third fail on SQLite; all pass on PostgreSQL, SQLite, and MongoDB with this change. Readiness assessment P0-5. Signed-off-by: Scott Robinson <robinnsc@amazon.com>
robinnsc
requested review from
LeeroyHannigan,
amrith,
c33howard,
jcshepherd,
pdf-amzn and
yesyayen
as code owners
October 5, 2026 02:45
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Stream shard ids on PostgreSQL and SQLite are built from the table id instead of the table name, and stream labels on all three backends carry milliseconds.
crates/storage-postgres/src/stream_engine.rs,crates/storage-sqlite/src/stream.rs:shardId-{table_id}-{i:016}in place ofshardId-{table_name}-{i:016}. The table id is a UUID, so new shard ids are 61 characters. MongoDB already builds its shard ids this way.stream_labelis nowYYYY-MM-DDThh:mm:ss.sss, the shape the service uses:to_char(clock_timestamp(), 'YYYY-MM-DD"T"HH24:MI:SS.MS')on PostgreSQL (stream_engine.rs,update_table.rs),strftime('%Y-%m-%dT%H:%M:%f','now')on SQLite (stream.rs,update_table.rs), andformat_stream_labelon MongoDB (table_engine.rs).docs/manuals/02-design-guide.md(shard id format),docs/design/13-storage-mongodb.md(label format).Why
No issue filed. Found in the 1.0 readiness review (P0-5).
stream_shardsis keyed by shard id alone, and the id contained only the table name, so any second stream-enabled table under one name collided:duplicate key value violates unique constraint "stream_shards_pkey"). Delete-and-recreate under one name is routine in test harnesses and works in DynamoDB.With recreate working, a second defect became reachable. Labels had one-second resolution, so a table deleted and recreated within the same second (control_plane_delay_seconds of 0 makes this easy) got the same stream ARN as the old table, and DescribeStream on the old ARN resolved to the new table's stream. On MongoDB, where recreate already worked, the new test hit this in 1 of 3 runs on main.
Testing done
New
tests/test_stream_table_reuse.py:On main, the first three fail on PostgreSQL and the second and third fail on SQLite. On this branch all four pass on PostgreSQL (control_plane_delay_seconds 0, 0.05, and 0.25), SQLite, and MongoDB, together with
test_streams.py(20 passed, 1 xfailed on each).The file is ExtendDB-only (module skip when
EXTENDDB_TEST_ENDPOINTis unset), liketest_streams.py: it signs Streams requests against the ExtendDB endpoint, and the two-account case needs the management API.Full PostgreSQL pytest run (
tests/, import/export excluded as in CI) and the comprehensive suite (tests/python), against servers built from this branch and from main: no test fails on the branch that passes on main, and the comprehensive suite passes 331 of 331 on both. I ran pytest directly rather than throughdevtools/run-tests, so the CLI lifecycle and GSI queue suites, which need the runner's PostgreSQL connection string and server restarts, errored the same way on both builds.Not changed here: PostgreSQL never removes a deleted table's shard rows (four per stream). They no longer block anything. Removing them safely needs a delete marker in the data database, because the catalog and the data database are separate and a sweep that infers "deleted" from a missing catalog row races CreateTable.
Checklist
cargo test --workspace)cargo fmt --check)cargo clippy -- -W clippy::pedantic)Storagetrait, auth model, on-diskformat, or public CLI surface, an RFC has been accepted or is linked
below. Otherwise, an ADR captures the decision (link below).
ADR / RFC: n/a. Shard ids and stream labels are opaque server-issued values; existing rows are left as they are, and no trait, schema, or CLI change.
Breaking changes
New streams get ShardIds and StreamLabels in a different shape. Both are opaque in the DynamoDB API, and the new label shape is the service's own. A client that parsed the table name out of a ShardId, or expected a label with no fractional seconds, would see the difference. Existing streams are unchanged.
By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache License 2.0 and I agree to the Developer Certificate of
Origin (DCO). See CONTRIBUTING.md for details.