Skip to content

Add TopicClient.reset_offset to rewind a consumer's committed offsets. - #907

Open
nshestakov wants to merge 2 commits into
ydb-platform:mainfrom
nshestakov:reset-offset
Open

nshestakov wants to merge 2 commits into
ydb-platform:mainfrom
nshestakov:reset-offset

Conversation

@nshestakov

@nshestakov nshestakov commented Sep 28, 2026 •

Copy link
Copy Markdown

Pull request type

Please check the type of change your PR introduces:

  • Bugfix
  • Feature
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • Documentation content changes
  • Other (please describe):

What is the current behavior?

The topic client can commit an offset for one partition (commit_offset) but cannot rewind a consumer's committed offsets. Python client for TopicService.ResetOffset, already available in the C++ SDK (ydb-platform/ydb#54149).

Issue Number: N/A

What is the new behavior?

TopicClient.reset_offset / TopicClientAsyncIO.reset_offset rewind a consumer's committed offsets on every topic partition, including inactive partitions left after a split or merge.

driver.topic_client.reset_offset(path, consumer, to=ydb.TopicResetOffset.EARLIEST)
driver.topic_client.reset_offset(path, consumer, to=ydb.TopicResetOffset.LATEST)
driver.topic_client.reset_offset(path, consumer, to=written_at)  # datetime

A datetime selects the first message whose write timestamp is greater than or equal to that time. If no such message exists, the partition end offset is used.

Partitions are updated independently: the call is not atomic, and a failure may still leave some partitions already rewritten. Any active read session of this consumer is dropped. The call is retried on cancellation the same way as commit_offset.

Other information

Stubs for protobuf 3–6 were regenerated with ResetOffset. ydb-api-protos does not contain this RPC yet, so the submodule pointer is unchanged; the proto additions used for generation are not part of this commit. The next regeneration from the current pin will drop ResetOffset until that change lands in ydb-api-protos.

The RPC moves every partition, including inactive ones after split or merge, to earliest, latest, or the first message written at a given time.

Co-authored-by: Cursor <cursoragent@cursor.com>
@codecov

codecov Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.14286% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 82.70%. Comparing base (87d561e) to head (ed11f9a).

Files with missing lines Patch % Lines
ydb/_grpc/grpcwrapper/ydb_topic.py 95.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #907      +/-   ##
==========================================
+ Coverage   82.66%   82.70%   +0.03%     
==========================================
  Files          99       99              
  Lines       12953    12987      +34     
  Branches     1268     1273       +5     
==========================================
+ Hits        10708    10741      +33     
  Misses       1794     1794              
- Partials      451      452       +1     
Flag Coverage Δ
integration 80.48% <91.42%> (+0.02%) ⬆️
unit 49.14% <80.00%> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
ydb/_apis.py 100.00% <100.00%> (ø)
ydb/_grpc/grpcwrapper/ydb_topic_public_types.py 100.00% <100.00%> (ø)
ydb/topic.py 86.58% <100.00%> (+0.56%) ⬆️
ydb/_grpc/grpcwrapper/ydb_topic.py 86.51% <95.00%> (+0.18%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Build the protobuf timestamp from a known datetime so CopyFrom is not given an optional value.

Co-authored-by: Cursor <cursoragent@cursor.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant