tests: stabilize asynchronous next-gen tests

[rleungx]

What problem does this PR solve?

Issue Number: Close #10353, Close #11229, Close #11233

TestHotRegionStorage can read history before the asynchronous flush has
persisted the complete batch. Client integration tests can also observe the
old leader immediately after a resignation or fail a one-shot embedded-etcd
leader transfer under CI load.

TestTSOKeyspaceGroupManagerSuite/TestMultiNodes can also race with the
automatic split finisher and time out after the split has already completed.

What is changed and how does it work?

Wait for the complete hot-region history batch before asserting its contents.

Reuse the bounded leader-resignation retry and wait until all running servers
agree on a leader different from the previous one.

Make the split-finishing test helper accept a concurrent completion only
after verifying that the target group is no longer splitting.

Check List

Tests

• Unit test
• Integration test

Release note

None.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: 16eb218f-bd4d-453e-88b5-249f51b5ff2d

📥 Commits

Reviewing files that changed from the base of the PR and between addfeb0 and b146426.

📒 Files selected for processing (3)

• tests/cluster.go
• tests/integrations/client/client_test.go
• tests/server/storage/hot_region_storage_test.go

Included review availability: Your plan provides up to 4 included reviews per hour; 1 remains after this review.

---

📝 Walkthrough

Walkthrough

The test utilities now wait for a leader different from a specified previous leader. Client integration tests use retrying resignation. Hot-region storage tests collect complete write and read histories before validation.

Changes

Leader transfer test reliability

Layer / File(s)	Summary
Leader-change waiting and integration coverage tests/cluster.go, tests/integrations/client/client_test.go	TestCluster adds WaitLeaderChange and excludes the previous leader during polling. Client tests use ResignLeaderWithRetry and wait for the new leader.

Hot-region storage test reliability

Layer / File(s)	Summary
History loading and validation tests/server/storage/hot_region_storage_test.go	The test collects write and read history records through loadHistoryHotRegions and validates both categories after loading.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Other · Severity of issue fixed: Low

Suggested reviewers: bufferflies

Merge Risk: ⚪ Minimal · up to b1464

The updated tests wait for complete history and a changed leader before asserting results, with no remaining actionable merge risk identified.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 3 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The changes satisfy the coding requirements in [#10353] and [#11229]. TestHotRegionStorage now polls until both write and read history contain two records, then validates all records. `ResignLeaderW…
Out of Scope Changes check	✅ Passed	The changed files support the linked flaky-test objectives. The iterator helper supports complete asynchronous history verification. WaitLeaderChange and retry reuse support leader-transfer stabiliz…
Title check	✅ Passed	The title clearly identifies test stabilization as the main change and uses a concise package-scoped format.
Description check	✅ Passed	The description explains the affected flaky tests, lists the linked issues, summarizes the implementation, identifies unit and integration tests, and includes the release note. The optional Side effec…

• Fix all pre-merge checks with AI

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[ti-chi-bot]

@coderabbitai[bot]: adding LGTM is restricted to approvers and reviewers in OWNERS files.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[rleungx]

/retest

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.83%. Comparing base (2d43fe9) to head (aa1cc23).
⚠️ Report is 6 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #11230      +/-   ##
==========================================
+ Coverage   79.79%   79.83%   +0.03%     
==========================================
  Files         544      545       +1     
  Lines       78879    78891      +12     
==========================================
+ Hits        62942    62981      +39     
+ Misses      11570    11550      -20     
+ Partials     4367     4360       -7     

Flag	Coverage Δ
unittests	79.83% <ø> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bufferflies, coderabbitai[bot]

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [bufferflies]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2026-09-11 11:47:23.723215622 +0000 UTC m=+27266.580996793: ☑️ agreed by bufferflies.