Repository navigation
Wait for a writable MongoDB single-member primary - #21439
David Fowler (davidfowl) wants to merge 1 commit into
Conversation
A newly elected primary reports replSetGetStatus myState 1 while it is still in drain mode and rejects writes with NotWritablePrimary. Gate single-member initialization and health on hello.isWritablePrimary instead. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 21439Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 21439" |
Tests selector4 / 100 PR test projects · 2 PR jobs, from 4 changed files. Selected PR test projects (4 / 100)
Selected PR jobs (2)
How these were chosen — grouped by what changed🔧 📦 affected project 🔧 🧪 🧪 Job reasons
Selection computed for commit |
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues were identified for the MongoDB replica-set readiness change.
No open issues 🎉
What changed in this PR
Fixes a readiness race in Aspire’s single-member MongoDB replica sets that could allow writes before the primary was writable.
Changes:
- Uses
hellofor initialization and health checks, requiring the expected replica-set name, a writable primary, and exactly one host. - Updates unit coverage for drain mode and missing replica-set metadata.
- Adds a functional regression test that writes immediately after initialization.
| File | Description |
|---|---|
| tests/Aspire.Hosting.MongoDB.Tests/SingleMemberReplicaSetTests.cs | Tests the writable-primary readiness predicate. |
| tests/Aspire.Hosting.MongoDB.Tests/SingleMemberReplicaSetFunctionalTests.cs | Adds an immediate-write initialization regression test. |
| src/Aspire.Hosting.MongoDB/MongoDBSingleMemberReplicaSet.cs | Waits for writable-primary readiness using hello. |
| src/Aspire.Hosting.MongoDB/MongoDBBuilderExtensions.cs | Applies the same readiness predicate to health checks. |
Review effort: Balanced
|
Closing in favor of #21638, which makes the same product fix: init and the health check both use
#21638 is also the PR currently going through paired CI validation. 🤖 Posted by AspireAgent (AI Agent) |
Description
Fixes an intermittent
MongoNotPrimaryException : Server returned not primary errorinSingleMemberReplicaSetFunctionalTestson Ubuntu (TransactionsAndChangeStreamsWorkWithoutADeveloperCertificateandTransactionsAndChangeStreamsWorkWithTls). The same race can affect real AppHosts that useWithReplicaSet().Root cause.
MongoDBSingleMemberReplicaSettreated the resource as ready oncereplSetGetStatus.myState == 1. A newly elected primary reportsPRIMARYwhile it is still in drain mode, and it rejects writes withNotWritablePrimaryuntil it logs "Transition to primary complete; database writes are now permitted". The resource could become healthy inside that window, so the test's first write failed. In the CI failure,replSetInitiateran at 06:57:31.520, the "is the primary" message was logged at 31.550,ordersturned healthy and the next write failed with not-primary.Fix. Initialization and the health check now use
hello. Readiness requiressetNameto match,isWritablePrimary == trueand exactly one host. The configuration validation and initiation flow are unchanged. No assertions were weakened and no sleeps or retries were added.Evidence
--replSet, polling right afterreplSetInitiate):myState: 1withisWritablePrimary: false, and an immediatecreateCollectionreturnedNotWritablePrimaryin 3/3 runs.InitializationWaitsUntilThePrimaryAcceptsWritesruns initialization, then writes immediately:MongoNotPrimaryException.ReadinessRequiresTheSingleMemberWritablePrimaryandReadinessRequiresAReplicaSetMembercover the predicate, including the drain-mode case (isWritablePrimary: false).ci.yml, Oct 16–30):Hosting.MongoDBUbuntu jobs, of which 27 failed.MongoDbReplicaSetFunctionalTestsfailures already tracked elsewhere (Retry MongoDB replica-set configuration during member recovery #20796) or whole-job outages. This PR does not touch them.Limitations
MongoExpressConnectsWithTls,TransactionsAndChangeStreamsWorkWithTlsandDataAndReplicaSetIdentitySurviveNewAppHostsAndPortsfail both with and without this change. CI covers them.Checklist
<remarks />and<code />elements on your triple slash comments?🤖 Posted by AspireAgent (AI Agent)