Outbox DI-registration hardening + heterogeneous-engine semantics (3.2.1) - #175
Merged
Merged
Conversation
…#15) Under modular / multi-datastore composition the transactional outbox silently stopped persisting durable events (0 rows, no error). WithEventTracking runs at the end of every WithPersistence call and TryAdds the in-memory IEntityEventTracker; when a non-outbox module registered first it pinned the in-memory tracker, so the outbox producer's TryAdd of OutboxEntityEventTracker no-opped and every durable event was dispatched in-process and dropped. AddOutboxProducer now Remove-then-Adds IEntityEventTracker and IEventRouter so the outbox wins regardless of registration order. OutboxEntityEventTracker is a strict superset of the in-memory tracker, so it is always correct for it to win. Adds a modular-composition regression test.
…tracker (#15) The startup diagnostic inspected IEventRouter, which is not what persists durable events (the tracker composes the concrete OutboxEventRouter directly). Because the core ctor pins IEventRouter to the in-memory router unconditionally, the old check false-positived on every working outbox host and missed the real failure. It now inspects IEntityEventTracker and warns only when the effective tracker is the in-memory one.
…hosts persist (#16) A host that ran the outbox poller (AddOutboxProcessor) and also committed domain entities silently dropped durable events, because only AddOutboxProducer registered the outbox tracker/router. Moves the authoritative tracker/router registration (plus its concrete collaborators) into AddOutboxCore, which both the producer and processor call. This also removes any path where the outbox tracker could be bound without its concrete InMemoryEntityEventTracker collaborator, which surfaced as an 'unable to resolve InMemoryEntityEventTracker' DI failure. Producer-only diagnostics stay on the producer path.
…ot shipped (#17) The 3.2.0 changelog listed outbox metrics (AC-18), IOutboxPayloadProtector (AC-19), and the deserialization allow-list (AC-20) under Added, but none shipped. Moves them to an explicit 'Planned — not shipped in 3.2.0' note in the changelog (current + versioned snapshot) with interim workarounds, and marks AC-18/19/20 DEFERRED in the spec (acceptance criteria, observability, security, and open questions) so the spec reflects the actual shipped state.
Adds Examples.EventHandling.Outbox.Modular: three independent bounded-context modules (Ordering, Billing, Shipping), each owning its own datastore and native transactional outbox, composed at a single root. This is the customer's shape (modular composition + multiple datastores + native outbox) that silently dropped durable events before 3.2.1. The runnable Program and the end-to-end test both prove every datastore's durable event persists to its own outbox and nowhere else, regardless of module registration order.
…s Remove-then-Add
Adds an integration test documenting the contract for a unit of work spanning more than one datastore (the heterogeneous-engine production question): - a single UnitOfWork writing two datastores promotes to a distributed transaction and fails loud (TransactionAbortedException / Postgres 55000 prepared transactions disabled) — RCommon gives no cross-datastore atomicity and never half-commits; - the correct pattern is a unit of work PER datastore, each committing locally and atomically with its own co-located outbox row. Modelled with two Postgres databases (any second connection triggers the same promotion; the engine pairing is not the trigger), reproducing the exact error a Postgres + SQL Server deployment reported.
Adds a caution to the outbox recipes guide (live + versioned 3.2.0): an outbox row is atomic with its own datastore only; a unit of work spanning two datastores promotes to a distributed transaction (2PC/MSDTC) and fails loud (e.g. Postgres 55000 prepared transactions disabled). One logical operation touching multiple datastores must use a unit of work per datastore; cross- datastore delivery is eventual via the poller.
… reference - Changelog: new 3.2.1 section (silent-data-loss fixes #15/#16, routing diagnostic correction, modular example, multi-datastore semantics test/note, #17 doc correction). - Migration guide: 'Upgrading to 3.2.1' (no code changes; remove any outbox Replace workaround; multi-datastore transaction-boundary guidance). - Recipes: reference the runnable Examples.EventHandling.Outbox.Modular from the Multiple datastores section. Verified with a clean Docusaurus build (onBrokenLinks: throw).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Behavior-only bug-fix release (3.2.1) that closes three defects a third party hit while adopting the 3.2.0 outbox, plus a worked modular example, an integration test pinning multi-datastore transaction semantics, and documentation. No public API changes — upgrading from 3.2.0 requires no code changes.
All three defects share one theme: the transactional outbox registered its load-bearing services with
TryAdd, so under realistic composition (modular apps, multiple datastores, split producer/processor hosts) the outbox silently lost durable events — no exception, no warning.What's fixed
#15 — silent data loss under modular / multi-datastore composition (High)
WithEventTrackingruns at the end of everyWithPersistencecall andTryAdds the in-memoryIEntityEventTracker. If a non-outbox module (or a second datastore) registered first, it pinned the in-memory tracker and the outbox's ownTryAddno-opped, so every durable event dispatched in-process and was never written to the outbox.Fix: the outbox now registers
IEntityEventTracker/IEventRouterauthoritatively (remove-then-add) in the shared outbox core, so it wins regardless of registration order.OutboxEntityEventTrackeris a strict superset of the in-memory tracker, so it's always correct for it to win.#16 — processor / combined hosts silently dropped durable events (Medium)
Outbox routing was registered only by
AddOutboxProducer; a host that ran the poller (AddOutboxProcessor) and also committed domain entities lost its durable events.Fix: routing moved into the shared core that both
AddOutboxProducerandAddOutboxProcessorcall. This also removes the registration path that could surface as an "unable to resolveInMemoryEntityEventTracker" DI error.#15 diagnostic — routing diagnostic inspected the wrong service
OutboxRoutingDiagnosticsHostedServicecheckedIEventRouter(which the core always binds to the in-memory router), so it false-warned on every working outbox host and missed the real failure. It now inspects the load-bearingIEntityEventTracker.#17 — changelog/spec described unshipped features (docs)
Outbox metrics
Meter,IOutboxPayloadProtector, and the deserialization allow-list (AC-18/19/20) were listed as shipped in the 3.2.0 changelog but never shipped. Moved to an explicit "Planned — not shipped in 3.2.0" note (live + versioned snapshot) with interim workarounds, and marked DEFERRED throughout the spec.Heterogeneous-engine transaction semantics (follow-up from adopter review)
The adopter's production topology spans two engines (Postgres + SQL Server) in one composition and asked whether
UnitOfWork.CommitAsyncforces cross-datastore 2PC. Confirmed empirically and documented:UnitOfWorkwraps a singleTransactionScope(Required), so writing a second datastore in one UoW promotes to a distributed transaction and fails loud (reproduced:TransactionAbortedException→ Postgres55000: prepared transactions are disabled). The trigger is the second connection, not the engine pairing.Added
Examples.EventHandling.Outbox.Modular— three bounded-context modules (Ordering/Billing/Shipping), each its own datastore + native outbox, composed at one root; runnableProgram+ end-to-end test proving every datastore persists regardless of module order.MultiDataStoreUnitOfWorkSemanticsTests(integration) — asserts the single-UoW-across-datastores failure and the per-datastore-scope success path.ReproOutboxModularDiTests(Feature/dapper orm support #15),ProcessorOnlyHostDiTests(Multitenancy Support #16).Verification
CrossDataStoreOutboxTests2/2 ·RecipeTwoBBrokerOutboxTests2/2 · MassTransit spike 2/2 · Wolverine spike 3/3 ·MultiDataStoreUnitOfWorkSemanticsTests2/2.onBrokenLinks: throw).Reviewed and confirmed by the reporting third party.