Repository navigation
Reshape LocalDiscovery to a single-owner model — Closes #106, #300, #433, #435, #436, #437 - #377
Merged
conradbzura merged 8 commits intoSep 17, 2026
Conversation
conradbzura
force-pushed
the
300-single-owner-local-discovery
branch
2 times, most recently
from
September 15, 2026 19:07
06dc075 to
07de281
Compare
This was referenced Sep 15, 2026
conradbzura
force-pushed
the
300-single-owner-local-discovery
branch
3 times, most recently
from
September 16, 2026 21:30
b867c6a to
20291ce
Compare
A single-type expect now types the yielded object as that type, so a caller that validates a dependency against a protocol receives it typed as that protocol. A tuple expect keeps the dependency's own type.
Entering LocalDiscovery creates a namespace's shared-memory registry and claims it, and exiting reclaims it. Publishers and subscribers borrow the registry, and a borrower that outlives its owner is orphaned. An exclusive create decides ownership, so no check-then-create race can hand one namespace to two owners. DiscoveryNamespaceInUse and DiscoveryNamespaceNotFound report the two failed claims and join the public API. BREAKING CHANGE: Entering LocalDiscovery on a namespace whose registry already exists raises DiscoveryNamespaceInUse, and a LocalDiscovery publisher or subscriber on a namespace without a registry raises DiscoveryNamespaceNotFound.
Each discovery exception passes its structured fields positionally to its base and renders its message in __str__. The worker's exception serializer rebuilds an unpicklable exception as cls(*exc.args), which previously restored the message in place of the fields. BREAKING CHANGE: args on DiscoveryCapacityExhausted, DiscoveryBlockExhausted and DiscoveryWorkerNotFound holds the exception's fields in place of its message, and the messages name the discovery registry.
SubscriberMeta returns a shared subscription in place of an instance of the subscriber class, a substitution no type checker observes. Each subscriber class declares a checker-only __new__ with its constructor's parameters and the DiscoverySubscriberLike return type.
WorkerProxy iterates its discovery subscriber and never resolves or enters it. A pool URI proxy borrows the named LocalDiscovery namespace through a shared subscriber, so it claims nothing. The proxy validates discovery at construction, and reducing a proxy that holds a context-manager subscriber raises TypeError in the dispatching process. BREAKING CHANGE: WorkerProxy's discovery parameter takes only a DiscoverySubscriberLike; a Factory, or any other value, raises TypeError at construction.
A durable pool publishes nothing, so a DiscoverySubscriberLike suffices as its discovery. Passing LocalDiscovery.Subscriber borrows a namespace another process owns.
The integration suite borrows owned namespaces through subscribers and covers cross-process ownership, borrowing, orphaning and teardown. Retried bodies mint their namespaces and events per attempt, so a retry starts from clean state.
The READMEs name the owner, borrower and orphan roles, show how a durable pool borrows a local namespace, and defer the contract to the LocalDiscovery docstring.
conradbzura
force-pushed
the
300-single-owner-local-discovery
branch
from
September 17, 2026 14:07
20291ce to
94c07ff
Compare
conradbzura
marked this pull request as ready for review
September 17, 2026 14:33
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Give each
LocalDiscoverynamespace exactly one owner. The owner is the entered instance whose entry created the namespace's shared-memory registry. Publishers and subscribers borrow that registry, and a borrower that outlives its owner is orphaned. Entry decides ownership through an exclusive create, which removes the race in which whichever process created a registry first held it and every later entrant attached without knowing its role.The model reaches the worker runtime in two places. A durable
WorkerPoolborrows a namespace through a bareDiscoverySubscriberLike(#106), andWorkerProxyconsumes its discovery subscriber as given, which leaves the pool as the sole owner of discovery's lifetime (#433). Three changes the model exposed ride along: discovery exceptions carry their fields inargs(#437),resolvednarrows its result for a single-typeexpect(#435), and subscriber construction is typed as the protocol it returns (#436).Closes #106
Closes #300
Closes #433
Closes #435
Closes #436
Closes #437
Scope
DiscoveryNamespaceNotFoundonly once its quorum wait expires. This PR pins the outcome; #376 owns the latency__exit__still leaves the lock and notification files behindcreate=Trueconstruction that fails partway still emits the stray tracker unregisterDiscoveryLikewhose publisher and subscriber both borrow_subscriber_factoriesretains one closure per namespace for the life of the process, which predates this changeloadbalancerstays out of this PRMigration
LocalDiscovery(ns)on an existing registry attaches to itDiscoveryNamespaceInUseLocalDiscovery.Subscriber(ns)orLocalDiscovery.Publisher(ns)WorkerPool(discovery=LocalDiscovery(ns))withoutspawnattaches to the namespaceDiscoveryNamespaceInUsewhere a registry existsLocalDiscovery.Subscriber(ns)LocalDiscoverynamespaceDiscoveryNamespaceInUseFileNotFoundErrorDiscoveryNamespaceNotFound, aWoolErrorDiscoveryNamespaceNotFoundDiscoveryCapacityExhausted,DiscoveryBlockExhaustedandDiscoveryWorkerNotFoundcarry their message inargsargs, and the first and last name the discovery registry in their messagestr(error)WorkerProxy(discovery=...)accepts anyFactoryformDiscoverySubscriberLikeand raisesTypeErrorat construction for anything elseWorkerPool, and pass the subscriberProposed changes
Claim a namespace on entry
LocalDiscoveryDiscoveryNamespaceInUseLocalDiscovery.Publisher,LocalDiscovery.SubscriberDiscoveryNamespaceNotFoundEntry creates the registry with
create=Trueand translatesFileExistsError; the attach fallback and the owner flag go. The claim belongs to entry, the one operation with a matching__exit__, sopublisher,subscriberandsubscribereturn borrowers whether or not their instance is entered. Borrowers bind through_registry, which translates only a missing registry, and each binds before it acquires anything else, so a failed bind leaves nothing to unwind.No entry adopts a stranded registry. A liveness check cannot be made atomic against a slow owner, so
DiscoveryNamespaceInUsenames the backing segment and leaves its removal to the caller. TheLocalDiscoverydocstring holds the ownership, borrowing and orphaning contract.Carry discovery exception fields in
argsEach discovery exception passes its fields positionally to
WoolError.__init__and renders its message in__str__. The worker's exception serializer rebuilds an unpicklable exception ascls(*exc.args), which bound the rendered message to the first field.DiscoveryNamespaceInUseandDiscoveryNamespaceNotFoundfollow the same convention and joinwool.__all__.Type a subscriber construction as the protocol it returns
SubscriberMetareturns a_SharedSubscriptionin place of an instance of the subscriber class.LocalDiscovery.SubscriberandLanDiscovery.Subscriberdeclare a checker-only__new__with their constructor's parameters and theDiscoverySubscriberLikereturn type, so a checker types construction as what the caller receives and still checks its arguments.LocalDiscovery.Subscriberdrops its__reduce__andnamespacemembers, which no caller could reach through the returned object.Consume a discovery subscriber as given in
WorkerProxyWorkerProxyiterates its discovery subscriber and never resolves or enters it. The pool had resolveddiscoveryand the proxy resolved it again, entering a context-manager subscriber twice. The proxy validatesdiscoveryat construction, and__wool_reduce__raisesTypeErrorin the dispatching process for a context-manager subscriber. A pool-URI proxy borrows throughafilter(tag_filter, LocalDiscovery.Subscriber(pool_uri))and claims nothing.A proxy constructed directly from a factory that produces an unpicklable subscriber re-ran that factory in the worker; it now raises at construction. No pool arm passed a factory down.
Accept a bare subscriber as discovery for a durable pool
A durable pool publishes nothing, so it accepts a
DiscoverySubscriberLikeas well as aDiscoveryLike. The durable arm classifies the resolved object once,DiscoveryLikefirst, since a runtime protocol check on Python 3.11 evaluates the service's properties. Rejection raisesTypeErrorat context entry inside the resolved block, so a rejected context-manager factory is exited. #106 proposed aValueErrorat construction, whichFactoryforms preclude because they resolve only on entry.Narrow
resolved's return type for a single-typeexpectOverloads type the result of a single-type
expectas that type. A tupleexpect, or none, keeps the dependency's type.Exercise single-owner discovery across processes
PoolMode.DURABLE_JOINEDbecomesDURABLE_BORROWED, which supplies a bare subscriber in eachFactoryform and asserts after resolution that the pool receives noDiscoveryLike. Pairs thatallpairspystarves out of the covering array are pinned from one tuple. The suite scopes the subscriber pool and factory registry to each test, as the unit suite does, and retried bodies mint their state per attempt. New suites cover cross-process ownership, borrowing, orphaning and teardown.The rework exposed four defects in existing tests:
_enter_lifecycle_forestsuppressed every claim below the root, so both Hypothesis forest tests passed against attach-on-entrypublish, the raise pool teardown depends on(pool mode, discovery)pairs reached the covering array only when Hypothesis drew themDescribe single-owner local discovery in the READMEs
The READMEs name the owner, borrower and orphan roles, show how a durable pool borrows a local namespace, and defer the contract to the
LocalDiscoverydocstring.Test cases
TestLocalDiscoveryLocalDiscoveryis entered and exitedTestLocalDiscoveryLocalDiscoveryis enteredDiscoveryNamespaceInUsenaming the segment and leaves the owner's registry intactTestLocalDiscoveryRuntimeErrorTestLocalDiscoveryTestLocalDiscoveryDiscoveryNamespaceNotFoundand the successor's registry serves a new workerTestLocalDiscoveryPublisherPublisheris entered or publishesDiscoveryNamespaceNotFoundTestLocalDiscoveryPublisherTestLocalDiscoveryPublisherDiscoveryCapacityExhaustedTestLocalDiscoveryPublisher,TestLocalDiscoverySubscribercapacityTypeErrorTestLocalDiscoverySubscriberSubscriberis iteratedDiscoveryNamespaceNotFoundand creates no registry or notification directoryTestLocalDiscoverySubscriberTestLocalDiscoverySubscriberTestLocalDiscoverySubscriberSubscriberMetaDiscoverySubscriberLikeand its reduction carries itspoll_intervalTestDiscoveryExceptionReconstruction__cause__TestWorkerPoolTestWorkerPoolDiscoveryLikeTypeErrornaming the accepted protocols and exits any context it enteredTestWorkerPoolTestWorkerProxydiscoveryTypeErrorTestWorkerProxyTypeErrornamingdiscoveryTestWorkerProxyDiscoveryNamespaceNotFoundnaming the namespaceTestWorkerProxyTestCrossProcessOwnershipTestCrossProcessOwnershipTestPoolNamespaceOwnershipDiscoveryNamespaceInUse, orDiscoveryNamespaceNotFound, and leaves the namespace usableTestCrossProcessBorrowingTestOverlappingNamespaceLifecyclesNoWorkersAvailablewithin ten seconds and a new owner serves the namespaceTestCrossProcessTrackerTestCrossProcessLockTimeout,TestWorkerOrphanPreventionTimeoutError, and the worker is reapedTestPoolComposition,test_integrationDURABLE_BORROWEDscenario in each supply form, and any drawable scenario