You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As described in #86, ForceRefreshKeys can block requests forever: the request goroutine waits on an unbounded channel handshake with the refresh goroutine, and JWKS fetches use a default http.Client with no timeout, so a hanging JWKS endpoint stalls requests indefinitely.
There is a second related failure mode: Traefik constructs a separate plugin instance per router chain referencing the middleware, and rebuilds them all on every dynamic configuration reload. The current code cancels "the previous" background refresh goroutine by plugin name, which kills the refresher of a sibling instance that is still serving — freezing its cached JWKS keys.
Changes
HTTPS timeouts: new JwksFetchTimeoutSecs config option bounds each JWKS request, and (with ForceRefreshKeys) bounds how long a request waits for a forced refresh before continuing with the currently cached keys. Defaults to 0 (no timeout) to preserve existing behavior.
Shared key registry: key material and the background refresh goroutine now live in a keyRegistry shared by every plugin instance built from the same configuration (keyed on plugin name + keys + JWKS headers + timeout), so one refresh goroutine serves all routes and config reloads no longer orphan live instances.
Tested this branch (a089f09) on a real GKE cluster, Traefik v3.6.15, kubernetesCRD provider — side by side with catalog v0.10.0 in the same Traefik process (v0.10.0 via experimental.plugins, this branch via experimental.localPlugins; same module name coexists fine). Middleware config: Required: true, ForceRefreshKeys: true, two Keycloak JWKS URLs + one static PEM key, JwksFetchTimeoutSecs: 5 on this branch. Two IngressRoutes referencing each middleware (two plugin instances per middleware), requests authenticated with an RS256 token carrying no kid header, so the refresh path fires on every request.
Results:
v0.10.0: the route served by the instance whose background refresher was cancelled by its sibling's construction hangs indefinitely (client timeout, reproducible with just 2 routes referencing the same middleware). It stays broken across dynamic config reloads. Matches Requests hangs indefinitely in forceRefreshKeys #86.
This branch: all routes answer 200 in ~115 ms — no measurable overhead vs the healthy v0.10.0 instance — before and after dynamic config reloads.
Fixes both failure modes for us. Would love to see this merged and released.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #86
Problem
As described in #86,
ForceRefreshKeyscan block requests forever: the request goroutine waits on an unbounded channel handshake with the refresh goroutine, and JWKS fetches use a defaulthttp.Clientwith no timeout, so a hanging JWKS endpoint stalls requests indefinitely.There is a second related failure mode: Traefik constructs a separate plugin instance per router chain referencing the middleware, and rebuilds them all on every dynamic configuration reload. The current code cancels "the previous" background refresh goroutine by plugin name, which kills the refresher of a sibling instance that is still serving — freezing its cached JWKS keys.
Changes
JwksFetchTimeoutSecsconfig option bounds each JWKS request, and (withForceRefreshKeys) bounds how long a request waits for a forced refresh before continuing with the currently cached keys. Defaults to0(no timeout) to preserve existing behavior.keyRegistryshared by every plugin instance built from the same configuration (keyed on plugin name + keys + JWKS headers + timeout), so one refresh goroutine serves all routes and config reloads no longer orphan live instances.