Skip to content

v4: per-command cache serve/store flags, and an external-storage policy mode - #3354

Merged
mgravell merged 1 commit into
v4from
marc/v4-cache-command-flags
Nov 13, 2026
Merged

mgravell merged 1 commit into
v4from
marc/v4-cache-command-flags

Conversation

@mgravell

Copy link
Copy Markdown
Member

Implements the HybridCache-side spec for per-command local-cache serve/store control, plus the external-cache-layer scenario discussed alongside it — with one deliberate split, below.

Two new public CommandFlags (bits 23–24, per the spec)

  • CacheNoLocalServe (1 << 23) — always a round trip. No probe, and no attaching to an identical request already in flight (that reply answers a question asked before this one). Everything else stands: the reply is still eligible for admission, and a RemovalHandler still arms on the fill — including when the store loses to a resident still-valid entry, which is first-fill-wins as for any racing fill.
  • CacheNoLocalStore (1 << 24) — an entry already present may be served (serving is not storing; the serve re-arms as usual), but the reply begins no fill at all: no store, no removal enrollment, and no single-flight leadership — a waiter attached to a reply that will never be stored would wake, miss, and send anyway, having gained only the wait.
  • Together: full non-participation for that call. Both are inert on a non-participating command, CacheOptOut included (the reserved-pair rejection stays OptIn|OptOut only). Neither has a wire half in any tracking mode. Both are in UserSelectableFlags, and the bits adjoin in HC's read-then-write order, so the HC side's branch-free (CommandFlags)(((int)flags & 0b11) << 23) mapping holds.

The external layer is a policy, not the flag pair

The spec pins CacheNoLocalStore as "nothing stored ⇒ no enrollment arises", which is right for HC-on-CSC — and exactly backwards for the external-layer scenario (FusionCache, the IDistributedCache-based HybridCache), which is enrollment without storage. Overloading one flag with both meanings would poison each, so the read-coupled external shape ("Shape A") lives where the coupling already does:

var external = db.Context.WithCachePolicy(new CachePolicy
{
    Storage = CacheStorage.External,
    RemovalHandler = static (key, reason) => EvictFromOuterCache(key),
});
  • The fill runs and refuses only the store: keys registered and tracked, tracked stays true, the handler arms — the same path that already arms on a raced or too-large fill, because the server is tracking either way.
  • The read implies CacheOptIn (unless the call said CacheOptOut): a read whose purpose is enrollment must not be silently dropped by the admission doorkeeper or an opt-in mode's default-off. Under TrackingOptIn/TrackingByPrefix it therefore goes out behind CLIENT CACHING YES as it must.
  • No in-flight registration is made for such a fill (new TryBeginFill overload takes the policy).
  • Entries other callers stored may still be served to it — current per the same invalidation stream — and the serve re-arms. LocalWrite arrives distinctly: the one removal whose replacement value the outer layer already has.
  • On CacheOptions.DefaultPolicy it is connection-wide. Per-call flags still win: CacheNoLocalStore skips the fill (and the enrollment) for that one command.
  • New counter: RefusedExternal.

The broadcast-shaped integration ("Shape B": a raw connection-level invalidation stream over CLIENT TRACKING BCAST, no read coupling) is not in this PR — it is a CacheOptions-level surface of its own, and can land independently.

The ToString() trap, as foretold

Declaring the new members beside the bit 20–22 cluster flipped which of the duplicate-valued names (DemandSlave/DemandReplica) ToString() picks — precisely the perturbation the enum's warning comment describes, caught by FormatTests.CommandFlagsFormatting. Fixed by declaration position: a brute-force check of every insertion point (scratch program over all 22 candidate layouts) shows positions 5–9 and end-of-enum keep the expected formatting on net8.0 and net10.0; they are declared at the end, with a pointer comment left in the bit cluster. netfx formatting is asserted by the same test on Windows CI.

Verification

  • 12 new tests (RespCacheParticipationTests): warm/cold serve- and store-disabled behaviour, both-flags non-participation, inertness alongside CacheOptOut, external mode enrolling from the first read under default (doorkeeper) admission, re-arm on re-read, serving from others' entries, LocalWrite, connection-wide default policy, TrackingOptIn mode, and the flag-beats-policy case.
  • Full cache/tracking/stale/format sweep: 430/430 on net10.0; formatting also green on net8.0. Library builds clean on all six TFMs (PublicAPI updated).
  • docs/ClientSideCaching.md: the two flags in the per-call section and the mode table's follow-up paragraph, a Storage row in the policy table, and a new "Feeding an external cache layer" section with the re-fetch-not-deleted caveat (the server sheds its bounded tracking table by invalidating).

CacheNoLocalServe (bit 23): always a round trip - no probe, no attaching to
an in-flight twin - while the reply stays eligible to store and a removal
handler still arms. CacheNoLocalStore (bit 24): serve if present, but the
reply begins no fill at all - no store, no enrollment, no single-flight
leadership for waiters to lose time on. Together: full non-participation
for that call. Both inert on non-participating commands, both in
UserSelectableFlags, no wire half in any tracking mode.

The external cache layer (FusionCache, IDistributedCache-based HybridCache)
wants the OPPOSITE of the store flag - nothing kept, everything watched -
and overloading one flag with both meanings would poison each, so that is
a policy instead: CachePolicy.Storage = CacheStorage.External. Fills run
and refuse only the store (tracked stays true, the handler arms), the read
implies CacheOptIn so the doorkeeper and the opt-in modes cannot silently
drop an enrollment, and no in-flight registration is made since a waiter
would wake to find nothing. Serving from entries others stored stays: it
is current per the same invalidation stream, and the serve re-arms.

Declaring the new members beside the bit 20-22 cluster flipped which of
the duplicate-valued names (DemandSlave/DemandReplica) ToString() picks -
the exact trap the enum's warning comment describes - so they are declared
at the end, a position verified against the formatting expectations on
net8.0 and net10.0 by brute force over all insertion points.
@mgravell
mgravell merged commit 7b33eea into v4 Nov 13, 2026
1 of 3 checks passed
@mgravell
mgravell deleted the marc/v4-cache-command-flags branch November 13, 2026 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant