perf: backend-sharding auto measures the whole proxy and becomes the default - #21
Merged
Merged
Conversation
…aster The worker rule (busy 85% and thin local batches) used to move every session on the worker to the shared pipes; a memtier sliding pipeline at 200 connections lost 16% that way, since its local path was not congested and the cross-worker hop cost more than the deeper batches returned. The rule now only starts an experiment: the worker moves its sessions, compares its command rate a second later with the second before, keeps the shared pipes on a 5% gain and otherwise takes the sessions back, waiting 30 s before the next try and doubling that up to 8 min while the answer holds; from the shared pipes it probes the other way on the same schedule. INFO reports pipe_probes, pipe_keeps, pipe_reverts and per-worker worker_shared. Nothing runs on the command path: the tick reads the worker's existing command counter.
…nnections The pause that lets a never-idle session drain and switch existed only for the move to the shared pipes; a session the worker had moved there stayed on them after a losing probe, since its depth never reached zero, and a reverse probe measured the shared path twice. The pause now applies whenever the session should switch. The per-worker INFO field is worker_prefers_shared: it reports the tuner's preference, not lane placement.
A session at score one with a reply outstanding was paused for the score-based move to the shared pipes, then the idle dispatch lowered its score again without a switch, so a lightly pipelined client paid a round trip every other command. The pause now applies to the worker's move to the shared pipes and to a pipelining session's move back, the two switches still wanted once the session has drained; an unpipelined session idles and moves on its own.
… rate The per-worker probe measured a worker's own rate while the cost of the shared pipes is process-wide: a worker moving its few sessions saw its own latency fall and kept, every worker concluded the same, and the proxy ended at the throughput of full sharding (memtier P16 c200: 3.61M against 4.33M without sharding). Each worker's tick now only publishes its busyness and batch depth and mirrors one process-wide preference; the lead worker runs the experiment when a majority of workers is busy with thin batches, moves every session, and compares the sum of all workers' command rates a second later with the second before. INFO reports pipes_shared, the probe counters and worker_busy.
The reverse probe ran on a fixed schedule (30 s, doubling), so after a workload change the proxy stayed on the losing answer until the next scheduled probe: the memtier P16 c200 cell, entered from the P16 c1000 cells on the shared pipes, measured 3.91M against 4.35M local because the flip came part-way through. The rate the current state was chosen on is now remembered, and a rate a quarter away from it ends the wait, so the next probe starts as soon as the trigger holds; with that the base wait is a minute, which keeps the scheduled reverse probe out of a steady workload's way.
After a decision the ring still holds the trial's rate for one window; a trial a quarter slower than the winner read as a workload shift on the next tick, ended the wait and started the same probe again. The shift check now waits one rate window after every decision.
A probe started right after a rate shift took its baseline from the second that held the shift itself, a gap between two benchmark phases in the acceptance runs, and a reverse trial then beat a near-zero baseline: the proxy went local for a whole cell (5.5M against 8.9M). The ten ticks of the baseline must now lie within a factor of two of each other; a gap or a ramp makes the probe wait until the rate has settled. An unsteady trial window needs no guard, since a low reading keeps the previous state.
…ne spreads under a quarter A step smaller than the spread bound left both rates in the baseline second, so a trial compared against a blend and the wrong answer held for the backoff. A detected shift now starts the same settling window a decision does, and the baseline's ticks must lie within a quarter of each other, the same margin the shift check uses.
…nges nothing Every pause between two benchmark phases read as a changed workload and bought a probe on the losing arm as soon as the rate settled: the SET cell ran 4% under full sharding from one two-second local window. The shift is now only noted; once the rate is steady again it is compared with the decided one, and only a rate that settled elsewhere voids the decision. A pause, or a blip, that returns to the same rate keeps the schedule.
A pause longer than the settling window left a steady all-zero ring, which read as a rate settled elsewhere and voided the decision; the returning workload then bought a probe it did not need. A shift is now confirmed only by a steady rate above the floor; while the proxy is idle the shift stays pending.
A rate that settled above the decided one also voided it: the GET cell, entered from the seed phase at half its rate, bought a reverse probe and ran 5% under full sharding. A heavier load batches at least as well on the shared pipes and the scheduled reverse probe still checks it; from the local path the busy trigger already watches for load. Only a rate that settled a quarter under the decided one, on the shared pipes, now ends the wait.
On the 128-master rig with 90-second GET and 60-second memtier cells, auto runs the P16 fan-out lane at 8.98M against 5.54M without sharding and 9.01M with full sharding, the sliding memtier lane at 4.23M against 4.31M and 3.58M, and P1 and the cache cells at parity with the best fixed mode. The 2% in the sliding lane is the experiment itself; the alternatives lose 62% or 17% somewhere. The docs name the old default where their tables were measured with it.
…-nothing claim With a single worker the local pool already holds the one connection per node the shared pipes would add, so auto only bought the fabric and mutex-backed reply queues; it now resolves to no. The threading model, the reply queue's module line and the sample name the setting the shared-nothing guarantee holds under.
… choose The one-worker normalization also rewrote auto where backend-conns spreads a worker's sessions over several connections per node, which the shared pipe consolidates; it now applies only with one connection per node.
…ate, its comments one line each The four cleanup lenses over the branch: must_drain is a filter over switch_pipes (equivalent on every input, one threshold instead of two), the ring's first fill is the same settling window a decision starts, the four rate checks at a probe's start and the two in the shift confirmation are one settled() predicate, the probing phase is an Option instead of an enum, the backoff is stored and doubled instead of derived from a shift, and the module's comments are back to one line each (25 lines from 34). benchmarks.md no longer describes the retired worker rule above the table measured with it, and the configuration row defers the mechanism to architecture.md.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
backend-sharding autonow runs one experiment for the whole proxy and becomes the default.The previous
automoved a worker's sessions to the shared pipes whenever the worker was busy with thin backend batches. That rule fires in every busy shape, including the one where the local path is not congested and the cross-worker hop costs more than the deeper batches return: a memtier sliding pipeline at 200 connections lost 16%.INFOreportspipes_prefer_shared,pipe_probes,pipe_keeps,pipe_revertsand per-workerworker_busy.Measured on a 128-master rig, 64 workers, arms rotated per round. Four rounds of 20-second cells: P16 1000 conns GET
no5.58M /auto8.70M /yes8.89M, SET 5.53 / 8.99 / 8.94, P1 2000 conns 2.18 / 2.62 / 2.62, memtier P16 200 conns 4.40 / 4.28 / 3.58, cache 7.77 vs auto+cache 8.30. Three rounds of long cells: 90-second GETno5.54M /auto8.98M /yes9.01M, 60-second memtier 4.31 / 4.23 / 3.58. Auto sits within the noise floor of the best fixed mode in the fan-out lane and within 2% in the sliding lane, where the cost is the experiment itself; the fixed modes lose 62% or 17% somewhere, soautois the default.docs/benchmarks.mdmarks its older tables as measured underno.Gates: unit 117; the 15-cell compatibility matrix green (its default cells now pin
no, so every mode stays covered); ct1 IT ×8 green, functional suite 165 / 165 passed. Review converged.