Repository navigation
analysis: re-trim tempogram latency to center the grid-phase peak - #43
Merged
Merged
Conversation
The windowed phase estimator turned out to be start-anchored: the band-norm EMA warms up from zero, which inflates the first second of flux frames ~100x, and with the amplitude-cubed window weighting the phase then comes almost entirely from the track's first strong onsets. An A/B against a library of imported rekordbox grids shows this is load-bearing, not a bug: seeding the EMA honestly drops <50ms grid agreement by 12 points and grows the half-beat tail, and an explicit exponential early-window decay cannot reproduce the win either, so the anchor lives in the inflated first onset frames, not in any window preference. Both experiments stay behind new default-off knobs (BandNormWarmupSec, PhaseEarlyTauSec) whose comments warn the warmup must not be "fixed", and a new env-gated TestWarmupDiag measures the window-0 dominance directly. Rekordbox itself normalizes its onset novelty with the same kind of one-pole EMA, so its phase is plausibly start-anchored by the very same accident. The calibration that WAS wrong: TempogramLatencyMs was centered on the mean error, but the well-aligned population sits ~9ms early while a long late tail drags the mean positive. Re-trimming 30 -> 34ms centers the error distribution's peak instead, taking <10ms grid agreement from 23.9% to 34.3% with <20ms, <50ms and the half-beat tail unchanged (tuned and held-out validated on disjoint slices; latency shifts can now be swept offline from a VYNULL_BEAT_REF_DUMP CSV). cacheVersion bumps to 28 so existing libraries re-analyze onto the corrected phase.
loadFromDisk discarded (and deleted) any stale-version cache entry unconditionally, GridEdited or not - every cacheVersion bump so far has silently wiped all hand-fixed beat grids and re-analyzed those tracks back to the detector's wrong phase. Found while preparing the v29 bump, which would have destroyed a grid fixed minutes earlier. A stale entry carrying GridEdited now stashes its beats/BPM/downbeat (and keeps the file on disk, so the edit also survives a restart that happens before re-analysis); Set re-applies the stash onto the fresh result - new waveforms, the user's grid - and regenerates the grid blobs from the carried beats. TestGridEditSurvivesCacheBump pins the miss-but-keep-file behavior, the re-application on Set, and persistence across a store restart.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
This started as a bug hunt and ended as a calibration fix. The band-norm EMA in the multiband onset warms up from zero, which inflates the first second of flux frames roughly 100x, and with the amplitude-cubed window weighting the grid phase effectively comes from the track's first strong onsets alone.
Removing that "bug" makes everything worse. Seeding the EMA from the track's real leading level drops <50ms grid agreement with rekordbox by 12 points and grows the half-beat tail, and replacing it with an explicit exponential early-window decay is worse at every time constant. The anchor lives in the inflated first onset frames themselves, not in any preference for early windows, and rekordbox normalizes its own onset novelty with the same kind of one-pole EMA, so its phase is plausibly start-anchored by the very same accident. The warmup stays, and the two refuted experiments stay behind default-off knobs (
BandNormWarmupSec,PhaseEarlyTauSec) whose comments warn future readers away from "fixing" it. A new env-gatedTestWarmupDiagmeasures the window-0 dominance directly.What WAS wrong is the latency constant.
TempogramLatencyMshad been centered on the mean phase error, but the error distribution is asymmetric: the well-aligned population sits about 9ms early while a long late tail drags the mean positive. Re-trimming 30 -> 34ms centers the distribution's peak instead of its mean, which takes <10ms grid agreement with rekordbox from 23.9% to 34.3% while <20ms, <50ms and the half-beat tail stay unchanged. The trim was tuned and held-out validated on disjoint slices of a library of imported rekordbox grids, then confirmed with a full run, and the accuracy harness gained env knobs (VYNULL_BEAT_REF_{WARMUP,EARLYTAU}) plus offline latency sweeping from aVYNULL_BEAT_REF_DUMPCSV.cacheVersionbumps to 29 (28 went to the PWV4 clamp that merged first) so existing libraries re-analyze once and every grid picks up the corrected phase.Preparing that bump surfaced a long-standing bug it would have triggered: the cache loader discarded (and deleted) any stale-version entry unconditionally, so every cacheVersion bump since v26 has silently wiped user-edited beat grids and re-analyzed those tracks back to the detector's phase. A stale entry carrying GridEdited now stashes its beats/BPM/downbeat and keeps its file until re-analysis lands; the fresh result gets the user's grid re-applied (new waveforms, hand-fixed grid). TestGridEditSurvivesCacheBump pins the behavior, including persistence across a restart.
Hardware testing
No protocol, load-path, or serve-side change, but this DOES change analysis output that decks consume: every beat grid shifts +4ms and the cacheVersion bump re-analyzes existing libraries on upgrade. A hardware smoke (load a track, check the grid/beat indicator looks sane, one re-analyzed track after upgrade) is the honest bar here.
Checklist
go build ./...,go vet ./..., andgo test ./...passgofmt -l .is cleanGPL-3.0-or-later)