fix(market-maker): correct the realized-volatility estimate inflating spreads - #75
Merged
Merged
Conversation
… spreads Quoted spreads were far wider than intended because sigma was overstated by two independent bugs. Decimal mismatch in the backfill. OracleTracker seeded its window from the hashprice subgraph (8 decimals) and then appended live on-chain answers (6 decimals), so a single ln(1/100) return dominated the window. The history source now declares the aggregator address and decimals it is reporting, and OracleTracker reconciles that against the live oracle before rebasing; a mismatched aggregator skips the backfill instead of poisoning sigma. This replaces the previous magnitude-based guess at the scale factor. Wrong estimator for an irregularly sampled feed. volatilityPerSecond divided each log return by sqrt(dt), which is only valid if the feed samples a Brownian process at times unrelated to its moves. These oracles publish on a deviation threshold, so |r| is essentially independent of elapsed time (measured corr(log dt, log|r|) = -0.13 over a stable day, -0.11 over 96h) and dividing by sqrt(dt) injects a spurious 1/sqrt(dt) term. It read ~1.74x high in every period sampled. Replaced with the realized-variance estimator sqrt(sum r^2 / sum dt), which is unbiased under threshold publication and far more robust to bursts: rolling sigma spread over 96h was 20x for the old form versus 4x for the new one. Verified against production in dry run, where sigma fell from 69 to 41 bps over a 1h horizon. Also in this change: - Gate vault deposits on dryRun. CollateralTracker.maybeTopUp() submitted real deposits even in dry-run mode. - Replace oracle.historyLookbackMultiplier with historyMaxAgeSec. The old knob scaled the backfill window by pollInterval, which has no bearing on how often the oracle updates. Backfill now requests exactly windowSize samples and uses the age bound only to keep a stale regime out of sigma. - Move shared config documentation from the YAML files into the TypeBox schema descriptions so hover docs reach every environment, leaving only env-specific rationale inline. Corrects a stale claim that hashprice updates every ~5 min (measured median is ~3 min) and drops a comment describing a tx-weight field that no longer exists. - Consolidate the three copies of fractionToNumber into math.ts and use it in oracleTracker, which was calling Fraction.valueOf() directly and logging null for sigma. Sigma is now reported at info level on backfill. - Finish the migration to unified portfolio.*.yml configs: drop the per-app perps/futures configs and the stg variants, retire the matching package scripts, and replace the two per-app smoke tests with one portfolio test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The market maker was quoting a much wider spread than the config implied. Both causes turned out to be in the volatility estimate feeding
volatilityMultiplier.The two bugs
Decimal mismatch in the backfill.
OracleTrackerseeded its rolling window from the hashprice subgraph (8 decimals) and then appended live on-chain answers (6 decimals). A singleln(1/100)return dominated the window and sigma was meaningless.The history source now declares which aggregator it is reporting and at what precision, and
OracleTrackerreconciles that against the live oracle before rebasing. A mismatched aggregator skips the backfill rather than poisoning sigma. This replaces an earlier heuristic that guessed the scale factor from the magnitude of the numbers.Wrong estimator for an irregularly sampled feed.
volatilityPerSeconddivided each log return bysqrt(dt). That is correct only if the feed samples a Brownian process at times unrelated to its moves. These oracles publish on a deviation threshold instead, so the size of a move is essentially independent of how long it took:With
abs(r)independent ofdt, dividing bysqrt(dt)does not remove a time effect, it injects one —corr(log dt, log abs x)comes out at −0.48, almost exactly the −0.5 you would get from the division alone. Sigma was partly measuring update cadence.It read about 1.74× high in every period I sampled. Replaced with the realized-variance estimator
sqrt(sum r^2 / sum dt), the standard form for irregularly spaced observations, which is unbiased under threshold publication and much more robust to bursts:sd(r/sqrt(dt))sqrt(sum r^2/sum dt)Verification
Run against production config in dry run: backfill pulled 60 samples spanning 4.1h,
decimalShift: 2confirming the reconciliation is active, and sigma settled at 6.807e-5/s — about 41 bps over 1h, down from 69.math.test.tsgains a regression test that replays one price path twice, once evenly spaced and once with a compressed burst in the middle. The old estimator inflates sigma 6.6× on the bursty version; the new one moves 1%.Also in this PR
dryRun.CollateralTracker.maybeTopUp()submitted real on-chain deposits even in dry-run mode. This is the one item here worth reviewing as a safety fix rather than a tuning change.oracle.historyLookbackMultiplierwithhistoryMaxAgeSec. The old knob scaled the backfill window bypollInterval, which has nothing to do with how often the oracle updates. Backfill now asks for exactlywindowSizesamples and uses the age bound only to keep a stale regime out of sigma.fractionToNumberintomath.ts. Three copies existed to guard againstFraction.valueOf()overflowing to NaN on 1000-bit rationals;oracleTrackerlacked it and was loggingnullfor sigma. Sigma is now logged at info level when backfill completes.Note for reviewers
Spreads will tighten noticeably once this lands — roughly 40% from the estimator alone, on top of the decimal fix. Worth re-tuning
volatilityMultiplieragainst the corrected scale before or shortly after deploy rather than discovering the new width in production.Separately,
portfolio.dev.ymlsizes futures at 120/side against a perps 10/side. That predates this PR and I left it alone, but it looks unintentional.Test plan
npx tsc --noEmitcleanbiome lintclean (21 pre-existing warnings, none in changed files)