Skip to content

orts 2.4.0: ContextualLogisticBandit, OR-TS over arm-by-cell contrasts - #17

Merged
sulgik merged 1 commit into
mainfrom
claude/growthbook-discussion-analysis-0be9b7
Oct 1, 2026
Merged

sulgik merged 1 commit into
mainfrom
claude/growthbook-discussion-analysis-0be9b7

Conversation

@sulgik

@sulgik sulgik commented Oct 1, 2026

Copy link
Copy Markdown
Owner

What

  • ContextualLogisticBandit (experimental, not in the paper): OR-TS over discrete context cells. One logistic fit per batch with a fresh flat intercept per cell, the joint posterior of the arm-by-cell contrasts as the carried state, and a hierarchical prior tying the cells together.
  • interaction_sd, the prior's one parameter, is re-estimated after each batch by marginal likelihood unless a number is given. No fixed value did well both when the best arm differs by cell and when it does not.
  • With one cell the class reproduces LogisticBandit.
  • benchmarks/growthbook/ctxbench.py runs it against GrowthBook's own contextual engine (packages/stats-ts), bundled from source and driven over stdin by gb_contextual_driver.ts.
  • Version 2.4.0, CHANGELOG and README.

Benchmark

Cumulative regret, 4 arms, 6 cells, 40 periods of 20,000 users, 20 paired runs.

best arm differs by cell one best arm everywhere differs, common shifts one best arm, common shifts
Contextual OR-TS 819.7 336.9 820.6 364.6
GrowthBook contextual (as shipped) 1221.8 526.1 1453.3 768.2
OR-TS, independent per cell 798.7 728.0 846.6 691.3
OR-TS, context ignored 3221.8 295.7 3384.1 364.9

Tests

11 new tests in tests/test_contextual.py; 105 pass locally on Python 3.11.

🤖 Generated with Claude Code

One logistic fit per batch with a fresh flat intercept per cell, the joint
posterior of the arm-by-cell contrasts as the carried state, and a
hierarchical prior tying the cells together. Its interaction_sd is
re-estimated after each batch by marginal likelihood unless a number is
given; no fixed value did well both when the best arm differs by cell and
when it does not. With one cell the class reproduces LogisticBandit.
Experimental and not in the paper.

benchmarks/growthbook/ctxbench.py runs it against GrowthBook's own
contextual engine (packages/stats-ts), bundled from source and driven over
stdin by gb_contextual_driver.ts. gb_row now takes the arm count from its
input so both benchmarks can share it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@sulgik
sulgik merged commit ec34bf6 into main Oct 1, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant