Skip to content

Interleave rollout requests across groups - #81

Open
nightlessbaron wants to merge 1 commit into
prodfrom
agent/stratified-rollout-dispatch
Open

nightlessbaron wants to merge 1 commit into
prodfrom
agent/stratified-rollout-dispatch

Conversation

@nightlessbaron

Copy link
Copy Markdown

What

  • Dispatch each candidate batch in a seeded, stratified order: shuffle groups and member positions, then submit one member per group per pass.
  • Apply --rolling-start-size to exact sample-level waves instead of truncating the batch to whole groups.
  • Reassemble each group in its original member order before group RM, filtering, tail cancellation, and partial-rollout handling.
  • Preserve deterministic sampling seeds by original within-group position.

Why

The existing rolling start submits complete prompt groups together. With 256 groups × 16 samples and --rolling-start-size 256, that clusters all 16 siblings from a small set of groups into the same burst. This change produces 16 waves of 256 requests, with one sample from every group in each wave and 15 intervals between waves.

Impact

Only the inference rollout scheduler changes. No arguments, configuration, or test files are added.

Validation

  • 256 × 16 behavioral check: 4,096 unique samples, 16 waves, one sample per group per wave, deterministic ordering.
  • Existing agentic rollout tests: 9 passed together; the socket-sensitive concurrency case passed separately.
  • Ruff, Python byte-compilation, and git diff --check passed.

@nightlessbaron
nightlessbaron marked this pull request as ready for review August 28, 2026 17:32
@nightlessbaron
nightlessbaron requested a review from a team August 28, 2026 17:32
@nightlessbaron
nightlessbaron force-pushed the agent/stratified-rollout-dispatch branch from 39d1af8 to 3725498 Compare August 28, 2026 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant