Skip to content

docs(rfc): cap how many attended chat turns run at once - #28633

Merged
iamwhatever merged 1 commit into
mainfrom
docs/rfc-attended-turn-cap
Nov 7, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
docs/rfc-attended-turn-cap

Conversation

@jeeshofone

@jeeshofone jeeshofone commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Part of #25055

Problem / Motivation

Goal: Main records a design for one cap on how many attended chat turns run at once, at the dispatch every send and Resume reaches, before any code for it is written.

Nothing limits concurrent attended dashboard turns today. The cold-start semaphore paces boots only, and the background-turn cap counts unattended turns only. Resuming many sessions, including with the bulk Resume in #27512, leaves every one of their turns running at once. The maintainers parked #25055 and asked for a design through docs/request-for-change.

Why it matters

A bulk Resume of tens of sessions holds tens of model streams, MCP servers and tool processes together, which falls on host memory and model rate limits.

Not a goal

  • Any code change. The implementation follows in its own PR once this design is accepted.
  • Changing the unattended cap, cold-start pacing or the admission gate.

What changed (motivation → approach → change)

Adds docs/request-for-change/rfc-attended-turn-cap.md (status draft) and its row in the index in docs/request-for-change/README.md. The document covers the problem, alternatives including doing nothing, the recommended design (an attended-turn semaphore beside _bg_turn_sema, taken in the guarded wrapper of _run_chat around gate_turn so no call site changes and none can bypass it, kept while a turn waits on a person (a tool approval or question card) and while it waits on SessionManager._start_sem, that queues and never rejects and serves typed sends before bulk Resumes), how it relates to #27512, a migration plan and a test plan.

Backwards compatibility

Compatible: documentation only.

Tests

N/A: documentation only. scripts/docs_lint.py runs in Source Gates.

Manual verification

N/A: documentation only.

Screenshots / video

N/A: no UI change.

@jeeshofone
jeeshofone requested review from a team and KiroCrewCaptain as code owners November 2, 2026 22:25
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 2, 2026
@jeeshofone
jeeshofone force-pushed the docs/rfc-attended-turn-cap branch from b756599 to 7cbc47c Compare November 2, 2026 23:07
@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 76a21288cb89c43bee7a3f6a1b1618f8285ad226 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 76a21288cb89c43bee7a3f6a1b1618f8285ad226 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 76a21288cb89c43bee7a3f6a1b1618f8285ad226 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

This adds a draft RFC and its index row, with no code and no take-away, and the seam it names (guarded → gate_turn in chat_runner.py) is on main.

Notes

  • §1.2 and §1.2a let a turn keep its permit while it waits on gate_turn admission and on a person's approval. With a small cap, N parked turns can block every new typed send for up to 1800 s. Say this cost plainly next to the priority order in §1.3, since priority does not help a send that is stuck behind parked turns.
  • §1.2's bounded wait counts against the spawn_guarded_turn ceiling but has no outer ceiling on the other paths. This gives two different failure messages for the same queue state, so pick one user-visible outcome in the implementation PR.

[DESIGN-REVIEWED] 76a2128

Description read

[DESCRIPTION-READ] 1a7adaaa19555e495e11720ef65dbddd0238066911d9d9d5ce7efbfcb3837999

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 76a21288cb89c43bee7a3f6a1b1618f8285ad226 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 76a2128

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 76a21288cb89c43bee7a3f6a1b1618f8285ad226 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 76a21288cb89c43bee7a3f6a1b1618f8285ad226 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 76a2128

Verdict parsed from the review's SHA-scoped output markers for commit 76a21288cb89c43bee7a3f6a1b1618f8285ad226.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 76a21288cb89c43bee7a3f6a1b1618f8285ad226 false-positive: <reason, 20 to 500 characters>

@jeeshofone
jeeshofone force-pushed the docs/rfc-attended-turn-cap branch from 7cbc47c to 599012d Compare November 2, 2026 23:58
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 3, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Hold: merge conflict. Reviewed at 599012d.

GitHub reports this PR as conflicting with main. The conflict is in docs/request-for-change/README.md, where newer RFC rows landed next to the row this PR adds. Rebase onto main and keep both rows; the new head will need another review.

The RFC itself reads as a sound draft. The two Design Review notes are clarity points: §1.2a already states the re-acquire is queued at priority 0, and the auto default is an open question. Saying in §1.2 that a turn waiting on _start_sem holds a permit would answer the second note. Checks are green.

— crew-pr-reviewer (r143)

@jeeshofone
jeeshofone force-pushed the docs/rfc-attended-turn-cap branch from 599012d to 69e34a5 Compare November 3, 2026 10:02
@jeeshofone

Copy link
Copy Markdown
Contributor Author

Rebased onto main (69e34a5), both index rows kept. §1.2 now says a turn waiting on SessionManager._start_sem keeps its attended permit, since that wait is on the system, not a person.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: passed Eligible automated validation passed for the current revision readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 3, 2026
@jeeshofone
jeeshofone force-pushed the docs/rfc-attended-turn-cap branch from 69e34a5 to 29aaf66 Compare November 3, 2026 12:00
@jeeshofone

Copy link
Copy Markdown
Contributor Author

New head 29aaf66 answers the Opus 5.5 block on 69e34a5. The permit is now taken in _run_chat's guard beside gate_turn, so direct producers (regenerate, rewind, slash commands, TaskRunner, OpenAI-compatible, Slack inbound) are covered. The nesting with fair admission is stated, with an alternatives row and open question 3. The wait is bounded at 1800 s with a named message, and a turn keeps its permit while it waits on a person.

@jeeshofone
jeeshofone force-pushed the docs/rfc-attended-turn-cap branch from 73436bb to c9fa177 Compare November 4, 2026 04:24
@jeeshofone

Copy link
Copy Markdown
Contributor Author

New head c9fa177 answers the Opus 5.5 block on 73436bb. The ceiling sentence now applies only to spawn_guarded_turn producers; on the enqueue_or_run_prompt and direct _run_chat paths the 1800 s permit bound is the only limit. A slot waiting on a permit carries an awaiting_attended_permit flag, reads as queued, holds steers until the turn starts and queues new sends behind it. The auto size now matches cold_start_sizing: twice the effective cold-start value, clamped to 2..64 (8..32 when cold-start is auto). The goal is scoped to attended turns and states its dependence on _ChatSlot.unattended; unattended turns outside run_background_turn are out of scope.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 4, 2026
@jeeshofone
jeeshofone force-pushed the docs/rfc-attended-turn-cap branch from c9fa177 to 8efbbb6 Compare November 4, 2026 12:04
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 4, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Design brief for rfc-attended-turn-cap (read in full at 8efbbb6).

What it does. It adds one attended-turn semaphore on DashboardState, taken in the exit guard of _run_chat beside gate_turn. That is the one seam every turn producer passes, so no call site changes and none can bypass it. Turns over the cap queue with a bounded 1800 s wait that ends in a visible message, and the person's message is kept. A typed send is served ahead of bulk Resumes. A nested turn reuses its outer permit, and a turn waiting on a person keeps its permit. The setting is dashboard.max_attended_turns: auto is twice the cold-start concurrency, clamped to 2..64, and 0 turns the cap off.

Pros.

  • It closes the gap #25055 measured: a bulk Resume of 40 sessions no longer runs 40 turns at once once the starts end.
  • One place, one check, guarded by a test, instead of per-path limits.
  • It is separate from the unattended pool, so crews and people cannot starve each other.
  • 0 restores today's behaviour exactly, and no stored format changes.

Cons.

  • With auto on by default, a typed send can now wait when N turns are running or parked on approval prompts. The waiting line and the priority order soften this, but it is a visible change at the cap.
  • With Shared Crew admission on, a queued turn holds an idle attended permit (open question 3).
  • spawn_guarded_turn turns spend their wait against their own wall-clock ceiling, so a short ceiling can end the turn before the cap message appears.

Tech debt. No. It adds one well-placed limit, and it names the remaining unattended gap as out of scope rather than hiding it.

Maintainer signs the direction. This is a comment, not an approval. I suggest settling open question 3 in the implementation PR.

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 5, 2026
@jeeshofone
jeeshofone force-pushed the docs/rfc-attended-turn-cap branch from 8efbbb6 to a2f8811 Compare November 5, 2026 12:06
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Nov 5, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

HOLD (conflict) at a2f8811.

The branch conflicts with main again in docs/request-for-change/README.md. Please rebase and keep both rows.

On the rebase, please fix the description's What changed, as the Design note says. It still says the permit is taken by one helper that spawn_guarded_turn and enqueue_or_run_prompt call, and released while a turn waits on a person. The RFC takes it in _run_chat's guard (§1.2) and keeps it while waiting on a person (§1.2a). A reader approving from the description would approve a different design.

The other Design note (parked turns can block a typed send for up to 1800 s) does not block: §1.2a already states that cost and how the waiting line shows it. Checks are green. A new head needs a fresh review.

— crew-pr-reviewer (r146)

@jeeshofone
jeeshofone force-pushed the docs/rfc-attended-turn-cap branch from a2f8811 to d45edb4 Compare November 6, 2026 07:15
@jeeshofone

Copy link
Copy Markdown
Contributor Author

@iamwhatever thanks for the r146 HOLD. Rebased onto main as d45edb4, still one commit.

  • Conflict: docs/request-for-change/README.md keeps both main's rows and this PR's row.
  • What changed: now says the permit is taken in the guarded wrapper of _run_chat around gate_turn (§1.2), and kept while a turn waits on a person (§1.2a) and on SessionManager._start_sem.
    Docs checks pass locally.

iamwhatever
iamwhatever previously approved these changes Nov 6, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve at d45edb45211f5d2806b0c8c2716ed90ce7874166.

This is a docs-only draft RFC. The r146 point is fixed: What changed now matches §1.2, where the permit is taken in _run_chat's guard and kept while a turn waits on a person (rfc#L64-L78). All checks are green, and @bolichen97 signed the direction.

AI findings judged non-blocking

  • Design: a turn that waits on another slot's turn would deadlock at cap 1. Does not block: I found no such wait on main. enqueue_or_run_prompt, which session_deliver uses, starts the turn with create_task and returns (chat_slot.py#L1100). A sentence ruling this out would still help the implementation PR.
  • Design: turns parked on approvals keep their permits, so at a small cap new sends can wait 1800 s. Does not block: §1.2a states and weighs this cost (rfc#L78).
  • Opus: the spawn_guarded_turn caller list leaves out chat_turn/agent_started.py. Does not block: the permit is taken inside _run_chat, so coverage does not depend on that list (rfc#L64).

Note: #28664 and #28632 edit the same README sentence, so the PRs that merge after this one will need another rebase.

— crew-pr-reviewer (r148)

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at 76a2128.

This is a docs-only draft RFC plus its index row. The RFC text is identical to d45edb4, which I approved in r148; this head is a rebase onto current main, and the README change is the one new row and audit sentence with no conflict. The description matches §1.2: the permit is taken in _run_chat's guard beside gate_turn and kept while a turn waits on a person (rfc#L64-L78). All checks are green, and @bolichen97 signed the direction.

AI findings judged non-blocking:

  • Design (note): parked turns keep their permits, so at a small cap a typed send can wait up to 1800 s. §1.2a already states and weighs this cost (rfc#L78).
  • Design (note): the wait ends at the spawn_guarded_turn ceiling on one path and at the 1800 s bound on the others. §1.2 names both outcomes; choosing one message belongs to the implementation PR (rfc#L70).

— crew-pr-reviewer (r149)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants