[Feature]: Demand-level routing for JEV combos — orchestrator-chosen levels, per-session pinning, probability escalation (opt-in series on #6364) #6665

Area

Proxy and routing

What are you trying to accomplish?

We route Claude Code subagents through one JEV combo (CLAUDE_CODE_SUBAGENT_MODEL=ocx-claude-combo--<combo> with ..._FORCE=1). The goal is to give each subagent a model and effort that match how demanding its assignment is, while keeping three properties:

• The orchestrating agent can state that level when it knows it, for example "this delegate does a cross-module refactor".
• The provider prompt cache survives for the whole subagent conversation.
• Operators keep owning the candidate lists and quota policy.

What prevents this today?

On current dev a JEV combo decides a target and effort per request. When we ran a level-mode build (the closed #6423 work) with Claude Code for about a week, we saw the following:

• Mid-conversation switches. In 2,975 consecutive request pairs within the same subagent, 28% switched model. When candidates span providers, every switch is a cold prompt cache. Anthropic also documents that changing output_config.effort invalidates message-level cache blocks.
• The first request is misread. A subagent's opening request reads and plans the whole assignment, yet it was classified at the lowest level 87% of the time. One reason is fixed separately in a bug-fix PR: the decision task for Claude Code requests starts with <system-reminder> blocks such as project CLAUDE.md.
• The orchestrator has no channel. The main agent cannot pass the demand level it already knows.
• Level mode is not on dev. The maintainer comment on feat(combos): self-hosted JEV decision service, level mode with quota-aware selection, configurable decision wording #6302 invited a new PR on dev on top of feat(combos): JEV decision methods — TypeSafe, System One server, or any opencodex model #6364's decision-backend interface. The author closed feat(combos): JEV quota signals and level decision mode #6423 as superseded; this proposal is that follow-up, rebuilt on feat(combos): JEV decision methods — TypeSafe, System One server, or any opencodex model #6364 rather than reopened.

What should OpenCodex do?

Every item is opt-in and leaves today's behavior and payloads unchanged when unset.

1. Demand-level routing through the existing dispatcher. decisionMode: "level" with decisionLevels holds per-level ordered candidates, each with an effort. decisionFallbackLevel sets the fallback. Selection goes through resolveJevComboDecision for the TypeSafe, self-hosted System One and model backends.
2. Quota signals with configurable tier boundaries. The defaults stay at 70% / 90%. An optional extra boundary (for example 40%) lets a combo prefer a provider with more headroom.
3. Fixed level (decisionFixedLevel). The combo uses one level and never calls the classifier.
4. Described custom level ids. A combo can use its own vocabulary (for example light/routine/heavy/deep), in config order from least to most demanding. Each custom id must have a description, and the orchestrator and classifier share the same definitions.
5. Richer classification evidence (decisionRichState). Instead of only the last 500 characters, the classifier gets:
   • the assignment, meaning the first user message with reminders removed;
   • the head of the agent role;
   • tool names and capability flags (edit / shell / web / spawn_agents).
     No schemas, arguments, reminders or credentials are sent.
6. Session pinning (decisionSessionPin). Decide the level, target and effort once per client conversation, then reuse them. Re-pin within the level, then the fallback level, only when the pinned target becomes ineligible or nearly exhausted, and record the level actually used. The model and effort move together, because changing either costs the cache anyway.
7. Orchestrator hints (decisionLevelHints). A system-prompt line ocx-level: <level> selects the level without a classifier call. The precedence is fixed level, then hint, then classifier.
8. Probability escalation (decisionLevelEscalation: {minProbability, maxLevel?}). When the answer carries per-level probabilities, pick the most demanding level at or above minProbability, capped at maxLevel. Without probabilities, nothing changes.

Example usage or interface

"combos": { "subagents": {
  "strategy": "jev", "decisionProvider": "<system-one or model backend>",
  "decisionMode": "level",                       // proposal
  "decisionLevels": {                            // custom ids: proposal
    "light":   { "description": "single-pass, checkable work …", "candidates": [{ "provider": "anthropic", "model": "claude-sonnet-5-5", "effort": "low" }, { "provider": "openai", "model": "gpt-6-luna", "effort": "medium" }] },
    "routine": { "description": "bounded implementation …",      "candidates": [{ "provider": "anthropic", "model": "claude-sonnet-5-5", "effort": "high" }] },
    "heavy":   { "description": "non-trivial engineering …",     "candidates": [{ "provider": "anthropic", "model": "claude-sonnet-5-5", "effort": "xhigh" }] },
    "deep":    { "description": "design/analysis, no implementation …", "candidates": [{ "provider": "anthropic", "model": "claude-opus-5-5", "effort": "xhigh" }] }
  },
  "decisionFallbackLevel": "routine",
  "decisionQuotaSignals": true, "decisionQuotaTiers": { "moderate": 40, "limited": 70, "nearlyExhausted": 90 },  // names: proposal
  "decisionSessionPin": true, "decisionRichState": true,
  "decisionLevelEscalation": { "minProbability": 0.3, "maxLevel": "heavy" },
  "targets": [ … ]
}}

A Claude Code agent definition opts in with one line in its prompt body: ocx-level: heavy. With none of these fields set, the combo behaves exactly as it does today.

Alternatives or workarounds

• Per-level combos with failover and per-agent model: + effort:. This works on stock dev, but loses per-candidate efforts, quota-tier preference, and classification for agents that carry no hint, such as built-ins.
• Per-request route mode. This is the cache and stability problem described above.
• Keeping a private build. This is what we do now, and it breaks on ocx update.

Additional context

Evidence. These numbers come from one self-hosted tev1:4b System One classifier and our own labelled tasks, so treat them as indicative:

• Setup: 85 subagent assignments in the exact shape Claude Code sends, no hint, 3 runs each.
• Plain argmax: 76.5% exact. Misses were mostly one level too low (54 vs 6 runs).
• Escalation (minProbability: 0.3, capped below the design level), computed on the same stored probabilities: correct answers rose from 65 to 73 of 85, and from 180 to 199 of 255 when the agent role was varied. Too-low errors fell from 18 to 8 and too-high errors rose from 2 to 4.
• Session pinning, observed live: a 16-request subagent stayed on one model and effort with about 60k cached input tokens per turn.
• Not tested: we have no TypeSafe key, so hosted jev-latest is untested with custom ids, rich state, and probabilities.

Proposed series. Each PR is about 100–400 production lines, default-off, with tests, docs-site and structure updates. Stacked PRs retarget dev after their parent lands. The list follows the port plan:

• P0: share the bounded System One exchange
• P1: quota signals and configurable boundaries
• P2: level definitions
• P3: level routing through all backends
• P4: read-only GUI
• P5: fixed level
• P6: custom ids
• P7: rich evidence
• P8: session pinning
• P9: hints
• P10: escalation
• P11: refresh quota only while quota-aware combos are active

Separate bug fixes are opened independently: #6662 (Claude Code <system-reminder> in the decision task), #6663 (the forced-effort usage label) and #6664 (Cursor redirect wording for Claude-family models).

Questions before coding, in priority order:

1. Is resolveJevComboDecision the right owner for mode and policy dispatch? Should level policy travel as fields or as a nested, versioned policy object?
2. Should the feat(combos): JEV quota signals and level decision mode #6423 field names be kept? Is the fixed → hint → classifier precedence right, and should hints require pinning?
3. Are custom ids and descriptions acceptable on hosted TypeSafe, or only on self-hosted and model backends? Should config order define demand order?
4. Pin identity and state. Should the server own a request-scoped session key, with the hash of system prompt plus first assignment only as a fallback? Is in-memory, bounded state with no restart survival acceptable? What invalidates a pin when config, credentials or targets change?
5. What is pinned (provider, model and effort, but not the pool account), and is it committed on classification or on first successful inference?
6. Should quota boundaries be per combo or global? May four tiers appear in the hosted route payload?
7. Which backends return validated per-level probabilities? Should escalation wait until that contract is agreed?
8. Telemetry: what should backend mean for fixed, hint and pinned decisions that never call a backend? Pin hashes would be left out of records unless needed.
9. Scope: is a read-only GUI enough? We do not plan endpoint changes, which overlaps with Allow arbitrary HTTPS JEV decision endpoints #6384.

Related.

• [Feature]: Dynamic Model Selection and Smart Routing via TypeSafe Jev #5626: the original JEV smart-routing request.
• feat(subagents): force every Claude Code subagent onto one model from the Subagents page #6313: force every Claude Code subagent onto one model. This is the setup this proposal builds on.
• stabilizePromptCache re-attaches Claude Code's <total_tokens> footer as a user message; routed models treat every tool result as a new user turn and re-plan #6646: Claude Code's <total_tokens> footer re-attached as a user turn. Session pinning strips the footer from its key for the same reason.
• Keeping authority stable while the model switches #4579: keeping authority stable while the model switches.
• Allow arbitrary HTTPS JEV decision endpoints #6384: configurable decision endpoints. No overlap is intended.

Credit. The level-mode and quota-signal pieces build on @SeongwoongCho's #6348 / #6423 and will carry Co-authored-by: SeongwoongCho <35558061+SeongwoongCho@users.noreply.github.com>.

Checks

• I searched existing issues and documentation.
• This request describes a concrete OpenCodex workflow rather than merely naming a desired technology.
• I removed secrets and personal data.