Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs-site/src/content/docs/guides/claude-code.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,13 @@ opencodex serves `POST /v1/messages` (plus `count_tokens`) alongside `/v1/respon
Code can use every routed provider — OAuth logins, account pools, key failover and sidecars
included — with zero extra auth work.

On Devin routes (including SWE-2) reached through the Messages API, text and tool calls wait
for the upstream turn to complete so its late reasoning signature can precede the answer. This prevents Claude Code's final
result from becoming empty; reasoning and keepalive progress still flow during generation.
The buffer shares the request's 32 MiB translation limit and cancellation stops the producer.
This output-order fix does not resolve Cognition's separate refusal of some generated system
text. It preserves system instructions and safety constraints.

For an Anthropic route on stored OAuth or an Anthropic API key, native Fast is available on
`claude-opus-5-5`, `claude-opus-5`, and `claude-opus-4-8`: pick the model's `--fast` row (listed
when Fast rows are enabled) or set `fastMode: true`. Claude Code's own `/fast` toggle is not
Expand Down
7 changes: 7 additions & 0 deletions docs-site/src/content/docs/ja/guides/claude-code.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,13 @@ opencodex は `/v1/responses` と共に `POST /v1/messages`(`count_tokens` も)
Code から OAuth ログイン、アカウントプール、キーフェイルオーバー、サイドカーを含むすべてのルーティングプロバイダーを別途の
認証作業なしで使えます。

Messages API 経由の Devin ルート(SWE-2 を含む)では、テキストとツール呼び出しの送信を
上流のターンが完了するまで待ち、遅れて届く推論署名を回答より先に送ります。これにより Claude Code の
最終結果が空になるのを防ぎます。生成中も推論と keepalive による進行通知は送信されます。
バッファはリクエストの 32 MiB の変換上限を共有し、キャンセルすると生成処理も停止します。
この出力順序の修正は、生成された一部のシステムテキストを Cognition が拒否する別の問題を解決するものではありません。
システム指示と安全上の制約は保持されます。

## クイックスタート

```bash
Expand Down
7 changes: 7 additions & 0 deletions docs-site/src/content/docs/ko/guides/claude-code.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,13 @@ opencodex는 `/v1/responses`와 함께 `POST /v1/messages`(및 `count_tokens`)
Code에서 OAuth 로그인, 계정 풀, 키 장애 조치, 사이드카를 포함한 모든 라우팅 제공자를 별도의
인증 작업 없이 사용할 수 있어요.

Messages API를 통해 사용하는 Devin 경로(SWE-2 포함)에서는 늦게 도착하는 추론 서명을 답변보다 먼저
보낼 수 있도록, 상위 제공자의 턴이 완료될 때까지 텍스트와 도구 호출 전송을 기다려요. 이렇게 하면 Claude Code의
최종 결과가 비어 버리는 것을 막을 수 있어요. 생성 중에도 추론과 keepalive 진행 알림은 계속 전달돼요.
버퍼에는 요청의 32 MiB 변환 한도가 함께 적용되며, 요청을 취소하면 생성 작업도 중단돼요.
이 출력 순서 수정으로 Cognition이 생성된 일부 시스템 텍스트를 거부하는 별개의 문제가 해결되지는 않아요.
시스템 지시와 안전 제약은 그대로 유지해요.

## 빠른 시작

```bash
Expand Down
8 changes: 8 additions & 0 deletions docs-site/src/content/docs/ru/guides/claude-code.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,14 @@ Claude Code может использовать все маршрутизиру
аккаунтов, отказоустойчивое переключение (failover) ключей и сайдкары — без какой-либо
дополнительной работы по аутентификации.

На маршрутах Devin (включая SWE-2), доступных через Messages API, отправка текста и вызовов
инструментов откладывается до завершения хода у провайдера, чтобы поздно поступившая подпись
рассуждений могла предшествовать ответу. Это предотвращает пустой итоговый результат в Claude Code;
рассуждения и уведомления keepalive продолжают поступать во время генерации.
Буфер использует общий с запросом лимит преобразования в 32 MiB, а отмена запроса останавливает генерацию.
Это исправление порядка вывода не решает отдельную проблему отказа Cognition принимать некоторые
сгенерированные системные тексты. Системные инструкции и ограничения безопасности сохраняются.

## Быстрый старт

```bash
Expand Down
7 changes: 7 additions & 0 deletions docs-site/src/content/docs/zh-cn/guides/claude-code.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,13 @@ opencodex 在 `/v1/responses` 之外还提供 `POST /v1/messages`(以及 `coun
Code 可以使用每一个已路由的提供商——包括 OAuth 登录、账户池、密钥故障转移和 sidecar——
而无需进行任何额外的身份验证配置。

通过 Messages API 使用 Devin 路由(包括 SWE-2)时,文本和工具调用会等到上游轮次完成后再发送,
以便将晚到的推理签名放在答案之前。这样可以避免 Claude Code 的最终结果为空;
生成期间,推理内容和 keepalive 进度通知仍会持续传输。
缓冲区与请求共用 32 MiB 的转换上限,取消请求会停止生成过程。
这项输出顺序修复不会解决 Cognition 拒绝部分生成的系统文本这一独立问题。
系统指令和安全约束会保持不变。

## 快速开始

```bash
Expand Down
1 change: 1 addition & 0 deletions scripts/test-layout/layout.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
"version": 1,
"root": "tests",
"explicit": {
"claude-devin-output-order.test.ts": "claude-integration",
"codex-config-preservation.test.ts": "codex-integration",
"codex-credits.test.ts": "codex-integration",
"codex-credits-settings.test.ts": "codex-integration",
Expand Down
98 changes: 98 additions & 0 deletions src/claude/devin-output-order.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
import type { AdapterEvent } from "../types";
import {
isTranslatorBudgetExceededError,
releaseTranslatedEvent,
retainTranslatedEvent,
type TranslatorBudget,
} from "../lib/translator-budget";

/**
* Cognition signs reasoning after text/tools. Claude Code treats a trailing empty
* thinking block as its final result, so Messages needs reasoning before content.
* Consume after raw-event preflight and drain on demand, avoiding a queue burst.
*/
export async function* orderDevinMessagesOutput(
source: AsyncIterable<AdapterEvent>,
budget: TranslatorBudget,
signal: AbortSignal,
abortProducer: () => void,
): AsyncGenerator<AdapterEvent> {
const held: Array<AdapterEvent | undefined> = [];
let cancelled = signal.aborted;
let terminalDelivered = false;
let delivering: AdapterEvent | undefined;
const release = () => {
for (const event of held) if (event) releaseTranslatedEvent(event, budget);
held.length = 0;
if (delivering) releaseTranslatedEvent(delivering, budget);
};
const cancel = () => { cancelled = true; release(); };
signal.addEventListener("abort", cancel, { once: true });
const cancelledTerminal = (event: AdapterEvent): AdapterEvent => {
if (event.type !== "done" && event.type !== "incomplete") return event;
return { type: "error", status: 499, message: "client closed request", retryable: false,
...(event.usage ? { usage: event.usage } : {}) };
};
async function* drain() {
for (let index = 0; index < held.length && !cancelled; index++) {
delivering = held[index];
held[index] = undefined;
try {
if (delivering) {
terminalDelivered = delivering.type === "done" || delivering.type === "error" || delivering.type === "incomplete";
yield delivering;
}
}
finally {
if (delivering) releaseTranslatedEvent(delivering, budget);
delivering = undefined;
}
}
held.length = 0;
}
try {
for await (const event of source) {
const terminal = event.type === "done" || event.type === "error" || event.type === "incomplete";
if (cancelled) {
// The adapter's cancellation terminal retains its measured usage. Client
// stream cancellation instead returns this iterator and stops consumption.
if (terminal) { yield cancelledTerminal(event); return; }
continue;
}
if (event.type === "heartbeat" || event.type === "thinking_delta"
|| event.type === "thinking_signature" || event.type === "redacted_thinking"
|| event.type === "reasoning_raw_delta" || event.type === "kiro_redacted_reasoning") {
yield event;
continue;
}
// Retention owns a snapshot, including nested usage, so later producer or
// consumer mutations cannot change the bytes measured by the budget.
const copy = structuredClone(event);
try {
retainTranslatedEvent(copy, budget, held.at(-1));
held.push(copy);
} catch (error) {
if (!isTranslatorBudgetExceededError(error)) throw error;
release();
// Stop only the producer: aborting the request signal here would let the
// hosted-search loop replace the typed overflow with a client-cancel error.
abortProducer();
yield { type: "error", status: 413, errorType: "request_too_large",
code: "translation_buffer_limit", message: error.message };
return;
}
if (terminal) {
yield* drain();
if (cancelled && !terminalDelivered) yield cancelledTerminal(event);
return;
}
// Raw preflight already observed real output; feed the stream watchdog while
// the answer waits for its signature. Original heartbeats stay verbatim.
yield { type: "heartbeat" };
}
if (!cancelled) yield* drain();
} finally {
signal.removeEventListener("abort", cancel);
release();
}
}
30 changes: 25 additions & 5 deletions src/server/responses/run-turn-execution.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ import { undeclaredToolCallMessage } from "../responses-undeclared-tool-guard";
import { planWebSearch } from "../../web-search";
import { runTurnWebSearchInitialParsed, runTurnWebSearchLoop } from "../../web-search/run-turn-loop";
import { WEB_SEARCH_TOOL_NAME } from "../../web-search/synthetic-tool";
import { orderDevinMessagesOutput } from "../../claude/devin-output-order";

// LOCAL PATCH (runturn-websearch): top-level fields route binding or the
// adapter itself may write during a turn. Iteration-local `turnParsed` objects
Expand Down Expand Up @@ -181,6 +182,13 @@ export async function executeResponsesRunTurn(

const runTurnAbort = new AbortController();
const cleanupRunTurnAbort = linkAbortSignal(runTurnAbort, options.abortSignal);
const devinProducers = new Set<AbortController>();
const orderMessagesOutput = (source: AsyncIterable<AdapterEvent>): AsyncIterable<AdapterEvent> =>
inboundWire === "anthropic" && transportState.runTurnAdapter.name === "devin" && !routedCompaction
? orderDevinMessagesOutput(source, translatorBudget, runTurnAbort.signal, () => {
for (const producer of devinProducers) producer.abort();
})
: source;
const queue = createAdapterEventQueue({
onBacklogExceeded: () => runTurnAbort.abort(),
});
Expand Down Expand Up @@ -229,6 +237,8 @@ export async function executeResponsesRunTurn(
options.onCompactionRecoveryAdapterEvent?.(event);
targetQueue.push(event);
};
let producerAbort = runTurnAbort;
let cleanupProducerAbort: (() => void) | undefined;
try {
if (!pacingSlotAcquired) {
pacingSlot = await waitForProviderRequestSlot(route.providerName, route.provider, route.modelId, runTurnAbort.signal);
Expand Down Expand Up @@ -261,11 +271,16 @@ export async function executeResponsesRunTurn(
turnScopedPacing: true,
},
);
if (inboundWire === "anthropic" && transportState.runTurnAdapter.name === "devin" && !routedCompaction) {
producerAbort = new AbortController();
cleanupProducerAbort = linkAbortSignal(producerAbort, runTurnAbort.signal);
devinProducers.add(producerAbort);
}
await transportState.runTurnAdapter.runTurn?.(
turnParsed,
{
headers: requestState.selectedForwardHeaders,
abortSignal: runTurnAbort.signal,
abortSignal: producerAbort.signal,
comboAttempt: options.comboAttempt === true,
translatorBudget,
providerFetch: runTurnProviderFetch,
Expand Down Expand Up @@ -321,6 +336,8 @@ export async function executeResponsesRunTurn(
message: err instanceof Error ? err.message : String(err),
});
} finally {
devinProducers.delete(producerAbort);
cleanupProducerAbort?.();
releaseProviderRequestSlot(pacingSlot);
// Cursor assigns a stable conversation id inside runTurn on the first headerless
// turn; backfill so Logs can filter/total that opening request (#330 / #522).
Expand All @@ -345,14 +362,14 @@ export async function executeResponsesRunTurn(
});
void runTurnAttempt(iterQueue, undefined, false, iterParsed);
const stream = iterQueue.stream();
if (!runTurnFailoverArmed()) return stream;
if (!runTurnFailoverArmed()) return orderMessagesOutput(stream);
// LOCAL PATCH (runturn-websearch): a post-search iteration can open on a
// 429 too — the search cells already reached the client, so only this
// answer call rotates. Preflight replays it on the next account with the
// grown history intact; a mid-stream error still ends the turn as before.
return (async function* () {
return orderMessagesOutput((async function* () {
yield* await preflightRunTurnFailover(stream, iterParsed);
})();
})());
};
// Rebind the turn to an admitted account. The failed attempt emitted no client-visible bytes,
// so replay is safe, but a Cursor conversation/checkpoint is credential-scoped: carrying its
Expand Down Expand Up @@ -614,7 +631,7 @@ export async function executeResponsesRunTurn(
onBacklogExceeded: () => runTurnAbort.abort(),
});
void runTurnAttempt(retryQueue, "empty-completion");
return retryQueue.stream();
return orderMessagesOutput(retryQueue.stream());
};

const { toolNsMap, declaredToolNames, toolParameterSchemas, freeformToolNames, bareCustomToolNames, toolSearchToolNames } = toolBridgeMaps;
Expand Down Expand Up @@ -651,6 +668,7 @@ export async function executeResponsesRunTurn(
retryAfter: resolveClientRetryAfter({ status: httpStatus, message: error.message }),
});
};
// Messages ingress forces internal streaming, including buffered client requests.
if (parsed.stream) {
try {
void runTurn();
Expand Down Expand Up @@ -699,6 +717,8 @@ export async function executeResponsesRunTurn(
}
eventSource = preflight.stream;
}
// Preflight sees raw output before Messages delays text for the late signature.
eventSource = orderMessagesOutput(eventSource);
// LOCAL PATCH (runturn-websearch): intercept web_search calls across
// iterations; terminal output keeps flowing through the same queue/bridge.
if (wsPlan) {
Expand Down
23 changes: 23 additions & 0 deletions structure/clients/claude-desktop.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,29 @@ Native OpenAI pool routing also accepts
[Orca-linked accounts](../codex-home.md#orca-source-owned-account-import), whose source resolution
belongs to the shared account store. The import CLI adds pool rows independently of Desktop profiles.

## Devin Messages output ordering

`src/claude/devin-output-order.ts` orders each physical Devin turn after raw-event preflight in
`src/server/responses/run-turn-execution.ts` when the original inbound wire is Anthropic
Messages. Provider names may be customized; the selected adapter determines applicability.
Text and tool events wait for that turn's terminal so Cognition's late reasoning signature
precedes them. Claude Code therefore receives a final text or tool block rather than an empty
signature-only thinking block. Reasoning and transport progress remain live; answer text and
tool dispatch incur turn-completion latency. Responses and Chat retain their original ordering,
and routed compaction is excluded.

Retained events own deep snapshots, including nested usage, so producer/consumer mutations
cannot alter their measured payload. They share the request translator budget and drain on demand without a synchronous
burst into the adapter queue. They are released on terminal, cancellation, overflow, or adapter
EOF. Overflow emits one typed `translation_buffer_limit` error and aborts only active Devin
producers, preserving error classification through hosted search. Cancellation drops held
semantic output; this consumer preserves error terminals and maps cancelled success/incomplete
terminals to a 499 with their original usage, so partial output cannot commit completed replay
state. Hosted search retains its existing independent cancellation mapping.
Adapter error and incomplete terminals retain partial output and their original usage.
Ordering occurs before hosted-search interception, independently for each physical iteration,
so one iteration's signature cannot be attached to another iteration's answer.

## Desktop modes: gateway and first-party

`src/claude/desktop-first-party.ts` owns the Desktop mode contract. Two modes exist and are
Expand Down
2 changes: 1 addition & 1 deletion structure/providers-and-adapters.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Providers And Adapters

Anthropic account pause, model routes, and quota labels follow the
[Anthropic account-pool contract](providers/anthropic-account-pool.md).
[Anthropic account-pool contract](providers/anthropic-account-pool.md). Devin Messages follows the [per-turn output ordering contract](clients/claude-desktop.md#devin-messages-output-ordering), preserving late signatures before text/tools without changing Responses or Chat ordering.

Per-account usage thresholds follow the [Anthropic account thresholds contract](providers/anthropic-account-thresholds.md).
An Anthropic 429 records the served account's cooldown even when the request has used its allowed retry sends. That final account remains excluded on the next request; combo target cooling is skipped only after the matching account cooldown is present.
Expand Down
4 changes: 4 additions & 0 deletions structure/transports/byte-accounting.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,10 @@ lifecycle, cancellation races, protocol envelopes, and the real HTTP admission b

## Stream-buffer accounting

Devin's [Messages ordering buffer](../clients/claude-desktop.md#devin-messages-output-ordering) charges retained
semantic events consumed from the independently bounded adapter queue to the shared translator
budget until downstream delivery. Cancellation and overflow release held events before producer shutdown.

`src/web-search/run-turn-loop.ts` charges retained iteration events and generated replay history to
the request translator budget. Each owner releases its own reservations on completion, error,
cancellation or consumer closure; a buffer-limit failure terminates without another search.
Expand Down
Loading
Loading