Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ Providers can expose a built-in shorthand, such as `agy` for `google-antigravity
| `modelSupportsServiceTier?` | `Record<string, boolean>` | Exact upstream model capability overrides. Exact `true` enables canonical Fast for that model; exact `false` narrows provider defaults. An explicit provider-level `supportsServiceTier: false` remains fail-closed and cannot be reopened. Exact `true` does not authorize foreign caller-tier forwarding on Chat. Undeclared models fall back to provider-wide behavior. Management `PATCH /api/providers` merges entries and accepts `null` to clear one. |
| `chatServiceTier?` | `boolean` | Provider-wide Chat-wire opt-in for forwarding caller `service_tier` values. On a classified route it governs foreign values such as `flex`, not proxy-owned canonical Fast after capability validation; on an unclassified route it governs every caller value because no Fast capability has been validated. Exact model capability does not authorize foreign forwarding. Responses routes retain their capability-based caller forwarding behavior. |
| `promptCacheKey?` | `boolean` | Provider-wide `openai-chat` opt-in for forwarding a `prompt_cache_key`. The adapter forwards the key it is given and never invents one, but the key is not always the caller's: Claude Messages translation derives one from `metadata.user_id`, or from a model/system/tools cohort when no metadata is sent. Default off. Enable only when the upstream documents support, because strict gateways may reject the unknown field with HTTP 400. |
| `preserveResponsesReasoningContent?` | `boolean` | Keep plaintext reasoning content on replayed Responses reasoning items instead of blanking it (blanking is the ChatGPT backend's rule). Enable for upstreams whose contract accepts reasoning replay, such as DeepSeek. Proxy-minted `ocxr1` envelopes are always stripped. |
| `preserveResponsesReasoningContent?` | `boolean` | Keep plaintext reasoning content on replayed Responses reasoning items instead of blanking it (blanking is the ChatGPT backend's rule). Enable for upstreams whose contract accepts reasoning replay, such as DeepSeek or a self-hosted Kimi on vLLM. Proxy-minted `ocxr1` envelopes are always stripped. A hand-configured `openai-responses` provider without it logs one warning per host the first time plaintext reasoning is dropped. `preserveReasoningContentModels` does not affect this wire. |
| `preserveResponsesInputItemIds?` | `boolean` | Default off. Preserve valid `input[].id` when `store: false` for explicitly trusted launcher relays that require turn identity. Ordinary Responses providers keep stripping IDs; xAI custom-call ID repair remains active. |
| `preserveResponsesMessageMetadata?` | `boolean` | Default off. Preserve private `internal_chat_message_metadata_passthrough` on input items for explicitly trusted launcher relays. This forwards private turn/session metadata to that destination; do not enable for ordinary gateways. |
| `dropResponsesReasoningItems?` | `boolean` | Remove replayed Responses `reasoning` items from continuation input entirely, instead of forwarding them with blanked content. This is lossy: summaries, item ids, and `encrypted_content` go with them, so no reasoning item from the previous turn remains in the forwarded input. It does not remove `previous_response_id`, so an upstream holding its own turn state may still reach it. Enable only for an upstream that rejects reasoning replay outright — the built-in Volcengine Ark Coding Plan preset sets it because validated tool continuations answered `400 InvalidParameter` otherwise. Set it to `false` on that provider to forward reasoning items again. |
Expand Down
2 changes: 2 additions & 0 deletions src/adapters/openai-responses/passthrough.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ import {
} from "../../providers/fastwire";
import { dropResponsesReasoningInputItems, mapRoutedResponsesReasoningEffort, normalizeConfiguredReasoningSummaryDelivery, sanitizeReasoningInputContent, stripDisabledReasoningSummaries, stripDisabledVerbosity, stripNoneReasoningSummary, stripUnsupportedReasoningSummaryDelivery } from "./reasoning";
import { scrubOcxCompactionItems, stripCanonicalOnlyToolFields, stripCanonicalOnlyTopLevelFields, stripInternalChatMessageMetadataPassthrough, stripInvalidItemIds, stripItemIdsWhenUnstored, stripRejectedSamplingParams } from "./request-strips";
import { notePlaintextReasoningDropped } from "./plaintext-reasoning-notice";
import { stripCanonicalForwardPromptCacheOptions, stripDeprecatedPromptCacheRetention } from "./prompt-cache";
import { isPlainObject } from "./internal";
import { normalizeToolSchemas, promoteClientLoadedTools, stripUnsupportedHostedTools } from "./tool-schema";
Expand Down Expand Up @@ -505,6 +506,7 @@ export function createResponsesPassthroughAdapter(provider: OcxProviderConfig):
stripEncryptedContent: threadServingIdentityChanged || requiresPlaintextReasoningReplay(provider),
dropForeignItemId: parsed._dropForeignReasoningItemIds === true,
requirePlaintextReasoning: requiresPlaintextReasoningReplay(provider),
onPlaintextReasoningBlanked: () => notePlaintextReasoningDropped(provider),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Update the owned structure contract for the warning

This callback adds new process-wide, user-visible adapter behavior and provider-classification rules, but the commit updates no mapped structure/ document; for example, structure/providers/chat-compat.md already describes the reasoning-input sanitization contract and now omits its warning and exemption behavior. Update the adapter area's owned structure documentation in the same change so the maintainer contract remains synchronized.

AGENTS.md reference: src/AGENTS.md:L11-L11

Useful? React with 👍 / 👎.

},
),
provider,
Expand Down
85 changes: 85 additions & 0 deletions src/adapters/openai-responses/plaintext-reasoning-notice.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
import { sanitizeLogMetadataString } from "../../lib/redact";
import { isOpenAiOperatedResponsesDestination } from "../../providers/openai-tiers";
import { normalizedProviderEndpoint, PROVIDER_REGISTRY } from "../../providers/registry";
import type { OcxProviderConfig } from "../../types";

/**
* #6675: a custom Responses provider drops replayed plaintext reasoning unless it sets
* `preserveResponsesReasoningContent`. That default is deliberate (the ChatGPT backend rejects
* echoed raw reasoning, and forwarding it costs tokens), but it is silent, and the similarly named
* Chat-wire `preserveReasoningContentModels` does not opt in. Say so once per destination.
*
* Only a hand-configured `openai-responses` row with key auth qualifies. Registry presets (matched
* by endpoint on either OpenAI wire, since a preset may be routed to Responses per model), OAuth and
* forward rows, Azure, and OpenAI-operated destinations are skipped: their replay contract is
* curated, so blanking there is a decision rather than a missing setting. The notice carries only
* the destination host and option names, never request content.
*/
const MAX_NOTICED_DESTINATIONS = 64;
const noticedDestinations = new Set<string>();

function destinationHost(baseUrl: unknown): string | undefined {
if (typeof baseUrl !== "string") return undefined;
try {
return sanitizeLogMetadataString(new URL(baseUrl).host, 128);
} catch {
return undefined;
}
}

/** Azure OpenAI and Azure AI Foundry serve OpenAI-operated models under per-resource hosts. */
const AZURE_OPENAI_HOST_SUFFIXES = [".openai.azure.com", ".services.ai.azure.com", ".cognitiveservices.azure.com"];

function hostnameOf(url: string): string | undefined {
try {
return new URL(url).hostname.toLowerCase();
} catch {
return undefined;
}
}

/** A templated preset (`https://{resource}.openai.azure.com/...`) matches by host pattern. */
function templatedHostMatches(template: string, hostname: string): boolean {
const host = /^[a-z][a-z0-9+.-]*:\/\/([^/?#]+)/i.exec(template)?.[1];
if (!host || !/\{[^}]*\}/.test(host)) return false;
const pattern = host.toLowerCase().split(/\{[^}]*\}/).map(part => part.replace(/[.*+?^$()|[\]\\]/g, "\\$&")).join("[^.]+");
return new RegExp(`^${pattern}$`).test(hostname);
}

function isRegistryDestination(baseUrl: string): boolean {
const endpoint = normalizedProviderEndpoint(baseUrl);
const hostname = hostnameOf(baseUrl);
if (hostname && AZURE_OPENAI_HOST_SUFFIXES.some(suffix => hostname.endsWith(suffix))) return true;
return PROVIDER_REGISTRY.some(entry =>
normalizedProviderEndpoint(entry.baseUrl) === endpoint
|| (hostname !== undefined && templatedHostMatches(entry.baseUrl, hostname))
|| (entry.destinationAliases ?? []).some(alias => normalizedProviderEndpoint(alias.baseUrl) === endpoint));
Comment on lines +53 to +56

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Warn custom providers on vLLM's default URL

When a hand-configured openai-responses provider uses vLLM's standard http://localhost:8000/v1 endpoint, this URL-only scan matches the unrelated, chat-only, overrideable vllm registry row in src/providers/registry/entries-core.ts:1201; isCustomResponsesProvider therefore returns false even while the serializer blanks nonempty reasoning content. This suppresses the newly documented warning for one of the feature's primary self-hosted scenarios, so the exemption should be restricted to registry entries with a curated Responses contract rather than every registry row sharing the URL.

Useful? React with 👍 / 👎.

}

function isCustomResponsesProvider(provider: OcxProviderConfig): boolean {
if (provider.adapter !== "openai-responses") return false;
if (provider.authMode !== undefined && provider.authMode !== "key") return false;
if (typeof provider.baseUrl !== "string" || !provider.baseUrl) return false;
return !isOpenAiOperatedResponsesDestination(provider) && !isRegistryDestination(provider.baseUrl);
}

export function plaintextReasoningDroppedNotice(host: string): string {
return `[opencodex] Replayed plaintext reasoning was dropped for the custom Responses provider at ${host}. `
+ "If that endpoint accepts reasoning_text replay, set \"preserveResponsesReasoningContent\": true on the provider; "
+ "\"preserveReasoningContentModels\" applies only to the Chat wire.";
}

export function notePlaintextReasoningDropped(
provider: OcxProviderConfig,
warn: (message: string) => void = message => console.warn(message),
): void {
if (!isCustomResponsesProvider(provider)) return;
const host = destinationHost(provider.baseUrl);
if (!host || noticedDestinations.has(host) || noticedDestinations.size >= MAX_NOTICED_DESTINATIONS) return;
noticedDestinations.add(host);
warn(plaintextReasoningDroppedNotice(host));
}

export function resetPlaintextReasoningNoticesForTests(): void {
noticedDestinations.clear();
}
12 changes: 11 additions & 1 deletion src/adapters/openai-responses/reasoning.ts
Original file line number Diff line number Diff line change
Expand Up @@ -78,13 +78,19 @@ export function sanitizeReasoningInputContent(
* placeholder, rather than emitted in the shape the upstream 400s on.
*/
requirePlaintextReasoning?: boolean;
/**
* Called once per sanitized body when a reasoning item that carried nonempty plaintext
* `reasoning_text` was blanked. Observability only; it never changes the body (#6675).
*/
onPlaintextReasoningBlanked?: () => void;
},
): unknown {
if (!body || typeof body !== "object" || Array.isArray(body)) return body;
const raw = body as Record<string, unknown>;
if (!Array.isArray(raw.input)) return body;

let changed = false;
let blankedPlaintext = false;
const input = raw.input.map(item => {
if (!item || typeof item !== "object" || Array.isArray(item)) return item;
const rec = item as Record<string, unknown>;
Expand Down Expand Up @@ -148,11 +154,15 @@ export function sanitizeReasoningInputContent(
// guide merges reasoning items into the adjacent assistant message), so providers flagged
// `preserveResponsesReasoningContent` keep it — deleting valid replay content there breaks
// continuations after tool calls (issue #875 family).
if (blankContent) next.content = [];
if (blankContent) {
if (hasPlaintextReasoningContent(rec.content)) blankedPlaintext = true;
next.content = [];
}
if (missingPlaintextReasoning) next.content = plaintextReasoningBackfill(next);
return next;
});

if (blankedPlaintext) opts?.onPlaintextReasoningBlanked?.();
return changed ? { ...raw, input } : body;
}

Expand Down
80 changes: 79 additions & 1 deletion tests/providers/deepseek-reasoning-replay.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,13 @@
* flagged `preserveResponsesReasoningContent` now keep valid replay content while
* still stripping proxy-minted `ocxr1` envelopes no upstream can decrypt.
*/
import { describe, expect, test } from "bun:test";
import { afterEach, describe, expect, spyOn, test } from "bun:test";
import { createResponsesPassthroughAdapter as createResponsesPassthroughAdapterProduction, sanitizeReasoningInputContent } from "../../src/adapters/openai-responses";
import {
notePlaintextReasoningDropped,
plaintextReasoningDroppedNotice,
resetPlaintextReasoningNoticesForTests,
} from "../../src/adapters/openai-responses/plaintext-reasoning-notice";
import { enrichProviderFromRegistry, providerConfigSeed } from "../../src/providers/derive";
import { getProviderRegistryEntry } from "../../src/providers/registry";
import { OCX_REASONING_PREFIX } from "../../src/responses/reasoning-envelope";
Expand Down Expand Up @@ -270,3 +275,76 @@ describe("DeepSeek Responses replay keeps reasoning on the wire", () => {
expect(item.content).toEqual([]);
});
});

// #6675: the default blanking stays, but a custom Responses provider is told once how to opt out.
describe("dropped plaintext reasoning notice", () => {
afterEach(() => resetPlaintextReasoningNoticesForTests());
const customVllm = (): OcxProviderConfig => ({ adapter: "openai-responses", baseUrl: "http://gpu-box.local:8000/v1", apiKey: "k" } as OcxProviderConfig);
const buildBody = (provider: OcxProviderConfig): Record<string, unknown> => {
const built = createResponsesPassthroughAdapter(provider).buildRequest({
modelId: "kimi-k3",
context: { messages: [] },
stream: true,
options: {},
_rawBody: { model: "kimi-k3", input: [reasoningItem()] },
} as Parameters<ReturnType<typeof createResponsesPassthroughAdapter>["buildRequest"]>[0], { headers: new Headers() });
return JSON.parse(String(built.body)) as Record<string, unknown>;
};

test("the sanitizer reports blanking of nonempty plaintext only", () => {
let calls = 0;
const onPlaintextReasoningBlanked = () => { calls += 1; };
sanitizeReasoningInputContent({ model: "m", input: [reasoningItem(), reasoningItem({ id: "rs_2" })] }, { onPlaintextReasoningBlanked });
expect(calls).toBe(1);
sanitizeReasoningInputContent({ model: "m", input: [reasoningItem()] }, { preserveRawReasoningContent: true, onPlaintextReasoningBlanked });
sanitizeReasoningInputContent({ model: "m", input: [reasoningItem({ content: [] })] }, { onPlaintextReasoningBlanked });
sanitizeReasoningInputContent({ model: "m", input: [reasoningItem({ content: [{ type: "reasoning_text", text: "" }] })] }, { onPlaintextReasoningBlanked });
expect(calls).toBe(1);
});

test("a custom destination is warned once, with no request content", () => {
const messages: string[] = [];
notePlaintextReasoningDropped(customVllm(), message => messages.push(message));
notePlaintextReasoningDropped(customVllm(), message => messages.push(message));
expect(messages).toEqual([plaintextReasoningDroppedNotice("gpu-box.local:8000")]);
expect(messages[0]).toContain("preserveResponsesReasoningContent");
expect(messages[0]).not.toContain("think step by step");
});

test("registry presets and OpenAI-operated destinations stay silent", () => {
const messages: string[] = [];
const warn = (message: string) => messages.push(message);
notePlaintextReasoningDropped({ ...providerConfigSeed(getProviderRegistryEntry("openai-apikey")!), apiKey: "sk-test" }, warn);
notePlaintextReasoningDropped({ ...providerConfigSeed(getProviderRegistryEntry("opengateway")!), adapter: "openai-responses", apiKey: "k" }, warn);
notePlaintextReasoningDropped({ adapter: "openai-responses", authMode: "forward", baseUrl: "https://chatgpt.com/backend-api/codex" } as OcxProviderConfig, warn);
notePlaintextReasoningDropped({ ...customVllm(), authMode: "oauth" } as OcxProviderConfig, warn);
notePlaintextReasoningDropped({ ...customVllm(), adapter: "openai-chat" } as OcxProviderConfig, warn);
// Hand-configured openai-responses rows pointing at Azure's per-resource hosts.
for (const baseUrl of [
"https://contoso.openai.azure.com/openai/v1",
"https://contoso.services.ai.azure.com/openai/v1",
"https://contoso.cognitiveservices.azure.com/openai/v1",
]) notePlaintextReasoningDropped({ ...customVllm(), baseUrl } as OcxProviderConfig, warn);
expect(messages).toEqual([]);
});

test("a lookalike host outside Azure is still treated as custom", () => {
const messages: string[] = [];
notePlaintextReasoningDropped({ ...customVllm(), baseUrl: "https://openai.azure.com.example.net/v1" } as OcxProviderConfig, message => messages.push(message));
expect(messages).toHaveLength(1);
});

test("the passthrough serializer still blanks a custom provider's reasoning and emits the notice", () => {
const warnSpy = spyOn(console, "warn").mockImplementation(() => {});
try {
const body = buildBody(customVllm());
expect((body.input as Record<string, unknown>[])[0]!.content).toEqual([]);
expect(warnSpy.mock.calls.map(call => String(call[0]))).toEqual([plaintextReasoningDroppedNotice("gpu-box.local:8000")]);
const preserved = buildBody({ ...customVllm(), baseUrl: "http://other-box.local/v1", preserveResponsesReasoningContent: true });
expect((preserved.input as Record<string, unknown>[])[0]!.content).toEqual([{ type: "reasoning_text", text: "think step by step" }]);
expect(warnSpy.mock.calls).toHaveLength(1);
} finally {
warnSpy.mockRestore();
}
});
});
Loading