Repository navigation
Feature: configurable inbound body limit - 922k context sessions 413 on remote compact (256 MiB hard-coded in request-decompress) #3573
Description
Area
Proxy and routing
What are you trying to accomplish?
With the GPT-5.6 1M (922k opt-in) context window configured, a long session eventually triggers Codex's automatic remote compaction. That compaction request replays the full conversation history to the compaction model, and once the session passes roughly 700-800k tokens the serialized request body exceeds the proxy's hard-coded 256 MiB inbound limit. The compact task then fails with:
Error running remote compact task: unexpected status 413 Payload Too Large: Decompressed request body exceeds 268435456 bytes, url: http://127.0.0.1:10100/v1/responses
Users on the large opt-in window cannot use automatic (or late manual) compaction at all, because the compaction request itself is what crosses the limit. I want the body limit to be configurable (or scaled with the model window) so large-context sessions can compact.
What prevents this today?
- src/server/request-decompress.ts hard-codes MAX_DECOMPRESSED_BODY_BYTES = 256 * 1024 * 1024 for /v1/responses inbound bodies (readJsonRequestBody always uses it; there is no config lever).
- A 922k-token session history serializes to well over 256 MiB of JSON, so remote compact (and any full-history request) returns 413 before parsing.
- Manual /compact suffers the same 413 once the history is large enough - the user is stuck until they abandon the session.
- The limit is independent of the configured context window: smaller windows (e.g. 272k) stay far below it, which is why this only bites the opt-in large-window setups.
What should OpenCodex do?
- Make the inbound decompressed-body limit configurable (e.g. a config key with a sensible default), and/or
- Derive it from the routed model's configured context window (limit >= window size serialized with headroom), so enabling the 1M opt-in does not silently break compaction, and/or
- At minimum, return a clearer error explaining that the session is too large to compact through the proxy (instead of an opaque 413), and suggest a smaller window or manual compaction earlier.
Example usage or interface
# config.json
"inboundRequestBodyLimitBytes": 536870912 # 512 MiB opt-in
# or per-model auto-scaling when the 922k window is enabled
"providers": { "openai": { "modelContextWindows": { "gpt-5.6-sol": 922000 } } }
# -> inbound limit scales to cover the window, compaction works
Alternatives or workarounds
- Manually /compact before the session exceeds ~600k tokens (works, but easy to forget; automatic compaction still 413s if it fires late).
- Revert to the 272k default window (compaction always safe, but the large-window capability is lost).
Additional context
Related: the 1M opt-in mechanism for gpt-5.6 (NATIVE_GPT56_MAX_INPUT_TOKENS / modelContextWindows), and #3196 which added an opt-in ceiling for oversized outbound passthrough bodies - this is the inbound analogue. Reproduced on 2.42.0.
Checks
- I searched existing issues and documentation.
- This request describes a concrete OpenCodex workflow rather than merely naming a desired technology.
- I removed secrets and personal data.
Activity
Ingwannu commentedon Sep 5, 2026
The 413 path is real: current
devstill applies the shared 256 MiB decompressed admission cap before parsing Responses and compact bodies. I do not think the token-count explanation is established yet, though. A 700–922k text-token history normally should not serialize anywhere near 256 MiB; crossing this limit usually implies large base64 image parts, very large tool outputs, repeated embedded payloads, or another replay-shape expansion.Please provide content-free measurements from one failing request: compressed
Content-Lengthwhen present, decompressed byte count from the 413, Codex version, and counts plus total serialized bytes grouped by input item/content-part type. In particular, note the total bytes ininput_imagedata URLs, file parts, and tool outputs. Do not share prompts, credentials, images, or raw request bodies.A safe fix should not simply expose an unbounded/global 512 MiB knob. The parser temporarily owns raw, decoded, UTF-8 text, and parsed JSON representations, so doubling the admission cap can multiply peak memory and make a shared/non-loopback proxy vulnerable to memory exhaustion. Model context size is also unavailable until after this pre-parse gate. The likely safe direction is a bounded, hard-capped opt-in with aggregate admission accounting, plus a compaction-specific diagnostic; if one payload class is responsible, earlier compaction or pruning that class may be better than raising every data-plane route. I am keeping this open pending the sanitized size breakdown.
lidge-jun commentedon Sep 5, 2026
리뷰 · 우선순위 50 / 80
이 이슈는 GPT-5.6 1M(922k opt-in) 같은 큰 컨텍스트 세션에서 Codex 자동(또는 늦은 수동) remote compact가 전체 히스토리를 다시
/v1/responses로 보낼 때, 프록시의 하드코딩된 256 MiB 인바운드 압축 해제 한도에 걸려 413으로 죽는다는 기능 요청이다. 제보 에러는Decompressed request body exceeds 268435456 bytes이고, 대략 700–800k 토큰을 넘긴 세션에서 compact 자체가 한도를 넘기 때문에 창은 켰는데 compact를 못 해서 세션을 버려야 하는 상태에 빠진다. 원하는 것은 (1) 설정으로 인바운드 한도를 올리거나, (2) 모델 컨텍스트 창에 맞춰 자동 스케일하거나, (3) 최소한 413 대신 세션이 너무 커서 compact가 안 된다는 안내다.지금
dev(HEAD24cc558d5, package 2.43.0)에서 원인은 제보 그대로다.src/server/request-decompress.ts22줄MAX_DECOMPRESSED_BODY_BYTES = 256 * 1024 * 1024이고, 주석도 루프백 Codex 폭탄 위협은 약하고 OOM 가드라고 적혀 있다.readJsonRequestBody(237–238줄)는 항상 이 상수를readBoundedJsonRequestBody에 넘긴다. 호출부는 Responses 데이터면 전반이다:src/server/responses/core.ts2724줄(본요청),src/server/responses/compact.ts498줄(remote compact), collaboration/encrypted-payload/policy-fallback도 같은 헬퍼를 쓴다.DecompressedBodyTooLargeError는core.ts1438줄대에서 처리되어 제보의 413 문구로 나간다. 아웃바운드 쪽에는 이미 #3196이maxUpstreamBodyBytes(default-off,src/types/config.ts761줄,src/config.tszod,core.tscheckOutboundBodySize)를 넣어 큰 바디 가드의 나가는 짝을 만들었다. 이 이슈는 그 들어오는 짝이다.컨텍스트 창과 한도가 분리돼 있다는 점도 맞다.
modelContextWindows와 GPT-5.6 922k/272k 쌍은 routing/capability·input-admission·registry·관리 API에 이미 있다. 창을 키워도MAX_DECOMPRESSED_BODY_BYTES는 안 따라간다. 272k 기본 창에서는 JSON이 256 MiB 아래에 잘 남고, 922k opt-in + 긴 히스토리(+이미지 base64)에서만 compact가 먼저 터지는 구조다. 다만 한도를 창 토큰 × 어떤 바이트 계수로 자동 추론하는 건 추정이 거칠고, 이미지·툴 결과가 있으면 토큰 대비 바이트가 크게 뛴다. decompress 주석이 말하듯 12장 풀해상도 스크린샷만으로도 수십 MB가 나온다. 그래서 (2) 자동 스케일보다 (1) opt-in 설정 키(예:inboundRequestBodyLimitBytes, 기본 256 MiB 유지)가 #3196 패턴과 가장 잘 맞고, OOM 리스크도 운영자가 의도적으로 연다. (3) 에러 메시지는 설정 없이도 싸게 개선할 수 있다.현재 머지 트레인(OAuth/Kiro #3561/#3562, #3348 combo 바운디드 스택, Windows suite)과는 파일이 겹치지 않는다. types/config 스플릿 캠페인에 새 키를 넣을 때는 types/config zod 위치에만 조심하면 되고, 이 이슈 자체는 close-don't-rebase 대상이 아니다. 열린 이슈 중 같은 inbound 256 MiB / remote compact 413 중복은 보이지 않는다. 우선순위는 922k opt-in 사용자만, 워크어라운드(일찍 /compact 또는 272k 유지) 있음, 보안·메모리 트레이드오프 있음이라 중상위 enhancement로 둔다.
src/server/request-decompress.ts22 / 237–238 - 인바운드 한도가 상수 256 MiB로 고정이고readJsonRequestBody가 설정을 받지 않는다.src/server/responses/compact.ts498 - remote compact가 같은readJsonRequestBody를 써서, 큰 세션일수록 compact 요청 자체가 413으로 실패한다.src/server/responses/core.ts2724 / 1438대 - 일반/v1/responses도 동일 캡·에러 경로라, compact만이 아니라 full-history 재전송 전반에 영향이 있다.src/types/config.tsmaxUpstreamBodyBytes/ #3196 - 아웃바운드 opt-in 선례. 인바운드도 기본 유지 + 상한 올리기 옵션이 대칭이다.경로
modelContextWindows/ GPT-5.6 922k - 창 설정과 decompress 캡이 연결되어 있지 않아 opt-in 큰 창이 silent break를 만든다.메인테이너의 판단이 필요한 지점
TranslatorBudget)·동시 연결과의 OOM 가드 정책(절대 상한 캡이 필요한지).inboundRequestBodyLimitBytes로 할지, #3196과 대칭인 다른 이름/네임스페이스로 할지.너의 추천
라벨
enhancement/proxy맞다. 1차 구현은 #3196 대칭으로 기본 256 MiB 유지 + opt-in 상한 설정을readJsonRequestBody/readBoundedJsonRequestBody에 주입하고, 413 본문에 inbound decompressed body limit / earlier compact or raise limit 안내를 넣는 쪽을 추천한다. 창 자동 스케일은 2차로 두자(바이트 추정 부정확). 922k를 쓰는 운영자만 설정을 올리면 되고, 기본 사용자는 OOM 가드가 그대로다. 우선순위 50 — 재현·원인 명확, 영향 범위는 opt-in 대형 창, 지금 핫픽스 트레인은 아님.이 댓글은 grok-bot이 작성했습니다
nowhere1975 commentedon Sep 6, 2026
Thanks for the detailed review — here are content-free measurements from the actual failing request, plus a before/after control that directly brackets the request size.
Environment
The failing request (09-05 01:27–01:34 UTC)
The 413 burst came from one explicit compaction of a long-lived rolling session (created 07-31; rollout journal is 50,257 lines with 27,255 response_items; the session had already performed 56 window compactions — "compacted" records with replacement_history — so it is a multi-window session, not a single giant context).
thread/compact/startat 01:28:28 UTC (conversation 019fb74b-…).Decompressed request body exceeds 268435456 bytesunknown/unknownin all of them → the body is rejected before parsing (as your review notes, at the decompress admission cap), so no upstream error body exists to quote.Session content composition (rollout journal, serialized bytes per item type)
This is the full journal, not the compact payload, but it shows where the bytes live:
Tool outputs dominate (custom_tool_call_output 92.0 MB + function_call_output 27.5 MB ≈ 62.7% of bytes). There are no input_image data-URL parts in this session (text/tool-output only); the 256 MiB crossing therefore comes from the compaction replay shape carrying the session's accumulated tool outputs across window boundaries.
Before/after control (local patch, same session, same client)
I temporarily raised the inbound decompress cap to 1.5 GiB in the proxy source (issue #3573 workaround) and restarted:
Since the same client+session succeeds at 1.5 GiB and fails at 256 MiB, the failing compact body is definitively between 256 MiB and 1.5 GiB decompressed — consistent with the 181.8 MiB journal being expanded further by compaction replay framing (system prompt + all window summaries + repeated tool outputs).
Requests
inboundRequestBodyLimitBytesconfig key (default 256 MiB) as suggested — that would have unblocked this session without a source patch.Happy to provide more measurements (e.g. gzip Content-Length from a re-captured attempt with the cap restored) if useful.
Ingwannu commentedon Sep 6, 2026
The exact decompression error is strong evidence that OCX's 256 MiB admission limit blocked these requests before routing. Thank you for providing content-free measurements.
One qualification: the posted post-change 200 rows are normal model requests. Unless the same compact request was observed completing under the larger limit, they do not by themselves prove an upper bound for the original compact payload. Likewise, the journal's serialized size is not the transmitted compact body's size. The decisive next measurement is the decompressed byte count for that exact compact attempt and its endpoint/result, without its contents.
An opt-in limit cannot be implemented by changing the decoder constant alone. Current
src/server/index.tsalso passes that constant toBun.serve({ maxRequestBodySize }); the runtime listener and decompression reader must use one validated policy. Preserve the default, bound the opt-in maximum, and cover plain/compressed requests and concurrency/memory behavior. A 1.5 GiB body can cost substantially more than that once JSON and replay structures are materialized, so it is not a safe general default or a sufficient memory budget by itself.A bounded byte-count diagnostic is useful; item counts may not be available when admission rejects the body before parsing, so the 413 path must not parse/retain the rejected payload merely to report them. Keeping the issue open; no global size increase or retry-policy change has been applied.
fix(diagnostics): distinguish inbound size measurement provenance [sk…
Merge pull request #4138 from lidge-jun/codex/3573-configurable-inbou…
lidge-jun commentedon Sep 9, 2026
Fixed on
devby #4138 (merge commit1b64bd35b4ca494c87b50693dd8b2b6c82ede49a).maxInboundBodyBytesis now an opt-in config key shaped like the existingmaxUpstreamBodyBytes. Omitted or0keeps the current 256 MiB, so nothing changes for an unconfigured proxy; a configured value is clamped to [1 MiB, 512 MiB], which covers the 512 MiB you proposed.The ceiling is deliberate rather than an oversight of option 1 in your request:
readBoundedJsonRequestBodymaterializes the body several times over (retained wire bytes, decoded bytes, the decoded string, and the parsed object graph), so peak memory is a multiple of whatever is admitted, and an unbounded inbound cap would be a memory-exhaustion lever. It is enforced in a single resolver that every reader goes through rather than in the config schema, because the schema degrades an invalid hand edit toundefinedinstead of failing the parse and so cannot be the enforcement point.Option 3 is covered too. Bun rejects an oversized body before the handler runs, so the listener's accept size now follows the config at bind time (a change takes effect on restart, and an out-of-range value is clamped with a startup warning naming the value actually used). A local admission refusal now answers HTTP 413 with
code: inbound_body_too_largeand a message naming OpenCodex as the refuser, the observed and configured sizes, and the key that moves the limit. That is deliberately distinct from thecontext_length_exceeded413 that #4127 introduced on this same surface for an upstream size refusal, since the two look alike to a client but have opposite remedies.Option 2 (deriving the limit from the routed model's context window) was not implemented. The byte size of a serialized history is not a function of its token count once images are involved, so an explicit ceiling is the honest lever rather than an inferred one.
fix(diagnostics): distinguish inbound size measurement provenance [sk…
Merge pull request lidge-jun#4138 from lidge-jun/codex/3573-configura…