Repository navigation
Auto-review appears to fall back to Sol low through OpenCodex; direct ChatGPT login uses codex-auto-review #6712
Description
Client or integration
Codex App
Area
Proxy and routing
Summary
I sign in to Codex with a ChatGPT account and have Auto-review enabled.
While using Codex through OpenCodex, the relevant call statistics I inspected showed GPT-6.1 Sol / low. After disabling the OpenCodex integration and restoring native Codex direct access, the records showed codex-auto-review / low.
An old explicit reviewer-model override is not still enabled:
auto_review_modelis commented out and inactive.- I checked the model catalog; the relevant
auto_review_model_overridefields are empty/unset. - Auto-review remains enabled, with approval policy
on-requestand sandbox modeworkspace-write.
| Connection | Observed model identifier or display name | Reasoning effort | Calls | Total tokens |
|---|---|---|---|---|
| Through OpenCodex, before disabling the integration | GPT-6.1 Sol | low | 853 | 95,481,177 |
| Native Codex direct access after disabling OpenCodex | codex-auto-review |
low | 2 | 140,189 |
These totals cover different observation periods. They are not a controlled cost or performance comparison using identical workloads. Per-request evidence attributing all 853 records to approval reviews has not yet been attached; low effort alone must not be used to classify ordinary inference requests as Guardian requests.
This report concerns a difference in reviewer-model selection or identification between the proxied and direct paths. I am not claiming that all 95,481,177 tokens were incorrectly charged against my weekly allowance.
Expected behavior
With ChatGPT account authentication and no reviewer-model override, OpenCodex should preserve Codex's official default Auto-review behavior and request semantics, rather than unintentionally falling back to an ordinary task model because of a missing catalog entry or an integration compatibility issue.
OpenAI's documentation states that Auto-review safety checks are free and do not count against plan usage limits when signed in with ChatGPT:
https://help.openai.com/en/articles/11369540-using-codex-with-your-chatgpt-plan
The observed difference may therefore affect the official free Auto-review path and warrants investigation. I do not have per-request server-side billing evidence, so the actual billing impact is not established.
Reproduction
The following describes the before-and-after behavior I observed. It is not yet a repeatedly verified, deterministic minimal reproduction:
- Sign in to Codex App with a ChatGPT account and enable OpenCodex's Codex integration.
- Use the configuration shown under Redacted configuration. Confirm that the old Luna reviewer override is commented out and that the catalog's reviewer override fields are empty/unset.
- Use Codex for tasks that require automatic approval reviews and inspect the relevant call statistics. Before disabling OpenCodex, the inspected records showed GPT-6.1 Sol / low, totaling 853 calls and 95,481,177 tokens.
- Disable the OpenCodex integration and restore native Codex direct access. This means restoring direct connectivity, not merely stopping the proxy process while leaving Codex pointed at local port 10100.
- Continue using Auto-review. The direct-access records show
codex-auto-review/ low, with 2 calls and 140,189 tokens.
Comparison conditions
- Exact steps used to disable the integration and restore direct access: [Disabled the OpenCodex Codex integration and restored the native Codex configuration. After restoration, the openai_base_url entry was no longer present in config.toml. I then fully restarted Codex App and verified that subsequent requests used the native ChatGPT/Codex connection rather than the local OpenCodex proxy.]
- Whether both paths used the same ChatGPT account and workspace: [Yes. Both paths used the same ChatGPT account and the same workspace. No account or workspace switch was performed between the two observations.]
- Whether Codex App and its bundled runtime were the same versions on both paths: [Yes. Codex App was not updated between the proxied and direct observations. Both observations were made with the same installed Codex App build.]
- Whether the main task model, task, and thread were the same on both paths: [No. The direct-path observation was made in a different/new thread. The purpose of this comparison is only to compare the Auto-review model identity, not identical workload or token usage.]
- Whether Codex was fully quit and reopened after configuration/catalog changes: [Yes. Codex App was fully quit and reopened after disabling the OpenCodex integration and restoring the direct configuration.]
- Proxied observation period and time zone: [2026-10-07 11:45 to 2026-10-07 19:11, UTC+8.]
- Direct observation period and time zone: [2026-10-07 19:15–20:08, UTC+8.]
- Statistics source and filtering criteria: [The statistics were filtered to Auto-review/Guardian requests, then aggregated by model identifier and reasoning effort.]
Version
2.79.0
Operating system
WIN11 24H2
Provider and model
chatpgt 6.1 sol high
Logs or error output
No exception stack trace is attached. The main observation is a difference in model selection or identification:
OpenCodex enabled:
observed model/display name: GPT-6.1 Sol
reasoning effort: low
calls: 853
total tokens: 95,481,177
OpenCodex disabled; native Codex direct access:
model: codex-auto-review
reasoning effort: low
calls: 2
total tokens: 140,189
See **Reproduction** for the statistics source, filtering criteria, and observation periods. These aggregates alone do not prove that every Sol request was an approval review or that all corresponding tokens reduced the weekly allowance.Screenshots and supporting files
Suggested redacted attachments:
- Statistics screenshots for both connection modes, retaining the observation periods, source, and filters.
- Evidence showing whether the default reviewer model exists in the effective model catalog, plus the main model's reviewer override field.
- Corresponding records for one proxied approval review and one direct approval review, retaining the model, effort, Guardian classification, and result status. Do not include full conversations or credentials.
- Screenshots of the actual OpenCodex and Codex App versions.
Investigation leads — not confirmed root causes
A. Fallback when the default reviewer is missing from the catalog
Codex's select_review_model falls back to the parent model when no override is set and the default reviewer cannot be found in the catalog. It prefers low effort when supported.
This is consistent with the observed GPT-6.1 Sol / low records, but the actual local runtime and effective catalog still need to be checked.
Source reference at a fixed commit; this does not establish that the installed binary uses this exact code:
Related historical issue: #1225. Existing routing fix: #833.
This report is about preserving the official default reviewer without an override, not requesting another custom reviewer-model feature.
B. Local /v1 URL and Codex backend recognition
In the same Codex source snapshot, supports_codex_backend_routes() checks for the /backend-api/codex suffix on an explicitly configured base URL.
Please verify whether the local /v1 URL used here affects the generation of official reviewer request markers and associated metadata.
Sources:
C. Reviewer marker forwarding
The inspected public v2.80.0 source does not list x-codex-guardian in FORWARD_HEADERS. The configurable forwardClientHeaders allowlist does not include it either.
This is a source-level investigation lead, not evidence from a local capture proving that a header was dropped, and it does not establish the server's billing rules.
https://github.com/lidge-jun/opencodex/blob/v2.80.0/src/adapters/openai-responses/passthrough.ts
https://github.com/lidge-jun/opencodex/blob/v2.80.0/src/lib/provider-client-headers.ts
Questions for maintainers
- For an actual proxied approval review, is the incoming model already Sol when the request reaches OpenCodex, or is it still
codex-auto-review? - Does the catalog actually loaded by the Codex runtime contain the default reviewer model?
- Are genuine Codex-generated reviewer markers and parent-request correlation metadata preserved across both HTTP and WebSocket paths?
- Is this a catalog-selection issue, runtime compatibility issue, request-forwarding issue, or statistics-classification issue?
I am not proposing to work around this by statically marking ordinary requests as Guardian, fabricating reviewer metadata, or disabling safety approvals.
Questions for maintainers
- For an actual proxied approval review, is the incoming model already Sol when the request reaches OpenCodex, or is it still
codex-auto-review? - Does the catalog actually loaded by the Codex runtime contain the default reviewer model?
- Are genuine Codex-generated reviewer markers and parent-request correlation metadata preserved across both HTTP and WebSocket paths?
- Is this a catalog-selection issue, runtime compatibility issue, request-forwarding issue, or statistics-classification issue?
I am not proposing to work around this by statically marking ordinary requests as Guardian, fabricating reviewer metadata, or disabling safety approvals.
Redacted configuration
The following is a snippet from Codex's `config.toml` (**TOML, not JSON**):
approvals_reviewer = "auto_review"
approval_policy = "on-request"
sandbox_mode = "workspace-write"
# auto_review_model = "gpt-6-luna"
openai_base_url = "http://127.0.0.1:10100/v1"
Model catalog:
- The relevant `auto_review_model_override` fields are empty/unset.
- Whether an entry with `slug: "codex-auto-review"` exists: [- Whether an entry with `slug: "codex-auto-review"` exists:
Yes. The OpenCodex-generated catalog contains a `codex-auto-review` entry.
It is marked with `visibility: "hide"` and supports low/medium/high/xhigh/max reasoning levels.
No `auto_review_model_override` is configured for the main model.]
- Effective `model_catalog_json` / profile: [- Effective `model_catalog_json` / profile:
`model_catalog_json = "C:\Users\<redacted>\.codex\opencodex-catalog.json"`
No explicit Codex profile was configured; the root/default configuration was used.]Checks
- I searched existing issues and documentation.
- I removed secrets, tokens, account details, request credentials, and personal data.
Activity
sungyongcho commentedon Oct 7, 2026
I checked this against my local checkout and could reproduce part of the forwarding concern, but not the end-to-end Sol fallback or billing impact.
Environment: OpenCodex package version 2.81.0, local commit 8c6e273, Linux; desktop package 26.1002.52244, bundled
codex-cli 0.162.0-alpha.2. This differs from the Windows/version combination in the report.Findings
model_catalog_jsonand mymodels_cache.jsoneach contain 44 model entries, with nocodex-auto-reviewentry. Neither has a non-emptyauto_review_model_override, and the root config has noauto_review_model. This is an on-disk observation, not proof of what an already-running Codex process loaded. It also differs from the report's updated configuration, which says its catalog contains the reviewer.http://127.0.0.1:10100/v1. I have not established whether that changes backend recognition in the installed runtime.selectForwardHeadershelper both omitx-codex-guardian. Both preservex-codex-parent-thread-id; the HTTP builder also preservesx-openai-subagent.normalizeForwardedClientHeaderName("x-codex-guardian")returnsnull, so the configurable client-header allowlist does not admit it either.codex-auto-reviewmodel is routed unchanged to the canonical OpenAI provider. This does not establish which model Codex selects before making that request.Offline forwarding reproduction
From the repository root, run the following with Bun. It builds a request in memory and calls the WebSocket header-selection helper; it does not send a request, access real credentials, or change the live configuration. The synthetic marker is only a diagnostic input, not a proposed workaround.
Observed output:
{"httpGuardian":false,"wsGuardian":false,"httpParent":true,"wsParent":true,"httpSubagent":true,"configAllowsGuardian":false}Existing-test result and limits
I also ran:
Result: 1 passed, 1 failed. The router case passed. The catalog case failed at line 174, expecting
gpt-5.4to be absent; the resulting catalog still contained the seeded entries, includingcodex-auto-review. Its later assertion expects the reviewer to be removed, but execution never reached that assertion. This run therefore does not reproduce reviewer removal or establish why my live catalog lacks it. I made no source changes.The remaining discriminating check is one genuine Auto-review request: determine the effective loaded catalog, the model selected before proxy ingress, and whether Codex-generated reviewer markers survive the full HTTP/WebSocket path. My disk config says
approvals_reviewer = "user", while this chat's effective execution policy reportsauto_review, so the disk setting alone is insufficient to classify the session.These findings do not establish a Sol fallback, a live Guardian-header capture, or any effect on plan usage/billing.
sungyongcho commentedon Oct 8, 2026
Follow-up: reproduced with genuine Auto-review requests
I ran real approval reviews against the ChatGPT backend with the Codex binary bundled in the desktop app (
codex-cli 0.162.0-alpha.2) and ChatGPT auth. Settings:approvals_reviewer = "auto_review",approval_policy = "on-request",sandbox_mode = "workspace-write", parent modelgpt-6.1-sol/high. Each arm used its own isolatedCODEX_HOME. In place of OpenCodex I put a local recording reverse proxy that forwarded requests unchanged tochatgpt.com/backend-api/codexand logged only the model, the effort, and the reviewer marker headers and metadata keys. Each run gave one task that needed one escalated command, so each arm produced exactly one approval review.model_catalog_jsonx-openai-subagent: guardian)x-codex-guardianparent_response_idguardian_credits_requestedon parent turnscodex-auto-review)…/v1gpt-6.1-sol/lowcodex-auto-reviewrow…/v1codex-auto-review/low…/backend-api/codexcodex-auto-review/lowreviewertrueAll upstream responses were 200. Arm A reproduces the report exactly. The two causes are independent.
1. The catalog drops the reviewer (OpenCodex)
isUnsupportedOpenAiNativeSluginsrc/codex/catalog/metadata.tstreats every baregpt-/codex-slug outsideNATIVE_OPENAI_MODELSas unsupported.CANONICAL_NATIVE_CATALOG_CONTENT_POLICYinsrc/codex/catalog/build-entries.tsappliesunsupportedNativeEntries: "drop". Every catalog writer therefore removescodex-auto-review. This is intentional and has tests:Gap Bintests/codex-integration/codex-catalog-sync-hardening.test.tsasserts// legacy dropped, and it has been in place since 13c03e6 (v2.6.2).When
model_catalog_jsonis set, Codex uses aStaticModelsManager. That file is the whole catalog andmodels_cache.jsonis never read. This probably explains the report's note that the catalog "contains" the reviewer:models_cache.jsondoes contain a hiddencodex-auto-reviewrow with low–max effort, but Codex never reads it. With the row missing,select_review_model(codex-rs/ext/guardian-reviewer/src/model.rs) falls back to the parent slug and choosesLow.2. The
/v1base URL suppresses the reviewer markers (Codex gate, triggered by our injected URL)ext/guardian-v2/src/sync_reviewerseedsx-codex-guardian: reviewerasCodexResponsesHeadersfor the preferred review model.core/src/client.rsapplies those headers only whensupports_codex_backend_routes()is true, which requires the provider to beopenaiand the base URL to be unset or end in/backend-api/codex. The same gate controlsparent_response_idandguardian_credits_requested. OpenCodex injectsopenai_base_url = "http://127.0.0.1:<port>/v1"(src/codex/inject/config-toml.ts), so Codex never generates the markers at all. The forwarding gap from my earlier comment (x-codex-guardianis missing from the HTTP and WS forward lists and from the client-header allowlist) is real. With the injected config it is currently unreachable. Arm C is a hypothetical configuration, because OpenCodex does not serve a/backend-api/codexingress.What this means for a fix
codex-auto-reviewrow fixes reviewer model selection (arm B). One way is to exempt Codex control-plane ids from the unsupported-native drop and backfill the row from the pinned upstream entry. The router already pins this id to canonical OpenAI (fix(router): pin Codex auto review to OpenAI #833). Upstream acceptedcodex-auto-reviewwithout the marker.x-codex-guardianon both HTTP and WS. This needs its own design. Fabricating markers is not an option.Not established
Billing impact is still not established. The rate-limit headers in the responses are coarse snapshots and did not distinguish the arms. I also have no evidence on whether
codex-auto-reviewwithout the marker is treated as a free review.The
Gap Bfailure I reported earlier no longer reproduces. The test passes on8c6e2736fe.Ingwannu commentedon Oct 8, 2026
Current-source triage at dev bbab02a confirms separate leads, not one proven cause: the shared HTTP/WebSocket forward-header list omits x-codex-guardian and the configurable metadata allowlist does not admit it; the router explicitly preserves an incoming codex-auto-review model. The catalog sync fixture also explicitly expects that legacy reviewer row to be dropped, but the reported test failed before reaching that assertion. Neither a pinned source snapshot nor an on-disk catalog establishes what the installed Codex process actually selected/loaded.
The official Auto-review documentation describes a separate reviewer for eligible approval prompts and effective policy/session selection. It does not establish the reviewer wire-model fallback or billing treatment for this proxied request. We should not infer quota/billing from a Sol/low statistics label or a synthetic missing-header test.
The useful next evidence is metadata from one already-occurring genuine approval: exact installed Codex binary/version; effective reviewer overrides and whether its loaded catalog contains codex-auto-review; incoming requested model/effort before routing; and presence/absence (names/booleans only) of genuine reviewer/parent markers at ingress and final HTTP or WebSocket egress. Please omit bearer/header values, parent IDs, prompts and full config/session files. Distinguish /v1 backend recognition from catalog selection and from forwarding before changing any of them. No reviewer override, fabricated Guardian metadata, live request or approval-policy change was made here.
sungyongcho commentedon Oct 8, 2026
@Ingwannu Thanks for the triage. I think your comment and my follow-up above (posted a few minutes earlier) crossed. That follow-up already covers the binary, the loaded catalog, the incoming model and effort, and how the three factors separate. This run fills the one gap left: genuine approvals through an actual OpenCodex instance, with marker presence recorded at both ingress and final egress, on both transports.
Setup
bbab02ab27, run as an isolated instance: separateHOME,OPENCODEX_HOMEandCODEX_HOME, and a different port.https://chatgpt.com/backend-api/codex).codex-cli 0.162.0-alpha.2, signed in with ChatGPT, withapprovals_reviewer = "auto_review",approval_policy = "on-request"andsandbox_mode = "workspace-write".auto_review_modelin the config, andauto_review_model_overridewas null on every catalog row.--preloadhook recorded, for/responsesonly, at ingress (Bun.serve) and at egress (fetchand the upstreamWebSocket): model, effort, and header and metadata-key presence as booleans. It recorded no values.Loaded catalog. I seeded the isolated
CODEX_HOMEcatalog with the upstreamcodex-auto-reviewrow (hidden, low–max) and ran OpenCodex's ownsyncCatalogModels. The result went from 47 rows to 9, andcodex-auto-reviewwent from 1 to 0. Codex then loaded this file throughmodel_catalog_json.Reviewer request (
x-openai-subagent: guardian), ingress → egressx-codex-guardianin / outx-openai-subagent,x-codex-parent-thread-idin / outparent_response_idin / outgpt-6.1-sol/ lowgpt-6.1-sol/ lowcodex-auto-review/ lowcodex-auto-review/ lowgpt-6.1-sol/ lowgpt-6.1-sol/ lowcodex-auto-review/ lowcodex-auto-review/ lowupstreamWebsocket: false)gpt-6.1-sol/ lowgpt-6.1-sol/ lowcodex-auto-review/ lowcodex-auto-review/ lowIn every arm the parent turns were
gpt-6.1-sol/ high at both ends, andguardian_credits_requestedwas absent at both ends. Every upstream HTTP response was 200.How the three factors separate
codex-auto-review/low (E, G, I). The proxy never changed the model on any path./v1backend recognition (Codex gate). Under/v1, Codex never generatesx-codex-guardian,parent_response_idorguardian_credits_requested. They are false at ingress in every arm, whatever the catalog. In my earlier recorder run, the same Codex binary with a base URL ending in/backend-api/codexdid generate all three.x-openai-subagent,x-codex-parent-thread-idandclient_metadatawere preserved on both WS and HTTP egress. Thex-codex-guardianomission in the forward list is real, but these runs cannot exercise it, because no genuine marker ever reaches ingress.So the observed Sol/low is caused by factor 1 alone. Fixing it is independent of factors 2 and 3. Restoring the markers would need factor 2, a backend-route ingress, together with factor 3.
Not established: billing or quota treatment for either reviewer model.
I wrote nothing to the live install. The isolated instance never connected to the live proxy port (verified with
strace -e connect), and its Codex-config write was refused by theservice-homeownership gate, which is why I ran the sync by hand into the isolated home.lidge-jun commentedon Oct 8, 2026
Fixed on
devby #6727 (squashbf9ecf3d79). Thanks @iWishingStar for the report and @sungyongcho for the A/B reproduction, which made the cause unambiguous.Cause (confirmed). Codex chooses the Auto-review model in
select_review_model(codex-rsext/guardian-reviewer/src/model.rs, unchanged on upstreammainat ec8251c54). It looks upcodex-auto-reviewin the loaded catalog, and when the row is absent and no reviewer override is set, it falls back to the parent task model at Low. OpenCodex writesmodel_catalog_json, which is Codex's entire catalog, and every catalog writer droppedcodex-auto-reviewas an unsupported barecodex-slug. That drop dates from 13c03e6 (v2.6.2), where it kept legacy rows out of the picker. So reviews through OpenCodex ran asgpt-6.1-sol/ low. Arm A versus arm B in the reproduction above isolates exactly this.Fix. When the final catalog contains an ordinary native OpenAI row, OpenCodex now keeps exactly one hidden
codex-auto-reviewrow. It reuses a live or persisted row when one exists and otherwise backfills from the pinned upstream entry. The row keeps upstream's reasoning ladder, so Codex picks Low. It stays out of the picker, subagent choices, Desktop lists and/v1/models, and provider or rootautoReviewModeloverrides keep their precedence. Reserve-only and native-less catalogs omit it, so Codex still falls back to the task model there. The router already pins this id to canonical OpenAI (#833).Not changed. Codex generates
x-codex-guardian,parent_response_idandguardian_credits_requestedonly when its base URL ends in/backend-api/codex. OpenCodex injects a/v1base URL, so these markers are never produced on the proxied path. This fix neither forwards nor synthesizes them; carrying them would need a separate design. Whether a marker-lesscodex-auto-reviewrequest is treated like a native review for plan usage is still unknown.The fix ships with the next release after it reaches
dev. After updating, a catalog sync (or restarting the proxy) rewrites the catalog. Fully quit and reopen Codex so it reloads the catalog.gateway: Codex's model list keeps codex-auto-review when the user pic…