[Bug][Linux] native-main 503 when systemctl exists but the user bus is unreachable (#1612 regression gap)

Client or integration

Codex CLI

Area

Service lifecycle

Summary

OpenCodex 2.26.0 blocks every native OpenAI request on a systemd-less Linux container when systemctl is present on PATH but the user bus is unreachable. The proxy stays live, /healthz returns 200, and ocx ready reports {"ready":true,"status":"ready"}, but every native request fails locally with HTTP 503 before reaching upstream:

{"error":{"message":"OpenCodex local native-main profile maintenance is active; retry this request","type":"server_error","code":"server_is_overloaded"}}

This is the same admission fence as #1612, but that fix does not cover this environment. #1612 was closed by treating a systemctl spawn failure as absent. In this container systemctl spawns fine — it exits non-zero because there is no user bus:

$ command -v systemctl
/usr/bin/systemctl

$ systemctl --user show opencodex-proxy -p LoadState -p ActiveState -p FragmentPath -p NeedDaemonReload
Failed to connect to user scope bus via local transport: $DBUS_SESSION_BUS_ADDRESS and $XDG_RUNTIME_DIR not defined
EXIT=1

inspectSystemd() only maps spawnFailed to absent; a non-zero exit still returns unknown, which re-triggers the native-main block. The /.dockerenv marker is present and PID 1 is tini, so this is exactly the externally-supervised foreground shape src/service.ts recommends (Service: unsupported in Docker, "run ocx start directly").

Reproduction

Debian 13 container (PID 1 = tini, /.dockerenv present) with systemd installed but no user bus: /run/user/$UID empty, XDG_RUNTIME_DIR and DBUS_SESSION_BUS_ADDRESS unset.
npm install -g @openai/codex@0.148.0 @bitkyc08/opencodex@2.26.0
codex login (ChatGPT), confirm native codex exec works.
ocx start, then ocx sync.
Send a native request:

$ curl -s -X POST http://127.0.0.1:10100/v1/responses \
    -H 'Content-Type: application/json' \
    -d '{"model":"gpt-5.5","input":"hi","stream":false}'
{"error":{"message":"OpenCodex local native-main profile maintenance is active; retry this request",...}}

codex exec surfaces this as a misleading message, which is what a user actually sees:

ERROR: Selected model is at capacity. Please try a different model.

ocx restart does not clear it; ocx ready still reports ready:true while every request 503s.

Control flow

systemd installed, no user bus (Docker/tini)
-> inspectSystemd() runs `systemctl --user show` -> spawn SUCCEEDS, exit 1 (no bus)
-> service-manager-probe.ts: !spawnFailed, status !== 0 -> unknown("systemctl show exited 1: ...")
-> ownership-preflight.ts: manager.kind === "unknown" -> ownership: "unknown"
-> server/index.ts: blockNativeMainStartupForUnownedServiceHome("ownership-unknown")
-> native-profile-startup.ts: isNativeMainTrafficBlocked() === true
-> auth-context.ts: throw CodexMainProfileDrainingError -> 503

The #1612 fix (if (shown.spawnFailed) return { kind: "absent" }) sits one line above the branch that fires here.

Verified workaround

Removing systemctl from the proxy's PATH makes the probe hit the spawnFailed path fixed in #1612, and native traffic recovers immediately:

$ mkdir -p ~/ocx-nosysd && for f in /usr/bin/*; do b=$(basename $f); [ "$b" = systemctl ] && continue; ln -sf $f ~/ocx-nosysd/$b; done
$ ocx stop
$ setsid env PATH=$HOME/ocx-nosysd:/usr/local/bin ocx start &

$ curl -s -X POST http://127.0.0.1:10100/v1/responses -H 'Content-Type: application/json' -d '{"model":"gpt-5.5","input":"hi"}'
{"detail":"Input must be a list"}          # request reached the handler; no 503

$ codex exec "Reply with exactly: PROXY_OK"
PROXY_OK                                    # tokens used 3,922

Same binary, same config, same login — the only variable is whether systemctl is reachable on PATH. That isolates the probe result as the sole cause.

Setting XDG_RUNTIME_DIR=/run/user/1000 does not help: the directory exists but has no bus socket, so systemctl still exits non-zero ("Failed to connect to user scope bus via local transport: No such file or directory").

Suggested fix

Treat "the question never reached a user bus" the same as an unaskable manager, while keeping fail-closed behavior for real evidence.

In inspectSystemd(), classify a no-bus failure as absent rather than unknown. Prefer matching the bus-connection failure specifically (stderr Failed to connect to ... bus), not any non-zero exit.
Guard it with the container/foreground signal already used elsewhere (/.dockerenv, PID 1 not systemd), so a real desktop Linux box with a transient bus problem still returns unknown.
Keep every other path fail-closed: a present unit, NeedDaemonReload=yes, malformed state, conflicting mirrors, or foreign-home evidence must still yield unknown/foreign.

Regression coverage worth adding:

Docker + systemctl present + bus unreachable (exit 1) -> native-main admits traffic.
Docker + systemctl present + unit actually installed for a foreign home -> still blocked.
Non-Docker Linux + bus unreachable -> still unknown (no widening on real hosts).

Two secondary issues

1. The user-facing error is wrong. A local admission fence is reported to the CLI as ERROR: Selected model is at capacity. Please try a different model. That sends users to change models or wait for quota, when the cause is local and the fix is ocx restore or the PATH workaround. The 503 body is accurate; the CLI-visible mapping is not.

2. ocx ready reports ready while every request 503s. {"ready":true,"status":"ready"} with a 100% failure rate makes readiness unusable as a health signal in exactly the automated/container context where it matters most. Readiness could reflect isNativeMainTrafficBlocked() when native-main is the only configured route.

Also worth noting: ocx status correctly says Service: unsupported in Docker and ocx doctor warns AT RISK after restart (no viable background service; run 'ocx restore'). The product already knows it is in an unsupported-service environment at the moment it blocks traffic for not being able to prove service ownership.

Environment

OpenCodex 2.26.0 (npm global), Codex CLI 0.148.0
Debian GNU/Linux 13 (trixie), kernel 6.12.94, x86_64
Container: /.dockerenv present, PID 1 = tini, /run/user/1000 empty
systemd installed (/usr/bin/systemctl), user bus unreachable
Bundled Bun 1.3.14 runtime, port 10100
Auth: codex login (ChatGPT), native codex exec verified working before ocx sync

🔗 Related PRs

#306 - feat(windows): add tray controls and restart-safety diagnostics [merged]
#403 - feat(grok): auto-configure Grok Build and harden strict-client compatibility [merged]
#569 - Add post-sync readiness endpoint and bounded ocx ready wait [merged]
#576 - fix(codex): warn about stale Codex app-servers after a catalog write [merged]
#770 - fix: research-batch reliability and security-boundary issues [closed]
#1160 - fix(service): preserve supervision across restart [open]
#1647 - fix(tray): let the service wrapper exit 0 on an already-live proxy [open]
#1721 - refactor(adapters): centralize runtime adapter authority - Part 1 [merged]

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

Create Plan

🧪 Issue enrichment is currently in open beta.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

리뷰 · 우선순위 74 / 80

Docker/tini 컨테이너에서 systemctl은 PATH에 있는데 user bus는 없는 환경이다. ocx start와 ocx sync는 되고 /healthz·ocx ready도 ready인데, 네이티브 OpenAI 요청만 로컬 HTTP 503으로 막힌다. 본문은 OpenCodex local native-main profile maintenance is active; retry this request이고, Codex CLI는 이걸 Selected model is at capacity로 보여 준다. #1612가 고친 건 systemctl spawn 실패를 absent로 보는 길이었고, 이번엔 spawn은 성공하고 exit 1이라 그 우회를 못 탄다. PATH에서 systemctl만 빼면 바로 살아나는 게 원인 분리이고, 기본 native 경로가 컨테이너 설치 직후부터 막혀서 74다. 데스크톱 systemd 호스트의 기본 설치/재부팅 사고는 아니라 80은 아니다.

막히는 지점은 src/service-manager-probe.ts의 inspectSystemd()다. systemctl --user show opencodex-proxy -p LoadState -p ActiveState -p FragmentPath -p NeedDaemonReload를 치고, shown.spawnFailed일 때만 { kind: "absent" }다. shown.status !== 0이면 주석 그대로 “the question never reached the bus”인데도 unknown("systemctl show exited 1: …")으로 돌려보낸다. 리포터 stderr인 Failed to connect to user scope bus via local transport가 바로 그 분기다. XDG_RUNTIME_DIR만 채워도 소켓이 없으면 같은 exit 1이 난다.

그 unknown이 서비스홈을 미소유로 고정한다. src/integrations/native/ownership-preflight.ts의 inspectNativeCodexOwnership()은 manager.kind === "unknown"이면 { ownership: "unknown" }이다. src/server/index.ts의 startServer()는 inspectStartupOwnership() 결과가 owned가 아니면 복구 라이프사이클을 안 타고 blockNativeMainStartupForUnownedServiceHome("ownership-unknown")으로 간다. src/codex/native-profile-startup.ts 스냅샷이 blocked가 되면 isNativeMainTrafficBlocked()가 true고, src/codex/auth-context.ts가 CodexMainProfileDrainingError를 던져 Responses가 503을 만든다. ocx restart는 같은 프로브를 다시 돌릴 뿐이라 풀리지 않는다.

제품은 이미 이 환경을 서비스 불가로 알고 있다. src/service.ts의 platformOps()는 /.dockerenv가 있으면 “Run ocx start directly”로 서비스 설치를 거절하고, status는 unsupported in Docker를 준다. doctor도 백그라운드 서비스가 없다고 경고한다. 그런데 ownership 프로브는 그 신호를 안 보고, systemctl이 PATH에만 있으면 질문을 버스에 던진 뒤 실패를 unknown으로 승격한다. 서비스가 아예 없는 foreground 컨테이너에서 “소유를 증명 못 함”으로 native-main을 닫는 셈이다. 옆 이슈 #1939도 같은 systemctl show exited 1 / D-Bus 실패 계열이다.

부수 증상 두 개도 코드와 맞다. ocx ready는 /readyz의 동기화 게이트만 보고 isNativeMainTrafficBlocked()는 안 본다. 그래서 native-main이 유일한 경로여도 {ready:true}가 나온다. CLI의 “model at capacity”는 로컬 admission 펜스를 업스트림 용량 부족으로 오역한다. 503 바디는 맞고, 사용자에게 보이는 문장이 틀리다.

해결방안

inspectSystemd()에서 버스에 질문이 도달하지 못한 실패를 unknown이 아니라 absent로 분류한다. 아무 non-zero나 열지 말고, stderr의 Failed to connect to … bus / $DBUS_SESSION_BUS_ADDRESS and $XDG_RUNTIME_DIR not defined처럼 버스 연결 실패만 본다. 이 완화는 /.dockerenv 또는 PID 1이 systemd가 아닌 컨테이너/foreground 신호와 같이 걸어서, 실제 데스크톱에서 버스가 잠깐 죽은 경우는 지금처럼 unknown으로 남긴다. unit이 있거나 NeedDaemonReload=yes이거나 파편/홈이 어긋나면 계속 fail-closed다. src/service.ts가 이미 가진 Docker 신호를 ownership-preflight까지 넘기면, 서비스 불가 환경에서 “소유 미증명”으로 native를 닫지 않게 맞출 수 있다. 회귀는 세 갈래다. Docker + systemctl 있음 + 버스 없음 → native-main 통과. Docker + 다른 홈 unit 실설치 → 계속 차단. 비Docker 호스트 + 버스 실패 → 계속 unknown. ready는 native-main이 유일한 경로일 때 isNativeMainTrafficBlocked()를 반영하고, CLI 오역은 로컬 503을 용량 부족으로 바꾸지 않게 한다.

이 댓글은 grok-bot이 작성했습니다

Triage note: this issue is already owned by open PR #2029, not unclaimed.

That PR rewrites the exact function this needs changed — the non-zero branch of inspectSystemd in src/service-manager-probe.ts — for #1939. It currently maps two deterministic no-bus messages to absent:

Failed to get D-Bus connection: No such file or directory
System has not been booted with systemd

and deliberately keeps everything else unknown, with a test that pins that choice. Your stderr is a third shape:

Failed to connect to user scope bus via local transport: $DBUS_SESSION_BUS_ADDRESS and $XDG_RUNTIME_DIR not defined

so it currently falls on the unknown side and the fence stays armed.

Why this is one decision, not two

#1939 and this issue are the same classifier with different symptoms. There, an unprovable service home refuses a sync. Here it closes native-main for the process lifetime:

inspectSystemd -> unknown
  -> ownership-preflight.ts:155      ownership: "unknown"
  -> server/index.ts                 blockNativeMainStartupForUnownedServiceHome("ownership-unknown")
  -> native-profile-startup.ts       isNativeMainTrafficBlocked() == true
  -> auth-context.ts                 CodexMainProfileDrainingError -> 503

Opening a second PR against that function would mean two people making the same fail-closed, security-adjacent call in two places, with whichever lands second silently overwriting the first. So this is deliberately not getting its own code change.

What is happening instead

#2029 is CHANGES_REQUESTED for a good reason — a missing bus does not prove the unit file is absent, and returning absent on stderr alone would let the preflight overwrite a foreign installation. I have posted the containment for that on #2029: consult the unit file on disk when the bus cannot answer, which the same function already knows how to do a few lines below, plus the regression that keeps a foreign definition fenced.

Your case is included in the suggested tests, so when #2029 lands with the containment this closes with it.

Your workaround stands meanwhile

Removing systemctl from the proxy's PATH forces the spawnFailed → absent branch from #1612 and restores native traffic. Thank you for isolating that with a single variable — it is what made the classifier the obvious suspect rather than the fence itself.

Keeping this open and linked rather than closing it as a duplicate: the 503 symptom is distinct enough that someone hitting it should be able to find this thread.

fix(probe,codex): close the fail-open and the refcount bug an audit f…

fix(codex): let the fence pay for itself, so the probe cannot pay twice

Fixed on dev as of #2130 (890d339de).

The probe no longer treats an unanswerable bus as evidence about ownership. When systemctl --user fails with a bus-unreachable error, the classification reads the unit file off disk instead — the disk needs no bus, and the homes a unit names are what ownership is actually decided on. No unit means nothing can own this home, so native-main is not fenced and your GPT requests stop returning 503.

The part that took the longest was making sure this does not fail open in the other direction, and it is worth being explicit about it. With the bus down systemctl cannot see a foreign unit either, so "no answer" must not be read as "no owner". The first cut of this checked only ~/.config/systemd/user and an audit proved it: a foreign unit in ~/.local/share/systemd/user came back as owned on a host someone else owned. The shipped version walks every directory systemd itself honors, including the XDG_CONFIG_HOME and XDG_DATA_HOME overrides, and refuses rather than choosing when two units claim the proxy.

A limitation worth knowing. systemd localizes these stderr strings, so a non-English host will not match and keeps the old unknown. That errs toward fencing rather than admitting, which is the safe direction, but it does mean this may not reach you if your system messages are not in English. Forcing LC_ALL=C on the probe would remove the caveat and is the obvious follow-up.

Could you send a follow-up probe? Two things, once you are on a build with this:

systemctl --user show opencodex-proxy -p LoadState -p FragmentPath 2>&1
ls -la ~/.config/systemd/user/opencodex-proxy.service ~/.local/share/systemd/user/opencodex-proxy.service 2>&1

The stderr from the first tells us whether your message shape is matched; the second tells us which directory your unit actually lives in. If native requests still 503 after this, that pair is what identifies whether it is the locale gap or something else.

Closing manually — PRs here target dev, so GitHub's auto-close does not fire. Reopen if it recurs.

Credit where it is due: draft PR #2029 was already working the same function for #1939 and classifies two other bus messages the same way. That PR was not touched here, and the two changes agree in direction.

fix(probe): read the unit off disk when the session bus cannot answer

fix(probe,codex): close the fail-open and the refcount bug an audit f…

fix(codex): let the fence pay for itself, so the probe cannot pay twice

Navigation Menu

Repository navigation

[Bug][Linux] native-main 503 when systemctl exists but the user bus is unreachable (#1612 regression gap) #2114

Client or integration

Area

Summary

Reproduction

Control flow

Verified workaround

Suggested fix

Two secondary issues

Environment

리뷰 · 우선순위 74 / 80

해결방안

Why this is one decision, not two

What is happening instead

Your workaround stands meanwhile

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

[Bug][Linux] native-main 503 when systemctl exists but the user bus is unreachable (#1612 regression gap) #2114

Description

Client or integration

Area

Summary

Reproduction

Control flow

Verified workaround

Suggested fix

Two secondary issues

Environment

Activity

coderabbitai commented on Aug 19, 2026

lidge-jun commented on Aug 19, 2026

리뷰 · 우선순위 74 / 80

해결방안

lidge-jun commented on Aug 19, 2026

Why this is one decision, not two

What is happening instead

Your workaround stands meanwhile

lidge-jun commented on Aug 19, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions