Skip to content

Dashboard shows no data when ACL hardening cannot be verified (management API fails closed), plus a permanent false startup-protection warning and a 10s first paint #766

Closed

Description

@brunoflma

Client or integration

OpenCodex dashboard

Area

Dashboard

Summary

Three problems that leave the dashboard blank, slow, or misleading. The first is the severe one: on hosts where icacls hardening cannot be verified, the entire management API fails closed and the dashboard shows no data at all.

1. Management API fails closed when ACL hardening cannot complete.

initializeManagementAuthState() needs a token, and every path to one requires hardenSecretPath(..., { required: true }) to succeed — including readExistingToken(), so pre-creating the token file does not help. On this host hardenSecretDir() consumes its whole budget and returns ok: false, so the state becomes available: false and every /api/* request returns 503 {"error":"management API unavailable"}. The GUI loads and then shows nothing.

What makes this look like a bug rather than the host's fault:

  • icacls itself is not slow here. Running the same grant / /inheritance:r / verify sequence manually against a directory completes in ~220ms total, well inside the 5s budget, and spawning icacls from within bun measures ~35ms. I could not identify what consumes the budget.
  • Raising OPENCODEX_ACL_TIMEOUT_MS to 30000 does not help. Lowering it to 1000 just fails faster (and is a real ~4.7s saving per process start, since loadConfig() hardens dir+config+auth sequentially).
  • Elsewhere the codebase deliberately soft-fails ACL hardening so that it "cannot block OAuth logins or token refresh". The management token is the one place that fail-closes, and the cost is the entire dashboard.

2. The startup-health probe always times out, so a false warning is permanent.

startup-health-cache.ts spawns a full bun subprocess (__startup-health) every 30s with PROBE_TIMEOUT_MS = 5000. Here a bare bun -e 0 takes ~450ms and loadConfig() adds ~3s, so a subprocess that also loads the CLI entry never finishes inside 5s. The probe therefore always falls back to conservativeFallback() with diagnosticStale: true and the dashboard permanently shows "Could not read startup protection." — while the actual state is healthy (status: native, rebootSafe: true).

The stale-while-revalidate design correctly never blocks a request, but it also means a bun process is spawned and discarded every 30s forever and the warning never clears.

3. GET /api/settings gates the dashboard's first paint.

/api/settings calls resolveCodexRuntime() for "full alternative discovery (memoized) so newerAvailable warnings work". First call: 6-10s here. Subsequent calls: 2ms. dashboard-core-poll.ts fetches five endpoints in one Promise.all, so Status, Version, Uptime and Providers all stay blank and the header reads Offline until the slowest one returns. Opening the GUI shortly after a service start shows "Offline" for a long stretch while the proxy is already serving traffic.

4. Smaller: the request log has no surface filter.

/api/logs returns every entry regardless of surface, and the Logs page treats only surface === "claude" as Claude, so claude-desktop requests appear under the Codex tab. A ?surface= query parameter plus including claude-desktop in the Claude bucket fixes both ends.

Reproduction

Problem 1:

  1. On a host where icacls hardening cannot be verified, start the proxy. ~/.opencodex/service.log fills with ACL hardening timed out (ETIMEDOUT) ... (budget exhausted).
  2. GET http://127.0.0.1:10100/healthz -> 200.
  3. GET http://127.0.0.1:10100/api/logs -> 503 {"error":"management API unavailable"}.
  4. Open the dashboard: it renders but every panel is empty.
  5. Calling initializeManagementAuthState() directly reports reason: management token directory ACL hardening did not complete.
  6. Setting OPENCODEX_ADMIN_AUTH_TOKEN makes it available: true, source: environment — it short-circuits before any filesystem work.

Problem 2: leave the dashboard open. The startup-protection warning never clears, and a new bun process appears every 30s. /api/settings reports startupHealth.diagnosticStale: true indefinitely.

Problem 3: restart the service, then immediately load the dashboard and time how long it reads "Offline". Timing the five batch endpoints individually shows /api/settings at 6-10s cold and 2ms warm while the others answer in single-digit milliseconds.

Version

2.7.43

Operating system

Windows 11 (build 10.0.26200), npm -g install, scheduler service backend. A third-party antivirus does on-access scanning, which is relevant to problems 2 and 3 because both assume subprocess spawns are cheap.

Provider and model

Not provider-specific.

Logs or error output

[opencodex] ACL hardening timed out (ETIMEDOUT) - transient icacls stall; the volume may still
support per-user NTFS ACLs; ACL state unverified (budget exhausted) - continuing without NTFS ACL harden

GET /api/logs -> 503
{ "error": "management API unavailable" }

initializeManagementAuthState():
  available: false
  reason   : management token directory ACL hardening did not complete

Measured latency of the dashboard's Promise.all batch, cold:

/healthz                     233 ms
/api/providers                 6 ms
/api/settings              10 107 ms   <- gates the first paint
/api/sidecar-settings         16 ms
/api/shadow-call-settings      1 ms

Redacted configuration

{ "port": 10100, "hostname": "127.0.0.1", "claudeCode": { "enabled": true, "authMode": "proxy" } }

isApiAuthRequired(config) is false here (loopback hostname), so the session-issuing path is not the blocker — only state.available is.

Suggested fixes

  1. Treat unverified ACL state as a warning for the management token as well, the way the OAuth paths already do, rather than disabling the whole management API. Failing closed costs the user the entire dashboard on hosts where icacls is unreliable, which seems worse than proceeding with a warning.
  2. Collect startup health in-process where possible, or back off after repeated probe timeouts instead of re-spawning every 30s — and distinguish "could not determine" from "unprotected" in the UI.
  3. Return /api/settings without the discovery result and fill newerAvailable from a background refresh, or let the dashboard render as soon as /healthz answers rather than gating on the whole batch.
  4. Add ?surface= to /api/logs and include claude-desktop in the Claude bucket in the Logs page.

Checks

  • I searched existing issues and documentation.
  • I removed secrets, tokens, account details, request credentials, and personal data.

Activity

  1. github-actions commented on Jul 30, 2026

    @github-actions
    Contributor

    Automated translation bookkeeping — detected language: unknown.

  2. Wibias commented on Jul 30, 2026

    @Wibias
    Contributor

    [shipping-github] Research review

    Claim: When Windows ACL harden cannot verify, management API fails closed (blank dashboard); startup-health probe permanently stale; /api/settings cold path delays first paint.
    Checked against: dev@be177ea5

    Field Finding
    Still an issue on latest development? yes for ACL fail-closed + startup-health probe; first-paint partially mitigated
    Exact issue initializeManagementAuthState uses hardenSecret* with required: true; probe still spawns __startup-health with 5s timeout
    Fixed on development? partial — merged #740 progressive dashboard paint helps overview independence; ACL/management and startup-health probe logic still as reported. Claude surface bucketing in Logs GUI already includes claude-desktop
    Open PR covering this? none for ACL soft-fail / probe redesign
    Duplicate of? none (ACL family touches #612/#645 history)
    Security relevance possible — soft-failing management-token ACL is an availability vs secret-file hardening tradeoff
    Priority high — blank dashboard when proxy itself is healthy
    Verdict Needs fix on development

    Evidence: src/server/management-auth.ts required harden paths; 503 management API unavailable; src/server/startup-health-cache.ts PROBE_TIMEOUT_MS = 5000. #740 merged 2026-07-30 for progressive paint.
    Gaps: Did not reproduce icacls budget exhaustion on this host; root cause of budget burn still unknown per reporter.

  3. Wibias commented on Jul 30, 2026

    @Wibias
    Contributor

    [shipping-github] Security review

    Checked against: dev@be177ea5
    Scope: Management-token init fail-closed on Windows ACL harden timeout; soft-fail proposal

    Severity Area Finding Next
    medium (if soft-failed naively) secrets / local multi-user initializeManagementAuthState correctly fails closed when required ACL harden returns ok: false, yielding management API unavailable. Soft-failing that path without compensating controls can leave the admin token file under weaker NTFS isolation on shared Windows hosts — full management-plane access for other local users. Prefer env OPENCODEX_ADMIN_AUTH_TOKEN (already bypasses file harden), diagnose/fix the icacls stall, or soft-fail only with an explicit operator opt-in + persistent warning — not silently.
    low (current tip) availability vs confidentiality Current fail-closed matches structure/05_gui-and-management-api.md: confidentiality of the management credential wins over dashboard availability when harden cannot be verified. Keep fail-closed as default; document the env-token escape hatch for affected hosts.

    Summary: No auth bypass found. The dangerous change would be relaxing required ACL verification without a clear multi-user threat-model decision.
    Residual: Startup-health probe / first-paint items are operational, not security findings.

  4. Wibias commented on Jul 30, 2026

    @Wibias
    Contributor

    [shipping-github] Opened PR #782 to address this.

  5. added
    guiDashboard, tray, settings UI
    account-poolOAuth, credentials, Codex pool, quota, failover, plans
    proxyHTTP proxy, routing, reverse-proxy / management auth
    platformOS/service/tray/ACL (Windows-heavy, not Windows-only)
    on Aug 1, 2026
  6. brunoflma commented on Aug 2, 2026

    @brunoflma
    Author

    Re-tested on 2.10.0 after migrating from 2.8.0. The message is better — the failure reason now points at the escape hatch — but the fail-closed behaviour itself is unchanged, so the dashboard is still empty on this host without a manually provisioned token.

    Management token, with no OPENCODEX_ADMIN_AUTH_TOKEN set:

    available: false
    reason   : management token directory ACL hardening did not complete;
               set OPENCODEX_ADMIN_AUTH_TOKEN to use an environment token instead
    

    ~/.opencodex/admin-api-token is never created, and every /api/* request returns 503 {"error":"management API unavailable"}. Setting the environment variable still fixes it immediately, so the hint is accurate and genuinely useful — I just wanted to flag that the underlying behaviour did not change, in case the issue was closed on the assumption that it had.

    Timing, same host, 2.10.0:

    OPENCODEX_ACL_TIMEOUT_MS hardenSecretDir loadConfig() (3 runs)
    default (5000) 5008 ms 5747 / 5658 / 5642 ms — avg 5.68 s
    1000 1015 ms 1646 / 1661 / 1668 ms — avg 1.66 s

    The harden fails either way; the lower budget only makes it fail sooner. That is ~4 s saved on every bun process start, which is worth knowing for anyone else on a host where icacls verification does not complete.

    One behavioural change I noticed while measuring: hardenSecretDir now throws on timeout where it previously returned { ok: false, diagnostics }. Not a problem for me, but if any caller still relies on the soft-fail contract it would now propagate. The required: true call sites are the ones to check.

    For context on why the harden never completes here: icacls itself is not slow. Running the same grant / /inheritance:r / verify sequence by hand against a directory finishes in ~220 ms total, and spawning icacls from inside bun measures ~35 ms. I still cannot account for the budget being consumed, so I am not claiming this is opencodex's fault — only that fail-closed turns an unverifiable ACL into a completely empty dashboard, where every other consumer of the same helper degrades with a warning.

    Happy for this to stay closed if documenting the escape hatch is the intended resolution. If you would rather track the fail-closed behaviour separately, say the word and I will open a narrower issue for just that.

  7. added a commit that references this issue on Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    account-poolOAuth, credentials, Codex pool, quota, failover, plansbugSomething isn't workingguiDashboard, tray, settings UIplatformOS/service/tray/ACL (Windows-heavy, not Windows-only)proxyHTTP proxy, routing, reverse-proxy / management auth

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions