Skip to content

fix(sim-cli): report an embedded SimApiError with its own exit code - #8462

Merged
waleedlatif1 merged 2 commits into
stagingfrom
fix/cli-embedded-exit-codes
Sep 30, 2026
Merged

waleedlatif1 merged 2 commits into
stagingfrom
fix/cli-embedded-exit-codes

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

The embedded CLI now reports a SimApiError with that error's own exit code, as the installed CLI already does.

renderEmbeddedError in packages/sim-cli/src/embed.ts returned 1 for every SimApiError and ignored error.exitCode. The terminal renderer (terminal.ts) returns it. So when a workspace operation wait timed out (exit 4, OPERATION_WAIT_TIMEOUT) or the operation needed destination configuration (exit 3, REQUIRES_CONFIGURATION), an in-process run reported a generic failure. The installed CLI reported the real outcome. Now both report the same code. The stderr text is unchanged.

Exit codes this surfaces

  • The only SimApiErrors that carry a code other than 1 are the workspace-operation outcomes in workspace-operation-wait.ts:
    • 3 when the operation requires configuration;
    • 4 when the wait times out.
  • Wrappers that rethrow an error keep its code.
  • Soft exits such as workflows runs wait (0/1/2/3/4) already reached embedded callers through the embed context and are unaffected.

Consumers
Every caller of runEmbeddedCli branches only on zero vs non-zero:

  • the Sim sim_cli tool handler;
  • agent-CLI sink and curation;
  • the Chat worker's result shaping and pipeline stages.

A 3 or 4 that used to arrive as 1 is still a failure to each of them. A caller that wants to tell the outcomes apart, such as a wait that continues after "still running", can now read the real code.

Test plan

  • New embed test: an operation wait that times out returns exit 4, the operation receipt on stdout, and OPERATION_WAIT_TIMEOUT on stderr. It failed with exit 1 before the fix and passes after.
  • sim-cli suite, bun run lint, type-check (sim-cli and apps/sim), bun run check:audits, and the apps/sim agent-CLI and tool-handler suites.

Follow-ups (not in this PR)

Detached synchronous runs. A workflows run that submits and then polls in bounded slices, so no single request stays open for the whole run. It is not built on async: true, because a queued run is a different run from a synchronous one:

  • it gets the async execution timeout instead of the plan's synchronous limit;
  • it uses the async rate-limit bucket and queue concurrency;
  • a time-based pause keeps polling instead of returning;
  • the run resource carries no output for a failed or paused run, while the synchronous response carries the partial output.

It needs:

  • a server option to queue a run under the caller's synchronous execution limits and rate bucket;
  • the run resource returning partial output for failed and paused runs.

The CLI would then submit with that option, poll with the loop workflows runs wait uses, print exactly what a synchronous run prints when it settles, and on timeout print the run record with exit 4.

Block attribution on the v2 run resource. The polled run resource reclassifies a failed run's stored error string, so it reports no failing block, and a timeout or usage-limit failure known only from its HTTP status is not recognized. Recording the classification (code, plus the block when one is to blame) with the log at error finalization, and applying it on read, would make it match the synchronous response. That change also needs:

  • updated public contract descriptions for code, blockId/blockName/blockType, and the run error (runs logged earlier and the resume response keep reclassifying);
  • tests that fail when the classification is dropped by log compaction or skipped on the background-execution fallback path.

The embedded renderer returned 1 for every SimApiError, so an operation
wait that timed out (exit 4) or needs configuration (exit 3) read as a
generic failure in-process while the installed CLI reported the real
status. Return error.exitCode, as the terminal renderer does.
@vercel

vercel Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Sep 30, 2026 10:32am UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium risk] Changes how the embedded CLI reports error exit codes.

The PR appears safe to merge.

Summary

The embedded CLI now returns a SimApiError’s exit code instead of always returning 1.

  • Adds a workspace-operation timeout test that checks exit code 4, the stdout receipt, and the stderr error code.
  • Makes the test’s timeout independent of runner scheduling by advancing a mocked clock when the status response is served.

Reviews (2) · Last reviewed commit: "test(sim-cli): make the embedded wait-ti..."

Comment thread packages/sim-cli/src/embed.test.ts Outdated
…ner speed

The test gave the wait 10 ms of real time, so a slow runner could reach
the deadline before the first status check and exit 4 without printing
the receipt. The clock now advances only when the stubbed status is
served, so the wait times out only after it has a receipt to print.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit c873ed2 into staging Sep 30, 2026
32 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/cli-embedded-exit-codes branch September 30, 2026 18:28

This branch was previously deployed

1 inactive deployment
Preview — 1634d0c0 Deployed Sep 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant