Skip to content

Improve timeout errors on the llm command - #915

Open
jviotti wants to merge 2 commits into
mainfrom
better-timeout
Open

jviotti wants to merge 2 commits into
mainfrom
better-timeout

Conversation

@jviotti

@jviotti jviotti commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Signed-off-by: Juan Cruz Viotti jv@jviotti.com

Review in cubic

Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 5 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/command_llm.cc">

<violation number="1" location="src/command_llm.cc:788">
P2: `--json` exposes the timeout duration only inside the prose `error` field, so callers cannot consume it structurally. Store the duration on `LLMTimeoutError` and emit a dedicated JSON field instead of interpolating it into `what()`.

(Based on your team's feedback about structured exception data.)</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread src/command_llm.cc
// What the backend says about a failure is its own business, so whether the
// time ran out is read off the clock rather than off the message
if (std::chrono::steady_clock::now() - started >= timeout) {
throw LLMTimeoutError{"The completion request did not answer within " +

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: --json exposes the timeout duration only inside the prose error field, so callers cannot consume it structurally. Store the duration on LLMTimeoutError and emit a dedicated JSON field instead of interpolating it into what().

(Based on your team's feedback about structured exception data.)

View Feedback

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At src/command_llm.cc, line 788:

<comment>`--json` exposes the timeout duration only inside the prose `error` field, so callers cannot consume it structurally. Store the duration on `LLMTimeoutError` and emit a dedicated JSON field instead of interpolating it into `what()`.

(Based on your team's feedback about structured exception data.) </comment>

<file context>
@@ -777,7 +777,21 @@ auto sourcemeta::jsonschema::llm(const sourcemeta::core::Options &options)
+    // What the backend says about a failure is its own business, so whether the
+    // time ran out is read off the clock rather than off the message
+    if (std::chrono::steady_clock::now() - started >= timeout) {
+      throw LLMTimeoutError{"The completion request did not answer within " +
+                                std::to_string(timeout.count()) + " seconds",
+                            std::string{url}};
</file context>

Comment thread src/command_llm.cc
sourcemeta::core::HTTPResponse response;
try {
response = request.send();
} catch (const sourcemeta::core::HTTPError &) {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: We probably need a better upstream error for this...

@augmentcode

augmentcode Bot commented Oct 6, 2026

Copy link
Copy Markdown
🤖 Augment PR Summary

Improves timeout reporting for the llm command.
Measures completion-request duration without adding retries.
Introduces LLMTimeoutError with the configured timeout and endpoint URL.
Adds human-readable guidance while keeping --json errors structured.
Registers a Unix CI test for non-responsive endpoints in both output modes.
Removes the Doubleword-specific latency note from the documentation.

🤖 Was this summary useful? React with 👍 or 👎

@augmentcode augmentcode Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 2 suggestions posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

Comment thread src/command_llm.cc
} catch (const sourcemeta::core::HTTPError &) {
// What the backend says about a failure is its own business, so whether the
// time ran out is read off the clock rather than off the message
if (std::chrono::steady_clock::now() - started >= timeout) {

@augmentcode augmentcode Bot Oct 6, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you check the silent-endpoint case on macOS at src/command_llm.cc:787? The Darwin backend leaves timeoutIntervalForRequest at its 60-second default, so the request fails before the default 120-second timeout and bypasses both LLMTimeoutError and the new guidance.

Severity: medium

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

Comment thread src/command_llm.cc
// What the backend says about a failure is its own business, so whether the
// time ran out is read off the clock rather than off the message
if (std::chrono::steady_clock::now() - started >= timeout) {
throw LLMTimeoutError{"The completion request did not answer within " +

@augmentcode augmentcode Bot Oct 6, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you check the late response-size failure case on Windows at src/command_llm.cc:788? WinHTTP uses per-operation timeouts, so a slowly arriving response can exceed the 16 MiB limit after timeout; this discards the actual response-size HTTPError and incorrectly advises waiting longer.

Severity: medium

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant