Skip to content

fix(openai-compat): answer a turn that ended on its error row as turn_failed, not an empty stop - #25555

Merged
iamwhatever merged 1 commit into
mainfrom
fix/openai-compat-error-turn-14695
Oct 29, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
fix/openai-compat-error-turn-14695

Conversation

@CrysisDeu

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: a POST /v1/chat/completions turn the backend could not run is reported as a failure, not as an empty successful completion.

When the ACP runtime cannot start a turn, the caller gets HTTP 200, content: "", finish_reason: "stop" and zero usage. The turn's failure only reaches the gateway's own log.

Why it matters

A failed turn looks the same as a model that chose to say nothing. A caller that retries gets empty replies forever, and a script or dashboard reading finish_reason records a normal completion. The issue records a Fargate task where this hid an AcpRuntimeError on every attempt and cost a full e2e round to find.

Not a goal

  • Changing what the dashboard transcript shows for a failed turn.
  • Inferring failure from "no content and no usage". An empty reply with no error row stays a 200 stop.

Intended? / Repro

Intended? No. docs/system-specs/modules/learn-cron-dashboard.md already says a failed turn must answer server_error instead of an empty reply, but only the pre-try failure row carries the meta.turn_failed flag the endpoint checks.

Repro (read on current main, 8f8cc940e7):

  • src/kiro_crew/dashboard/chat_turn/turn_failures.py:604 _on_error: an AcpRuntimeError raised by the session start (the issue's the default agent spec .../kirocrew.json is missing) is appended as slot.append("error", _err_text, "msg msg-err", meta=_err_meta), with no turn_failed key. _on_acp_error_card (turn_failures.py:402) and the other terminal arms do the same.
  • The cycle then ends with its done row.
  • src/kiro_crew/dashboard/openai_compat.py:649 _turn_failed only matches meta.turn_failed (set by chat_turn/queue_cycle.py:686 for an exception above the turn's try). _blocking_response (openai_compat.py:795-834) and _stream_response (openai_compat.py:686-725) skip every other row that is not chunk/assistant, so at done they answer 200 with content: "" and finish_reason: "stop".
  • Input: any completion request whose turn hits a runtime error at session start. The user sees {"choices": [{"message": {"content": ""}, "finish_reason": "stop"}]} with status 200.

What changed (motivation → approach → change)

Symptom: a failed turn answers 200 / stop. Cause: the readers ignore the turn's own error row. Change: read the error rows and answer the failure at done.

  • openai_compat._TurnOutcome: both readers feed every row to it. An error row that no later chunk/assistant text follows ends the turn as failed, with that row's redacted text as the reason.
    • A retry notice (meta.kind == TRANSIENT_RETRY_KIND) is not a failure.
    • An error followed by the reply the turn recovered with is a normal stop.
    • A meta.turn_failed row (failed before it started) holds whatever the queue behind it answers, as on main.
  • Blocking: 502 {"error": {"message": <error row, redacted>, "type": "server_error", "code": "turn_failed"}, "code": "turn_failed"}. The pre-try failure, which answered 500 "internal error", now gets the same named 502.
  • Streaming: flushes the text it held back for redaction, then sends that error object as the last frame and [DONE], with no finish_reason: stop chunk. The container front's stream.py already passes error frames through.
  • The slot.task.result() exception path is unchanged (500 internal error).
  • docs/system-specs/modules/learn-cron-dashboard.md: describes the rule and the response shapes; the old paragraph said the collectors read only chunk/assistant.

Backwards compatibility

A caller that got 200 with empty content for a failed turn now gets 502 (blocking) or an error frame (stream). The pre-start failure moves from 500 server_error to 502 turn_failed; error.type stays server_error. Successful turns, including empty ones, are unchanged.

Removes nothing: no key, field or route is removed.

Tests

test/test_openai_compat.py, TestTurnErrorRow (new), both transports where it applies:

  • test_blocking_answers_502_naming_the_error_row: error row then done → 502, code: turn_failed, message names the error.
  • test_stream_ends_on_the_error_frame_without_a_stop: last frame is the turn_failed error, no finish_reason: stop, ends with [DONE].
  • test_the_error_message_is_redacted: a credential in the error row does not reach the response.
  • test_a_reply_after_a_recovered_error_is_a_normal_stop
  • test_an_error_after_partial_text_still_fails
  • test_a_retry_notice_alone_is_not_a_failure
  • test_an_empty_reply_with_no_error_row_stays_a_200_stop
  • test_a_turn_that_failed_before_it_started_holds_over_later_text

test/test_dashboard_chat_refusal.py::test_a_setup_error_is_a_server_error_on_the_openai_endpoint: now expects 502 with code: turn_failed and the row's message.

On main the blocking, stream, redaction and partial-text tests fail, because the error row is dropped and the reply is a 200 stop.

Proof: not provable -- other: this host runs no local tests; CI runs the new tests

Manual verification

N/A: needs a crew container whose runtime fails at session start. Static gates run locally: black, isort, flake8, scripts/check_comment_history.py, scripts/check_file_line_cap.py, scripts/docs_lint.py. Tests run in CI.

Related Issues

Fixes #14695

Sibling sweep

Swept: git grep -n "_blocking_response|_stream_response|role.*chunk.*assistant" -- src/kiro_crew/dashboard
Sibling: src/kiro_crew/dashboard/openai_compat.py -- changed here (both readers)
Sibling: src/kiro_crew/apps/builtins/aws_control/crew/runtime/container/front/stream.py -- not affected (already relays error frames from the backend)

Pattern harvest

Not generalizable: the only HTTP reader that turns slot rows into a completion.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

@CrysisDeu
CrysisDeu requested review from a team and KiroCrewCaptain as code owners October 29, 2026 02:23
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 29, 2026
@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 0cd2637ab8cdbdca29c21b3bb09ad3614e6aa68b touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 0cd2637ab8cdbdca29c21b3bb09ad3614e6aa68b — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

One rider is worth a look: the turn that failed before it started had a working 500 already, and the PR changes it to a 502 just so both cases match.

Notes

  • Item 3 — rides along, inherited: "The pre-try failure, which answered 500 "internal error", now gets the same named 502." The main fix doesn't need this. That path already answered a failure, and the only reason given for changing its status and code is symmetry with the new path. Subtraction: keep status=500 / code: "server_error" for the _pinned case, and leave the test_dashboard_chat_refusal.py assertion change out.

What this change ships

Inventory (6 items) — 5 justified

Intent: an OpenAI-compatible API caller can tell a turn the backend could not run apart from an empty reply. FIX (#14695).

  1. A blocking completion whose turn ended on an error row now gets a 502 turn_failed with the redacted error text, not a 200 with empty content and stop — justified
  2. A streamed completion in that case now ends on an error frame and [DONE] with no stop chunk — justified
  3. A turn that failed before it started moves from 500 internal error to 502 turn_failed with its message — rides along, done for symmetry
  4. A retry notice (transient_retry) does not count as a failure — justified
  5. An error row followed by recovered reply text is still a normal stop — justified
  6. The spec paragraph and the handler comment describe the new rule — justified

[FIRST-PRINCIPLES-REVIEWED] 0cd2637

Description read

[DESCRIPTION-READ] e043abc00c011ea985ebd517e2ac96a00ed5468c347a9f5817c4d5267488419e

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 0cd2637ab8cdbdca29c21b3bb09ad3614e6aa68b — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

No design-level concerns. Reading the turn's own terminal error row at done fixes the root cause. Retry notices and turns that recover are excluded, and an empty reply with no error row stays a 200 stop.

Notes

  • Some error rows carry no TRANSIENT_RETRY_KIND meta yet are not final: acp_recovery.py:482 "⟳ Session busy — please retry." and permission_prompt.py:550 (the approval card). A turn that ends on one of these with no text after it now answers 502. That fits the goal, but say so in the spec paragraph.
  • The 502 message now carries the redacted error-row text, which can include host paths such as the agent spec location, where it used to say "internal error". A caller on a non-loopback gateway, who still needs a token, can now read those paths.

[DESIGN-REVIEWED] 0cd2637

Description read

[DESCRIPTION-READ] e758ed46cfdace11e9ec761bb2abe00d7f205315f6fca05a01a5d77560d23576

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 0cd2637ab8cdbdca29c21b3bb09ad3614e6aa68b — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 0cd2637

Verdict parsed from the review's SHA-scoped output markers for commit 0cd2637ab8cdbdca29c21b3bb09ad3614e6aa68b.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 0cd2637ab8cdbdca29c21b3bb09ad3614e6aa68b false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 0cd2637ab8cdbdca29c21b3bb09ad3614e6aa68b and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 0cd2637

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 0cd2637ab8cdbdca29c21b3bb09ad3614e6aa68b false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 29, 2026
…_failed, not an empty stop

A turn the runtime could not run still ends its cycle with a done row, and
both /v1/chat/completions readers read only chunk/assistant rows, so the
caller got HTTP 200 with empty content and finish_reason=stop. Track error
rows: one no later reply text follows ends the turn as failed. Blocking
answers 502 {code: turn_failed} with the redacted error row as message;
streaming ends with that error frame and [DONE], with no stop chunk. Retry
notices are not failures, a recovered turn stays a stop, and an empty reply
with no error row stays a 200.

Fixes #14695
@CrysisDeu
CrysisDeu force-pushed the fix/openai-compat-error-turn-14695 branch from 80814c9 to 0cd2637 Compare October 29, 2026 15:29
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running labels Oct 29, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at 0cd2637ab8cdbdca29c21b3bb09ad3614e6aa68b.

A turn that ends on its own error row now answers 502 turn_failed (blocking) or an error frame plus [DONE] with no stop (streaming), in place of an empty 200 stop. Retry notices are ignored, a turn that recovers with later text is still a normal stop, and an empty reply with no error row stays 200 (

). The new TestTurnErrorRow cases cover both transports and would fail on the base. Opus 5.5 and GPT 6.1 found nothing, and all checks on this head are green. This PR and #25562 both edit the top of api_completions, so whichever lands second needs a rebase.

AI findings judged non-blocking:

  • Design (Notes): error rows without TRANSIENT_RETRY_KIND that are not final ('Session busy — please retry', approval card) now end a turn as 502; asks the spec to say so. Both are failures from the caller's point of view: the busy notice is the no-requeue branch, and the approval card is only appended on no-budget or timeout denial. A turn that recovers is still a stop because later text clears the reason. This is a spec-wording nit.
    _approval_card = format_approval_timeout_card(_approval_window)
  • Design (Notes): the 502 message now carries the redacted error-row text, which can include host paths, to a token-holding caller. The text passes through the endpoint's existing _redact. The caller is the token-authenticated principal who already sees the same card in the dashboard and drives the agent, so no new trust boundary is crossed.
    self.reason = _redact(str(msg.get("content") or "")).strip() or "the turn failed"
  • First Principles (Notes): the pre-start failure moving from 500 'internal error' to 502 turn_failed is a symmetry rider. Scope and style only, and documented under Backwards compatibility. error.type stays server_error, and no user is harmed.

— crew-pr-reviewer (r135)

@iamwhatever
iamwhatever enabled auto-merge (squash) October 29, 2026 16:55

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: fix (4 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: bug fix in the OpenAI-compatible endpoint so a turn that ends on its error row answers turn_failed instead of an empty 200 stop, with tests for the blocking and streamed readers. Spec files changed as a ride-along (a minority of the diff on both file count and changed lines), not reviewed as a design decision: docs/system-specs/modules/learn-cron-dashboard.md.

@iamwhatever
iamwhatever merged commit dcc4c7e into main Oct 29, 2026
95 checks passed
@iamwhatever
iamwhatever deleted the fix/openai-compat-error-turn-14695 branch October 29, 2026 17:08
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Crew container front: a turn the backend could not run answers HTTP 200 with empty content and finish_reason=stop

3 participants