Repository navigation
fix(openai-compat): answer a turn that ended on its error row as turn_failed, not an empty stop - #25555
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS One rider is worth a look: the turn that failed before it started had a working 500 already, and the PR changes it to a 502 just so both cases match. Notes
What this change shipsInventory (6 items) — 5 justifiedIntent: an OpenAI-compatible API caller can tell a turn the backend could not run apart from an empty reply. FIX (#14695).
[FIRST-PRINCIPLES-REVIEWED] 0cd2637 Description read[DESCRIPTION-READ] e043abc00c011ea985ebd517e2ac96a00ed5468c347a9f5817c4d5267488419e That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS No design-level concerns. Reading the turn's own terminal Notes
[DESIGN-REVIEWED] 0cd2637 Description read[DESCRIPTION-READ] e758ed46cfdace11e9ec761bb2abe00d7f205315f6fca05a01a5d77560d23576 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
…_failed, not an empty stop
A turn the runtime could not run still ends its cycle with a done row, and
both /v1/chat/completions readers read only chunk/assistant rows, so the
caller got HTTP 200 with empty content and finish_reason=stop. Track error
rows: one no later reply text follows ends the turn as failed. Blocking
answers 502 {code: turn_failed} with the redacted error row as message;
streaming ends with that error frame and [DONE], with no stop chunk. Retry
notices are not failures, a recovered turn stays a stop, and an empty reply
with no error row stays a 200.
Fixes #14695
80814c9 to
0cd2637
Compare
iamwhatever
left a comment
There was a problem hiding this comment.
APPROVE at 0cd2637ab8cdbdca29c21b3bb09ad3614e6aa68b.
A turn that ends on its own error row now answers 502 turn_failed (blocking) or an error frame plus [DONE] with no stop (streaming), in place of an empty 200 stop. Retry notices are ignored, a turn that recovers with later text is still a normal stop, and an empty reply with no error row stays 200 (
TestTurnErrorRow cases cover both transports and would fail on the base. Opus 5.5 and GPT 6.1 found nothing, and all checks on this head are green. This PR and #25562 both edit the top of api_completions, so whichever lands second needs a rebase.
AI findings judged non-blocking:
- Design (Notes): error rows without TRANSIENT_RETRY_KIND that are not final ('Session busy — please retry', approval card) now end a turn as 502; asks the spec to say so. Both are failures from the caller's point of view: the busy notice is the no-requeue branch, and the approval card is only appended on no-budget or timeout denial. A turn that recovers is still a stop because later text clears the reason. This is a spec-wording nit.
- Design (Notes): the 502 message now carries the redacted error-row text, which can include host paths, to a token-holding caller. The text passes through the endpoint's existing _redact. The caller is the token-authenticated principal who already sees the same card in the dashboard and drives the agent, so no new trust boundary is crossed.
- First Principles (Notes): the pre-start failure moving from 500 'internal error' to 502 turn_failed is a symmetry rider. Scope and style only, and documented under Backwards compatibility. error.type stays server_error, and no user is harmed.
— crew-pr-reviewer (r135)
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: fix (4 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: bug fix in the OpenAI-compatible endpoint so a turn that ends on its error row answers turn_failed instead of an empty 200 stop, with tests for the blocking and streamed readers. Spec files changed as a ride-along (a minority of the diff on both file count and changed lines), not reviewed as a design decision: docs/system-specs/modules/learn-cron-dashboard.md.
Problem / Motivation
Goal: a
POST /v1/chat/completionsturn the backend could not run is reported as a failure, not as an empty successful completion.When the ACP runtime cannot start a turn, the caller gets HTTP 200,
content: "",finish_reason: "stop"and zero usage. The turn's failure only reaches the gateway's own log.Why it matters
A failed turn looks the same as a model that chose to say nothing. A caller that retries gets empty replies forever, and a script or dashboard reading
finish_reasonrecords a normal completion. The issue records a Fargate task where this hid anAcpRuntimeErroron every attempt and cost a full e2e round to find.Not a goal
stop.Intended? / Repro
Intended? No.
docs/system-specs/modules/learn-cron-dashboard.mdalready says a failed turn must answerserver_errorinstead of an empty reply, but only the pre-tryfailure row carries themeta.turn_failedflag the endpoint checks.Repro (read on current main,
8f8cc940e7):src/kiro_crew/dashboard/chat_turn/turn_failures.py:604_on_error: anAcpRuntimeErrorraised by the session start (the issue'sthe default agent spec .../kirocrew.json is missing) is appended asslot.append("error", _err_text, "msg msg-err", meta=_err_meta), with noturn_failedkey._on_acp_error_card(turn_failures.py:402) and the other terminal arms do the same.donerow.src/kiro_crew/dashboard/openai_compat.py:649_turn_failedonly matchesmeta.turn_failed(set bychat_turn/queue_cycle.py:686for an exception above the turn'stry)._blocking_response(openai_compat.py:795-834) and_stream_response(openai_compat.py:686-725) skip every other row that is notchunk/assistant, so atdonethey answer200withcontent: ""andfinish_reason: "stop".{"choices": [{"message": {"content": ""}, "finish_reason": "stop"}]}with status 200.What changed (motivation → approach → change)
Symptom: a failed turn answers 200 /
stop. Cause: the readers ignore the turn's own error row. Change: read the error rows and answer the failure atdone.openai_compat._TurnOutcome: both readers feed every row to it. Anerrorrow that no laterchunk/assistanttext follows ends the turn as failed, with that row's redacted text as the reason.meta.kind == TRANSIENT_RETRY_KIND) is not a failure.stop.meta.turn_failedrow (failed before it started) holds whatever the queue behind it answers, as on main.{"error": {"message": <error row, redacted>, "type": "server_error", "code": "turn_failed"}, "code": "turn_failed"}. The pre-tryfailure, which answered500 "internal error", now gets the same named 502.[DONE], with nofinish_reason: stopchunk. The container front'sstream.pyalready passeserrorframes through.slot.task.result()exception path is unchanged (500 internal error).docs/system-specs/modules/learn-cron-dashboard.md: describes the rule and the response shapes; the old paragraph said the collectors read onlychunk/assistant.Backwards compatibility
A caller that got 200 with empty content for a failed turn now gets 502 (blocking) or an error frame (stream). The pre-start failure moves from 500
server_errorto 502turn_failed;error.typestaysserver_error. Successful turns, including empty ones, are unchanged.Removes nothing: no key, field or route is removed.
Tests
test/test_openai_compat.py,TestTurnErrorRow(new), both transports where it applies:test_blocking_answers_502_naming_the_error_row: error row thendone→ 502,code: turn_failed, message names the error.test_stream_ends_on_the_error_frame_without_a_stop: last frame is theturn_failederror, nofinish_reason: stop, ends with[DONE].test_the_error_message_is_redacted: a credential in the error row does not reach the response.test_a_reply_after_a_recovered_error_is_a_normal_stoptest_an_error_after_partial_text_still_failstest_a_retry_notice_alone_is_not_a_failuretest_an_empty_reply_with_no_error_row_stays_a_200_stoptest_a_turn_that_failed_before_it_started_holds_over_later_texttest/test_dashboard_chat_refusal.py::test_a_setup_error_is_a_server_error_on_the_openai_endpoint: now expects 502 withcode: turn_failedand the row's message.On main the blocking, stream, redaction and partial-text tests fail, because the error row is dropped and the reply is a 200
stop.Proof: not provable -- other: this host runs no local tests; CI runs the new tests
Manual verification
N/A: needs a crew container whose runtime fails at session start. Static gates run locally: black, isort, flake8,
scripts/check_comment_history.py,scripts/check_file_line_cap.py,scripts/docs_lint.py. Tests run in CI.Related Issues
Fixes #14695
Sibling sweep
Swept: git grep -n "_blocking_response|_stream_response|role.*chunk.*assistant" -- src/kiro_crew/dashboard
Sibling: src/kiro_crew/dashboard/openai_compat.py -- changed here (both readers)
Sibling: src/kiro_crew/apps/builtins/aws_control/crew/runtime/container/front/stream.py -- not affected (already relays
errorframes from the backend)Pattern harvest
Not generalizable: the only HTTP reader that turns slot rows into a completion.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributed under the project's Apache-2.0 license.