Skip to content

fix(gateway): pass the invalid_guard_verdict worker code through - #297

Merged
dragosboca merged 1 commit into
mainfrom
fix/gateway-invalid-guard-verdict-code
Sep 17, 2026
Merged

dragosboca merged 1 commit into
mainfrom
fix/gateway-invalid-guard-verdict-code

Conversation

@dragosboca

@dragosboca dragosboca commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Why

Since #285 a guard model whose first output positions carry no complete Yes/No distribution terminates with error_code="invalid_guard_verdict" and "guard model did not produce a valid thresholded verdict". The Python and TypeScript SDK READMEs already tell callers to handle that code. The gateway's worker error allowlist (queue/streaming.rs::client_safe_worker_error_code) predates it, so the terminal collapsed to inference_error / HTTP 500 / "internal error during generation" on the buffered path and "Generation terminated with an upstream error." on the streaming path. A caller has no way to tell "the guard could not produce a verdict" from a backend fault.

Reproducible with POST /v1/generate/ibm-granite__granite-guardian-3.0-2b {"prompt": "...", "max_new_tokens": 4} (a raw prompt has no Guardian template, so the model answers in prose): the body is the generic 500 instead of the typed code.

What

  • http_error.rs: openai_code::INVALID_GUARD_VERDICT.
  • queue/streaming.rs: admit the code in client_safe_worker_error_code and pin it in the contract test.

Status and OpenAI type stay the defaults (500 / server_error), the same settlement as empty_model_output: terminal, non-retryable, tokens were consumed.

Verification

  • cargo test --lib worker_error_public_contract: 2 passed.
  • cargo fmt --check and cargo clippy --lib -D warnings: clean.

Summary by CodeRabbit

  • Bug Fixes
    • Guard-model responses with incomplete Yes/No verdicts now return a clear, non-retryable error code.
    • This error code is preserved when passed through the worker, improving error reporting for clients.
    • Added coverage to verify the error-code handling.

Guard models emit invalid_guard_verdict when their first output positions
carry no complete Yes/No distribution (#285), and the SDK READMEs tell
callers to expect that code. The gateway's worker error allowlist predates
it, so the terminal collapsed to a generic inference_error with the
"internal error during generation" message on both the buffered and the
streaming path.

Admit the code so the typed terminal and its message reach the client. It
settles like empty_model_output: terminal, non-retryable, server_error.
@dragosboca
dragosboca requested a review from a team as a code owner September 17, 2026 16:08
@coderabbitai

coderabbitai Bot commented Sep 17, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 29ea26ec-71ed-46f4-b2f6-9729b32ec729

📥 Commits

Reviewing files that changed from the base of the PR and between 120059f and 86a2ae2.

📒 Files selected for processing (2)
  • packages/sie_gateway/src/http_error.rs
  • packages/sie_gateway/src/queue/streaming.rs

Included review availability: Your plan provides up to 10 included reviews per hour; 7 remain after this review.


📝 Walkthrough

Walkthrough

The gateway adds invalid_guard_verdict as a public OpenAI error code. Client-safe worker error mapping preserves this code, and a regression test covers the allowlist.

Changes

Guard verdict error code handling

Layer / File(s) Summary
Error code declaration and propagation
packages/sie_gateway/src/http_error.rs, packages/sie_gateway/src/queue/streaming.rs
The gateway declares INVALID_GUARD_VERDICT, preserves it in client-safe worker error mapping, and tests the sanctioned error-code list.

Suggested reviewers: huronat

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to 86a2a

The new terminal error code is preserved for clients across buffered and streaming responses without changing the intended server-error behavior.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: passing the invalid_guard_verdict worker code through the gateway.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 2 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@dragosboca
dragosboca merged commit 7a8b09b into main Sep 17, 2026
21 checks passed
@dragosboca
dragosboca deleted the fix/gateway-invalid-guard-verdict-code branch September 17, 2026 16:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant