Skip to content

exp/api/remote: never answer a failed write with a 2xx status - #2158

Open
mrueg wants to merge 1 commit into
prometheus:mainfrom
mrueg:exp-remote-write-error-status
Open

mrueg wants to merge 1 commit into
prometheus:mainfrom
mrueg:exp-remote-write-error-status

Conversation

@mrueg

@mrueg mrueg commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Summary

NewWriteResponse() defaults the status code to 204, but the write handler only fell back to 500 when the status code was 0. A Store implementation that returned an error together with a default (or nil) WriteResponse therefore made the handler reply 204 No Content with the error message as the body. The sender treats that as success and drops the batch, so data is lost silently.

This change:

  • Replaces any status code below 400 with 500 when Store returns an error. Explicit 4xx/5xx codes are kept.
  • Defaults a zero status code to 204 on success. Previously, returning a zero value &WriteResponse{} made WriteHeader(0) panic.
  • Updates the SetStatusCode doc comment accordingly.

Behaviour change to note: a Store returning an error together with an explicit 2xx status now results in a 500. An error from Store should never be reported as success to the sender.

Testing

Added the table-driven TestWriteHandler_StatusCode, covering default, nil, zero value and explicit status responses with and without a storage error. Before the fix it fails (204 on errors, panic on the zero value response); with the fix it passes, as does go test -race ./api/remote/ in exp/.

Note: #2153 touches writeHeaders and the same test file; any conflict should be trivial.

🤖 Generated with Claude Code

NewWriteResponse defaults the status code to 204, but the write handler
only fell back to 500 when the status code was 0. A Store implementation
returning an error together with a default (or nil) WriteResponse thus
made the handler answer with "204 No Content", so the sender treated the
batch as accepted and dropped it.

Replace any status code below 400 with 500 when Store returns an error.
Also default a zero status code to 204 on success, which previously made
WriteHeader(0) panic for a zero value WriteResponse.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Manuel Rüger <manuel@rueg.eu>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The implementation matches the stated behavior and includes focused regression coverage.

Review effort: Balanced
Findings: None

What changed in this PR

Prevents failed remote writes from returning successful HTTP statuses, avoiding silent data loss.

Changes:

  • Converts sub-400 statuses to 500 when storage fails.
  • Defaults zero status to 204 on success.
  • Adds table-driven regression coverage.
File Description
exp/​api/​remote/​remote_headers.go Documents status fallback behavior.
exp/​api/​remote/​remote_api.go Correctly normalizes success and error statuses.
exp/​api/​remote/​remote_api_test.go Tests default, nil, zero, and explicit statuses.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants