Skip to content

fix: return a clear error when Query over HTTP gets a non-Native format - #2140

Open
Indian-boult wants to merge 1 commit into
ClickHouse:mainfrom
Indian-boult:fix/http-query-non-native-format
Open

Indian-boult wants to merge 1 commit into
ClickHouse:mainfrom
Indian-boult:fix/http-query-non-native-format

Conversation

@Indian-boult

@Indian-boult Indian-boult commented Nov 18, 2026 •

Copy link
Copy Markdown

Summary

Fixes #2130

Over HTTP, Query decoded every response body as Native. dialHttp puts default_format=Native on the connection URL, but a FORMAT clause in the query text takes priority, so the server sends the body in that format. JSONEachRow and CSV then failed with readData: block decode for exception: unexpected value 34 for boolean, and RowBinary gave an empty result with no error. QueryRow, Select and database/sql go through the same httpConnect.query.

The server names the body format in the X-ClickHouse-Format response header. I checked it with curl against ClickHouse 26.10.4.207 with the parameters the driver sends. It is Native with no FORMAT clause, the format name as written for any FORMAT clause, also one placed before SETTINGS, Null with an empty body for FORMAT Null, and missing for statements without a result. It was the same with wait_end_of_query=1, send_progress_in_http_headers=1, compress=1 and gzip.

httpConnect.query now reads this header after sendQuery. When it names a format other than Native or Null, without regard to case, Query reads at most maxCloseDrainBytes of the body, closes it, puts the healthy connection back in the pool with release(h, nil), the same as httpFormatStream.Close, and returns this error.

clickhouse: the server sent the result in format JSONEachRow, but Query over HTTP can read only Native, so remove the FORMAT clause from the query or use QueryFormat to read the JSONEachRow bytes

FORMAT Null stays allowed because the body is empty and the native protocol also returns no rows. A response without the header keeps the old path. Exec does not read the body and does not change. QueryFormat and InsertFormat do not use this path. The change is after sendQuery, away from the compression switch that #2121 changes.

One behavior changes. Over HTTP, a FORMAT clause other than Native or Null now gives this error also for an empty result, where before an empty JSONEachRow, CSV or RowBinary result gave no rows and no error.

tests/issues/issue_2130_test.go adds Test2130 for clickhouse.Open and Test2130Std for database/sql. Over HTTP, the JSONEachRow, CSV, RowBinary, lowercase format jsoneachrow and FORMAT JSONEachRow SETTINGS max_threads = 1 cases expect the exact error from Query, QueryRow and Select, and from QueryContext and QueryRowContext, with LZ4 and GZIP too. After each error the tests check that the connection is back in the pool and that the next query returns rows. The native protocol, Native bodies, FORMAT Null and Exec are control cases. On main the 12 HTTP error cases fail and the control cases pass. With the fix all pass. The root package tests, the HTTP and format tests in tests/, the HTTP tests in tests/std/ and tests/issues/ pass against 26.10.4.207, except Test1421BatchFlushBrokenConn, which starts its own Docker container and was skipped.

Checklist

  • Unit and integration tests covering the common scenarios were added

Note

Medium Risk
Touches the core HTTP query path used by Query/QueryRow/Select, but the new branch is narrow and preserves connection pooling; empty non-Native results now error instead of returning no rows.

Overview
HTTP Query now inspects X-ClickHouse-Format after sendQuery instead of always decoding the body as Native. When the header names a format other than Native or Null (case-insensitive), it partially drains the body, closes it, returns the connection to the pool, and returns an error that names the format and suggests removing the FORMAT clause or using QueryFormat.

Previously, a FORMAT clause in the SQL could override default_format=Native, leading to misleading decode errors or silent empty results for formats like JSONEachRow or RowBinary. FORMAT Null, missing headers, and Native responses are unchanged.

Adds tests/issues/issue_2130_test.go covering driver and database/sql paths, multiple non-Native formats, compression, pool reuse after the error, and control cases (native protocol, Exec, etc.).

Reviewed by Cursor Bugbot for commit d734e04. Bugbot is set up for automated code reviews on this repo. Configure here.

Over HTTP, a FORMAT clause in the query text takes priority over the
default_format=Native that the connection sends, so the server sends the
body in that format. Query decoded the body as Native anyway. JSONEachRow
and CSV failed with a block decode error that did not tell the cause, and
RowBinary gave an empty result and no error.

Query now reads the X-ClickHouse-Format response header. When the header
names a format other than Native or Null, Query reads at most 64 KiB of
the body, closes it, puts the connection back in the pool and returns an
error that names the format and points to QueryFormat. FORMAT Null and
responses without the header keep the old behavior.

Fixes ClickHouse#2130
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HTTP: Query with a non-Native FORMAT clause decodes the body as Native (opaque decode error, or an empty result with no error)

1 participant