Skip to content

Use lookup tables for string and bytes parameter escaping - #1167

Merged
joe-clickhouse merged 3 commits into
ClickHouse:mainfrom
patrick-immometrica:perf-bind-escape
Nov 10, 2026
Merged

joe-clickhouse merged 3 commits into
ClickHouse:mainfrom
patrick-immometrica:perf-bind-escape

Conversation

@patrick-immometrica

Copy link
Copy Markdown
Contributor

Summary

escape_str and escape_bytes build their output one character/byte at a time in a generator with an f-string per item. They run for every string and bytes parameter, client-side and server-side, plus identifier quoting, so large IN lists and binary params spend most of their bind time in them.

escape_str now uses str.translate with a table built from must_escape, and escape_bytes a precomputed 256-entry table. Output is identical (checked against the old functions on random input, including bytearray and memoryview).

bind_query timings, Python 3.14:

case main this PR
IN %(ids)s, 10k strings 24.0 ms 5.6 ms
{ids:Array(String)}, 10k strings 24.3 ms 7.5 ms
{b:String}, 1 MB bytes 202 ms 31 ms

Checklist

  • Unit test pinning the escaped output
  • CHANGELOG entry

@patrick-immometrica
patrick-immometrica marked this pull request as ready for review November 9, 2026 12:07
Signed-off-by: Joe Spadola <joe.spadola@clickhouse.com>
joe-clickhouse
joe-clickhouse previously approved these changes Nov 9, 2026

@joe-clickhouse joe-clickhouse left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks!

chernser
chernser previously approved these changes Nov 10, 2026
Conflict in CHANGELOG.md: kept both Improvements entries.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@joe-clickhouse
joe-clickhouse merged commit 8de8ad1 into ClickHouse:main Nov 10, 2026
54 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants