Repository navigation
Truncate Decimal inserts consistently in the Python codec - #1165
joe-clickhouse merged 4 commits into
Conversation
# Conflicts: # CHANGELOG.md # tests/unit_tests/test_driver/test_native_write.py
joe-clickhouse
left a comment
There was a problem hiding this comment.
Approved. I added wider regression tests, clarified the changelog, and removed the comment. These cover the carry case and both clients. I also merged current main. The fix itself is unchanged.
There was a problem hiding this comment.
🟡 Changes recommended
Both decimal writers inherit clamp=1, which can cause MemoryError on previously valid inserts after precision is raised to MAX_PREC; they need clamp disabled and regression coverage.
1 open issue
What changed in this PR
This PR updates Python codec Decimal inserts to truncate excess fractional digits consistently, without rounding first at larger magnitudes. The read path is unchanged.
Changes:
- Use
decimal.MAX_PRECduring scaling in both Decimal writers. - Add unit and integration coverage for decimal widths, input types, nullable values, and nested types.
- Document the behavior correction in the changelog.
Validation was limited to a focused standard-library check. Project tests, linting, and type checks were not run.
| File | Description |
|---|---|
tests/unit_tests/test_driver/test_native_write.py |
Check encoded integers for Decimal truncation. |
tests/integration_tests/test_numeric.py |
Add round-trip tests, including nullable and nested Decimal values. |
clickhouse_connect/datatypes/numeric.py |
Increase scaling precision in both Decimal write paths. |
CHANGELOG.md |
Describe the truncation fix and previous rounding behavior. |
Review effort: Balanced

Summary
The Python codec's
Decimalwrite runsint(Decimal(str(x)) * 10**scale)inside a context withprec = self.prec. When the scaled value has more digits than the column precision, the multiply rounds (half-even) beforeint()gets to truncate. So the result depends on the magnitude:Same for
Decimal128/256viaBigDecimal. The Rust codec and the server (toDecimal*, text input) truncate in every case.The multiply by
10**scaleis exact once precision isn't capped, so this setsprectodecimal.MAX_PRECfor the write and letsint()truncate. Overflow behaviour is unchanged since the int still has to fit the column width. The read path is untouched.Checklist
Decimal(9, 2), nullable andDecimal(38, 2)(fail on main)