Repository navigation
Convert NumPy datetime64 inserts to the column unit - #1175
Merged
joe-clickhouse merged 4 commits intoNov 11, 2026
Merged
Conversation
mansur-akhmadi
force-pushed
the
fix/numpy-datetime64-unit
branch
from
November 10, 2026 18:18
5b05926 to
e31ad9b
Compare
joe-clickhouse
approved these changes
Nov 11, 2026
Contributor
There was a problem hiding this comment.
Thanks for this. Looks good.
I pushed two commits on top:
- Renamed
_datetime64_ticksto_np_datetime_ticks. The old name matched an unrelated method onDateTime64. - Added type hints to the helper and removed its comment.
- Merged main.
NaT as NULL and the other gaps you listed can go in a separate PR.
joe-clickhouse
approved these changes
Nov 11, 2026
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Using np_array.T reverses nested cell dimensions, causing previously accepted row-oriented (2, 1, 3) datetime arrays for an Array(DateTime) column to raise ProgrammingError.
1 open issue
What changed in this PR
This PR addresses #1174 by converting NumPy datetime64 values to the target column's time unit before insertion, rather than writing raw array ticks.
Changes:
- Adds per-column conversion for structured and two-dimensional NumPy arrays.
- Adds unit and sync/async integration tests for unit conversion, mixed column types, and orientation.
- Documents the user-visible fix in the changelog.
| File | Description |
|---|---|
tests/unit_tests/test_driver/test_insert.py |
Checks converted ticks, orientation, and column-count validation. |
tests/integration_tests/test_numpy.py |
Covers date/time units, array layouts, and mixed columns. |
clickhouse_connect/driver/insert.py |
Converts NumPy date/time columns to target units before extracting ticks. |
CHANGELOG.md |
Describes the insertion fix and previous failure modes. |
Review effort: Balanced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Fixes #1174
InsertContext._convert_numpyturneddatetime64columns into integers withastype(int)and marked them with the"int"format. Those integers are ticks in the array unit, but the writer expects ticks in the column unit: seconds forDateTime, days forDateandDate32, and10**scaleticks forDateTime64. Only an array with the same unit as the column worked.This change casts each date or time column to
col_type.np_typebefore it takes the ticks, the same as the pandas path in_convert_pandas. The 2-D branch now converts each column on its own, so a 2-D array can feedDateandDateTime64(6)columns together, and it keeps the row or column orientation the caller asked for. Values before the epoch round down, which matchestoDateTime64on the server.Not changed: NaT in a NumPy array is still not written as NULL, a cast to a finer unit outside the NumPy range still wraps,
DateTime64scales other than 0, 3, 6 and 9 still raise the existingProgrammingError, and anumpy.datetime64scalar inside row data still fails as before. I can send those separately if you want them.Tests:
tests/integration_tests/test_numpy.pycoversDate,Date32,DateTime,DateTime64(3/6/9)andNullable(DateTime64(3))with units fromDtons, a pre-epoch value, and structured, row and column-oriented 2-D arrays, through sync and async clients. A unit test intests/unit_tests/test_driver/test_insert.pychecks the ticks and the orientation of a mixed 2-D array. The new tests fail on main and pass with this change, withCLICKHOUSE_CONNECT_USE_C=0and1.Checklist