Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 98 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
name: Python Package Publish

on:
# 手动触发发布工作流
workflow_dispatch:
inputs:
destination:
description: '发布目标'
required: true
default: 'testpypi'
type: choice
options:
- testpypi
- pypi

permissions:
contents: write
packages: write

jobs:
# 检查测试工作流是否成功完成
check-tests:
runs-on: ubuntu-latest
if: >
(github.event_name == 'workflow_run' && github.event.workflow_run.conclusion == 'success') ||
github.event_name == 'workflow_dispatch'
steps:
- name: Success check
run: echo "前置测试通过,可以继续执行发布流程"

# 发布到TestPyPI
publish-testpypi:
name: Publish to TestPyPI
needs: check-tests
if: >
(github.event_name == 'workflow_run' && github.event.workflow_run.conclusion == 'success') ||
(github.event_name == 'workflow_dispatch' && github.event.inputs.destination == 'testpypi')
runs-on: ubuntu-latest
environment: publish-testpypi # 使用需要审批的环境
permissions:
# Required for trusted publishing
id-token: write

steps:
- name: Check out the repository
uses: actions/checkout@v6

- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build twine

- name: Build package
run: python -m build

- name: Publish to TestPyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
repository-url: https://test.pypi.org/legacy/
skip-existing: true
# Note: removed user and password fields to use Trusted Publishers

# 发布到PyPI
publish-pypi:
name: Publish to PyPI
needs: check-tests
if: github.event_name == 'workflow_dispatch' && github.event.inputs.destination == 'pypi'
runs-on: ubuntu-latest
environment: publish-pypi # 使用需要严格审批的环境
permissions:
# Required for trusted publishing
id-token: write

steps:
- name: Check out the repository
uses: actions/checkout@v6

- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build twine

- name: Build package
run: python -m build

- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
# Note: removed user and password fields to use Trusted Publishers
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ jobs:
uses: actions/checkout@v6
with:
repository: rhosocial/python-activerecord-testsuite
ref: feature/expression-schema-object-refactor
ref: test/typed-column-contracts
path: _testsuite

- name: Install activerecord and testsuite (editable)
Expand Down
4 changes: 3 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,9 @@ and raise `UnsupportedFeatureError` instead of being emulated:
- MySQL-style spatial types (`GEOMETRY`/`POINT`/...) and the `ST_*` function
family
- MySQL 9.0 `VECTOR` type and `STRING_TO_VECTOR` / `VECTOR_TO_STRING` /
`VECTOR_DIM` / `DISTANCE_*` functions (use `Array(Float32)` + `L2Distance`)
`VECTOR_DIM` / `DISTANCE_*` functions — no such API here, by design; ClickHouse's
own vector type is `QBit`, rendered as `ClickHouseVectorType` →
`QBit(Float32, n)`, and `Array(Float32)` + `L2Distance` also works
- MySQL `SET` type and `FIND_IN_SET`
- Stored procedures / stored functions / `CALL`
- `LOAD DATA INFILE` / `LOAD XML`
Expand Down
4 changes: 3 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,9 @@ and raise `UnsupportedFeatureError` instead of being emulated:
- MySQL-style spatial types (`GEOMETRY`/`POINT`/...) and the `ST_*` function
family
- MySQL 9.0 `VECTOR` type and `STRING_TO_VECTOR` / `VECTOR_TO_STRING` /
`VECTOR_DIM` / `DISTANCE_*` functions (use `Array(Float32)` + `L2Distance`)
`VECTOR_DIM` / `DISTANCE_*` functions — no such API here, by design; ClickHouse's
own vector type is `QBit`, rendered as `ClickHouseVectorType` →
`QBit(Float32, n)`, and `Array(Float32)` + `L2Distance` also works
- MySQL `SET` type and `FIND_IN_SET`
- Stored procedures / stored functions / `CALL`
- `LOAD DATA INFILE` / `LOAD XML`
Expand Down
37 changes: 34 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,11 @@ _ec7d664_
- **CI-tested server versions**: 25.8, 26.3, 26.7
- **Published**: _not yet on PyPI_

The `<3.15` ceiling is inherited from the driver: `clickhouse-connect` declares
`Requires-Python: <3.15,>=3.10`, which is why `requires-python` is pinned to
`>=3.10,<3.15` in `pyproject.toml` rather than chosen here. The pin moves when
`clickhouse-connect` lifts its own bound.

## Installation

```bash
Expand Down Expand Up @@ -140,9 +145,35 @@ than being silently emulated.
version. Beyond the ones in the table, that includes: `check_constraint`,
`create_sequence`, `drop_sequence`, `auto_increment`, `clone`, `call`,
`binlog`, `analyze_table`, `check_table`, `checksum_table`, `descending_index`,
`add_constraint`, `drop_constraint`, `add_partition`, `attach_partition`,
`detach_partition`, `deferrable_*`, and every MySQL spatial `ST_*` surface
plus the MySQL 9.0 `VECTOR` type.
`add_constraint`, `drop_constraint`, `deferrable_*`, and every MySQL spatial
`ST_*` surface plus the MySQL 9.0 `VECTOR` type.

## Partitioning

ClickHouse partitions a `MergeTree` table with `PARTITION BY <expr>` in
`CREATE TABLE` — any expression over the table's columns, or a tuple of them —
declared through `storage_options` alongside `ENGINE` / `ORDER BY`. There is no
partitioning strategy to choose, no `PARTITION ... VALUES LESS THAN` /
`VALUES IN` / `MAXVALUE` boundary and no subpartitioning; a partition comes
into existence when a row lands in it. Partitions are introspected through
`system.parts` (`partition`, `partition_id`, `name`) and maintained by
partition id:

```python
from rhosocial.activerecord.backend.impl.clickhouse.expression import (
ClickHouseDetachPartitionExpression, # ... also DROP / ATTACH
)
sql, params = ClickHouseDetachPartitionExpression(dialect, "visits", "202601").to_sql()
# ALTER TABLE `visits` DETACH PARTITION ID '202601'
```

`DROP` / `DETACH` / `ATTACH` partition are the only partition clauses this
backend renders. ClickHouse has no `ADD PARTITION`, `TRUNCATE PARTITION`,
`REORGANIZE PARTITION`, `EXCHANGE PARTITION`, `SUBPARTITION BY` or
`ANALYZE`/`CHECK`/`OPTIMIZE`/`REBUILD`/`REPAIR PARTITION`; the full inventory
is at <https://clickhouse.com/docs/sql-reference/statements/alter/partition>.
Empty a partition with `ALTER TABLE ... DELETE IN PARTITION ... WHERE ...`, and
a table with `TRUNCATE TABLE`.

## Version Gates

Expand Down
1 change: 1 addition & 0 deletions changelog.d/5.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Register the column-type protocol and render the new pad/repeat/trim nodes. `ClickHouseColumnTypeMixin` answers `suggested_column_types()` for all eighteen common Python types — every entry a column class, none refused — with each cell backed by a probe against ClickHouse 26.7.3.19, because core's rebuild removed the neutral table this backend used to derive from. `ClickHouseTrimMixin` (`mixins/trim.py`) is written from scratch: the ANSI `TRIM(BOTH FROM x)` form the shared renderer emits is a syntax error on this server (measured on the public playground), so the direction moves into the function name and the operand order becomes target-first, characters-second — `BOTH → trimBoth(x[, chars])`, `LEADING → trimLeft`, `TRAILING → trimRight`. `LpadMixin` / `RpadMixin` / `RepeatMixin` need no override, because ClickHouse spells all three natively. `ClickHouseTypeSupport` is split into `protocols/type.py`, `ColumnTypeSupport` is declared in the protocol-conformance list, and `feature/backend/test_expression_roundtrip_all.py` now enumerates both core's and this backend's expression packages, classifying every outcome instead of swallowing it: the 31 dispatch gaps are grouped by feature in `UNMODELLED_FORMATTERS` and pinned in both directions, so a formatter nobody listed fails the run and a listed method no class needs fails too.
1 change: 1 addition & 0 deletions changelog.d/5.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Render `TRIM`, `LPAD`, `RPAD` and `REPEAT` through the node formatters, and render the `ALTER COLUMN` default change ClickHouse actually accepts: `SET DEFAULT` becomes `DEFAULT` and `DROP DEFAULT` becomes `MODIFY COLUMN ... REMOVE DEFAULT`, both spellings quoted from the server's own rejection messages in `mixins/ddl_column.py`. `SHOW [FULL] COLUMNS`, `SHOW INDEX`, `SHOW PROCESSLIST`, `SHOW ENGINES` and `SHOW GRANTS` no longer refuse — five statements the server does accept that this backend used to raise on. Their columns are ClickHouse's, and the differences from MySQL's (where a parser reading MySQL's names returns `None` rather than raising) are documented rather than left for a caller to discover.
1 change: 1 addition & 0 deletions changelog.d/5.docs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Correct the capability-boundary documentation against measurements taken on ClickHouse 26.7.3.19, in both `docs/en_US` and `docs/zh_CN`: the `text` inverted index (GA from 26.2) replacing `tokenbf_v1` / `ngrambf_v1` for full-text search, the seven real `ST_*` WKB/MVT codecs and the `Geometry` / `Point` column types, the optimizer-hint comment the server parses and discards, the skip-index keyword being `TYPE` rather than MySQL's `USING`, and `FINAL` being accepted only by the engines that collapse duplicates. The README also records why `requires-python` is `>=3.10,<3.15`: the ceiling is inherited from `clickhouse-connect` (which declares `Requires-Python: <3.15,>=3.10`) rather than chosen here, which is why there is no Python 3.15 leg in CI.
1 change: 1 addition & 0 deletions changelog.d/5.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Refuse the constraints whose silent outcome is worse than the error. `UNIQUE (col)` is a `SYNTAX_ERROR` on the server, and an unnamed `FOREIGN KEY (...) REFERENCES ...` is *accepted and silently discarded* — measured on 26.7.3.19, the DDL succeeds, `SHOW CREATE TABLE` reports no foreign key, `system.tables.create_table_query` does not contain the string, an orphan row inserts and reads back fine, and dropping the referenced parent succeeds as well. Emitting either would produce a schema with no constraint and nothing in the server's metadata to explain the missing enforcement, so `mixins/ddl_column.py` raises `UnsupportedFeatureError` at the point of declaration with that measured reason attached.
1 change: 1 addition & 0 deletions changelog.d/5.internal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Adapt the tests and CI to core `release/v1.0.0.dev30`. Tests that read SQL types move from `column_type()` / `columns_type()` to `column_data_type()` / `columns_data_type()`; predicates the bare `Column` used to accept are built explicitly as `ComparisonPredicate` / `InPredicate` / `BinaryArithmeticExpression`; `DateTimeSupport` becomes `TimestampSupport` in the protocol-conformance list, `UUIDSupport` joins the not-implemented list, and `ClickHouseVectorSupport` leaves it. CI checks out core from `release/v1.0.0.dev30` now that the typed-columns branch has merged ([python-activerecord#126](https://github.com/rhosocial/python-activerecord/pull/126)), while the testsuite stays on `test/typed-column-contracts` because that branch has not landed on its release branch yet. `requirements-dev.txt` pins pydantic 2.12.2 / pydantic-core 2.41.4 for Python 3.13, since an unpinned resolve there picks a pydantic-core whose only cp313t distribution is an sdist.
1 change: 1 addition & 0 deletions changelog.d/5.removed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Remove the MySQL 9.0 `VECTOR` emulation surface and the MySQL declarative-partitioning strategy family. ClickHouse has no `STRING_TO_VECTOR` / `VECTOR_TO_STRING` / `VECTOR_DIM` / `DISTANCE_*` API, so `expression/vector.py`, `mixins/vector.py`, `protocols/vector.py`, `ClickHouseVectorMixin`, `ClickHouseVectorSupport` and the testsuite provider `tests/providers/partition.py` are gone; ClickHouse's own vector type is `QBit`, rendered by `ClickHouseVectorType` → `QBit(Float32, n)`. The `PARTITION ... VALUES` strategy family, subpartitioning, and `ADD` / `TRUNCATE` / `REORGANIZE` / `EXCHANGE` / `COALESCE` / `ANALYZE` / `CHECK` / `REBUILD` / `REPAIR PARTITION` are removed too, along with `expression/partition_lifecycle.py`. What is left is ClickHouse's own maintenance by partition id (`DROP` / `DETACH` / `ATTACH PARTITION ID`) and `system.parts` introspection, both now covered by `feature/backend/ddl/test_partition_maintenance.py`.
Loading
Loading