feat(dialect): follow the core's column-layer rebuild — column types, node formatters, and the version gates by vistart · Pull Request #62 · rhosocial/python-activerecord-postgres

vistart · 2026-10-11T07:52:49Z

Description

This is the PostgreSQL side of the core column-layer rebuild: the branch that makes a column class a capability bundle, turns the bare Column into a reference rather than an operator surface, and moves trim/lpad/rpad/repeat from function names routed through format_function_call into dedicated nodes. A dialect has to answer for that rebuild in five places, and this PR answers all five.

The column-type protocol is now implemented rather than assumed. The new mixins/column_type.py carries POSTGRES_COLUMN_TYPES — the protocol's closed list of eighteen common Python types, every one of them answered — and PostgresColumnTypeMixin(ColumnTypeMixin), composed into PostgresDialect next to PostgresDataTypeMixin. Putting the two side by side is deliberate: one says which operations a value carries, the other says how it is spelled, and the protocol exists to keep them apart. Two answers moved with the rebuild rather than being edited: float and decimal.Decimal now answer NumericColumn, because core removed FloatColumn / DecimalColumn / RealColumn / DoubleColumn and the numeric family is deliberately one class — PostgreSQL really does have distinct numeric / real / double precision families, but that is a storage fact the DataType layer spells, not something an operation class can express. One answer is a deliberate refusal: datetime.timedelta answers None, because this server has a real interval type and core has no IntervalColumn to name its operations with, and the previous NumericColumn answer handed callers sqrt(interval '1 day') — which the engine refuses outright. A field declared timedelta now fails to resolve with a message naming the fix (UseColumnType), which is the honest version of that gap.

The four new nodes are composed, not overridden. PostgresDialect now mixes in TrimMixin, LpadMixin, RpadMixin and RepeatMixin. PostgreSQL spells all four natively — TRIM([BOTH|LEADING|TRAILING] [chars] FROM str), LPAD/RPAD(str, len, padstr), REPEAT(str, n) — so the shared defaults are the answer and no format_* override is written. That absence is the point worth reviewing: the mixins are listed with the reason no override is needed, recorded in the class body rather than left for the next reader to re-derive.

The tests follow the accessor and naming rebuilds. The DDL-source accessors split: column_type() is a declaration accessor, SQL-type reads go to column_data_type() / columns_data_type()(), and columns_type is gone. DateTimeColumn became TimestampColumn (the column layer models values, and datetime is a vendor spelling), and DateTimeSupport became TimestampSupport. Test code that built predicates through the bare Column's removed operator sugar now builds them explicitly as nodes.

The round-trip matrix was made load-bearing-aware. Core reports a missing formatter and a declined feature through the same UnsupportedFeatureError, and this matrix allowed both identically — which let one of them hide. A class whose formatter this dialect never mixed in read as "PostgreSQL lacks the feature" and passed. Turning trim/lpad/rpad/repeat into nodes made the hole matter: a backend that shipped no format_lpad_expression would answer every .lpad(...) with that refusal, and nothing could tell it apart from a genuine "not on this engine". The two are now told apart by who is refusing — a probe inside a formatter names the feature it lacks, while the dispatch frames the method name it could not find. Only the second is a wiring fact about this backend, so only the second has to be accounted for: UNMODELLED_FORMATTERS groups the formatters PostgreSQL genuinely does not have (SQL:2003 PIVOT/UNPIVOT and the row-source root, today) with the reason, and test_unmodelled_formatter_list_is_exact pins the table in both directions — a formatter that starts existing moves its classes out and the pin fails until the entry goes, and a class that starts needing an unlisted method fails instead of passing as "unsupported".

The version gates were found empty by CI, four times over. A gate that returns True unconditionally tells the dialect it can render a type on any server, and the server disagrees at execution — the expensive way to find out, because the DDL is built, sent and rejected, and the error names a table rather than the version gap that caused it. postgres_jsonpath (12), postgres_xid8 (13), the six multiranges (14) and postgres_macaddr8 (10) were each found by CI, one at a time, and are now gated. Two more were simply wrong and are corrected with them: supports_jsonb_subscript claimed 14.0+ for a capability PostgreSQL does not have at all (SQL subscripting covers arrays and composites; jsonb is read with -> / ->> or the SQL/JSON path language), and supports_json_arrow_operators returned True unconditionally, which on 9.2 advertised SQL the parser rejects. The boundaries are measured against the scenario servers rather than read off the release notes, because the notes and the servers disagreed about jsonpath — the notes say 13, and 12 has it. KNOWN_VERSIONED_TYPES records every floor and the server it was read off, and the two test classes pin the two directions separately: a versioned type must not answer yes unconditionally, while text/integer/varchar saying yes is correct on purpose and is not a to-do list.

Coincidental fixes found while aligning. PostgresUUIDMixin and PostgresUUIDAdapter are new, registered with allow_override because they claim the same {uuid.UUID: {str}} pair the core adapter does, and placed before core's UUIDMixin so PostgreSQL's version-13-built-in-or-uuid-ossp answer wins over a single fixed spelling. PostgresArrayEnhancedMixin moved from the backend block to before ArrayMixin, where it answers True to a question the core answers False for — listed after, the core default was what every caller actually saw. JSONSupport is de-listed from the protocol block because PostgresJSONBEnhancedSupport derives from it and listing a base ahead of its own subclass is not a consistent MRO (it also put supports_json_type in the MRO four times); isinstance(dialect, JSONSupport) is unchanged. The two supports_uuid_generation probes that shared one name and asked two different questions — "is uuid-ossp installed" versus "can this server generate a UUID" — are separated: supports_uuid_ossp_extension is the narrower question, and the merged answer lives only in core's UUIDSupport, folded by PostgresUUIDMixin. format_cast_expression renders the target through expr.target_type.to_sql() instead of interpolating it, which had produced col::IntegerType() rather than col::INTEGER. And PostgresTableMixin's data-type validation regex now accepts underscores, which no valid PostgreSQL type name was allowed to contain.

The CI legs. Python 3.15 becomes a required leg rather than an experimental one: the previous test-experimental-python-315 job was if: ${{ false }} with continue-on-error: true on a 3.15.0-rc.2 pair. The free-threaded leg is dropped — psycopg does not support free-threaded builds (psycopg/psycopg#1095) — but the free-threaded marker stays in the job name, because two legs named alike is how a GIL pass and a free-threaded failure came to look like one job that both passed and failed. The benchmark job moves to 3.15. requirements-dev.txt pins pydantic 2.12.2 / pydantic-core 2.41.4 for Python 3.13 including the free-threaded 3.13t build, restated locally because that file is installed before the core package and an unpinned pydantic there picks a pydantic-core whose only cp313t distribution is an sdist that cannot build. CI installs the core from its release/v1.0.0.dev30 branch now that the rebuild has merged, and the testsuite from test/typed-column-contracts.

Type of change

feat: A new feature
docs
perf
test: Adding missing tests and correcting existing tests for the new feature
chore
ci: Changes to CI configuration (Python 3.15 required leg, benchmark interpreter, Pydantic pins)
build: Pydantic pinning for Python 3.13 / 3.13t
revert

Breaking Change

Yes, for end-users (backward-incompatible API changes)
Yes, for backend developers (internal architectural changes)

Applicable rhosocial-activerecord Version Range: >=1.0.0,<2.0.0 — and, concretely, this branch requires the core at or above 1.0.0.dev30, the release that carries the rebuild. It is not compatible with the previous core.

Impact on End-Users — yes. This backend is consumed by people building on PostgreSQL-backed ActiveRecord, and the core's six end-user changes reach them through the dialect. The ones visible in this repo's own surface:

Change	Migration
`supports_data_type_postgres_jsonpath` answered True unconditionally	now refuses below PostgreSQL 12; catch `UnsupportedFeatureError` or target 12+
`supports_data_type_postgres_xid8`	now refuses below 13
the six `postgres_*multirange` probes	now refuse below 14
`supports_data_type_postgres_macaddr8`	now refuses below 10
`supports_jsonb_subscript` claimed 14.0+	now always False — PostgreSQL has no `jsonb['key']`; read with `->` / `->>` or the SQL/JSON path language
`supports_json_arrow_operators` returned True unconditionally	now refuses below 9.3; on 9.2 an arrow was a syntax error the probe advertised
`DateTimeColumn` / `DateTimeSupport` names	renamed `TimestampColumn` / `TimestampSupport`; rename the import
bare `Column` lost its operators	use a typed column (`StringColumn`, `NumericColumn`, `ArrayColumn`, …) or build the node explicitly (`ComparisonPredicate`, `InPredicate`, `BinaryArithmeticExpression`)
`DDLSource.column_type()` is now a declaration accessor	SQL-type reads move to `column_data_type()` / `columns_data_type()()`; `columns_type` is gone
`PostgresUuidOssSupport.supports_uuid_generation`	renamed `supports_uuid_ossp_extension` for the narrower question; ask `supports_uuid_generation()` for the merged answer
`PostgresCharacterVaryingType`	no longer exported from `...impl.postgres.expression`

Five of the rows above are the expected consequence of the repair rather than of the rebuild: those probes used to promise the wrong answer, and a caller that took them at their word rendered SQL the server rejects. The failure is now a refusal that names the version gap.

Impact on Backend Developers (Custom Implementations) — yes.

A dialect that renders must compose the mixins whose operations it supports — for the new nodes, TrimMixin / LpadMixin / RpadMixin / RepeatMixin — or write a format_* override where its spelling differs. This backend composes all four and writes no override; that is the case reviewers should check rather than assume.
ColumnTypeSupport is a protocol a backend must implement. The shape is a table of common Python type → column class, with None as an allowed refusal, and a refusal has to be a considered answer: timedelta is refused here because no core column class names interval's operations, not because the entry was forgotten.
format_* signature changes from core: expr is annotated with its concrete expression type rather than Any or absent.
MRO discipline is load-bearing in a way it was not before. A backend mixin that overrides a core default must be listed before the core mixin, which is why PostgresUUIDMixin precedes UUIDMixin and PostgresArrayEnhancedMixin now precedes ArrayMixin; and a derived protocol should not be listed alongside its base, which is why JSONSupport is de-listed. Both were live bugs, not tidying.
The conformance surfaces (positive/negative protocol lists) must classify ColumnTypeSupport and, after the rename, TimestampSupport.

Related Repositories/Packages

This change cannot be used alone; the coordinated branch in each repo is feature/typed-columns (testsuite: test/typed-column-contracts).

Repository	Branch	State	Required before this merge?
`python-activerecord`	`release/v1.0.0.dev30` (#126)	merged	Yes — already landed
`python-activerecord-testsuite`	`test/typed-column-contracts`	pushed, not yet merged to a release branch	Yes, and it must land before the backends
`python-activerecord-{mysql,mariadb,oracle,sqlserver,firebird,clickhouse,snowflake,bigquery}`	`feature/typed-columns`	each ahead of its own release	No — sibling PRs, independent of this one

Note on the install references: this branch's CI installs the core from @release/v1.0.0.dev30 (its release branch, now that the rebuild has merged) and the testsuite from @test/typed-column-contracts. That testsuite reference is intentionally still a branch rather than a release — it is what makes this PR's test legs meaningful now, and it must be re-pointed to the testsuite's release branch once test/typed-column-contracts has landed, so the release branch does not keep depending on a branch that will be deleted.

Testing

I have added new tests to cover my changes.
I have updated existing tests to reflect my changes.
All existing tests pass — CI is the gate here, see Additional Notes.
Tested against PostgreSQL (the backend this repository owns): the CI grid pairs Python 3.8–3.14 with PostgreSQL 9–18, adds a required Python 3.15 / PostgreSQL 18 leg and a PostgreSQL 19 leg on 3.14, and the version-gate tests additionally construct dialects at 9.0, 10, 11, 12, 13 and 14 to pin the refusals.

Test Plan:

# the column-type suggestion table: every entry answered, both directions pinned
pytest tests/rhosocial/activerecord_postgres_test/feature/backend/types/test_column_type_suggestions.py

# the version gates: empty gates, and every versioned type gated both ways
pytest tests/rhosocial/activerecord_postgres_test/feature/backend/types/test_data_type_formatting.py

# the node-class matrix: rendering dispatch gaps told apart from feature probes
pytest tests/rhosocial/activerecord_postgres_test/feature/backend/test_expression_roundtrip_all.py

# the new coverage added alongside the alignment
pytest tests/rhosocial/activerecord_postgres_test/feature/backend/postgres/test_cast_integration.py
pytest tests/rhosocial/activerecord_postgres_test/feature/backend/postgres/test_domain_expression_roundtrip.py
pytest tests/rhosocial/activerecord_postgres_test/feature/backend/postgres/test_extension_type_execution.py
pytest tests/rhosocial/activerecord_postgres_test/feature/backend/postgres/test_extension_type_operations.py
pytest tests/rhosocial/activerecord_postgres_test/feature/backend/postgres/test_json_path_modes.py
pytest tests/rhosocial/activerecord_postgres_test/feature/backend/postgres/test_uuid_values.py

# the DDL-source accessor and DDL-object contracts
pytest tests/rhosocial/activerecord_postgres_test/feature/backend/postgres/test_ddl_source_collection.py
pytest tests/rhosocial/activerecord_postgres_test/feature/backend/postgres/test_ddl_objects.py

Local verification is intentionally partial: this project forbids local full-suite runs, so CI is the gate. What answers for the whole is the GitHub Actions test.yml matrix, which pairs Python 3.8–3.14 with PostgreSQL 9–18, adds the required Python 3.15 / PostgreSQL 18 leg and a PostgreSQL 19 leg on 3.14, and runs the FastAPI benchmark job on 3.15 / 18.

Checklist

My code follows the project's code style guidelines.
I have performed a self-review of my own code.
I have commented my code, particularly in hard-to-understand areas.
I have made corresponding changes to the documentation (docstrings, README.md).
My changes generate no new warnings.
Changelog fragment — not yet written. By this runbook's PR-first rule the fragment is created after the PR exists, named with the real PR number (Steps 14–15).
I have verified that my changes do not introduce SQL injection vulnerabilities (all rendering stays parameterised; the broken predicate path the rebuild closes was the one that silently produced a Python bool).
I have checked for potential performance regressions (the new gates are a version-tuple comparison on a path that already read self.version).

Additional Notes

Two things a reviewer should not expect to find here. The format_* annotations that core typed on its own side (44 of 691 org-wide) are not backfilled in this backend — the remaining ones are follow-up work, as they are in every backend. And the supports_data_type_* probe family that core retired is not yet removed from this backend's declaration surface beyond the two probes noted above; the four backends that carry their own large probe sets (mysql, mariadb, firebird, clickhouse) are the shape of that rework, and this backend is not one of them.

What the alignment surfaced, stated plainly. Six of the eleven rows in the breaking-change table are not consequences of the core rebuild but bugs it exposed: four probes that answered True for a server version that predates the type, one subscript PostgreSQL does not have at all, and one arrow operator on a version whose parser rejects it. Beyond the table, two MRO-order bugs surfaced the same way — PostgresArrayEnhancedMixin listed after ArrayMixin and PostgresUUIDMixin after UUIDMixin, each of which made the backend's own answer dead code in favour of a core default. All of it is fixed here because the rebuild's tests are what found it, and the probe repairs are listed as breaking because a caller that took the old answer at its word rendered SQL the server rejects, where it now gets a refusal naming the version gap.

… column_data_type)

… operators

…. 3.13t)

…tively)

…tay silent

setup-python now ships Python 3.15.0, so the release-candidate pins are replaced with plain 3.15 (both the GIL and the free-threaded legs), and the exemptions that made the leg optional are removed: - the "(experimental)" tag in the job names, and the experimental job ids; - `continue-on-error: true`, which let the leg report failure without failing the run; - postgres' disable gate (`if:` set to false) and its comment. The remaining matrix entries are unchanged: every backend keeps a GIL and a free-threaded leg on 3.15, now on equal footing with the other interpreters.

psycopg does not support free-threaded builds -- it is tracked upstream as psycopg/psycopg#1095 -- so a free-threaded leg on this backend can only ever fail at install or at connection. The 3.15 job keeps its GIL leg only, which is what every other interpreter here already runs. The job name now carries the free-threaded marker. It renders empty today, but the matrix had two 3.15 legs that both displayed as "Python 3.15 / postgres 18": the GIL leg passed and the free-threaded leg failed, which read as one job that had simultaneously passed and failed. With one leg the collision is invisible; keeping the marker stops it recurring if a second leg is ever added.

… Python The benchmark job was pinned to 3.14 while the test matrix had already moved to 3.15, so the newest interpreter this package supports was never exercised under load. It now runs on 3.15 (core, postgres and mysql) -- job id, job name and the summary text included -- while the per-version test legs stay on their own interpreters. The README carried the Pydantic requirement as one coarse line. It now carries the actual pairing: 3.8 pins an exact pair (the last installable there), 3.13 and 3.13t pin the pair whose pydantic-core still has a wheel, and the rest stay open. The reason column records why the 3.13 row cannot simply track the newest release, and the free-threaded paragraphs now state per backend which legs exist and why.

…as merged While these branches were in development, CI installed the core from `@feature/typed-columns`, because the release branch did not carry the rebuilt column layer yet. That branch has now merged into `release/v1.0.0.dev30` (PR #126), so the feature reference is both stale and about to disappear with branch deletion. Point the core dependency at `release/v1.0.0.dev30` instead — the same branch these backends are merging into, which is what the release-side CI already used. The testsuite reference is deliberately left alone: that branch (`test/typed-column-contracts`) has not merged into its release yet, and pointing at a release that lacks the new contracts would break every backend at once. It moves when the testsuite merges. ClickHouse expresses the dependency as a `ref:` on `actions/checkout` rather than as a pip requirement; that one is switched the same way.

sonarqubecloud · 2026-10-11T07:58:38Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

codecov · 2026-10-11T08:07:26Z

Codecov Report

❌ Patch coverage is 80.37975% with 155 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.93%. Comparing base (6733846) to head (ad028cc).

Files with missing lines	Patch %	Lines
...record/backend/impl/postgres/functions/pgvector.py	42.64%	24 Missing and 15 partials ⚠️
...erecord/backend/impl/postgres/expression/values.py	86.12%	24 Missing and 5 partials ⚠️
...tiverecord/backend/impl/postgres/functions/enum.py	10.52%	17 Missing ⚠️
...ecord/backend/impl/postgres/functions/geometric.py	7.69%	12 Missing ⚠️
...ckend/impl/postgres/mixins/types/jsonb_enhanced.py	89.81%	7 Missing and 4 partials ⚠️
...impl/postgres/mixins/types/data_type_formatting.py	91.57%	5 Missing and 3 partials ⚠️
...verecord/backend/impl/postgres/functions/orafce.py	12.50%	7 Missing ⚠️
...erecord/backend/impl/postgres/functions/postgis.py	72.22%	3 Missing and 2 partials ⚠️
...ord/backend/impl/postgres/functions/text_search.py	16.66%	5 Missing ⚠️
...verecord/backend/impl/postgres/expression/types.py	88.88%	3 Missing and 1 partial ⚠️
... and 8 more

Additional details and impacted files

@@                   Coverage Diff                    @@
##           release/v1.0.0.dev17      #62      +/-   ##
========================================================
+ Coverage                 75.42%   75.93%   +0.51%     
========================================================
  Files                       346      349       +3     
  Lines                     16111    16343     +232     
  Branches                   2741     2731      -10     
========================================================
+ Hits                      12151    12410     +259     
+ Misses                     2969     2952      -17     
+ Partials                    991      981      -10

Flag	Coverage Δ
postgres-18	`75.93% <80.37%> (+0.51%)`	⬆️
python-3.14	`75.93% <80.37%> (+0.51%)`	⬆️
unittests	`75.93% <80.37%> (+0.51%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

vistart added 14 commits October 9, 2026 16:13

feat: the typed-columns column-type suggestion table

dc0835a

feat: align the column-type table with the rebuilt core protocol

d01e5de

fix: follow the core's DateTimeColumn -> TimestampColumn rename

120a1d7

test: follow the rebuilt DDL-source accessor semantics (column_type /…

a4a7c6e

… column_data_type)

test: build predicates explicitly, now that the bare Column offers no…

e234cba

… operators

test: build predicates explicitly, now that the bare Column offers no…

5680cb3

… operators

deps: pin pydantic 2.12.2 / pydantic-core 2.41.4 on Python 3.13 (incl…

56ea312

…. 3.13t)

feat: render the new pad/repeat/trim nodes (PostgreSQL spells them na…

57f1c29

…tively)

test: enumerate every core node class so a missing formatter cannot s…

ae7b044

…tay silent

test: follow the core's rename of DateTimeSupport to TimestampSupport

c87bcf9

vistart added the enhancement New feature or request label Oct 11, 2026

vistart self-assigned this Oct 11, 2026

chore: add changelog fragment for PR #62

ad028cc

vistart merged commit b773da9 into release/v1.0.0.dev17 Oct 11, 2026
17 checks passed

Navigation Menu

Repository navigation

feat(dialect): follow the core's column-layer rebuild — column types, node formatters, and the version gates - #62

feat(dialect): follow the core's column-layer rebuild — column types, node formatters, and the version gates#62
vistart merged 15 commits into
release/v1.0.0.dev17from
feature/typed-columns

vistart commented Oct 11, 2026 •

edited

Loading

Uh oh!

sonarqubecloud Bot commented Oct 11, 2026

Uh oh!

codecov Bot commented Oct 11, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

vistart commented Oct 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Breaking Change

Related Repositories/Packages

Testing

Checklist

Additional Notes

Uh oh!

sonarqubecloud Bot commented Oct 11, 2026

Quality Gate passed

Uh oh!

codecov Bot commented Oct 11, 2026

Codecov Report

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vistart commented Oct 11, 2026 •

edited

Loading