Skip to content

feat(postgres): add pg_partman partition support - #43

Merged
vistart merged 17 commits into
release/v1.0.0.dev15from
feature/partitioning-support
Jun 12, 2026
Merged

vistart merged 17 commits into
release/v1.0.0.dev15from
feature/partitioning-support

Conversation

@vistart

@vistart vistart commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

Description

Adds PostgreSQL partitioning support and pg_partman integration for the PostgreSQL backend.

This PR introduces pg_partman-related DDL expressions, protocol exports, dialect/mixin support, backend integration points, provider coverage, and focused tests for partition operations and pg_partman behavior. It also updates examples and existing test providers to work with the new partitioning capability.

Adds # (issue)

Type of change

  • feat: A new feature
  • docs: Documentation only changes (if related to the new feature)
  • perf: A code change that improves performance (if part of the feature)
  • test: Adding missing tests or correcting existing tests (for the new feature)
  • chore: Changes to the build process or auxiliary tools and libraries suchs as documentation generation (if related to the new feature)
  • ci: Changes to our CI configuration files and scripts (if related to the new feature)
  • build: Changes that affect the build system or external dependencies (if related to the feature)
  • revert: Reverts a previous commit

Breaking Change

  • Yes
  • No

N/A

Applicable rhosocial-activerecord Version Range:

Compatible with rhosocial-activerecord 1.0.0.dev15 release branch expectations.

Related Repositories/Packages

  • python-activerecord: depends on core expression/dialect/protocol architecture already present in the release branch.
  • python-activerecord-postgres: this PR implements PostgreSQL-specific partitioning and pg_partman support.

Testing

  • I have added new tests to cover my changes.
  • I have updated existing tests to reflect my changes.
  • All existing tests pass.
  • This change has been tested on [specify backend/DB versions].

Test Plan:

Local tests were skipped during PR preparation by request. CI/CD should run after PR creation.

Recommended verification:

PYTHONPATH=src .venv3.8/bin/pytest tests/ -v --tb=short

Checklist

  • My code follows the project's code style guidelines.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation (Docstrings, README.md, etc.).
  • My changes generate no new warnings.
  • I have added a Changelog fragment (changelog.d/<issue_number>.<type>.md).
  • I have verified that my changes do not introduce SQL injection vulnerabilities.
  • I have checked for potential performance regressions.

Additional Notes

Commit included in this PR:

  • 036669d feat(postgres): add pg_partman partition support

Primary changed areas:

  • PostgreSQL DDL partition expressions and pg_partman expressions.
  • PostgreSQL dialect, mixins, protocols, backend exports, and transaction/backend integration.
  • Partition provider and partition operation tests.
  • Existing PostgreSQL tests and examples updated for compatibility with the new feature.

Add PostgreSQL partition and pg_partman support with dialect,
expression, protocol, mixin, provider, and test updates.

Also apply ruff formatting/lint cleanup across the current change set
and add low-risk backend signature annotations.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@vistart vistart self-assigned this Jun 6, 2026
@vistart vistart added the feature label Jun 6, 2026
vistart and others added 3 commits June 7, 2026 02:38
Update partition value security tests to exercise the public
format_partition_value expression path instead of the removed
_format_partition_value helper.

Declare supports_table_partitioning in PostgresTableSupport so the
protocol conformance reverse coverage matches PostgresTableMixin.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove supports_table_partitioning from PostgresTableMixin so the
partition capability remains owned by PartitionSupport and
PostgresPartitionMixin. This keeps PostgreSQL protocol interfaces
non-overlapping while preserving dialect support through the partition
mixin.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move PostgreSQL implementations of generic PartitionSupport capability
methods into PostgresPartitionMixin and ensure that mixin takes
precedence over the generic PartitionMixin in the dialect MRO.

Keep PostgresTableMixin focused on table capabilities only, and move
partition feature detection coverage from table tests into DDL protocol
and partition mixin tests. Explicitly include PartitionSupport in the
protocol conformance checks to document the inheritance boundary.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@codecov

codecov Bot commented Jun 6, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.39779% with 42 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.65%. Comparing base (df84e80) to head (f796a99).

Files with missing lines Patch % Lines
...cord/backend/impl/postgres/mixins/ddl/partition.py 85.21% 11 Missing and 10 partials ⚠️
...activerecord/backend/impl/postgres/mixins/table.py 76.19% 4 Missing and 1 partial ⚠️
...d/impl/postgres/protocols/extensions/pg_partman.py 50.00% 4 Missing and 1 partial ⚠️
...d/backend/impl/postgres/protocols/ddl/partition.py 62.50% 3 Missing ⚠️
.../backend/impl/postgres/expression/ddl/partition.py 93.33% 1 Missing and 1 partial ⚠️
...backend/impl/postgres/expression/ddl/pg_partman.py 95.91% 1 Missing and 1 partial ⚠️
...kend/impl/postgres/mixins/extensions/pg_partman.py 96.66% 1 Missing and 1 partial ⚠️
...iverecord/backend/impl/postgres/protocols/table.py 60.00% 2 Missing ⚠️
Additional details and impacted files
@@                   Coverage Diff                    @@
##           release/v1.0.0.dev15      #43      +/-   ##
========================================================
+ Coverage                 71.13%   71.65%   +0.51%     
========================================================
  Files                       232      233       +1     
  Lines                     10284    10563     +279     
  Branches                   1619     1665      +46     
========================================================
+ Hits                       7316     7569     +253     
- Misses                     2441     2461      +20     
- Partials                    527      533       +6     
Flag Coverage Δ
postgres-18 71.65% <88.39%> (+0.51%) ⬆️
python-3.14 71.65% <88.39%> (+0.51%) ⬆️
unittests 71.65% <88.39%> (+0.51%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

vistart and others added 13 commits June 7, 2026 03:43
Disable the Python 3.15 experimental workflow job until the beta
toolchain stabilizes.

Move pg_partman part_config cleanup into a dedicated expression and
formatter, then update partition operation tests to use expression
helpers for DDL and pg_partman operations instead of hand-built SQL.
Validation queries remain raw SQL because they inspect PostgreSQL
runtime metadata and routing behavior directly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add PostgreSQL partition metadata query expression and align generic partition capability methods with PostgreSQL declarative partitioning.

Update provider and tests to use Expression-Dialect generated SQL for partition metadata, truncate, and drop paths.

Also update PR template links to repository-local .claude rule documents.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add production-style PostgreSQL time partition scenarios covering microsecond boundaries, future partition preparation, continuous partition pruning, index usage, metadata changes, and cold archival via DETACH PARTITION.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fix PostgreSQL DEFAULT partition DDL by omitting FOR VALUES for DEFAULT RANGE and LIST partitions.

Also add expression and real backend coverage for DEFAULT partition maintenance and pg_partman configuration maintenance.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…dge files

- Add 5 eager loading test bridge files (one-to-one import)
- Split test_relations_with.py into sync/async bridge files
- Remove old combined test_relations_with.py bridge
- Remove partition.interfaces dependency from provider registry
- Fix async_blog_fixtures to import AsyncUser from async_models for fixture parity

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add setup_profile_fixtures() and setup_async_profile_fixtures() to Postgres QueryProvider
- Create profiles.sql schema with FK reference to users table
- Add bridge files for eager_loading_scenarios_sync/async tests

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test_eager_loading_scenarios.py imported
rhosocial.activerecord.testsuite.feature.query.test_eager_loading_scenarios
which does not exist in the testsuite's release/v1.0.0.dev15 branch.
This caused ModuleNotFoundError during test collection on all
Python/PostgreSQL versions in CI.

The async and sync variants
(test_eager_loading_scenarios_async.py,
test_eager_loading_scenarios_sync.py) already exist and work correctly.
setup_profile_fixtures() imported Profile from the base models module,
but User is selected from a version-specific module (e.g. models_py312)
via _select_model_class(). The profile relation on the selected User
class resolves to the version-specific Profile class, which had no
backend configured, causing 'No backend configured' errors in all
eager loading tests using with_('profile').

Fix: add Profile to the module-level version-conditional imports
using the same _select_model_class() pattern as User, Order, etc.
… concurrently assertion

- Override format_column_definition in PostgresTableMixin to permit
  PostgreSQL array types (e.g. TEXT[], INTEGER[]) that contain brackets
- Fix test_detach_concurrently_pg14 assertion: PR refactoring moved
  CONCURRENTLY from adjacent to DETACH to a separate token after the
  partition name
The method was added to PostgresTableMixin but not declared in
the matching protocol, causing protocol conformance test failure.
…, attach improvements, and comprehensive docstrings

- Add ATTACH CONCURRENTLY support (PG 14+) to PostgresAttachPartitionExpression
- Add ATTACH DEFAULT partition support via partition_values={'default': True}
- Refactor format_partition_metadata_query to use dynamic schema filtering
- Complete pg_partman create_parent with all 7 optional parameters
- Add named expression functions for partition auto-management (create_next_monthly_partition,
  create_range_partition_for_month, create_range_partition_for_quarter,
  create_range_partitions_for_interval) via rhosocial-activerecord resolver
- Add english docstrings (Args/Returns/Raises) to all partition classes and methods
- Add tests: ATTACH CONCURRENTLY, ATTACH DEFAULT, multi-column partition keys,
  expression boundary values, DETACH FINALIZE, pg_partman optional params,
  metadata query schema filtering, concurrent_attach capability
- Verify on real PostgreSQL 18, 14, 11, 10, 9: all 150+ unit + 27 integration tests pass
@sonarqubecloud

Copy link
Copy Markdown

@vistart
vistart merged commit 58665f3 into release/v1.0.0.dev15 Jun 12, 2026
16 checks passed
@vistart vistart mentioned this pull request Jun 15, 2026
7 of 12 tasks
vistart added a commit that referenced this pull request Jun 15, 2026
* feat(postgres): add pg_partman partition support

Add PostgreSQL partition and pg_partman support with dialect,
expression, protocol, mixin, provider, and test updates.

Also apply ruff formatting/lint cleanup across the current change set
and add low-risk backend signature annotations.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(test): align partition tests with public protocol API

Update partition value security tests to exercise the public
format_partition_value expression path instead of the removed
_format_partition_value helper.

Declare supports_table_partitioning in PostgresTableSupport so the
protocol conformance reverse coverage matches PostgresTableMixin.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(postgres): avoid table and partition protocol overlap

Remove supports_table_partitioning from PostgresTableMixin so the
partition capability remains owned by PartitionSupport and
PostgresPartitionMixin. This keeps PostgreSQL protocol interfaces
non-overlapping while preserving dialect support through the partition
mixin.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(postgres): separate table and partition capabilities

Move PostgreSQL implementations of generic PartitionSupport capability
methods into PostgresPartitionMixin and ensure that mixin takes
precedence over the generic PartitionMixin in the dialect MRO.

Keep PostgresTableMixin focused on table capabilities only, and move
partition feature detection coverage from table tests into DDL protocol
and partition mixin tests. Explicitly include PartitionSupport in the
protocol conformance checks to document the inheritance boundary.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(postgres): expressionize partition operation helpers

Disable the Python 3.15 experimental workflow job until the beta
toolchain stabilizes.

Move pg_partman part_config cleanup into a dedicated expression and
formatter, then update partition operation tests to use expression
helpers for DDL and pg_partman operations instead of hand-built SQL.
Validation queries remain raw SQL because they inspect PostgreSQL
runtime metadata and routing behavior directly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(postgres): align partition capability mapping

Add PostgreSQL partition metadata query expression and align generic partition capability methods with PostgreSQL declarative partitioning.

Update provider and tests to use Expression-Dialect generated SQL for partition metadata, truncate, and drop paths.

Also update PR template links to repository-local .claude rule documents.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(partition): add production time partition operation scenarios

Add production-style PostgreSQL time partition scenarios covering microsecond boundaries, future partition preparation, continuous partition pruning, index usage, metadata changes, and cold archival via DETACH PARTITION.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(ddl): generate DEFAULT partition clauses correctly

Fix PostgreSQL DEFAULT partition DDL by omitting FOR VALUES for DEFAULT RANGE and LIST partitions.

Also add expression and real backend coverage for DEFAULT partition maintenance and pg_partman configuration maintenance.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(query): add eager loading and split sync/async relation with bridge files

- Add 5 eager loading test bridge files (one-to-one import)
- Split test_relations_with.py into sync/async bridge files
- Remove old combined test_relations_with.py bridge
- Remove partition.interfaces dependency from provider registry
- Fix async_blog_fixtures to import AsyncUser from async_models for fixture parity

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(provider): implement setup_profile_fixtures for Postgres sync/async

- Add setup_profile_fixtures() and setup_async_profile_fixtures() to Postgres QueryProvider
- Create profiles.sql schema with FK reference to users table
- Add bridge files for eager_loading_scenarios_sync/async tests

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(dialect): split mixins into directory-based modules

* fix(ci): remove bridge file referencing non-existent testsuite module

test_eager_loading_scenarios.py imported
rhosocial.activerecord.testsuite.feature.query.test_eager_loading_scenarios
which does not exist in the testsuite's release/v1.0.0.dev15 branch.
This caused ModuleNotFoundError during test collection on all
Python/PostgreSQL versions in CI.

The async and sync variants
(test_eager_loading_scenarios_async.py,
test_eager_loading_scenarios_sync.py) already exist and work correctly.

* fix(test): import Profile from version-specific module, not base models

setup_profile_fixtures() imported Profile from the base models module,
but User is selected from a version-specific module (e.g. models_py312)
via _select_model_class(). The profile relation on the selected User
class resolves to the version-specific Profile class, which had no
backend configured, causing 'No backend configured' errors in all
eager loading tests using with_('profile').

Fix: add Profile to the module-level version-conditional imports
using the same _select_model_class() pattern as User, Order, etc.

* fix(test): allow TEXT[]/INTEGER[] in column definition and fix detach concurrently assertion

- Override format_column_definition in PostgresTableMixin to permit
  PostgreSQL array types (e.g. TEXT[], INTEGER[]) that contain brackets
- Fix test_detach_concurrently_pg14 assertion: PR refactoring moved
  CONCURRENTLY from adjacent to DETACH to a separate token after the
  partition name

* fix(protocol): declare format_column_definition in PostgresTableSupport

The method was added to PostgresTableMixin but not declared in
the matching protocol, causing protocol conformance test failure.

* feat(partition): complete partitioning support with named expressions, attach improvements, and comprehensive docstrings

- Add ATTACH CONCURRENTLY support (PG 14+) to PostgresAttachPartitionExpression
- Add ATTACH DEFAULT partition support via partition_values={'default': True}
- Refactor format_partition_metadata_query to use dynamic schema filtering
- Complete pg_partman create_parent with all 7 optional parameters
- Add named expression functions for partition auto-management (create_next_monthly_partition,
  create_range_partition_for_month, create_range_partition_for_quarter,
  create_range_partitions_for_interval) via rhosocial-activerecord resolver
- Add english docstrings (Args/Returns/Raises) to all partition classes and methods
- Add tests: ATTACH CONCURRENTLY, ATTACH DEFAULT, multi-column partition keys,
  expression boundary values, DETACH FINALIZE, pg_partman optional params,
  metadata query schema filtering, concurrent_attach capability
- Verify on real PostgreSQL 18, 14, 11, 10, 9: all 150+ unit + 27 integration tests pass

* chore: add changelog fragment for PR #43

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@vistart
vistart deleted the feature/partitioning-support branch October 1, 2026 03:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant