Repository navigation
feat: PGQ enhancement - fulltext search, cursor pollution fix, and PG 19 compatibility - #49
Merged
Merged
Conversation
…ported on PG 19
PostgreSQL 19 does not support:
- Variable-length (quantified) path patterns (+/*/{n,m})
- Comma-separated multiple patterns in a single MATCH clause
Add capability overrides returning False, document known limitations,
and add tests for the new gating methods.
…ackend' in cursor pollution tests The cursor pollution tests referenced 'pg_backend' and 'async_pg_backend' which do not exist as pytest fixtures. The correct names are 'postgres_backend' and 'async_postgres_backend', matching the actual fixture definitions used by the rest of the test suite.
…n bridges - Update bridge files to match testsuite rename: test_sync_queries -> test_queries - Add 4 new async relation bridge files (test_base_async, test_cache_async, test_interfaces_async, test_nested_relationship_access_async) - Include pgq/ example directory
… relation benchmark - derived_field/conftest.py + test_derived_field_testsuite.py - interface/test_threadsafe_dict.py (sync + async combined) - benchmark/relation/conftest.py + test_cache_comparison.py
execute() is atomic per DB-API 2.0 — the extra fetchall() after fetchone() is unnecessary and adds a round-trip for no benefit. Follow-up to commit ab3be40 which introduced these defensive calls.
…ssion-based DDL - Delete composite_pk/derived_field bridge files, schema SQL, and providers - Create tests/providers/fixtures/ with TABLE_EXPRESSIONS (basic, query, mixins, events) - Replace _load_postgres_schema with CreateTableExpression/DropTableExpression DDL - Update all query/relation bridge imports to new subdirectory paths - Fix _setup_model_async -> _setup_async_model typo in query provider
…est, fix composite_pk setup - Move 70 bridge files from flat paths to testsuite-matching subdirectories - Create 6 missing bridge files (composite_pk crud/query/cte/set_operations, derived_field_basic, example_basic_fixtures) - Add missing basic/worker, query/worker, interface conftest bridge files - Rename test_worker_transaction_isolation -> test_transaction_isolation - Fix setup_order_item_model to use basic TABLE_EXPRESSIONS for composite PK order_items table
…nftest - Use PROVIDER_KEY_ASYNC instead of undefined PROVIDER_KEY - Fix method call names: setup_async_user_model -> setup_user_model (await) - Fix cleanup_after_test_async -> cleanup_after_test (await) - Align method calls with testsuite IRelationAsyncProvider interface
…hema call from relation conftest RelationAsyncProvider._setup_user_post_comment_async() already handles all three tables in a single call.
…ss-test contamination Add missing tables to PostgreSQL provider cleanup lists so all tables created during a test are reliably removed afterwards: - basic provider (sync + async): bulk_users - query provider (sync + async): profiles These were created by setup methods (setup_bulk_user_model, setup_profile_fixtures) but absent from the cleanup drop list, leading to orphaned tables across sequential test runs and cross-scenario contamination.
…p, fix bare excepts - RelationSyncProvider/RelationAsyncProvider cleanup_after_test now drops all 12 created tables (employees, departments, authors, books, chapters, profiles, users, posts, comments, boundary_*) with CASCADE before disconnect. - BasicConnectionProvider/QueryConnectionProvider cleanup now drops test_users table. - CrudBenchmarkProvider cleanup now drops benchmark_users table. - All 8 bare 'except:' -> 'except Exception:' across basic.py, query.py, mixins.py, events.py providers.
- Generate 61 _async.py bridge files for sync/async pairing compliance - Rename test_eager_loading_scenarios_sync.py -> test_eager_loading_scenarios.py - Rename test_relations_with_sync.py -> test_relations_with.py - Split test_derived_field_basic.py into sync-only + test_derived_field_basic_async.py - Add missing bridge files for basic/fields/ subcategories - Fix testsuite module path in relation/derived/test_derived_field.py
…ields/ Same fix as python-activerecord-mysql: identical leaf filename test_derived_field.py from two different testsuite modules collide under pytest's import model. Drop the basic/fields duplicates to keep only the relation/derived/ bridges, matching SQLite.
Same fix as python-activerecord-mysql: the four __init__.py markers
under tests/benchmark/{,backend,activerecord_bulk,relation} were
removed in commit f146d54, breaking pytest collection on tests that
use relative imports (e.g. 'from .workloads import ...' in
tests/benchmark/backend/test_execute_crud_*.py).
Restore them so pytest treats those directories as packages and the
relative imports resolve.
…-mismatch
- Move cursor.description assertions before cursor.close() in all
8 cursor pollution tests (both sync and async) to fix assert None
- Replace string type names ('INTEGER', 'VARCHAR', 'NUMERIC') with
proper DataType instances (IntegerType, VarCharType, DecimalType)
in test_recursive_cte_graph_traversal.py to fix TypeError
- Rename test_transaction_isolation.py → test_worker_transaction_isolation.py
to avoid import-file-mismatch collision with backend/transaction/
…ssion for depth+1 in CTE tests PostgreSQL does not support prefix function-call syntax for arithmetic operators (+(a, b)). Use the overloaded + operator (creates BinaryArithmeticExpression) which generates correct infix a + b SQL.
…ssertion bugs in CTE tests - Add GroupByHavingClause to fix column 'fund_trace.depth' must appear in GROUP BY - Fix depth 3 expected sum to 800000 (500000 Diana + 300000 Eve) - Fix social network assertions: Charlie at depth 2 (not 1), no Diana in results - These bugs were pre-existing: tests were never runnable before DataType fix
…condition in cleanup_after_test
Instead of hardcoding tables_to_drop list in cleanup_after_test (which
drops ALL tables including those used by concurrent xdist workers),
track only tables that were actually created by _setup_{model,multiple}
calls in each provider instance. Since each test fixture creates a fresh
provider, this naturally isolates table cleanup to just the tables the
current test created, eliminating the race where one test's teardown
drops users/type_adapter_tests/pydantic_validated_models tables that
another concurrent worker is still using.
PostgreSQL supports only a single ON CONFLICT clause per INSERT. Explicitly declare supports_on_conflict_clause=True and supports_multiple_on_conflict_clauses=False to enable the generic gate in DMLMixin.format_on_conflict_clauses. Add targeted tests for the capability switches and single-clause rendering (incl. EXCLUDED).
Confirm PostgreSQL inherits the correct optimistic True/True defaults from the core TableMixin and emits standard CASCADE/RESTRICT tokens. No code change required on the PostgreSQL backend.
…ultiple_on_conflict_clauses to PostgresUpsertSupport protocol
…o avoid cross-scenario table conflicts
…overriding type_adapter_fixtures
…eaking other fixtures
…oid cross-scenario table collisions
…ge file The bridge file imported type_adapter_fixtures directly from the testsuite conftest into the test module namespace. pytest registered that fixture with the test module as its most specific baseid, shadowing the conftest override (feature/basic/conftest.py) that passes the scenario name to the provider. As a result, the sync [postgres_exts] type_adapter tests connected to the default scenario database (5432) instead of postgres_exts (5433), colliding with async [postgres_default] tests on the same type_adapter_tests table and causing 'relation does not exist' failures under xdist concurrency. Remove the import so pytest resolves the conftest override, which correctly passes the scenario name.
- Fix relative imports in parse_type (..expression.types -> ...expression.types) that crashed serial/binary/jsonpath/uuid/net/geom/range/oid/ts/vector parsing with ModuleNotFoundError. - Fix DATETIME mis-parsed as DATE (dispatch order), and CHARACTER/ CHARACTER VARYING/TIMESTAMPTZ/TIMETZ falling back to CustomType due to \b boundary regex. - Add test_data_type_formatting.py covering all PostgreSQL type format and parse branches (data_type_formatting.py coverage 55.67% -> 98%). - Add test_expression_types.py covering synonyms/equality/hash/array equivalence (expression/types.py coverage 76.19% -> 100%). - Cover index fulltext PHRASE mode + multi-column, and upsert capability switches (index.py and upsert.py -> 100%).
11 of 12 tasks
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## release/v1.0.0.dev16 #49 +/- ##
=======================================================
Coverage ? 73.18%
=======================================================
Files ? 277
Lines ? 11333
Branches ? 1837
=======================================================
Hits ? 8294
Misses ? 2439
Partials ? 600
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
8 of 12 tasks
vistart
added a commit
that referenced
this pull request
Aug 15, 2026
… 19 compatibility (#49) * feat(pgq): gate quantified path and comma-separated patterns as unsupported on PG 19 PostgreSQL 19 does not support: - Variable-length (quantified) path patterns (+/*/{n,m}) - Comma-separated multiple patterns in a single MATCH clause Add capability overrides returning False, document known limitations, and add tests for the new gating methods. * fix: use correct fixture names 'postgres_backend' / 'async_postgres_backend' in cursor pollution tests The cursor pollution tests referenced 'pg_backend' and 'async_pg_backend' which do not exist as pytest fixtures. The correct names are 'postgres_backend' and 'async_postgres_backend', matching the actual fixture definitions used by the rest of the test suite. * feat: sync bridge files with testsuite rename and add 4 async relation bridges - Update bridge files to match testsuite rename: test_sync_queries -> test_queries - Add 4 new async relation bridge files (test_base_async, test_cache_async, test_interfaces_async, test_nested_relationship_access_async) - Include pgq/ example directory * feat: add bridge files for derived_field, interface feature tests and relation benchmark - derived_field/conftest.py + test_derived_field_testsuite.py - interface/test_threadsafe_dict.py (sync + async combined) - benchmark/relation/conftest.py + test_cache_comparison.py * perf(backend): remove redundant fetchall() in get_server_version() execute() is atomic per DB-API 2.0 — the extra fetchall() after fetchone() is unnecessary and adds a round-trip for no benefit. Follow-up to commit ab3be40 which introduced these defensive calls. * refactor(postgres): merge composite_pk/derived_field, switch to expression-based DDL - Delete composite_pk/derived_field bridge files, schema SQL, and providers - Create tests/providers/fixtures/ with TABLE_EXPRESSIONS (basic, query, mixins, events) - Replace _load_postgres_schema with CreateTableExpression/DropTableExpression DDL - Update all query/relation bridge imports to new subdirectory paths - Fix _setup_model_async -> _setup_async_model typo in query provider * fix(postgres): move bridge files to subdirectories, add missing conftest, fix composite_pk setup - Move 70 bridge files from flat paths to testsuite-matching subdirectories - Create 6 missing bridge files (composite_pk crud/query/cte/set_operations, derived_field_basic, example_basic_fixtures) - Add missing basic/worker, query/worker, interface conftest bridge files - Rename test_worker_transaction_isolation -> test_transaction_isolation - Fix setup_order_item_model to use basic TABLE_EXPRESSIONS for composite PK order_items table * fix(postgres): fix PROVIDER_KEY_ASYNC and method names in relation conftest - Use PROVIDER_KEY_ASYNC instead of undefined PROVIDER_KEY - Fix method call names: setup_async_user_model -> setup_user_model (await) - Fix cleanup_after_test_async -> cleanup_after_test (await) - Align method calls with testsuite IRelationAsyncProvider interface * fix(postgres): remove non-existent _ensure_user_post_comment_async_schema call from relation conftest RelationAsyncProvider._setup_user_post_comment_async() already handles all three tables in a single call. * fix(postgres): extend cleanup_after_test table lists to ensure no cross-test contamination Add missing tables to PostgreSQL provider cleanup lists so all tables created during a test are reliably removed afterwards: - basic provider (sync + async): bulk_users - query provider (sync + async): profiles These were created by setup methods (setup_bulk_user_model, setup_profile_fixtures) but absent from the cleanup drop list, leading to orphaned tables across sequential test runs and cross-scenario contamination. * fix(postgres): add DROP TABLE to relation/connection/benchmark cleanup, fix bare excepts - RelationSyncProvider/RelationAsyncProvider cleanup_after_test now drops all 12 created tables (employees, departments, authors, books, chapters, profiles, users, posts, comments, boundary_*) with CASCADE before disconnect. - BasicConnectionProvider/QueryConnectionProvider cleanup now drops test_users table. - CrudBenchmarkProvider cleanup now drops benchmark_users table. - All 8 bare 'except:' -> 'except Exception:' across basic.py, query.py, mixins.py, events.py providers. * refactor(test): add missing _async bridge files (61), fix naming - Generate 61 _async.py bridge files for sync/async pairing compliance - Rename test_eager_loading_scenarios_sync.py -> test_eager_loading_scenarios.py - Rename test_relations_with_sync.py -> test_relations_with.py - Split test_derived_field_basic.py into sync-only + test_derived_field_basic_async.py - Add missing bridge files for basic/fields/ subcategories - Fix testsuite module path in relation/derived/test_derived_field.py * fix(test): remove conflicting test_derived_field bridges from basic/fields/ Same fix as python-activerecord-mysql: identical leaf filename test_derived_field.py from two different testsuite modules collide under pytest's import model. Drop the basic/fields duplicates to keep only the relation/derived/ bridges, matching SQLite. * fix(test): restore __init__.py under tests/benchmark/packages Same fix as python-activerecord-mysql: the four __init__.py markers under tests/benchmark/{,backend,activerecord_bulk,relation} were removed in commit f146d54, breaking pytest collection on tests that use relative imports (e.g. 'from .workloads import ...' in tests/benchmark/backend/test_execute_crud_*.py). Restore them so pytest treats those directories as packages and the relative imports resolve. * fix(test): resolve cursor pollution, DataType errors, and import-file-mismatch - Move cursor.description assertions before cursor.close() in all 8 cursor pollution tests (both sync and async) to fix assert None - Replace string type names ('INTEGER', 'VARCHAR', 'NUMERIC') with proper DataType instances (IntegerType, VarCharType, DecimalType) in test_recursive_cte_graph_traversal.py to fix TypeError - Rename test_transaction_isolation.py → test_worker_transaction_isolation.py to avoid import-file-mismatch collision with backend/transaction/ * fix(ci): update benchmark test file names to match testsuite (remove _sync suffix) * fix(test): replace FunctionCall('+') with infix BinaryArithmeticExpression for depth+1 in CTE tests PostgreSQL does not support prefix function-call syntax for arithmetic operators (+(a, b)). Use the overloaded + operator (creates BinaryArithmeticExpression) which generates correct infix a + b SQL. * fix(test): add GROUP BY to fund_trace_aggregation, fix pre-existing assertion bugs in CTE tests - Add GroupByHavingClause to fix column 'fund_trace.depth' must appear in GROUP BY - Fix depth 3 expected sum to 800000 (500000 Diana + 300000 Eve) - Fix social network assertions: Charlie at depth 2 (not 1), no Diana in results - These bugs were pre-existing: tests were never runnable before DataType fix * fix(test): track created tables per provider to eliminate xdist race condition in cleanup_after_test Instead of hardcoding tables_to_drop list in cleanup_after_test (which drops ALL tables including those used by concurrent xdist workers), track only tables that were actually created by _setup_{model,multiple} calls in each provider instance. Since each test fixture creates a fresh provider, this naturally isolates table cleanup to just the tables the current test created, eliminating the race where one test's teardown drops users/type_adapter_tests/pydantic_validated_models tables that another concurrent worker is still using. * feat(postgres): set supports_json_arrow_operators=True * feat(postgres): add ON CONFLICT capability switches PostgreSQL supports only a single ON CONFLICT clause per INSERT. Explicitly declare supports_on_conflict_clause=True and supports_multiple_on_conflict_clauses=False to enable the generic gate in DMLMixin.format_on_conflict_clauses. Add targeted tests for the capability switches and single-clause rendering (incl. EXCLUDED). * test(postgres): add DROP TABLE CASCADE/RESTRICT rendering test Confirm PostgreSQL inherits the correct optimistic True/True defaults from the core TableMixin and emits standard CASCADE/RESTRICT tokens. No code change required on the PostgreSQL backend. * fix(postgres): add missing supports_on_conflict_clause and supports_multiple_on_conflict_clauses to PostgresUpsertSupport protocol * fix(postgres): remove sync savepoint test import, only async tests apply * fix(postgres): pass scenario name to type_adapter_fixtures provider to avoid cross-scenario table conflicts * fix(postgres): import all testsuite basic fixtures in conftest while overriding type_adapter_fixtures * fix(postgres): revert conftest override, restore import * to avoid breaking other fixtures * fix(postgres): pass scenario name to sync type_adapter_fixtures to avoid cross-scenario table collisions * fix(postgres): remove test bridge files accidentally added in protocol fix * fix(postgres): remove shadowing fixture import from type_adapter bridge file The bridge file imported type_adapter_fixtures directly from the testsuite conftest into the test module namespace. pytest registered that fixture with the test module as its most specific baseid, shadowing the conftest override (feature/basic/conftest.py) that passes the scenario name to the provider. As a result, the sync [postgres_exts] type_adapter tests connected to the default scenario database (5432) instead of postgres_exts (5433), colliding with async [postgres_default] tests on the same type_adapter_tests table and causing 'relation does not exist' failures under xdist concurrency. Remove the import so pytest resolves the conftest override, which correctly passes the scenario name. * test(postgres): add unit tests for data type formatting and parsing - Fix relative imports in parse_type (..expression.types -> ...expression.types) that crashed serial/binary/jsonpath/uuid/net/geom/range/oid/ts/vector parsing with ModuleNotFoundError. - Fix DATETIME mis-parsed as DATE (dispatch order), and CHARACTER/ CHARACTER VARYING/TIMESTAMPTZ/TIMETZ falling back to CustomType due to \b boundary regex. - Add test_data_type_formatting.py covering all PostgreSQL type format and parse branches (data_type_formatting.py coverage 55.67% -> 98%). - Add test_expression_types.py covering synonyms/equality/hash/array equivalence (expression/types.py coverage 76.19% -> 100%). - Cover index fulltext PHRASE mode + multi-column, and upsert capability switches (index.py and upsert.py -> 100%). * chore: add changelog fragment for PR #49
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds PGQ (Property Graph Query) enhancements for the PostgreSQL backend: fulltext search capability, cursor result set pollution fix, and PostgreSQL 19 compatibility gates.
Changes:
FulltextSearchCapabilityto PostgreSQL backend, enabling full-text search operations using tsvector/tsquerydescriptionis properly cleaned after internal queries likeget_server_version()get_server_version()to prevent stalecursor.descriptionfrom affecting subsequent queries on the same connectionpg_backend→postgres_backendandasync_pg_backend→async_postgres_backendin cursor pollution testsType of change
feat: A new featurefix: A bug fixtest: Adding missing tests or correcting existing testsBreaking Change
Impact on End-Users:
No backward-incompatible changes. Fulltext search is a new capability; cursor pollution fix only affects internal cursor management.
Impact on Backend Developers (Custom Implementations):
No backward-incompatible changes.
Related Repositories/Packages
Testing
Checklist
changelog.d/<pr_number>.added.mdAdditional Notes
This is the PostgreSQL backend counterpart of the core PGQ enhancement (PR #111).