Repository navigation
feat: PGQ enhancement - fulltext search, cursor pollution fix, and PG 19 compatibility - #48
Merged
Merged
Conversation
…dev15 PyPI rhosocial-activerecord-testsuite==1.0.0.dev15 does not include the composite_pk module added in testsuite v1.0.0.dev16. Install from the release branch directly to resolve ModuleNotFoundError.
…for PostgreSQL backend (#46) * feat(types): implement PostgreSQL DataType subclasses and type support mixin - Add ~45 PostgreSQL-specific DataType subclasses in expression/types.py - Add PostgresTypeFormatSupportMixin with TypeFormattingSupport + TypeParsingSupport - Integrate type support into PostgresDialect - Update exports in __init__.py, expression/__init__.py, mixins/__init__.py, mixins/types/__init__.py * refactor(types): add __eq__/__hash__, remove synonyms, add CharacterVaryingType - Replace _type_params() with __eq__/__hash__ in PostgresBitType, PostgresVarBitType, PostgresVectorType - Remove synonyms() from PostgresSmallSerialType, PostgresSerialType, PostgresBigSerialType - Add PostgresCharacterVaryingType(VarCharType) for CHARACTER VARYING(n) - Register new type in data_type_formatting mixin - Clean up unused imports (ClassVar, Set, Tuple) * feat(schema): add parsed_data_type to PostgreSQL _parse_columns Call DataType.parse_data_type_str(dialect, data_type) when constructing ColumnInfo so that introspection results carry structured DataType instances for schema comparison. * feat(types): add synonyms to PostgreSQL types - PostgresSerialType ↔ IntegerType/SerialType - PostgresSmallSerialType ↔ SmallIntType/SmallSerialType - PostgresBigSerialType ↔ BigIntType/BigSerialType - PostgresCharacterVaryingType ↔ VarCharType * feat(types): add synonyms to PostgresByteaType PostgresByteaType ↔ BlobType for cross-backend schema comparison. * refactor(types): inherit DDLTypeMixin in Postgres type formatting * refactor(types): update to DDLTypeSupport protocol * refactor(types): use @DDLTypeMixin.handles, remove format_data_type override - Add @DDLTypeMixin.handles decorators to all 56 formatter methods - Remove custom format_data_type() dispatch and supports_data_types() list - Move Postgres type imports from TYPE_CHECKING to runtime * fix(types): rewrite Postgres core type formatters, fix missing PostgresTypeFormatSupportMixin base - Replace _default_sql() calls with explicit PostgreSQL SQL strings for all 18 core type handlers (SmallIntType→SMALLINT, etc.) - Add PostgresTypeFormatSupportMixin as base of PostgresDialect (was imported but never used as base, causing format_data_type to resolve to protocol stub and return None) - Create test_core_type_mappings.py with 15 tests for core type→SQL rendering via PostgreSQL dialect * feat(postgres): implement array type support - Add PostgresArrayType parameterised container type (element_type, dimensions) - Add format_data_type_array handler that recursively formats element types - Update parse_type to detect ARRAY keyword and [] suffix before standard dispatch - Add 25 tests covering core type mappings and array rendering/parsing * refactor(postgres): use core ArrayType, keep PostgresArrayType as alias - PostgresArrayType is now just 'PostgresArrayType = ArrayType' - Update handler registration to use core ArrayType - Update parse_type to return ArrayType directly - Add test for is_element_type_equivalent * feat(postgres): add element type validation and PostgresArrayType subclass - format_data_type_array validates element_type is in _type_formatters - PostgresArrayType is now a proper subclass with is_equivalent ignoring dimensions (PG normalises multi-dim arrays to 1-D internally) - parse_type always creates PostgresArrayType with dimensions=1 - Update tests for new PostgresArrayType subclass and PG array semantics * feat(schema): add PostgresSchemaDiffer Column order has no semantic meaning in PostgreSQL; array dimension equivalence is handled by PostgresArrayType.is_equivalent(). * feat(schema): add schema-diff examples and refactor for PostgreSQL backend - Add 6 schema-diff example scripts: table_changes, column_order_irrelevant, postgres10_identity, array_type_diff, index_type_diff, serialization_roundtrip - All DDL uses expression objects, call introspect_and_adapt() after connect - Refactor format_column_definition to use col_def.data_type.to_sql() and add identity dialect_options support - Update existing DDL examples to use expression API (table param, QueryExpression for verification) * refactor(types): remove _default_sql(), make format_data_type return Tuple[str, tuple] - Delete all _default_sql() methods from Postgres expression types - Change all PostgresTypeFormatSupportMixin formatters to return Tuple[str, tuple] - Inline _default_sql() logic directly into each format_data_type_* method - Unpack format_data_type() call in array formatter * style(types): add complete type annotations to all format_data_type_* methods * feat(examples): strengthen serialization roundtrip with table data assertions * fix(tests): replace string data_type with DataType instances across 22 test files ColumnDefinition now validates that data_type must be a DataType instance. Replace all string data_type values (SERIAL, TEXT, INTEGER, etc.) with proper instance references. Also add PostgresCitextType, PostgresCubeType, PostgresLtreeType, PostgresRasterType missing extension DataType subclasses. Update security injection tests to verify TypeError at construction time instead of ValueError at format time. * docs: add changelog fragment for PR #46 * fix(tests): replace positional string data_type with DataType instances in 6 more files * fix(types): register PostgresCitextType, PostgresCubeType, PostgresLtreeType, PostgresRasterType formatters * fix: restore PostgresCidrType import accidentally removed in type formatter registration
* feat(cli): add named-migration subcommand (PostgreSQL adapter) Registers the 'named-migration' CLI subcommand for the PostgreSQL backend. Delegates to the shared implementation in rhosocial.activerecord.backend.migration.cli. - cli/named_migration.py: parser + handler adapter - cli/__init__.py: register in COMMAND_NAMES and register_commands() * feat(migration): add --async support, example migrations, and demo scripts for PostgreSQL - Extend named_migration.py adapter with --async support (AsyncPostgresBackend) - Add extensive CLI epilog with 12 scenario examples and demo script references - Create examples/named_migrations/ with migrations.py, expressions.py, run_basic.py, run_chain.py - Add demo_basic.sh, demo_chain.sh, demo_all.sh end-to-end shell scripts - expressions.py uses PostgresSerialType and PostgresCharacterVaryingType * chore: add changelog fragment for PR #47 * chore: add parameterized migration, async, incompatible, and single-transaction demo scripts for PostgreSQL * fix(postgres): use positional arg instead of keyword 'size' for PostgresCharacterVaryingType * ci: trigger rebuild
…ite_pk mapped provider, partition explain test - mixins/ddl/index: implement supports_fulltext_search() (tsvector/tsquery via GIN), supports_fulltext_boolean_mode() (tsquery boolean operators), and refine docstrings distinguishing DDL (CREATE FULLTEXT INDEX, False) from query capability (True). - pyproject: register benchmark* pytest markers. - tests/providers/composite_pk: add MappedOrderItem/Async setup helpers. - Add tests/.../feature/backend/test_partition_explain.py covering partition EXPLAIN plan introspection.
…ability The TestFulltextNotSupported class previously assumed PostgreSQL opposes ALL fulltext features and that the format_* helpers raise UnsupportedFeatureError. That contract is stale: PostgresIndexMixin overrides supports_fulltext_search/supports_fulltext_boolean_mode to return True (queried via tsvector @@ tsquery), and format_fulltext_match / format_create_fulltext_index_statement / format_drop_fulltext_index_statement return working SQL (plainto_tsquery / to_tsquery / CREATE INDEX USING GIN / DROP INDEX). - Rename to TestFulltextDdlNotSupportedSearchSupported to express the decoupled DDL-vs-query contract introduced by core PR #110. - Keep the still-unsupported DDL flags (supports_fulltext_index, supports_fulltext_parser, supports_fulltext_query_expansion) as False. - Add positive coverage for supports_fulltext_search, supports_fulltext_boolean_mode. - Add tsvector/tsquery SQL-shape assertions for natural-language mode, boolean mode, and multi-column variants. - Add CREATE INDEX ... USING GIN / DROP INDEX shape assertions (with and without IF [NOT] EXISTS), constructing CreateFulltextIndexExpression / DropFulltextIndexExpression with a dialect instance as required by BaseExpression.
get_server_version() called fetchone() on SELECT version() but did not consume any remaining results before closing the cursor. Some database drivers can leave stale result-set metadata on the connection, causing the next cursor opened on the same connection to read a polluted cursor.description. This mirrors the same fix applied to the core SQLite backend and the MySQL backend. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ported on PG 19
PostgreSQL 19 does not support:
- Variable-length (quantified) path patterns (+/*/{n,m})
- Comma-separated multiple patterns in a single MATCH clause
Add capability overrides returning False, document known limitations,
and add tests for the new gating methods.
…ackend' in cursor pollution tests The cursor pollution tests referenced 'pg_backend' and 'async_pg_backend' which do not exist as pytest fixtures. The correct names are 'postgres_backend' and 'async_postgres_backend', matching the actual fixture definitions used by the rest of the test suite.
…n bridges - Update bridge files to match testsuite rename: test_sync_queries -> test_queries - Add 4 new async relation bridge files (test_base_async, test_cache_async, test_interfaces_async, test_nested_relationship_access_async) - Include pgq/ example directory
… relation benchmark - derived_field/conftest.py + test_derived_field_testsuite.py - interface/test_threadsafe_dict.py (sync + async combined) - benchmark/relation/conftest.py + test_cache_comparison.py
execute() is atomic per DB-API 2.0 — the extra fetchall() after fetchone() is unnecessary and adds a round-trip for no benefit. Follow-up to commit ab3be40 which introduced these defensive calls.
…ssion-based DDL - Delete composite_pk/derived_field bridge files, schema SQL, and providers - Create tests/providers/fixtures/ with TABLE_EXPRESSIONS (basic, query, mixins, events) - Replace _load_postgres_schema with CreateTableExpression/DropTableExpression DDL - Update all query/relation bridge imports to new subdirectory paths - Fix _setup_model_async -> _setup_async_model typo in query provider
…est, fix composite_pk setup - Move 70 bridge files from flat paths to testsuite-matching subdirectories - Create 6 missing bridge files (composite_pk crud/query/cte/set_operations, derived_field_basic, example_basic_fixtures) - Add missing basic/worker, query/worker, interface conftest bridge files - Rename test_worker_transaction_isolation -> test_transaction_isolation - Fix setup_order_item_model to use basic TABLE_EXPRESSIONS for composite PK order_items table
…nftest - Use PROVIDER_KEY_ASYNC instead of undefined PROVIDER_KEY - Fix method call names: setup_async_user_model -> setup_user_model (await) - Fix cleanup_after_test_async -> cleanup_after_test (await) - Align method calls with testsuite IRelationAsyncProvider interface
…hema call from relation conftest RelationAsyncProvider._setup_user_post_comment_async() already handles all three tables in a single call.
…ss-test contamination Add missing tables to PostgreSQL provider cleanup lists so all tables created during a test are reliably removed afterwards: - basic provider (sync + async): bulk_users - query provider (sync + async): profiles These were created by setup methods (setup_bulk_user_model, setup_profile_fixtures) but absent from the cleanup drop list, leading to orphaned tables across sequential test runs and cross-scenario contamination.
…p, fix bare excepts - RelationSyncProvider/RelationAsyncProvider cleanup_after_test now drops all 12 created tables (employees, departments, authors, books, chapters, profiles, users, posts, comments, boundary_*) with CASCADE before disconnect. - BasicConnectionProvider/QueryConnectionProvider cleanup now drops test_users table. - CrudBenchmarkProvider cleanup now drops benchmark_users table. - All 8 bare 'except:' -> 'except Exception:' across basic.py, query.py, mixins.py, events.py providers.
- Generate 61 _async.py bridge files for sync/async pairing compliance - Rename test_eager_loading_scenarios_sync.py -> test_eager_loading_scenarios.py - Rename test_relations_with_sync.py -> test_relations_with.py - Split test_derived_field_basic.py into sync-only + test_derived_field_basic_async.py - Add missing bridge files for basic/fields/ subcategories - Fix testsuite module path in relation/derived/test_derived_field.py
…ields/ Same fix as python-activerecord-mysql: identical leaf filename test_derived_field.py from two different testsuite modules collide under pytest's import model. Drop the basic/fields duplicates to keep only the relation/derived/ bridges, matching SQLite.
Same fix as python-activerecord-mysql: the four __init__.py markers
under tests/benchmark/{,backend,activerecord_bulk,relation} were
removed in commit f146d54, breaking pytest collection on tests that
use relative imports (e.g. 'from .workloads import ...' in
tests/benchmark/backend/test_execute_crud_*.py).
Restore them so pytest treats those directories as packages and the
relative imports resolve.
…-mismatch
- Move cursor.description assertions before cursor.close() in all
8 cursor pollution tests (both sync and async) to fix assert None
- Replace string type names ('INTEGER', 'VARCHAR', 'NUMERIC') with
proper DataType instances (IntegerType, VarCharType, DecimalType)
in test_recursive_cte_graph_traversal.py to fix TypeError
- Rename test_transaction_isolation.py → test_worker_transaction_isolation.py
to avoid import-file-mismatch collision with backend/transaction/
…ssion for depth+1 in CTE tests PostgreSQL does not support prefix function-call syntax for arithmetic operators (+(a, b)). Use the overloaded + operator (creates BinaryArithmeticExpression) which generates correct infix a + b SQL.
…ssertion bugs in CTE tests - Add GroupByHavingClause to fix column 'fund_trace.depth' must appear in GROUP BY - Fix depth 3 expected sum to 800000 (500000 Diana + 300000 Eve) - Fix social network assertions: Charlie at depth 2 (not 1), no Diana in results - These bugs were pre-existing: tests were never runnable before DataType fix
…condition in cleanup_after_test
Instead of hardcoding tables_to_drop list in cleanup_after_test (which
drops ALL tables including those used by concurrent xdist workers),
track only tables that were actually created by _setup_{model,multiple}
calls in each provider instance. Since each test fixture creates a fresh
provider, this naturally isolates table cleanup to just the tables the
current test created, eliminating the race where one test's teardown
drops users/type_adapter_tests/pydantic_validated_models tables that
another concurrent worker is still using.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #48 +/- ##
==========================================
+ Coverage 71.66% 73.18% +1.51%
==========================================
Files 274 277 +3
Lines 10687 11333 +646
Branches 1743 1837 +94
==========================================
+ Hits 7659 8294 +635
- Misses 2438 2439 +1
- Partials 590 600 +10
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
PostgreSQL supports only a single ON CONFLICT clause per INSERT. Explicitly declare supports_on_conflict_clause=True and supports_multiple_on_conflict_clauses=False to enable the generic gate in DMLMixin.format_on_conflict_clauses. Add targeted tests for the capability switches and single-clause rendering (incl. EXCLUDED).
Confirm PostgreSQL inherits the correct optimistic True/True defaults from the core TableMixin and emits standard CASCADE/RESTRICT tokens. No code change required on the PostgreSQL backend.
…ultiple_on_conflict_clauses to PostgresUpsertSupport protocol
…o avoid cross-scenario table conflicts
…overriding type_adapter_fixtures
…eaking other fixtures
…oid cross-scenario table collisions
…ge file The bridge file imported type_adapter_fixtures directly from the testsuite conftest into the test module namespace. pytest registered that fixture with the test module as its most specific baseid, shadowing the conftest override (feature/basic/conftest.py) that passes the scenario name to the provider. As a result, the sync [postgres_exts] type_adapter tests connected to the default scenario database (5432) instead of postgres_exts (5433), colliding with async [postgres_default] tests on the same type_adapter_tests table and causing 'relation does not exist' failures under xdist concurrency. Remove the import so pytest resolves the conftest override, which correctly passes the scenario name.
- Fix relative imports in parse_type (..expression.types -> ...expression.types) that crashed serial/binary/jsonpath/uuid/net/geom/range/oid/ts/vector parsing with ModuleNotFoundError. - Fix DATETIME mis-parsed as DATE (dispatch order), and CHARACTER/ CHARACTER VARYING/TIMESTAMPTZ/TIMETZ falling back to CustomType due to \b boundary regex. - Add test_data_type_formatting.py covering all PostgreSQL type format and parse branches (data_type_formatting.py coverage 55.67% -> 98%). - Add test_expression_types.py covering synonyms/equality/hash/array equivalence (expression/types.py coverage 76.19% -> 100%). - Cover index fulltext PHRASE mode + multi-column, and upsert capability switches (index.py and upsert.py -> 100%).
|
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds PGQ (Property Graph Query) enhancements for the PostgreSQL backend: fulltext search capability, cursor result set pollution fix, and PostgreSQL 19 compatibility gates.
Changes:
FulltextSearchCapabilityto PostgreSQL backend, enabling full-text search operations using tsvector/tsquerydescriptionis properly cleaned after internal queries likeget_server_version()get_server_version()to prevent stalecursor.descriptionfrom affecting subsequent queries on the same connectionpg_backend→postgres_backendandasync_pg_backend→async_postgres_backendin cursor pollution testsType of change
feat: A new featurefix: A bug fixtest: Adding missing tests or correcting existing testsBreaking Change
Impact on End-Users:
No backward-incompatible changes. Fulltext search is a new capability; cursor pollution fix only affects internal cursor management.
Impact on Backend Developers (Custom Implementations):
No backward-incompatible changes.
Related Repositories/Packages
Testing
Checklist
changelog.d/<pr_number>.added.mdAdditional Notes
This is the PostgreSQL backend counterpart of the core PGQ enhancement (PR #111).