Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ jobs:

- name: Install activerecord and testsuite from PyPI
run: |
pip install rhosocial-activerecord==1.0.0.dev20
pip install rhosocial-activerecord-testsuite==1.0.0.dev7
pip install rhosocial-activerecord==1.0.0.dev21
pip install rhosocial-activerecord-testsuite==1.0.0.dev8

- name: Create postgres scenario config
run: |
Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
## [v1.0.0.dev8] - 2026-04-06

### Added

- Added EXPLAIN clause support with typed PostgresExplainResult and dual-layer connection recovery for PostgreSQL backend ([#16](https://github.com/rhosocial/python-activerecord-postgres/issues/16))
- Improved PostgreSQL backend CLI info command with connection status display and added comprehensive introspect usage examples. ([#17](https://github.com/rhosocial/python-activerecord-postgres/issues/17))


## [v1.0.0.dev7] - 2026-03-28

### Added
Expand Down
3 changes: 3 additions & 0 deletions docs/en_US/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,4 +48,7 @@
* **[Performance Issues](troubleshooting/performance.md)**: performance bottleneck analysis
* **[SQL Standard Compliance](troubleshooting/sql_standard.md)**: strict SQL compliance handling

8. **[Scenarios](scenarios/README.md)**
* **[Parallel Worker Processing](scenarios/parallel_workers.md)**: correct usage in multi-process/async concurrent scenarios

> 📖 **Core Library Documentation**: To learn about the complete functionality of the ActiveRecord framework, please refer to [rhosocial-activerecord documentation](https://github.com/rhosocial/python-activerecord/tree/main/docs/en_US).
1 change: 1 addition & 0 deletions docs/en_US/postgres_specific_features/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ PostgreSQL offers many advanced features beyond standard SQL. This section cover
- **[RETURNING Clause](./returning.md)**: Get data back from DML operations
- **[Protocol Support Matrix](./protocol_support.md)**: Comprehensive protocol support and version compatibility
- **[Database Introspection](./introspection.md)**: Query metadata using pg_catalog
- **[EXPLAIN Support](./explain.md)**: Query execution plan analysis and performance diagnostics

## Feature Highlights

Expand Down
294 changes: 294 additions & 0 deletions docs/en_US/postgres_specific_features/explain.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,294 @@
# EXPLAIN Support

## Overview

The PostgreSQL backend provides comprehensive support for the `EXPLAIN` statement, enabling query performance analysis and execution plan inspection. This feature integrates with the ActiveRecord query builder through a unified API while exposing PostgreSQL-specific behaviors.

Key capabilities:
- Basic `EXPLAIN` for execution plan visualization
- `EXPLAIN ANALYZE` for actual runtime statistics (all modern versions)
- Multiple output formats: `TEXT`, `XML`, `JSON`, `YAML`
- PostgreSQL's bracketed option syntax (`EXPLAIN (ANALYZE, FORMAT JSON)`)
- Structured result parsing with `PostgresExplainResult`
- Index usage analysis helpers (`Seq Scan`, `Index Scan`, `Index Only Scan`)
- Full sync and async support

## Basic Usage

### Accessing EXPLAIN via Query Builder

```python
from rhosocial.activerecord.backend.expression.statements import ExplainFormat

# Simple EXPLAIN — returns PostgresExplainResult
result = User.query().explain().all()

# Inspect the generated SQL prefix
print(result.sql) # EXPLAIN SELECT ...

# Print the execution time (seconds)
print(result.duration)
```

### Reading the Result Rows

```python
result = User.query().explain().all()

for row in result.rows:
print(row.line)
```

Each row is a `PostgresExplainPlanLine` with a single field:

| Field | Type | Description |
|---|---|---|
| `line` | `str` | One line of the query plan text (e.g. `"Seq Scan on users (cost=0.00..8.27 rows=1 width=...)"`). |

PostgreSQL returns each plan node as an indented text line in the `"QUERY PLAN"` column. All lines together form the full execution plan tree.

## Output Formats

PostgreSQL supports four output formats, all available on any modern server version:

### TEXT Format (Default)

```python
# Default — no FORMAT option emitted
result = User.query().explain().all()
# Generates: EXPLAIN SELECT ...
```

### JSON Format

```python
result = User.query().explain(format=ExplainFormat.JSON).all()
# Generates: EXPLAIN (FORMAT JSON) SELECT ...
```

### XML Format

```python
result = User.query().explain(format=ExplainFormat.XML).all()
# Generates: EXPLAIN (FORMAT XML) SELECT ...
```

### YAML Format

```python
result = User.query().explain(format=ExplainFormat.YAML).all()
# Generates: EXPLAIN (FORMAT YAML) SELECT ...
```

## EXPLAIN ANALYZE

`EXPLAIN ANALYZE` executes the query and returns actual runtime statistics alongside cost estimates. Use it to compare estimated vs. actual row counts and timing.

```python
# Basic ANALYZE
result = User.query().explain(analyze=True).all()
# Generates: EXPLAIN (ANALYZE) SELECT ...

# ANALYZE with JSON format
result = User.query().explain(analyze=True, format=ExplainFormat.JSON).all()
# Generates: EXPLAIN (ANALYZE, FORMAT JSON) SELECT ...

# ANALYZE with YAML format
result = User.query().explain(analyze=True, format=ExplainFormat.YAML).all()
# Generates: EXPLAIN (ANALYZE, FORMAT YAML) SELECT ...
```

**Important Notes:**
- `EXPLAIN ANALYZE` **executes the query** against the database. Avoid using it on write statements in production without understanding the side effects.
- Supported on all modern PostgreSQL versions (no minimum version constraint within the supported range).

## SQL Syntax Generated

PostgreSQL uses a parenthesized option syntax, distinct from MySQL's flat syntax:

```
EXPLAIN [ ( option [, ...] ) ] statement
```

Options are comma-separated inside parentheses:

```python
# No options
# → EXPLAIN SELECT ...

# analyze only
# → EXPLAIN (ANALYZE) SELECT ...

# format only
# → EXPLAIN (FORMAT JSON) SELECT ...

# both analyze and format
# → EXPLAIN (ANALYZE, FORMAT JSON) SELECT ...
```

The `ANALYZE` option always appears before `FORMAT` in the option list.

## Index Usage Analysis

`PostgresExplainResult` provides helpers for a quick index usage assessment based on text matching across all plan lines:

```python
result = User.query().where(User.c.email == "alice@example.com").explain().all()

usage = result.analyze_index_usage()
print(usage) # "full_scan" | "index_with_lookup" | "covering_index" | "unknown"

# Convenience properties
if result.is_full_scan:
print("Warning: sequential scan detected")

if result.is_covering_index:
print("Optimal: index only scan in use")

if result.is_index_used:
print("An index is being used")
```

### Index Usage Rules

The analysis concatenates all plan lines and performs case-insensitive text matching:

| Matched keyword (case-insensitive) | `analyze_index_usage()` return |
|---|---|
| `"INDEX ONLY SCAN"` (checked first) | `"covering_index"` |
| `"INDEX SCAN"`, `"BITMAP INDEX SCAN"`, or `"BITMAP HEAP SCAN"` | `"index_with_lookup"` |
| `"SEQ SCAN"` | `"full_scan"` |
| None of the above | `"unknown"` |

**Note:** `"INDEX ONLY SCAN"` is checked before `"INDEX SCAN"` to avoid misclassification.

## Async API

The async backend provides the same interface. Use `await` for the terminal method:

```python
async def analyze_queries():
# Simple async EXPLAIN
result = await User.query().explain().all_async()

# ANALYZE with JSON format
result = await User.query().explain(
analyze=True,
format=ExplainFormat.JSON
).all_async()

for row in result.rows:
print(row.line)
```

## Checking Format Support at Runtime

Use `dialect` methods to guard option availability:

```python
dialect = backend.dialect

# Check ANALYZE support (always True for PostgreSQL)
if dialect.supports_explain_analyze():
result = User.query().explain(analyze=True).all()

# Check a specific format (all four formats always supported)
if dialect.supports_explain_format("JSON"):
result = User.query().explain(format=ExplainFormat.JSON).all()
```

## Combining EXPLAIN with Complex Queries

EXPLAIN works with the full query builder chain:

```python
# JOIN query
result = (
User.query()
.join(Order, User.c.id == Order.c.user_id)
.where(User.c.status == "active")
.explain(format=ExplainFormat.JSON)
.all()
)

# GROUP BY / aggregate
result = (
Order.query()
.group_by(Order.c.user_id)
.explain(analyze=True)
.count(Order.c.id)
)

# Subquery
result = (
User.query()
.where(User.c.id.in_(Order.query().select(Order.c.user_id)))
.explain()
.all()
)
```

## Comparison with MySQL EXPLAIN

Both backends share the same query builder API but differ in SQL syntax and result structure:

| Feature | PostgreSQL | MySQL |
|---|---|---|
| SQL syntax | `EXPLAIN (ANALYZE, FORMAT JSON)` | `EXPLAIN ANALYZE FORMAT=JSON` |
| Option delimiter | Comma-separated, parenthesized | Space-separated, no parentheses |
| Supported formats | TEXT, XML, JSON, YAML | TEXT, JSON (5.6.5+), TREE (8.0.16+) |
| ANALYZE support | All modern versions | 8.0.18+ |
| Result row type | `PostgresExplainPlanLine` (1 field) | `MySQLExplainRow` (12 fields) |
| Covering index indicator | `"Index Only Scan"` in plan text | `"Using index"` in `extra` column |

## Version Compatibility

| Feature | Minimum PostgreSQL Version |
|---|---|
| Basic `EXPLAIN` | All supported versions |
| `EXPLAIN FORMAT=TEXT/XML/JSON/YAML` | All supported versions |
| `EXPLAIN ANALYZE` | All supported versions |
| `EXPLAIN (ANALYZE, FORMAT JSON)` | All supported versions |

The PostgreSQL backend targets PostgreSQL 9.6 and later; all EXPLAIN features described here are available on 9.6+.

## API Reference

### `ExplainOptions` Fields Used by PostgreSQL

| Field | Type | Default | Description |
|---|---|---|---|
| `analyze` | `bool` | `False` | Emit `ANALYZE` option |
| `format` | `Optional[ExplainFormat]` | `None` | Output format (`TEXT`/`XML`/`JSON`/`YAML`) |

Other `ExplainOptions` fields (such as `verbose`, `buffers`, `timing`, `settings`) are accepted but not yet emitted by the current PostgreSQL dialect implementation.

### `PostgresExplainResult` Methods

| Method / Property | Description |
|---|---|
| `rows` | `List[PostgresExplainPlanLine]` — parsed plan lines |
| `sql` | The full SQL string that was explained |
| `duration` | Query execution time in seconds |
| `raw_rows` | Raw rows as returned by psycopg (list of dicts with `"QUERY PLAN"` key) |
| `analyze_index_usage()` | Returns `"full_scan"`, `"index_with_lookup"`, `"covering_index"`, or `"unknown"` |
| `is_full_scan` | `True` if `analyze_index_usage() == "full_scan"` |
| `is_index_used` | `True` if any index is used |
| `is_covering_index` | `True` if an index-only scan is used |

### `PostgresDialect` Methods

| Method | Return | Description |
|---|---|---|
| `supports_explain_analyze()` | `bool` | Always `True` for PostgreSQL |
| `supports_explain_format(fmt)` | `bool` | `True` for `TEXT`, `XML`, `JSON`, `YAML` |
| `format_explain_statement(expr)` | `str` | Builds `EXPLAIN [(ANALYZE, FORMAT X)]` prefix |

## Best Practices

- **Use JSON format for programmatic analysis.** `FORMAT JSON` produces a structured plan tree that is easier to parse than plain text and provides the most detail.
- **Reserve `EXPLAIN ANALYZE` for development.** The statement executes the query, so use it only when you need actual row counts and timing figures.
- **Leverage all four formats for different audiences.** Use `TEXT` for human inspection, `JSON`/`YAML` for tooling, and `XML` for XML-based reporting pipelines.
- **Combine with index analysis.** Use `result.is_full_scan` as a quick smoke test in integration tests to detect inadvertent sequential scans.

💡 *AI Prompt: Explain how to use EXPLAIN ANALYZE with JSON format on PostgreSQL to diagnose slow queries in rhosocial-activerecord.*
27 changes: 27 additions & 0 deletions docs/en_US/scenarios/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Scenarios

This chapter provides complete application examples of the PostgreSQL backend in real-world business scenarios, helping developers understand how to correctly use rhosocial-activerecord in actual projects.

## Scenario List

### [Parallel Worker Processing](parallel_workers.md)

Demonstrates correct usage of PostgreSQL backend in multi-process/async concurrent scenarios:

- **Multi-process Correct Usage**: `configure()` must be called within child processes
- **PostgreSQL Async Advantage**: psycopg native async support, true network I/O concurrency
- **Deadlock Handling**: PostgreSQL automatically detects deadlocks, recommended retry mechanism for production
- **Multi-threading Pitfalls**: psycopg connections also do not support multi-threaded concurrency

> 📖 **Companion Code**: Complete runnable experiment code is located in the `docs/examples/chapter_08_scenarios/parallel_workers/` directory.

## Relationship with Core Library Scenarios

This chapter is a PostgreSQL-specific supplement to the [Core Library Scenarios Documentation](https://github.com/Rhosocial/python-activerecord/tree/main/docs/en_US/scenarios), focusing on:

- PostgreSQL-specific concurrent behavior (MVCC, row-level locking, deadlock detection)
- psycopg async driver characteristics (single-connection model, true async I/O)
- RETURNING clause advantages (INSERT/UPDATE/DELETE returning data)
- Key differences compared to MySQL and SQLite

The general ActiveRecord usage patterns introduced in the core library scenarios (such as relationships, query building) also apply to the PostgreSQL backend.
Loading
Loading