Skip to content

Add refreshable and inner-engine materialized view forms to Alembic ops - #1204

Open
joe-clickhouse wants to merge 2 commits into
mainfrom
joe/mv-refresh-inner-engine
Open

joe-clickhouse wants to merge 2 commits into
mainfrom
joe/mv-refresh-inner-engine

Conversation

@joe-clickhouse

@joe-clickhouse joe-clickhouse commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

create_clickhouse_materialized_view only handled the TO form. It now creates refreshable and inner-engine views too, so policy hooks can classify them by field instead of parsing SQL.

  • New keyword-only engine, refresh, empty and comment arguments. The plain TO form renders the same SQL and revision text as before.
  • New ClickHouseRefresh dataclass for the REFRESH clause.
  • New modify_clickhouse_materialized_view_refresh and modify_clickhouse_materialized_view_query ops.
  • All create forms share CreateClickHouseMaterializedViewOp. Hooks that approve by isinstance should check engine and refresh.
  • Adds type annotations to parts of TableEngine so the public type check does not regress.

Known limit: create still collapses %% to % online in the SELECT and comment. The modify ops keep %%.

Checklist

  • Unit and integration tests covering the common scenarios were added
  • A human-readable description of the changes was provided to include in CHANGELOG
  • For significant changes, documentation in https://github.com/ClickHouse/clickhouse-docs was updated with further explanations or tutorials

🤖 Generated with Claude Code


Note

Medium Risk
Adds DDL migration paths that can create destructive inner-engine materialized views and alter refresh/query definitions; mistakes in migrations could drop or reshape stored data on apply.

Overview
Extends Alembic create_clickhouse_materialized_view beyond the TO-table form so migrations can create inner-engine views (to_table=None + engine) and refreshable views via a new ClickHouseRefresh helper that renders the full REFRESH clause (intervals, dependencies, settings, APPEND, etc.). Create also gains empty, comment, and validation that exactly one of to_table or engine is set.

Adds modify_clickhouse_materialized_view_refresh and modify_clickhouse_materialized_view_query Alembic ops (with reversible behavior when prior definitions are supplied), updates autogen rendering/exports (23 public ops), and documents policy-hook guidance: all create shapes share CreateClickHouseMaterializedViewOp, so reviewers should inspect engine and refresh; drops may delete inner-table data for inner-engine views.

TableEngine gets minor type annotations; integration and unit tests cover offline SQL, rendering round-trips, and live refresh/query behavior.

Reviewed by Cursor Bugbot for commit 18f7c3e. Bugbot is set up for automated code reviews on this repo. Configure here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The refresh clause lacks online literal-percent escaping, so a dependency name such as dep%% becomes dep%, potentially referencing the wrong view or causing creation to fail.

1 open issue
What changed in this PR

This PR extends the driver's Alembic support to create refreshable and inner-engine materialized views, while retaining the existing target-table form.

Changes:

  • Adds ClickHouseRefresh, engine selection, comments, and initial-refresh control.
  • Adds operations to modify refresh schedules and SELECT queries.
  • Adds tests, public exports, typing updates, and migration policy guidance.
File Description
tests/​unit_tests/​test_sqlalchemy/​test_alembic.py Tests validation, SQL generation, rendering, and reversal.
tests/​unit_tests/​test_sqlalchemy/​test_alembic_public_operations.py Covers the new public operations.
tests/​integration_tests/​test_sqlalchemy/​test_alembic.py Tests live creation, modification, and data behavior.
docs/​sqlalchemy.mdx Documents new forms, limits, and policy checks.
clickhouse_connect/​cc_sqlalchemy/​ddl/​tableengine.py Adds engine type annotations.
clickhouse_connect/​cc_sqlalchemy/​ddl/​refresh.py Defines and renders refresh specifications.
clickhouse_connect/​cc_sqlalchemy/​alembic/​WORKED_EXAMPLE.md Adds migration examples and safety guidance.
clickhouse_connect/​cc_sqlalchemy/​alembic/​operations.py Implements expanded creation and modification operations.
clickhouse_connect/​cc_sqlalchemy/​alembic/​adapter.py Renders new options and operations into revisions.
clickhouse_connect/​cc_sqlalchemy/​alembic/​__init__.py Exports refresh specifications and modification operations.
clickhouse_connect/​cc_sqlalchemy/​__init__.py Exports ClickHouseRefresh from the SQLAlchemy API.
CHANGELOG.md Records new capabilities and data-loss considerations.

Review effort: Balanced

f"TO {full_table(operation.to_table, operation.to_schema)} AS {operation.select}"
parts = [f"CREATE MATERIALIZED VIEW{exists} {full_table(operation.name, operation.schema)}"]
if operation.refresh is not None:
parts.append(operation.refresh.render())

@security-ai-gate-reviewer security-ai-gate-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto-approved by the AI review gate on d22af93. Details in the approver's comment on this pull request.

@security-ai-gate-reviewer

security-ai-gate-reviewer Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

AI review gate — auto-approved

18f7c3ec8ad8 is approved by security-ai-gate-reviewer[bot], an automated identity, not a human. The approval stands on the required AI review: Cursor Bugbot concluded green on this commit and the AI review gate is SUCCESS (its own comment has the verdict), and the change touches no path that requires a human reviewer. It is bound to this commit: a new push dismisses it and the approver re-evaluates.

History (latest 5 heads, newest first)

  • 18f7c3ec8ad8 approved — green: Cursor Bugbot
  • d22af93bc272 approved — green: Cursor Bugbot

@security-ai-gate-reviewer security-ai-gate-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto-approved by the AI review gate on 18f7c3e. Details in the approver's comment on this pull request.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants