Skip to content

fix(snapshot): drop a leftover __snapshot_upsert table before recreating it - #806

Open
gedaliakliger wants to merge 2 commits into
ClickHouse:mainfrom
gedaliakliger:fix/600-snapshot-upsert-leftover
Open

gedaliakliger wants to merge 2 commits into
ClickHouse:mainfrom
gedaliakliger:fix/600-snapshot-upsert-leftover

Conversation

@gedaliakliger

@gedaliakliger gedaliakliger commented Nov 17, 2026 •

Copy link
Copy Markdown

Summary

clickhouse__snapshot_merge_sql creates <snapshot>__snapshot_upsert with create table if not exists and drops it only at the end of a successful run. A run that died after filling it left the table behind, and the next run inserted into it again, so every snapshot row was duplicated.

The upsert table is now dropped before it is created, the same way __dbt_tmp is. The new test leaves a filled upsert table behind and checks that the next snapshot run keeps 10 rows; without the fix it has 20.

Ran here on ClickHouse 26.3, single node, over HTTP and native: the snapshot tests in tests/integration/adapter/basic, plus ruff and mypy on the changed test file. The cluster and Cloud paths, newer ClickHouse versions and the full make lint are left to CI.

Fixes #600

Checklist

  • Unit and integration tests covering the common scenarios were added
  • A changelog entry was added following Updating the Changelog (user-facing, concise, PR link)

Note

Low Risk
Small, targeted snapshot merge change aligned with existing tmp-table cleanup, with a regression test; no auth or broad behavioral shifts.

Overview
Fixes duplicate snapshot rows when a prior run failed after populating <snapshot>__snapshot_upsert. clickhouse__snapshot_merge_sql now calls drop_relation_if_exists on that upsert relation before CREATE TABLE IF NOT EXISTS, matching staging cleanup for __dbt_tmp.

Adds a regression integration test that simulates a leftover filled upsert table and asserts row and dbt_scd_id counts stay correct, plus a changelog note under Bugs.

Reviewed by Cursor Bugbot for commit 25edcea. Bugbot is set up for automated code reviews on this repo. Configure here.

…ing it

A snapshot run that died after filling <snapshot>__snapshot_upsert left the table behind, and the next run reused it through create table if not exists, so its rows were inserted twice.

Fixes ClickHouse#600
@cla-assistant

cla-assistant Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Duplicates in case if snapshot didn't finished successfully

1 participant