Skip to content

chore(logging): log debug sql lines from typed SQLQuery and SQLQueryStatus events - #807

Merged
canbekley merged 4 commits into
mainfrom
chore/typed-sql-debug-log-events
Nov 17, 2026
Merged

canbekley merged 4 commits into
mainfrom
chore/typed-sql-debug-log-events

Conversation

@canbekley

@canbekley canbekley commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The default implementation of SQLConnectionManager and other adapters fire sql debug log events as typed SQLQuery and SQLQueryStatus, while dbt-clickhouse fires generic AdapterEventDebug events. This has led to limitations in how debug log events could be processed (no structured fields, and no query id).
This PR aligns query debug logging with dbt's standard way:

  • sql logs no longer carry the custom dbt_clickhouse adapter: prefix and the trailing ...
  • json logs now carry structured fields
  • dbtRunner receive typed events (making adapter base tests such as UDFsBasic easier to run without clickhouse work-arounds)
  • seed sql logs are shortened to the first 512 characters.
  • timing in logs now show seconds with 3 decimals (ms precision) instead of 2

Example Output

Text log (logs/dbt.log), view model

Before:

09:55:42  dbt_clickhouse adapter: On model.logdemo.my_model: /* {"app": "dbt", ... "node_id": "model.logdemo.my_model"} */
  create or replace view `logdemo`.`my_model`
  as (
    select id, name from `logdemo`.`people`
  )
                    -- end_of_sql
                    ...
09:55:42  dbt_clickhouse adapter: SQL status: OK in 0.00 seconds

After:

09:55:19  On model.logdemo.my_model: /* {"app": "dbt", ... "node_id": "model.logdemo.my_model"} */
  create or replace view `logdemo`.`my_model`
  as (
    select id, name from `logdemo`.`people`
  )
                    -- end_of_sql
09:55:19  SQL status: OK in 0.002 seconds

JSON log (--log-format-file json), view model

Before:

{"code": "E001", "name": "AdapterEventDebug", "level": "debug",
 "data": {"name": "dbt_clickhouse", "args": [], "base_msg": "On model.logdemo.my_model: /* {\"app\": ..."}}
{"code": "E001", "name": "AdapterEventDebug", "level": "debug",
 "data": {"name": "dbt_clickhouse", "args": [], "base_msg": "SQL status: OK in 0.00 seconds"}}

After:

{"code": "E016", "name": "SQLQuery", "level": "debug",
 "data": {"conn_name": "model.logdemo.my_model", "sql": "/* {\"app\": \"dbt\", ..."}}
{"code": "E017", "name": "SQLQueryStatus", "level": "debug",
 "data": {"status": "OK", "elapsed": 0.002111166, "query_id": "a6d1a0ee-759f-47a7-abab-ea484776b540"}}

Checklist


Note

Low Risk
Observability-only change to debug logging; query execution behavior is unchanged aside from consolidating execute/add_query through _run_statement.

Overview
The ClickHouse adapter now emits dbt’s standard SQLQuery and SQLQueryStatus debug events instead of custom adapter debug log lines, so JSON logs, dbtRunner callbacks, and shared adapter tests see structured SQL metadata including the ClickHouse query ID and node context.

execute and add_query share a new _run_statement path: both assign a UUID query_id for the client call, use fire_event for start/status, and measure elapsed time with perf_counter. Optional abridge_sql_log on add_query logs only the first 512 characters while still running the full SQL.

Unit tests in test_connections.py assert the typed events, query ID wiring, and abridged logging behavior.

Reviewed by Cursor Bugbot for commit 9b17086. Bugbot is set up for automated code reviews on this repo. Configure here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new event contract and SQL abridgement lack regression coverage, and the changelog still contains unresolved PR-link placeholders.

2 open issues
What changed in this PR

Aligns ClickHouse SQL debug logging with dbt’s standard typed events, improving structured logs and dbtRunner callback support.

Changes:

  • Shares statement execution through _run_statement, emitting SQLQuery and SQLQueryStatus events with query IDs.
  • Uses high-resolution elapsed timing and honors SQL log abridgement without truncating executed statements.
  • Adds a user-facing changelog entry.
File Description
dbt/​adapters/​clickhouse/​connections.py Unifies execution logging with typed events, query IDs, timing, and optional abridgement.
CHANGELOG.md Documents the structured SQL logging improvement.

Review effort: Balanced

Comment thread CHANGELOG.md Resolved
Comment thread dbt/adapters/clickhouse/connections.py Resolved
@canbekley
canbekley requested review from koletzilla and a balanced review from Copilot November 17, 2026 10:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the structured SQL logging changes.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

@canbekley
canbekley merged commit 0cbec71 into main Nov 17, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants