Summary
dbt-coves generate sources reads Snowflake metadata for quoted, case-sensitive identifiers correctly (for example, a table created as "my_table"). However, the sources.yml it generates doesn't keep the case and never sets quoting. As a result, the generated source and staging models fail at dbt run.
What works
- Relations come from
adapter.list_relations() (dbt_coves/tasks/generate/base.py). dbt-snowflake builds them with quote_policy set to true for database, schema and identifier.
get_columns_in_relation() therefore runs describe table "DB"."RAW"."my_table" and finds the correct table.
- Column names in the generated staging SQL are quoted with their exact case (
"my_col"::varchar as my_col).
What's broken
dbt_coves/templates/source_props.yml:
sources:
- name: {% if not relation.schema.isupper() and not relation.schema.islower() %}{{ relation.schema }}{% else %}{{ relation.schema.lower() }}{% endif %}
tables:
- name: {% if not relation.name.isupper() and not relation.name.islower() %}{{ relation.name }}{% else %}{{ relation.name.lower() }}{% endif %}
- Case information is lost:
MY_TABLE and "my_table" both render as name: my_table.
- Quoting is never enabled: dbt-snowflake defaults to no quoting, so
source('raw', 'my_table') compiles to db.raw.my_table. Snowflake resolves that to MY_TABLE, which fails with "Object does not exist". Mixed-case tables ("MyTable") and lowercase or mixed-case schemas fail the same way.
Steps to reproduce
- In Snowflake:
create table raw."my_table" ("id" int);
dbt-coves generate sources --schemas raw --select-relations my_table
dbt run -s stg_... → Object 'DB.RAW.MY_TABLE' does not exist or not authorized
Proposed fix
In source_props.yml, emit quoting only when a name isn't all uppercase, so output for standard uppercase identifiers stays the same:
sources:
- name: raw
quoting:
schema: true # when schema != schema.upper()
tables:
- name: my_table
quoting:
identifier: true # when name != name.upper()
Snowflake-only. Other adapters shouldn't be affected.
Related
get_metadata_map_key (dbt_coves/tasks/generate/base.py) lowercases every key part, so columns "MyCol" and MYCOL in the same relation map to the same metadata entry.
Summary
dbt-coves generate sourcesreads Snowflake metadata for quoted, case-sensitive identifiers correctly (for example, a table created as"my_table"). However, thesources.ymlit generates doesn't keep the case and never setsquoting. As a result, the generated source and staging models fail atdbt run.What works
adapter.list_relations()(dbt_coves/tasks/generate/base.py). dbt-snowflake builds them withquote_policyset to true for database, schema and identifier.get_columns_in_relation()therefore runsdescribe table "DB"."RAW"."my_table"and finds the correct table."my_col"::varchar as my_col).What's broken
dbt_coves/templates/source_props.yml:MY_TABLEand"my_table"both render asname: my_table.source('raw', 'my_table')compiles todb.raw.my_table. Snowflake resolves that toMY_TABLE, which fails with "Object does not exist". Mixed-case tables ("MyTable") and lowercase or mixed-case schemas fail the same way.Steps to reproduce
create table raw."my_table" ("id" int);dbt-coves generate sources --schemas raw --select-relations my_tabledbt run -s stg_...→Object 'DB.RAW.MY_TABLE' does not exist or not authorizedProposed fix
In
source_props.yml, emit quoting only when a name isn't all uppercase, so output for standard uppercase identifiers stays the same:Snowflake-only. Other adapters shouldn't be affected.
Related
get_metadata_map_key(dbt_coves/tasks/generate/base.py) lowercases every key part, so columns"MyCol"andMYCOLin the same relation map to the same metadata entry.