Skip to content

generate sources: Snowflake quoted lowercase/mixed-case identifiers not preserved in sources.yml #649

Description

@k0x2e

Summary

dbt-coves generate sources reads Snowflake metadata for quoted, case-sensitive identifiers correctly (for example, a table created as "my_table"). However, the sources.yml it generates doesn't keep the case and never sets quoting. As a result, the generated source and staging models fail at dbt run.

What works

  • Relations come from adapter.list_relations() (dbt_coves/tasks/generate/base.py). dbt-snowflake builds them with quote_policy set to true for database, schema and identifier.
  • get_columns_in_relation() therefore runs describe table "DB"."RAW"."my_table" and finds the correct table.
  • Column names in the generated staging SQL are quoted with their exact case ("my_col"::varchar as my_col).

What's broken

dbt_coves/templates/source_props.yml:

sources:
  - name: {% if not relation.schema.isupper() and not relation.schema.islower() %}{{ relation.schema }}{% else %}{{ relation.schema.lower() }}{% endif %}
    tables:
      - name: {% if not relation.name.isupper() and not relation.name.islower() %}{{ relation.name }}{% else %}{{ relation.name.lower() }}{% endif %}
  1. Case information is lost: MY_TABLE and "my_table" both render as name: my_table.
  2. Quoting is never enabled: dbt-snowflake defaults to no quoting, so source('raw', 'my_table') compiles to db.raw.my_table. Snowflake resolves that to MY_TABLE, which fails with "Object does not exist". Mixed-case tables ("MyTable") and lowercase or mixed-case schemas fail the same way.

Steps to reproduce

  1. In Snowflake: create table raw."my_table" ("id" int);
  2. dbt-coves generate sources --schemas raw --select-relations my_table
  3. dbt run -s stg_... → Object 'DB.RAW.MY_TABLE' does not exist or not authorized

Proposed fix

In source_props.yml, emit quoting only when a name isn't all uppercase, so output for standard uppercase identifiers stays the same:

sources:
  - name: raw
    quoting:
      schema: true        # when schema != schema.upper()
    tables:
      - name: my_table
        quoting:
          identifier: true  # when name != name.upper()

Snowflake-only. Other adapters shouldn't be affected.

Related

get_metadata_map_key (dbt_coves/tasks/generate/base.py) lowercases every key part, so columns "MyCol" and MYCOL in the same relation map to the same metadata entry.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtriaged: noHasn't been approved for future implementation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions