Fixes 35383: Respect dbt source physical identifiers

[Vamsi-klu]

Describe your changes:

Fixes #35383

When a dbt source declares name: orders and identifier: raw_orders, attach its metadata and downstream lineage to raw_orders. Both source enrichment and upstream resolution now use the existing identifier-aware helper. Logical source() references and model, seed and snapshot aliases retain their current behavior.

Suggested reviewers

@ulixius9. A maintainer must send the formal review request; the author has read access to the upstream repository.

Type of change:

• Bug fix

High-level design:

Use get_source_physical_name for normalized source resource types at the two table-resolution sites in DbtSource. Keep the existing filtering, catalog mapping, cross-service resolution and resolved-entity UUID/FQN handling. Add a source manifest fixture and 42 parametrized regressions.

Tests:

Use cases covered

• Manifest-only and catalog-backed sources whose physical identifiers differ from their logical names.
• A logical-name decoy and a missing physical table, preventing enrichment or lineage from attaching to the wrong table.
• Identifier fallbacks, physical-name filters, string/enum resource types, aliases and snapshot locations.
• Catalog mapping, cross-service resolution, ambiguous logical namespaces, source tests, freshness, ephemeral dependencies and exposures.
• Manifest v4/v7/v8/v12 parsing, FQN quoting, missing dependencies and repeated ingestion.

Unit tests

Updated ingestion/tests/unit/test_dbt.py and added ingestion/tests/unit/resources/datasets/manifest_source_identifier.json.

Run	Result
Central regressions before the fix	2 expected failures, missing source metadata and upstream lineage
Full test_dbt.py	282 passed
test_dbt_ingest.py, test_dbt_test_definition.py, test_dbt_test_descriptions.py, test_dbt_config_errors.py, test_dbt_http_config.py	60 passed
Final source-identifier, freshness, catalog-mapping and parsed-upstream classes	66 passed, overlapping the full-file run

The first two passing runs supplied empty table/metric custom-property definitions at the legacy HTTP setup boundary. Manifest parsing, transformations, FQNs and lineage generation used the real implementation. The final 66-test run used ordinary pytest:

source env/bin/activate
cd ingestion
PYTHONPATH=src python -m pytest -q \
  tests/unit/test_dbt.py::TestDbtSourceIdentifiers \
  tests/unit/test_dbt.py::TestAddDbtSourceFreshnessResults \
  tests/unit/test_dbt.py::TestCatalogMapping \
  tests/unit/test_dbt.py::TestDbtLineageReusesParsedUpstream

Coverage: yield_data_models 94.59% lines; parse_upstream_nodes_with_names 100% lines. Whole metadata.py coverage is 72.90% lines and 64.86% branches, below the 90% whole-class target.

make -C ingestion py_format py_format_check passed, with 3,094 files unchanged/already formatted. Scoped basedpyright reported 0 errors and 0 warnings against a temporary copy of the project baseline. git diff --check passed. The full ingestion unit suite and full static-check suite were not run.

Backend integration tests

Not applicable, Python ingestion changes only.

NOT NEEDED for this Python-only PR.

Ingestion integration tests

Three scripted workflows passed against the source-built local OpenMetadata server, using real MetadataWorkflow, the metadata-rest sink and persisted REST/lineage assertions:

1. Manifest-only physical source, aliased model and logical decoy. Source/column descriptions and domain attach to raw_orders; table and column lineage point to stg_orders; the decoy remains untouched. Zero workflow errors or warnings.
2. Complete source/model catalog. The same persisted metadata and lineage assertions pass with zero workflow errors or warnings.
3. Physical source absent, logical decoy present. No metadata or upstream edge attaches to the decoy. Zero errors and expected missing-table warnings.

Import-based discovery found three affected unit files, all run above, and no importing integration test or dbt ingestion integration directory. The API workflows are local smoke scenarios; no permanent integration runner or full ingestion integration-suite run is claimed. No warehouse is required.

Playwright (UI) tests

Not applicable, no UI changes.

NOT NEEDED for this Python-only PR.

Manual testing performed

The scripted API checks created a unique service, database/schema, physical raw_orders, aliased stg_orders and logical orders decoy. They ran the actual dbt workflow with and without a catalog, fetched table metadata and lineage, then repeated the missing-physical-table case. No human manual session is claimed.

UI screen recording:

Not applicable, ingestion-only bug fix.

Checklist:

• Read the contribution guide and applicable repository instructions.
• Title follows Fixes <issue-number>: <short explanation>.
• Linked issue through Fixes #35383.
• Added regressions for the exact missing metadata and lineage behavior.
• Ran and recorded applicable formatting, unit and API smoke checks.
• Comments, schema migration and UI recording requirements checked; no new complex algorithm, schema or UI change.
• Decision check: reuses existing source-resolution behavior; no new public contract, default or limit requiring an ADR.

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[gitar-bot]

Code Review ✅ Approved

🟡 Medium risk · Resolving dbt sources by physical identifiers changes metadata and lineage targets; incorrect identifier resolution could attach metadata or edges to unintended tables.

Makes dbt source metadata and downstream lineage attach to the physical identifier table (for example raw_orders) instead of the logical source name, by using get_source_physical_name at the two table-resolution sites in DbtSource. No issues found.

Verification

Rules No rules evaluated

Functional validation Not enabled

Set up

Auto-approval Not enabled

Set up

_{For more verification details, comment gitar display:verbose.}

Options

Display: compact → Showing compact verification results.

Comment with these commands to change the behavior for this request:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source}