fix(iceberg): one catalog-kind table, so every emitter agrees on location.catalog

[fas89]

Description

binding.location.catalog is a free string, and every emitter classified it by hand. They disagreed:

• catalog: lakekeeper: streamed over REST, while dbt's catalogs.yml wrote a Snowflake-managed table and the AWS IaC created a second, metadata-less Glue table of the same name.
• The sink validator: checked only the literal rest.
• catalog: hive on Snowflake: became a Snowflake-managed table.
• A Confluent binding: published to Glue whatever catalog it named.

A separate defect: the Kafka Connect deriver set both iceberg.catalog.type and iceberg.catalog.catalog-impl for Glue. Apache Iceberg's CatalogUtil refuses that, so every Glue sink failed at startup:

java.lang.IllegalArgumentException: Cannot create catalog iceberg, both type and catalog-impl are set

This PR adds one catalog-kind table in fluid_build/providers/_iceberg_catalog.py, and every emitter now reads from it.

• Shape: borrowed from dbt-adapters' _V2_TO_V1_TYPE alias hook and Airbyte's typed Polaris preset.
• Wire values: come only from Iceberg's CatalogUtil set. No engine or SDK has a vendor type such as lakekeeper.
• Readers: the streaming sinks, dbt catalogs.yml, the Snowflake/AWS/Confluent IaC, the native AWS planner, fluid policy compile, fluid diff, fluid test and fluid validate.

Behaviour changes, all fail-closed:

• Kafka Connect: emits catalog-impl or type, never both (the Debezium twin already did). fluid validate refuses overrides that would add the other key back.
• AWS: an Iceberg expose in another catalog keeps its S3 bucket but gets no Glue database, table, import, Glue IAM grant or Lake Formation resource.
  • LF, column restrictions and row filters on such an expose are refused by catalog name, at validate and at apply.
  • Unknown catalog values are refused at apply.
  • No catalog or glue produces byte-identical output.
• fluid apply (AWS): stops with iceberg_catalog_move_blocked when state holds Glue resources an earlier release created for such a table, and prints the tofu state rm commands. Without the guard, the next plan would destroy them, and destroying a Glue database deletes every table in it. This mirrors the packaging ownership-transition guard.
• fluid validate refuses:
  • unknown kinds;
  • a sink.catalog that disagrees with the expose;
  • a missing uri/warehouse on any REST catalog;
  • Snowflake catalogs that have no integration (hive, jdbc, hadoop, dynamodb);
  • two catalog: snowflake exposes that share one EXTERNAL VOLUME.
• fluid validate gate crashes: a crash in an Iceberg or Confluent gate now fails validation instead of printing only with --verbose.
• Runner preflight: the Kafka Connect runner and the embedded Debezium Server runner run the same checks before they create anything.
• RFC-streaming-extension: corrected. The Apache sink creates namespaces on auto-create since Iceberg 1.6.0 (Kafka-connect: Handle namespace creation for auto table creation apache/iceberg#10186).

See CHANGELOG [Unreleased] for the upgrade note and every fixed behaviour.

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would break existing functionality)
• Documentation update
• Refactor / code cleanup
• New provider or provider enhancement

Contracts that were previously accepted and wrote to the wrong catalog now fail fluid validate on purpose. Each case has a CHANGELOG line.

Documentation

• Docs PR linked:
  • Docs PR: docs(iceberg): document how location.catalog is classified (forge-cli #707) forge_docs#144 (source-aligned-acquisition, snowflake, aws, gcp, validate, generate-iac, apply, diff; new behaviour marked "forge-cli fix(iceberg): one catalog-kind table, so every emitter agrees on location.catalog #707, unreleased")
• No docs needed
• Docs TODO

Checklist

• I have read the Contributing Guide
• My code follows the project's coding standards
• I have added tests that prove my fix/feature works
• All new and existing tests pass (pytest) — see Testing for 4 local-environment failures that fail identically on main
• I have run ruff check and black (24.10.0) with no errors
• New maintained Python files include license headers

Testing

Unit and contract tests. The new tests fail against the old code (negative controls run per slice):

• tests/providers/test_iceberg_catalog_kinds.py: the cross-emitter agreement matrix. Every emitter runs over every kind, including absent and unknown, and each answer is checked against the kind's row.
• test_iac_aws_iceberg_catalog.py and test_iac_catalog_moves.py.
• The sink-validator and runner-preflight suites.
• The dbt, Snowflake, Confluent, planner and policy-compiler suites.
• The schema suite.

Full suite: 20772 passed, 552 skipped on the rebased commit. Four failures also fail identically on main (e0a8b258) with the same venv:

• tests/iac/test_iac_moto_e2e.py: three tests.
• tests/test_coding_agent_live.py: "NOT run in CI"; it drives a local claude CLI.

Lint: ruff check, black --check (24.10.0) and scripts/mypy_strict.py are clean.

Security review of e0a8b258..aa940e36: no findings.

• Every governance input on a non-Glue Iceberg table is refused up front, never dropped without an error.
• The tofu state rm commands are shlex-quoted.
• No connector config is logged.

Tested live

New CI lane: lakekeeper-integration in the emulated-heavy workflow. It runs a real Kafka Connect worker (cp-kafka-connect 7.9.10, Apache Iceberg sink 1.9.2) against a real Lakekeeper v0.13.6. Credentials are vended over STS from an S3-compatible store, so no static storage keys are used.

Test	Asserts
test_derived_lakekeeper_config_streams_into_lakekeeper	Forge's derived config for catalog: Lakekeeper streams 5 records and commits a snapshot with total-records = 5. The sink auto-creates the namespace.
test_both_type_and_catalog_impl_fail_on_a_real_worker	The task FAILS. Its status trace and the worker log both carry the CatalogUtil refusal above.
test_derived_glue_config_gets_past_the_catalog_gate	Forge's derived Glue config (impl only) reaches RUNNING with no catalog refusal.

assert_lane_coverage.py --require fails the job if every test skipped.

Local smoke run:

tests/integration/test_lakekeeper_kafka_connect_live.py::test_both_type_and_catalog_impl_fail_on_a_real_worker PASSED
tests/integration/test_lakekeeper_kafka_connect_live.py::test_derived_glue_config_gets_past_the_catalog_gate PASSED
tests/integration/test_lakekeeper_kafka_connect_live.py::test_derived_lakekeeper_config_streams_into_lakekeeper PASSED
======================== 3 passed in 157.82s (0:02:37) =========================

Negative control: restoring the old "type always, plus impl" emission fails the Glue test and the unit matrix.