Skip to content

docs(iceberg): correct the catalog-move, GCP and sink-config rules (forge-cli #709) - #146

Merged
fas89 merged 1 commit into
mainfrom
docs/iceberg-catalog-followups-2
Oct 7, 2026
Merged

fas89 merged 1 commit into
mainfrom
docs/iceberg-catalog-followups-2

Conversation

@fas89

@fas89 fas89 commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

Follow-up to #145. Corrects the pages #145 wrote for forge-cli #709, whose branch fix/iceberg-catalog-followups now ends at 6e6eb000 ("fix(iceberg): four gaps the defect scan found in this PR's own checks"). Same marker style: (forge-cli #709, unreleased). No page or heading moved or renamed. cli/apply.html#iceberg-catalog-move-guard, which the CLI routes to, is unchanged.

What changed

1. Catalog-move guard (cli/apply.md, advanced/source-aligned-acquisition.md, providers/aws.md, providers/snowflake.md, advanced/production-troubleshooting.md)

  • AWS: a Glue database is flagged when the moved expose's own Glue table is in state, or when the expose names a location.database and no location.table. An earlier release created only the database for that expose, so the database is flagged when state holds it and the module no longer declares it. docs(iceberg): document the forge-cli catalog follow-up fixes (fix/iceberg-catalog-followups) #145 said "only when its own Glue table is in state". Code: catalog_moves._detect, which skips an expose only when it has evidence resources (own) and none of them is in state.
  • Snowflake: the guard finds the EXTERNAL VOLUME by the name derived from the contract id (_snowflake_volume_before), so it also stops an upgrade whose edit changed location.warehouse to the catalog's warehouse name or removed location.iam_role_arn. docs(iceberg): document the forge-cli catalog follow-up fixes (fix/iceberg-catalog-followups) #145 said the volume is flagged "when a moved expose would have derived it".

2. GCP "must name the catalog" (providers/gcp.md, cli/validate.md, advanced/source-aligned-acquisition.md)

  • The gate reads the catalog that reaches the worker, whether a type or a catalog-impl class selects it (_reaching_catalog, _IMPL_CATALOGS). It refuses every catalog but BigLake, so Glue is refused too, from sink.catalog: glue or from a hand-written GlueCatalog catalog-impl. Quoted the real error for sink.catalog: glue.

3. New rule: a hand-written selector must match the expose (cli/validate.md, advanced/source-aligned-acquisition.md, plus one sentence in providers/gcp.md)

  • A sink_connector_config, iceberg_catalog_overrides or embedded Debezium server.sink.config whose type/catalog-impl selects another catalog than the expose's is an error on every platform. A REST endpoint that fronts Glue (Glue's Iceberg REST endpoint) is catalog: rest. Quoted the real errors: AWS Glue default with iceberg.catalog.type: rest, and GCP catalog: bigquery with iceberg.catalog.type: rest.

4. Guidance that contradicted these

  • "A hand-written iceberg.catalog.type: rest for a bigquery or nessie expose draws neither [warning]": still true for the warnings, but that config is now an error. Both pages now say so.
  • The Nessie and BigQuery warnings follow a catalog-impl class too (NessieCatalog, BigQueryMetastoreCatalog), not only type.
  • production-troubleshooting.md: for a table-less expose, the guard flags a Glue database only, not a database and a table.

Evidence

Built from the forge-cli worktree's .venv/bin/fluid (FLUID Forge CLI v0.19.1.dev6, editable install at 6e6eb000). Every new quoted block is the exact fluid validate output for that contract. The guard claims were run through detect_catalog_moves / guard_catalog_moves with the real AwsIacPlugin / SnowflakeIacPlugin:

  • AWS, catalog: lakekeeper, database: streaming, no table, state = bucket + database: ('aws_glue_catalog_database.bronze_orders_stream_streaming',) flagged.
  • Snowflake, catalog: lakekeeper, warehouse: analytics (a name), no iam_role_arn, state = the volume: today's emit derives no volume, and the guard still flags snowflake_external_volume.sales_orders_lake_vol_FLUID_SALES_ORDERS_LAKE_VOL.
  • catalog: rest on AWS with a hand-written type: rest validates clean. Its module holds only aws_s3_bucket, and governance.lakeFormation on it is refused, which is what the "a table in another catalog" sentence says.
  • iceberg_sink_preflight returns the same errors for the AWS, GCP and Debezium contracts.

Checks

  • npm ci && npm run docs:build: exit 0 (236 pages; the only warning is vite's existing chunk-size notice)
  • node scripts/check-dist-links.mjs: exit 0, canaries ok, 137,794 absolute references across 242 built pages all clean
  • Base-prefix check, extracted verbatim from .github/workflows/link-check.yml: exit 0 ("Watching 48 top-level routes. All internal absolute body links carry the /forge_docs/ base.")
  • New in-page anchors (#how-a-value-is-read, #on-aws-a-table-in-another-catalog, validate.html#iceberg-catalog-checks) exist in the built HTML
  • No overlap with open PRs (all open PRs are dependabot package*.json bumps)

…orge-cli #709)

Follow-up to #145 for forge-cli #709 at 6e6eb000 ("four gaps the defect
scan found in this PR's own checks"). Quoted output regenerated from
6e6eb000.

- Catalog-move guard: a Glue database is flagged when the moved expose's
  own Glue table is in state, or when the expose names no table (its
  database is all an earlier release created for it). The Snowflake
  volume is found by its contract-derived name, so an upgrade that also
  changed location.warehouse to a catalog name or dropped iam_role_arn
  is still stopped. (apply, source-aligned-acquisition, aws, snowflake,
  production-troubleshooting)
- GCP: an expose with no location.catalog is refused when any catalog
  other than BigLake reaches the worker, by type or by catalog-impl,
  Glue included (sink.catalog: glue). (gcp, validate,
  source-aligned-acquisition)
- New rule: a sink_connector_config, server.sink.config or
  iceberg_catalog_overrides whose type or catalog-impl selects another
  catalog than the expose's is an error on every platform; a REST
  endpoint fronting Glue is catalog: rest. The Nessie and BigQuery
  warnings follow a catalog-impl class too. (validate,
  source-aligned-acquisition, gcp)
@fas89
fas89 merged commit 6e94337 into main Oct 7, 2026
9 checks passed

This branch was successfully deployed

1 active deployment
github-pages — d1ed2da1 Deployed Oct 7, 2026 by fas89 via deploy #181
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant