Repository navigation
docs(iceberg): document the forge-cli catalog follow-up fixes (fix/iceberg-catalog-followups) - #145
Merged
Merged
Conversation
Companion to forge-cli fix/iceberg-catalog-followups (b87fd3a7), which closes the gaps the post-merge inspection of #707 found. Marked "(forge-cli fix/iceberg-catalog-followups, unreleased)". - Quoted CLI output regenerated from b87fd3a7: findings keep their square brackets (exposes[orders], [error]) and print on one line. - The streaming runners check a hand-written sink config too, for a build that declares sink.format: iceberg, and read the build they run. - GCP: an Iceberg expose a streaming sink writes must name its catalog. - catalog: bigquery on Kafka Connect warns (published sink 1.9.2). - Catalog-move guard: Snowflake EXTERNAL VOLUMEs, the Glue-table evidence rule, the iceberg_catalog_move_probe_skipped WARNING, and both events on the troubleshooting page. - The Snowflake module still emits a native snowflake_table whatever the catalog; fluid diff refuses an unknown catalog value.
Merged
9 of 15 tasks
fas89
added a commit
that referenced
this pull request
Oct 7, 2026
…orge-cli #709) (#146) Follow-up to #145 for forge-cli #709 at 6e6eb000 ("four gaps the defect scan found in this PR's own checks"). Quoted output regenerated from 6e6eb000. - Catalog-move guard: a Glue database is flagged when the moved expose's own Glue table is in state, or when the expose names no table (its database is all an earlier release created for it). The Snowflake volume is found by its contract-derived name, so an upgrade that also changed location.warehouse to a catalog name or dropped iam_role_arn is still stopped. (apply, source-aligned-acquisition, aws, snowflake, production-troubleshooting) - GCP: an expose with no location.catalog is refused when any catalog other than BigLake reaches the worker, by type or by catalog-impl, Glue included (sink.catalog: glue). (gcp, validate, source-aligned-acquisition) - New rule: a sink_connector_config, server.sink.config or iceberg_catalog_overrides whose type or catalog-impl selects another catalog than the expose's is an error on every platform; a REST endpoint fronting Glue is catalog: rest. The Nessie and BigQuery warnings follow a catalog-impl class too. (validate, source-aligned-acquisition, gcp)
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Companion docs for the forge-cli fix wave on branch
fix/iceberg-catalog-followups(headb87fd3a7, "fix(iceberg): close the gaps the post-merge inspection of #707 found"). It follows #144, which documented forge-cli #707.The fix wave is in no release, and its forge-cli PR has no number yet. Each new statement is marked (forge-cli fix/iceberg-catalog-followups, unreleased). When the PR number is known, replace the substring
forge-cli fix/iceberg-catalog-followupswith[forge-cli #NNN](https://github.com/Agenticstiger/forge-cli/pull/NNN)acrossdocs/.Related CLI PR
fix/iceberg-catalog-followups(PR number to be added)Pages changed
advanced/source-aligned-acquisition.mdexposes[orders] declares .... Narrowed the "one table" sentence: the Snowflake module readslocation.catalogonly for the EXTERNAL VOLUME and the Glue catalog integration, and still emitssnowflake_database/snowflake_schema/snowflake_tablewhatever the catalog. The GCP no-catalog case is now an error, quoted. Added thebigqueryKafka Connect warning (table row and quote). States exactly which builds the runner preflight covers, and that each runner reads the build it runs.fluid diff's planner refuses a typo, and a typo draws no Lake Formation refusal. Rewrote the AWS guard output and added the Glue-table evidence rule and theiceberg_catalog_move_probe_skippedWARNING. New heading Upgrading a Snowflake contract that names another catalog.cli/apply.mdtofu state rmshape), and the probe WARNING. Theiceberg_catalog_move_blockederror row covers Snowflake.providers/snowflake.mdtofu state rmcommand. A pointer to it from thefluid applybullet.providers/gcp.mdcatalog: bigqueryKafka Connect warning. Regenerated the placeholder-principal output (exposes[orders]).providers/aws.mdaccessPolicy.grantswarning. A new "Upgrading" bullet: the evidence rule, the probe WARNING, andfluid diffrefusing an unknown catalog.cli/validate.mdexposes[customers],[error]severity), with a note on what 0.19.0 and earlier print. Iceberg catalog checks: thebigquerywarning, the GCP rule, the unknown-catalog/Lake Formation change, and the preflight scope.cli/generate-iac.mdprincipal-placeholderoutput (exposes[customers]). The Snowflake volume guard, in the #707 paragraph.cli/diff.mdfluid diffon AWS refuses an unknownlocation.catalog(diff_failed), with the real message.advanced/production-troubleshooting.mdiceberg_catalog_move_blockedandiceberg_catalog_move_probe_skippedin "State and region errors". This is the CLI's fallback docs page (_DOC_FALLBACKinfluid_build/_errors.py).No page moved, and no heading was renamed or removed. forge_docs URLs and routed anchors are a CLI contract (
_DOC_ROUTES). This PR adds two headings: one insource-aligned-acquisition.mdand one insnowflake.md.How the claims were checked
Every behavioural claim traces to the CHANGELOG
[Unreleased]entries or the code atb87fd3a7. The "0.19.0 and earlier" statements were checked against thev0.19.0tag, which is an ancestor ofb87fd3a7. Every quoted output below was produced by forge-cli atb87fd3a7(editable install in the forge-cli worktree's.venv):fluid validate: the Lakekeeper example and its variants (nouri, typo,sink.catalog: glue,governance.lakeFormation, typo plus Lake Formation,accessPolicy, override with both selectors). Also a GCP Kafka Connect build with no catalog, the same withcatalog: bigquery, and the governance example with its two bundles (fluid bundle --format tgz).fluid generate iac: the placeholder-principal refusal.fluid diffon AWS: the unknown-catalog refusal.fluid applyagainst a moto server with realtofu1.12.0. The contract was applied with the Glue catalog first, then switched tocatalog: lakekeeper. The guard fired, and the printedtofu state rmlines were run as copied. The next apply then planned past the guard, and the Glue table stayed in moto.fluid applywith realtofuand the real Snowflake provider (init only). There is no Snowflake account, so the state file was written by hand. It holds the four addressesfluid generate iacemits for the same contract withoutlocation.catalog. The guard fired, the printed command removed the volume, and the next apply got past the guard (its plan then failed on missing credentials, as expected). The same setup withcatalog: hivealso fires the guard.iceberg_catalog_move_probe_skippedWARNING: fromfluid applyrun in-process. The only stub replaced thetofushell-outs, withtofu state pullfailing (AccessDenied).snowflake_tableclaim was read fromfluid generate iacforcatalog: lakekeeper.The commit's own tests pass at
b87fd3a7: 246 tests across the sink validation, deriver, Debezium, catalog-move, wiring, bracket-output, planner and unknown-catalog files.Areas Updated
Checklist
npm run docs:devor built it withnpm run docs:buildb87fd3a7)Checks run locally:
npm ci && npm run docs:buildnode scripts/check-dist-links.mjslink-check.yml)--offlinesource check (mirror and canaries fromlink-check.yml; lychee 0.24.2)python scripts/check_cli_docs.py(incl.--version-only), withdata-product-forge==0.18.1python scripts/check_providers.pypython scripts/gen_contract_reference.py --checkidin the built HTMLNotes
rest,iceberg_rest,polaris,unityandnessieamong the catalogs that "got a Snowflake EXTERNAL VOLUME from earlier releases". The code (_SNOWFLAKE_NO_VOLUME_BEFOREiniac/catalog_moves.py) says those never got one, and the guard does not fire for them. It fires forlakekeeper,bigquery, theiceberg-restspelling, andhive/jdbc/hadoop/dynamodb. These pages follow the code.iceberg_catalog_move_blockedhas no entry in forge-cli's error catalog, so the CLI prints no docs link for it. The troubleshooting row is there for a reader who searches the event name.exposes accessPolicy: ...):recipes/one-contract-two-clouds.md,concepts/governance-parity.md,concepts/governance-policy.mdandreference/preview-fields.md.