Repository navigation
docs(iceberg): document how location.catalog is classified (forge-cli #707) - #144
Merged
Merged
Conversation
… #707) forge-cli #707 replaces the per-emitter reading of binding.location.catalog with one catalog-kind table. Document it, and correct the pages that described the old partition: - advanced/source-aligned-acquisition: new "Iceberg catalogs" section with a Lakekeeper example, the spelling and platform-default rules, the per-kind table (sink selector, dbt catalogs.yml on Snowflake, Snowflake and AWS modules, streaming-sink requirements), the Kafka Connect catalog-impl XOR type fix, the AWS behaviour for a non-Glue table, and the iceberg_catalog_move_blocked upgrade path. - providers/snowflake: the external-vs-managed list, the apply prerequisites, the --strict note and policy compile for Iceberg exposes. - cli/validate: new "Iceberg catalog checks" section; note on the 0.14.0 --strict warning. - cli/generate-iac, cli/apply, cli/diff, providers/aws, providers/gcp: the Glue-table, guard, not_checked, governance and BigLake claims the change makes false. Every new behaviour is marked unreleased; 0.19.0 and earlier behaviour is stated beside it. No page or heading moved.
8 of 15 tasks
Merged
6 of 9 tasks
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Companion docs for forge-cli #707 ("fix(iceberg): one catalog-kind table, so every emitter agrees on
location.catalog").#707 replaces each emitter's hand-written reading of
binding.location.catalogwith one table of catalog kinds. This PR documents that table and corrects the pages that describe the old partition. It documents only what #707 ships. Nothing from the held-back Lakekeeper branch is here: no/cataloguri warning, no warehouse-must-be-a-name error, nolakekeeper/polarisin thesink.catalogenum, and nosink.catalogAuth.#707 is in no release yet: forge-cli 0.19.0 is the latest tag, and it predates #707. So every new statement is marked (forge-cli #707, unreleased), and the 0.19.0-and-earlier behaviour is stated beside it. The site still tracks 0.18.1, and readers on that version need the old behaviour too.
Related CLI PR
Pages changed
advanced/source-aligned-acquisition.mdlocation.catalog). It covers:- a Lakekeeper example (binding, the validate error without
uri, the derived Kafka Connect catalog keys, the AWS module);- the spelling-folding rule and the platform defaults;
- the unknown-value and
sink.catalogrefusals;- a per-kind table (sink selector, dbt
catalogs.ymlon Snowflake, Snowflake module, AWS module, streaming-sink requirements);- the Kafka Connect
catalog-implXORtypefix;- the AWS behaviour for a non-Glue table;
- the
iceberg_catalog_move_blockedupgrade path with thetofu state rmshape;- the other commands, and what 0.19.0 and earlier did.
One sentence on
sink.catalogunder "Where the build lands data".providers/snowflake.mdcatalogs.ymlexternal/managed list (addslakekeeper,bigquery,iceberg_rest;hive/jdbc/hadoop/dynamodbleft out and refused), the apply prerequisites, the--strictCI note, and policy compile for an Iceberg expose.cli/validate.md--strictwarning.cli/generate-iac.mdawsrow of the provider table (no Glue table for a non-Glue Iceberg expose). The Snowflake--strictcaveat now covers every REST kind, andhive/jdbc/hadoop/dynamodbare an error.cli/apply.mdiceberg_catalog_move_blockedrow in "Errors you can hit".cli/diff.mdnot_checked, not looked up in Glue.providers/aws.mdaccessPolicy.grantswarning and policy compile for a non-Glue Iceberg expose, and a line in "Upgrading an existing AWS contract".providers/gcp.mdbigqueryis left out ofcatalogs.yml, and validate accepts its warehouse name.No page moved, and no heading was renamed or removed. forge_docs URLs and anchors are a CLI contract (
_DOC_ROUTES), so this PR only adds headings: 9 in the acquisition page, 1 invalidate.mdand 1 inapply.md.How the claims were checked
Every behavioural claim traces to #707's CHANGELOG
[Unreleased]entries or to its code at head05dbe18b.These outputs were produced by running forge-cli at
05dbe18b, not written by hand:fluid validateoutputs;fluid generate iacresult;iceberg_catalog_move_blockedmessage and itstofu state rmcommands.A script also ran every emitter over every kind to fill the per-kind table: the sink, dbt
catalogs.yml, the Snowflake and AWS modules, and validate.The "0.19.0 and earlier" statements were checked against the
v0.19.0tag. That tag is an ancestor of #707's base.Areas Updated
Checklist
npm run docs:devor built it withnpm run docs:buildChecks run locally:
npm ci && npm run docs:buildnode scripts/check-dist-links.mjslink-check.yml)--offlinesource check (mirror and canaries copied fromlink-check.yml; lychee 0.24.2)python scripts/check_cli_docs.py, withdata-product-forge==0.18.1installedpython scripts/check_providers.pypython scripts/gen_contract_reference.py --checkidin the built HTMLNotes
forge-cli/pull/707.