Skip to content

[920] Address the review comments on the 0.4.0 Spark runtime docs - #922

Open
rangareddy wants to merge 1 commit into
apache:mainfrom
rangareddy:920-spark-runtime-docs-review
Open

[920] Address the review comments on the 0.4.0 Spark runtime docs#922
rangareddy wants to merge 1 commit into
apache:mainfrom
rangareddy:920-spark-runtime-docs-review

Conversation

@rangareddy

Copy link
Copy Markdown
Contributor

Closes #920.

What is this?

#913 added the 0.4.0 release notes and the Spark runtime quickstart. It was merged at
2026-08-25T15:20:10Z, about ten hours after a round of review comments was left on it, and those
comments were never applied. This PR applies them. No new content beyond what the review asked for.

Changes

website/docs/how-to-spark-runtime.md

Review comment Change
Section order confuses new users; orders-job.jar appears without context "Running a sync as its own job" now comes first, the in-job section after it
Cover building the job before running it The in-job section is now Writing the job, Building the job, Running the job, so the code precedes the spark-submit that submits it. Building the job documents the provided-scope dependency needed to compile against the runtime classes
Other quickstarts use local paths, this one used S3 All examples now use file:///tmp/hudi-dataset/people, the table the Quickstart creates
Mention the Hudi table has to exist first New Prerequisites section, linking the Quickstart
Add a section on running with and without a config file XTableSparkSync now has a subsection per mode: "Syncing a single table" and "Syncing several tables from a config file"
Move the Delta paragraph into a note It is now a :::note Which Delta implementation you get
"and newer" implies Spark 4.x Now states Spark 3.4.x and 3.5.x explicitly (see Verification below)
The demo/spark-runtime link 404s No change needed. It 404'd because the directory did not exist on main at review time; merging #913 created it and the link now resolves

website/releases/downloads.mdx

  • Each ### Release x.y.z-incubating heading now links to its release notes page.
  • Downloads moves to the top of the Releases sidebar. It rendered last because the release-notes
    pages use negative sidebar_position values (-4, -3, -2) while downloads.mdx used 1.

The existing "Release Notes:" bullet under each heading is left in place; it is now somewhat
redundant with the linked heading, and I am happy to drop it if reviewers prefer.

website/releases/release-0.4.0-incubating.mdx

  • Spark version support moves from mid-sentence into a :::note callout.

Verification

Facts were read from the 0.4.0-incubating tag rather than main, because xtable-spark-runtime
exists only on the 0.4.x release line (branch-0.4 and the release branches); main's pom.xml
does not list the module.

The Spark version claim specifically:

  • root pom.xml on the tag sets spark.version 3.4.2 and spark.version.prefix 3.4, with no
    3.5 anywhere in the file and no Spark profile;
  • xtable-spark-runtime/pom.xml depends on hudi-spark${spark.version.prefix}-bundle and
    iceberg-spark-runtime-${spark.version.prefix};
  • ITXTableSparkRuntimeBundle documents the 3.4 and 3.5 lines only.

Nothing supports Spark 4, so "3.5.x and newer" was removed.

Site checks, run locally on this branch:

  • npm run build succeeds with no new warnings (the remaining ones are pre-existing blog
    truncation-marker and inline-author warnings).
  • All 22 external links in the three changed files return 200.
  • All 8 internal doc links resolve to a built page. Worth noting because
    docusaurus.config.js sets onBrokenLinks: 'ignore', so a broken internal link would not fail
    the build.
  • Rendered output confirmed: heading order, both admonitions, the Scala/Java tabs, all nine code
    block titles, the linked release headings, and the Releases sidebar now ordered Downloads,
    0.4.0, 0.3.0, 0.2.0.

Out of scope

While verifying comments 6 and 7 I found two further inaccuracies in the same "Spark version
support" section, which this PR deliberately leaves alone so it stays scoped to the review
comments. Both are filed as #921:

  1. The page says Delta Kernel is "selected automatically" on Spark 3.5 "with no flag needed".
    That is true only for the XTableSparkSync CLI, which auto-enables it via isSparkAtLeast35.
    The in-job API the page leads with has no version detection: XTableSyncService.sync reads
    spec.isUseDeltaKernel(), and TableSyncSpec.useDeltaKernel defaults to false.
  2. The table labels the Spark 3.4 path "Delta Standalone", but xtable-spark-runtime/pom.xml
    depends on delta-core, delta-kernel-api and delta-kernel-defaults. It contains no
    delta-standalone dependency.

PR apache#913 was merged before the review comments left on it were addressed.
This applies them.

website/docs/how-to-spark-runtime.md
- Lead with "Running a sync as its own job" and move the in-job section
  after it. The page previously opened on a spark-submit that referenced
  an application jar nothing had introduced yet.
- Split the in-job section into writing, building and running the job, so
  the code appears before the command that submits it, and document the
  provided-scope dependency needed to compile against the runtime classes.
- Add a Prerequisites section pointing at the Quickstart for creating the
  source table every example on the page syncs.
- Give XTableSparkSync's two modes a subsection each, single table from
  command line options and several tables from a --datasetconfig file.
- Use the Quickstart's local file:///tmp/hudi-dataset/people paths rather
  than s3://example-warehouse/db/orders, so both pages can be followed
  with the same setup.
- Promote the paragraph on which Delta implementation is used to a note.
- State the supported Spark versions as 3.4.x and 3.5.x. "3.5.x and newer"
  read as a promise of Spark 4 support: the 0.4.0 build sets spark.version
  3.4.2 with hudi-spark3.4-bundle and iceberg-spark-runtime-3.4, and
  ITXTableSparkRuntimeBundle covers the 3.4 and 3.5 lines only.

website/releases/downloads.mdx
- Link each release heading to its release notes page.
- Move Downloads to the top of the Releases sidebar. It rendered below
  every release entry because those use negative sidebar_position values
  (-4, -3, -2) while downloads.mdx used 1.

website/releases/release-0.4.0-incubating.mdx
- Call out Spark 3.4.x and 3.5.x support instead of noting it mid-sentence.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Docs: address the open review comments on the 0.4.0 Spark runtime quickstart and Downloads page

1 participant