feat(raster): byte-bounded batches for RS_EnsureLoaded (EnsureLoadedExec + sedona.raster.max_batch_bytes)

[james-willis]

First step of the plan on #1220 (Linear DB-273): bound the bytes RS_EnsureLoaded materializes per batch, sized dynamically per input batch from metadata, inside a Sedona-owned operator.

Problem

datafusion.execution.batch_size counts rows, but a row holding a raster can carry megabytes of pixels, so the batch DataFusion hands RS_EnsureLoaded can be gigabytes once loaded — 8192 × 1 MiB spatialbench tiles is 8 GiB per partition, which is the sf1 OOM. The obvious knobs don't help: DataFusion's AsyncFuncExec re-coalesces its input to exactly batch_size rows before evaluating, and the UDF's ideal_batch_size only chunks the invocation — the per-chunk results are concated back into one array. Same in DataFusion 54.1.

What this does

• sedona_raster::size — metadata-only estimates: Σ_bands Π raw_source_shape × pixel bytes. One formula for both storage kinds: for InDb bands it equals the buffer length, for OutDb bands it is the post-load size. Uses the raw source shape (not the visible one) so views don't distort it; shared buffers over-count, the safe direction. Never touches data, so it works on OutDb rows before anything is loaded.
• sedona.raster.max_batch_bytes — new raster namespace on SedonaOptions, SET-able. Default 256 MiB; when a memory limit is configured, SedonaContext lowers it to 1/8 of the per-partition limit (floor 16 MiB), mirroring how the spill threshold is derived. 0 disables.
• RasterBatchBudgetRule + EnsureLoadedExec (sedona-query-planner) — a physical optimizer rule appended after DataFusion's own replaces every AsyncFuncExec carrying rs_ensureloaded with EnsureLoadedExec: same expressions, same schema and plan properties (including the __async_fn_N output columns the planner projects on top), but each input batch is sliced so the estimated bytes about to be materialized stay within the budget, and each slice is evaluated and emitted as its own batch. A row over budget goes alone; null rows cost nothing; empty batches flow through; other async expressions in the same node are evaluated per slice. Slices are zero-copy RecordBatch::slices, so order is preserved.
• Wired into SedonaContext::new_local_interactive_with_runtime_env next to the existing planner rules (SedonaContext::new() wraps an external SessionContext and, as today, installs no rules).

Rebased on main with #1251 merged and DataFusion 54.1: each slice now reaches the loader as one bundled load() per loader, which the end-to-end test asserts.

Measurements

2048 OutDb rasters of 1024 × 1024 UInt8 (1 MiB each) through SELECT RS_EnsureLoaded(rast), mock loader returning committed pages, single partition, output streamed and dropped, debug build, macOS max RSS of the test process:

sedona.raster.max_batch_bytes	output batches	max RSS
0 (unbounded — today's behaviour)	1	2.20 GB
256 MiB (default)	8	779 MB
64 MiB	32	591 MB
16 MiB	128	145 MB

RSS is a high-water mark that includes allocator retention, so it tracks the budget loosely rather than matching it; the point is that peak memory now follows the budget instead of the row count.

Not in this PR (later steps of the same plan)

• Stopping FilterExec / CoalesceBatchesExec / RepartitionExec from re-batching downstream. The end-to-end test pins target_partitions = 1 so the round-robin repartition the planner puts above the exec doesn't obscure its output.
• Byte-bounded slicing of already in-db columns below raster-producing projections; scan-side batch sizing; memory-pool reservations with adaptive shrinking.
• Output-size estimation for kernels whose output exceeds their input (RS_FromGDALRaster, RS_AsRaster, upsampling).

Testing

• sedona-raster: estimator tests — InDb equals buffer length, OutDb from metadata without loading, raw vs visible shape under a view, multi-band + null + zero-band rows, overflow rejected instead of wrapping.
• sedona-query-planner: slice_ranges unit test; exec tests with a mock rs_ensureloaded async UDF — uniform rasters sliced to the budget, oversized rows alone with nulls free, 0 disables, output schema and __async_fn_0 column parity with AsyncFuncExec, empty batch, the rule ignores unrelated async UDFs, mixed async expressions sized by the ensure-loaded argument only.
• sedona: end-to-end SQL through SedonaContext with a mock loader and SET sedona.raster.max_batch_bytes = 600: EXPLAIN shows EnsureLoadedExec and no AsyncFuncExec; output batches are [2, 2, 2] and the loader receives exactly one bundled call per slice ([2, 2, 2] requests).
• Full suites locally on the rebased branch: sedona-raster 195, sedona-common 10, sedona-query-planner 90, sedona 101 (3 skipped for missing geoarrow-data assets). clippy and fmt clean.