Support native `schema_of_variant` and `schema_of_variant_agg`

[peterxcli]

What is the problem the feature request solves?

Spark 4.x provides schema_of_variant for a single value and schema_of_variant_agg for the merged schema of a Variant column. Comet does not implement either expression, so schema discovery over a natively scanned Variant column falls back:

SELECT schema_of_variant(v) FROM t;
SELECT schema_of_variant_agg(v) FROM t;

Spark's scalar implementation infers and prints the schema, using OBJECT<...> rather than STRUCT<...>, in SchemaOfVariant. The aggregate shares that inference and merges schemas across rows and partial buffers in SchemaOfVariantAgg.

Describe the potential solution

Implement one shared Spark-compatible Variant schema inference/merge helper, then expose it through the scalar expression and aggregate:

• infer scalar, decimal precision/scale, date/timestamp, binary, UUID, array, object, and Variant-null (VOID) schemas;
• keep object fields in Spark's required order and print OBJECT<...> names with Spark-compatible quoting;
• merge heterogeneous array elements and object fields with Spark's compatible-type rules;
• ignore SQL NULL rows in the aggregate, start/finish an empty buffer as VOID, and support partial-buffer merge/serialization; and
• admit Variant only for these two expressions while preserving general fallback gates.

Add focused parity and native-plan tests for every scalar kind, JSON versus SQL NULL, nested arrays/objects, heterogeneous values, decimal widening, field-name quoting and ordering, empty/all-null inputs, grouping, and multi-partition partial aggregation.

Additional context

Related work: #4295, #5407, #5424, and #5425.

Non-goals: schema-driven Variant casts, subfield pruning, predicate pushdown, writing, shuffle/spill of Variant values, C2R, Python transport, and Iceberg-specific work.

[peterxcli]

take