wire-datafusion-function

What problem does it solve?

It solves the challenge of matching Spark expression semantics to Comet by wiring an existing DataFusion or datafusion-spark function into the Comet Scala serde layer and (when needed) the native UDF registration path.

Core Features & Use Cases

• Semantics-matched wiring patterns: Selects the lightest correct integration approach (passthrough, register + passthrough, or custom serde) based on Spark contract requirements.
• Correct Spark-to-Comet behavior: Ensures return types, null handling, ANSI error paths, type restrictions, and foldable-only argument rules are preserved.
• Testable integration: Adds Spark SQL file tests that exercise validity bitmaps, edge cases (NaN, infinities, boundary values), and fallback behavior when serde returns None.
• Documentation updates: Updates the expressions user guide and support matrix while relying on generated compatibility pages for per-expression details.

Quick Start

Use wire-datafusion-function to integrate the existing upstream $ARGUMENTS function into Comet by following the wiring-pattern decision, applying the Scala serde wiring, registering the UDF only when required, and adding the corresponding SQL file tests for the directly supported input types.