Convert Apache Spark UDFs into equivalent Spark SQL expressions.
npx skills add https://github.com/NVIDIA/cudf-spark --skill udf-convert-to-sql
Please help me install this Agent Skill. Skill: udf-convert-to-sql Source: https://github.com/NVIDIA/cudf-spark/tree/main/skills/udf-convert-to-sql Command: npx skills add https://github.com/NVIDIA/cudf-spark --skill udf-convert-to-sql
This Skill removes the manual effort of translating an Apache Spark UDF into an equivalent Spark SQL expression while keeping the CPU behavior as the reference.
Use this skill to convert the UDF class into a Spark SQL expression and update the comparison test so it can be validated against the CPU implementation.
High-intent search queries and answers about installing and using this skill.
To convert a Spark UDF to a SQL expression for GPU acceleration, you translate the user-defined function into a functionally equivalent Spark SQL resource. This shifts execution from CPU-bound JVM logic to GPU-accelerated SQL operations.
Migrating Apache Spark UDFs to GPU-ready SQL involves generating a resource-backed SQL file and updating the SqlComparisonTest. You then iterate test execution until CPU and SQL outputs match for equivalence checking.
Testing CPU and SQL output parity is required when converting Spark UDFs to prevent logic discrepancies. The CPU implementation serves as the reference behavior, ensuring the GPU-accelerated SQL expression matches exactly without overfitting to sample cases.
Yes, converting Spark UDFs to SQL works without external dependencies. The conversion requires only a correct snake-case SQL resource and an updated comparison test to validate functional equivalence within your Apache Spark environment.
You can use Spark SQL expressions directly in a GPU acceleration workflow by replacing UDFs with equivalent SQL. This allows the pipeline to execute logic natively on the GPU, bypassing CPU bottlenecks during large-scale data processing.
NVIDIAReview converted UDFs for correctness gaps and hidden fallback risks.
drunkcodingBenchmark FlashInfer kernel performance across backends using CUPTI timing.
emilneuraz-aiOptimize Apache Spark jobs with partitioning, caching, and memory tuning.
NVIDIAConvert Apache Spark CPU UDFs into native CUDA RapidsUDF implementations.
