datafusion-query-advisor

Optimize DataFusion queries by applying predicate pushdown to move filters to storage layers, enabling partition pruning on partition columns, projecting only needed columns, and evaluating join ordering. This Skill analyzes your queries and DataFrame operations to identify these bottlenecks and provide rewritten code examples with EXPLAIN plan verification.

Predicate pushdown moves WHERE clause filters down to the storage layer, reducing the amount of data scanned. For DataFusion queries, this technique dramatically improves performance by filtering data earlier in the execution plan rather than after scanning full datasets.

Partition pruning skips unnecessary partitions during scans by filtering on partition columns. This Skill detects when your filters target partition columns and suggests how to structure queries to enable pruning, reducing I/O and query execution time.

Column projection reads only the columns your query needs, minimizing I/O overhead. This Skill identifies SELECT * statements on wide tables and recommends explicit column selection to reduce data movement and improve query speed.

Yes. This Skill evaluates join strategies and ordering in DataFusion queries, analyzes execution plans via EXPLAIN, and suggests more efficient join sequences to reduce intermediate data shuffling and improve overall query performance.

This Skill identifies non-pushed filters, SELECT * on wide tables, improper join ordering, and missing partition column filters. It provides actionable rewrite suggestions with example code blocks and EXPLAIN plan verification for each optimization.