apacheOfficial·7 Agent Skills Included

datafusion-comet

AI-assisted development toolkit for the Apache Spark Comet accelerator

Guides AI agents through building, reviewing, and optimizing Apache DataFusion Comet, a native accelerator that speeds up Apache Spark queries. Automates pull request reviews, bug triage, expression implementation, and performance audits against multiple Spark versions. Removes manual cross-checking of Spark source code, test coverage gaps, and benchmark regressions so contributors ship compatible changes faster.

1,262|364

↗

npx skills add apache/datafusion-comet --all -g -y

Available:

Points the AI agent to the project's contributor guides and enforces critical build rules, such as the JVM/native build order and the ban on hand-editing Spark SQL test diff files.

All Skills in This Repository (7)

Pure Emerald Level Indicators

📦 In Repo

apache

review-comet-pr

Guide structured review of DataFusion Comet PRs for Spark compatibility.

audit-comet-expression

Audit Comet expressions for correctness and test coverage across Spark versions.

bug-triage

Classifies and labels open GitHub issues marked requires-triage per the project triage guide.

wire-datafusion-function

Wire existing DataFusion functions into Comet for Spark expression support.

optimize-comet-expression

Optimizes existing native scalar expressions in the datafusion-comet spark-expr crate with benchmarked, regression-gated changes.

implement-comet-expression

Implements new Spark expressions in DataFusion Comet with serde, Rust wire-up, and test iteration.

suggest-native-expression

Assesses Spark expressions for native Rust implementation in Comet and files GitHub issues.

Official

Advanced

Frequently Asked Questions

FAQPage Schema

How to install DataFusion Comet skills?▼

Run `npx skills add apache/datafusion-comet --all -g -y` in your terminal to install all skills in this suite globally.

What is Apache DataFusion Comet?▼

Comet is a high-performance accelerator for Apache Spark that runs queries on Apache Arrow columnar data via the Rust-based DataFusion engine. It delivers roughly 2x speedups on standard benchmarks with no changes to your Spark code.

How do these skills help Comet contributors?▼

They automate PR reviews, bug triage, new expression implementation, correctness audits, and performance optimization while checking everything against the canonical Spark source code.

Do these skills work with Claude Code?▼

Yes. All skills follow the standard SKILL.md format and run in Claude Code and other compatible AI coding agents.

Do I need to know Rust to use these skills?▼

No. The agent reads the Spark and Rust sources for you and guides each step, though basic familiarity with the Comet codebase helps when reviewing its output.