skills.rest
apacheapacheOfficial·5 Agent Skills Included

datafusion-python

Query CSV, Parquet, and JSON data fast with SQL or Python DataFrames

Runs fast SQL and DataFrame queries on CSV, Parquet, JSON, Pandas, and Arrow data directly inside Python. Eliminates the need for a separate database server, slow row-by-row loops, and manual data conversion between formats. Teaches AI agents to write correct DataFusion code, audit API coverage, and keep function signatures Pythonic.
599|167
↗
npx skills add apache/datafusion-python --all -g -y
Available:

Directs the AI agent to the right sub-skill for each task and enforces project rules on FFI capsule protocols, docstring examples, and documentation updates when writing or reviewing DataFusion Python code.

All Skills in This Repository (5)

Pure Emerald Level Indicators
📦 In Repo
apacheapache

datafusion-python

Write DataFusion DataFrame and SQL queries with Apache Arrow integration in Python.

Official
Advanced
📦 In Repo
apacheapache

make-pythonic

Audits and refactors Apache DataFusion Python API to accept native types for literal arguments.

Official
Advanced
📦 In Repo
apacheapache

check-upstream

Audits python-api coverage of upstream DataFusion Rust bindings and reports gaps.

Official
Intermediate
📦 In Repo
apacheapache

ffi-capsule-protocol

Enforces the settled PyCapsule FFI protocol for DataFusion extension getters and codecs.

Official
Advanced
📦 In Repo
apacheapache

audit-skill-md

Audits the datafusion-python skill documentation against the current public Python API.

Official
Advanced

Frequently Asked Questions

FAQPage Schema
How to install datafusion-python?▼

Run `npx skills add apache/datafusion-python --all -g -y` in your terminal to install all skills in this suite globally.

What is Apache DataFusion for Python?▼

It is a fast in-process query engine built on Apache Arrow that lets you run SQL or DataFrame queries on CSV, Parquet, JSON, and Pandas data without setting up a database server.

How do I query a Parquet file in Python?▼

Create a SessionContext, register the Parquet file, then run a SQL query or chain DataFrame methods like filter, aggregate, and sort before collecting results.

Does datafusion-python work with AI coding assistants?▼

Yes. It ships a SKILL.md following the Agent Skills open standard, so Claude Code, Cursor, Codex, and other agents can write idiomatic DataFusion code for you.

Can I convert results to Pandas or Polars?▼

Yes. Any DataFrame result can be converted with to_pandas(), to_polars(), to_arrow_table(), or to_pydict() for use in the rest of your Python workflow.

Related Repositories in Data & Analytics

View All in Data & Analytics→
PaddlePaddlePaddlePaddle
87.3k

PaddleOCR

Extract text, tables, and formulas from PDFs and images

Official📦 2 Skills
D4VinciD4Vinci
73.3k

Scrapling

Scrape any website and bypass anti-bot protection with AI

Community📦 1 Skills
mvanhornmvanhorn
60.8k

last30days-skill

Research any topic across Reddit, X, YouTube, and the web

Community📦 2 Skills