skills.rest
trmaphi
trmaphiCOMMUNITY
Explore all skills by this author →
GitHub

datafusion-cli

Query Parquet, Arrow, IPC, and AVRO files with SQL via DataFusion CLI.

Updated Mar 2, 2019
Search Tags:#cli#sql#parquet#datafusion#data-exploration#arrow#columnar
One-click install
npx skills add https://github.com/trmaphi/dotfiles --skill datafusion-cli
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: datafusion-cli
Source: https://github.com/trmaphi/dotfiles/tree/main/.claude/skills/datafusion-cli
Command: npx skills add https://github.com/trmaphi/dotfiles --skill datafusion-cli

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Analyzing and querying columnar data files (Parquet, Arrow, Feather, IPC, AVRO) typically requires a full database setup. DataFusion CLI lets you run SQL directly against your files, simplifying exploration and quick analytics.

Core Features & Use Cases

  • Direct SQL queries on Parquet, Arrow/Feather, IPC, and AVRO files without loading into a database.
  • Interactive REPL and one-shot query modes for rapid experimentation.
  • Useful for data exploration, prototyping data pipelines, and validating schemas.

Quick Start

Run a sample SQL query against a Parquet file to verify the setup and start exploring data.

Frequently Asked Questions about datafusion-cli

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I query Parquet files with SQL without setting up a database?▼

Yes, you can run SQL queries directly against AVRO and Arrow files using DataFusion CLI. It supports querying columnar data formats like Arrow, Feather, and IPC without needing to load them into a database first.

Can I run SQL queries directly on Arrow and AVRO files?▼

Yes, you can run SQL queries directly against AVRO and Arrow files using DataFusion CLI. It supports querying columnar data formats like Arrow, Feather, and IPC without needing to load them into a database first.

What is the best way to explore columnar data for ad-hoc analysis?▼

The best way to explore columnar data for ad-hoc analysis is using DataFusion CLI. It allows you to execute SQL queries directly on files like Parquet and Arrow, skipping full database setup for quick data exploration and schema validation.

Does DataFusion CLI support interactive REPL for data exploration?▼

Yes, DataFusion CLI supports an interactive REPL mode for data exploration. You can also use one-shot query modes to run single SQL statements against your columnar data files for rapid experimentation.

Do I need a database to run SQL for data pipeline prototyping?▼

No, you do not need a database to run SQL for data pipeline prototyping. DataFusion CLI lets you execute SQL directly against columnar data files like Parquet and AVRO, simplifying schema validation and quick analytics.

Related Skills

Explore More Data & Analytics Skills→
tondevreltondevrel
19

duckdb

Run in-process analytical SQL queries on Python DataFrames and files.

Community
Intermediate
itsadijmbtitsadijmbt
1

duckdb-en

Translate user intent into DuckDB CLI SQL, flags, dot commands, and export formats.

Community
Intermediate
spqwspqw

observable-framework-lib-arrow

Load Apache Arrow IPC and Parquet files in Observable Framework projects.

Community
Intermediate
Leonard-DataLeonard-Data

sqlite

Provide a serverless relational database with SQL queries and ACID transactions.

Community
Intermediate