duckdb

Analyze and manipulate local CSV, Parquet, and JSON files using DuckDB SQL.

Updated Aug 23, 2026

Search Tags:#json #data analysis #sql #csv #parquet #duckdb #local files

Ready to run

npx skills add https://github.com/dtsong/data-engineering-skills --skill duckdb-dtsong

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: duckdb
Source: https://github.com/dtsong/data-engineering-skills/tree/main/duckdb
Command: npx skills add https://github.com/dtsong/data-engineering-skills --skill duckdb-dtsong

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill empowers you to perform SQL-based data analysis directly on your local files (CSV, Parquet, JSON, Excel) without needing a full data warehouse, making data exploration and cleaning faster and more accessible.

Core Features & Use Cases

Local File Ingestion: Read various file formats directly into DuckDB tables using SQL.
Data Profiling & Cleaning: Analyze column statistics, identify nulls, and perform SQL-based transformations.
Use Case: You have a large CSV file containing customer data downloaded from a SaaS tool. Use this Skill to load it into DuckDB, profile the data to check for missing values and data types, and then write SQL to clean and deduplicate records before exporting to Parquet.

Quick Start

Use the duckdb skill to read the file 'sales_data.csv' and count the number of rows.

Frequently Asked Questions about duckdb

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I run SQL queries on local CSV and Parquet files without a data warehouse?▼

You can run SQL queries on local CSV and Parquet files without a data warehouse by using DuckDB to ingest files directly into local tables. This allows you to profile, clean, and manipulate data entirely on your machine without requiring cloud warehouse connections.

What is the best way to profile and clean large JSON datasets locally?▼

The best way to profile and clean large JSON datasets locally is to load them into DuckDB and execute SQL commands. You can analyze column statistics, identify nulls, and perform SQL-based transformations directly on your local JSON files without needing a cloud warehouse.

Can I use SQL to deduplicate records in a CSV file before exporting to Parquet?▼

Yes, you can use SQL to deduplicate records in a CSV file before exporting to Parquet. By loading the CSV into DuckDB, you can write SQL queries to clean the data and then export the manipulated results directly to a Parquet file format.

Does local file analysis with DuckDB require any external dependencies?▼

Local file analysis with DuckDB does not require external dependencies or cloud warehouse connections. It operates entirely on local files, supporting data ingestion and manipulation for formats like CSV, Parquet, and JSON directly on your machine.

How do I check for missing values and data types in local data files?▼

To check for missing values and data types in local data files, load your CSV, Parquet, or JSON files into DuckDB. You can then use SQL to profile the data, analyzing column statistics and identifying nulls to understand your dataset's quality.