hugging-face-datasets

Query and transform Hugging Face datasets with DuckDB SQL.

10.9k|724|Updated Nov 24, 2025
One-click install
npx skills add https://github.com/huggingface/skills --skill hugging-face-datasets
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: hugging-face-datasets
Source: https://github.com/huggingface/skills/tree/main/skills/hugging-face-datasets
Command: npx skills add https://github.com/huggingface/skills --skill hugging-face-datasets

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires duckdb, huggingface_hub, datasets, and includes scripts (resource) components.

What problem does it solve?

Hugging Face datasets often require manual setup for repo creation, configuration, and data processing. This Skill provides an end-to-end workflow to initialize, configure, and edit datasets, plus SQL-based discovery, transformation, and export capabilities.

Core Features & Use Cases

  • Dataset lifecycle management: initialize repos, configure system prompts, and manage content with templates.
  • SQL-based querying and transformation: query HF datasets using DuckDB, describe schemas, sample data, join datasets, and export to Parquet/JSONL.
  • HF Hub integration: push results to new datasets, manage access, and organize multi-split workflows.

Quick Start

Use uv run scripts/dataset_manager.py init to create a new dataset, then uv run scripts/dataset_manager.py quick_setup --template chat --repo_id "your-username/your-dataset" to bootstrap a dataset with chat templates. Then run uv run scripts/sql_manager.py query --dataset "your-username/your-dataset" --sql "SELECT * FROM data" to inspect.

Frequently Asked Questions about hugging-face-datasets

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I query Hugging Face datasets using SQL?

You can query Hugging Face datasets using SQL by leveraging DuckDB integration. The skill supports SQL-based data manipulation, allowing you to describe schemas, sample data, and join datasets directly across multiple splits.

What is the best way to initialize and configure a Hugging Face dataset repository?

The best way to initialize a Hugging Face dataset is using the provided Python scripts to create repos and bootstrap configurations. You can apply templates, like chat templates, and manage content across multiple splits programmatically.

Can I export Hugging Face dataset query results to Parquet or JSONL?

Yes, you can export Hugging Face dataset query results to Parquet or JSONL. The skill supports SQL-based transformation and export capabilities, enabling you to push processed results to new datasets on the Hugging Face Hub.

Do I need an HF_TOKEN to manage private Hugging Face datasets?

Yes, you need an HF_TOKEN environment variable set up to manage private Hugging Face datasets. The skill relies on Hugging Face Hub APIs and requires this token for authentication during initialization, querying, and pushing results.

Does this DuckDB integration support joining multiple Hugging Face datasets?

Yes, the DuckDB integration supports joining multiple Hugging Face datasets. You can use standard SQL commands to query, join, and transform data across different dataset splits and configurations seamlessly.