hugging-face-datasets

Name: hugging-face-datasets
Availability: InStock
Author: Yog-Sotho

Automate Hugging Face Hub dataset lifecycle for LLM training pipelines.

4|Updated Mar 21, 2026

Search Tags:#duckdb #data-pipeline #datasets #huggingface #ml-training #dataset-management

Ready to run

npx skills add https://github.com/Yog-Sotho/claude-skills --skill hugging-face-datasets-yog-sotho

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: hugging-face-datasets
Source: https://github.com/Yog-Sotho/claude-skills/tree/main/hugging-face-datasets
Command: npx skills add https://github.com/Yog-Sotho/claude-skills --skill hugging-face-datasets-yog-sotho

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires duckdb, huggingface_hub, and includes scripts (resource) components.

What problem does it solve?

Create, manage, and transform Hugging Face Hub datasets for LLM training, SFT, RLHF, and DPO pipelines, enabling seamless data handling across the full lifecycle.

Core Features & Use Cases

Initialize and configure dataset repos on Hugging Face Hub, including system prompts and metadata.
Query, transform, and export data using DuckDB-backed SQL over HF datasets, then push subsets back to the Hub.
Stream new rows and manage iterative data improvements within a single workflow.

Quick Start

Create a new Hugging Face dataset, configure its system prompts, and start streaming initial rows using the provided scripts.

Frequently Asked Questions about hugging-face-datasets

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I manage Hugging Face Hub datasets for LLM training pipelines?▼

Manage Hugging Face Hub datasets for LLM training by automating repo initialization, configuring system prompts, streaming rows, and pushing data subsets to the Hub. It handles the end-to-end dataset lifecycle across SFT, RLHF, and DPO pipelines.

Can I query and transform Hugging Face datasets using SQL?▼

Query and transform Hugging Face datasets using DuckDB-backed SQL. Apply SQL operations to filter and manipulate dataset rows directly, then export the transformed subsets back to the Hugging Face Hub.

Do I need a Hugging Face token to create and push datasets to the Hub?▼

A valid HF_TOKEN with write access is required to create, manage, and push datasets to the Hugging Face Hub. The environment also requires uv installed and optionally supports the HF MCP server.

What is the best way to stream new rows into a Hugging Face dataset?▼

Stream new rows into a Hugging Face dataset by executing the provided scripts within the workflow. This enables iterative data improvements and row streaming directly into the configured repository.

Does this approach support dataset configuration for SFT and DPO pipelines?▼

Dataset configuration supports SFT, RLHF, and DPO pipelines by initializing repos, setting system prompts, and managing metadata. Transformations apply SQL to prepare and export subsets for specific training workflows.

Why use DuckDB for Hugging Face dataset transformations?▼

DuckDB provides SQL-based querying and transformation capabilities over Hugging Face datasets. It allows efficient filtering and manipulation of data before pushing the resulting subsets to the Hub.