skills.rest
patchy631
patchy631COMMUNITY
Explore all skills by this author →
𝕏@akshay_pachaarGitHub

hugging-face-datasets

Manage Hugging Face Hub datasets with SQL queries via DuckDB.

36.9k|6.1k|Updated Oct 21, 2024
Search Tags:#SQL#MCP#DuckDB#data-pipelines#datasets#manage#HuggingFace Hub
One-click install
npx skills add https://github.com/patchy631/ai-engineering-hub --skill hugging-face-datasets-patchy631
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hugging-face-datasets
Source: https://github.com/patchy631/ai-engineering-hub/tree/main/hugging-face-skills/skills/hugging-face-datasets
Command: npx skills add https://github.com/patchy631/ai-engineering-hub --skill hugging-face-datasets-patchy631

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires huggingface_hub, duckdb, datasets, pandas, and includes scripts (resource) components.

What problem does it solve?

This skill provides tools to manage datasets on the Hugging Face Hub with a focus on creation, configuration, content management, and SQL-based data manipulation. It is designed to complement the existing Hugging Face MCP server by providing dataset editing and querying capabilities.

Core Features & Use Cases

  • Dataset Lifecycle Management: Initialize repos with proper structure, configure with detailed system prompts, and stream updates to datasets.
  • SQL-Based Dataset Querying (NEW): Query Hugging Face datasets using DuckDB SQL via the hf:// protocol; supports schema discovery, sampling, aggregations, transformations, and exporting/pushing results to the Hub.
  • Multi-Format Dataset Support: Supports templates for Chat, Text Classification, Question-Answering, Text Completion, Tabular Data, and Custom Formats.
  • Quality Assurance Features: JSON validation, batch processing, and error recovery during data operations.

Quick Start

  • Initialize a new dataset: python scripts/dataset_manager.py init --repo_id "your-username/dataset-name"
  • Configure with a system prompt: python scripts/dataset_manager.py config --repo_id "your-username/dataset-name" --system_prompt "$(cat system_prompt.txt)"
  • Add sample data: python scripts/dataset_manager.py add_rows --repo_id "your-username/dataset-name" --rows_json '[{"messages": [...]}]'

Frequently Asked Questions about hugging-face-datasets

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I query Hugging Face datasets using SQL?▼

You can query Hugging Face datasets using SQL by leveraging DuckDB SQL via the hf:// protocol. This workflow supports schema discovery, sampling, aggregations, transformations, and exporting results back to the Hub.

Can I create and manage dataset repos on the Hugging Face Hub programmatically?▼

Yes, you can initialize dataset repos with proper structure, configure them using detailed system prompts, and stream updates. This workflow handles the full dataset lifecycle management directly from your scripts.

What is the best way to edit and transform Hugging Face datasets for machine learning?▼

The best way to edit and transform datasets is using SQL-based data manipulation through DuckDB. It enables on-demand transformations, JSON validation, batch processing, and error recovery during data operations.

Does this workflow support multi-format datasets like chat and text classification?▼

Yes, the workflow supports multi-format datasets. It includes templates for Chat, Text Classification, Question-Answering, Text Completion, Tabular Data, and Custom Formats to suit various ML tasks.

How do I add sample data rows to a Hugging Face dataset repository?▼

You can add sample data rows by executing the dataset manager script with the add_rows command, providing your repo_id and a JSON array of your rows. This enables streaming updates and batch processing.

Can I push SQL query results directly back to the Hugging Face Hub?▼

Yes, you can push SQL query results directly back to the Hugging Face Hub. The workflow supports exporting aggregations and transformations performed via DuckDB SQL back to your Hub repositories.

Related Skills

Explore More Data & Analytics Skills→
huggingfacehuggingface
10.9k

hugging-face-cli

Manage Hugging Face Hub resources via the hf CLI.

Official
Intermediate
jluo41jluo41

haipipe-data-1-source

Convert CSV, XML, Parquet, and JSON files into standardized SourceSet assets.

Community
Advanced
crossxwillcrossxwill
5

duckdb-parquet-lab-workflow

Query Parquet data with DuckDB and convert results to pandas DataFrames.

Community
Intermediate
Himanshu040604Himanshu040604

hugging-face-datasets

Query and transform Hugging Face datasets using SQL via DuckDB.

Community
Advanced