skills.rest
xiaoyu-ops
xiaoyu-opsCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

dataset-discovery

Search HuggingFace Hub, OpenML, GitHub, and Semantic Scholar for relevant datasets.

Updated Apr 1, 2026
Search Tags:#deduplication#ranking#huggingface#semantic-scholar#multisource-search#dataset-discovery#ml-datasets
One-click install
npx skills add https://github.com/xiaoyu-ops/the_work_of_dedup --skill dataset-discovery-xiaoyu-ops
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: dataset-discovery
Source: https://github.com/xiaoyu-ops/the_work_of_dedup/tree/main/.claude/skills/dataset-discovery
Command: npx skills add https://github.com/xiaoyu-ops/the_work_of_dedup --skill dataset-discovery-xiaoyu-ops

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires requests.

What problem does it solve?

Multi-source ML dataset discovery scans diverse sources (HuggingFace Hub, OpenML, GitHub, Semantic Scholar) to surface datasets relevant to a research task and reduces duplication across sources.

Core Features & Use Cases

  • Cross-source search across HuggingFace Hub, OpenML, GitHub, and Semantic Scholar to assemble a prioritized dataset list.
  • Deduplication and ranking by relevance, popularity, and recency, with optional modality filtering.
  • Generates markdown summaries and supports detail and pull workflows for downstream data collection.
  • Use Case: Researchers can quickly assemble a concise, licensed dataset set for experiments such as image classification or text benchmarking.

Quick Start

Find datasets for a given ML task by running a multi-source search to obtain a ranked, deduplicated list.

Frequently Asked Questions about dataset-discovery

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find relevant ML datasets across HuggingFace, OpenML, and GitHub?▼

Multi-source dataset discovery searches HuggingFace Hub, OpenML, GitHub, and Semantic Scholar to surface ML datasets relevant to a research task. It returns a ranked, deduplicated list with basic metadata for rapid scoping.

Can I filter datasets by modality and popularity during a cross-source search?▼

Cross-source dataset search supports ranking by relevance, popularity, and recency, with optional modality filtering. This allows you to assemble a prioritized dataset list tailored to your specific ML task.

What is the best way to remove duplicate datasets found on multiple platforms?▼

Deduplication across sources is handled automatically during the multi-source search. The workflow scans HuggingFace, OpenML, GitHub, and Semantic Scholar, returning a clean, deduplicated dataset list.

Does this multi-source dataset search require complex dependencies to run?▼

Multi-source dataset search requires only the requests dependency. It provides an extensible workflow that can be augmented with dedicated scripts for detail and pull operations.

How do I generate markdown summaries for discovered ML datasets?▼

The dataset discovery workflow generates markdown summaries automatically after searching across multiple sources. These summaries support detail and pull workflows for downstream data collection.

Why use Semantic Scholar for dataset discovery instead of standard dataset repositories?▼

Semantic Scholar is included alongside HuggingFace, OpenML, and GitHub to surface datasets cited in research papers. This provides a broader, cross-source view of datasets relevant to your research task than standard repositories alone.

Related Skills

Explore More Data & Analytics Skills→
LigphiDonkLigphiDonk
708

dataset-discovery

Search HuggingFace Hub, OpenML, GitHub, and Semantic Scholar for relevant ML datasets.

Community
Advanced
crazymsncrazymsn
22

flowio

Parse FCS files into NumPy arrays, metadata, and CSV/DataFrame outputs.

Community
Intermediate
OpenLAIROpenLAIR
1.0k

dataset-discovery

Find and rank datasets from HuggingFace, OpenML, GitHub, and papers.

Official
Intermediate
easyspace-aieasyspace-ai

paper-research

Search arXiv, Google Scholar, and Semantic Scholar for papers and summarize key contributions.

Community
Advanced