Plans and executes ML/LLM dataset creation and curation workflows.
npx skills add https://github.com/xcrrr/claude-skills --skill dataset-curator
Please help me install this Agent Skill. Skill: dataset-curator Source: https://github.com/xcrrr/claude-skills/tree/main/skills/ai-ml/dataset-curator Command: npx skills add https://github.com/xcrrr/claude-skills --skill dataset-curator
This skill covers the full lifecycle of dataset creation and curation for machine learning and LLM tasks. It addresses dataset schema design, data collection strategies, quality filtering, deduplication, class imbalance mitigation, stratified train/val/test splits, annotation guideline writing, and dataset card documentation. Good datasets are the foundation of reliable models — this skill helps teams avoid the most common data quality pitfalls that lead to poor generalization, evaluation leakage, and biased models.
Define the dataset schema, collection plan, annotation guidelines, deduplication strategy, and stratified train/val/test splits for a new dataset.
High-intent search queries and answers about installing and using this skill.
Dataset schema and annotation guidelines require explicit field definitions, quality filtering rules, and deduplication strategies to ensure reproducible machine learning training data. This structured approach mitigates class imbalance and prevents evaluation leakage.
Handling class imbalance and data deduplication involves applying quality filtering and mitigation strategies during dataset curation. This ensures clean, balanced training data, preventing biased models and poor generalization caused by duplicate or skewed samples.
Creating reproducible train, validation, and test splits requires stratified data partitioning during dataset curation. This ensures consistent evaluation and prevents leakage by maintaining class distributions across splits, which is critical for reliable LLM model assessment.
You can audit an existing dataset for quality, coverage, and potential biases by applying curation steps like quality filtering and deduplication. This process identifies data gaps and leakage, ensuring the dataset meets reliability standards for model training.
Dataset cards and documentation are required for machine learning dataset curation to ensure reproducibility and transparency. They capture schema definitions, annotation guidelines, and auditing steps, which prevent poor generalization and biased model outcomes.
Poor generalization often stems from dataset quality issues like duplicates, class imbalance, or evaluation leakage. Applying dataset curation practices such as quality filtering, deduplication, and stratified splits resolves these foundational data pitfalls.
GonglitianValidate dataset formats, schema consistency, and data quality for ML pipelines.
mvillmowPrepare and validate datasets for machine learning model training.
Ethical-AI-SyndicateDesign data annotation workflows with guidelines, quality control, and labeler training.
nimadorostkarImplement data validation gates for automated pipelines using Pandera or Great Expectations.
