skills.rest
ehadziabdic
ehadziabdicCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

llm-data-preparation

Generate, curate, deduplicate, and format training datasets for LLM fine-tuning and alignment.

Updated Sep 23, 2026
Search Tags:#deduplication#synthetic data#data annotation#dpo#rlhf#llm fine-tuning#dataset curation
One-click install
npx skills add https://github.com/ehadziabdic/WAgents --skill llm-data-preparation-ehadziabdic
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llm-data-preparation
Source: https://github.com/ehadziabdic/WAgents/tree/main/opencode/skills/llm-data-preparation
Command: npx skills add https://github.com/ehadziabdic/WAgents --skill llm-data-preparation-ehadziabdic

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires openai, datasets, datasketch, argilla, pandas, pyarrow, langdetect, numpy, scikit-learn, and includes scripts (resource) and references (resource) components.

What problem does it solve? Preparing high-quality training data is the most time-consuming and error-prone part of LLM fine-tuning: raw data contains duplicates, PII, low-quality examples, benchmark contamination, and inconsistent formats. This Skill provides a complete pipeline for creating, cleaning, validating, and versioning instruction and preference datasets. ## Core Features & Use Cases - Synthetic Data Generation: Bootstrap instruction-response pairs with Self-Instruct, Evol-Instruct, and topic-based generation via OpenAI-compatible APIs, with built-in quality filtering and MinHash LSH deduplication. - Dataset Curation CLI: Compute statistics, remove exact and near-duplicates, redact PII (credit cards with Luhn validation, SSNs, emails, phones, IPs), filter by length and language, and create stratified train/val/test splits. - Preference & Annotation Workflows: Build chosen/rejected pairs for DPO/RLHF and set up annotation pipelines with Argilla 2.0, Label Studio, or Prodigy. - Use Case: You need 5,000 domain-specific instruction examples for a customer support fine-tune. Generate synthetic pairs from seed tasks, deduplicate against your eval set, redact PII, score quality, and export in Alpaca, ShareGPT, or OpenAI chat format. ## Quick Start Generate 100 synthetic instruction-response pairs about MLOps, deduplicate them, and save the result in chat format to a JSONL file.

Frequently Asked Questions about llm-data-preparation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate synthetic training data for LLM fine-tuning?▼

Use the generate_synthetic.py script with Self-Instruct (seed tasks), Evol-Instruct (complexity evolution), or topic-based generation through an OpenAI-compatible API. Output is automatically length-filtered, deduplicated with MinHash LSH, and formatted as Alpaca, ShareGPT, or chat JSONL.

How to remove near-duplicates from a training dataset?▼

Run curate_dataset.py with the deduplicate action, which applies SHA-256 exact dedup followed by MinHash near-duplicate detection at a configurable Jaccard threshold (default 0.8). Word-trigram shingles are used so documents sharing vocabulary but not phrasing are not falsely merged.

Argilla vs Label Studio for RLHF preference data annotation?▼

Argilla offers native preference/RLHF question types and LLM suggestions via its Python SDK, making it the default choice for fine-tuning data. Label Studio is more general-purpose with ML backend integration, while Prodigy suits rapid single-user NLP annotation.

Does the Argilla 1.x API still work for dataset logging?▼

No. Argilla 2.0 removed rg.init, rg.FeedbackDataset, and push_to_argilla entirely. Use the rg.Argilla client with rg.Dataset and rg.Settings, and log records through dataset.records.log with optional model-generated suggestions.

Why is benchmark decontamination necessary before fine-tuning?▼

Training rows overlapping evaluation sets cause inflated scores that measure memorization rather than capability. The standard approach removes training examples sharing 13-gram overlap with eval sets, complemented by MinHash or embedding similarity to catch paraphrased contamination.

What are the limitations of off-policy DPO preference pairs?▼

Sampling rejected responses from a weaker model introduces style and length confounds, so DPO learns to prefer verbose outputs rather than better answers. Standard practice is on-policy sampling from the model being trained, with comparable length distributions or length-debiasing penalties.

Related Skills

Explore More Data & Analytics Skills→
forcedotcomforcedotcom
803

salesforce-web-app-creating-records

Create Salesforce records from React web apps using createRecord and handle returned record IDs.

Official
Advanced
microsoftmicrosoft
1.4k

finetuning

Fine-tune models on Microsoft Foundry using SFT, DPO, and RFT training workflows.

Official
Advanced
intertwineintertwine
3

sv-hf

Push metadata and canonical datasets to HuggingFace repositories with gated access.

Community
Intermediate
open-thoughtsopen-thoughts
276

datagen-standard-launch

Generate synthetic datasets via Curator sharding or declarative data scripts.

Official
Intermediate