data-version-control

Version datasets and orchestrate reproducible pipeline stages with DVC.

33|6|Updated Mar 17, 2026
One-click install
npx skills add https://github.com/xjtulyc/awesome-rosetta-skills --skill data-version-control
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: data-version-control
Source: https://github.com/xjtulyc/awesome-rosetta-skills/tree/main/skills/23-research-workflow/data-version-control
Command: npx skills add https://github.com/xjtulyc/awesome-rosetta-skills --skill data-version-control

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

Data-intensive research often becomes impossible to reproduce when datasets, preprocessing logic, and experiment outputs change over time without a reliable version history.

Core Features & Use Cases

  • Git-like dataset versioning for large files: track dataset states with DVC so you can retrieve exact inputs later.
  • Reproducible pipelines with dvc.yaml: define stages (preprocess → train → evaluate) so only changed dependencies rerun.
  • Experiment tracking and comparison: use DVC experiments to compare metrics, artifacts, and hyperparameter settings across runs.

Use case example: you train multiple ML models across different preprocessing settings and hyperparameters, then reproduce the exact best-performing dataset + pipeline + metrics on another machine or in CI.

Quick Start

Use the DVC pipeline to add a dataset to version control and run the full reproducible workflow with a single command: dvc repro.

Frequently Asked Questions about data-version-control

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I version large datasets for machine learning reproducibility?

Dataset versioning tracks large file states with DVC so you can retrieve exact inputs later. It enables Git-like snapshot capabilities for data-intensive ML research without storing heavy files directly in Git.

How do I automate ML pipeline stages so only changed dependencies rerun?

Pipeline automation orchestrates ML stages by defining preprocess, train, and evaluate steps in a dvc.yaml file. DVC automatically detects changed dependencies and executes only the modified pipeline stages during reproduction.

What is the best way to track and compare ML experiment metrics across runs?

Experiment tracking compares metrics, artifacts, and hyperparameter settings across multiple runs using DVC experiments. It allows researchers to evaluate different preprocessing settings and identify the best-performing model configuration.

Do I need a remote cache configured to share reproducible data with collaborators?

Remote cache configuration is required to share reproducible data workflows with collaborators and CI systems. DVC remotes store versioned dataset snapshots and pipeline outputs, enabling exact environment reproduction across machines.

How to reproduce a specific machine learning pipeline execution from scratch?

Reproducing a pipeline execution requires running the dvc repro command in a configured DVC repository. This single command executes the full workflow defined in dvc.yaml, regenerating exact dataset states and model metrics.