skills.rest
owid
owidOFFICIAL
Explore all skills by this author →
Website𝕏@OurWorldInDataGitHub

update-dataset

Orchestrate end-to-end dataset updates with checkpointed approvals and PR creation.

156|30|Updated Sep 17, 2021
Search Tags:#automation#workflow#checkpoints#etl#pr#update#dataset
One-click install
npx skills add https://github.com/owid/etl --skill update-dataset
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: update-dataset
Source: https://github.com/owid/etl/tree/main/.claude/skills/update-dataset
Command: npx skills add https://github.com/owid/etl --skill update-dataset

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

End-to-end dataset updates orchestrate PR creation, snapshot, meadow, garden, and grapher steps with auditable progress and per-step approvals.

Core Features & Use Cases

  • Orchestrates the full dataset update flow from versioning to downstream steps, including PR creation, snapshot generation, and transformation steps.
  • Provides checkpointed progress tracking by persisting status to workbench paths and requiring user approvals after each workflow step.
  • Supports optional branch specification and uses today's date as the new version, enabling reproducible updates across datasets.

Quick Start

Invoke the workflow with <namespace>/<old_version>/<name> to start an end-to-end dataset update.

Frequently Asked Questions about update-dataset

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate end-to-end ETL dataset updates with checkpointed approvals?▼

Automating end-to-end dataset updates involves orchestrating PR creation, snapshot, meadow, garden, and grapher steps with checkpointed progress tracking that requires user approvals after each workflow step.

What is the best way to manage PR creation and versioning for production data pipelines?▼

The best way to manage PR creation and versioning for production data pipelines is by automating the full dataset update flow, using today's date as the new version to enable reproducible updates across datasets.

How do I persist ETL workflow progress across multiple transformation steps?▼

You persist ETL workflow progress by saving checkpointed status to workbench paths after each transformation step, ensuring auditable progress and per-step approvals before continuing.

Can I specify a custom branch when running a dataset update workflow?▼

Yes, the dataset update workflow supports optional branch specification alongside using today's date as the new version for reproducible updates across datasets.

How do I start an end-to-end dataset update using a namespace and version?▼

You start an end-to-end dataset update by invoking the workflow with the namespace, old version, and name parameters to trigger the full orchestration from snapshot generation through grapher steps.

When do I need checkpointed approvals for ETL data refreshes?▼

You need checkpointed approvals for ETL data refreshes when running production pipelines that require auditable progress and provenance, ensuring each transformation step is validated before proceeding to the next.

Related Skills

Explore More Data & Analytics Skills→
1molchuan1molchuan
1

oh-my-ccg-init

Verify MCP, Codex, and Gemini endpoints and initialize RPI state.

Community
Intermediate
blankuzrblankuzr
6

r-eda

Execute end-to-end exploratory data analysis workflows in R on raw datasets.

Community
Advanced
softmgsoftmg

github-workflow-automation

Automate GitHub Actions workflow generation and CI/CD pipeline management.

Official
Advanced
jeninhjeninh

scripting-with-go

Execute Go scripts directly as standalone programs with go run.

Community
Advanced