skills.rest
KaranKathur06
KaranKathur06COMMUNITY
Explore all skills by this author →
GitHub

data-pipeline

Build scheduled ETL workflows that extract, transform, and load data into analytics destinations.

Updated Feb 22, 2026
Search Tags:#automation#monitoring#data quality#data pipeline#scheduling#etl#analytics integration
One-click install
npx skills add https://github.com/KaranKathur06/Metal-Hub --skill data-pipeline-karankathur06
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-pipeline
Source: https://github.com/KaranKathur06/Metal-Hub/tree/main/.cursor/skills/data-pipeline
Command: npx skills add https://github.com/KaranKathur06/Metal-Hub --skill data-pipeline-karankathur06

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Data teams waste time wiring together extraction, transformation, loading, and repeatable schedules for analytics, which makes integrations brittle and reporting late.

Core Features & Use Cases

  • ETL pipeline design: define Extract → Transform → Load flows across databases, APIs, and files.
  • Transformation & data quality: apply cleaning, mapping, enrichment, aggregation, and validation checks.
  • Scheduling, monitoring, and alerts: run pipelines on cron/webhooks, track metrics, and trigger failure/data-quality notifications.
  • Real-world use case: combine daily sales from Shopify and Stripe, transform fields into a normalized schema, and load into BigQuery and Google Sheets on a fixed schedule.

Quick Start

Use the data-pipeline skill to create a daily ETL workflow that extracts Shopify orders and Stripe payments, transforms them into a sales_daily schema, and loads results into BigQuery and Google Sheets.

Frequently Asked Questions about data-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate ETL workflows to load data from APIs and databases into an analytics warehouse?▼

Automate ETL workflows by defining extraction from heterogeneous sources, transforming data into normalized datasets, and loading it into analytics destinations. This builds end-to-end pipelines that eliminate manual integration and ensure reporting is never delayed.

How do I schedule a data pipeline to run daily sales reporting from Shopify and Stripe?▼

Schedule data pipelines using cron or webhook triggers to run on a fixed schedule. You can extract daily sales from Shopify and Stripe, transform fields into a normalized schema, and load results into BigQuery and Google Sheets automatically.

How do I enforce data quality and validation checks during an ETL pipeline?▼

Enforce data quality during ETL by applying cleaning, mapping, enrichment, aggregation, and validation checks within the transformation stage. This ensures datasets remain normalized and reliable before loading into analytics destinations.

Can I trigger data quality alerts when a scheduled ETL pipeline fails or finds invalid records?▼

You can configure monitoring and alerts to trigger failure and data-quality notifications during pipeline execution. This applies to scheduled or event-driven scenarios, allowing you to track metrics and catch invalid records immediately.

What is the best way to normalize data from spreadsheets and databases before loading into BigQuery?▼

The best way to normalize data is using configurable extract and transform mappings that apply cleaning and aggregation across warehouses, databases, and spreadsheets. This creates normalized datasets ready for loading into BigQuery.

Related Skills

Explore More Data & Analytics Skills→
urivauriva

google-docs

Perform CRUD operations on Google Docs and Sheets via APIs.

Community
Intermediate
akillnessakillness
40

data-analysis

Analyze CSV, JSON, and SQL datasets with Python pandas workflows.

Community
Intermediate
afel6afel6

webhook-subscriptions

Create and manage webhook subscriptions that trigger Hermes agent runs from external POSTed events.

Community
Advanced
CODE-SAURABHCODE-SAURABH
2

data-engineer

Design reliable data pipelines with idempotent processing and schema validation.

Community
Advanced