skills.rest
claude-office-skills
claude-office-skillsOFFICIAL
Explore all skills by this author →
GitHub

ETL Pipeline

Design and automate ETL pipelines from SQL, NoSQL, APIs, and files to BigQuery and Snowflake.

368|75|Updated Jan 29, 2026
Search Tags:#automation#data pipeline#etl#data integration#bigquery#snowflake#data warehousing
One-click install
npx skills add https://github.com/claude-office-skills/skills --skill etl-pipeline
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ETL Pipeline
Source: https://github.com/claude-office-skills/skills/tree/main/etl-pipeline
Command: npx skills add https://github.com/claude-office-skills/skills --skill etl-pipeline

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill automates the complex process of moving data from various sources, transforming it into a usable format, and loading it into target systems, enabling efficient data integration and analytics.

Core Features & Use Cases

  • Data Extraction: Connects to diverse sources like SQL databases (Postgres, MySQL), NoSQL databases (MongoDB), APIs (Stripe, Salesforce), and file formats (CSV, JSON).
  • Data Transformation: Cleans, validates, normalizes, aggregates, and joins data according to defined business rules.
  • Data Loading: Loads transformed data into data warehouses and data lakes such as BigQuery and Snowflake.
  • Pipeline Orchestration: Defines and schedules complex data workflows with dependencies and error handling.
  • Use Case: Automatically extract daily sales data from a transactional PostgreSQL database, transform it to calculate daily revenue by product category, and load it into a BigQuery data warehouse for business intelligence reporting.

Quick Start

Design an ETL pipeline to extract data from a PostgreSQL database, transform it by joining with customer data, and load it into BigQuery.

Frequently Asked Questions about ETL Pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate a data pipeline to extract from PostgreSQL and load into BigQuery?▼

To automate a data pipeline, this Skill designs ETL workflows that extract data from PostgreSQL sources, apply transformations like joining customer records, and load the results directly into BigQuery for analytics.

What sources can I connect to for data integration and pipeline automation?▼

For data integration, you can connect to SQL databases like PostgreSQL and MySQL, NoSQL databases like MongoDB, APIs such as Stripe and Salesforce, and file formats including CSV and JSON.

Can I use this to orchestrate ETL workflows with error handling and scheduling?▼

Yes, you can orchestrate complex ETL workflows with scheduling, manage task dependencies, and implement error handling to ensure robust data integration and automated pipeline execution.

How do I transform raw data before loading it into a data warehouse?▼

You transform raw data by applying rules to clean, validate, normalize, aggregate, and join datasets before loading them into target data warehouses like BigQuery or Snowflake.

Does this approach work for loading data into Snowflake as well as BigQuery?▼

Yes, this ETL pipeline approach supports loading transformed data into both Snowflake and BigQuery, enabling seamless data warehousing and business intelligence reporting across platforms.

What is the best way to handle data quality checks during ETL pipeline automation?▼

The best way to handle data quality is by orchestration workflows that include built-in validation checks, ensuring data is cleaned and validated during the transformation step before final loading.

Related Skills

Explore More Data & Analytics Skills→
PhucMPhamPhucMPham

backend-development

Design production-ready backend systems across Node.js, Python, Go, and Rust.

Community
Advanced
nirajrathod19nirajrathod19

database

Manage Replit PostgreSQL databases and execute SQL queries across data warehouses.

Community
Advanced
SpiralCloudOmegaSpiralCloudOmega
1

Multi-Agent Coordinator

Coordinate multiple AI agents through decompose, assign, schedule, dispatch, merge, and verify loops.

Community
Advanced
sandeepmvlsandeepmvl
21

data-warehouse-integration

Design ELT pipelines syncing Rails Postgres to cloud warehouses with dbt.

Community
Advanced