skills.rest
AubreyHan
AubreyHanCOMMUNITY
Explore all skills by this author →
GitHub

gcp-data-pipelines

Route Google Cloud data pipeline requests to the correct build, deployment, or orchestration skill.

Updated Jun 10, 2026
Search Tags:#gcp#dbt#data pipelines#spark#dataflow#dataform#cloud composer
One-click install
npx skills add https://github.com/AubreyHan/SKILL_Repo --skill gcp-data-pipelines-aubreyhan
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: gcp-data-pipelines
Source: https://github.com/AubreyHan/SKILL_Repo/tree/main/gcp-data-pipelines
Command: npx skills add https://github.com/AubreyHan/SKILL_Repo --skill gcp-data-pipelines-aubreyhan

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps users identify, choose, and proceed with the correct Google Cloud data pipeline approach when a repository may contain dbt, Dataflow, Dataform, Spark, BigQuery DTS, or orchestration assets. It reduces ambiguity by scanning the workspace first and steering users toward the most relevant workflow instead of making assumptions.

Core Features & Use Cases

  • Pipeline detection: Detects existing pipeline indicators such as dbt project files, Dataform configuration, Spark notebooks or PySpark code, and orchestration or deployment manifests.
  • Tool selection guidance: Recommends the best-fit GCP pipeline skill for ingestion, transformation, batch or streaming processing, provisioning, or orchestration.
  • Clarification workflow: Handles ambiguous or multi-pipeline repositories by asking the right follow-up question before any implementation begins.
  • Use case: A user wants to run or update a data workflow in a mixed repository, and this Skill determines whether to use dbt, Dataflow, Dataform, Spark, Cloud Composer, or provisioning guidance.

Quick Start

Ask the skill to inspect the repository and tell you which Google Cloud data pipeline tool should be used for the current workspace and request.

Frequently Asked Questions about gcp-data-pipelines

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I choose the right GCP data pipeline tool for my repository?▼

GCP data pipeline routing requires scanning the workspace for dbt, Dataform, Spark, or orchestration manifests to match the request to the correct build, deployment, or provisioning workflow.

What is the best way to handle mixed data pipeline assets in a GCP repository?▼

Handling mixed GCP pipeline assets requires a clarification workflow that asks follow-up questions before execution, steering users toward the most relevant workflow instead of making assumptions.

Does this approach support both dbt transformations and Cloud Composer orchestration?▼

Yes, the routing approach supports dbt transformations, Dataflow, Dataform, Spark, BigQuery DTS, provisioning, and Cloud Composer workflows by detecting existing assets and confirming the stepwise execution path.

How do I detect which data pipeline framework is already configured in my workspace?▼

Pipeline detection scans the workspace for dbt project files, Dataform configuration, Spark notebooks, PySpark code, and deployment manifests to identify existing frameworks before recommending a tool.

Can I route a streaming data processing request to Dataflow from an ambiguous repository?▼

Yes, ambiguous repositories are resolved through a clarification workflow that identifies whether Dataflow, Spark, or another GCP pipeline skill fits the streaming or batch processing request.

What are the limitations of routing Google Cloud data pipeline requests automatically?▼

Routing limitations include dependency on accurate workspace scanning and pipeline detection; ambiguous or multi-pipeline repositories require manual stepwise confirmation before any implementation begins.

Related Skills

Explore More Data & Analytics Skills→
AubreyHanAubreyHan

gcp-pipeline-orchestration

Generate and update Google Cloud Composer orchestration YAML for data pipelines.

Community
Advanced
erichowenserichowens
181

data-pipeline-engineer

Automate ETL/ELT pipeline design, validation, and operation across medallion architectures.

Community
Advanced
gemini-cli-extensionsgemini-cli-extensions
161

gcp-data-pipelines

Guide selection and implementation of Google Cloud data pipelines with dbt, Dataflow, Dataform, Spark, BigQuery DTS, and Cloud Composer.

Official
Advanced
RxFitRxFit

gcp-data-pipelines

Detect existing Google Cloud data pipelines and recommend suitable tools.

Community
Advanced