skills.rest
gemini-cli-extensions
gemini-cli-extensionsOFFICIAL
Explore all skills by this author →
GitHub

gcp-pipeline-orchestration

Automate Cloud Composer pipeline creation and management with gcloud validation.

161|37|Updated Apr 2, 2026
Search Tags:#orchestration#dbt#gcloud#notebooks#spark#data-pipelines#cloud-composer
One-click install
npx skills add https://github.com/gemini-cli-extensions/data-agent-kit-starter-pack --skill gcp-pipeline-orchestration
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: gcp-pipeline-orchestration
Source: https://github.com/gemini-cli-extensions/data-agent-kit-starter-pack/tree/main/skills/gcp-pipeline-orchestration
Command: npx skills add https://github.com/gemini-cli-extensions/data-agent-kit-starter-pack --skill gcp-pipeline-orchestration

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires google-auth, requests, google-auth-transport-requests, and includes scripts (resource) and references (resource) components.

What problem does it solve?

Automates the creation and management of Cloud Composer orchestration pipelines.

Core Features & Use Cases

  • Generate orchestration YAML and deployment.yaml for Cloud Composer-based pipelines
  • Validate configurations with gcloud beta orchestration-pipelines validate
  • Deploy, trigger, and monitor pipelines across environments (dev, prod)

Quick Start

Create a new orchestration pipeline named 'orchestration_pipeline' in the dev environment and generate the required deployment.yaml and orchestration YAML, then validate and prepare for deployment.

Frequently Asked Questions about gcp-pipeline-orchestration

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate Cloud Composer orchestration pipelines for dbt and Spark workflows?▼

Cloud Composer orchestration pipelines are generated by creating standardized deployment.yaml and orchestration YAML files. This process automates data workflows involving dbt, notebooks, Spark, Dataform, and Python scripts across environments.

What is the required YAML structure for Cloud Composer pipeline definitions?▼

Cloud Composer pipeline definitions require a mandatory YAML structure using camelCase for top-level fields. All schedules must enforce an endTime, and configurations are validated using gcloud beta orchestration-pipelines before deployment.

How do I validate and deploy orchestration pipelines across dev and prod environments?▼

Orchestration pipelines are validated using gcloud beta orchestration-pipelines validate. After validation, you can deploy, trigger, and monitor the pipelines across different environments like dev and prod.

Can I use this to manage data pipelines that include Dataform and Python scripts?▼

Yes, Cloud Composer orchestration pipelines support data workflows that include Dataform, Python scripts, Spark, dbt, and notebooks. The generated YAML handles configuration and deployment for these components.

What is the best way to handle placeholders in Cloud Composer deployment.yaml files?▼

Cloud Composer pipeline generation handles placeholders within the deployment.yaml and orchestration YAML structures. It guides deployment and triggering steps while ensuring configurations meet gcloud validation requirements.

Why does my orchestration pipeline validation fail when scheduling data workflows?▼

Orchestration pipeline validation fails if mandatory YAML structures are incorrect, such as missing endTime on schedules or not using camelCase for top-level fields. Configurations must pass gcloud beta orchestration-pipelines validate checks.

Related Skills

Explore More Data & Analytics Skills→
RxFitRxFit

gcloud-auth-verification

Check gcloud authentication status and provide login commands for CLI and ADC setup.

Community
Basic
briannadoubtbriannadoubt

observability

Configure structured logs, metrics, and traces for Trebuchet actors with CloudWatch export.

Community
Intermediate
RxFitRxFit

gcp-pipeline-orchestration

Generate and deploy Cloud Composer orchestration pipeline definitions for GCP workflows.

Community
Advanced
RxFitRxFit

gcp-pipeline-resource-provisioning

Provision GCP data pipeline resources from deployment.yaml configurations.

Community
Advanced