skills.rest
ricardolui
ricardoluiCOMMUNITY
Explore all skills by this author →
GitHub

gcp-managed-airflow-migrations

Migrates Apache Airflow DAGs to Airflow 2.11.1 or Airflow 3 on Google Cloud Managed Service for Apache Airflow.

Updated Jul 7, 2026
Search Tags:#migration#gcp#airflow#dag#msaa#cloud composer#airflow 3
One-click install
npx skills add https://github.com/ricardolui/gcp-custom-agent-skills --skill gcp-managed-airflow-migrations-ricardolui
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: gcp-managed-airflow-migrations
Source: https://github.com/ricardolui/gcp-custom-agent-skills/tree/main/gcp-managed-airflow-migrations
Command: npx skills add https://github.com/ricardolui/gcp-custom-agent-skills --skill gcp-managed-airflow-migrations-ricardolui

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Upgrading Apache Airflow DAGs to Airflow 2.11.1 or Airflow 3 on Google Cloud Managed Service for Apache Airflow (formerly Cloud Composer) involves many breaking changes across core Airflow and provider packages, and missing any of them causes DAG import failures or runtime errors after deployment. ## Core Features & Use Cases - Dependency Mapping: Traces version progression of Google, SSH, and HTTP providers across Composer releases to identify every breaking change between your source and target versions. - Automated Scanning & Remediation: Provides grep-based scan commands to detect deprecated operators, imports, context variables, SubDAGs, SLAs, and Dataset usage, with concrete remediation steps for each finding. - Deployment & Verification: Guides GCS upload of migrated DAGs, Airflow CLI verification of parsing and import errors, Cloud Logging checks, and local testing with the composer-dev CLI. - Use Case: You have DAGs running on Composer 2.10.0 (Airflow 2.10.2) and need them on Airflow 3. The skill walks you through downloading the DAGs, scanning for removed operators like BigQueryExecuteQueryOperator, rewriting imports to airflow.sdk, and verifying the result in the target environment. ## Quick Start Migrate my Airflow DAGs from Composer 2.10.0 to Airflow 3 on MSAA Gen 3, scanning for breaking changes and fixing deprecated operators.

Frequently Asked Questions about gcp-managed-airflow-migrations

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I migrate Airflow DAGs from Cloud Composer to Airflow 3?▼

Download the DAGs from the source environment's GCS bucket, scan them for breaking changes using the provided grep patterns, and rewrite imports to airflow.sdk while replacing removed features like SubDAGs and SLAs. Then upload to the target bucket and verify with the Airflow CLI dags list and list-import-errors commands.

What breaking changes occur when upgrading to Airflow 2.11.1 on Composer?▼

Key changes include SSH provider 5.0.0 removing sshtunnel, HTTP provider 6.0.0 changing JSON serialization, and Google provider 20.0.0 migrating ADLS Gen2. Earlier removals include BigQueryExecuteQueryOperator, AutoML operators, and SimpleHttpOperator, each with documented replacements.

How do I replace BigQueryExecuteQueryOperator in Airflow?▼

Replace BigQueryExecuteQueryOperator with BigQueryInsertJobOperator, passing the query through a configuration dictionary. This operator was removed in Google provider 11.0.0, so the change is required when upgrading to Airflow 2.11.1 or Airflow 3.

Does Airflow 3 still support SubDAGs and SLAs?▼

No, Airflow 3 removes SubDagOperator and the sla parameter entirely. Refactor SubDAGs into nested TaskGroups and delete sla from operator calls and DAG default_args before migrating.

How do I verify migrated DAGs after uploading to Cloud Composer?▼

Run gcloud composer environments run with dags list to confirm the DAGs are registered, and dags list-import-errors to detect parsing failures. You can also query Cloud Logging for airflow-scheduler errors at severity ERROR or higher.

Can I test Airflow 3 DAG changes locally before deploying?▼

Yes, use the Composer Local Development CLI tool composer-dev. Copy migrated DAGs into the local environment's DAGs directory, then run composer-dev run-airflow-cmd with dags list and list-import-errors to validate parsing.

Related Skills

Explore More Software Engineering Skills→
jerrylin96jerrylin96
9

gcp-managed-airflow-migrations

Guides Airflow 3 migrations and Cloud Composer environment inspection on GCP.

Community
Basic
tottenjordantottenjordan
1

gcp-composer-troubleshooting

Diagnoses Cloud Composer and Airflow DAG failures using gcloud logs and remote code for root cause analysis.

Community
Intermediate
googlegoogle
19.1k

managed-airflow-migrations

Migrates Apache Airflow DAGs to Airflow 2.11.1 or Airflow 3 on Managed Service for Apache Airflow.

Official
Advanced
cuoreinpacecuoreinpace

ml-pipeline-workflow

Build end-to-end MLOps pipelines from data preparation to deployment.

Community
Advanced