skills.rest
google
googleOFFICIAL
Explore all skills by this author →
Website𝕏@GoogleOSSGitHub

managed-airflow-migrations

Migrates Apache Airflow DAGs to Airflow 2.11.1 or Airflow 3 on Managed Service for Apache Airflow.

19.1k|1.5k|Updated Mar 31, 2026
Search Tags:#Airflow#Google Cloud#DAG#Migration#Cloud Composer#Breaking Changes#Data engineering
One-click install
npx skills add https://github.com/google/skills --skill managed-airflow-migrations
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: managed-airflow-migrations
Source: https://github.com/google/skills/tree/main/skills/cloud/managed-airflow-migrations
Command: npx skills add https://github.com/google/skills --skill managed-airflow-migrations

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Upgrading Apache Airflow DAGs to newer versions on Managed Service for Apache Airflow (formerly Cloud Composer) involves tracking dozens of breaking changes across core Airflow and provider packages, and missing one causes DAG import failures after deployment.

Core Features & Use Cases

  • Dependency Mapping: Traces breaking changes across Composer versions and Google, SSH, and HTTP provider releases up to Airflow 2.11.1 and Airflow 3.
  • Scan-and-Remediate Workflow: Provides grep-based scan commands to detect deprecated operators, imports, context variables, SubDAGs, and SLAs, paired with concrete remediation steps.
  • Deployment & Verification: Covers GCS upload of migrated DAGs, Airflow CLI import-error checks, Cloud Logging queries, and local verification with the composer-dev CLI.
  • Use Case: A data engineer migrating a Composer 2.10 environment to Airflow 3 scans their DAGs, replaces removed operators like BigQueryExecuteQueryOperator, updates imports to airflow.sdk, and verifies the DAGs parse without import errors.

Quick Start

Migrate my Airflow DAGs in this workspace to Airflow 3 on Managed Service for Apache Airflow and scan them for breaking changes.

Frequently Asked Questions about managed-airflow-migrations

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I migrate Airflow DAGs to Airflow 3 on Cloud Composer?▼

Download the DAGs from the source environment's GCS bucket, scan them for breaking changes such as removed operators and deprecated context variables, apply the remediations, then upload to the target bucket. Verify with the Airflow CLI using dags list and dags list-import-errors.

What breaking changes occur when upgrading to Airflow 2.11.1?▼

Key changes include SimpleHttpOperator replaced by HttpOperator, BigQueryExecuteQueryOperator removed in favor of BigQueryInsertJobOperator, SSHHook timeout renamed to conn_timeout, and Pendulum Period renamed to Interval. Dataset-scheduled DAGs also only trigger while unpaused.

How do I replace removed BigQuery operators in Airflow?▼

Replace BigQueryExecuteQueryOperator with BigQueryInsertJobOperator using a configuration dict, and replace BigQueryCreateEmptyTableOperator or BigQueryCreateExternalTableOperator with BigQueryCreateTableOperator using a table_resource dict.

Does Airflow 3 still support SubDAGs and SLA parameters?▼

No, Airflow 3 removes SubDagOperator and the sla parameter entirely. Refactor SubDAGs into nested TaskGroups and delete sla from operator calls and DAG default_args.

Why do my DAGs show import errors after migrating to Airflow 3?▼

Import errors usually come from outdated imports such as airflow.models.dag or airflow.operators.bash, which moved to airflow.sdk and the standard provider bundle. Run dags list-import-errors via the Airflow CLI or ruff check --select AIR30 to locate them.

When should I not use this migration workflow?▼

Do not use it when troubleshooting DAG run failures unrelated to code migration, such as runtime or infrastructure issues. It is scoped to upgrading DAG code compatibility for Airflow 2.11.1 and Airflow 3.

Related Skills

Explore More Data & Analytics Skills→
RealEmmettSRealEmmettS

shaughv-gcs-storage

Automate file upload, download, and management in a Google Cloud Storage bucket.

Community
Advanced
miptah21miptah21
2

airflow

Manage Apache Airflow DAGs, runs, and task logs via the af CLI.

Community
Advanced
astronomerastronomer
419

migrating-airflow-2-to-3

Migrate Airflow 2.x DAGs and code to Airflow 3.x with rule-based fixes.

Official
Advanced
wshobsonwshobson
38.6k

ml-pipeline-workflow

Build and orchestrate end-to-end MLOps pipelines for machine learning projects.

Community
Advanced