Skills
.Work. You
Rest

dataops-airflow-ha-review

Community

Harden Airflow for zero-schedule-failure uptime

Software Engineering#postgresql#kubernetes#high availability#airflow#pgbouncer#scheduler heartbeat#remote logging
Authorivanshamaev
Version1.0.0
Installs0

System Documentation

What problem does it solve?

This Skill helps you review and harden an Apache Airflow deployment to eliminate single points of failure and prevent DAG execution pauses caused by scheduler, metadata DB, logging, or executor misconfiguration.

Core Features & Use Cases

  • Scheduler HA review (multi-scheduler): Validates multi-scheduler behavior using PostgreSQL row-level locking, heartbeat and health-check thresholds to detect scheduler hangs early.
  • Metadata DB HA and pooling: Recommends PostgreSQL HA patterns (e.g., Multi-AZ) and connection pooling with PgBouncer to sustain concurrency and reduce connection storms.
  • Production operational readiness: Covers DAG sync strategy (git-sync/S3 bundle), remote log storage (S3/GCS), webserver/triggerer HA, worker sizing for CeleryExecutor or KubernetesExecutor, and upgrade/rollout procedures aimed at minimizing downtime.

Quick Start

Ask an AI to review your Airflow Helm/Kubernetes configuration for HA gaps by checking scheduler replicas, PostgreSQL + PgBouncer metadata DB setup, remote logging, DAG sync, and your upgrade plan.

Dependency Matrix

Required Modules

None required

Components

Standard package

💻 Claude Code Installation

Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.

Please help me install this Skill:
Name: dataops-airflow-ha-review
Download link: https://github.com/ivanshamaev/de-agent-skills/archive/main.zip#dataops-airflow-ha-review

Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
View Source Repository

Agent Skills Search Helper

Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.