github-actions-dataops

What problem does it solve?

This Skill solves the problem of reliably building, linting, validating, testing, and deploying data engineering workflows using GitHub Actions with strong DataOps/CI/CD guardrails.

Core Features & Use Cases

• dbt slim CI/CD: Runs only changed models using state:modified+, --defer, and manifest.json artifact passing to keep CI fast while preserving correctness through downstream awareness.
• Quality gates in the pipeline: Integrates SQLFluff linting with PR annotations, plus Great Expectations and Soda checks as CI blockers.
• Airflow integrity testing: Validates DAG structure and prevents breaking changes via pytest/DagBag import and structural tests.
• Secure authentication and secrets: Uses OIDC for AWS/GCP with environment-scoped secrets (no static long-lived credentials).
• Scalable workflow patterns: Supports reusable workflows (workflow_call), matrix builds, and composite actions, plus multi-stage Docker builds and pushing to ghcr.io.

Quick Start

Use the github-actions-dataops skill to generate a complete end-to-end GitHub Actions workflow for dbt CI/CD that includes SQLFluff linting, dbt slim CI with prod manifest/defer, Airflow DAG validation, and Great Expectations/Soda quality gates.