dataops-root-cause-analysis

Diagnoses DataOps incidents by analyzing pipeline failures and data quality anomalies.

14|1|Updated May 5, 2026

Search Tags:#root cause analysis #data quality #airflow #kafka #dataops #spark #incident timeline

Ready to run

npx skills add https://github.com/ivanshamaev/de-agent-skills --skill dataops-root-cause-analysis

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: dataops-root-cause-analysis
Source: https://github.com/ivanshamaev/de-agent-skills/tree/main/group_skills/infra_dataops_group_skills/dataops_root_cause_analysis
Command: npx skills add https://github.com/ivanshamaev/de-agent-skills --skill dataops-root-cause-analysis

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps teams quickly identify the true root cause of DataOps incidents across pipeline failures, data quality anomalies, and downstream operational impact, instead of repeatedly applying superficial fixes.

Core Features & Use Cases

Failure taxonomy + structured RCA workflow: Classify incidents by infrastructure, data, logic, dependencies, configuration, and concurrency, then drive investigation in that order.
Airflow, Spark, and Kafka diagnosis playbooks: Triage task states, scheduler health, retry patterns, Spark OOM/skew/FetchFailed/serialization issues, and Kafka consumer lag spikes with targeted checks.
Data quality anomaly investigation + timeline reconstruction: Detect volume/freshness/distribution anomalies (e.g., z-scores, null spikes) and rebuild an incident timeline across components for postmortems.

Quick Start

Ask the AI to perform a root cause analysis for an Airflow DAG failure on your incident date and to produce the most likely failure taxonomy path with the exact SQL and command checks to run.

Frequently Asked Questions about dataops-root-cause-analysis

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I perform root cause analysis for Airflow DAG failures?▼

Root cause analysis for Airflow DAG failures involves classifying incidents by infrastructure, data, logic, dependencies, configuration, and concurrency, then triaging task states, scheduler health, and retry patterns with targeted diagnostic queries.

Why does my Spark job fail with OOM or FetchFailed errors?▼

Spark job failures like OOM, data skew, FetchFailed, and serialization issues require targeted diagnostic checks within a structured failure taxonomy to pinpoint whether the root cause is infrastructure, logic, or concurrency-related.

How do I investigate Kafka consumer lag spikes and data quality anomalies?▼

Investigating Kafka consumer lag spikes and data quality anomalies requires quantifying distribution anomalies like z-scores and null spikes, then reconstructing the incident timeline across pipeline components for synthesis.

What is the best way to structure a DataOps incident postmortem?▼

The best way to structure a DataOps incident postmortem is to apply a 5-Why synthesis template, reconstruct the incident timeline across components, and classify the failure to drive investigation systematically rather than applying superficial fixes.

Can I diagnose data pipeline outages without prior failure taxonomy knowledge?▼

Yes, diagnosing data pipeline outages is possible by following a structured RCA workflow that classifies incidents across six taxonomy paths and provides component-specific diagnostic commands to run directly.

When should I not use a structured RCA workflow for data pipeline issues?▼

A structured RCA workflow is not ideal for isolated, immediately obvious syntax errors, but is necessary for complex DataOps incidents involving data quality anomalies, cross-component dependencies, or repeated superficial fixes.