workflow-management

Debugging a failed TD workflow involves checking session logs with `digdag log <session_id>` to identify the root cause, examining task-level errors, and reviewing slow queries or resource issues. The Skill guides you through log analysis, common failure patterns, and targeted fixes to restore your pipeline quickly.

Monitor Digdag pipelines by tracking session status, implementing alerting patterns via Slack or email on failures, and setting up data quality checks across projects. This Skill provides pre-built notification templates and monitoring patterns to catch issues before they impact downstream processes.

Retry and backfill patterns in TD workflows let you rerun failed sessions or fill historical data gaps with controlled execution. This Skill covers structured retry logic, backfill strategies, and session management to ensure reliable recovery without duplicating valid data.

Yes, data quality validation integrates into Digdag pipelines through SQL queries and predefined checks within workflow definitions. This Skill provides patterns for embedding quality gates that flag anomalies, validate row counts, and enforce schema constraints across pipeline stages.

Slow ETL workflows often stem from unoptimized queries, resource contention, or inefficient task sequencing. This Skill offers strategies for identifying bottlenecks in logs, optimizing query performance, managing resource limits, and improving overall pipeline execution speed to reduce costs and time.