skills.rest
slysik
slysikCOMMUNITY
Explore all skills by this author →
GitHub

databricks-jobs

Manage Databricks Jobs for multi-task DAGs and scheduling triggers.

Updated Mar 10, 2026
Search Tags:#workflow#monitoring#orchestration#scheduling#etl#databricks#jobs
One-click install
npx skills add https://github.com/slysik/databricks-claude-coding --skill databricks-jobs-slysik
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: databricks-jobs
Source: https://github.com/slysik/databricks-claude-coding/tree/main/.agents/skills/databricks-jobs
Command: npx skills add https://github.com/slysik/databricks-claude-coding --skill databricks-jobs-slysik

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill simplifies the creation, management, and monitoring of complex data workflows on Databricks, reducing manual effort and potential errors.

Core Features & Use Cases

  • Workflow Orchestration: Define multi-task Directed Acyclic Graphs (DAGs) with task dependencies.
  • Flexible Scheduling: Set up cron-based schedules, periodic triggers, or event-driven executions (file arrival, table updates).
  • Monitoring & Notifications: Configure email and webhook alerts, health rules, and retry policies for robust job management.
  • Use Case: Automate a daily ETL pipeline that extracts data, transforms it, loads it into a data warehouse, and then triggers a BI dashboard refresh, with notifications sent on success or failure.

Quick Start

Create a new Databricks job named 'my-etl-job' that runs a notebook located at '/Workspace/Users/[email protected]/extract'.

Frequently Asked Questions about databricks-jobs

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I orchestrate multi-task ETL workflows in Databricks?▼

Databricks workflow orchestration supports multi-task Directed Acyclic Graphs (DAGs) with dependencies, handling notebook, Python, SQL, and dbt task types. You can configure these via Python SDK, CLI, or Asset Bundles.

Can I schedule Databricks jobs with cron or trigger them based on file arrival?▼

Yes, Databricks jobs support flexible scheduling including cron-based schedules, periodic triggers, and event-driven executions triggered by file arrival or table updates.

How do I set up notifications and retries for failed Databricks job tasks?▼

Databricks job monitoring allows you to configure email and webhook alerts, define health rules, and establish retry policies to ensure robust job management and automated failure recovery.

Does this Databricks jobs configuration support dbt and SQL task types?▼

Yes, Databricks jobs orchestration supports diverse task types including dbt, SQL, Python scripts, and notebooks within your multi-task DAGs.

What is the best way to configure compute and parameters for Databricks job workflows?▼

Configuring Databricks job workflows is best handled through Python SDK, CLI, or Asset Bundles, allowing you to explicitly define compute resources, pass parameters, and manage health monitoring.

Related Skills

Explore More Software Engineering Skills→
rywerth-dbxrywerth-dbx

databricks-job-orchestrator

Automate creation, execution, and monitoring of Databricks jobs via CLI.

Community
Advanced
RichJacobs69RichJacobs69

gha-analyzer

Analyze GitHub Actions workflow runs to debug failures and optimize batch timing.

Community
Advanced
ThomazRossitoThomazRossito
31

databricks-dbsql

Extend Databricks SQL with procedural scripts, stored procedures, and recursive queries.

Community
Advanced
p-o-ke-naep-o-ke-nae

airflow-dag-patterns

Build production Apache Airflow DAGs with reusable operator, sensor, and branching patterns.

Community
Advanced