skills.rest
JingyiWng
JingyiWngCOMMUNITY
Explore all skills by this author →
GitHub

databricks-jobs

Manage Databricks Jobs lifecycle via Python SDK, CLI, and Asset Bundles.

Updated Mar 25, 2026
Search Tags:#automation#monitoring#orchestration#databricks#jobs#scheduler
One-click install
npx skills add https://github.com/JingyiWng/databricks_ai_dev_kit_price_watcher --skill databricks-jobs-jingyiwng
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: databricks-jobs
Source: https://github.com/JingyiWng/databricks_ai_dev_kit_price_watcher/tree/main/.claude/skills/databricks-jobs
Command: npx skills add https://github.com/JingyiWng/databricks_ai_dev_kit_price_watcher --skill databricks-jobs-jingyiwng

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Managing Databricks Jobs involves creating complex DAGs, handling schedules, triggers, cluster configs, and monitoring, which can be error‑prone and time‑consuming for data teams.

Core Features & Use Cases

  • Job Lifecycle Management: Create, list, run, update, and delete jobs via CLI, Python SDK, or Asset Bundles.
  • Advanced Scheduling & Triggers: Define cron schedules, periodic, file‑arrival, table‑update, and continuous jobs.
  • Monitoring & Notifications: Configure email/webhook alerts, health rules, timeout, retries, and concurrency controls.
  • Cluster & Parameter Configuration: Specify job clusters, autoscaling, serverless options, and job‑level parameters.

Quick Start

Ask the assistant to create a Databricks job that runs a notebook at “/Workspace/Users/me/etl” every day at 6 AM.

Frequently Asked Questions about databricks-jobs

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create and schedule Databricks Jobs using Python SDK or CLI?▼

Yes, Databricks Jobs supports event-driven triggers such as file-arrival and table-update events. You can configure these triggers programmatically alongside cron schedules to orchestrate complex multi-task DAGs based on real-time data changes.

How do I configure monitoring and notifications for Databricks Jobs?▼

Configuring monitoring for Databricks Jobs involves setting up email or webhook alerts, defining health rules, and applying timeout and retry policies. You can also manage concurrency controls to ensure reliable job execution and immediate failure notifications.

Can I manage Databricks Job clusters and autoscaling configurations programmatically?▼

You can manage Databricks Job clusters and autoscaling configurations programmatically by specifying job clusters and serverless options during job creation. This integration handles cluster configurations and job-level parameters efficiently via the Python SDK or Asset Bundles.

What is the best way to orchestrate multi-task DAGs in Databricks?▼

Yes, you can update and delete Databricks Jobs programmatically. The Skill handles full job lifecycle management, allowing you to modify existing DAGs, adjust cluster configurations, and remove obsolete jobs via the Python SDK, CLI, or Asset Bundles.

Related Skills

Explore More Data & Analytics Skills→
rbutinarrbutinar

databricks-lineage

Extract orchestration lineage from Databricks jobs and notebooks.

Community
Intermediate
0xmariowu0xmariowu
4

verification-before-completion

Enforces verification evidence such as command output and logs before task completion in automated workflows, code reviews, or deployment processes.

Community
Intermediate
ScottHMcKeanScottHMcKean

databricks-unity-catalog

Query Databricks Unity Catalog system tables and manage volume files.

Community
Intermediate
AradotsoAradotso
5

apache-airflow-orchestration

Automate data pipeline scheduling and monitoring with Apache Airflow DAGs.

Official
Advanced