Create, schedule, and monitor Databricks Jobs using Python SDK, CLI, or Asset Bundles.
npx skills add https://github.com/FMurray/mlfts --skill databricks-jobs-fmurray
Please help me install this Agent Skill. Skill: databricks-jobs Source: https://github.com/FMurray/mlfts/tree/main/.agents/skills/databricks-jobs Command: npx skills add https://github.com/FMurray/mlfts --skill databricks-jobs-fmurray
depends_on dependencies and conditional run_if logic across notebook, Python, SQL, dbt, pipeline, JAR, run-job, and for-each task types. - Triggers & Scheduling: Configure cron schedules, periodic intervals, file arrival triggers, table update triggers, and continuous jobs. - Monitoring & Reliability: Set up email and webhook notifications, health rules, timeouts, retries, and run queue settings. - Use Case: You need a daily ETL pipeline that extracts from three sources, transforms the data, loads a warehouse, and alerts the on-call team on failure—deployed to dev, staging, and prod via Asset Bundles. This Skill provides the complete YAML and SDK code for exactly that. ## Quick Start Ask the agent to create a Databricks job that runs your notebook on a daily cron schedule with failure email notifications.High-intent search queries and answers about installing and using this skill.
Define tasks with unique task_key values and link them using depends_on to form a DAG. You can configure this via the Python SDK w.jobs.create(), the CLI with JSON, or Asset Bundles YAML, and control execution with run_if conditions like ALL_SUCCESS.
Use a schedule block with a quartz_cron_expression and timezone_id for cron-based runs, or a trigger block for periodic intervals, file arrival, or table update events. Set pause_status to UNPAUSED to activate the schedule.
Databricks Jobs support notebook, Spark Python, Python wheel, SQL, dbt, pipeline, Spark JAR, run-job, and for-each task types. Each type has specific required parameters, such as notebook_path for notebooks or warehouse_id for SQL tasks.
Yes, use a file_arrival trigger with the cloud storage URL, such as s3://bucket/path/. Optional settings like min_time_between_triggers_seconds and wait_after_last_change_seconds control trigger frequency and batching behavior.
Check that pause_status is set to UNPAUSED and the timezone_id is valid. For file arrival triggers, verify storage path permissions; for table update triggers, confirm the tables are in Unity Catalog with proper SELECT grants.
Use Databricks Asset Bundles with targets defined in databricks.yml for dev, staging, and prod. Reference bundle variables like ${bundle.target} in job names and parameters to environment-specific cluster sizes and notification emails.
databricksDevelop and deploy Databricks Lakeflow Jobs using Databricks Asset Bundles.
itsadijmbtQuery Unity Catalog system tables for audit, lineage, billing, and compute history.
vivekganaDeploy Databricks Asset Bundles across dev, staging, and prod environments.
as4584Generate Apache Airflow DAG patterns with task dependencies and testing examples.