databricks-jobs

Automate Databricks job creation, management, monitoring via API.

Updated Mar 23, 2024
One-click install
npx skills add https://github.com/m19c/dotfiles --skill databricks-jobs-m19c
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: databricks-jobs
Source: https://github.com/m19c/dotfiles/tree/main/claude/.claude/skills/databricks-jobs
Command: npx skills add https://github.com/m19c/dotfiles --skill databricks-jobs-m19c

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires databricks.sdk, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill streamlines the management of Databricks Jobs, including creation, listing, running, updating, and deleting, making it easier to orchestrate data workflows and monitor job status.

Core Features & Use Cases

  • Job Creation and Management: Create, list, run, update, and delete Databricks Jobs.
  • Task Orchestration: Support for DAG-based task dependencies and flexible triggers.
  • Monitoring and Alerts: Configure notifications and health monitoring for jobs.
  • Use Case: Imagine you need to schedule a daily ETL job to process data. This Skill allows you to define the job, set up triggers, and configure notifications for successful or failed runs.

Quick Start

Use the 'databricks-jobs' skill to create a new job with the following command: databricks-jobs create --name "my-job" --tasks [extract, transform, load]

Frequently Asked Questions about databricks-jobs

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate creating and running Databricks jobs?

You can automate Databricks jobs by using this skill to create, run, and delete jobs through commands. It handles API access to define jobs, set triggers, and orchestrate data workflows automatically.

Can I set up task dependencies for Databricks data workflows?

Yes, you can set up task dependencies for Databricks data workflows. The skill supports DAG-based task orchestration, allowing you to define sequential and conditional triggers for complex ETL pipelines.

How do I configure notifications for Databricks job monitoring?

To configure notifications for Databricks job monitoring, use this skill to set up health monitoring and alerts. It enables configurations for successful or failed job runs to track workflow status.

Do I need Databricks API access to manage task orchestration?

Yes, you need Databricks API access to manage task orchestration. The skill requires the databricks.sdk dependency and knowledge of Databricks job management to execute workflow commands.

What is the best way to update and delete Databricks jobs programmatically?

The best way to update and delete Databricks jobs programmatically is using this skill. It directly interfaces with the Databricks API to modify existing job configurations or remove obsolete workflows.