skills.rest
itsadijmbt
itsadijmbtCOMMUNITY
Explore all skills by this author →
GitHub

skill-test

Evaluate and optimize Databricks agent skills against ground truth YAML test cases.

1|Updated Apr 24, 2026
Search Tags:#optimization#evaluation#yaml#mlflow#databricks#trace analysis#gepa
One-click install
npx skills add https://github.com/itsadijmbt/SecureMCP-Servers --skill skill-test-itsadijmbt
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: skill-test
Source: https://github.com/itsadijmbt/SecureMCP-Servers/tree/main/TEST_SERVERS/PORTED_TO_SECUREMCP/databrickslab-mcp/ai-dev-kit/.test
Command: npx skills add https://github.com/itsadijmbt/SecureMCP-Servers --skill skill-test-itsadijmbt

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pyyaml, mlflow[databricks]>=3.10.1, protobuf>=5.26, python-dotenv>=1.0.0, gepa>=0.1.0, tiktoken>=0.7.0, litellm<=1.82.6, claude-agent-sdk>=0.1.39, databricks-sdk>=0.20.0, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill solves the challenge of measuring whether Databricks-focused agent instructions (SKILL.md) actually improve real outputs versus running without them, while providing repeatable ground-truth evaluation and iterative optimization.

Core Features & Use Cases

  • Ground-truth driven evaluation: Runs deterministic assertions and LLM-based judges against ground_truth.yaml to produce pass/fail quality signals.
  • GEPA-based optimization: Uses evolutionary optimization to mutate SKILL.md based on judge rationales and assertion failures.
  • Regression testing and baselines: Saves baseline evaluation runs and compares future results to detect regressions.
  • Trace evaluation support: Validates agent tool usage, token budgets, and execution behavior using Claude Code traces from MLflow or local JSONL.
  • Workflow automation for test cases: Supports init, add, interactive review, and generate-review-promote flows to maintain high-quality test sets.

Quick Start

Run: /skill-test databricks-spark-declarative-pipelines run

Frequently Asked Questions about skill-test

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate Databricks agent instructions against ground truth test cases?▼

You can evaluate Databricks agent instructions by running deterministic assertions and LLM-based judges against a ground_truth.yaml file to produce pass/fail quality signals for WITH-skill vs WITHOUT-skill responses.

How does GEPA optimization improve agent instruction skills?▼

GEPA optimization uses an evolutionary loop to mutate SKILL.md files based on judge rationales and assertion failures, iteratively improving Databricks agent behaviors through automated instruction refinement.

Can I use MLflow to score and trace Databricks agent evaluations?▼

MLflow-compatible scoring is fully supported for evaluation runs, and you can validate agent tool usage and token budgets using Claude Code traces from MLflow or local JSONL files.

What is the best way to detect regressions in Databricks agent skills?▼

Regression testing for Databricks agent skills works by saving baseline evaluation runs and comparing future results against them to detect quality degradations across instruction iterations.

Does this YAML-first evaluation approach support interactive test case management?▼

The workflow supports interactive review and generate-review-promote flows to build and maintain high-quality ground_truth.yaml and manifest.yaml datasets for Databricks-focused agent behaviors.

What limitations exist when optimizing Databricks skills with trace-based evaluation?▼

Trace-based evaluation requires Claude Code transcripts from either MLflow or local JSONL, and optimization depends on structured judge rationales from ground_truth.yaml datasets to drive GEPA mutation loops effectively.

Related Skills

Explore More Software Engineering Skills→
elasticdotventureselasticdotventures
14

direnv-pattern

Automate loading of per-project environment variables via direnv and dotenv with Rust validation.

Community
Advanced
juanlamadrid20juanlamadrid20

agent-mlops

Automate production deployment and monitoring of AI agents on Databricks.

Community
Advanced
RamVegirajuRamVegiraju
4

agent-evaluation

Evaluate GenAI agents with MLflow datasets, scorers, and traces.

Community
Advanced
databrickslabsdatabrickslabs
38

databricks-mlflow-evaluation

Evaluate GenAI agents with MLflow 3 GenAI evaluation APIs and built-in scorers.

Official
Advanced