skills.rest
opendatahub-io
opendatahub-ioOFFICIAL
Explore all skills by this author →
WebsiteGitHub

eval-setup

Configure and validate the agent-eval-harness environment with MLflow tracking and API key checks.

36|39|Updated Apr 2, 2026
Search Tags:#workflow#validation#dependencies#config#environment#preflight#mlflow
One-click install
npx skills add https://github.com/opendatahub-io/agent-eval-harness --skill eval-setup
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-setup
Source: https://github.com/opendatahub-io/agent-eval-harness/tree/main/skills/eval-setup
Command: npx skills add https://github.com/opendatahub-io/agent-eval-harness --skill eval-setup

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires mlflow, pyyaml, anthropic, and includes scripts (resource) components.

What problem does it solve?

Verifies and configures optional components for the agent-eval-harness, including MLflow tracking, API keys, and environment readiness; non-destructive and triggers on common setup signals.

Core Features & Use Cases

  • Configure MLflow tracking and experiment context
  • Verify API keys and environment readiness, with non-destructive fixes
  • Run preflight checks and guide follow-up steps for evaluation pipelines
  • Support remote MLflow server configuration and per-project pinning

Quick Start

Run /eval-setup to initialize environment checks and configure optional MLflow tracking.

Frequently Asked Questions about eval-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I configure MLflow tracking for my evaluation pipeline?▼

Configure MLflow tracking by running an environment setup script that parses variables, verifies API keys, creates missing directories, and prepares an experiment context non-destructively for local or remote deployments.

What is a preflight check for MLflow and API dependencies?▼

A preflight check validates environment readiness by parsing environment variables, verifying API keys, checking dependency health, and preparing the MLflow experiment context without applying destructive changes.

Can I run environment validation for remote MLflow servers?▼

Yes, environment validation supports remote MLflow server configuration and per-project pinning by parsing environment variables and verifying API keys to ensure readiness across local or remote deployments.

How do I verify API keys and environment readiness for evaluation workflows?▼

Verify API keys and environment readiness by running preflight checks that parse environment variables, confirm dependency health, and prepare the experiment context for your evaluation pipelines.

Does environment setup modify or destroy existing configuration files?▼

No, environment setup is explicitly non-destructive. It safely parses environment variables, checks dependency health, creates missing directories, and prepares the experiment context without overwriting or destroying existing configuration files.

Related Skills

Explore More Software Engineering Skills→
AutumnsGroveAutumnsGrove
26

env-config

Manage Python environment configurations and encrypt secrets across dev, staging, and prod.

Community
Intermediate
ovachieverovachiever
52

weights-and-biases

Automate ML experiment tracking and model registry management via the wandb API.

Community
Advanced
alyamenfactoryalyamenfactory

ai-integrations-anthropic

Integrate Anthropic AI via Replit AI Integrations proxy without an API key.

Community
Intermediate
tianhao909tianhao909
1

tensorboard

Visualize machine learning training metrics and model graphs with TensorBoard.

Community
Intermediate