skills.rest
filipexyz
filipexyzCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

tasks-eval

Run JSON task specifications against Ravi sessions for regression testing.

6|2|Updated Jan 29, 2026
Search Tags:#testing#benchmarking#evaluation#agents#regression#ravi
One-click install
npx skills add https://github.com/filipexyz/ravi --skill tasks-eval
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: tasks-eval
Source: https://github.com/filipexyz/ravi/tree/main/src/plugins/internal/ravi-system/skills/tasks-eval
Command: npx skills add https://github.com/filipexyz/ravi --skill tasks-eval

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill solves the lack of objective, reproducible measurement in agent development by providing a structured harness to validate behavior and prevent regressions.

Core Features & Use Cases

  • Reproducible Benchmarking: Run task specifications against live Ravi sessions to measure performance consistently.
  • Automated Validation: Apply deterministic rubrics to evaluate responses, file changes, and transcripts.
  • Use Case: When refactoring an agent's routing logic, use this Skill to run a suite of regression tests to ensure the agent still produces the expected output for specific prompts.

Quick Start

Execute the evaluation harness for the specified task configuration file by running the ravi eval run command followed by the path to your json spec file.

Frequently Asked Questions about tasks-eval

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run regression testing for AI agents to prevent behavior regressions?▼

Regression testing for AI agents is executed by running JSON-based task specifications against live sessions to measure performance and validate behavior using reproducible task execution.

How do I benchmark AI agent performance and reliability reproducibly?▼

Benchmarking AI agent performance reproducibly involves executing structured task specifications within the Ravi ecosystem to consistently measure reliability and compare runtime configurations.

Can I use JSON task specifications to perform automated validation on agent workflows?▼

Yes, JSON task specifications support automated validation by applying deterministic rubrics to evaluate agent responses, file changes, and transcripts for structured output grading.

What is the best way to compare prompts and agent configurations during refactoring?▼

Comparing prompts and agent configurations is achieved through comparative analysis using snapshot-based diffing and rubric-based grading of agent outputs to ensure expected behavior.

Does the Ravi ecosystem support snapshot-based diffing for agent evaluation?▼

Yes, the Ravi ecosystem supports snapshot-based diffing to evaluate agent outputs, enabling precise comparative analysis and automated validation of agent responses during task execution.

Related Skills

Explore More Software Engineering Skills→
jyang234jyang234

edi-core

Establish standardized identity, recall integration, and flight logging for agents.

Community
Basic
brianruggieribrianruggieri

eval-extract

Run blind extraction evaluations against fixtures and grade results with a rubric.

Community
Advanced
filipexyzfilipexyz
6

meetings

Join Google Meet sessions to record and transcribe calls into meet.md artifacts.

Community
Advanced
filipexyzfilipexyz
6

cli-creator

Create and refactor Ravi runtime CLI tools with decorator-based auto-discovery.

Community
Advanced