tasks-eval

Run JSON task specifications against Ravi sessions for regression testing.

6|2|Updated Jan 29, 2026

Search Tags:#testing #benchmarking #evaluation #agents #regression #ravi

Ready to run

npx skills add https://github.com/filipexyz/ravi --skill tasks-eval

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: tasks-eval
Source: https://github.com/filipexyz/ravi/tree/main/src/plugins/internal/ravi-system/skills/tasks-eval
Command: npx skills add https://github.com/filipexyz/ravi --skill tasks-eval

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill solves the lack of objective, reproducible measurement in agent development by providing a structured harness to validate behavior and prevent regressions.

Core Features & Use Cases

Reproducible Benchmarking: Run task specifications against live Ravi sessions to measure performance consistently.
Automated Validation: Apply deterministic rubrics to evaluate responses, file changes, and transcripts.
Use Case: When refactoring an agent's routing logic, use this Skill to run a suite of regression tests to ensure the agent still produces the expected output for specific prompts.

Quick Start

Execute the evaluation harness for the specified task configuration file by running the ravi eval run command followed by the path to your json spec file.

Frequently Asked Questions about tasks-eval

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I run regression testing for AI agents to prevent behavior regressions?▼

Regression testing for AI agents is executed by running JSON-based task specifications against live sessions to measure performance and validate behavior using reproducible task execution.

How do I benchmark AI agent performance and reliability reproducibly?▼

Benchmarking AI agent performance reproducibly involves executing structured task specifications within the Ravi ecosystem to consistently measure reliability and compare runtime configurations.

Can I use JSON task specifications to perform automated validation on agent workflows?▼

Yes, JSON task specifications support automated validation by applying deterministic rubrics to evaluate agent responses, file changes, and transcripts for structured output grading.

What is the best way to compare prompts and agent configurations during refactoring?▼

Comparing prompts and agent configurations is achieved through comparative analysis using snapshot-based diffing and rubric-based grading of agent outputs to ensure expected behavior.

Does the Ravi ecosystem support snapshot-based diffing for agent evaluation?▼

Yes, the Ravi ecosystem supports snapshot-based diffing to evaluate agent outputs, enabling precise comparative analysis and automated validation of agent responses during task execution.