skills.rest
souliane
soulianeCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

running-evals

Automates AI and deterministic evaluations for codebase analysis with Claude Code.

1|Updated Mar 5, 2026
Search Tags:#code quality#agent behavior#AI evaluation#deterministic testing#codebase assessment
One-click install
npx skills add https://github.com/souliane/teatree --skill running-evals
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: running-evals
Source: https://github.com/souliane/teatree/tree/main/skills/running-evals
Command: npx skills add https://github.com/souliane/teatree --skill running-evals

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill simplifies the process of running both AI evaluations and deterministic tests on a codebase, reducing the complexity of evaluating agent behavior and ensuring code quality.

Core Features & Use Cases

  • Automated Evaluation: Run both AI evaluations and deterministic tests in one command.
  • AI/trajectory lane: Prepare, produce, and grade in-session transcripts.
  • Deterministic lanes: Run tests like pinned-regressions without a model.
  • Use Case: For a code review, use this Skill to automatically evaluate both the AI agent's behavior and the deterministic test results to get a comprehensive picture of code quality.

Quick Start

Run the full eval suite with a single command: /t3:running-evals

Frequently Asked Questions about running-evals

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run AI evaluations and deterministic tests on a codebase at the same time?▼

To run AI evaluations and deterministic tests simultaneously, you can use a single command to automate the orchestration of both agent behavior assessments and pinned-regression tests, providing a unified results table for comprehensive codebase evaluation.

What is the difference between AI trajectory evaluation and deterministic testing for agent behavior?▼

AI trajectory evaluation prepares, produces, and grades in-session transcripts to assess agent behavior, whereas deterministic testing runs functionality tests like pinned-regressions without requiring a model, ensuring code quality through fixed expected outcomes.

Do I need Claude Code to automate codebase assessment and AI evals?▼

Yes, you need Claude Code to automate codebase assessment and AI evals. The Skill requires Claude Code and supports various backends for AI evaluations to properly orchestrate both agent behavior and deterministic test lanes.

Can I evaluate code quality during a code review without using an AI model?▼

Yes, you can evaluate code quality without an AI model by using deterministic testing lanes. These lanes run functionality tests like pinned-regressions that validate code behavior independently of AI-driven agent evaluations.

What's the best way to get a comprehensive picture of code quality from agent behavior and test results?▼

The best way to get a comprehensive picture of code quality is to run an automated evaluation suite that handles both AI-driven agent behavior assessments and deterministic functionality tests, outputting a unified results table for direct comparison.

Are there limitations when running AI evaluations for codebase assessment?▼

A key limitation is the dependency on Claude Code and supported backends for AI evaluations. Deterministic tests run without a model, but assessing agent behavior requires the full environment to execute the AI-driven trajectory lane.

Related Skills

Explore More Software Engineering Skills→
chiptoe-svgchiptoe-svg

add-discord

Install and configure the official Discord chat adapter for NanoClaw agents.

Community
Intermediate
jamesjlundinjamesjlundin
32

pr-review

Analyze pull requests and generate structured review reports.

Community
Intermediate
beyonaibeyonai
45

skillify

Standardize agent tool development through multi-phase audits and cross-modal evaluations.

Community
Advanced
LinkedInLearningLinkedInLearning
2

convert-web-app

Convert existing web applications into MCP Apps with a hybrid architecture.

Official
Advanced