behavioral-evals

Create, debug, and promote behavioral evaluations that validate agent tool-choice decisions.

Updated May 23, 2026
One-click install
npx skills add https://github.com/kiranimmadi2/promptforge-ai --skill behavioral-evals-kiranimmadi2
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: behavioral-evals
Source: https://github.com/kiranimmadi2/promptforge-ai/tree/main/gemini-cli/.gemini/skills/behavioral-evals
Command: npx skills add https://github.com/kiranimmadi2/promptforge-ai --skill behavioral-evals-kiranimmadi2

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) and assets (resource) components.

What problem does it solve? Prompt and tool changes can silently alter an AI agent's decision-making, and standard unit tests cannot catch these regressions. This Skill provides a structured workflow for writing, fixing, and promoting behavioral evaluations that verify the agent chooses the right tools and follows intended reasoning paths. ## Core Features & Use Cases - Eval Creation Guidance: Choose between evalTest (subprocess TestRig) and appEvalTest (in-process AppRig), seed realistic workspaces, and write assertions using breakpoints, tool-log audits, and mock MCP servers. - Failure Diagnosis: Follow a step-by-step investigation process using nightly CI logs, local Vitest runs, and prompt-tuning strategies that favor general engineering principles over narrow fixes. - Test Promotion: Identify stable incubated tests (USUALLY_PASSES) with 100% pass rates over 7 nightly runs and promote them to ALWAYS_PASSES regression suites. - Use Case: After editing a system prompt in snippets.ts, write a behavioral eval that asserts the agent calls read_file before editing, run it locally across Gemini models, and promote it once stable to lock in the fix. ## Quick Start Ask the agent to create a behavioral eval that verifies the agent reads a file before modifying it, using the standard evalTest template.

Frequently Asked Questions about behavioral-evals

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I write a behavioral eval for an AI agent?

Use evalTest for standard subprocess tests or appEvalTest for UI interactions, seed a realistic workspace with files like package.json, and assert on tool logs via rig.readToolLogs(). New tests must start with the USUALLY_PASSES policy and include a timeout budget.

What is the difference between evalTest and appEvalTest?

evalTest runs the CLI in a subprocess and waits for exit, so it only supports auditing tool logs after completion. appEvalTest runs in-process, making it safe for setBreakpoint and interactive confirmation flows, but breakpoints in evalTest will hang indefinitely.

How do I fix a failing behavioral eval?

Investigate locally using logs in evals/logs/ and GEMINI_DEBUG_LOG_FILE, then fix by adjusting tool descriptions or system prompts in snippets.ts with general principles rather than narrow rules. Verify stability by running the test 3 times across Gemini 3.0, Gemini 3 Flash, and Gemini 2.5 Pro.

When can a USUALLY_PASSES test be promoted to ALWAYS_PASSES?

A test qualifies for promotion after passing 100% of the time across all enabled models over the last 7 nightly runs. Promotion changes only the policy argument to ALWAYS_PASSES without refactoring the test or its fixtures.

Why does my behavioral eval hang when using setBreakpoint?

Breakpoints pause execution, but standard evalTest waits for the process to exit before assertions run, causing a deadlock. Use setBreakpoint only with appEvalTest, or switch to auditing tool logs with rig.readToolLogs() for subprocess tests.