behavioral-evals

Create, debug, and promote behavioral evaluations that validate agent tool-choice decisions.

Updated Aug 26, 2026
One-click install
npx skills add https://github.com/rameshkumarchouhanr/gemni-cli --skill behavioral-evals-rameshkumarchouhanr
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: behavioral-evals
Source: https://github.com/rameshkumarchouhanr/gemni-cli/tree/main/.gemini/skills/behavioral-evals
Command: npx skills add https://github.com/rameshkumarchouhanr/gemni-cli --skill behavioral-evals-rameshkumarchouhanr

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) and assets (resource) components.

What problem does it solve? Prompt and tool changes in an AI agent can silently alter its decision-making behavior, and standard integration tests cannot catch these regressions. This Skill provides a structured workflow for writing, fixing, and promoting behavioral evaluations that verify the agent's tool choices and reasoning trajectory. ## Core Features & Use Cases - Eval Creation Guidance: Choose between evalTest (subprocess TestRig) and appEvalTest (in-process AppRig), seed realistic workspaces, and write assertions using breakpoints, tool-log audits, and mock MCP servers. - Failure Diagnosis: Follow a step-by-step investigation process using nightly CI logs, local Vitest runs, and prompt-tuning strategies grounded in instructional generality principles. - Test Promotion: Identify stable incubated tests (USUALLY_PASSES) with 100% pass rates over 7 nightly runs and promote them to ALWAYS_PASSES regression suites. - Use Case: After editing the system prompt in snippets.ts, write a behavioral eval that asserts the agent calls read_file within its first five turns, run it locally three times across Gemini models, then promote it once stable. ## Quick Start Ask the agent to create a behavioral eval that verifies the agent reads a file before editing it in a seeded NodeJS workspace.

Frequently Asked Questions about behavioral-evals

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I write a behavioral eval for an AI agent?

Use evalTest for standard subprocess tests or appEvalTest for in-process UI tests, seed a realistic workspace with files like package.json, and assert on tool logs via rig.readToolLogs(). New tests must start with the USUALLY_PASSES policy and a timeout budget.

What is the difference between evalTest and appEvalTest?

evalTest runs the CLI in a subprocess and waits for exit, so breakpoints will deadlock it; audit tool logs instead. appEvalTest runs in-process inside the runner loop, making it safe for setBreakpoint and interactive confirmation flows.

Why does my behavioral eval hang when using setBreakpoint?

Breakpoints pause execution, but standard evalTest waits for the process to exit before assertions run, causing an indefinite hang. Use breakpoints only with appEvalTest, or switch to auditing tool logs after the run completes.

How do I fix a failing behavioral eval without changing the test prompt?

Adjust tool descriptions, system prompts in snippets.ts, or modules contributing to the prompt template, keeping changes general rather than test-specific. Changing the test prompt is a last resort since prompts are intentionally vague to preserve fidelity.

When can a USUALLY_PASSES eval be promoted to ALWAYS_PASSES?

A test qualifies for promotion after passing 100% of the time across all enabled models over the last 7 consecutive nightly runs. Promotion changes only the policy argument and must remain a minimal, targeted edit.