skills.rest
phuhao00
phuhao00COMMUNITY
Explore all skills by this author →
WebsiteGitHub

agent-eval

Benchmarks AI agent performance by comparing tool-assisted navigation against baseline grep and read operations.

12|3|Updated Jun 17, 2026
Search Tags:#benchmarking#developer-tools#code-navigation#performance-analysis#agent-evaluation#codegraph
One-click install
npx skills add https://github.com/phuhao00/bony-agent --skill agent-eval-phuhao00
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: agent-eval
Source: https://github.com/phuhao00/bony-agent/tree/main/vendor/codegraph/.claude/skills/agent-eval
Command: npx skills add https://github.com/phuhao00/bony-agent --skill agent-eval-phuhao00

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires tmux, node, git, and includes scripts (resource) components.

What problem does it solve?

This Skill solves the difficulty of objectively measuring how much a specific tool or version of CodeGraph improves an AI agent's ability to navigate and understand complex, real-world codebases compared to standard methods.

Core Features & Use Cases

  • Comparative Benchmarking: Run controlled experiments comparing agent behavior with and without CodeGraph integration.
  • Multi-Harness Support: Execute tests in headless mode for speed and cost metrics, or interactive tmux sessions for faithful TUI behavior analysis.
  • Use Case: A developer wants to verify if a new version of their indexing tool reduces the number of file reads and total token cost when an agent is asked to trace a complex cross-module call path in a large repository like Django or the Cosmos SDK.

Quick Start

Use the agent-eval skill to benchmark the latest published version of CodeGraph against the Django repository using the headless harness.

Frequently Asked Questions about agent-eval

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I benchmark AI agent performance on a real-world codebase?▼

Benchmarking agent code navigation compares tool-assisted file discovery against baseline grep and read operations. It measures token consumption and task success rates to evaluate tool efficacy on real-world repositories.

What is the best way to measure if a code navigation tool reduces token cost?▼

Measuring code navigation tool token cost requires orchestrating automated test harnesses on target repositories. You compare baseline file reads against tool-assisted runs to generate actionable metrics for agent efficiency and token reduction.

Can I run agent evaluation tests in headless mode and interactive tmux sessions?▼

Yes, you can run agent evaluation tests in headless mode for speed and cost metrics, or use interactive tmux sessions for faithful TUI behavior analysis. Multi-harness support allows both execution environments.

Do I need tmux and node installed to use agent-eval?▼

Yes, you need tmux, node, and git installed. These dependencies are required to execute the benchmark scripts and orchestrate the automated test harnesses on local repositories.

Does agent benchmarking work with large repositories like Django or the Cosmos SDK?▼

Yes, agent benchmarking works with large repositories like Django or the Cosmos SDK. It orchestrates test harnesses across various programming languages and project sizes to trace complex cross-module call paths.

Related Skills

Explore More Software Engineering Skills→
flashinfer-aiflashinfer-ai
264

clone-repos

Clone and update SGLang and FlashInfer repositories with submodules from GitHub.

Official
Advanced
mahmoud20138mahmoud20138
10

skill-analytics

Analyze skill usage data and generate optimization recommendations.

Community
Intermediate
MSApps-MobileMSApps-Mobile
8

kotlin-lsp

Analyze Kotlin .kt and .kts files with LSP diagnostics and navigation.

Official
Intermediate
mindmorassmindmorass
2

mcp-server-builder

Scaffold MCP servers with environment-based configuration and tool registration.

Community
Intermediate