eval-model-effectiveness
CommunityAnalyze pipeline data to tune agent models.
Authorpaulingham
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill analyzes pipeline observations and per-role costs to generate a data-driven recommendation report on model downgrades or upgrades, guiding humans to optimize resource use without automatically changing configurations.
Core Features & Use Cases
- Analyze per-role effectiveness across pipeline observations and costs to identify downgrade or upgrade opportunities.
- Produce a markdown report that summarizes recommendations and evidence for human-in-the-loop action.
- Use in scenarios where teams want to optimize model-tier choices (e.g., Opus, Sonnet, Haiku) based on success-rate and cost metrics.
Quick Start
Run the analyze.py script for the current project.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: eval-model-effectiveness Download link: https://github.com/paulingham/.claude/archive/main.zip#eval-model-effectiveness Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 545,000+ vetted skills library on demand.