Design and run AI evaluations to measure model quality.
npx skills add https://github.com/PSkinnerTech/lenny-skills --skill ai-evals-pskinnertech
Please help me install this Agent Skill. Skill: ai-evals Source: https://github.com/PSkinnerTech/lenny-skills/tree/main/skills/ai-evals Command: npx skills add https://github.com/PSkinnerTech/lenny-skills --skill ai-evals-pskinnertech
Teams building AI products need a repeatable way to evaluate model quality, create test cases, and benchmark performance across iterations.
Describe an end-to-end AI evaluation plan for a specific model feature and generate the rubrics, test cases, and benchmarks you will use.
High-intent search queries and answers about installing and using this skill.
Design evaluation rubrics by translating product goals into measurable criteria. This Skill generates rubrics and benchmarks that map directly to your LLM feature requirements, allowing you to evaluate model quality against specific product outcomes.
Benchmark LLM outputs by measuring performance against established rubrics and test cases. This Skill supports benchmarking across model iterations to track quality improvements and guide product decisions based on result analysis.
Generate test cases by creating diverse evaluation datasets tailored to your model feature. This Skill produces test cases designed to expose weaknesses in LLM outputs and provide actionable data for iterating on improvements.
Yes, you can run end-to-end AI evaluations tailored for product teams. This Skill plans an evaluation plan for a specific model feature and generates the necessary rubrics, test cases, and benchmarks to validate LLM functionality.
A repeatable process to evaluate model quality ensures consistent measurement across updates. This Skill provides a structured approach to test-case generation and benchmarking, enabling teams to reliably assess LLM performance and iterate based on results.
yu13130122297Identify uncertainties in research framing, datasets, metrics, and baselines.
zhouguoqingCreate and refine Claude Skills with structured workflows for testing and iteration.
dtsongDesign evaluation frameworks for AI features with golden datasets and scoring rubrics.
OpenLAIRTrace, evaluate, and monitor LLM applications with OpenTelemetry and self-hosted storage.
