...
npx skills add https://github.com/liqiongyu/lenny_skills_plus --skill ai-evals-liqiongyu
Please help me install this Agent Skill. Skill: ai-evals Source: https://github.com/liqiongyu/lenny_skills_plus/tree/main/skills/ai-evals Command: npx skills add https://github.com/liqiongyu/lenny_skills_plus --skill ai-evals-liqiongyu
This Skill streamlines the creation of comprehensive evaluation plans for AI and LLM features, ensuring quality and enabling confident deployment.
Use the ai-evals skill to design evals for a customer-support reply drafting assistant, including a safety rubric and a golden set.
High-intent search queries and answers about installing and using this skill.
Design an LLM evaluation rubric by defining scoring criteria, acceptance thresholds, and a judging plan to assess AI response quality. This process structures human or LLM-as-judge execution for consistent testing.
An error taxonomy in AI evaluation categorizes observed LLM failures into structured groups for analysis. It works by classifying errors to establish quality gates and guide iteration loops for feature development.
Create a golden test set by developing structured user queries and expected outputs from observed failures. This structured dataset serves as a benchmark to measure AI system accuracy and safety.
An error taxonomy in AI evaluation categorizes observed LLM failures into structured groups for analysis. It works by classifying errors to establish quality gates and guide iteration loops for feature development.
Yes, generating an evaluation PRD requires structured input on the system under test and decision criteria. This input defines clear requirements, scope, and acceptance thresholds for AI feature evaluations.
Yes, you can plan LLM-as-judge execution by designing scoring rubrics and harness plans within an evaluation pack. This facilitates automated assessment of AI responses against defined quality standards.
pcasarettoAnalyze specifications to map user flows, identify gaps, and formulate clarifying questions.
LSDIPPOLLCDesign automated quality gates for Copilot harnesses with hook-driven validation.
dtsongDesign evaluation frameworks for AI features with golden datasets and scoring rubrics.
marinkobabicRun OWASP Top 10 (2021)-aligned security audits for web applications.
