What problem does it solve? Prompts that pass one manual test often fail silently on real-world inputs, and non-deterministic LLM outputs make single-run quality checks unreliable. This Skill replaces guesswork with a measurable 5-step cycle: draft prompt, design rubric, build dataset, grade outputs, and iterate until the average score reaches a production threshold. ## Core Features & Use Cases - Rubric-Based Grading: Define 2-5 weighted criteria with anchor examples, then grade outputs using a hybrid of code-based checks and LLM judgment via RubricGrader. - Dataset-Driven Testing: Run prompts against 3-10 representative test cases and compute aggregate metrics including average score, pass rate, standard deviation, and weakest criteria. - Multi-Run Confidence Intervals: Execute 3-5 evaluation runs with varied temperatures and report mean scores with 95% confidence intervals per IEEE 29119-2 guidance for non-deterministic systems. - Prompt Comparison: Benchmark two prompt versions on the same dataset to identify the winner and quantify the improvement. - Use Case: A team building a code-review agent evaluates its review prompt against Python snippets with known bugs, discovers it misses race conditions, refines the prompt, and verifies the score climbs from 68% to 85% before release. ## Quick Start Ask the AI to evaluate your prompt by defining a rubric and a small dataset of test cases, then run the evaluation pipeline and report the average score with the weakest criteria.
