kr-evaluate

Categorize engine test results into core gaps, extension opportunities, and lessons learned.

Updated Mar 3, 2026
One-click install
npx skills add https://github.com/rayanino/kr --skill kr-evaluate
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: kr-evaluate
Source: https://github.com/rayanino/kr/tree/main/skills/kr-evaluate
Command: npx skills add https://github.com/rayanino/kr --skill kr-evaluate

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill streamlines the review of engine test results, ensuring thorough categorization of findings and clear distinction between critical issues and potential improvements.

Core Features & Use Cases

  • Categorize Findings: Assigns each test result to a specific category (e.g., CORE GAP, LLM QUALITY, EXTENSION OPPORTUNITY).
  • Distinguish Gaps: Differentiates between fundamental flaws requiring immediate fixes (CORE GAP) and areas for future enhancement (EXTENSION OPPORTUNITY).
  • Owner Spot-Check: Facilitates targeted review of Arabic content and LLM-generated outputs with domain experts.
  • Aggregate Assessment: Provides a comprehensive summary of engine reliability, core gaps, lessons learned, and an overall verdict.
  • Use Case: After running a batch of tests on a new language engine, use this Skill to systematically analyze the output, identify bugs, document insights, and determine if the engine is ready for the next stage of development.

Quick Start

Use the kr-evaluate skill to review the test output for the 'arabic-names' engine.

Frequently Asked Questions about kr-evaluate

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I review automated engine test results and categorize failures?

To review automated engine test results, categorize each finding into core gaps, extension opportunities, or lessons learned. You must distinguish between code bugs, prompt issues, data problems, and evaluator noise to ensure thorough quality assurance.

What is the best way to evaluate LLM-worker quality assessment outputs?

Evaluating LLM-worker quality assessment outputs involves categorizing findings into core gaps or extension opportunities. You need to distinguish prompt issues and evaluator noise from fundamental code bugs to determine overall engine reliability.

How do I distinguish between code bugs and prompt issues in LLM evaluator reviews?

Distinguishing code bugs from prompt issues in LLM evaluator reviews requires categorizing test findings. You separate fundamental flaws requiring immediate fixes from data problems or evaluator noise to produce a comprehensive aggregate assessment.

Can I use automated test categorization for Arabic content and LLM-generated outputs?

Yes, automated test categorization supports Arabic content and LLM-generated outputs. It facilitates targeted owner spot-check reviews with domain experts, categorizing findings into core gaps, extension opportunities, and lessons learned for quality assurance.

When do I need to categorize engine test findings into core gaps and extension opportunities?

You need to categorize engine test findings into core gaps and extension opportunities after running a batch of tests on a new language engine. This process identifies fundamental flaws requiring fixes and areas for future enhancement to determine readiness.

Why does my LLM-evaluator review produce inconsistent pass/fail test results?

Inconsistent pass/fail test results during an LLM-evaluator review often stem from evaluator noise, prompt issues, or data problems. Categorizing these findings helps distinguish evaluator noise from fundamental code bugs to reach an overall verdict.