regulation-quality

What problem does it solve?

This Skill addresses the critical need for robust quality assurance of Regulation Q&A systems by simulating diverse user interactions and objectively evaluating response accuracy and relevance.

Core Features & Use Cases

• Comprehensive Testing: Simulates 6 distinct user personas (student, professor, staff, etc.) to test the Q&A system under realistic conditions.
• Automated Evaluation: Employs LLM-as-Judge to score responses across multiple metrics (accuracy, completeness, citations, relevance).
• Use Case: Before deploying a new version of the university's regulation Q&A chatbot, use this Skill to run over 150 test scenarios across all user types to ensure it provides accurate and helpful answers, identifying any regressions or new issues.

Quick Start

Run a quick quality evaluation of the regulation Q&A system.