checkpoint-qa

What problem does it solve?

Coordinates end-to-end Q&A evaluation for AI model comparisons by brainstorming questions, proposing answers, and validating responses against evaluation criteria.

Core Features & Use Cases

• Brainstorm questions (preset or custom) and prepare evaluation prompts
• Suggest answers with evidence, references, and structured reasoning
• Evaluate CTV responses and generate an Overall Preference Justification
• Supports multi-turn workflows and exports per-turn QA artifacts (questions.json, q_*.md, and overall_justification)

Quick Start

Invoke the skill with a workspace path and a turn number to auto-generate questions and kick off the evaluation workflow.