What problem does it solve? Evaluating pre-recorded or externally generated LLM responses normally requires either re-running live models (expensive, non-deterministic) or bypassing the evaluation pipeline entirely. This Skill lets you feed stored traces into karenina's complete verification pipeline—parsing, template verification, and rubric evaluation—without making live answering-model API calls. ## Core Features & Use Cases - Trace Replay via ManualAdapter: Register pre-recorded responses as strings, karenina port messages, or LangChain message lists, keyed by question text or MD5 hash. - Full Pipeline Execution: Only the answer-generation stage is intercepted; validation, autofails, abstention/sufficiency checks, parsing, verification, and rubric stages all run identically to live runs. - CLI and Python Support: Run via karenina verify --interface manual --manual-traces traces.json or programmatically with ManualTraces and ModelConfig(interface="manual"). - Use Case: Capture responses from a live benchmark run once, then iterate on answer templates or compare parsing judge models repeatedly at zero generation cost. ## Quick Start Ask the agent to run a karenina manual evaluation by registering your pre-recorded answers as ManualTraces for your benchmark questions and running verification with interface set to manual and a live parsing model configured.
