pdd-to-test-prompts-eval

Test prompt quality is graded across 7 weighted dimensions including expected answer specificity, adversarial prompt quality, archetype coverage, and out-of-chain failure mode coverage to ensure reliable LLO chatbot evaluation results.

Adversarial testing coverage evaluates prompt suites against safety-critical, out-of-knowledge-base, and adversarial scenarios a real LLO supervisor would encounter, enforcing a hard floor for failure mode coverage to prevent false confidence in bot performance.

Validate PDD-derived test prompts by checking archetype coverage, expected-tag correctness, and escalation-prompt quality. This enforces a failure mode coverage floor to prevent thin, PDD-faithful prompt suites from passing evals without stress-testing real-world scenarios.

PDD validation includes out-of-chain fitness checks that enforce a hard floor for failure mode coverage. This prevents structurally valid but thin PDD-aligned prompt suites from passing evals without testing out-of-knowledge-base edge cases.

LLO chatbot eval prompt validation grades 7 dimensions: expected-answer specificity, adversarial-prompt quality, archetype coverage, prompt phrasing realism, expected-tag correctness, escalation-prompt quality, and out-of-chain failure-mode coverage.

Generate a structured eval verdict by producing a standardized YAML output with weighted scores, auto-surfaced blockers and warnings, and calibration tracking for consistent LLO chatbot test prompt grading results.