pdd-to-test-prompts-qa

Validate test prompt markdown structure and output YAML QA results.

1|2|Updated Apr 1, 2026
One-click install
npx skills add https://github.com/dimagi-internal/ace --skill pdd-to-test-prompts-qa
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: pdd-to-test-prompts-qa
Source: https://github.com/dimagi-internal/ace/tree/main/skills/pdd-to-test-prompts-qa
Command: npx skills add https://github.com/dimagi-internal/ace --skill pdd-to-test-prompts-qa

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Manually verifying the structural correctness of test prompt markdown files is time-consuming and error-prone, especially when they must meet strict requirements for adversarial coverage, required fields, and mandatory prompt types before quality evaluation.

Core Features & Use Cases

  • 8 Static Structural Checks: Validates header accuracy, prompt count bounds, required field presence for every prompt, full adversarial category coverage, minimum 20% adversarial share, and mandatory training-gap, product-feedback, and escalation prompts.
  • Auto-Fix Hints: Provides specific, actionable suggestions for fixing each failed check to speed up remediation.
  • Gating for Quality Evaluation: Produces a binary pass/fail verdict that gates the companion pdd-to-test-prompts-eval skill, ensuring only structurally valid prompt suites move to expensive quality grading.
  • Use Case: ACE lifecycle teams building test prompt suites for Connect LLO supervisor chatbots can run this skill to catch structural issues early, before running resource-intensive quality evaluations.

Quick Start

Use the pdd-to-test-prompts-qa skill to run all structural checks on the pdd-to-test-prompts.md file for the current Connect opportunity and return the pass/fail verdict.

Frequently Asked Questions about pdd-to-test-prompts-qa

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I validate test prompt markdown structure for chatbot evaluation?

Structural validation of test prompt markdown checks header alignment, prompt count bounds, required field presence, and full adversarial category coverage before quality evaluation. This skill runs 8 static checks and returns a binary pass/fail verdict with YAML QA results.

What adversarial prompt coverage is required for ACE Connect test prompt suites?

Adversarial prompt coverage requires full coverage of 7 adversarial categories and a minimum 20% adversarial share of total prompts. Test prompt suites must also include mandatory training-gap, product-feedback, and escalation prompts to pass structural validation.

How do I fix failing structural checks in test prompt markdown files?

Failing structural checks can be fixed using auto-fix hints that provide specific, actionable suggestions for each failed validation. Review the structured YAML QA results to remedy header mismatches, missing required fields, insufficient adversarial counts, or missing mandatory prompt types.

Why does my test prompt suite need a binary pass/fail verdict before quality grading?

A binary pass/fail verdict gates quality grading to ensure only structurally valid prompt suites move to resource-intensive quality evaluation. This prevents wasted resources on malformed markdown artifacts lacking required fields, sufficient adversarial coverage, or mandatory prompt types.

Can I use structural validation for test prompts outside the ACE Connect lifecycle?

Structural validation is designed specifically for ACE Connect lifecycle opportunities and supervisor chatbots. The 8 static checks verify requirements tailored to this workflow, including 7 specific adversarial categories and mandatory training-gap, product-feedback, and escalation prompts.

What are the limitations of automated test prompt structural validation?

Automated structural validation only checks formatting and coverage requirements like field presence and adversarial share, not semantic quality or prompt effectiveness. It gates the companion quality grading skill but does not evaluate chatbot response accuracy or conversational relevance.