Grade Product Design Documents against source idea packs and output verdict YAML.
npx skills add https://github.com/dimagi-internal/ace --skill idea-to-pdd-eval
Please help me install this Agent Skill. Skill: idea-to-pdd-eval Source: https://github.com/dimagi-internal/ace/tree/main/skills/idea-to-pdd-eval Command: npx skills add https://github.com/dimagi-internal/ace --skill idea-to-pdd-eval
The built-in self-evaluation for Product Design Documents (PDDs) has critical blind spots: the same AI model that writes the PDD also grades it, leading to over-generosity bias, and it only checks the PDD against its own rubric rather than the original source idea pack. This means PDDs that miss reviewer comments from the source material or silently resolve source conflicts can still earn perfect self-graded scores, leading to low-quality designs moving forward in the ACE lifecycle.
Use the idea-to-pdd-eval skill to grade the PDD for your current Connect opportunity against its source idea pack and generate a structured quality verdict.
High-intent search queries and answers about installing and using this skill.
Evaluating a Product Design Document against its original source idea pack requires cross-checking reviewer comment fidelity and source conflict honesty to eliminate over-generosity bias from self-evaluations. This process generates a structured verdict YAML with weighted dimension scores and auto-surfaced warnings.
PDD self-evaluation over-generosity bias happens when the same AI model authors and grades the document, ignoring original source material. An independent grading process re-runs stress tests from outside the authoring workflow to catch missed reviewer comments and silently resolved source conflicts.
Detecting silent source conflicts in ACE Connect opportunity design workflows involves cross-checking PDD content against the full original source idea pack. This independent validation flags unaddressed concerns and integrity issues that self-evaluations often miss.
Independently grading PDDs across weighted quality and viability dimensions is possible by assessing document fidelity, program viability, demand reality, and mission alignment. This produces a structured quality verdict YAML with calibration signals for self-eval rubric accuracy.
Run an independent PDD evaluation immediately after generating the document and before moving to implementation. This ensures the design is grounded in the original source material and identifies gaps or integrity issues early in the ACE lifecycle.
Idea-to-pdd-eval is an independent PDD grading skill for ACE Connect workflows that requires no external dependencies. It validates PDD content against source idea packs to output a structured verdict YAML with weighted scores and auto-surfaced warnings.
dimagi-internalValidate PDD structure against 11 sections and archetype enums.
dimagi-internalGrade Nova-built Learn training apps against their Product Design Document.
dimagi-internalGrade OCS chatbot transcripts against LLM-as-Judge rubrics.
aarushlohitConducts web research with source verification and fact-checking for documents.
