skills.rest
dimagi-internal
dimagi-internalOFFICIAL
Explore all skills by this author →
GitHub

synthetic-summary-eval

Grade synthetic Connect opportunity summaries against rubric dimensions and source documents.

1|2|Updated Apr 1, 2026
Search Tags:#quality-assurance#substrate-fidelity#synthetic-summary#connect-opportunity#rubric-grading#stakeholder-readiness#demo-evaluation
One-click install
npx skills add https://github.com/dimagi-internal/ace --skill synthetic-summary-eval
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: synthetic-summary-eval
Source: https://github.com/dimagi-internal/ace/tree/main/skills/synthetic-summary-eval
Command: npx skills add https://github.com/dimagi-internal/ace --skill synthetic-summary-eval

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill solves the critical gap where synthetic Connect opportunity summaries look polished in draft but fail when forwarded to stakeholders: they may contain internal implementation jargon, missing or broken links to labs/fixture folders, unsubstantiated headline claims, or numbers that don't match the actual live demo environment, leading to stakeholder confusion or mistrust.

Core Features & Use Cases

  • Multi-dimensional rubric grading: Evaluates summaries across 6 weighted dimensions: stakeholder-readiness, narrative coherence, completeness, substrate fidelity (cross-checking claims against the live rendered demo), source fidelity, and what's-next clarity.
  • Hard-gate safety rules: Automatically flags or fails summaries with critical issues like missing labs/fixture URLs, unsubstituted placeholder tokens, or headline claims that contradict the live demo state.
  • Use Case: Dimagi teams building Connect opportunity demos use this Skill to validate the one-page synthetic summary before forwarding it to external stakeholders, ensuring it is accurate, complete, and free of internal implementation details.

Quick Start

Grade the synthetic summary for the current Connect opportunity using the synthetic-summary-eval skill to confirm it is safe to forward to external stakeholders.

Frequently Asked Questions about synthetic-summary-eval

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I validate a synthetic summary for stakeholder readiness before forwarding it?▼

To validate synthetic summary stakeholder readiness, grade the artifact against a multi-dimensional rubric that checks narrative coherence, link completeness, and cross-references claims against the actual live demo state.

What is substrate fidelity evaluation for demo summaries?▼

Substrate fidelity evaluation cross-checks stated claims in synthetic summaries against the live rendered demo environment, ensuring headline numbers and functionality descriptions match actual demo behavior before stakeholder delivery.

How do I check if my summary contains unsubstantiated claims or broken links?▼

Apply hard-gate safety rules to automatically flag summaries containing missing labs or fixture URLs, unsubstituted placeholder tokens, or headline claims that contradict the live demo state, preventing inaccurate forwarding.

Can I use rubric grading to detect internal jargon in external stakeholder summaries?▼

Yes, rubric-based grading evaluates stakeholder-readiness by detecting internal implementation jargon and missing links, ensuring synthetic Connect opportunity summaries are polished and suitable for external audiences.

What are the limitations of automated summary evaluation for Connect opportunity demos?▼

Automated summary evaluation focuses on Phase 7 synthetic summary artifacts and cross-checks claims against upstream source documents, but requires a live rendered demo state to accurately verify substrate fidelity and link completeness.

Does synthetic summary evaluation work without upstream source documents?▼

No, synthetic summary evaluation requires upstream source documents and the live rendered demo state to cross-check claims, verify substrate fidelity, and ensure headline numbers match actual demo functionality.

Related Skills

Explore More Product & Management Skills→
dimagi-internaldimagi-internal
1

synthetic-summary

Aggregates Phase 7 synthetic data demo artifacts into a stakeholder-facing markdown summary.

Official
Advanced
AlsKozlovAlsKozlov
17

stakeholder-summary

Convert legal contract review memos into plain-language stakeholder summaries.

Community
Advanced
sajalsuhanesajalsuhane

connect-chrome

Connect Claude to a headed Chrome window with the gstack extension auto-loaded.

Community
Advanced
EdgeAppEdgeApp
1

agent-eval

Evaluate agent run transcripts against a compliance rubric for process fidelity.

Official
Advanced