Design and test LLM prompts with evaluation rubrics and test suites.
npx skills add https://github.com/Mathews-Tom/praxis-skills --skill prompt-lab-mathews-tom
Please help me install this Agent Skill. Skill: prompt-lab Source: https://github.com/Mathews-Tom/praxis-skills/tree/main/skills/prompt-lab Command: npx skills add https://github.com/Mathews-Tom/praxis-skills --skill prompt-lab-mathews-tom
Replaces trial-and-error prompt iteration with a repeatable methodology that analyzes existing prompts for failure modes, generates focused variants, and produces evaluation rubrics and test suites to reliably compare prompt performance and enforce output formats.
Use prompt-lab to generate two few-shot and one structured-output prompt variant plus a five-case test suite and a weighted rubric for classifying customer support tickets.
High-intent search queries and answers about installing and using this skill.
A/B testing for LLM prompts involves generating structured variants like few-shot or chain-of-thought, then evaluating them against weighted rubrics and test suites to reliably compare performance and enforce format compliance.
Designing a prompt evaluation test suite requires building a minimum five-case suite covering happy path, edge, adversarial, and boundary scenarios, paired with weighted rubrics for automated scoring across models like GPT-4 and Claude.
Analyzing existing prompts for failure modes involves identifying ambiguities, mapping to known failure patterns, and applying format-enforcement strategies to guide targeted fixes for classification or extraction tasks.
Yes, generating structured output variants can combine few-shot and chain-of-thought techniques, where each variant changes a single variable and includes hypothesis and risk notes to isolate performance impacts during A/B testing.
Inconsistent prompt results often stem from missing constraints or ambiguities; fixing them requires designing weighted evaluation rubrics and applying format constraints to systematically score and compare prompt variants.
Chain-of-thought prompt engineering is needed for multi-step reasoning tasks where direct prompts fail, allowing you to generate targeted variants and evaluate them with test suites to ensure reliable format compliance.
sunny0826Analyze user prompts for ambiguities and risks, then generate an optimized Markdown review.
slowman2084Score agent outputs against YAML rubrics or gold references with verified calculations.
cabeznoGenerate schema-constrained JSON, XML, and code with grammar-based token filtering.
nroze22Conduct FMEA risk assessments for clinical trial EDC systems.
