eval-generator

Generate pytest evaluation tests from Jira acceptance criteria.

2|Updated May 27, 2026

Search Tags:#test automation #pytest #evaluation #acceptance criteria #Jira

Ready to run

npx skills add https://github.com/EmergenceAI/EM-AISoftwareFactory --skill eval-generator-emergenceai

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: eval-generator
Source: https://github.com/EmergenceAI/EM-AISoftwareFactory/tree/main/skills/eval-generator
Command: npx skills add https://github.com/EmergenceAI/EM-AISoftwareFactory --skill eval-generator-emergenceai

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires mcp__atlassian__jira, pytest, requests, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill automates the generation of evaluation tests from Jira acceptance criteria, streamlining the validation process for software development.

Core Features & Use Cases

Jira Acceptance Criteria Parsing: Extracts and parses acceptance criteria from Jira issues.
Automated Test Generation: Converts acceptance criteria into executable test cases.
Test Execution: Provides commands for running the generated tests.
Use Case: After implementing a feature, use this Skill to generate tests from the Jira issue and ensure the feature meets its acceptance criteria.

Quick Start

Use the eval-generator skill to generate evaluation tests for the Jira issue 'ABI-123'.

Frequently Asked Questions about eval-generator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I generate pytest tests from Jira acceptance criteria?▼

Generating pytest tests from Jira acceptance criteria involves automatically extracting issue details via the Jira API and converting them into executable test cases. This streamlines validation by ensuring new features meet their defined acceptance criteria.

Can I automate test generation for Jira issues using pytest?▼

Yes, automating test generation for Jira issues using pytest is possible by extracting acceptance criteria and converting them into executable test cases. This provides automated test validation for software development workflows requiring integration with Jira.

Do I need Jira API access to automate acceptance criteria parsing?▼

Yes, Jira API access is required to automate acceptance criteria parsing. The process relies on the mcp__atlassian__jira dependency to extract and parse acceptance criteria from Jira issues before generating executable test cases.

What is the best way to convert Jira acceptance criteria into executable test cases?▼

The best way to convert Jira acceptance criteria into executable test cases is through an automated generation process that parses Jira issues and outputs pytest scripts. This ensures features meet validation requirements by automating test execution.

What are the limitations of generating evaluation tests from Jira issues?▼

Limitations of generating evaluation tests from Jira issues include the dependency on the Jira API for parsing and the requirement of codebase knowledge for test generation. It is tailored for pytest execution and may not suit other testing frameworks.