Create structured evaluation plans for GenAI applications with metrics and datasets.
npx skills add https://github.com/polikeiji/skills --skill eval-pipeline-plan
Please help me install this Agent Skill. Skill: eval-pipeline-plan Source: https://github.com/polikeiji/skills/tree/main/skills/eval-pipeline-plan Command: npx skills add https://github.com/polikeiji/skills --skill eval-pipeline-plan
This Skill turns a vague need to evaluate a GenAI application into a structured, implementation-ready evaluation plan with clear targets, datasets, metrics, and deployment guidance.
Ask this Skill to analyze your repository and produce a complete evaluation plan for your GenAI application, including datasets, metrics, model configuration, and Azure ML deployment guidance.
High-intent search queries and answers about installing and using this skill.
To create a structured GenAI evaluation plan, define the testing scope by identifying agents and end-to-end flows, then author markdown plans, test datasets, and model configuration files. This process organizes metrics and deployment guidance for repeatable evaluation workflows.
The best way to set up regression testing for LLM features is to operationalize scheduled evaluation runs using Azure Machine Learning batch runs and MLflow reporting, guided by a structured plan that defines test datasets and scoring metrics.
Yes, LangSmith works with Azure Machine Learning and MLflow for evaluation tracking. This approach centers workflows on LangSmith for tracing and Azure ML with MLflow for scheduled batch runs and visualizing regression test results.
You generate test datasets and model configuration files by analyzing your GenAI application repository to identify required agents and skills, then authoring structured markdown plans with corresponding schemas to ensure repeatable evaluation.
Yes, you can use this approach to define what to test and how to score GenAI agents. It identifies specific agents, skills, and end-to-end flows requiring testing, then produces organized plans with clear metrics and deployment documentation.
To operationalize GenAI evaluation pipelines with Azure ML, you need markdown frontmatter for plan structure, model configuration files, defined dataset schemas, and deployment utilities that support repeatable batch runs and MLflow reporting.
getkyoOrchestrate analyze, draft, critique, and verify sub-skills for README quality assurance.
Zack2626-okFetches and analyzes MLflow performance metrics including token usage, latency, and trace counts.
databricks-solutionsAutomate GenAI evaluation of ML agents with MLflow workflows.
choijinwonIntegrate MLflow into GenAI and ML projects with guided onboarding.
