Evaluate and tune Myco LLM prompts using structured fixtures and EVAL_MODEL.
npx skills add https://github.com/goondocks-co/myco --skill prompt-tune
Please help me install this Agent Skill. Skill: prompt-tune Source: https://github.com/goondocks-co/myco/tree/main/.agents/skills/prompt-tune Command: npx skills add https://github.com/goondocks-co/myco --skill prompt-tune
This Skill helps development teams tune and evaluate Myco's LLM prompts to improve reliability, safety, and usefulness across internal AI workflows.
Run the prompt tuning workflow against your prompt fixtures to start evaluating and iterating on prompts.
High-intent search queries and answers about installing and using this skill.
You evaluate LLM prompts by running a tuning workflow against structured test fixtures stored under tests/prompts, which surfaces weaknesses and edge cases by comparing model outputs against expected results.
Prompt tuning iterates on prompt configurations while model comparison benchmarks different LLMs using the EVAL_MODEL variable, identifying best-performing setups by testing iterations against your structured fixtures.
Start prompt testing by organizing test fixtures and scripts under tests/prompts, then executing the tuning workflow to apply structured evaluation across development workflows for Myco's AI agents.
Yes, prompt evaluation requires fixtures and testing scripts stored under tests/prompts, a model selection defined by the EVAL_MODEL variable, and a documented methodology in references for traceability.
Document prompt changes and learning in a reference log to enable traceability and reuse, storing a documented methodology in references so structured evaluation outcomes are recorded for future iterations.
Yes, you compare different models and prompt iterations using the same structured fixtures by selecting an evaluation model with the EVAL_MODEL variable, identifying best-performing configurations across Myco's LLMs.
pv-udpvRun RED-GREEN-REFACTOR tests to harden skill documentation and close loopholes.
tbroadleyExtract samples, messages, events, and metrics from Inspect AI .eval logs.
coreymayprayAudit AI models with evaluation, calibration, and SHAP analysis.
Dianshu-LiaoImplement LLM evaluation with automated metrics and LLM-as-judge patterns.
