Automate LLM prompt evaluation tests with a Go-based harness.
npx skills add https://github.com/C-Ross/LlamaOfFate --skill llm-eval-tests
Please help me install this Agent Skill. Skill: llm-eval-tests Source: https://github.com/C-Ross/LlamaOfFate/tree/main/.github/skills/llm-eval-tests Command: npx skills add https://github.com/C-Ross/LlamaOfFate --skill llm-eval-tests
LLM eval tests verify that prompts produce expected model behavior. Located in test/llmeval and implemented as a Go-based harness that renders prompts, executes evaluations, and collects results for analysis.
Run the llm-eval-tests suite with the llmeval build tag using go test.
High-intent search queries and answers about installing and using this skill.
Automate LLM prompt evaluation tests using a Go-based harness that renders prompts, executes evaluations, and collects results. Run the suite with the llmeval build tag via go test to verify expected model behavior.
Reproducible LLM prompt evaluation results are ensured by using configurable test cases, prompt templates, and build-tagged Go test harnesses. This framework systematically logs outcomes for positive and negative test cases.
LLM prompt evaluation supports both positive and negative test cases. The framework executes these configured cases to validate model behavior and aggregates performance metrics for analysis.
Configure test data and prompt templates for LLM evaluation by defining them within the test/llmeval directory. The Go-based harness renders these templates against test data to execute systematic evaluations.
You can run LLM evaluation tests without external dependencies because the skill operates with zero dependencies. It uses standard Go build tags to execute the test harness locally for immediate result summarization.
way-platformAutomate Go project builds and tasks using Mage build tool.
bennybennisonDefine expected outcomes and run pass@k evals to validate agent behavior changes.
Nuwanda04Analyze prompts, evaluate RAG systems, and design agentic workflows.
vuralserhat86Evaluate LLM applications with automated metrics, human feedback, and benchmarking.
