llm-eval-tests

Automate LLM prompt evaluation tests with a Go-based harness.

2|Updated Aug 16, 2025

Search Tags:#testing #llm #evaluation #go #prompts #prompt-engineering #test-automation

Ready to run

npx skills add https://github.com/C-Ross/LlamaOfFate --skill llm-eval-tests

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: llm-eval-tests
Source: https://github.com/C-Ross/LlamaOfFate/tree/main/.github/skills/llm-eval-tests
Command: npx skills add https://github.com/C-Ross/LlamaOfFate --skill llm-eval-tests

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

LLM eval tests verify that prompts produce expected model behavior. Located in test/llmeval and implemented as a Go-based harness that renders prompts, executes evaluations, and collects results for analysis.

Core Features & Use Cases

Configurable test cases and prompts for systematic LLM evaluation
Build-tagged Go test harness with llmeval support for reproducible results
Prompt rendering, evaluation execution, and result summarization
Supports positive/negative test cases and aggregated performance metrics

Quick Start

Run the llm-eval-tests suite with the llmeval build tag using go test.

Frequently Asked Questions about llm-eval-tests

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate LLM prompt evaluation tests in Go?▼

Automate LLM prompt evaluation tests using a Go-based harness that renders prompts, executes evaluations, and collects results. Run the suite with the llmeval build tag via go test to verify expected model behavior.

What is the best way to ensure reproducible LLM prompt evaluation results?▼

Reproducible LLM prompt evaluation results are ensured by using configurable test cases, prompt templates, and build-tagged Go test harnesses. This framework systematically logs outcomes for positive and negative test cases.

Does LLM prompt evaluation support positive and negative test cases?▼

LLM prompt evaluation supports both positive and negative test cases. The framework executes these configured cases to validate model behavior and aggregates performance metrics for analysis.

How to configure test data and prompt templates for LLM evaluation?▼

Configure test data and prompt templates for LLM evaluation by defining them within the test/llmeval directory. The Go-based harness renders these templates against test data to execute systematic evaluations.

Can I run LLM evaluation tests without setting up complex external dependencies?▼

You can run LLM evaluation tests without external dependencies because the skill operates with zero dependencies. It uses standard Go build tags to execute the test harness locally for immediate result summarization.