skills.rest
drdave-flexnetos
drdave-flexnetosCOMMUNITY
Explore all skills by this author →
GitHub

llm-evaluation

Evaluate LLM prompts and applications for correctness, safety, and performance.

Updated Jan 12, 2026
Search Tags:#rag#llm evaluation#latency#json validation#promptfoo#prompt testing#trulens
One-click install
npx skills add https://github.com/drdave-flexnetos/ripple-env --skill llm-evaluation-drdave-flexnetos
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llm-evaluation
Source: https://github.com/drdave-flexnetos/ripple-env/tree/main/.claude/skills/llm-evaluation
Command: npx skills add https://github.com/drdave-flexnetos/ripple-env --skill llm-evaluation-drdave-flexnetos

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

It helps teams find which LLM prompts and apps actually produce reliable, safe, and cost-effective outputs instead of relying on ad-hoc testing.

Core Features & Use Cases

  • Prompt evaluation with promptfoo: Run systematic prompt test suites with assertions like JSON validity, rubric-based quality, and latency/cost thresholds for robotics agents and assistants.
  • Application feedback and monitoring with TruLens: Trace LLM chains and compute feedback signals such as groundedness, relevance, and custom safety/format checks for RAG and agent workflows.
  • Evals-style testing across frameworks: Define repeatable evaluation pipelines that can be integrated into CI/CD to gate deployments based on measurable metrics.

Use case example: Before deploying a robot command parser, evaluate multiple provider models and prompt variants with JSON structure checks, rubric scoring, and latency thresholds, then require CI to fail if quality gates are not met.

Quick Start

Tell your AI to set up a promptfoo config that runs evaluation tests for a robot command parser and includes assertions for JSON validity, rubric-based correctness, and a latency threshold.

Frequently Asked Questions about llm-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up automated LLM evaluation tests for prompt variants?▼

Automated LLM evaluation uses tools like promptfoo to run systematic test suites with assertions for JSON validity, rubric-based quality scoring, and latency thresholds. Configure test cases to compare prompt variants and gate deployments based on measurable metrics.

What is RAG groundedness measurement and how does it work?▼

RAG groundedness measurement evaluates whether LLM responses are factually based on retrieved context. Using TruLens, it traces LLM chains and computes feedback signals like groundedness and relevance to ensure responses stay anchored to source documents.

Can I integrate prompt testing into CI/CD pipelines to block bad deployments?▼

Prompt testing integrates into CI/CD by defining repeatable evaluation pipelines with quality gates. Configure promptfoo test suites with assertions for correctness, safety, and latency thresholds, then fail the CI pipeline if quality metrics are not met.

Does promptfoo support JSON validation and latency threshold assertions?▼

Promptfoo supports JSON validation and latency threshold assertions within its test configurations. You can define assertions that check structured output validity, enforce rubric-based correctness scoring, and set maximum latency or cost limits per evaluation.

What's the best way to monitor LLM application performance across different providers?▼

Monitoring LLM applications across providers requires tracing chains and computing feedback signals with TruLens. It tracks groundedness, relevance, and custom safety checks continuously, enabling ongoing evaluation of response quality and cost performance across models.

How do I evaluate if a robot command parser prompt produces safe and correct outputs?▼

Evaluating robot command parser prompts involves running test suites that apply JSON structure checks, rubric-based correctness scoring, and safety assertions. Use promptfoo to compare provider models and prompt variants against predefined quality gates before deployment.

Related Skills

Explore More Education & Research Skills→
VjayRamVjayRam

dspy-expert

Define DSPy signatures and modules with deterministic evaluation across splits.

Community
Advanced
qiphonqiphon
2

Verification & Quality Assurance

Score truth, validate code, and rollback failed changes via CLI.

Community
Intermediate
JNZader-VaultJNZader-Vault

langchain

Build LLM chains, agents, and RAG systems with LangChain and LangGraph.

Official
Advanced
HuuBarHuuBar

promptfoo-evaluation

Automates LLM evaluation configuration and execution using Promptfoo framework.

Community
Intermediate