llm-obs-eval
OfficialRun end-to-end LLM evaluation experiments.
Authorspeqqai
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This skill enables automated, instrumented evaluation of agent behavior and tool usage using Datadog LLM Observability Experiments.
Core Features & Use Cases
- End-to-end experiment orchestration across multiple scenarios (greeting, product_question, feature_question).
- Data collection of per-turn prompts, responses, tool calls, and token usage with Datadog spans and export data.
- Programmatic result querying and export via Datadog's Export API to analyze costs, tokens, and tool performance.
Quick Start
Run the Python experiment runner to launch all scenarios and view the resulting Datadog Experiment URL.
Dependency Matrix
Required Modules
None requiredComponents
scriptsreferences
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: llm-obs-eval Download link: https://github.com/speqqai/claude/archive/main.zip#llm-obs-eval Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.