measure-agent-task-success
OfficialMeasure and ensure AI agent task completion accuracy.
Software Engineering#AI agent#task completion#AI monitoring#performance evaluation#agent benchmarking
AuthorContextJet-ai
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill assesses whether an AI agent successfully completes its task end-to-end, not just individual steps, by scoring outcomes and task paths.
Core Features & Use Cases
- End-to-End Task Measurement: Evaluate whether the agent achieves the correct end state for each task.
- Task Success Rate Analysis: Define "success" explicitly per task type, checking if conditions are met.
- Diagnostic Metrics: Review the steps taken to complete the task, including tool-call success rate and recovery from errors.
- Evaluation Frameworks: Use task datasets with defined success conditions, run agents in a sandbox, and analyze trace data for failures.
- CI and Real-time Monitoring: Integrate success rate checks into CI and monitor real tasks for drift alerts.
Quick Start
Measure the success rate of your AI agent on the task 'book a flight', ensuring it reaches the correct end state.
Dependency Matrix
Required Modules
None requiredComponents
scriptsreferences
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: measure-agent-task-success Download link: https://github.com/ContextJet-ai/awesome-llm-observability/archive/main.zip#measure-agent-task-success Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 620,000+ vetted skills library on demand.