skills.rest
reaatech
reaatechCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

Trajectory Evaluation

Evaluate multi-turn agent trajectories for coherence, goal completion, and turn-level quality.

Updated Apr 19, 2026
Search Tags:#quality#regression#trajectory#coherence#evaluate#goal-completion
One-click install
npx skills add https://github.com/reaatech/agent-eval-harness --skill trajectory-evaluation
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: Trajectory Evaluation
Source: https://github.com/reaatech/agent-eval-harness/tree/main/skills/trajectory-eval
Command: npx skills add https://github.com/reaatech/agent-eval-harness --skill trajectory-evaluation

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Trajectory evaluation assesses the quality of complete agent executions—multi-turn conversations with tool invocations and task completion—by measuring coherence, goal completion, and turn-level quality.

Core Features & Use Cases

  • Holistic evaluation of conversation flow and outcome.
  • Coherence assessment, goal completion tracking, and turn-level analysis.
  • Regression detection across releases and comparisons against golden trajectories.
  • Use cases include validating agent performance in multi-turn tasks and surfacing quality issues for tuning.

Quick Start

Evaluate a trajectory file with the CLI or programmatic API to generate a comprehensive quality report and per-turn diagnostics.

Frequently Asked Questions about Trajectory Evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate multi-turn agent trajectories for quality and coherence?▼

You can evaluate multi-turn agent trajectories by processing trajectory JSONL outputs and tool-invocation traces to measure coherence, goal completion, and turn-level quality. The Skill generates a comprehensive quality report and per-turn diagnostics.

What is trajectory regression detection and when do I need it?▼

Trajectory regression detection compares current multi-turn agent executions against golden trajectories to identify quality degradation across releases. You need it when validating agent performance to ensure new updates maintain conversation flow and task completion.

Can I compare agent trajectories against a golden trajectory benchmark?▼

Yes, you can compare agent trajectories against golden trajectory benchmarks. The Skill provides dedicated evaluate and compare operations that quantify quality differences and exports structured results for downstream regression analysis.

Does trajectory evaluation work with tool-invocation traces and JSONL outputs?▼

Yes, trajectory evaluation works directly with trajectory JSONL outputs and tool-invocation traces. It applies holistic evaluation to conversation flow and outcomes, surfacing quality issues for agent tuning.

How do I detect quality regressions in multi-turn agent releases?▼

Detect quality regressions by running the compare operation on trajectory JSONL outputs across agent releases to measure coherence and goal completion against golden trajectories. This surfaces turn-level quality issues for tuning.

Related Skills

Explore More Data & Analytics Skills→
sergicalsergical
1

session-reader

Parse and analyze pi agent session JSONL files for execution insights.

Community
Intermediate
wimbleroboticswimblerobotics
18

mppi-trajectory-visualization

Visualize MPPI trajectory rollouts as color-coded markers in RViz.

Community
Intermediate
mostafa-ismail-2004mostafa-ismail-2004

skill-comply

Compare expected behavioral sequences to observed tool-call traces for compliance.

Community
Advanced
carterwayneskhizeinecarterwayneskhizeine
19

hermes-atropos-environments

Implements Hermes-AgentRL environments integrating multi-turn tool-calling loops with the Atropos training framework.

Community
Advanced