eval-session-classify

Classify Datadog assistant sessions into satisfaction verdicts using session_id and LLM Observability traces.

150|23|Updated Feb 3, 2026
One-click install
npx skills add https://github.com/datadog-labs/agent-skills --skill eval-session-classify
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: eval-session-classify
Source: https://github.com/datadog-labs/agent-skills/tree/main/dd-llmo/eval-session-classify
Command: npx skills add https://github.com/datadog-labs/agent-skills --skill eval-session-classify

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill enables automated classification of user satisfaction and intent in Datadog assistant sessions by analyzing session identifiers and related observability traces to produce a structured verdict.

Core Features & Use Cases

  • Input-driven session classification using a session_id and ROUTE_CONTEXT-derived signals (pre-, during-, and post-session data).
  • Integration of LLM Observability evaluations, trace data, and RUM signals to produce robust verdicts.
  • Output structured satisfaction verdicts with actionable recommendations for product teams.

Quick Start

Provide a standalone classification for a session_id to determine whether the user’s intent was satisfied.

Frequently Asked Questions about eval-session-classify

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I classify user intent satisfaction using LLM observability traces in Datadog?

Session intent satisfaction is classified by analyzing a session_id alongside Datadog LLM Observability traces and RUM signals across pre-, during-, and post-event windows to produce a structured satisfaction verdict.

What is the best way to automate session classification verdicts for Datadog assistant sessions?

Automated session classification verdicts are generated by integrating ROUTE_CONTEXT details, LLM Observability evaluations, and RUM signals to robustly measure whether user intent was satisfied.

How does the classification pipeline handle missing LLM observability traces during session evaluation?

The classification pipeline validates the presence of required data sources and handles missing LLM observability traces safely to ensure robust satisfaction verdicts without processing failures.

Can I use RUM signals and session_id to determine if a user's intent was met during a Datadog session?

Yes, you can provide a standalone session_id to measure whether user intent was satisfied by integrating RUM signals and LLM Observability traces across the session timeline.

Why does my session satisfaction classification require pre-, during-, and post-event data windows?

Pre-, during-, and post-event data windows are required to capture the complete session context, ensuring the classification pipeline produces robust and accurate satisfaction verdicts.