analyzing-mlflow-session

What problem does it solve?

This Skill analyzes an MLflow session — a sequence of traces from a multi-turn chat conversation or interaction. It helps you debug a chat conversation, review a session or chat history, find where a multi-turn chat went wrong, or analyze patterns across turns. Traces within a session are linked by a shared session ID stored in trace metadata.

Core Features & Use Cases

• Reconstruct conversations by collecting per-turn inputs and outputs from root span attributes (e.g., mlflow.spanInputs/mlflow.spanOutputs) and linking traces by session ID.
• Identify schema and key fields, inspect assessments, and audit session-level vs per-turn evaluations to pinpoint where issues originate.
• Analyze multi-turn session quality across turns, detect drift or inconsistencies, and surface actionable insights for debugging.

Quick Start

Run the discover_schema.sh script to identify the session inputs/outputs, then inspect individual turns with the inspect_turn.sh script.