analyzing-mlflow-session

Analyze MLflow chat session traces to identify conversation issues.

Updated Mar 25, 2026

Search Tags:#debugging #analysis #trace #assessment #session #chat #mlflow

Ready to run

npx skills add https://github.com/JingyiWng/databricks_ai_dev_kit_price_watcher --skill analyzing-mlflow-session-jingyiwng

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: analyzing-mlflow-session
Source: https://github.com/JingyiWng/databricks_ai_dev_kit_price_watcher/tree/main/.claude/skills/analyze-mlflow-chat-session
Command: npx skills add https://github.com/JingyiWng/databricks_ai_dev_kit_price_watcher --skill analyzing-mlflow-session-jingyiwng

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

This Skill helps debug multi‑turn chat conversations by analyzing MLflow session traces, extracting inputs, outputs, and assessments to pinpoint where a dialogue went wrong.

Core Features & Use Cases

Schema Discovery: Identify trace schema and root span attributes for a given session.
Efficient Extraction: Pull relevant input/output fields and assessments across all traces without fetching full JSON for each turn.
Assessment Insight: Filter out scorer errors and use assessment rationales to understand quality issues.
CLI Guidance: Provides safe command patterns for handling metadata keys with dots and managing large CLI outputs.
Scripts: Ready‑to‑run bash scripts for schema discovery and turn inspection.

Quick Start

Ask the skill to analyze a specific MLflow session by providing the experiment ID and session ID.

Frequently Asked Questions about analyzing-mlflow-session

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I debug multi-turn chat issues using MLflow session traces?▼

To debug multi-turn chat issues with MLflow session traces, you can analyze conversation traces by specifying the experiment ID and session ID to extract inputs, outputs, and assessments across all turns to pinpoint dialogue failures.

How do I extract assessment rationales from MLflow chat sessions without fetching full JSON?▼

You can extract assessment rationales from MLflow chat sessions without full JSON payloads by using CLI filtering with escaped metadata keys to pull relevant input/output fields and assessments across all traces.

How do I handle metadata keys with dots when querying MLflow traces in the CLI?▼

When querying MLflow traces in the CLI, handle metadata keys containing dots by using safe command patterns with escaped metadata keys to prevent parsing errors and manage large outputs.

What is the best way to identify the root span schema for an MLflow chat session?▼

The best way to identify the root span schema for an MLflow chat session is to run a schema discovery script that inspects trace attributes and root span structures for the specified session ID.

Why does my MLflow chat session assessment return scorer errors?▼

MLflow chat session assessments may return scorer errors due to malformed inputs or evaluation failures, which can be filtered out during trace analysis to focus on valid assessment rationales and quality issues.

Do I need to provide a session ID to analyze MLflow chat traces?▼

Yes, you need to provide both the experiment ID and the session ID to analyze MLflow chat traces, as the Skill relies on these identifiers to filter traces and extract conversation data across all turns.