instrumenting-with-mlflow-tracing

Name: instrumenting-with-mlflow-tracing
Availability: InStock
Author: aaronachermann

Instruments Python and TypeScript apps with MLflow tracing for LLM calls and retrievals.

Updated Apr 18, 2026

Search Tags:#python #llm #agents #observability #tracing #instrumentation #mlflow

Ready to run

npx skills add https://github.com/aaronachermann/PolentaEncoders --skill instrumenting-with-mlflow-tracing-aaronachermann

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: instrumenting-with-mlflow-tracing
Source: https://github.com/aaronachermann/PolentaEncoders/tree/main/.github/skills/instrumenting-with-mlflow-tracing
Command: npx skills add https://github.com/aaronachermann/PolentaEncoders --skill instrumenting-with-mlflow-tracing-aaronachermann

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Provides clear, actionable guidance to add MLflow tracing to Python and TypeScript agent and LLM applications so developers can see end-to-end behavior, latency, and span-level details for debugging and evaluation.

Core Features & Use Cases

Language-specific guidance: Step-by-step pointers for Python and TypeScript projects and how to detect the project type.
What to trace: Recommendations for tracing root operations, LLM calls, retrievals, tool calls, agent decisions, and external services while avoiding noisy low-value traces.
Verification and validation: Practical verification steps using mlflow.search_traces and MlflowClient to confirm traces and spans were recorded.
Operational guidance: Notes on tracking URI, sampling, autolog pitfalls, production deployment, distributed context propagation, and feedback collection for evaluation.

Quick Start

Instrument my Python agent to record MLflow traces for LLM calls, retrievals, and agent decisions and then verify traces using mlflow.search_traces.

Frequently Asked Questions about instrumenting-with-mlflow-tracing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I add MLflow tracing to my LangChain or LangGraph agents?▼

You instrument your Python application with MLflow tracing to capture traces and spans for LLM calls, retrievals, and agent decisions within LangChain or LangGraph workflows for end-to-end observability and debugging.

What is the best way to capture LLM calls and retrievals for observability?▼

Instrumenting applications with MLflow tracing captures LLM calls, retrievals, and tool calls as spans, providing span-level latency and behavior details for debugging and performance analysis.

Does MLflow tracing support both Python and TypeScript LLM applications?▼

Yes, MLflow tracing supports instrumenting both Python and TypeScript LLM applications, providing language-specific step-by-step guidance to capture traces for debugging and performance analysis.

How do I verify that MLflow traces and spans were recorded successfully?▼

You verify recorded MLflow traces and spans by executing practical verification steps using mlflow.search_traces and MlflowClient to confirm your LLM application behavior was captured correctly.

How should I configure MLflow tracing for production deployment and sampling?▼

Configuring MLflow tracing for production deployment involves setting the tracking URI, managing sampling rates, avoiding autolog pitfalls, and handling distributed context propagation for evaluation runs.

Why am I getting noisy low-value traces when instrumenting my LLM workflow?▼

Noisy low-value traces in your LLM workflow occur when instrumentation captures unnecessary low-level operations; you should selectively trace root operations, LLM calls, retrievals, and tool calls to avoid this.