skills.rest
Blackkadder
BlackkadderCOMMUNITY
Explore all skills by this author →
GitHub

instrumenting-with-mlflow-tracing

Instrument Python and TypeScript applications with MLflow tracing for observability.

Updated Mar 19, 2026
Search Tags:#typescript#python#llm#observability#tracing#instrumentation#mlflow
One-click install
npx skills add https://github.com/Blackkadder/databricks-apps-and-agents-workshop --skill instrumenting-with-mlflow-tracing-blackkadder
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: instrumenting-with-mlflow-tracing
Source: https://github.com/Blackkadder/databricks-apps-and-agents-workshop/tree/main/.claude/skills/instrumenting-with-mlflow-tracing
Command: npx skills add https://github.com/Blackkadder/databricks-apps-and-agents-workshop --skill instrumenting-with-mlflow-tracing-blackkadder

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps developers and engineers instrument Python and TypeScript applications, agents, and LLM integrations with MLflow tracing so they can capture spans, monitor LLM calls, retrievals, tool calls, and external operations for observability and debugging.

Core Features & Use Cases

  • Language-specific guidance: Directs users to the appropriate instrumentation guide for Python or TypeScript projects and suggests how to detect project type.
  • What to trace and what to skip: Recommends high-value trace points (root operations, LLM calls, retrievals, tool calls, agent decisions, external services) and advises against tracing low-value utility operations to reduce noise.
  • Verification and production readiness: Provides verification steps using mlflow.search_traces, instructions for feedback collection, sampling and deployment considerations, and references for advanced and distributed tracing patterns.
  • Use case: Instrument an agent before running an evaluation to capture trace data for LLM latency, token usage, retrieval relevance, and decision reasoning.

Quick Start

Instrument your application by adding MLflow tracing to key operations, run the application to generate traces, and confirm traces are visible using mlflow.search_traces().

Frequently Asked Questions about instrumenting-with-mlflow-tracing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add MLflow tracing to Python or TypeScript LLM applications?▼

To add MLflow tracing, instrument your Python or TypeScript application code to record spans for LLM calls, retrievals, tool calls, and external operations. This captures observability data for debugging and evaluation workflows.

What operations should I trace in my LLM agent for effective observability?▼

You should trace high-value operations in your LLM agent such as root operations, LLM calls, retrievals, tool calls, agent decisions, and external service requests. Avoid tracing low-value utility operations to reduce noise and maintain clear observability.

Can I use MLflow tracing with frameworks like LangChain, OpenAI, and AutoGen?▼

Yes, MLflow tracing supports integrations including LangChain, LangGraph, OpenAI, DSPy, CrewAI, and AutoGen. It provides language-specific guidance to instrument these frameworks and capture spans for observability.

How do I verify that MLflow tracing is working in my application?▼

You can verify MLflow tracing by running your application to generate traces and then confirming they are visible using the mlflow.search_traces() function. This ensures your instrumentation correctly records spans and operations.

Does MLflow tracing support sampling configuration for production deployments?▼

Yes, MLflow tracing includes sampling configuration and deployment considerations for production environments. It provides reference guides for advanced patterns and distributed tracing to manage observability data at scale.

Why should I instrument an agent with MLflow tracing before running an evaluation?▼

Instrumenting an agent with MLflow tracing before evaluation captures trace data for LLM latency, token usage, retrieval relevance, and decision reasoning. This observability is essential for accurately assessing agent performance.

Related Skills

Explore More Software Engineering Skills→
ice-droid-99ice-droid-99

mlflow-onboarding

Detect GenAI versus traditional ML use cases and guide MLflow onboarding.

Community
Intermediate
aaronachermannaaronachermann

instrumenting-with-mlflow-tracing

Instruments Python and TypeScript apps with MLflow tracing for LLM calls and retrievals.

Community
Intermediate
alessandro9110alessandro9110
4

instrumenting-with-mlflow-tracing

Instrument Python and TypeScript code with MLflow Tracing for LLM calls and tool usage.

Community
Intermediate
ronniepinnellronniepinnell

langfuse

Automate Langfuse CLI operations and implement LLM tracing in applications.

Community
Advanced