skills.rest
informatico-madrid
informatico-madridCOMMUNITY
Explore all skills by this author →
GitHub

langsmith-observability

Traces, evaluates, and monitors LLM applications for performance and errors.

1|Updated Feb 10, 2026
Search Tags:#debugging#monitoring#llm#evaluation#observability#tracing
One-click install
npx skills add https://github.com/informatico-madrid/Architect-Expert-Gap-Forge --skill langsmith-observability-informatico-madrid
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: langsmith-observability
Source: https://github.com/informatico-madrid/Architect-Expert-Gap-Forge/tree/main/.github/skills/langsmith
Command: npx skills add https://github.com/informatico-madrid/Architect-Expert-Gap-Forge --skill langsmith-observability-informatico-madrid

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires langsmith, and includes references (resource) components.

What problem does it solve?

This Skill provides comprehensive tools for understanding, debugging, and improving Large Language Model (LLM) applications by offering detailed tracing, evaluation, and monitoring capabilities.

Core Features & Use Cases

  • Tracing: Capture and visualize the execution flow of LLM calls, chains, and agents.
  • Evaluation: Systematically test and score model performance against datasets using built-in or custom metrics.
  • Monitoring: Track production LLM systems for performance, errors, and costs.
  • Use Case: Debug a complex agent by tracing its decision-making process, identify a specific prompt that leads to poor performance, and then create an evaluation dataset to systematically test improvements.

Quick Start

Use the langsmith-observability skill to trace your OpenAI API calls by setting the LANGSMITH_TRACING environment variable to true.

Frequently Asked Questions about langsmith-observability

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I trace LLM calls and agent decisions to debug poor performance?▼

Trace LLM calls by setting the LANGSMITH_TRACING environment variable to true. This captures and visualizes the execution flow of prompts, chains, and agents to pinpoint specific decisions causing poor performance.

What is LLM observability and when do I need it for my application?▼

LLM observability is the systematic tracing, evaluation, and monitoring of AI applications. You need it to understand execution flows, systematically test performance against datasets, and track production latency, cost, and errors.

Can I monitor production LLM systems for cost and latency using LangSmith?▼

Yes, you can monitor production LLM systems to track performance metrics, errors, and costs. LangSmith provides built-in capabilities to oversee latency and operational health across major LLM providers.

Does LangSmith observability work with LangChain and major LLM providers?▼

Yes, LangSmith observability integrates with major LLM providers and frameworks like LangChain. This compatibility allows you to trace chains and agents directly within your existing application architecture.

How do I systematically evaluate model performance against custom datasets?▼

Systematically evaluate model performance by creating an evaluation dataset and applying built-in or custom metrics. This allows you to score model outputs, test improvements, and identify prompts causing poor results.

Related Skills

Explore More Software Engineering Skills→
HemantSudarshanHemantSudarshan

langfuse

Instrument LLM calls for tracing and observability in Python or TypeScript pipelines.

Community
Advanced
ethan-rioethan-rio

arize-prompt-optimization

Optimize LLM prompts using Arize trace data and evaluation scores.

Community
Advanced
JNZader-VaultJNZader-Vault

langchain

Build LLM chains, agents, and RAG systems with LangChain and LangGraph.

Official
Advanced
giosuetedeschi-specgiosuetedeschi-spec

langfuse

Track LLM traces, spans, and metrics for production AI applications.

Community
Advanced