skills.rest
supercent-io
supercent-ioOFFICIAL
Explore all skills by this author →
GitHub

langsmith

Instrument LLM pipelines with tracing and run evaluations via LangSmith.

Updated Aug 23, 2026
Search Tags:#debugging#evaluation#prompt management#observability#langsmith#llm tracing
One-click install
npx skills add https://github.com/supercent-io/skills-template --skill langsmith-supercent-io
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: langsmith
Source: https://github.com/supercent-io/skills-template/tree/main/.agent-skills/langsmith
Command: npx skills add https://github.com/supercent-io/skills-template --skill langsmith-supercent-io

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides comprehensive tools for monitoring, tracing, evaluating, and managing Large Language Model (LLM) applications and AI agents, ensuring their quality and performance.

Core Features & Use Cases

  • LLM Tracing: Instrument and track LLM calls, tool usage, and agentic workflows end-to-end.
  • Evaluation: Run offline and online evaluations using datasets and LLM-as-judge capabilities.
  • Prompt Management: Version and manage prompts efficiently using the Prompt Hub.
  • Use Case: Debug a failing AI agent by inspecting its execution traces in LangSmith, identify where it went wrong, and use evaluation datasets to test fixes.

Quick Start

Install the LangSmith SDK and set your API key, then instrument your Python code with the @traceable decorator.

Frequently Asked Questions about langsmith

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I trace LLM calls and agent workflows for debugging?▼

Trace LLM calls and agent workflows by instrumenting your Python code with the `@traceable` decorator. This captures end-to-end execution data, allowing you to inspect tool usage and identify exactly where an AI agent went wrong.

What is LLM observability and when do I need it?▼

LLM observability is the process of monitoring, tracing, and evaluating LLM applications to ensure quality. You need it when managing complex AI agents, requiring insights into execution traces and prompt performance to fix failures.

How do I run offline and online evaluations for LLM applications?▼

Run offline and online evaluations by creating datasets for regression testing and utilizing LLM-as-judge capabilities. This allows you to systematically test fixes against known data and measure LLM performance.

Can I manage and version prompts within my LLM pipeline?▼

You can manage and version prompts efficiently using the Prompt Hub. This feature allows you to track prompt iterations and integrate them into your LLM pipelines for consistent application behavior.

Do I need an API key and SDK installation to set up LLM tracing?▼

You must install the LangSmith SDK and configure an API key to enable LLM tracing. This setup is required to integrate the observability platform with your application and start capturing execution data.

Related Skills

Explore More Software Engineering Skills→
lguibrlguibr

langsmith-trace

Instrument applications to generate LangSmith traces for LLM observability.

Community
Advanced
Avi977Avi977

error-analysis

Categorize LLM pipeline failures by analyzing structured trace data.

Community
Advanced
maragudkmaragudk
10

llm-as-a-judge

Build and validate LLM-as-Judge evaluators for automated quality assessment.

Official
Advanced
tianhao909tianhao909
1

phoenix-observability

Provides an open-source AI observability platform for LLM tracing, evaluation, and monitoring.

Community
Advanced