skills.rest
Clay-HHK
Clay-HHKCOMMUNITY
Explore all skills by this author →
GitHub

phoenix-observability

Collect and analyze LLM execution traces and evaluation results via OpenTelemetry.

2|Updated Apr 12, 2026
Search Tags:#monitoring#llm#evaluation#observability#tracing#opentelemetry#phoenix
One-click install
npx skills add https://github.com/Clay-HHK/claude-config --skill phoenix-observability-clay-hhk
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: phoenix-observability
Source: https://github.com/Clay-HHK/claude-config/tree/main/skills/AI-research-SKILLs/17-observability/phoenix
Command: npx skills add https://github.com/Clay-HHK/claude-config --skill phoenix-observability-clay-hhk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Phoenix solves the lack of visibility into LLM-powered applications by collecting detailed traces, evaluations, and dataset-backed experiments so teams can debug, measure quality, and monitor performance over time.

Core Features & Use Cases

  • Tracing: Capture OpenTelemetry traces and spans from OpenAI, LangChain, LlamaIndex, Anthropic, and other instrumented clients to diagnose request flows and failures.
  • Evaluation: Run built-in and custom LLM evaluators (hallucination, relevance, toxicity) and log evaluation results back to projects for regression testing.
  • Datasets & Experiments: Version datasets, run experiments to compare prompts and models, and aggregate metrics for A/B comparisons and CI checks.
  • Monitoring & Deployment: Self-host with PostgreSQL or SQLite, expose REST/gRPC endpoints, and support Docker/Kubernetes production deployments for real-time dashboards and alerts.
  • Use Case: Debug a production chatbot by tracing a failed conversation, running automated evaluations on recent spans, and rolling back to a safer prompt variant based on experiment results.

Quick Start

Start a Phoenix server locally and configure your LLM client to send OpenTelemetry traces to http://localhost:6006/v1/traces so you can inspect spans and run evaluations immediately.

Frequently Asked Questions about phoenix-observability

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I trace LLM execution flows and debug failures in production?▼

You can trace LLM execution flows by collecting OpenTelemetry traces and spans from clients like OpenAI, LangChain, and Anthropic to diagnose request flows and pinpoint production failures.

What is LLM observability and when do I need it for my application?▼

LLM observability is the process of collecting traces, evaluations, and experiments to measure quality and monitor performance. You need it when you lack visibility into your LLM-powered application's behavior.

How do I run automated evaluations on LLM spans for regression testing?▼

You run automated evaluations on LLM spans by applying built-in and custom evaluators such as hallucination, relevance, and toxicity checks, then logging the evaluation results back to your projects for regression testing.

Does Phoenix observability support OpenTelemetry trace ingestion from LlamaIndex?▼

Yes, Phoenix observability supports OpenTelemetry trace ingestion from LlamaIndex, allowing you to capture and inspect spans to diagnose request flows and failures across your LLM applications.

Can I use SQLite or PostgreSQL for self-hosted LLM monitoring deployments?▼

Yes, you can use SQLite or PostgreSQL as backends for self-hosted LLM monitoring deployments, exposing REST and gRPC endpoints to support real-time dashboards and alerts.

What's the best way to compare prompts and models for LLM applications?▼

The best way to compare prompts and models is by versioning datasets and running experiments to aggregate metrics, enabling A/B comparisons and CI checks for your LLM applications.

Related Skills

Explore More Software Engineering Skills→
giosuetedeschi-specgiosuetedeschi-spec

langfuse

Track LLM traces, spans, and metrics for production AI applications.

Community
Advanced
teegin-gteegin-g

querying-mlflow-metrics

Fetch aggregated MLflow tracking metrics from servers in table or JSON formats.

Community
Intermediate
h4vzzh4vzz
33

model-deployment

Deploy trained ML models as REST services on Docker or Kubernetes.

Community
Advanced
qcmuuqcmuu

phoenix-observability

Capture OpenTelemetry traces and evaluation results for LLM debugging.

Community
Advanced