skills.rest
Orchestra-Research
Orchestra-ResearchOFFICIAL
Explore all skills by this author →
GitHub

phoenix-observability

Trace, evaluate, and monitor LLM applications with OpenTelemetry.

11.5k|842|Updated Nov 3, 2025
Search Tags:#monitoring#mlops#evaluation#observability#phoenix#llm tracing
One-click install
npx skills add https://github.com/Orchestra-Research/AI-Research-SKILLs --skill phoenix-observability
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: phoenix-observability
Source: https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/17-observability/phoenix
Command: npx skills add https://github.com/Orchestra-Research/AI-Research-SKILLs --skill phoenix-observability

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires arize-phoenix, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides deep visibility into AI applications, enabling debugging, evaluation, and monitoring of LLM performance and behavior.

Core Features & Use Cases

  • LLM Tracing: Visualize the entire lifecycle of LLM requests, including prompts, responses, and intermediate steps.
  • Evaluation: Run systematic tests and quality assessments on your LLM outputs against datasets or references.
  • Monitoring: Track key metrics and identify issues in production AI systems in real-time.
  • Use Case: When your chatbot starts giving nonsensical answers, use Phoenix to trace the problematic conversation, identify where the LLM went wrong, and pinpoint the faulty prompt or context.

Quick Start

Launch the Phoenix observability UI by running phoenix serve in your terminal.

Frequently Asked Questions about phoenix-observability

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I trace LLM requests to debug bad chatbot responses?▼

LLM tracing visualizes the entire request lifecycle, including prompts, responses, and intermediate steps, enabling you to pinpoint exactly where your LLM went wrong and identify faulty context.

Can I run systematic evaluations on my LLM outputs against a dataset?▼

Yes, you can run systematic evaluations to perform quality assessments on your LLM outputs against datasets or references, ensuring your application meets expected performance standards.

Does LLM observability with OpenTelemetry support LangChain and LlamaIndex?▼

Yes, this open-source observability solution supports integration with various LLM frameworks including OpenAI, LangChain, and LlamaIndex using OpenTelemetry for comprehensive tracing.

What is the best way to monitor production AI systems for real-time issues?▼

AI monitoring tracks key metrics and identifies issues in production AI systems with real-time insights, allowing you to observe behavior and system health continuously.

How do I start the observability UI to visualize my LLM traces?▼

You can launch the observability UI by running the `phoenix serve` command in your terminal to immediately visualize traces and monitor your application.

Related Skills

Explore More Software Engineering Skills→
Supporter09Supporter09

phoenix-observability

Capture and analyze OpenTelemetry-based LLM traces for debugging and quality evaluation.

Community
Advanced
ihatesea69ihatesea69
3

phoenix-observability

Provides an open-source platform for AI observability with LLM tracing, evaluation, and monitoring.

Community
Advanced
giosuetedeschi-specgiosuetedeschi-spec

langfuse

Track LLM traces, spans, and metrics for production AI applications.

Community
Advanced
qcmuuqcmuu

phoenix-observability

Capture OpenTelemetry traces and evaluation results for LLM debugging.

Community
Advanced