skills.rest
tadod12
tadod12COMMUNITY
Explore all skills by this author →
GitHub

phoenix-observability

Trace, evaluate, and monitor LLM applications with OpenTelemetry.

Updated Mar 18, 2026
Search Tags:#monitoring#production#llm#evaluation#observability#tracing#opentelemetry
One-click install
npx skills add https://github.com/tadod12/fraud-detection-research --skill phoenix-observability-tadod12
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: phoenix-observability
Source: https://github.com/tadod12/fraud-detection-research/tree/main/.agent/skills/17-observability/phoenix
Command: npx skills add https://github.com/tadod12/fraud-detection-research --skill phoenix-observability-tadod12

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires arize-phoenix>=12.0.0, and includes references (resource) components.

What problem does it solve?

Open-source AI observability and evaluation platform for LLM applications with tracing, evaluation, datasets, experiments, and real-time monitoring. Use when debugging LLM applications with detailed traces, evaluating model outputs on datasets, or monitoring production AI systems in real-time.

Core Features & Use Cases

  • Tracing: OpenTelemetry-based trace collection for any LLM framework
  • Evaluation: LLM-as-judge evaluators for quality assessment
  • Datasets: Versioned test sets for regression testing
  • Experiments: Compare prompts, models, and configurations
  • Playground: Interactive prompt testing with multiple models
  • Open-source: Self-hosted with PostgreSQL or SQLite

Quick Start

Install arize-phoenix, instrument your app, and launch the Phoenix server to begin tracing and evaluating your LLM workflows.

Frequently Asked Questions about phoenix-observability

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add LLM tracing and observability to my production AI application?▼

LLM observability uses OpenTelemetry-based tracing to collect end-to-end execution data. It captures LLM inputs, outputs, and internal workflow steps to provide visibility into debugging, real-time production monitoring, and performance measurement.

Can I evaluate and compare different LLM prompts and models using datasets?▼

Yes, you can evaluate model outputs on versioned datasets for regression testing and run experiments to compare prompts, models, and configurations. LLM-as-judge evaluators provide automated quality assessment across different setups.

Does this observability platform support self-hosting with an open-source backend?▼

Yes, the platform is fully open-source and supports self-hosting using either PostgreSQL or SQLite backends. This allows you to maintain full control over your tracing, evaluation, and production monitoring data.

What is the best way to debug LLM workflows with detailed execution traces?▼

The best way to debug LLM workflows is using OpenTelemetry-based trace collection. It captures end-to-end execution paths across any LLM framework, allowing you to inspect prompt inputs, model outputs, and internal steps to identify failures.

Do I need OpenTelemetry to use LLM evaluation and monitoring features?▼

Yes, OpenTelemetry is required for tracing. The platform relies on OpenTelemetry-based trace collection to instrument your application, which feeds data into the evaluation, dataset, and real-time monitoring dashboards.

Related Skills

Explore More Software Engineering Skills→
Arize-aiArize-ai
11.0k

arize-phoenix

Capture AI application traces, evaluations, and prompts with OpenTelemetry.

Official
Advanced
nimadorostkarnimadorostkar
25

prompt-engineering

Design, test, and refine prompts for large language models.

Community
Advanced
informatico-madridinformatico-madrid
1

phoenix-observability

Trace, evaluate, and monitor LLM applications with OpenTelemetry.

Community
Advanced
rusirusi
3

jr-roadmap

Align product roadmap direction using dependency-aware, timeline-free sequencing.

Community
Advanced