arize-phoenix

Capture AI application traces, evaluations, and prompts with OpenTelemetry.

11.0k|1.0k|Updated Nov 9, 2022

Search Tags:#debugging #evaluation #tracing #prompts #OpenInference #openTelemetry #ai-observability

Ready to run

npx skills add https://github.com/Arize-ai/phoenix --skill arize-phoenix

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: arize-phoenix
Source: https://github.com/Arize-ai/phoenix/tree/main/docs/phoenix
Command: npx skills add https://github.com/Arize-ai/phoenix --skill arize-phoenix

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Phoenix provides end-to-end observability for AI applications, enabling you to trace execution, evaluate outputs, and iterate on prompts in a single platform.

Core Features & Use Cases

Tracing: Capture LLM calls, tool executions, retrievals, and embeddings with OpenTelemetry.
Evaluation & Annotations: Run LLM-based or code-based evaluators and attach quality signals to traces.
Prompt Engineering & Experiments: Iterate prompts, manage versions, and run experiments against datasets to compare versions.
Use Case: Debug multi-turn agents and optimize retrieval quality in production environments.

Quick Start

Get started by inspecting the Phoenix docs, instrumenting your application with OTLP, and running a simple trace in a local environment.

Frequently Asked Questions about arize-phoenix

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I trace LLM calls and tool executions in production AI applications?▼

You can trace LLM calls, tool executions, retrievals, and embeddings by instrumenting your AI applications with OpenTelemetry. Phoenix captures these traces to provide end-to-end observability for debugging and optimizing production environments.

What is OpenInference and how does it relate to AI observability?▼

OpenInference is an instrumentation standard used alongside OpenTelemetry to capture AI application traces. It enables Phoenix to record execution paths, prompts, and retrievals, ensuring production-grade visibility for LLM-based systems.

Can I evaluate LLM outputs and attach quality signals to traces?▼

Yes, you can run LLM-based or code-based evaluators to assess outputs and attach quality annotations directly to traces. This allows you to measure retrieval quality and iterate on prompts using data-driven signals.

How do I manage prompt engineering and run experiments against datasets?▼

You can iterate prompts, manage versions, and run experiments against datasets to compare prompt versions. This allows you to evaluate different configurations and optimize multi-turn agents within a single platform.

Do I need OpenTelemetry to use Phoenix for tracing?▼

Yes, deploying Phoenix requires OpenTelemetry instrumentation and OpenInference where applicable. You can self-host the environment via Docker or Kubernetes, or alternatively use Phoenix Cloud for observability.

What's the best way to debug multi-turn agents and optimize retrieval quality?▼

The best way to debug multi-turn agents is to capture end-to-end traces and run evaluations on retrieval quality. Phoenix enables this by recording execution paths and attaching evaluation signals to improve production AI apps.