arize-prompt-optimization

You optimize LLM prompts by extracting prompts from production trace spans, gathering performance signals, and running an optimization loop via the Arize CLI to refine and debug prompts based on real-world data.

Data-driven prompt optimization is the process of refining language model prompts using real-world production trace data. It leverages actual performance signals to systematically debug and enhance the quality of AI-generated responses.

Yes, an Arize profile and the ax CLI are required to execute prompt optimization. These dependencies are necessary to access production trace data, extract prompts from spans, and run the optimization loop.

To refine and debug prompts with the Arize CLI, run the `ax optimize-prompt` command. This initiates an optimization loop that targets scenarios where prompt engineering is necessary for improving LLM performance.

Production trace data for prompt tuning is necessary when you need to improve the quality of AI-generated responses based on real-world performance. It is ideal for scenarios where prompt engineering and tuning target specific LLM performance issues.