Cost Tracking

Track LLM evaluation costs by calculating per-task and per-trajectory expenses across LLM calls, tool invocations, and judge evaluations. This cost tracking applies built-in pricing to generate a structured cost report with daily totals.

Budget enforcement for AI evaluation workflows triggers tiered alerts at specific thresholds: logging at 50% budget usage, notifications at 75%, and blocking further evaluations at 90% to prevent cost overruns.

Apply different cost tracking presets for AI evaluations using strict, moderate, and lenient configurations. These presets adjust budget thresholds and cost reporting to fit various organizational policies and evaluation scales.

Calculate per-task LLM and judge costs by applying built-in pricing models to evaluation workflows. Cost tracking separates llm_cost, tool_cost, and judge_cost to provide a precise cost breakdown for each task.

Identify wasteful spending patterns in AI evaluation campaigns by generating a structured cost report with daily totals. Cost tracking analyzes per-task and per-trajectory expenses to highlight inefficient model usage and optimization opportunities.

Cost tracking works without external pricing dependencies by using built-in pricing models to calculate llm_cost, tool_cost, and judge_cost. It operates independently to enforce budget limits and generate alerts across the evaluation suite.