void-llm-cost-discipline

Reduce Anthropic API costs by defaulting to Sonnet instead of Opus, enabling prompt caching on system prompts over 1024 tokens for roughly 90% savings on cached blocks, and using the Batch API for non-interactive workloads at a 50% discount.

Use Sonnet 4.6 as the default for agentic work and code generation, Haiku 4.5 for high-volume classification at roughly 3x lower cost, and Opus 4.7 only for high-stakes reasoning with a justifying comment at the call site, since it costs about 5x Sonnet.

Prompt caching works by adding cache_control with type ephemeral to stable blocks like system prompts, tool definitions, and few-shot examples. Cached blocks cost about 90% less per call, with a default 5-minute TTL and an optional 1-hour TTL for stable prompts.

Use the Batch API when the workload is non-interactive and tolerates latency over one hour, such as overnight processing, bulk analysis, or periodic reports, since it offers a 50% discount. Keep user-facing interactive calls on real-time streaming endpoints.

Retrying on invalid_request_error is pointless because the request itself is malformed and will fail identically every time, wasting tokens. Retries should be bounded with exponential backoff and jitter, and skipped for malformed requests and prompt-injection refusals.

This approach does not choose your provider, since Anthropic versus OpenAI versus Gemini is a project-level decision, and it does not impose a model-router abstraction by default. It also forbids caching prompts containing user PII without per-user cache keys.