llm-cost-optimization

You can reduce LLM API costs by implementing model tier routing, prompt caching, batch processing, and token budgeting to systematically control token spend and cut costs by 50% or more. This Skill applies these strategies to lower high, unpredictable API spend for production systems.

Prompt caching leverages native caching features from Anthropic, OpenAI, and Gemini to cut input token costs by up to 90% for repeated static content like system prompts and reference documents. This avoids repeatedly processing the same context, significantly lowering API expenses.

Model tier routing matches task complexity to the appropriate model tier, routing simple FAQ queries to low-cost small models while reserving complex reasoning tasks for advanced models. This prevents overspending on simple tasks and under-provisioning complex ones.

Batch processing and semantic caching support async workloads by using discounted batch APIs for nightly tasks like ticket classification and semantic similarity caching for repeated similar queries. This combination reduces costs further for non-real-time data processing.

Token budgeting and output length controls reduce RAG pipeline expenses by constraining the maximum tokens generated per request and monitoring overall spend. These cost monitoring controls prevent high, unpredictable token consumption from exceeding budget limits.

Semantic caching should be used instead of standard prompt caching when handling repeated similar queries that are not exact matches, catching near-duplicate requests. Standard prompt caching only applies to repeated static content like system prompts and reference documents.