prompt-caching

Anthropic prompt caching stores repeated prompt prefixes so subsequent requests only process new tokens. Structure your prompt so stable content like system instructions and documents comes first, then mark the cache breakpoint on that prefix.

Cache augmented generation pre-loads documents into the cached prompt instead of retrieving them at query time like RAG. CAG works well when the document set is small and stable, avoiding retrieval latency and embedding infrastructure.

Cache full responses when queries repeat exactly or are semantically similar, such as FAQ-style chatbots. Avoid response caching with high temperature settings or time-sensitive answers, since cached output can become stale or incorrect.

Prompt caching fails when the prefix changes between requests, since the cache matches on exact prefix content. Keep dynamic content like timestamps and user data at the end of the prompt so the shared prefix stays stable.

Cache invalidation requires TTLs or event-based purging tied to when underlying data changes. Without invalidation, cached responses become incorrect over time, so pair every cache entry with an explicit expiration or update trigger.