prompt-caching

Add cache_control with type ephemeral to stable system prompt blocks in your messages.create call. Claude caches the marked prefix, and subsequent requests with the identical prefix read from cache, reducing cached token costs by up to 90%.

Use Cache Augmented Generation when your document corpus is stable, fits within the context window (under roughly 100K tokens), and latency is critical. Use RAG when the corpus is large, updates frequently, or queries target specific documents.

Prompt caching requires an exact prefix match. Dynamic content like timestamps inside cached blocks, reordered components, or any change to the prefix invalidates the cache. Move dynamic content into user messages and keep cached blocks static.

No. Responses generated with temperature above 0.5 are non-deterministic, so caching them serves inconsistent results. Only cache low-temperature responses and include model and temperature in the cache key.

Use versioned cache keys to invalidate all entries at once, content-hash validation to detect source changes, or event-based invalidation tied to source updates. Always set a TTL appropriate to your data freshness requirements.