llama-slot-pinning

To persist KV caches across llama-server restarts, configure slot pinning by launching the server with a designated slot-save-path and verifying available slots via the /slots endpoint. This setup preserves prompt context and enables stable performance in multi-model deployments.

Slot pinning is a configuration technique that allocates dedicated slots and independent save paths to prevent cross-model cache contamination. You need it for multi-model deployments where each model requires separate slots to maintain stable performance and preserve prompt context across restarts.

Yes, you can run multiple llama-server instances on different ports with consistent slot management by configuring each instance with its own dedicated slots and independent save paths. This approach ensures per-model slot isolation and prevents cross-model cache contamination.

Yes, configuring llama-server for slot pinning requires a server build with KV-cache support, a writable slot-save-path directory, and appropriate --parallel and --ctx-size settings to allocate the correct number of slots and context window sizes for your models.

Cross-model cache contamination happens when multiple models share the same slots without isolation, causing prompt context overwrites. Configuring per-model slot isolation with dedicated slots and independent save paths prevents this contamination in multi-model deployments.