context-optimization

Apply observation masking first to replace verbose tool outputs with compact retrievable references, then compact remaining history when utilization exceeds 70-80%. Stabilize prompt prefixes to maximize KV-cache hits for additional cost and latency savings.

KV-cache optimization orders prompts so stable content (system prompt, tool definitions) forms a reusable prefix while dynamic content goes last. Identical prefixes reuse cached tensors, targeting 70%+ hit rates and 50%+ cost reduction on cached tokens.

Mask observations first when tool outputs dominate context, since masking removes low-value bulk with under 2% quality impact. Compact afterward when utilization still exceeds 70%, targeting 50-70% token reduction with under 5% quality degradation.

Any change in the prompt prefix, even a single whitespace or an interpolated timestamp, invalidates the entire cached block downstream. Pin system prompts as immutable strings and move dynamic metadata into user messages or tool results.

Partition when estimated task context exceeds 60% of the window and the task decomposes into at least 3 independent subtasks. Below that threshold, coordinator overhead typically exceeds the token savings from isolation.

The estimate_token_count function uses a rough 4-characters-per-token heuristic that varies by model, content type, and language. Production systems should use model-specific tokenizers such as tiktoken for OpenAI models or HuggingFace tokenizers for local models.