context-optimization

Apply observation masking first to replace verbose tool outputs with compact references, then trigger compaction when utilization exceeds 70-80%. The ContextBudget class monitors usage per category and signals when optimization should fire.

Place stable content first in the prompt: system prompt, tool definitions, templates, then dynamic content last. Remove timestamps and session IDs from the prefix, since even a single whitespace change invalidates the cached block downstream.

Mask first when tool outputs dominate context, since masking removes low-value bulk with under 2% quality impact. Compact afterward when total utilization still exceeds 70%, summarizing remaining turns while preserving decisions and task state.

Cache misses spike when prompt prefixes change between deployments, such as reworded system prompts or reordered tools. The entire cached block downstream of any change is invalidated, so roll out prompt changes gradually and monitor hit rates.

Partitioning adds coordination overhead: each sub-agent needs its own system prompt, tools, and result aggregation. For tasks with fewer than 3 independent subtasks, the overhead often exceeds the token savings, so estimate total cost first.

The estimate_token_count function uses a rough heuristic of about 4 characters per token for English. Production systems should use model-specific tokenizers like tiktoken for OpenAI models or HuggingFace tokenizers for local models.