common-context-optimization

Reduce token usage in long-running AI conversations by applying observation masking to replace raw outputs with semantic summaries and using context compaction to preserve essential state. This prevents token budget overflows and keeps context concise.

Context compaction is a recursive summarization technique that preserves state for AI systems by retaining the User Goal, Active Task, Current Errors, and Key Decisions. It prevents information loss when compacting long chat histories or tool logs.

Optimize AI latency when processing large tool outputs and logs by enforcing a strict System, Tools, RAG, and User order with an append-only history. This KV-cache awareness pattern maximizes pre-fill cache hits to significantly reduce response times.

Prevent lost in the middle issues with large language model context by strategically masking large outputs after consumption and applying recursive state compaction. This maximizes context window efficiency and keeps critical information visible.

Observation masking works for managing chat-history.json and log files in tool-assisted reasoning by replacing raw output with semantic summaries after consumption. This noise reduction technique keeps the context concise without losing critical reasoning state.

Avoid recursive summarization for context management when preserving exact verbatim outputs is strictly required, because context compaction intentionally replaces full histories with summarized states to optimize token budgets and reduce latency.