latent-briefing

To reduce token costs in a multi-agent system, share relevant parts of the orchestrator's reasoning state at the representation level instead of replaying the full trajectory as text. This prevents paying for the same context twice during worker handoffs.

Representation-level memory sharing works by using task-guided query vectors to determine which parts of the orchestrator's trajectory are relevant to the worker task. It aggregates scores across layers and heads into one per-position score for batched operations.

You need task-specific memory sharing in orchestrator-worker systems when workers require access to prior orchestrator state without incurring high token costs from replaying the full historical trajectory as text.

You determine which trajectory positions to keep by applying a robust thresholding method to the aggregated per-position scores, filtering the orchestrator's reasoning state to share only the most relevant context for the specific worker task.

Yes, this memory sharing approach works with supervisor-specialist systems. It is optimized for hierarchical systems where specialists need access to prior supervisor state without replaying the full trajectory as text.

A limitation of using shared global masks for multi-agent inference optimization is that it aggregates scores across all layers and heads into a single per-position score, which may oversimplify complex attention patterns required by highly specialized worker tasks.