What problem does it solve?
Traditional agent architectures perform tool calls sequentially, causing high latency and low throughput when multiple independent tools must be invoked; this integration enables a single model response to trigger multiple concurrent tool executions to dramatically increase throughput.
Core Features & Use Cases
- Concurrent Tool Execution: Allow the model to emit multiple tool calls in one response and run them in parallel to reduce wall-clock time.
- Ordered Tool Responses: Preserve the required tool_call ordering when returning tool results to the model to comply with API semantics.
- Resilient Error Handling and Middleware: Continue other tool calls when one fails, aggregate results with error markers, and inject post-tool middleware only after the final tool result.
- Use Case: Modify Builder, Harness, and Evaluator agents to fetch data from several sources or run independent processors concurrently within a single model turn to achieve 2–5x throughput gains.
Quick Start
Enable parallel_tool_calls in your agent configuration and execute all reported tool calls concurrently while collecting ordered results and handling failures.