parallel-tool-calls-integration

What problem does it solve?

Traditional agent architectures perform tool calls sequentially, causing high latency and low throughput when multiple independent tools must be invoked; this integration enables a single model response to trigger multiple concurrent tool executions to dramatically increase throughput.

Core Features & Use Cases

• Concurrent Tool Execution: Allow the model to emit multiple tool calls in one response and run them in parallel to reduce wall-clock time.
• Ordered Tool Responses: Preserve the required tool_call ordering when returning tool results to the model to comply with API semantics.
• Resilient Error Handling and Middleware: Continue other tool calls when one fails, aggregate results with error markers, and inject post-tool middleware only after the final tool result.
• Use Case: Modify Builder, Harness, and Evaluator agents to fetch data from several sources or run independent processors concurrently within a single model turn to achieve 2–5x throughput gains.

Quick Start

Enable parallel_tool_calls in your agent configuration and execute all reported tool calls concurrently while collecting ordered results and handling failures.