agentsop-map-reduce-fanout

What problem does it solve?

This Skill helps you design a reliable map-reduce / dynamic fan-out decision protocol for language-model pipelines that need to run per-item work in parallel and then merge results coherently.

Core Features & Use Cases

• Activation guidance: Determines when map-reduce fan-out is appropriate versus when you should use batching, chaining, or sequential/DAG workflows.
• Concurrency-by-design: Defines how to choose and enforce a bounded concurrency level (e.g., semaphores, max concurrency) to avoid 429/TPM storms.
• Failure policy & timeouts: Provides explicit abort-all, best-effort, retry-then-skip, and quorum strategies plus per-call and total wall-clock timeouts.
• Reduction shape: Guides reducer selection (concatenate, summarize, vote/majority, rank-top-K, dedupe/merge, tree-reduce) based on the downstream consumer’s input requirements.
• Cross-framework SOP: Covers mechanics and gotchas across asyncio, LangGraph Send, CrewAI parallelism, and retrieval fan-out patterns.

Quick Start

Tell your coding agent to apply the agentsop-map-reduce-fanout skill when it is about to write a loop that performs independent LM calls per N items and then merge the outputs with a reducer that matches your downstream needs.