What problem does it solve? Building benchmark test datasets for tool-calling agents (customer service, assistants, query/operation agents) is error-prone: distributions drift, dialogue histories contradict the data snapshot, injected language noise looks artificial, and edge-case coverage is uneven. This Skill provides a complete, project-agnostic methodology for constructing controlled, reproducible multi-turn dialogue benchmark cases. ## Core Features & Use Cases - CaseSpec-driven generation architecture: Orthogonally enumerate case specifications with exact quota allocation (largest remainder method) for hard distribution targets like turn-count tiers, and weighted sampling with conditional degradation for style dimensions. - Four-layer history constraints: Structural skeleton, data self-consistency (no forward/reverse fabrication, progressive state visibility), reference naturalness (real and unique antecedents, no verbatim repetition), plus lightweight heuristic validation with transparent failure marking. - Language noise injection: Prompt-driven noise styles including ASR errors split by severity, homophone typos, colloquialisms, and ambiguous references, with noise restricted to the current user turn only. - Boundary/fallback test pool: Subtype design for chit-chat, misrouted commands, out-of-scope requests, factually invalid premises, truly unresolvable ambiguity, and sensitive content with safe construction patterns. - Use Case: When asked to build a benchmark or test dataset for any tool-calling agent, use this Skill to design the case taxonomy, align the design draft with the user at a mandatory checkpoint, then generate validated cases at scale. ## Quick Start Use this skill to design and generate a multi-turn dialogue benchmark test dataset for my customer service agent, starting with a design draft of case categories, turn distributions, and noise styles for my review.
