Design controlled benchmark setups with fixed seeds and multiple trials.
npx skills add https://github.com/tnn1t1s/iterator --skill benchmark-design
Please help me install this Agent Skill. Skill: benchmark_design Source: https://github.com/tnn1t1s/iterator/tree/main/.claude/skills/CS500/benchmark_design Command: npx skills add https://github.com/tnn1t1s/iterator --skill benchmark-design
Generates controlled benchmark setups with workload patterns, throughput/latency metrics, and reproducible seeds.
Set up a Java benchmark with k=100 and 10M total elements, then run measurements.
High-intent search queries and answers about installing and using this skill.
Reproducible benchmarks use fixed seeds, warmup routines, and multiple trials to isolate performance characteristics. Set controlled workloads with defined throughput and latency metrics, document compiler flags and runtime versions, and handle cache state between trials to ensure consistent results across runs.
Yes. Use JMH for Java, Google Benchmark for C++, and Criterion for Rust. Each framework supports fixed seeds, high-resolution timing, and multiple trial patterns. The same principles—small, medium, and large workload patterns with documented platform details—apply across all three languages.
Throughput and latency benchmarks require controlled workloads with fixed seeds to eliminate randomness. Run multiple trials, apply warmup routines to stabilize JIT compilation and cache state, and capture high-resolution timing data. Document all compiler flags, runtime versions, and platform characteristics for reproducibility.
Both adversarial and realistic workload patterns reveal different performance characteristics. Adversarial patterns expose worst-case behavior and cache misses; realistic patterns reflect production use. Including both in small, medium, and large scales provides comprehensive performance insight with controlled, reproducible conditions.
Cache state affects latency and throughput measurements significantly. Managing cache between trials—clearing it or leaving it warm—ensures consistent conditions and prevents spurious results from carryover effects. Documented cache handling is essential for reproducible benchmarks across multiple runs and environments.
Record compiler flags, runtime versions, platform details (CPU, memory, OS), warmup duration, trial count, seed values, and workload patterns. This documentation enables others to replicate your benchmark and verify that performance changes come from code, not environment drift or measurement variance.
BurburtonEstablish performance baselines and compare benchmark results across builds.
howdeployBenchmark web pages against baselines to detect performance regressions.
Harmeet10000Profile and benchmark Go applications to identify and fix performance bottlenecks.
ab604Profile and benchmark R code using profvis, bench, and Rprof.
