Translate natural language requests into EvalScope CLI commands for benchmark evaluations and stress tests.
npx skills add https://github.com/wenerme/ai --skill evalscope-cli
Please help me install this Agent Skill. Skill: evalscope-cli Source: https://github.com/wenerme/ai/tree/main/skills/evalscope-cli Command: npx skills add https://github.com/wenerme/ai --skill evalscope-cli
Converts natural-language model evaluation and performance testing requests into the exact EvalScope CLI workflows, so you can benchmark accuracy and latency without manually assembling complex command flags.
Typical use case: you want to evaluate a deployed model’s math + coding accuracy, then stress test its throughput and find the maximum concurrency that keeps latency under a target SLA.
Use the evalscope-cli skill to run an accuracy eval for a local model on gsm8k with a small sanity sample by telling it: evaluate Qwen2.5-0.5B on gsm8k with limit 10, then show me the commands to run and where to read the report.
High-intent search queries and answers about installing and using this skill.
To benchmark LLM latency and throughput, this translates natural language into EvalScope CLI commands to run performance stress tests, measuring TTFT, TPOT, and throughput under configurable concurrency gradients against your model endpoints.
Yes, you can evaluate model accuracy by describing your intent in plain language to map eval parameters, select datasets like gsm8k, and produce valid CLI commands for running benchmarks against local or OpenAI-compatible endpoints.
Benchmark discovery by capability tags lists and filters 156+ available benchmarks by categories like math, coding, or multimodal, retrieving rich metadata and examples to help you select the right dataset for accuracy evaluation.
It works with both local model checkpoints and OpenAI-compatible or Anthropic API endpoints, mapping your evaluation and performance testing intent to the correct evalscope subcommands and endpoint configurations for valid execution.
To find maximum concurrency under a target SLA, it generates performance stress test commands with SLA auto-tuning, profiling latency and throughput across configurable concurrency gradients to identify your model's performance limits.
To visualize generated reports, it provides CLI instructions to launch a Web dashboard, allowing you to compare accuracy evaluation and performance testing outputs directly from the EvalScope execution results.
xiaomochnConfigure AI model providers and default model selection in openclaw.json.
Open330Benchmark ML models by measuring latency, throughput, memory, and accuracy.
GitnappDeploy large language model inference services with vLLM and OpenAI-compatible endpoints.
DhumitechAutomate AI agent evaluation and benchmarking with repeatable test pipelines.
