Runs mesh-llm benchmark tune sweeps to optimize model-serving throughput settings.
npx skills add https://github.com/Mesh-LLM/mesh-llm --skill benchmark-tune
Please help me install this Agent Skill. Skill: benchmark-tune Source: https://github.com/Mesh-LLM/mesh-llm/tree/main/.agents/skills/benchmark-tune Command: npx skills add https://github.com/Mesh-LLM/mesh-llm --skill benchmark-tune
Tuning LLM serving performance requires testing many combinations of context size, batch size, mmap, mlock, and speculative decoding settings, which is tedious and error-prone to do manually. This Skill guides running, debugging, and interpreting mesh-llm benchmark tune trials so you get tolerance-aware, evidence-backed configuration recommendations.
Ask the assistant to run mesh-llm benchmark tune on a local GGUF model with a bounded ctx and batch sweep and interpret the JSON results.
High-intent search queries and answers about installing and using this skill.
Run mesh-llm benchmark tune with --model pointing to a downloaded GGUF file, plus sweep flags like --ctx-sizes, --batch-sizes, and --ubatch-sizes. Add --json and tee the output to a file for machine-readable evidence.
Benchmark tune is the correct command for model-serving throughput tuning. The gpus namespace is only for hardware inventory and raw fingerprinting, so gpu tune or gpus tune should not be used for serving optimization.
Yes, benchmark tune can run on remote hosts over SSH, preferably in a TTY login shell with tee logging. Verify the intended CUDA runtime is selected via mesh-llm runtime list before recording performance results.
By default, --speculative-types auto tries MTP for MTP-style models, then local draft-model candidates, then ngram fallbacks, plus a disabled baseline. Use --speculative-types mtp,draft,ngram,disabled to force an explicit sweep or --no-speculative-tune to disable it.
Trials commonly fail due to invalid ubatch greater than batch pairs, missing local model targets, or wrong runtime selection. Check benchmarks[].trials[].error and the per-trial log_path under target/gpu-tune/ for the shared failure reason.
Benchmark tune only accepts already-downloaded local or configured model targets and will not fetch remote-only refs. Without an explicit target, it uses configured local models from ~/.mesh-llm/config.toml.
fikriafAutomate discovery, download, and local inference of llama.cpp models from Hugging Face Hub.
HouseGarofaloDeploy and serve LLMs with Hugging Face Text Generation Inference.
Wenwen555...
informatico-madridOptimizes LLM serving on NVIDIA hardware with batched GPU inference.
