Orchestrate host GPU benchmark operations via uv run and AskUserQuestion prompts.
npx skills add https://github.com/aviralmansingka/dotfiles --skill hostbench
Please help me install this Agent Skill. Skill: hostbench Source: https://github.com/aviralmansingka/dotfiles/tree/main/claude/.claude/skills/hostbench Command: npx skills add https://github.com/aviralmansingka/dotfiles --skill hostbench
Host benchmark workflows require repeatedly collecting the right run parameters (subcommands, cloud/region, instance types, IPs/worker IDs, date ranges, and local-vs-S3 intent) and then safely executing the correct operational command without manual, error-prone retyping.
Ask your AI to run hostbench to evaluate benchmark pass rates by saying: "Run /hostbench evaluate with the relevant since/until dates from this chat."
High-intent search queries and answers about installing and using this skill.
SSH-driven GPU benchmark jobs can be executed by selecting the correct subcommand and assembling arguments directly from conversation context, pre-filling instance types, cloud regions, IPs, and date ranges to run jobs via uv run main.py.
Comparing percentile distributions from S3 benchmark JSON artifacts is done by selecting the compare subcommand, which inspects local versus cloud/region results to evaluate performance differences across benchmark runs.
Evaluating GPU benchmark pass rates supports conversation-aware prefill, scanning chat context for since/until date ranges and worker IDs to automatically populate the required evaluation parameters.
Cloud GPU benchmark evaluations require AWS credential wrapping when running cloud or region-specific subcommands, while local-only benchmark inspections can execute without cloud credential setup.
Host benchmark commands fail when local-only and cloud/region requirements conflict, because validation logic enforces correct parameter combinations based on whether the benchmark target is local or cloud-based.
Inspecting benchmark JSON artifacts to diagnose GPU performance issues uses the inspect subcommand, which reads local or S3-stored results to surface percentile distributions and pass rate metrics.
HarriesProvision Lambda Labs GPU cloud instances and manage SSH keys via lambda_cloud_client.
AGIBuildCompare BenchmarkDotNet JSON results against GitHub Actions artifact baselines.
Penitence1992Generate presigned URLs and upload or download files across S3 and OSS in Go backends.
FutureMLS-LabGenerate end-to-end performance profiling traces for SGLang server runs.
