Execute end-to-end AI benchmarking workflows with spec validation and smoke testing.
npx skills add https://github.com/surus-lat/benchy --skill run-benchmark-surus-lat
Please help me install this Agent Skill. Skill: run-benchmark Source: https://github.com/surus-lat/benchy/tree/main/.agent/skills/run-benchmark Command: npx skills add https://github.com/surus-lat/benchy --skill run-benchmark-surus-lat
This skill removes the complexity of managing AI benchmarking workflows by automating the validation, smoke testing, and full execution of performance evaluations.
Use the run-benchmark skill to execute the evaluation for the benchmark specification located at path-to-spec.
High-intent search queries and answers about installing and using this skill.
Automate AI benchmarking by providing a valid benchmark specification file to execute end-to-end workflows. The system coordinates spec validation, smoke testing, and full-scale performance evaluation to generate a final metrics summary automatically.
A two-stage workflow ensures system stability by running an initial smoke test before committing to a full-scale LLM performance evaluation. This prevents wasted compute resources by validating system stability prior to full execution.
Validate benchmark specifications using automated validation checks that scan for errors before execution. This automated process prevents wasted compute by ensuring the spec is completely valid before initiating the full benchmark run.
Yes, you need access to the benchy CLI environment to manage task runners and output aggregation. A valid benchmark specification file and benchy CLI access are required to execute end-to-end AI benchmarking workflows.
Yes, you can use this skill for endpoint performance analysis and task-specific capability verification. It applies to model testing, endpoint performance analysis, and task-specific capability verification across AI systems.
surus-latValidate YAML benchmark specification files for completeness and structural integrity.
surus-latMerge local benchmark results into the HuggingFace LatamBoard leaderboard dataset.
eron1703Orchestrate autonomous agents for parallel configuration and deployment workflows.
aibangjuxinTest RESTful APIs with manual and automated scripts.
