Executes standardized AI benchmarking workflows with smoke tests and configurable interfaces.
npx skills add https://github.com/surus-lat/benchy --skill evaluate-surus-lat
Please help me install this Agent Skill. Skill: evaluate Source: https://github.com/surus-lat/benchy/tree/main/.agent/skills/evaluate Command: npx skills add https://github.com/surus-lat/benchy --skill evaluate-surus-lat
This skill solves the challenge of inconsistent AI performance evaluation by providing a standardized, reproducible workflow for benchmarking models and systems against specific tasks.
Run the evaluate skill to perform a smoke test on the specified model configuration using the provided task list.
High-intent search queries and answers about installing and using this skill.
To benchmark AI models with reproducible results, execute a canonical two-stage workflow that enforces an initial smoke test for quality control before running full-scale evaluation against task-specific metrics.
Yes, you can evaluate both cloud API endpoints and local vLLM instances using a unified interface architecture that decouples tasks from the specific AI system, ensuring consistent benchmarking.
Task and interface decoupling separates the evaluation logic from the target system, allowing diverse AI architectures like custom HTTP pipelines and local vLLM to be benchmarked through a single standardized workflow.
The best way to validate model outputs is by running a smoke test to verify accuracy and schema compliance, ensuring the AI system meets task-specific requirements before executing full benchmarking.
A smoke test is required before full-scale evaluation to enforce quality control, ensuring the AI model configuration meets baseline accuracy and schema requirements before committing to comprehensive benchmarking.
stakpakDeploy vLLM models with OpenAI-compatible API endpoints across CPU, GPU, and cloud.
ArogyaReddyAutomates LLM output evaluation with metrics, human feedback, and benchmarking.
surus-latIntegrate AI model providers into the Benchy benchmarking framework via YAML configuration.
amadeus-dlcExecute the AI-DLC nfr-design stage in isolation without advancing the main workflow.
