Generate benchmark task templates and register them in the Benchy evaluation engine.
npx skills add https://github.com/surus-lat/benchy --skill add-task-surus-lat
Please help me install this Agent Skill. Skill: add-task Source: https://github.com/surus-lat/benchy/tree/main/.agent/skills/add-task Command: npx skills add https://github.com/surus-lat/benchy --skill add-task-surus-lat
This Skill simplifies the process of adding new evaluation tasks to the Benchy benchmarking suite by providing a standardized, handler-based template system that reduces boilerplate code.
Use the add-task skill to generate a new benchmark directory structure and template files for a custom evaluation task.
High-intent search queries and answers about installing and using this skill.
Adding a new benchmark task to an AI evaluation engine involves using a standardized class inheritance template to define the task format, register metadata, and generate the required directory structure for reproducible performance evaluation.
Creating structured extraction or multiple choice benchmarks is best handled through a handler-based architecture that provides predefined templates, ensuring consistent task registration and reducing boilerplate code for diverse evaluation formats.
Standardized task registration improves AI model performance evaluation by automating metadata configuration and directory structure generation, which ensures consistent formatting and enables reproducible benchmarking across diverse datasets.
Yes, you can integrate multimodal analysis tasks into your benchmarking suite without rewriting core logic by copying the handler-based template, defining the task class, and registering it within the configuration system.
Python task automation supports freeform evaluation formats for data science workflows by providing predefined handlers that standardize the integration of diverse task types into the evaluation engine.
You need a handler-based template system for AI testing benchmarks to eliminate repetitive boilerplate code, enforce consistent metadata management, and streamline the integration of new evaluation datasets.
surus-latGenerate benchmark task YAML configurations for the Benchy engine.
RheasilviaBenchmark Large Language Models on academic and reasoning tasks with standardized metrics.
caobingshengCoordinates AI agents and skills by discovering, matching, and generating orchestration plans from SKILL.md metadata.
yanochka11Benchmark large language models with lm-evaluation-harness across standardized metrics.
