Configure and validate benchmark datasets by mapping CSV or JSONL files to required schemas.
npx skills add https://github.com/surus-lat/benchy --skill setup-data
Please help me install this Agent Skill. Skill: setup-data Source: https://github.com/surus-lat/benchy/tree/main/.agent/skills/setup-data Command: npx skills add https://github.com/surus-lat/benchy --skill setup-data
This skill resolves the friction of preparing and formatting datasets for AI benchmarking, ensuring that input data is correctly structured for the evaluation engine.
Use the setup-data skill to validate my local dataset file and update the data section of my benchmark configuration.
High-intent search queries and answers about installing and using this skill.
To prepare a local dataset for AI benchmarking evaluation, you map existing CSV or JSONL input files to the required schema formats, ensuring the data contains necessary identifiers, inputs, and ground truth labels for validation.
You can remap CSV columns to match a required schema format by configuring the data section within your benchmark configuration file, allowing the evaluation engine to correctly ingest and process the adapted input data.
Yes, you can generate synthetic test data for benchmarking when real-world examples are unavailable by integrating with generation tools to create synthetic datasets that match the required evaluation schema.
The required YAML structure for configuring benchmark datasets involves defining a specific data section within your benchmark configuration files, which dictates how local files are ingested and validated for the evaluation engine.
Yes, benchmark dataset validation checks JSONL files to ensure they contain necessary identifiers, inputs, and ground truth labels before the data is correctly structured and passed to the evaluation engine.
The best way to validate an existing dataset against a benchmarking schema is to use a configuration skill that checks for necessary identifiers and ground truth labels while mapping input files to the required format.
surus-latGenerate synthetic benchmark datasets as JSONL input-output pairs from task specifications.
surus-latGenerate benchmark task templates and register them in the Benchy evaluation engine.
ovachieverCurate multimodal LLM training data with GPU-accelerated filtering and deduplication.
JayFareiAutomate OpenTraces dataset workflow creation with JSONL row schema validation.
