skills.rest
ErrDivine
ErrDivineCOMMUNITY
Explore all skills by this author →
GitHub

generate-benchmark-api

Automate creation of LearnwareScaling benchmark dataset classes with scoring and normalization logic.

Updated Mar 18, 2026
Search Tags:#benchmarking#GitHub#Python#dataset creation#LearnwareScaling
One-click install
npx skills add https://github.com/ErrDivine/LiteCode --skill generate-benchmark-api
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: generate-benchmark-api
Source: https://github.com/ErrDivine/LiteCode/tree/main/skills/system/codex/generate-benchmark-api
Command: npx skills add https://github.com/ErrDivine/LiteCode --skill generate-benchmark-api

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires python, learnwarescaling, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill streamlines the process of adding or updating LearnwareScaling benchmark dataset classes, ensuring they integrate seamlessly with the existing framework.

Core Features & Use Cases

  • Benchmark Integration: Automates the creation of dataset classes for LearnwareScaling benchmarks.
  • Evaluation Contract Compliance: Ensures that new benchmarks adhere to the evaluation contract used by LearnwareScaling.
  • Official Repo Porting: Facilitates the porting of scoring and normalization logic from official benchmark GitHub repos.
  • Use Case: When a new benchmark needs to be added to the LearnwareScaling suite, this Skill can be used to create the dataset class and integrate it with the existing evaluation system.

Quick Start

Run the generate-benchmark-api skill with the desired benchmark name and dataset details to create a new benchmark dataset class.

Frequently Asked Questions about generate-benchmark-api

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add new benchmark datasets to the LearnwareScaling framework?▼

Adding benchmark datasets to LearnwareScaling involves automating dataset class creation and mapping scoring with normalization logic. This ensures new benchmarks integrate seamlessly with the existing evaluation contract.

Can I port scoring and normalization logic from official GitHub benchmark repositories?▼

You can port scoring and normalization logic directly from official GitHub benchmark repositories. The framework integrates with these official GitHub evaluators when available to maintain evaluation contract compliance.

Do I need Python and LearnwareScaling installed to generate benchmark dataset classes?▼

You need Python and the LearnwareScaling framework installed. These dependencies are required to automate the creation of benchmark dataset classes and ensure they adhere to the evaluation contract.

What is the best way to ensure new LearnwareScaling benchmarks comply with the evaluation contract?▼

To ensure evaluation contract compliance, automate benchmark dataset class creation. This process handles dataset mapping and integrates official scoring logic to seamlessly fit the LearnwareScaling suite.

How does automated benchmark dataset creation handle the mapping of datasets and scoring?▼

Automated benchmark dataset creation handles mapping by generating dataset classes that encapsulate scoring and normalization logic. It directly integrates official GitHub evaluators to maintain evaluation consistency.

Related Skills

Explore More Software Engineering Skills→
arsityarsity

nemo-evaluator-sdk

Benchmark LLMs across 100+ benchmarks using containerized backends.

Community
Advanced
surus-latsurus-lat
8

push-to-latamboard

Merge local benchmark results into the HuggingFace LatamBoard leaderboard dataset.

Official
Advanced
EurecaMomentEurecaMoment

benchclaw-stage3-evidence-compiler

Compile, clean, and generate ground truth for benchmark data using Python scripts.

Official
Advanced
apacheapache
79

magpie-setup

Automate Apache Magpie framework installation, version control, and skill configuration in repositories.

Official
Advanced