skills.rest
tmuskal
tmuskalCOMMUNITY
Explore all skills by this author →
𝕏@tmuskalGitHub

setup

Set up the LongMemEval benchmarking environment with conda and HuggingFace datasets.

3|1|Updated Apr 11, 2026
Search Tags:#benchmarking#Python#conda#dataset download#LongMemEval
One-click install
npx skills add https://github.com/tmuskal/arc-agi-benchmarker --skill setup-tmuskal
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: setup
Source: https://github.com/tmuskal/arc-agi-benchmarker/tree/main/plugins/longmemeval-benchmarker/skills/setup
Command: npx skills add https://github.com/tmuskal/arc-agi-benchmarker --skill setup-tmuskal

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires conda, git, python, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill guides users through setting up the LongMemEval benchmarking environment, including creating a conda environment, cloning the upstream code, downloading datasets, and initializing configuration.

Core Features & Use Cases

  • Environment Setup: Guides through creating a conda environment and setting up Python dependencies.
  • Upstream Clone: Provides instructions to clone the LongMemEval repository.
  • Dataset Download: Offers guidance on downloading dataset variants from HuggingFace.
  • Configuration Initialization: Initializes the configuration file for benchmarking.
  • API Key Verification: Checks for the presence of necessary API keys.
  • Harness Configuration Detection: Detects harness configuration from environment variables and settings files.

Quick Start

Run the setup command to begin the LongMemEval environment setup process.

Frequently Asked Questions about setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up the LongMemEval benchmarking environment from scratch?▼

Setting up the LongMemEval benchmarking environment requires creating a conda environment, cloning the LongMemEval repository, installing Python dependencies, downloading datasets from HuggingFace, and initializing the configuration file.

What dependencies do I need to install LongMemEval datasets and run benchmarking?▼

Installing LongMemEval datasets and running benchmarking requires Python, conda, and git installed on your system, along with valid API keys and access to HuggingFace datasets to successfully download and configure the required data.

Can I configure the LongMemEval benchmarking harness using environment variables?▼

Configuring the LongMemEval benchmarking harness using environment variables is supported. The setup process detects harness configuration directly from environment variables and settings files to initialize the benchmarking environment.

How do I download LongMemEval dataset variants from HuggingFace?▼

Downloading LongMemEval dataset variants from HuggingFace is guided through the setup process, which provides instructions for retrieving the necessary dataset files and verifying that required API keys are present for access.

Does setting up LongMemEval require manual configuration file initialization?▼

Setting up LongMemEval includes automated configuration initialization. The setup process initializes the configuration file for benchmarking automatically after the conda environment is created and the upstream repository is cloned.

Related Skills

Explore More Software Engineering Skills→
sayalinvidiasayalinvidia

cupynumeric-install

Install cuPyNumeric via conda or pip and verify in an isolated environment.

Community
Advanced
tmuskaltmuskal
3

compare-runs

Compare two LongMemEval runs by scorecards, harness configurations, and per-question-type accuracy.

Community
Intermediate
tmuskaltmuskal
3

LongMemEval Judge

Evaluate QA answers in the LongMemEval benchmark using Anthropic or OpenAI services.

Community
Intermediate
pytorchpytorch
4.9k

setup

Automate local ExecuTorch environment setup with conda and dependencies.

Official
Intermediate