lmms-eval-guide

What problem does it solve?

This Skill provides a comprehensive guide to the lmms-eval codebase, simplifying the process of integrating new models, adding evaluation tasks, and running complex evaluations for Large Multimodal Models (LMMs).

Core Features & Use Cases

• Codebase Navigation: Understand the architecture and key components of lmms-eval.
• Workflow Guidance: Get step-by-step instructions for common tasks like adding models or running evaluations.
• Use Case: A researcher wants to evaluate a new LMM. They use this Skill to understand how to register their model, define a new evaluation task, and run the evaluation using the framework's CLI or API server.

Quick Start

Use the lmms-eval-guide skill to understand how to add a new model backend to the lmms-eval framework.