local-ml-hosting

What problem does it solve?

This Skill addresses the challenges of hosting and evaluating machine learning models locally, ensuring optimal performance and compatibility with various environments.

Core Features & Use Cases

• Local ML Model Hosting: Self-host models like Ollama for on-premise evaluation and deployment.
• Model Selection by VRAM: Efficiently select models based on available VRAM, optimizing performance.
• Hermes Integration: Seamlessly integrate with Hermes for advanced functionality and customization.
• LLM Evaluation Benchmarks: Offers benchmarks for evaluating the performance of large language models.
• Use Case: Ideal for data scientists and ML engineers looking to evaluate and deploy models in a controlled, self-hosted environment.

Quick Start

Install the 'local-ml-hosting' skill and run the following command to evaluate a model: lm_eval --model ollama --model_args model=deepseek-r1:8b --tasks mmlu