mlops-suite

What problem does it solve?

Streamlines end-to-end Machine Learning operations by providing a suite of tools for model serving, inference, tracking, and more, enabling efficient experimentation and deployment.

Core Features & Use Cases

• Model Serving: Utilizes OpenAI API for high-throughput LLM serving.
• Local Inference: Supports local inference with llama.cpp and GGUF.
• Model Hub: Integrates with HuggingFace Hub for model sharing.
• Experiment Tracking: Offers integration with Weights & Biases.
• Audio Generation: Includes support for text-to-sound via AudioCraft.
• Image Segmentation: Supports image segmentation using SAM.
• Use Case: Imagine you have a large dataset with a trained ML model and want to perform local inference with fine-grained control over model parameters, track your experiments, and serve the model on a server.

Quick Start

Load the skill and start the vLLM server with: mlops-suite serve-vllm.