fleet-inference

What problem does it solve?

This Skill enables direct querying of any model within a fleet, whether local or cloud-based, reducing latency and simplifying model access for AI tasks.

Core Features & Use Cases

• Model Discovery and Access: Listed models can be retrieved across various providers like MLX, Ollama, Cerebras, Groq, Gemini, and OpenRouter.
• Flexible Querying: Send prompts to specific models or allow auto-routing to the best available provider for tasks such as inference, analysis, and generation.
• Use Case: Easily run large language model inferences on the local network or cloud, such as generating content or analyzing data without manual switching between endpoints.

Quick Start

Query models by specifying a prompt or list all available models, enabling quick integration into AI workflows.