skills.rest
kettleofketchup
kettleofketchupCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

litellm

Deploy an OpenAI-compatible API gateway routing requests to 100+ LLM providers.

Updated Apr 3, 2026
Search Tags:#llm#rbac#kubernetes#docker#api-gateway#openai-compatible#spend-tracking
One-click install
npx skills add https://github.com/kettleofketchup/KettleOfSkills --skill litellm-kettleofketchup
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: litellm
Source: https://github.com/kettleofketchup/KettleOfSkills/tree/main/plugins/litellm/skills/litellm
Command: npx skills add https://github.com/kettleofketchup/KettleOfSkills --skill litellm-kettleofketchup

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill eliminates the need to build and maintain custom integrations for each LLM provider, centralizing API access, authentication, spend tracking, and routing for 100+ LLM services in a single OpenAI-compatible gateway.

Core Features & Use Cases

  • Unified API Access: Expose an OpenAI-compatible endpoint that routes requests to any supported LLM provider, so client code does not need to change when switching models.
  • Built-in Governance: Manage virtual API keys, set per-team budgets and rate limits, and enforce RBAC and JWT/OIDC auth for secure access.
  • Use Case: For a team using multiple LLMs (OpenAI, Anthropic, self-hosted vLLM), use this Skill to deploy a proxy that tracks spend per department, routes requests to the lowest-cost available model, and integrates with Langfuse for observability.

Quick Start

Use the litellm skill to deploy a production-ready LLM API proxy with Docker, configure your model list and access controls, and connect it to your existing LLM providers for unified, tracked access.

Frequently Asked Questions about litellm

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up a unified LLM API gateway to route requests to multiple providers?▼

To set up a unified LLM API gateway, deploy a proxy using Docker, Helm, or Kubernetes, configure your model list, and route requests through a single OpenAI-compatible endpoint to access 100+ LLM providers without changing client code.

Can I enforce RBAC and JWT authentication for an LLM proxy in Kubernetes?▼

Yes, you can enforce RBAC and JWT/OIDC authentication in a Kubernetes-deployed LLM proxy by configuring virtual API keys and access controls to secure per-team routing and model usage.

What is the best way to track LLM spend per department across different providers?▼

The best way to track LLM spend per department is to deploy a unified API gateway that applies virtual keys and budget limits, centralizing spend tracking across 100+ providers in one proxy.

Does an OpenAI-compatible API gateway support fallback routing and rate limiting?▼

Yes, an OpenAI-compatible API gateway supports fallback routing and rate limiting to meet production requirements, automatically directing requests to available models and controlling traffic flow.

How do logging integrations work for a self-hosted LLM proxy?▼

Logging integrations for a self-hosted LLM proxy work by connecting the gateway to 20+ supported logging backends, capturing request data and observability metrics for monitoring LLM usage.

Do I need separate client integrations for switching between OpenAI, Anthropic, and vLLM?▼

No, you do not need separate client integrations when switching between OpenAI, Anthropic, and vLLM, because a unified API gateway exposes a single OpenAI-compatible endpoint for all providers.

Related Skills

Explore More Software Engineering Skills→
algusaemalgusaem

devops

Deploy and manage cloud infrastructure across Cloudflare, Docker, and Google Cloud Platform.

Community
Advanced
giosuetedeschi-specgiosuetedeschi-spec

langfuse

Track LLM traces, spans, and metrics for production AI applications.

Community
Advanced
HZeroxiumHZeroxium

java-authn-authz-jwt

Implement JWT authentication and authorization in Java backends with JWKS verification.

Community
Advanced
fgarofalo56fgarofalo56
6

litellm

Provide a unified API for over 100 LLM providers with fallback mechanisms.

Community
Advanced