ai-gateway

Route LLM requests across multiple AI providers by setting environment variables for your chosen provider—OLLAMA_BASE_URL and OLLAMA_MODEL for local Ollama, ANTHROPIC_API_KEY and ANTHROPIC_MODEL for Anthropic, or HF_API_KEY and HF_MODEL for HuggingFace—then use the get_llm function to switch providers at runtime based on configuration.

Yes, provider switching happens automatically through environment-driven configuration. Set AI_PROVIDER and the corresponding API credentials or endpoints, and the gateway routes requests to your selected provider without code changes across development, testing, and production environments.

Configure OLLAMA_BASE_URL and OLLAMA_MODEL for Ollama, ANTHROPIC_API_KEY and ANTHROPIC_MODEL for Anthropic, or HF_API_KEY and HF_MODEL for HuggingFace. The gateway reads these variables to enable seamless provider selection and supports optional defaults for fallback behavior.

Yes, the gateway applies rate limiting and API proxying to manage LLM requests across multiple providers, enabling controlled access, load distribution, and centralized request handling in development, testing, and production environments.

Test provider switching using the provided test endpoint after setting your AI_PROVIDER and environment variables. A simple request confirms that the gateway correctly routes to your selected provider and returns responses without errors.