ai-gateway-vivid-fish

You can route OpenAI-compatible chat completions across local backends by using a unified HTTP gateway that exposes the /openai/v1/* endpoint, automatically directing requests to local LLMs like Tabby or Kobold. It provides a consistent API surface for all routed calls.

Vision routing works by automatically detecting image-bearing chat requests and directing them to a vision-capable backend. If no vision support is available, the gateway rejects the request with a clear error instead of failing silently.

The best way to handle long-running media jobs is to submit them asynchronously and poll for completion. This durable job approach fetches signed output handles without keeping long HTTP requests open, preventing timeouts during image generation or audio processing.

Yes, you can use this gateway with Ollama and Anthropic APIs. It provides Anthropic-shaped APIs at /anthropic/v1/messages and Ollama-shaped APIs at /ollama/api/*, ensuring consistent request and response behavior across all supported providers.

Yes, you need an authenticated runtime bearer key to make calls. You must use the AI_GATEWAY_API_KEY with an opk_ prefix for runtime operations, while reserving oak_ prefixed admin keys strictly for control-plane and admin routes.

You can discover available models and voices by calling the gateway's health and discovery routes using your app bearer token. This allows you to identify supported providers for TTS, STT, embeddings, and image generation before executing tasks.