What problem does it solve?
Different LLM providers (Claude, GPT, Gemini, Ollama, Cohere) require different prompt and message formats, making multi-provider inference and RAG pipelines tedious to build and maintain.
Core Features & Use Cases
- Multi-Provider Prompt Formatting: Convert chat messages into the native format for Anthropic, OpenAI, Gemini, Ollama, or Cohere.
- HNSW Context Retrieval: Create an HNSW index, add documents, and route queries to retrieve relevant context for RAG workflows.
- Provider Status Checks: Verify provider availability before sending inference requests.
- Use Case: Build a RAG pipeline that indexes your documentation, retrieves the most relevant passages for a user question, and formats the final prompt for whichever LLM provider is available.
Quick Start
Format my prompt for the OpenAI provider and retrieve relevant context from the indexed documents using HNSW routing.