truefoundry-ai-gateway

An AI gateway exposes diverse LLM providers behind a single OpenAI-compatible API endpoint. You need it to centralize authentication, routing, and governance for safer and simpler AI deployments across cloud and self-hosted models.

You configure rate limiting and routing by applying gateway manifests via tfy apply. This deploys configurations to the gateway in production, enabling per-model routing, budget controls, and cost constraints for your LLM traffic.

Yes, the gateway supports centralized authentication using Personal Access Tokens (PAT) or Virtual Access Tokens (VAT). This enables per-model access control to securely manage which clients can query specific models.

Yes, the gateway integrates directly with MCP servers and tool providers. This enables secure tool access for AI agents, allowing them to utilize external functions while governed by the gateway's budget controls and guardrails.

You monitor usage and apply guardrails by deploying the gateway with tfy apply. The gateway provides built-in observability and guardrail configurations to track requests and enforce safety policies across all routed models.

The gateway requires OpenAI-compatible clients to function properly. While it supports self-hosted models, deployments are managed via tfy apply manifests, meaning infrastructure must support this deployment method.