a6-plugin-ai-proxy

Configure APISIX routes to proxy LLM requests across multiple providers.

1|2|Updated Mar 6, 2026
One-click install
npx skills add https://github.com/api7/a6 --skill a6-plugin-ai-proxy-api7
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: a6-plugin-ai-proxy
Source: https://github.com/api7/a6/tree/main/skills/a6-plugin-ai-proxy
Command: npx skills add https://github.com/api7/a6 --skill a6-plugin-ai-proxy-api7

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

The ai-proxy plugin turns APISIX into an AI gateway, proxying requests to multiple LLM providers while handling authentication, endpoint routing, and streaming to deliver a unified gateway experience.

Core Features & Use Cases

  • Proxy requests to OpenAI, Azure OpenAI, Anthropic, Gemini, Vertex AI, and other providers through a single route.
  • Centralize API keys and improve observability with token and latency metrics.
  • Enable streaming, load-balancing, and failover across providers using ai-proxy-multi.

Quick Start

Create a route using the ai-proxy plugin and send a chat-completions request to test routing.

Frequently Asked Questions about a6-plugin-ai-proxy

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I configure APISIX as an AI gateway for proxying LLM requests?

Configure APISIX as an AI gateway by creating routes with the ai-proxy plugin, setting provider credentials, and proxying requests to multiple LLM providers through a single endpoint.

Can I route LLM traffic to multiple providers like OpenAI and Anthropic through APISIX?

Yes, APISIX can proxy requests to multiple LLM providers including OpenAI, Azure OpenAI, Anthropic, Gemini, and Vertex AI through a unified route while centralizing API keys.

Does the APISIX AI gateway support streaming responses for LLM providers?

Yes, the APISIX AI gateway supports enabling streaming for LLM requests, allowing you to handle real-time streaming responses from providers alongside standard proxy routing.

How do I set up cross-provider failover and load balancing for LLM requests?

Enable cross-provider routing, load-balancing, and failover across LLM providers using the ai-proxy-multi plugin to ensure high availability for proxy requests.

Can I collect observability metrics like token usage and latency for LLM requests?

Yes, the APISIX AI gateway improves observability by collecting token usage and latency metrics for proxied LLM requests, alongside optional logging and timeout configurations.