rasa-configuring-model-groups

Name: rasa-configuring-model-groups
Availability: InStock
Author: Rasa

Configures model groups in endpoints.yml for LLM and embedding providers.

5|3|Updated Mar 2, 2026

Search Tags:#embeddings #llm #configuration #rasa #endpoints.yml #model groups

Ready to run

npx skills add https://github.com/RasaHQ/rasa-agent-skills --skill rasa-configuring-model-groups

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: rasa-configuring-model-groups
Source: https://github.com/RasaHQ/rasa-agent-skills/tree/main/skills/rasa-configuring-model-groups
Command: npx skills add https://github.com/RasaHQ/rasa-agent-skills --skill rasa-configuring-model-groups

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill simplifies the complex task of configuring model groups in endpoints.yml, enabling seamless integration and routing for various LLM and embedding providers within Rasa.

Core Features & Use Cases

Provider Configuration: Set up LLM and embedding providers (OpenAI, Azure, self-hosted, etc.) in endpoints.yml.
Multi-Deployment Routing: Configure strategies like least-busy or latency-based routing for multiple model deployments within a single group.
Caching & Failover: Implement response caching and define failover mechanisms for robust model group operation.
Use Case: When setting up a new Rasa project that requires routing requests to multiple LLM instances for load balancing, use this Skill to define the model_groups in your endpoints.yml.

Quick Start

Configure a model group for OpenAI embeddings in your endpoints.yml file.

Frequently Asked Questions about rasa-configuring-model-groups

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I configure multiple LLM deployments for load balancing in Rasa endpoints.yml?▼

Configure multiple LLM deployments in Rasa endpoints.yml by defining model_groups with routing strategies such as least-busy or latency-based, enabling efficient distribution of requests across available model instances.

What are model groups in Rasa and when do I need them?▼

Model groups in Rasa are endpoints.yml configurations that organize LLM and embedding providers. You need them when integrating multiple language models to enable flexible routing, failover mechanisms, and response caching for robust conversational agents.

Can I set up failover and caching for self-hosted LLM providers in Rasa?▼

Yes, you can set up failover and caching for self-hosted LLM providers in Rasa. Configure model_groups in endpoints.yml to define failover mechanisms and implement response caching, ensuring robust operation during outages or high traffic.

Does Rasa support configuring Azure OpenAI and other external embedding providers?▼

Rasa supports configuring Azure OpenAI and other external embedding providers. Set up various LLM and embedding providers within endpoints.yml using model_groups to manage routing and integration seamlessly.

What is the best way to route LLM requests based on latency in a Rasa chatbot?▼

The best way to route LLM requests based on latency in a Rasa chatbot is configuring model_groups in endpoints.yml with a latency-based routing strategy. This directs traffic to the fastest responding deployment, optimizing agent performance.