skills.rest
EndUser123
EndUser123COMMUNITY
Explore all skills by this author →
GitHub

ai-groq

Streams Llama, Mixtral, Gemma, and DeepSeek inference via Groq's OpenAI-compatible endpoint.

Updated Apr 26, 2026
Search Tags:#llm#real-time#streaming#groq#inference#openai-sdk
One-click install
npx skills add https://github.com/EndUser123/cc-marketplace --skill ai-groq
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ai-groq
Source: https://github.com/EndUser123/cc-marketplace/tree/main/plugins/cc-skills-ai-api/skills/ai-groq
Command: npx skills add https://github.com/EndUser123/cc-marketplace --skill ai-groq

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

Groq API provides ultra-fast, OpenAI-compatible LLM inference, enabling real-time applications and simplified model access across multiple providers.

Core Features & Use Cases

  • Real-time LLM inference with streaming support for Llama, Mixtral, Gemma, and DeepSeek
  • OpenAI SDK-compatible integration via base_url and GROQ_API_KEY management
  • Use cases include live chat bots, prototyping, and cost-efficient inference

Quick Start

Set GROQ_API_KEY in your environment and start sending chat requests to the Groq OpenAI-compatible endpoint.

Frequently Asked Questions about ai-groq

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I use the Groq API for real-time LLM inference?▼

Real-time LLM inference with the Groq API is achieved by sending chat requests to an OpenAI-compatible endpoint. You must set your GROQ_API_KEY in the environment to enable streaming responses across Llama, Mixtral, Gemma, and DeepSeek models.

Can I use the OpenAI SDK with Groq for streaming chat completions?▼

Yes, the OpenAI SDK works with Groq for streaming chat completions by configuring a custom base_url for the /v1/chat/completions endpoint. You manage authentication using your GROQ_API_KEY while maintaining SDK compatibility.

What LLM models are supported for ultra-fast inference via Groq?▼

Groq supports ultra-fast inference for Llama, Mixtral, Gemma, and DeepSeek models. It delivers high-speed token generation tailored for real-time applications and streaming use cases across these specific architectures.

Do I need a specific environment setup to run Groq inference at scale?▼

You need to set the GROQ_API_KEY environment variable to run Groq inference at scale. No other dependencies are required, allowing you to directly integrate the OpenAI SDK with a configurable base_url for scalable applications.

What are the best use cases for ultra-fast Groq LLM inference?▼

Ultra-fast Groq LLM inference is best suited for live chat bots, rapid prototyping, and cost-efficient inference workflows. It leverages streaming support to deliver real-time responsiveness across multiple open-source model architectures.

Why does Groq offer faster LLM inference compared to other compatible endpoints?▼

Groq provides faster LLM inference by utilizing specialized hardware designed for high-speed token generation. This architecture targets real-time applications, streaming use cases, and scalable OpenAI-compatible chat completions without latency bottlenecks.

Related Skills

Explore More Software Engineering Skills→
rajitsaharajitsaha
10

connect

Automate installation and authentication of SaaS CLI tools.

Community
Basic
oren-arguoren-argu

trigger-realtime

Stream Trigger.dev task run statuses and outputs to React UIs.

Community
Intermediate
antonyfmunozantonyfmunoz

openai

Generate chat responses, embeddings, and structured tool outputs via the OpenAI Python SDK.

Community
Intermediate
jayll1303jayll1303
17

vllm-tgi-inference

Deploy local vLLM or HuggingFace TGI inference servers with OpenAI-compatible APIs.

Community
Advanced