gem-llm-overview

What problem does it solve?

GEM-LLM newcomers and operators need a fast way to understand the system’s overall architecture, where each component lives in the repo, and how external requests map to internal services and models.

Core Features & Use Cases

• System architecture briefing: Explains how Cloudflare Tunnel traffic reaches the FastAPI gateway and then fans out to the vLLM Dense and MoE instances for dual-model serving.
• Operational port-to-role mapping: Provides a clear mapping of ports (8001/8002/8080/8090) to services and model responsibilities so operators can verify routing quickly.
• Repo navigation by intent: Summarizes key top-level directories (src/docs/plan/tests/scripts) and points to the right follow-up skills for deeper debugging (deploy, admin, load test, routing, troubleshooting).
• Use case: When a new team member receives an “it’s not working” report, they can immediately confirm which port should respond, which models are served on which vLLM endpoints, and where the relevant code and docs are stored before escalating to the specialized debug skills.

Quick Start

Ask the assistant: “Give me a gem-llm-overview of the full GEM-LLM structure, including component locations and port mappings.”