llm-app-deployer

Deploy LLM prototypes as containerized production microservices on cloud compute.

Updated Jul 16, 2026
One-click install
npx skills add https://github.com/Cloud-Byte-Consulting/plugins --skill llm-app-deployer-cloud-byte-consulting
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: llm-app-deployer
Source: https://github.com/Cloud-Byte-Consulting/plugins/tree/main/ai-engineering/skills/llm-app-deployer
Command: npx skills add https://github.com/Cloud-Byte-Consulting/plugins --skill llm-app-deployer-cloud-byte-consulting

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Moving an LLM app from a notebook or CLI prototype to a live production service involves many failure-prone steps: extracting a REST API, externalizing state, building a lean Docker image, handling secrets safely, and tuning the vector store. This Skill provides a verified 9-step promotion workflow so nothing critical is missed. ## Core Features & Use Cases - Prototype-to-service extraction: Convert notebooks or CLI loops into a FastAPI microservice with caller-supplied user_id/session_id identity and guardrails before the chain. - Production containerization: Multi-stage Docker builds with uv, non-root users, Gunicorn+Uvicorn workers, .dockerignore hygiene, and image-size checks (~600-700 MB target). - Managed state migration: Move local vector stores, Redis history, and tracing to managed cloud services with only connection-call changes. - Vector store tuning: Benchmark precision@k against exact search, sweep hnsw_ef, and apply scalar quantization with rescoring. - Use Case: You have a working RAG chatbot running locally with Qdrant and want to ship it on AWS. The Skill walks you through API extraction, ECR push, EC2 deployment, smoke testing, and setting scaling-ladder triggers. ## Quick Start Ask the assistant to take your local LLM prototype and deploy it as a production containerized service on your cloud account.

Frequently Asked Questions about llm-app-deployer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I deploy my LLM app to production?

Follow a four-step promotion: extract a REST API from your prototype, externalize state to managed services, build a multi-stage Docker image, then push to a registry and run it on cloud compute. Verify each step's exit condition before moving on.

How to dockerize a FastAPI LLM service for production?

Use a multi-stage Dockerfile with uv for locked dependency installs, copy only the venv and app into a slim runtime stage, run as a non-root user, and serve with Gunicorn managing Uvicorn workers. Expect roughly 600-700 MB for this stack.

Should I use Qdrant Cloud or a local Qdrant instance in production?

Production containers should hold no state, so move the vector store to a managed instance like Qdrant Cloud. The change touches only the client-construction call, swapping in the URL and API key from environment variables.

Why is my Docker image too big for the registry?

Oversized images usually come from fat base images, dev dependencies, or build-context leaks. Add a .dockerignore for .env, .venv, and .git, use a slim runtime stage, and check whether you exceed free-tier caps like private ECR's 500 MB.

When do I need Kubernetes for my LLM app?

Only when a single VM measurably fails: climb to load-balanced replicas when CPU or uptime demands it, serverless when traffic is spiky, and orchestrated containers like ECS or EKS when running multiple microservices at sustained high traffic.

How do I tune Qdrant search accuracy without reindexing?

Sweep the query-time hnsw_ef parameter and measure precision@k against exact search as ground truth. Scalar quantization with rescoring can also be applied live, while m and ef_construct require a full index rebuild.