lmdeploy
Deploy and serve large language models with fast inference
All Skills in This Repository (6)
Pure Emerald Level Indicators
InternLMcode-navigation
Generate a repo-wide directory tree of LMDeploy's major subsystems.
InternLMsupport-new-model
Integrate new LLMs or VLMs into LMDeploy's PyTorch backend.
InternLMcheck-env
Validate conda, Python, and CUDA setup for LMDeploy environments.
InternLMresolve-review
Fetch GitHub PR review comments, apply fixes, run pre-commit linting, and stage changes.
InternLMsubmit-pr
Automate GitHub pull request creation for LMDeploy branches with titles and bodies.
InternLMdocker-build
Build and push CUDA 13.0 or 12.8 LMDeploy Docker images to an inner registry.
Frequently Asked Questions
FAQPage SchemaHow to install LMDeploy?▼
Run `npx skills add InternLM/lmdeploy --all -g -y` in your terminal to install all skills in this suite globally.
What does LMDeploy do?▼
LMDeploy compresses, deploys, and serves large language and vision models with fast inference, delivering up to 1.8x higher throughput than vLLM through continuous batching and optimized CUDA kernels.
How to add a new model to LMDeploy?▼
The support-new-model skill walks your AI agent through creating the model file, registering it in the module map, and verifying weight loading step by step.
Can LMDeploy quantize models to 4-bit?▼
Yes. It supports AWQ weight-only and KV cache quantization, with 4-bit inference running up to 2.4x faster than FP16.
Which models does LMDeploy support?▼
It supports popular families like Llama, Qwen, DeepSeek, InternLM, Mixtral, GLM, and vision models such as InternVL and Qwen-VL.
Related Repositories in Software Engineering
View All in Software Engineering→
openclawopenclaw
Run a personal AI assistant across your devices and chat apps
obrasuperpowers
Gives coding agents a disciplined workflow from idea to merged code
facebookreact
AI agent skills for building, testing, and porting React core