dlinfer
Run large language models on domestic AI accelerators
All Skills in This Repository (3)
Pure Emerald Level Indicators
DeepLink-orgsupport-new-model
Guide new LLM or VLM adaptation to dlinfer on domestic hardware.
DeepLink-orgprecision-align
Diagnose precision mismatches between lmdeploy+dlinfer and reference implementations on Ascend, CAMB, and MACA backends.
DeepLink-orggraph-mode-internals
Explain graph mode capture and replay workflows across Ascend, Camb, MACA, and PPU backends.
Frequently Asked Questions
FAQPage SchemaHow to install dlinfer?▼
Run `npx skills add DeepLink-org/dlinfer --all -g -y` in your terminal to install all skills in this suite globally.
What does dlinfer do?▼
It is a hardware abstraction layer that lets lmdeploy run large language and vision-language models on Ascend NPU, Cambricon MLU, and Moore Threads GPU without modifying model code.
How to add support for a new model on Ascend or Cambricon?▼
The support-new-model skill walks the agent through the four-layer op stack, vendor backend changes, and graph buffer updates needed to port a model that already runs on CUDA.
How to debug wrong outputs on domestic AI chips?▼
The precision-align skill provides a step-by-step method to compare lmdeploy+dlinfer against a reference framework, isolating KV cache, operator, and parallelism bugs with tensor dumps.
Can I benchmark inference performance with these skills?▼
Yes. The service-launch and pressure-benchmark skills start an lmdeploy API server, run ShareGPT or random-length workloads, and report throughput, batch behavior, and KV cache usage.
Related Repositories in Software Engineering
View All in Software Engineering→
openclawopenclaw
Run a personal AI assistant across your devices and chat apps
obrasuperpowers
Gives coding agents a disciplined workflow from idea to merged code
facebookreact
AI agent skills for building, testing, and porting React core