skills.rest
ascend-ai-coding
ascend-ai-codingOFFICIAL
Explore all skills by this author →
GitHub

vllm-ascend

Serve OpenAI-compatible LLM inference on Huawei Ascend NPUs.

156|53|Updated Feb 14, 2026
Search Tags:#quantization#vllm#api-server#llm-serving#ascend#tensor-parallelism#pipeline-parallelism
One-click install
npx skills add https://github.com/ascend-ai-coding/awesome-ascend-skills --skill vllm-ascend
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: vllm-ascend
Source: https://github.com/ascend-ai-coding/awesome-ascend-skills/tree/main/vllm-ascend
Command: npx skills add https://github.com/ascend-ai-coding/awesome-ascend-skills --skill vllm-ascend

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

Enables efficient LLM inference serving on Huawei Ascend NPUs with OpenAI-compatible APIs, reducing deployment complexity and enabling scalable production workloads.

Core Features & Use Cases

  • OpenAI-compatible API server for LLM inference
  • Ascend-optimized kernels with support for quantization (msmodelslim)
  • Tensor and pipeline parallelism for multi-NPU deployments
  • Model support across Qwen, DeepSeek, GLM, and LLaMA families

Quick Start

Install the vLLM-Ascend plugin and start the API server with your model path to begin serving.

Frequently Asked Questions about vllm-ascend

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I serve LLM inference with an OpenAI-compatible API on Ascend NPUs?▼

Serving LLM inference with an OpenAI-compatible API on Ascend NPUs is achieved by deploying the vLLM-Ascend API server, which reduces deployment complexity and handles scalable production workloads.

Does vLLM support tensor and pipeline parallelism for multi-NPU deployments on Ascend?▼

Yes, vLLM supports multi-NPU deployments on Ascend by utilizing tensor and pipeline parallelism, enabling efficient distributed setups for large language model serving across multiple hardware units.

Can I run quantized model inference on Huawei Ascend NPUs?▼

Yes, quantized model inference on Ascend NPUs is supported through msmodelslim, providing Ascend-optimized kernels for efficient LLM serving.

What LLM model families are compatible with Ascend CANN for inference?▼

Model families compatible with Ascend CANN for inference include Qwen, DeepSeek, GLM, and LLaMA, ensuring broad architectural coverage for LLM serving.

What is the best way to perform offline batch inference on Ascend NPUs?▼

The best way to perform offline batch inference on Ascend NPUs is using the vLLM-Ascend setup, which natively supports batch processing alongside distributed configurations.

Do I need specific hardware configurations to run an OpenAI-compatible API server on Ascend?▼

Running an OpenAI-compatible API server on Ascend requires Huawei Ascend NPUs with CANN compatibility, scaling across multi-NPU configurations using tensor and pipeline parallelism for distributed serving.

Related Skills

Explore More Software Engineering Skills→
AscendAscend
31

cann-operator-env-config

Install and configure the CANN toolkit on Ascend NPU hardware.

Official
Advanced
qcmuuqcmuu

serving-llms-vllm

Deploy vLLM inference endpoints with quantization and tensor parallelism.

Community
Advanced
peteromalletpeteromallet
97

serving-llms-vllm

Deploy LLM inference with vLLM using OpenAI-compatible endpoints, quantization, and tensor parallelism.

Community
Intermediate
ovachieverovachiever
52

serving-llms-vllm

Deploy OpenAI-compatible LLM APIs with vLLM for high-throughput inference.

Community
Advanced