skills.rest
DeepLink-org
DeepLink-orgOFFICIAL
Explore all skills by this author →
WebsiteGitHub

graph-mode-internals

Explain graph mode capture and replay workflows across Ascend, Camb, MACA, and PPU backends.

74|15|Updated Jul 9, 2024
Search Tags:#model-inference#buffer-management#lmdeploy#backend-compatibility#graph-mode#graph-capture#dlinfer
One-click install
npx skills add https://github.com/DeepLink-org/dlinfer --skill graph-mode-internals
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: graph-mode-internals
Source: https://github.com/DeepLink-org/dlinfer/tree/main/.claude/skills/zh_cn/graph-mode-internals
Command: npx skills add https://github.com/DeepLink-org/dlinfer --skill graph-mode-internals

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Understand the end-to-end graph mode workflow used by lmdeploy+dlinfer, including runner architecture, buffer management, capture/replay flows, and vendor differences.

Core Features & Use Cases

  • Clarifies how graph mode captures and replays computations to reduce Python dispatch overhead.
  • Details the buffer layout, KV cache handling, and per-step data flows across Ascend, Camb, MACA, and PPU backends.
  • Provides practical guidance on common pitfalls and troubleshooting for graph-mode deployments.

Quick Start

Review the guide to understand how graph-mode capture and replay works and how to troubleshoot common issues.

Frequently Asked Questions about graph-mode-internals

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How does graph mode capture and replay work in lmdeploy dlinfer?▼

Graph mode in lmdeploy dlinfer captures and replays neural network computations to reduce Python dispatch overhead. It replays pre-recorded computation graphs during decode workflows across Ascend, Camb, MACA, and PPU backends.

How do I manage buffers and update KV cache in graph mode deployments?▼

Buffer management in graph mode involves creating, filling, and updating context for KV cache handling per step. You must specifically update kv_seqlens during the capture and replay flow to ensure correct data execution.

Does lmdeploy graph mode support Ascend, Camb, MACA, and PPU backends?▼

Yes, lmdeploy graph mode supports decode and deployment workflows across Ascend, Camb, MACA, and PPU backends. It details specific capture and replay steps alongside vendor differences for each hardware backend.

How do I troubleshoot common pitfalls and limitations in graph mode deployments?▼

Troubleshooting graph mode deployments involves checking runner architecture, buffer layouts, and vendor-specific graph update APIs. Common pitfalls often stem from incorrect kv_seqlens updates or incompatible torch_npu versions across different backends.

Related Skills

Explore More Software Engineering Skills→
DeepLink-orgDeepLink-org
74

support-new-model

Guide new LLM or VLM adaptation to dlinfer on domestic hardware.

Official
Advanced
ascend-ai-codingascend-ai-coding
156

msmodelslim

Quantize and optimize Huawei Ascend NPU models for MindIE or vLLM-Ascend deployment.

Official
Intermediate
SciMate-AISciMate-AI
1

torchdrug

Build, train, and deploy graph neural network workflows for chemistry and biology.

Official
Advanced
tenstorrenttenstorrent
341

tt-enable-tracing

Enable TTNN trace capture and replay to eliminate host dispatch overhead.

Official
Advanced