FastDeploy
Deploy, test, and benchmark large language models on GPUs
All Skills in This Repository (5)
Pure Emerald Level Indicators
PaddlePaddlefastdeploy-llm-integration
Automate adding inference deployment support for open-source LLMs to FastDeploy.
PaddlePaddlebenchmark
Benchmark FastDeploy versus SGLang with end-to-end tests and HTML reports.
PaddlePaddleresearch-report
Generate a self-contained HTML research report with embedded CSS and Mermaid diagrams.
PaddlePaddlensys-capture
Inject nsys profiling into FastDeploy GPU inference startup and collect .nsys-rep files.
PaddlePaddlewrite-fastdeploy-unittest
Generates CI-compliant pytest unit tests for FastDeploy source files.
Frequently Asked Questions
FAQPage SchemaHow to install FastDeploy skills?▼
Run `npx skills add PaddlePaddle/FastDeploy --all -g -y` in your terminal to install all skills in this suite globally.
How to deploy an LLM with FastDeploy?▼
FastDeploy serves models through an OpenAI-compatible API with a single command, supporting quantization, multi-GPU parallelism, and speculative decoding.
How to add a new model to FastDeploy?▼
The fastdeploy-llm-integration skill analyzes the model's config, picks the right base class, and generates the implementation, docs, and test scripts automatically.
How to compare FastDeploy vs SGLang performance?▼
The benchmark skill installs both frameworks, runs load tests across concurrency and quantization settings, and outputs an interactive HTML comparison report.
Can I profile GPU inference performance automatically?▼
Yes. The nsys-capture skill injects profiling code into FastDeploy, builds the launch script, and produces a ready-to-analyze .nsys-rep file.
Related Repositories in Software Engineering
View All in Software Engineering→
openclawopenclaw
Run a personal AI assistant across your devices and chat apps
obrasuperpowers
Gives coding agents a disciplined workflow from idea to merged code
facebookreact
AI agent skills for building, testing, and porting React core