fastdeploy-llm-integration

Name: fastdeploy-llm-integration
Availability: InStock
Author: PaddlePaddle

Automate adding inference deployment support for open-source LLMs to FastDeploy.

3.7k|758|Updated Jun 27, 2022

Search Tags:#deployment #llm #inference #huggingface #model-integration #modelscope #fastdeploy

Ready to run

npx skills add https://github.com/PaddlePaddle/FastDeploy --skill fastdeploy-llm-integration

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: fastdeploy-llm-integration
Source: https://github.com/PaddlePaddle/FastDeploy/tree/main/.claude/skills/fastdeploy-llm-integration
Command: npx skills add https://github.com/PaddlePaddle/FastDeploy --skill fastdeploy-llm-integration

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Guides you through adding inference deployment support for a new open-source LLM to the FastDeploy repository. Given a model path (local or HuggingFace/ModelScope hub), this skill walks through analyzing the model architecture, choosing the right base class, generating the model implementation file, updating registries, writing docs, and producing a deployment test script.

Core Features & Use Cases

Analyze a model's architecture and select the appropriate base class.
Generate model implementation files, update registries, write documentation, and create a deployment test script.
Use case: port a HuggingFace/ModelScope LLM to FastDeploy and validate the integration with a deployment test.

Quick Start

Provide the model path (local or hub) and follow the guided steps to generate the FastDeploy integration artifacts.

Frequently Asked Questions about fastdeploy-llm-integration

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I add HuggingFace LLM inference deployment to FastDeploy?▼

To add HuggingFace LLM inference deployment to FastDeploy, provide the model path and follow the guided steps to analyze the architecture, generate implementation files, update registries, and create a deployment test script.

What is the process for porting a ModelScope model to FastDeploy?▼

Porting a ModelScope model to FastDeploy involves analyzing the model architecture, choosing the appropriate base class, generating implementation files, writing documentation, and producing a deployment test script for validation.

Can I use a local model path for FastDeploy integration?▼

Yes, you can use a local model path for FastDeploy integration. The process accepts both local directories and HuggingFace or ModelScope hub paths to generate the necessary deployment artifacts.

What's the best way to generate deployment test scripts for new LLM integrations?▼

The best way to generate deployment test scripts for new LLM integrations is to use automated templates that analyze the model architecture and produce ready-to-test integration artifacts within the FastDeploy repository.

Does FastDeploy integration require manually updating model registries?▼

No, FastDeploy integration does not require manually updating model registries. The automated process handles updating registries while generating the model implementation files and documentation.