Orchestrate large model deployment with GPU optimization and distributed serving.
npx skills add https://github.com/jstzwj/ai-infra-plugins --skill sglang-jstzwj
Please help me install this Agent Skill. Skill: sglang Source: https://github.com/jstzwj/ai-infra-plugins/tree/main/plugins/sglang/skills/sglang Command: npx skills add https://github.com/jstzwj/ai-infra-plugins --skill sglang-jstzwj
This Skill provides a comprehensive AI assistance platform that integrates large language models, multimodal inputs, and advanced execution workflows to streamline enterprise AI deployment.
Launch the sglang server with a preloaded model and invoke it through API or CLI commands for instant AI assistance in your enterprise environment.
High-intent search queries and answers about installing and using this skill.
Deploy large language models using distributed serving and GPU optimization by orchestrating model pipelines, applying quantization techniques, and leveraging CUDA graph acceleration to achieve scalable, high-throughput enterprise AI solutions.
Processing multi-modal inputs like text, images, audio, and video in a single interface streamlines enterprise AI workflows, enabling versatile applications such as automated content moderation and compliance safety checks.
Enterprise model deployment requires integration of FastAPI and PyTorch alongside CUDA, NCCL, and Dapr to orchestrate large model pipelines, execute multi-modal data processing, and ensure high availability runtime optimization.
Increase AI serving throughput by applying quantization techniques and CUDA graph acceleration within the runtime optimization process, reducing memory overhead and accelerating execution for distributed enterprise workloads.
You can automate content moderation workflows that analyze text, images, and video simultaneously by utilizing multi-modal input handling capabilities to ensure compliance and safety across social media platforms.
Avoid using distributed servers for large model deployment when your enterprise environment lacks underlying GPU hardware or the necessary CUDA and NCCL dependencies required to execute runtime optimizations effectively.
erfanzarGenerate music from lyrics and tags using PyTorch on CUDA-enabled GPUs or CPUs.
HeldinhowImplement Prometheus metrics collection and scrape configuration for application monitoring.
TehminanazDocument architectural standards for cloud-native microservices with Dapr, Kafka, and Helm.
Ced3-hanAutomate 4-bit activation-aware quantization for large language models.
