skills.rest
yakeworld
yakeworldCOMMUNITY
Explore all skills by this author →
GitHub

llm-inference-os

Select operating systems and validate model variants for LLM inference deployments.

6|1|Updated May 11, 2026
Search Tags:#mlops#linux#model selection#qwen#vllm#llm inference#kernel tuning
One-click install
npx skills add https://github.com/yakeworld/Synthos --skill llm-inference-os-yakeworld
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llm-inference-os
Source: https://github.com/yakeworld/Synthos/tree/main/skills/private/mlops/llm-inference-os
Command: npx skills add https://github.com/yakeworld/Synthos --skill llm-inference-os-yakeworld

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Deploying LLM inference on the wrong operating system or with the wrong model variant wastes effort and produces failed workloads, such as requesting image tasks from text-only models like Qwen3.5. ## Core Features & Use Cases - Model Variant Validation: Distinguishes text-only Qwen models (Qwen3/3.5/3.6) from multimodal VL variants, rejecting image tasks on text-only models and recommending Qwen2.5-VL or Qwen3-VL instead. - OS Platform Selection: Ranks deployment platforms for LLM inference, prioritizing Linux (Ubuntu/Debian) over WSL2, native Windows, and macOS based on cited research. - Kernel Tuning Guidance: Applies Linux kernel tuning for high-concurrency inference clusters, targeting the 20-30% throughput gains reported in referenced studies. - Use Case: A user asks to run image classification with Qwen3.5; the skill detects the missing VL suffix, rejects the image task, and recommends a multimodal VL model with recovery advice. ## Quick Start Ask the assistant to deploy a Qwen3-VL high-concurrency inference cluster on Linux and verify the model variant and platform choice.

Frequently Asked Questions about llm-inference-os

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I choose an operating system for LLM inference deployment?▼

Linux (Ubuntu/Debian) is the top-rated platform for LLM inference, followed by RHEL/CentOS and WSL2. Native Windows and macOS rank lowest due to container overhead and kernel tuning limitations documented in cited research.

Can Qwen3.5 process images for classification tasks?▼

No, Qwen3.5 is a text-only model and cannot process images. For image tasks, use a multimodal variant with the VL suffix such as Qwen2.5-VL or the Qwen3-VL series.

What is the difference between Qwen3.5 and Qwen3-VL models?▼

Qwen3.5 is a pure text model while Qwen3-VL is multimodal with vision capability. The VL suffix in the model name indicates image input support, so always check for it before assigning image tasks.

How much throughput gain does Linux kernel tuning provide for inference clusters?▼

Referenced research on optimizing LLM inference clusters reports 20-30% throughput improvement from Linux kernel tuning under high concurrency. OS context switches and interrupts also measurably affect latency.

Why avoid Windows native or macOS for vLLM inference nodes?▼

Windows native and macOS receive the lowest platform ratings for LLM inference due to containerization overhead and limited kernel tuning options. Migrating vLLM nodes to Linux (Ubuntu/Debian) yields better performance and reproducibility.

Related Skills

Explore More Software Engineering Skills→
cloudrift-aicloudrift-ai
75

discover-new-models

Rank newly released open-weight LLMs by demand and VRAM compatibility.

Official
Advanced
KarlinskySKarlinskyS

serving-llms-vllm

Configure vLLM for high-throughput, low-latency LLM serving with PagedAttention and tensor parallelism.

Community
Advanced
yzlnewyzlnew
141

megatron-memory-estimator

Estimate GPU memory usage for Megatron dense and MoE models from HuggingFace configs.

Community
Advanced
shepherdjerredshepherdjerred

grafana-helper

Automate Grafana dashboards, datasources, alerts, and annotations via REST API.

Community
Advanced