skills.rest
infantesromeroadrian
infantesromeroadrianCOMMUNITY
Explore all skills by this author →
Website𝕏@infantes_adrianGitHub

inference-optimization

Quantize production inference models and export to ONNX with TensorRT optimization.

Updated Aug 23, 2026
Search Tags:#profiling#gpu#quantization#inference#onnx#tensorrt
One-click install
npx skills add https://github.com/infantesromeroadrian/arca-claude-code --skill inference-optimization-infantesromeroadrian
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: inference-optimization
Source: https://github.com/infantesromeroadrian/arca-claude-code/tree/main/skills/inference-optimization
Command: npx skills add https://github.com/infantesromeroadrian/arca-claude-code --skill inference-optimization-infantesromeroadrian

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Automates optimization of production inference workloads by quantizing models, exporting to ONNX, and deploying optimized runtimes.

Core Features & Use Cases

  • Post-training quantization options (INT8, INT4, NF4) with PTQ tools, plus GPTQ and AWQ variants.
  • ONNX export, optimization, and RTX 2000 Ada-specific tuning with TensorRT and profiling.
  • Workflow guidance for benchmarking latency, memory usage, and accuracy across hardware targets.

Quick Start

Select a representative model, apply a quantization strategy (INT8/INT4/NF4), and evaluate latency vs. accuracy.

Frequently Asked Questions about inference-optimization

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I quantize a large language model for inference?▼

You can apply post-training quantization using INT8, INT4, or NF4 formats, including GPTQ and AWQ variants. The workflow covers selecting a representative model, applying a quantization strategy, and evaluating latency versus accuracy for deployment.

What's the best way to export an ML model to ONNX for TensorRT deployment?▼

ONNX export is a core step in the optimization pipeline, followed by ONNX optimization and TensorRT tuning. The process targets RTX 2000 Ada GPUs specifically, enabling hardware-tuned deployment for production inference workloads.

Does this inference optimization workflow support INT4 and NF4 quantization?▼

Yes, the workflow supports INT8, INT4, and NF4 quantization formats. It includes post-training quantization tools alongside GPTQ and AWQ variants, allowing ML engineers to reduce model size and memory usage for large language model deployment.

How do I profile inference latency and memory usage across different GPU hardware?▼

Profiling is integrated into the workflow to benchmark latency, memory usage, and accuracy across hardware targets. This enables ML engineers to measure the impact of quantization and runtime optimization before production deployment on RTX 2000 Ada GPUs.

Can I use GPTQ and AWQ for post-training quantization of production models?▼

Both GPTQ and AWQ are supported as post-training quantization variants. Combined with PTQ tools and INT8/INT4/NF4 formats, they provide multiple strategies for optimizing large language models during production inference deployment.

Related Skills

Explore More Software Engineering Skills→
ricablericable

gptq

Compress large language models into GPTQ 4-bit checkpoints.

Community
Advanced
m0atm0at
1

tensorrt-llm

Optimize LLM inference on NVIDIA GPUs with TensorRT-LLM.

Community
Advanced
Z43LZ43L

serving-llms-vllm

Deploy an OpenAI-compatible LLM inference API with vLLM.

Community
Intermediate
amdamd
154

quark-torch-export

Create export and evaluation plans for Quark quantized models.

Official
Intermediate