skills.rest
NVIDIA
NVIDIAOFFICIAL
Explore all skills by this author →
WebsiteGitHub

tilegym-monkey-patch-kernels-to-transformers

Integrates TileGym cuTile kernels into Hugging Face transformers models via non-intrusive monkey-patching.

3.2k|370|Updated Feb 25, 2026
Search Tags:#transformers#llm inference#cuda kernels#monkey-patch#cutile#tilegym#kernel integration
One-click install
npx skills add https://github.com/NVIDIA/skills --skill tilegym-monkey-patch-kernels-to-transformers
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: tilegym-monkey-patch-kernels-to-transformers
Source: https://github.com/NVIDIA/skills/tree/main/skills/tilegym-monkey-patch-kernels-to-transformers
Command: npx skills add https://github.com/NVIDIA/skills --skill tilegym-monkey-patch-kernels-to-transformers

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Accelerating LLM training and inference with custom CUDA Tile kernels normally requires modifying the Hugging Face transformers source code, which is invasive and hard to maintain. This Skill replaces transformer modules, classes, and methods at runtime through monkey-patching, so models invoke TileGym kernels under the hood without touching the library source.

Core Features & Use Cases

  • Non-intrusive kernel integration: Replaces transformer submodule implementations and patches init/forward/load-weight methods before model instantiation, keeping transformers source untouched.
  • Autonomous kernel creation loop: Runs an auto-research-style experiment loop that identifies uncovered PyTorch code, creates new cuTile kernels, benchmarks coverage and throughput, and keeps or discards each experiment based on results.
  • FlashInfer-Bench kernel inventory: Records reusable kernels with Definition and Solution JSON metadata so kernels can be discovered, compared, and reused across models.
  • Use Case: An engineer wants to speed up Qwen3.5 inference on a B200 GPU. The agent sets up a Docker environment, patches the model with existing TileGym kernels, then autonomously writes new cuTile kernels until GPU-time kernel coverage is maximized without throughput regression.

Quick Start

Ask your agent to monkey-patch TileGym kernels into a target model, for example: please monkey-patch-kernels-to-transformers Qwen/Qwen3.5-0.8B.

Frequently Asked Questions about tilegym-monkey-patch-kernels-to-transformers

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I integrate TileGym kernels into a Hugging Face transformers model?▼

Prompt your AI agent with the skill name and a target model ID, such as Qwen/Qwen3.5-0.8B. The agent prepares a Docker environment, patches the model's modules with existing TileGym kernels, then autonomously creates new cuTile kernels for uncovered PyTorch code.

Does this approach modify the transformers library source code?▼

No. It uses a non-intrusive monkey-patch approach that replaces modules, classes, and methods at runtime before model instantiation. The transformers source code remains untouched, and the model invokes TileGym kernels under the hood.

What GPUs are supported for cuTile kernel experiments?▼

Experiments must run on NVIDIA GPUs supported by CUDA Tile IR, currently Ampere, Ada, and Blackwell architectures. The environment setup verifies GPU UUIDs and runs all commands inside a dedicated Docker container.

How is kernel coverage and performance measured during integration?▼

The benchmark script reports PyTorch baseline throughput, cuTile kernelized throughput, and cuTile kernel coverage as a percentage of GPU time. New kernels are kept only if coverage improves while end-to-end throughput does not drop below baseline.

What metadata format is used for reusable kernels?▼

Reusable kernels use FlashInfer-Bench-style Definition and Solution JSON metadata. Definitions describe the compute pattern with a PyTorch reference and source permalinks, while Solutions point to the cuTile kernel implementation files and target hardware.

When should I not use monkey-patching for kernel integration?▼

Avoid it when the target model architecture is already patched, since the workflow exits in that case. It is also unsuitable when you need permanent upstream changes in transformers itself rather than runtime substitution.

Related Skills

Explore More Software Engineering Skills→
NVIDIANVIDIA
796

monkey-patch-kernels-to-transformers

Monkey-patch TileGym CUDA kernels into HuggingFace Transformers models.

Official
Advanced
RockyeastRockyeast

onboard-model

Orchestrate end-to-end LLM kernel onboarding into FlashInfer-Bench.

Community
Advanced
vipshopvipshop
1.2k

cute-dsl-kernel

Implement and optimize CuTe DSL GPU kernels across NVIDIA architectures.

Official
Advanced
yo-stevenyo-steven

ad-add-fusion-transformation

Add TensorRT-LLM AutoDeploy fusion transformation passes for graph pattern rewriting.

Community
Advanced