skills.rest
scragnog
scragnogCOMMUNITY
Explore all skills by this author →
GitHub

engine-performance

Diagnose and benchmark HOT-Step generation performance across GGML and TensorRT inference backends.

151|22|Updated Apr 19, 2026
Search Tags:#benchmarking#music generation#performance profiling#cuda#onnx#tensorrt#ggml
One-click install
npx skills add https://github.com/scragnog/HOT-Step-CPP --skill engine-performance-scragnog
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: engine-performance
Source: https://github.com/scragnog/HOT-Step-CPP/tree/main/.claude/skills/engine-performance
Command: npx skills add https://github.com/scragnog/HOT-Step-CPP --skill engine-performance-scragnog

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Slow music generations in HOT-Step are hard to attribute: time is split across the LM, DiT denoising, and VAE phases, and two inference backends (GGML and TensorRT) behave very differently. This Skill maps where generation time goes, how the TRT paths activate, and how to benchmark from logs so you can find the real bottleneck instead of guessing. ## Core Features & Use Cases - Phase-level profiling: Read the server-side [Timing] pipeline breakdown table and cross-check it against engine-side wall-clock log markers to locate slow stages. - TRT integration guidance: Understand path-driven backend selection (ONNX path triggers TRT, GGUF triggers GGML), engine build caching, adapter refit, and streaming requirements. - Quality-speed trade-offs: Apply implemented knobs like CFG cutoff, step cache, quantization tiers, co-resident models, and batched CFG to speed up generation. - Use Case: A generation takes 80 seconds and you suspect the adapter apply step. Use this Skill to pull the newest session logs, compare [Adapter-TRT] Applied in against the Node timing table, and discover the table was skewed by stdout pipe buffering. ## Quick Start Ask the assistant to profile why my last HOT-Step generation was slow and identify which phase consumed the most time.

Frequently Asked Questions about engine-performance

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I benchmark a slow HOT-Step generation?▼

Run a generation normally, then read the [Timing] pipeline breakdown table in the newest session's generation log under logs/. Cross-check it against engine-side wall-clock markers like [DiT-Generate] TRT Total and [Adapter-TRT] Applied in, since the Node table can be skewed by stdout pipe buffering.

TensorRT vs GGML for music generation, which is faster?▼

TensorRT wins on DiT denoising (about 25% faster) and VAE decode, while GGML wins on the LM phase and model load times. The last end-to-end comparison predates the adapter batching fix, so current overall totals are unverified and should be re-measured.

How do I enable TensorRT inference in the engine?▼

There is no use-TRT flag; backend selection is path-driven. Selecting an ONNX model path activates TensorRT, while selecting a GGUF file activates GGML. The first run builds a .engine cache file, which takes 5 to 30 minutes.

Why does the first TensorRT generation hang for 30 minutes?▼

That is the normal one-time engine build, logged as Engine build in progress every 30 seconds. The result is cached as a .engine file next to the ONNX model, so subsequent runs load from cache. Deleting the .engine file retriggers the build.

Why does my LoRA adapter have no effect on TensorRT?▼

Check the log for [Adapter-TRT] No weights matched TRT engine, which indicates a name-mapping failure, or a warning that adapters are unsupported on FP8 or fp32-I/O engines. Adapters are only supported on BF16-I/O engines.

What settings speed up generation without code changes?▼

Use CFG cutoff (around 0.5 for roughly 20% speedup), step cache (try 0.3-0.5), fewer diffusion steps or a turbo model, lower quantization GGUF variants, the ORT VAE decoder, and co-resident models to avoid the roughly 7.5 second reload between runs.

Related Skills

Explore More Software Engineering Skills→
AscendAscend
31

atc-model-converter

A structured approach to managing your finances and hitting your savings targets through automated transfers.

Official
Advanced
TobiasWooldridgeTobiasWooldridge

wavecap-filters

Adjust WaveCap audio pre-processing filters in state/config.yaml.

Community
Intermediate
itsmostafaitsmostafa
24

qlora

Fine-tune 7B+ models on consumer GPUs with 4-bit quantization and LoRA adapters.

Community
Advanced
m0atm0at
1

cuda

Analyze CUDA kernel performance bottlenecks with Nsight Systems and Nsight Compute.

Community
Advanced