skills.rest
DeepLink-org
DeepLink-orgOFFICIAL
Explore all skills by this author →
WebsiteGitHub

nvidia-nlp-finetune-llama-7b

Benchmarks Alpaca-LoRA LLaMA-7B fine-tuning throughput on NVIDIA GPUs.

7|1|Updated Apr 17, 2026
Search Tags:#fine-tuning#docker#benchmark#throughput#lora#nvidia#llama-7b
One-click install
npx skills add https://github.com/DeepLink-org/DeepEval-Skills --skill nvidia-nlp-finetune-llama-7b-deeplink-org
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: nvidia-nlp-finetune-llama-7b
Source: https://github.com/DeepLink-org/DeepEval-Skills/tree/main/skills/NVIDIA/nlp/nvidia-nlp-finetune/llama-7b
Command: npx skills add https://github.com/DeepLink-org/DeepEval-Skills --skill nvidia-nlp-finetune-llama-7b-deeplink-org

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? Measuring LoRA fine-tuning performance of LLaMA-7B on NVIDIA GPUs requires manual container setup, distributed training orchestration, and log parsing. This Skill automates the full workflow and produces a structured result.json with throughput metrics. ## Core Features & Use Cases - Automated Benchmark Execution: Launches a Docker container, patches transformers with instrumented trainer files, and runs torch.distributed training via scripts/batch_finetune.sh. - Metric Extraction: Parses the final training summary from logs to compute train_tokens_per_second, per-GPU throughput, runtime, and train_loss, writing them to /workspace/results/result.json. - Use Case: Ask the agent to test LoRA fine-tuning performance on 8 NVIDIA GPUs; it starts the container, runs Alpaca-LoRA training on LLaMA-7B, and returns TGS metrics for hardware comparison. ## Quick Start Ask the agent to run an Alpaca-LoRA fine-tuning performance test with LLaMA-7B on NVIDIA GPUs and collect the train_tokens_per_second metrics.

Frequently Asked Questions about nvidia-nlp-finetune-llama-7b

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I benchmark LoRA fine-tuning performance on NVIDIA GPUs?▼

Run the batch_finetune.sh script inside the provided Docker container with model, dataset, and results directories mounted. It launches torch.distributed training of LLaMA-7B with Alpaca-LoRA and writes throughput metrics to result.json.

What metrics does Alpaca-LoRA fine-tuning benchmarking collect?▼

It collects train_tokens_per_second as total throughput, tokens_per_sec_per_gpu for single-card throughput, plus train_samples_per_second, train_steps_per_second, train_runtime, and final train_loss from the Trainer summary line.

How many GPUs are required for LLaMA-7B LoRA fine-tuning evaluation?▼

The benchmark requires 8 NVIDIA GPUs by default, matching the torch.distributed.run --nproc_per_node setting. You can override the GPU count by setting the NLP_FIN_NGPU or CARD_COUNT environment variable.

Why is train_tokens_per_second missing from the training log?▼

The default transformers Trainer does not emit train_tokens_per_second. The script patches the container's transformers trainer.py and trainer_utils.py with instrumented versions from the code directory; if patching fails, the summary line lacks the field.

Can I change batch size for the fine-tuning benchmark?▼

Yes, set NLP_FIN_BATCH_SIZE and NLP_FIN_MICRO_BATCH_SIZE environment variables before running the script. The log filename changes accordingly, so also set NLP_FIN_LOG_FILE if you need a specific log path.

Related Skills

Explore More Software Engineering Skills→
davila7davila7
30.5k

llama-factory

Guides fine-tuning of large language models using LLaMA-Factory with LoRA, QLoRA, and WebUI workflows.

Community
Intermediate
mindspore-aimindspore-ai
258

performance-summary

Summarize and compare operator performance results from result.json files.

Official
Intermediate
jwu1980-testjwu1980-test

benchmark-kernel

Benchmark FlashInfer GPU kernels with CUPTI timing and output median throughput metrics.

Official
Intermediate
JamesFincherJamesFincher

peft-fine-tuning

Train low-rank adapters for parameter-efficient fine-tuning of large language models.

Community
Advanced