Benchmark ML models by measuring latency, throughput, memory, and accuracy.
npx skills add https://github.com/Open330/agt --skill benchmarking-ml-models
Please help me install this Agent Skill. Skill: benchmarking-ml-models Source: https://github.com/Open330/agt/tree/main/ml/ml-benchmark Command: npx skills add https://github.com/Open330/agt --skill benchmarking-ml-models
Runs ML model benchmarks and evaluations, measuring inference speed, memory usage, and accuracy across models and configurations.
To start benchmarking, run the ml-benchmark.sh script with the desired model and endpoint.
High-intent search queries and answers about installing and using this skill.
You can benchmark ML models by running a Bash orchestration script that measures inference latency, memory usage, throughput, and accuracy. This script coordinates benchmark runs across models and configurations, reporting the results in a structured format.
Yes, you can evaluate ML model performance across GPUs and batch sizes through hardware profiling. The benchmarking process measures inference speed and memory usage to help you understand performance variations across different hardware configurations.
To measure inference throughput, you need a Bash environment with dependencies like jq, yq, bc, curl, and nvidia-smi installed. These tools allow the orchestration script to coordinate benchmark runs and collect performance metrics.
The best way to compare models for a production pipeline is by measuring their latency, throughput, memory usage, and accuracy. This benchmarking approach helps you identify the optimal trade-off between speed and resource consumption for your deployment.
Benchmarking ML models relies on nvidia-smi for GPU profiling, so a GPU environment is expected. The benchmark orchestration also requires standard command-line utilities like jq, yq, bc, and curl to process and report the performance metrics.
olibartfastProfile NVIDIA DeepStream pipelines with Nsight Systems and nvidia-smi.
thanh-abaiiCompare AI model performance across gstack skills using identical prompts.
Imad-OuteAutomate machine learning workflows for chemistry, biology, and materials science datasets.
jiunbaeAutomate ML model benchmarking across versions and languages with JSON and Markdown reports.
