benchmarking-ml-models

What problem does it solve?

Runs ML model benchmarks and evaluations, measuring inference speed, memory usage, and accuracy across models and configurations.

Core Features & Use Cases

• Benchmarking: measure latency, throughput, memory, and accuracy to compare models.
• Hardware profiling: evaluate performance across GPUs and batch sizes.
• Use Case: comparing two models to select the best trade-off for a production pipeline.

Quick Start

To start benchmarking, run the ml-benchmark.sh script with the desired model and endpoint.