bench-mars

To benchmark MarS order-level checkpoints, run the bench script on your target checkpoint to automate GPU inference, decode outputs, and score with LOBbench. This produces reproducible performance metrics across multiple tickers using SLURM-managed HPC.

End-to-end checkpoint benchmarking evaluates model latency, throughput, and robustness by running GPU inference, decoding outputs, and scoring with LOBbench. It generates per-ticker results and visualizations for comparing configurations on live order-book data.

Yes, benchmarking MarS checkpoints requires SLURM for HPC job submission, along with Mars bench tooling, BinConverter, and LOBbench to handle inference, decoding, and scoring. These dependencies ensure reproducible performance evaluation.

Generate per-ticker metrics by running the Mars bench script, which automates inference and LOBbench scoring. The process outputs per-ticker results and visualizations, allowing easy comparison of latency and throughput across different model configurations.

Yes, you can evaluate model latency and throughput on live order-book data by running the MarS checkpoint benchmark. It automates GPU inference and LOBbench scoring to produce reproducible performance metrics suitable for research workflows.

LOBbench scores the decoded outputs from MarS GPU inference to produce reproducible performance metrics. It evaluates latency, throughput, and robustness across ticker subsets, generating per-ticker results for easy configuration comparison.