bench-mars

Name: bench-mars
Availability: InStock
Author: kangoxford

Benchmark MarS order-level checkpoints with GPU inference and LOBbench scoring.

1|Updated Apr 23, 2026

Search Tags:#checkpoint #benchmark #inference #decode #slurm #mars #lobbench

Ready to run

npx skills add https://github.com/KangOxford/auto-quant-research --skill bench-mars

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: bench-mars
Source: https://github.com/KangOxford/auto-quant-research/tree/main/execution-layer/skills/bench-mars
Command: npx skills add https://github.com/KangOxford/auto-quant-research --skill bench-mars

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

The bench-mars skill enables end-to-end benchmarking of MarS order-level model checkpoints by automating GPU inference, decoding, and LOBbench scoring to produce reproducible performance metrics.

Core Features & Use Cases

End-to-end benchmarking from checkpoint inference to LOBbench scoring across multiple tickers.
Generates per-ticker results and visualizations for easy comparison across configurations.
Suitable for researchers and engineers evaluating model latency, throughput, and robustness on live order-book data.

Quick Start

Run the Mars bench script on your target checkpoint to produce the benchmark results.

Frequently Asked Questions about bench-mars

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I benchmark MarS order-level checkpoints using GPU inference and SLURM?▼

To benchmark MarS order-level checkpoints, run the bench script on your target checkpoint to automate GPU inference, decode outputs, and score with LOBbench. This produces reproducible performance metrics across multiple tickers using SLURM-managed HPC.

What is end-to-end checkpoint benchmarking for order-level financial models?▼

End-to-end checkpoint benchmarking evaluates model latency, throughput, and robustness by running GPU inference, decoding outputs, and scoring with LOBbench. It generates per-ticker results and visualizations for comparing configurations on live order-book data.

Do I need SLURM and LOBbench to benchmark MarS checkpoints?▼

Yes, benchmarking MarS checkpoints requires SLURM for HPC job submission, along with Mars bench tooling, BinConverter, and LOBbench to handle inference, decoding, and scoring. These dependencies ensure reproducible performance evaluation.

How do I generate per-ticker performance metrics from MarS model checkpoints?▼

Generate per-ticker metrics by running the Mars bench script, which automates inference and LOBbench scoring. The process outputs per-ticker results and visualizations, allowing easy comparison of latency and throughput across different model configurations.

Can I evaluate model latency and throughput on live order-book data with MarS?▼

Yes, you can evaluate model latency and throughput on live order-book data by running the MarS checkpoint benchmark. It automates GPU inference and LOBbench scoring to produce reproducible performance metrics suitable for research workflows.

Why use LOBbench for scoring decoded outputs from MarS inference?▼

LOBbench scores the decoded outputs from MarS GPU inference to produce reproducible performance metrics. It evaluates latency, throughput, and robustness across ticker subsets, generating per-ticker results for easy configuration comparison.