Benchmark OpenAI-compatible LLMs across seven objective tests.
npx skills add https://github.com/OrangeViolin/01fish-llm-radar --skill 01fish-llm-radar
Please help me install this Agent Skill. Skill: 01fish-llm-radar Source: https://github.com/OrangeViolin/01fish-llm-radar/tree/main Command: npx skills add https://github.com/OrangeViolin/01fish-llm-radar --skill 01fish-llm-radar
Benchmark OpenAI-compatible LLMs across seven objective tests to help teams compare models quickly and reproducibly.
Install dependencies, configure API keys, and run a sample evaluation with a chosen model and benchmark.
High-intent search queries and answers about installing and using this skill.
You can benchmark OpenAI-compatible LLMs by running seven objective tests including MMLU and HumanEval. This Skill routes requests across providers like DeepSeek and OpenRouter to produce reproducible, horizontally comparative model evaluations.
Reproducible LLM evaluation uses seven objective tests: GSM8K, MATH, MMLU, MMLU-Pro, C-Eval, IFEval, and HumanEval. These benchmarks provide a broad capability snapshot for comparing multiple models transparently.
Yes, you can evaluate models from self-hosted endpoints and SiliconFlow. The Skill supports multi-provider routing for any OpenAI-compatible endpoint, enabling rapid comparisons across diverse hosting environments.
You set up API keys for multi-provider routing by configuring them with python-dotenv. After installing dependencies like openai and datasets, you configure your provider credentials to run deterministic benchmark orchestration.
Yes, the benchmarking approach provides per-sample result transparency. It enforces data provenance by outputting per-sample results alongside aggregated summaries and deep analysis reports for actionable insights.
The best way to compare DeepSeek and OpenRouter performance is through one-shot benchmarking across standard tests. This approach enforces deterministic orchestration to generate transparent, reproducible comparisons in 30 minutes.
handsomelong922Evaluate LLMs across 100+ benchmarks from 18+ harnesses.
jandan138Analyze ML experiment results from JSON or CSV and generate statistics and comparison tables.
jackspaceProvide AI-ready drug-discovery datasets and benchmarks from PyTDC for AI/ML modeling.
majiayu000Orchestrate multiple LLM providers through a unified Sage client with streaming and fallback.
