Extract final numeric answers from GSM8K model outputs using official ground-truth formats.
npx skills add https://github.com/JoaquinCampo/Skills --skill gsm8k-eval
Please help me install this Agent Skill. Skill: gsm8k-eval Source: https://github.com/JoaquinCampo/Skills/tree/main/gsm8k-eval Command: npx skills add https://github.com/JoaquinCampo/Skills --skill gsm8k-eval
GSM8K evaluation protocol and answer extraction standardizes how to score model reasoning on math problems by focusing on the final numeric answer and consistent ground-truth comparison.
Run the GSM8K evaluation workflow using the official ground-truth and the multi-pattern extractor to score model outputs.
High-intent search queries and answers about installing and using this skill.
GSM8K evaluation extracts final numeric answers from model outputs by matching common formats like ####, The answer is, Answer:, and boxed patterns. It normalizes extracted values before comparing them to ground-truth data.
The standard ground-truth format for scoring the GSM8K dataset uses the official openai/gsm8k main config test split. Evaluation applies normalization and exact-match comparison against this data to compute robust accuracy.
Score model reasoning on math word problems consistently by standardizing final answer extraction and applying robust normalization. Exact-match comparison against official ground-truth ensures reproducible benchmarking across different models and decoding strategies.
Yes, GSM8K evaluation supports multiple answer formats during scoring. The multi-pattern extractor identifies final numeric values from ####, The answer is, Answer:, and last-number patterns before applying normalization and exact-match comparison.
Exact-match comparison for math evaluation requires precise normalization to succeed. If model outputs contain formatting anomalies not covered by the ####, Answer:, or boxed extraction patterns, scoring may fail to recognize correct numeric answers.
meet1919Generate and execute deterministic Python code for GSM8K math word problems.
baekenoughEvaluate software engineering tasks with a 15-task benchmark and scoring rubric.
musical-basicsFine-tune language models with Group Relative Policy Optimization and composable reward functions.
sudarshansudarshanGenerate randomized complex number problems and validate answers via API.
