Create benchmark trap tasks from Azure DevOps pull requests.
npx skills add https://github.com/SShadowS/CentralGauge --skill extract-trap-task
Please help me install this Agent Skill. Skill: extract-trap-task Source: https://github.com/SShadowS/CentralGauge/tree/main/.claude/skills/extract-trap-task Command: npx skills add https://github.com/SShadowS/CentralGauge --skill extract-trap-task
This Skill automates the process of creating benchmark trap tasks for the CentralGauge platform from approved Azure DevOps pull requests, ensuring that the tasks are representative of real-world scenarios and challenging for large language models.
Use the extract-trap-task skill to create a benchmark task from an Azure DevOps pull request with ID '12345'.
High-intent search queries and answers about installing and using this skill.
To create benchmark trap tasks from Azure DevOps pull requests, analyze PR comments, code diff, and work item descriptions to identify non-obvious code decisions and translate them into self-contained Business Central AL tasks.
A trap task in large language model benchmarking is a challenge derived from non-obvious code decisions that requires a correct solution to pass and a naive solution to fail, ensuring accurate evaluation of LLM performance on Business Central AL code generation.
To ensure LLM benchmark tasks are challenging for Business Central AL code generation, probe task discrimination by requiring both correct and incorrect solutions to explicitly pass or fail, validating that tasks catch naive large language model outputs.
Yes, you can use Azure DevOps PR comments to generate self-contained AL tasks by analyzing code decisions and work item descriptions, translating identified non-obvious logic into base-app-faithful benchmark tasks for large language models.
Automating trap task creation from pull requests relies on analyzing PR comments, code diff, and work item descriptions, meaning tasks depend on the clarity of PR documentation and may not capture all non-obvious Business Central AL code decisions.
Yes, the extract-trap-task skill works with the CentralGauge benchmarking platform, generating benchmark tasks from Azure DevOps pull requests to evaluate large language models on Business Central AL code generation performance.
albertoiruruetaGenerates and updates pull request descriptions from code changes for GitHub, Bitbucket, Azure DevOps, or TFS.
breethomasBuild a binary LLM evaluator that returns JSON Pass or Fail verdicts.
javiarmestoDiagnose and fix bugs in Business Central AL extensions through a structured workflow.
ductrung-nguyenFetch GitHub pull request context and post code reviews with inline comments.
