extract-trap-task
CommunityAutomate the creation of benchmark trap tasks from Azure DevOps PRs.
AuthorSShadowS
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill automates the process of creating benchmark trap tasks for the CentralGauge platform from approved Azure DevOps pull requests, ensuring that the tasks are representative of real-world scenarios and challenging for large language models.
Core Features & Use Cases
- Pull Request Analysis: Extracts and analyzes code comments and decisions from pull requests to identify potential traps for large language models.
- Task Creation: Automatically creates self-contained or base-app-faithful tasks based on identified traps.
- Discrimination Probing: Ensures that the tasks are challenging by requiring both a correct and a naive solution to pass or fail, respectively.
- Use Case: Use this Skill to generate tasks that can be used in the CentralGauge benchmarking platform to evaluate the performance of large language models on Business Central AL code generation.
Quick Start
Use the extract-trap-task skill to create a benchmark task from an Azure DevOps pull request with ID '12345'.
Dependency Matrix
Required Modules
None requiredComponents
scriptsreferences
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: extract-trap-task Download link: https://github.com/SShadowS/CentralGauge/archive/main.zip#extract-trap-task Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 620,000+ vetted skills library on demand.