Test Semgrep Rule Skill

Validate Semgrep rules against vulnerable repositories and compute metrics.

Updated Dec 25, 2025
One-click install
npx skills add https://github.com/chrismcmacken/bounty-hunter --skill test-semgrep-rule-skill
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: Test Semgrep Rule Skill
Source: https://github.com/chrismcmacken/bounty-hunter/tree/main/.claude/skills/test-semgrep-rule
Command: npx skills add https://github.com/chrismcmacken/bounty-hunter --skill test-semgrep-rule-skill

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This skill helps security researchers and rule authors validate Semgrep rules against known vulnerable repositories to measure true and false positive rates, enabling reliable detection and benchmarking.

Core Features & Use Cases

  • Execute end-to-end rule evaluation against curated vulnerable repos.
  • Compute TP/FP/Recall/Precision metrics and summarize findings.
  • Use Case: Benchmark rule performance across multi-language samples and standardized test-beds.

Quick Start

Follow the workflow to test a Semgrep rule against known vulnerable apps by cloning the test repositories described in resources/test-repositories.md, configuring your rule, running semgrep --config rule.yaml target/ --json > findings.json, and examining the results with jq.

Frequently Asked Questions about Test Semgrep Rule Skill

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I benchmark Semgrep rules against known vulnerable repositories?

You can benchmark Semgrep rules by running them against curated vulnerable repos to compute true and false positive metrics. This skill automates executing scans, collecting JSON results, and summarizing detection accuracy.

How do I measure false positives when testing static analysis rules?

To measure false positives during static analysis testing, run your Semgrep rules against known vulnerable repositories and calculate precision metrics. This skill evaluates scan results to identify inaccurate rule matches.

What tools do I need to run Semgrep rule evaluations locally?

You need standard CLI tools including semgrep, jq, and git to run rule evaluations locally. This skill requires these tools to clone test repositories, execute scans, and process JSON output for metrics.

Can I validate security testing rules across multi-language code samples?

Yes, you can validate security testing rules across multi-language code samples using standardized test-beds. This skill evaluates Semgrep rule performance against diverse vulnerable repositories to ensure broad detection coverage.

How do I calculate recall and precision for custom vulnerability detection rules?

You calculate recall and precision by comparing your custom vulnerability detection rule findings against known vulnerable code. This skill processes semgrep JSON output to compute these standard benchmarking metrics automatically.

What is the best way to test Semgrep rules before deploying them?

The best way to test Semgrep rules before deployment is validating them against known vulnerable apps. This skill provides an end-to-end workflow to clone test repositories, run scans, and compute accuracy metrics.