aiml-malware-benchmark

Evaluate a DistilBERT malware classifier on threat intelligence reports.

934|152|Updated Mar 1, 2026

Search Tags:#benchmark #threat-intelligence #mitre-attack #aiml_malware #distilbert #malware-classification

Ready to run

npx skills add https://github.com/wuyoscar/ISC-Bench --skill aiml-malware-benchmark

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: aiml-malware-benchmark
Source: https://github.com/wuyoscar/ISC-Bench/tree/main/templates/aiml_malware
Command: npx skills add https://github.com/wuyoscar/ISC-Bench --skill aiml-malware-benchmark

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill enables automated evaluation of a DistilBERT malware classifier on threat intelligence reports, ensuring reproducible depth across multiple malware families and strict adherence to MITRE ATT&CK references and IO patterns.

Core Features & Use Cases

Anchored depth: uses a reference anchor (one authenticated threat report) to calibrate the required technical depth and IOCs.
Deterministic evaluation: enforces report validation rules (word counts, MITRE IDs, IOs) to produce repeatable benchmarks.
Use Case: researchers can benchmark model performance on real-world threat reports and compare classifier outputs with expected anchors.

Quick Start

Run the benchmark against threat_reports.json and observe the classifier outputs.

Frequently Asked Questions about aiml-malware-benchmark

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I benchmark a DistilBERT malware classifier on threat intelligence reports?▼

Benchmarking a malware classifier involves running it against a labeled threat_reports.json dataset to measure detection depth across multiple families. The skill enforces deterministic evaluation rules including word counts, MITRE ATT&CK IDs, and IOCs to produce repeatable results.

What is anchored depth evaluation in malware classification benchmarks?▼

Anchored depth evaluation uses a single authenticated threat report as a reference anchor to calibrate the required technical depth and IOCs. This mechanism ensures reproducible malware classification results across multiple malware families.

Do I need a specific dataset format to evaluate malware classification models?▼

Yes, you need a threat_reports.json dataset containing labeled threat reports. This input data, combined with anchor-provided reference data, enforces deterministic behavior and validates report depth during the malware classification evaluation.

How does deterministic evaluation validate MITRE ATT&CK references in threat reports?▼

Deterministic evaluation enforces strict report validation rules by checking for MITRE ATT&CK references, IO patterns, and word counts. This strict adherence ensures repeatable benchmarks when comparing classifier outputs against expected threat intelligence anchors.

Can I use this malware benchmark for cybersecurity research workflows requiring actionable IOCs?▼

Yes, this malware benchmark suits cybersecurity research workflows requiring MITRE ATT&CK references and actionable IOCs. It validates report depth and placeholder content while measuring classifier detection depth across multiple malware families.