skills.rest
rocklambros
rocklambrosCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

running-adversarial-perturbation-suite

Run FGSM, PGD-20, and AutoAttack-standard to measure adversarial robustness.

2|Updated May 23, 2026
Search Tags:#threat-model#adversarial-robustness#vision-models#pgd#fgsm#autoattack#tabular-models
One-click install
npx skills add https://github.com/rocklambros/rcs --skill running-adversarial-perturbation-suite
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: running-adversarial-perturbation-suite
Source: https://github.com/rocklambros/rcs/tree/main/skills/ml-datasci/running-adversarial-perturbation-suite
Command: npx skills add https://github.com/rocklambros/rcs --skill running-adversarial-perturbation-suite

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill measures how brittle a trained vision or tabular model is to small, adversarially chosen input changes, so you can report a defensible robustness number instead of relying on clean accuracy alone.

Core Features & Use Cases

  • Threat-model first evaluation: Requires a declared norm, epsilon with units, attacker access level, and targeted or untargeted goal before any attack runs.
  • Canonical attack stack: Runs FGSM, PGD-20, and AutoAttack-standard in increasing strength to produce comparable empirical robustness results.
  • Tabular safety checks: Applies feasibility constraints for categorical, integer, monotone, and other domain-limited features so adversarial examples stay realistic.
  • Reporting and auditability: Produces clean accuracy on the attacked subset, per-attack robust accuracy, monotone sanity checks, and saved adversarial examples for inspection.

Quick Start

Ask Claude to evaluate your trained model under a declared threat model, run FGSM, PGD-20, and AutoAttack-standard on a held-out subset, and produce the robustness report with saved adversarial examples.

Frequently Asked Questions about running-adversarial-perturbation-suite

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I measure adversarial robustness for a trained vision or tabular classifier?▼

To measure adversarial robustness, you must declare a threat model specifying norm, epsilon, and attacker access, then run FGSM, PGD-20, and AutoAttack-standard on a held-out test subset to produce empirical robust accuracy numbers.

What is a threat model in adversarial robustness evaluation?▼

A threat model defines the rules of an attack, requiring an explicit norm, epsilon with units, attacker access level, and a targeted or untargeted goal before evaluating how brittle a model is to adversarially chosen input changes.

Can I run AutoAttack and PGD on tabular models with categorical features?▼

Yes, you can run attacks on tabular models because the evaluation applies feasibility constraints for categorical, integer, monotone, and domain-limited features so generated adversarial examples remain realistic.

What's the best way to compare FGSM, PGD-20, and AutoAttack results?▼

The best way to compare attack results is running FGSM, PGD-20, and AutoAttack-standard in increasing strength, which produces per-attack robust accuracy alongside clean accuracy on the attacked subset for comparable empirical evaluation.

How does monotone sanity check work during adversarial robustness testing?▼

Monotone sanity checks validate that robust accuracy decreases as attack strength increases, ensuring the evaluated FGSM, PGD-20, and AutoAttack-standard results are reliable and the threat model parameters are correctly configured.

Do I need clean accuracy on the attacked subset before reporting robust accuracy?▼

Yes, you need clean accuracy on the attacked subset as a baseline before reporting robust accuracy, because the evaluation requires comparing original predictions against per-attack robust accuracy to produce defensible, regulator-facing evidence.

Related Skills

Explore More Data & Analytics Skills→
junhyekhjunhyekh
1

rlxp-dr-design

Design domain randomization ranges and schedules for reinforcement learning experiments.

Community
Intermediate
raddueraddue
10

red-team

Review artifacts adversarially with iterative attack, fix, and re-review loops.

Community
Advanced
maruakshaymaruakshay
4

adversarial-robustness

Assess adversarial robustness of AI systems against evasion attacks.

Community
Advanced
sovr610sovr610

Robustness & Adversarial Testing

Identify and quantify adversarial and distribution-shift vulnerabilities in brain_ai systems.

Community
Advanced