redteam-plugin-development

Standardize redteam plugin development and grading with rubric structures.

24.1k|2.2k|Updated Apr 28, 2023
One-click install
npx skills add https://github.com/promptfoo/promptfoo --skill redteam-plugin-development
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: redteam-plugin-development
Source: https://github.com/promptfoo/promptfoo/tree/main/.claude/skills/redteam-plugin-development
Command: npx skills add https://github.com/promptfoo/promptfoo --skill redteam-plugin-development

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Provides a standardized framework for building redteam plugins and graders, ensuring consistent structure, tagging, and registration across projects.

Core Features & Use Cases

  • Standardized tag sets and rubric templates for graders to enforce uniform evaluation criteria.
  • Clear plugin registration and project structure guidance, enabling scalable plugin development.
  • Comprehensive references and base patterns for redteam tooling, attack templates, and documentation.

Quick Start

Create a new redteam plugin under src/redteam/plugins, implement its grader using the base classes, and register it following the official plugin checklist.

Frequently Asked Questions about redteam-plugin-development

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a custom redteam plugin for security testing?

To create a custom redteam plugin, build it under the designated plugins directory, implement its grader using base classes, and register it by following the official plugin checklist to ensure proper structure and tagging.

What is the required rubric structure for redteam graders?

The required rubric structure for redteam graders enforces a standardized format that includes a UserQuery, a purpose, and an optional AllowedEntities tag, ensuring uniform evaluation criteria across all plugins.

How do I standardize evaluation criteria across multiple redteam plugins?

You standardize evaluation criteria by applying standardized tag sets and rubric templates to your graders, which enforces uniform structure and consistent tagging across all plugins in your security testing suite.

Can I modify attack templates and rubric structures for existing redteam plugins?

Yes, you can modify attack templates and rubric structures across the redteam suite, adjusting the UserQuery, purpose, and optional AllowedEntities tags to fit new security testing scenarios.

What are the limitations when writing graders for redteam automation?

Graders must strictly adhere to the enforced rubric structure containing UserQuery and purpose tags, limiting freeform grading logic and requiring optional AllowedEntities definitions only when specific exceptions apply.