trace-analysis-advisor

What problem does it solve?

This Skill eliminates guesswork and inefficient trial-and-error in AI red teaming by replacing ad-hoc attack planning with data-driven, evidence-based recommendations derived from historical OTEL trace data from previous assessments.

Core Features & Use Cases

• Attack Effectiveness Analysis: Recommends optimal attack types for specific target models and goal categories based on historical success rates.
• Transform Optimization: Suggests effective obfuscation transform sequences and avoids low-performing transforms based on target response patterns.
• Success Prediction & Vulnerability Fingerprinting: Estimates attack success probability before execution and identifies exploitable vulnerability patterns in target models.
• Use Case: A red teamer testing a new LLM for system prompt leakage can use this skill to quickly identify the highest-success-rate attack and transform combination from historical data, cutting down weeks of manual testing to hours.

Quick Start

Provide your target model identifier, red teaming goal category, and any available target response samples to receive prioritized attack recommendations, optimal transform sequences, and predicted success rates for your assessment.