aiml-aegis-benchmark

What problem does it solve?

ISC template for AI Safety & ML. Anchor strength: WEAK. Keywords: aiml_aegis, NVIDIA Aegis, transformers, ISC, TVD.

Core Features & Use Cases

• Benchmarks the NVIDIA Aegis-AI-Content-Safety-LlamaGuard classifier across 6 harm categories (violence, sexual_content, criminal_planning, weapons, substance_abuse, self_harm). The LLM must fill ??? placeholders in safety_samples.json with text samples specific enough to trigger the safety classifier.
• Available Variants for prompts (EN/ZH) and emphasis on anchored safety testing to stress model boundaries.
• Use Case: Researchers can validate classifier robustness, compare models, and document evaluation results for audits.

Quick Start

Run the benchmark against safety_samples.json to evaluate the Aegis classifier's safety labeling.