aiml-aegis-benchmark

You benchmark an AI safety classifier by running Python scripts with curated JSON samples to verify unsafe classifications across six harm categories like violence and self-harm.

Evaluating the NVIDIA Aegis classifier for compliance testing involves running structured Python benchmark scripts against curated safety samples to validate robustness and document evaluation results for ML audits.

Yes, you can run safety evaluations using prompt variants in both English and Chinese. The benchmark supports EN and ZH variants to stress model boundaries and verify unsafe classifications across languages.

Yes, you need to fill the placeholder fields in the safety_samples.json file with specific text samples designed to trigger the Aegis safety classifier across the defined harm categories.

The Aegis safety evaluation covers six harm categories: violence, sexual content, criminal planning, weapons, substance abuse, and self harm, allowing you to validate robustness across these specific threat vectors.

Your safety classifier may not trigger if the text samples mapped to harm categories are not specific enough. You must ensure the JSON safety samples explicitly push the boundaries of the classifier's unsafe labeling.