aiml-propaganda-benchmark

What problem does it solve?

This Skill provides a standardized benchmark template to evaluate a multilingual propaganda detection classifier against SemEval-2023 techniques, enabling consistent cross-language evaluation and safety research insights.

Core Features & Use Cases

• Evaluates nine propaganda techniques (loaded_language, name_calling, appeal_to_fear, flag_waving, causal_oversimplification, false_dilemma, whataboutism, appeal_to_authority, bandwagon) on multilingual texts using a configurable dataset.
• Enforces data integrity checks such as placeholder absence, text length, and technique-marker presence to ensure reliable benchmarks.
• Use Case: Researchers assess model robustness across languages, domains, and audiences, identify weaknesses, and compare competing detectors.

Quick Start

Run the propaganda benchmark against propaganda_samples.json in the repository, following the included benchmark and test scripts.