aiml-propaganda-benchmark

Benchmark multilingual propaganda detectors against SemEval-2023 techniques using provided samples.

934|152|Updated Mar 1, 2026

Search Tags:#nlp #classification #benchmark #multilingual #propaganda #SemEval

Ready to run

npx skills add https://github.com/wuyoscar/ISC-Bench --skill aiml-propaganda-benchmark

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: aiml-propaganda-benchmark
Source: https://github.com/wuyoscar/ISC-Bench/tree/main/templates/aiml_propaganda
Command: npx skills add https://github.com/wuyoscar/ISC-Bench --skill aiml-propaganda-benchmark

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill provides a standardized benchmark template to evaluate a multilingual propaganda detection classifier against SemEval-2023 techniques, enabling consistent cross-language evaluation and safety research insights.

Core Features & Use Cases

Evaluates nine propaganda techniques (loaded_language, name_calling, appeal_to_fear, flag_waving, causal_oversimplification, false_dilemma, whataboutism, appeal_to_authority, bandwagon) on multilingual texts using a configurable dataset.
Enforces data integrity checks such as placeholder absence, text length, and technique-marker presence to ensure reliable benchmarks.
Use Case: Researchers assess model robustness across languages, domains, and audiences, identify weaknesses, and compare competing detectors.

Quick Start

Run the propaganda benchmark against propaganda_samples.json in the repository, following the included benchmark and test scripts.

Frequently Asked Questions about aiml-propaganda-benchmark

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I benchmark a multilingual propaganda detection model against SemEval-2023 techniques?▼

To benchmark a multilingual propaganda detection model against SemEval-2023 techniques, run the included benchmark and test scripts against the provided propaganda_samples.json dataset to evaluate technique coverage and report missing labels.

What propaganda techniques are evaluated in a SemEval-2023 benchmark?▼

A SemEval-2023 propaganda benchmark evaluates nine specific techniques: loaded_language, name_calling, appeal_to_fear, flag_waving, causal_oversimplification, false_dilemma, whataboutism, appeal_to_authority, and bandwagon across multilingual texts.

How do I validate multilingual text samples for propaganda classification benchmarks?▼

Validating multilingual text samples for propaganda classification benchmarks requires enforcing data integrity checks for placeholder absence, text length limits, and the presence of technique-specific markers before running evaluation scripts.

Can I use my own dataset to test a propaganda classifier for cross-language evaluation?▼

You can use a configurable dataset to test a propaganda classifier for cross-language evaluation, ensuring your input texts pass placeholder absence and length checks to generate reliable benchmarking results.

What is the best way to identify weaknesses in a multilingual propaganda detector?▼

The best way to identify weaknesses in a multilingual propaganda detector is to run standardized benchmark evaluations across nine techniques and multiple languages, analyzing the reported results for missing techniques and classification gaps.

Why does my propaganda benchmark script report missing techniques?▼

Your propaganda benchmark script reports missing techniques when the evaluated multilingual detector fails to identify specific markers for all nine SemEval-2023 categories within the provided test samples.