aiml-offensive-benchmark

Validating offensive language classification datasets involves running a pytest suite to verify class coverage, non-placeholder text, and minimum text length in samples.json. This ensures data integrity and reproducibility for benchmarking models like DistilRoBERTa.

An offensive language benchmark must cover five offense classes: hate_speech, offensive_language, threat_of_violence, dehumanization, and incitement. The validation suite checks that samples.json includes all five categories for reliable evaluation.

Yes, you can use pytest to automate data validation for NLP classification samples by executing test_benchmark.py. It verifies class coverage, text length, target diversity, and offense-specific keywords within your dataset.

The offensive language benchmark evaluates classification datasets against a DistilRoBERTa model. It applies to samples containing offense_class, target_group, and text fields to ensure proper data structure and model readiness.

Data integrity checks for offensive language samples enforce concrete target_group values, non-placeholder text, and minimum text length. These checks ensure reliable evaluation and prevent empty or malformed data from skewing benchmark results.

Offensive language benchmark validation fails on target diversity when target_group values contain placeholders or lack variety. The pytest suite requires concrete target_group assignments to ensure comprehensive classification evaluation across different demographics.