aiml-detoxify-benchmark

To benchmark Detoxify toxicity detection, provide a test dataset with non-placeholder text and a Detoxify model to score predictions. The benchmark script automates cross-category evaluation to generate scores and recall metrics for six toxicity categories.

A Detoxify benchmark evaluates six Jigsaw toxicity categories: toxic, severe_toxic, obscene, threat, insult, and identity_hate. It assesses model performance across these categories and their styled variants to generate recall metrics.

Toxicity benchmarking requires a test dataset with non-placeholder text and minimum length thresholds. The benchmark validates dataset quality by checking for placeholders, minimum text length, and category coverage before running evaluations.

A recall-based toxicity benchmark supports AI safety evaluations by generating scores and recall metrics across six toxicity categories. This repeatable evaluation harness validates model performance and reliability for safety-critical deployments.

Yes, the benchmark automates ML evaluation for styled variants of toxic text. It applies a repeatable evaluation harness to Detoxify-based classifiers to validate model performance across six Jigsaw toxicity categories and their styled variants.

A toxicity benchmark fails dataset quality validation if the test data contains placeholders, does not meet minimum text length thresholds, or lacks category coverage. The benchmark script checks these conditions before scoring predictions with the Detoxify model.