media-factcheck-benchmark

This skill builds a zero-shot benchmark dataset for automated claim verification across six misinformation categories by forcing the LLM to fill in claims and fabricated evidence per entry, testing robustness in media analysis and fact-checking pipelines.

Automated fact-checking benchmarks cover six misinformation domains: health, politics, finance, science, history, and conspiracy, using domain-marker validation to ensure generated claims and fabricated evidence stay within these specific categories.

Automated claim verification enforces minimum word counts for claims and fabricated evidence through validator-driven checks, ensuring the generated benchmark dataset contains sufficiently detailed entries to robustly test LLM safety and factuality workflows.

Yes, you can use this benchmark dataset to evaluate LLM safety and factuality in media analysis pipelines by integrating a classifier to rate claims, providing a structured and reproducible evaluation framework for misinformation research.

The best way to test LLM robustness against fabricated evidence is using a zero-shot benchmark dataset that forces the model to fill in claims and fabricated evidence across six misinformation categories, validating outputs with domain markers and minimum word counts.