text-classification

To design a codebook for LLM text classification, define five components for each code: Label, Definition, Clarification, Negative clarification, and Examples. This structured approach ensures explicit prompt construction and reproducible results.

The best way to validate LLM text classification is to run a 50–100 response pilot test against human ground truth. This workflow evaluates accuracy, considers inter-coder reliability, and supports hybrid human-LLM workflows for uncertain classifications.

Choosing between open-weight and proprietary models for social science text classification depends on your task's learning regime—zero-shot, few-shot, fine-tuning, or instruction-tuning—and your specific safety considerations and reproducibility requirements.

Yes, you can use zero-shot learning for text classification without fine-tuning. It is one of the supported learning regimes, alongside few-shot and instruction-tuning, allowing you to classify social science text based purely on prompt construction.

To document reproducibility for LLM text classification pipelines, report the full pipeline including codebook design, model selection, prompt construction, and pilot testing results. Transparent reporting practices ensure your social science findings are publishable.

You should use hybrid human-LLM workflows for text classification when handling uncertain classifications. This approach combines automated processing with human review to maintain high accuracy and robust reporting in social science research.