llm-text-analysis

You can handle invalid JSON from LLM outputs during text annotation by applying robust parsing and defensive recovery strategies, while tracking failure rates to maintain transparent reporting of your structured coding results.

Designing a codebook for zero-shot text annotation involves creating structured, category-aligned prompts within a YAML-defined workflow, ensuring the LLM produces structured coded variables that match your social-science research schema.

Yes, you can use pandas and numpy to manage social-science text coding variables by structuring the parsed LLM outputs into dataframes, aggregating multiple annotation runs, and computing statistical agreement metrics for validation.

The best way to select an LLM model for reproducible text annotation is to evaluate models based on reproducibility, cost, task complexity, and stability requirements, documenting the model version and validation protocol transparently.

LLM text annotations produce different results across multiple runs due to nondeterminism; you must run multiple annotation passes, aggregate the outputs, and calculate agreement metrics like Cohen’s κ to ensure reliability.

Limitations of using prompt engineering for social-science bias checking include format drift, model instability, and silent error propagation, which require defensive parsing, bias checks, and transparent reporting of limitations to mitigate.