llm-text-analysis

What problem does it solve?

This Skill solves the problem of turning unstructured social-science text into reliable, structured annotations without silently propagating model errors, instability, or bias.

Core Features & Use Cases

• Prompt and codebook design: Produce structured, category-aligned prompts for social-science coding tasks.
• Validation and nondeterminism management: Run multiple LLM passes, aggregate results, and assess agreement using metrics like Cohen’s κ and F1.
• Robust output handling: Parse and recover from format drift (e.g., invalid JSON) with defensive strategies and failure-rate reporting.
• Model selection guidance: Choose models based on reproducibility, cost, task complexity, and stability requirements.
• Ethical and reporting standards: Document model/version, prompts, parameters, validation protocol, bias checks, and limitations.

Quick Start

Use the llm-text-analysis skill to annotate a set of social-science texts by specifying a codebook, running multiple annotation passes, and validating LLM labels against a human-coded subset.