text-as-data-social

To convert unstructured social science text into quantitative variables, apply dictionary-based content analysis, topic modeling, and sentiment analysis to generate analysis-ready feature tables from corpora like policy texts, news, or speeches.

Yes, you can use BERTopic and LDA for topic modeling on social media corpora. The workflow discovers themes and leverages sentence embeddings to make topics meaningful and comparable across documents, outputting a document-by-topic feature table.

Dictionary coding works for measuring ideology or framing in policy texts by building and applying LIWC-style dictionaries to generate interpretable category proportions and counts for specific word categories.

To validate LLM-assisted annotation for qualitative text datasets, apply validation principles that translate qualitative coding schemes into modelable datasets, ensuring LLM labeling workflows maintain consistency across the corpus.

Python dependencies for text embeddings and sentiment analysis include sentence-transformers, transformers, scikit-learn, numpy, and pandas, supporting embedding-based similarity measures and transformer-based sentiment scoring.

This approach is suitable for large-scale survey text corpora, applying topic modeling and embedding-based semantic measurement to process surveys, speeches, and social media texts into structured analytical variables.