register-variation-code-switching

Segment the corpus into context-defined sub-corpora, extract per-text feature distributions for vocabulary, formality, sentence length, and sentiment, then compare distributions pairwise using KS tests and effect sizes like Cohen's d and Cliff's delta. Comparing distributions rather than means is what reveals register variation.

The two-sample Kolmogorov-Smirnov test compares entire distribution shapes, while Mann-Whitney U tests location differences non-parametrically. Effect sizes such as Cohen's d and Cliff's delta separate meaningful variation from statistical noise, which matters because large corpora make trivial differences significant.

Each context needs at least 300 words and ideally 5 or more texts. Contexts below 300 words are excluded from pairwise comparison, and contexts with fewer than 5 texts are flagged as low-sample with unreliable effect sizes.

Skip it when the corpus comes from a single context, when fewer than two contexts have sufficient text, when the text is multilingual and the goal is detecting language-switching, or when the corpus has multiple authors. Single-context corpora should use stylometric fingerprinting instead.

A stable register means no feature dimension shows large effect sizes across context pairs, so one global style profile applies everywhere. A context-dependent register means multiple dimensions vary systematically, requiring conditional rules such as different formality and sentence-length targets per context.