taxonomic-interest-classification

Map each corpus item to a hierarchical taxonomy such as the IAB Content Taxonomy, build a category distribution table, then compute Shannon entropy, Simpson's diversity, and an orthogonality score. The orthogonality score classifies the profile from Deep Specialist to Polymathic.

The IAB Content Taxonomy v2.0+ is the recommended baseline, offering 26 Tier-1 categories and 366+ Tier-2 subcategories. Adapt it when the corpus is domain-specific or when more than 15% of items fall into Uncategorized.

Shannon entropy measures the information content of a distribution and is sensitive to rare categories, while Simpson's diversity measures the probability that two random items come from different categories. Using both together captures evenness and concentration resistance.

Corpora under 30 items are too small for diversity metrics; only raw category counts should be reported. Corpora of 30-100 items can be analyzed with low-confidence caveats, and over 100 items supports full analysis.

Avoid it for single-document topic extraction, where NMF or LDA topic modeling fits better, and for keyword frequency analysis without categorical framing. It measures what a distribution is, not what content should be recommended.