Topic Modelling

Bright Books has two years of old messages, with no labels at all. What do customers talk about? Nobody has time to read them all. Topic modelling can read a mountain of text, and tell you the main themes, without being told what they are.

• Two years of unlabelled messages
• What do people talk about?
• Find the themes automatically

The idea behind LDA

The classic method is called LDA. It imagines that each document is a mix of topics, and each topic is a mix of words. A space article might be eighty percent space and twenty percent health. Then it works backward from the words to find topics that explain the collection. It only finds groups of words that appear together. A human then has to name them.

Finding topics

We take posts from three subjects, and we throw away the labels. We count the words, removing the very common ones, and the very rare ones. Then we ask for three topics. Run it. Read the word lists. One's clearly about space. One's about health. One's about cars. Nobody told it. It discovered these groups, purely from which words appear together. Modern versions like BERTopic use embeddings instead, and they work even better.

from sklearn.datasets import fetch_20newsgroups
from sklearn.decomposition import LatentDirichletAllocation
from sklearn.feature_extraction.text import CountVectorizer

data = fetch_20newsgroups(subset="train", remove=("headers", "footers", "quotes"),
                          categories=["sci.space", "sci.med", "rec.autos"])

vec = CountVectorizer(stop_words="english", max_df=0.5, min_df=10)
X = vec.fit_transform(data.data)
lda = LatentDirichletAllocation(n_components=3, random_state=0).fit(X)

words = vec.get_feature_names_out()
for i, topic in enumerate(lda.components_):
    print(f"Topic {i}:", ", ".join(words[j] for j in topic.argsort()[-6:][::-1]))

Topic 0: space, nasa, edu, launch, com, data
Topic 1: car, new, use, cars, health, like
Topic 2: like, just, don, people, think, know

Summary