A Short History of NLP

In nineteen fifty four, researchers proudly demonstrated a machine that translated Russian into English. They predicted that translation would be solved within five years. It took more than sixty. The story of why it took so long teaches you nearly everything that you need to know about the field.

• 1954: machine translation demo
• Five years, they said
• It took over sixty

Three eras of NLP

The first era was rules. People wrote grammar by hand, and it was brittle. In the nineteen nineties came statistics. Models learned from counting words in large collections of text. In the twenty tens came neural networks and embeddings, which captured meaning as numbers. In twenty seventeen, the transformer arrived. And in twenty twenty two, language models reached the public, and everything changed.

• 1950sHand-written rules
• 1990sStatistics
• 2013Word embeddings
• 2017Transformers
• 2022ChatGPT

What we still use from each era

Here's the surprise. Nothing has been thrown away. Rules still clean text, and catch things like email addresses. Counting still powers fast search, and cheap classifiers. Embeddings power semantic search. And transformers handle the hard parts. A great engineer picks the simplest tool that works. That's why this course goes through all of them, instead of jumping straight to chatbots.

Summary