Parsing Grammar

Man bites dog is news. Dog bites man isn't. The very same words, but a different meaning, and the only difference is who did what to whom. To get that, a computer has to understand grammar.

• Man bites dog
• Same words, new meaning
• Who did what to whom?

Parts of speech and dependencies

SpaCy tags each word with a part of speech, such as noun or verb. It also draws a dependency tree. Each word points to the word that it depends on. Run it. Returned is the root, the main action. Customer is its subject, and book is its object. Angry describes the customer, and torn describes the book. The whole sentence has become a structure that a program can walk through.

import spacy

nlp = spacy.load("en_core_web_sm")
doc = nlp("The angry customer returned the torn book yesterday.")

for t in doc:
    print(f"{t.text:<10} {t.pos_:<6} {t.dep_:<8} -> {t.head.text}")

The        DET    det      -> customer
angry      ADJ    amod     -> customer
customer   NOUN   nsubj    -> returned
returned   VERB   ROOT     -> returned
the        DET    det      -> book
torn       ADJ    amod     -> book
book       NOUN   dobj     -> returned
yesterday  NOUN   npadvmod -> returned
.          PUNCT  punct    -> returned

Extracting who did what

Let's use this. For each sentence, find the root verb, then its subject, and its object. Run it. Clean little facts, from plain sentences. Sam returned the book. The courier lost a parcel. This is called relation extraction, and it's how people build knowledge graphs from documents. No training needed, for simple sentences, and it runs in a fraction of a second.

import spacy

nlp = spacy.load("en_core_web_sm")
texts = ["Sam returned the book.", "The courier lost my parcel.",
         "Bright Books refunded the customer."]

for text in texts:
    doc = nlp(text)
    root = [t for t in doc if t.dep_ == "ROOT"][0]
    who = [c.text for c in root.children if c.dep_ == "nsubj"]
    what = [c.text for c in root.children if c.dep_ == "dobj"]
    print(f"{who} -> {root.lemma_} -> {what}")

['Sam'] -> return -> ['book']
['courier'] -> lose -> ['parcel']
['Books'] -> refund -> ['customer']

Summary