Sentiment Analysis

The owner of Bright Books wants to know, are customers getting happier or angrier? A complaint from someone furious should go to the top of the pile. Sentiment analysis reads the feeling behind words. It's one of the most widely used NLP tasks, and it's far trickier than it looks.

• Happier or angrier?
• Furious customers first
• Trickier than it looks

A rule-based approach

VADER is a clever rule based tool, with a dictionary of words and their scores. It also knows some rules. Capital letters and exclamation marks make a feeling stronger, and not flips it. Run it. Great is positive. GREAT with exclamation marks scores higher. And not great flips to negative. It needs no training, and it's perfect for quick jobs. But look at the last one, with mixed feelings. It gets confused.

from nltk.sentiment import SentimentIntensityAnalyzer

sia = SentimentIntensityAnalyzer()

for text in ["The book is great!", "The book is GREAT!!!",
             "The book is not great.", "Lovely packaging, shame about the book."]:
    print(f"{sia.polarity_scores(text)['compound']:+.2f}  {text}")

+0.66  The book is great!
+0.77  The book is GREAT!!!
-0.51  The book is not great.
+0.18  Lovely packaging, shame about the book.

A model against sarcasm

Let's try a transformer, on three hard cases. Sarcasm, double negatives, and mixed opinions. Run it. Read each result. Double negatives are often handled well. But sarcasm remains hard, since the words themselves are positive. And a single label can't describe a mixed review at all. Sentiment isn't one number. A real review has a separate feeling for each thing it mentions.

from transformers import pipeline

clf = pipeline("sentiment-analysis", device="cpu",
               model="distilbert-base-uncased-finetuned-sst-2-english")

for text in ["Delivery took six weeks. Fantastic service.",
             "It was not bad at all.",
             "Lovely packaging, shame about the book."]:
    r = clf(text)[0]
    print(f"{r['label']:<9} {r['score']:.2f}  {text}")

POSITIVE  1.00  Delivery took six weeks. Fantastic service.
POSITIVE  1.00  It was not bad at all.
POSITIVE  1.00  Lovely packaging, shame about the book.

Sentiment per aspect

So let's do better. Aspect based sentiment finds what each opinion is about. We use grammar to link a feeling word to its subject. Our sentence says that the delivery was late, the book was lovely, and the staff were rude. Run it. Three subjects, each with its own feeling. Now the owner knows that the books are fine, but that delivery and staff need work. This is far more useful than a single score.

import spacy

nlp = spacy.load("en_core_web_sm")

good = {"great", "lovely", "fast", "friendly"}
bad = {"late", "torn", "rude", "slow"}
text = "The delivery was late but the book was lovely and the staff were rude"

for token in nlp(text):
    if token.lower_ in good | bad:
        subject = [c.text for c in token.head.children if c.dep_ == "nsubj"]
        mood = "good" if token.lower_ in good else "bad"
        print(f"{subject}: {token.text} ({mood})")

['delivery']: late (bad)
['book']: lovely (good)
['staff']: rude (bad)

Summary