Fuzzy Matching

Customers mention book titles in all sorts of ways. Hary poter. Great gatsbi. Pride prejudice. If Scribe's going to look up a book, then it has to work out which title they meant. And Python's got a tool for this in its standard library.

• Titles get mangled
• Scribe must work out which
• It's in the standard library

Closest match

Get close matches compares a query with a list, and returns the most similar. The cut-off sets how similar it must be. Run it. Hary poter finds Harry Potter. Great gatsbi works too. Pride prejudice works as well. But look at the last line. War and Peace isn't in our shop, and it wrongly matched Pride and Prejudice, simply because they share the word and. That's the danger. A fuzzy matcher will always find something. So raise your cut-off, or check the score, and be willing to say that there's no match.

import difflib

titles = ["Harry Potter and the Philosopher's Stone", "The Great Gatsby", "Pride and Prejudice",
          "To Kill a Mockingbird", "One Hundred Years of Solitude"]

for query in ["hary poter", "great gatsbi", "pride prejudice", "war and peace"]:
    match = difflib.get_close_matches(query.title(), titles, n=1, cutoff=0.4)
    print(f"{query:<18} {match[0] if match else 'no match'}")

hary poter         Harry Potter and the Philosopher's Stone
great gatsbi       The Great Gatsby
pride prejudice    Pride and Prejudice
war and peace      Pride and Prejudice

Fuzzy matching at scale

This simple method compares a query with every title, so it gets slow on a large catalogue. Libraries such as Rapid Fuzz are hundreds of times quicker. For big lists, you first shortlist candidates, using a search index or character n grams, and then you compare only those. And always keep a threshold, so that a confident wrong answer can be turned into a question.

Summary