A Transformer Toolbox
Extract, answer, summarise, translate.
In the next few minutes, Scribe will gain four new skills, with almost no code. Each of these took researchers decades. And today, you can use them with a single line. This is what it means to stand on the shoulders of giants.
- Four new skills
- Each took decades
- One line each
Better entity extraction
A transformer NER model. We give it a sentence with a person, a city, and two companies. Run it. Clean, confident results, even for tricky names. It's more accurate than our smaller spaCy model, though it's slower, and you'd want a GPU at high volume.
from transformers import pipeline
ner = pipeline("ner", model="dslim/bert-base-NER", aggregation_strategy="simple",
device="cpu")
text = "Maria Lopez from Madrid emailed Bright Books about her Amazon order."
for e in ner(text):
print(f"{e['word']:<15} {e['entity_group']:<5} {e['score']:.2f}")Maria Lopez PER 1.00 Madrid LOC 1.00 Bright Books ORG 1.00 Amazon ORG 0.53 [transformers] [1mBertForTokenClassification LOAD REPORT[0m from: dslim/bert-base-NER Key | Status | | -------------------------+------------+--+- bert.pooler.dense.weight | UNEXPECTED | | bert.pooler.dense.bias | UNEXPECTED | | Notes: - UNEXPECTED: can be ignored when loading from different task/architecture; not ok if you expect identical arch.
Answering from a passage
Question answering models read a passage, and point to the exact span that answers a question. We give it the shop's return policy, and ask two questions. Run it. It finds five working days, and for gift cards, it picks out the right part. But notice that it can only quote the passage. It can't compose an answer of its own. We'll fix that with RAG.
import torch
from transformers import AutoModelForQuestionAnswering, AutoTokenizer
name = "distilbert-base-cased-distilled-squad"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForQuestionAnswering.from_pretrained(name)
def ask(question, context):
enc = tok(question, context, return_tensors="pt")
with torch.no_grad():
out = model(**enc)
a, b = out.start_logits.argmax(), out.end_logits.argmax()
return tok.decode(enc["input_ids"][0][a:b + 1])
policy = ("Books can be returned within 30 days if unread. Refunds reach your "
"card in 5 working days. We do not refund gift cards.")
for q in ["How long do refunds take?", "Can I return a gift card?"]:
print(f"{q:<30} {ask(q, policy)}")How long do refunds take? 5 working days Can I return a gift card? We do not refund gift cards
Summarising a complaint
Staff don't have time for long complaints. This model reads the whole message, and writes a short version. Max length limits how long it can be. Run it. Six sentences become one or two. A model like this is writing new text, and that means it can sometimes get details wrong. So for important messages, a person should still check the original.
from transformers import AutoTokenizer, BartForConditionalGeneration
name = "sshleifer/distilbart-cnn-12-6"
tok = AutoTokenizer.from_pretrained(name)
model = BartForConditionalGeneration.from_pretrained(name)
complaint = ("I ordered three books for my daughter's birthday on the fifth of March. "
"The site said they would come within two days. A week later nothing "
"had arrived, so I phoned and waited forty minutes. The agent was polite "
"but could not find my order. When the box finally came, one book was "
"torn and another was the wrong title. I want a refund for both.")
ids = tok(complaint, return_tensors="pt", truncation=True)
out = model.generate(**ids, max_length=45, min_length=15, num_beams=2)
print(tok.decode(out[0], skip_special_tokens=True))One book was torn and another was the wrong title . The site said they would come within two days . A week later nothing had arrived . [transformers] Please make sure the generation config includes `forced_bos_token_id=0`.
Translating replies
One more. A translation model, from English to German. Run it. Two clean translations, on a laptop, for free. Models from the Helsinki team cover over a thousand language pairs. So Bright Books can now reach readers around the world.
from transformers import MarianMTModel, MarianTokenizer
name = "Helsinki-NLP/opus-mt-en-de"
tok = MarianTokenizer.from_pretrained(name)
model = MarianMTModel.from_pretrained(name)
for text in ["Your order has shipped.", "We are sorry for the delay."]:
out = model.generate(**tok(text, return_tensors="pt"), max_length=40)
print(f"{text:<30} {tok.decode(out[0], skip_special_tokens=True)}")Your order has shipped. Ihre Bestellung wurde versendet. We are sorry for the delay. Es tut uns leid für die Verzögerung.
Summary
Extraction, questions, summaries, and translation. Scribe has grown a lot. All of this used small specialised models. But one large language model can do all of these, and more, if you know how to ask. Next, we step up to LLMs.
Key takeaways- Stronger NER
- QA quotes from a passage
- Summaries can get details wrong