Many Languages

Over half of the world's internet users don't read English first. A Spanish customer shouldn't have to wait for a Spanish speaking agent. But most NLP tools were built for English. Luckily, modern embeddings can cross language barriers in a single line.

• Half the internet isn't English
• Most tools were English-first
• Embeddings cross languages

Search across languages

This model places sentences from dozens of languages on the same map of meaning. Our FAQ is in English. The questions arrive in Spanish and German. Run it. Each one finds the right English answer, with no translation step. Meaning doesn't care what language it's written in. So Bright Books can keep one set of documents, and serve the whole world.

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("paraphrase-multilingual-MiniLM-L12-v2", device="cpu")

faq = ["How do I return a book?", "When will my order arrive?"]
vectors = model.encode(faq)

for q in ["Quiero devolver un libro", "Wo ist meine Bestellung?"]:
    scores = util.cos_sim(model.encode(q), vectors)[0]
    print(f"{q:<28} -> {faq[int(scores.argmax())]}  ({scores.max():.2f})")

Quiero devolver un libro     -> How do I return a book?  (0.84)
Wo ist meine Bestellung?     -> When will my order arrive?  (0.75)

Which language is this?

Before replying, Scribe needs to know the language. Here's a simple approach, counting common words. Run it. It works on these tiny examples, and that's the core idea behind language detection. Real libraries count groups of characters, and they're very accurate, even on short text.

def guess(text):
    hints = {"es": ["el", "la", "que", "de", "un", "quiero"],
             "de": ["der", "die", "und", "ist", "wo", "meine"],
             "en": ["the", "and", "is", "my", "where", "a"]}
    words = text.lower().replace("?", "").split()
    return max(hints, key=lambda k: sum(w in hints[k] for w in words))

for t in ["Where is my book", "Quiero un libro", "Wo ist meine Bestellung"]:
    print(f"{guess(t)}  {t}")

en  Where is my book
es  Quiero un libro
de  Wo ist meine Bestellung

Replying naturally

For the reply, we ask a language model to translate, with instructions about tone, and to keep numbers exact. Run it. A warm, natural reply in Spanish, with the five days intact. Translations by language models read naturally, but always have native speakers check important text, like legal policies.

from google import genai

client = genai.Client()

reply = "Your refund will reach your card in 5 working days."

prompt = ("Translate into warm, natural Spanish for a customer. "
          f"Keep numbers exact.\n\n{reply}")

print(client.models.generate_content(model="gemini-flash-latest",
                                     contents=prompt).text.strip())

Here is a warm and natural translation:

**"Tu reembolso se verá reflejado en tu tarjeta en 5 días hábiles."**

*(If your company addresses customers formally with "usted"):*
**"Su reembolso se verá reflejado en su tarjeta en 5 días hábiles."**

Summary