Hard NLP Questions

Here are some questions which strong candidates often stumble on. Try to answer each one yourself, before you listen.

• Strong candidates stumble
• Try each one
• Then listen

Your RAG bot gives wrong answers. How do you debug it?

Split the problem. First, is the right passage being retrieved? If not, it's a search problem. So look at chunking, embeddings, and hybrid search. If it is retrieved, it's a generation problem, so look at the prompt. Measure each one separately. Never tune everything at once.

How do you reduce an LLM bill by 80 percent?

Layer it. Handle the easy cases with rules, and with small models. Cache repeated answers, and use prompt caching. Shorten your prompts, and retrieve less. Use a smaller model for routing. Batch whatever can wait. And measure quality at each step, so that you know what's been lost.

Your classifier is 95 percent accurate. Ship it?

Ask more questions. What's the base rate? If one class is ninety five percent of the data, this model may be doing nothing. Look at precision and recall for each class. What does a mistake cost? Was the test data similar to live data? Accuracy alone is never an answer.

Summary