Quiz: Safety and Evals

Last round. These are the questions which separate someone who's built a demo from someone who can run an agent in production.

• The last round
• Demo builders
• Production engineers

Question 1

Question 1. What is indirect prompt injection? A, A user typing an attack. B, An attack hidden in content the agent reads. C, A slow model. D, A broken tool. Pause the video and choose. The answer is B. The agent can be turned against its own user.

Question 2

Question 2. What's the most reliable defence against it? A, A stronger prompt. B, Limiting what a tricked model can do. C, A bigger model. D, Longer instructions. Pause the video and choose. The answer is B. Assume that it's been fooled.

Question 3

Question 3. Which is NOT part of the lethal trifecta? A, Private data. B, Untrusted content. C, A way to send data out. D, A large model. Pause the video and choose. The answer is D. Remove any one of the three.

Question 4

Question 4. Where should hard rules live? A, In the prompt. B, In code. C, In memory. D, In a summary. Pause the video and choose. The answer is B. Code can't be persuaded.

Question 5

Question 5. What should memory never do? A, Store dates. B, Grant permissions. C, Have sources. D, Expire. Pause the video and choose. The answer is B. Treat it like untrusted notes.

Question 6

Question 6. How should generated code be run? A, On your laptop. B, In a sandbox, without secrets. C, As root. D, In production. Pause the video and choose. The answer is B. No network, no keys, time limits.

Question 7

Question 7. What is a trajectory eval? A, Testing the tools used. B, Testing the speed. C, Testing a chart. D, Testing a model's weights. Pause the video and choose. The answer is A. Right answers can hide bad process.

Question 8

Question 8. Why repeat each eval case several times? A, To cost more. B, Outputs vary. C, It's required. D, To warm up. Pause the video and choose. The answer is B. Look at pass rates.

Question 9

Question 9. What's the best first stage of a launch? A, Full launch. B, Shadow mode. C, Marketing. D, A press release. Pause the video and choose. The answer is B. Nobody sees the replies.

Question 10

Question 10. Why put stable text first in a prompt? A, It reads better. B, Prompt caching matches from the start. C, It's shorter. D, It's safer. Pause the video and choose. The answer is B. Things that change go last.

Summary