Explaining Hard Ideas

Richard Feynman said that if you can't explain something simply, then you don't really understand it. For a data scientist, it's even more practical. If the founder can't follow your explanation, then your work won't be used. Let's practise on the hardest ideas in this course.

• Feynman's test
• Explain it simply
• Or it won't get used

Explaining a p-value

Don't say, the probability of data at least as extreme, under the null. Say this. Imagine that the new design did nothing at all. If we ran this test many times, we'd see a gap this large only about four times in a hundred, by luck. So it's probably a real effect, although it's not certain. Then state the size of the effect, because that's what people need.

Explaining a confidence interval

Say, our best estimate is a lift of one point four points, and it's probably somewhere between a half and two and a half. If I had to bet, I'd say it's positive. But it could be small. A range shows honesty, and it invites a good discussion. Should we act, if the low end is true?

Explaining regression

Say, it's like comparing two customers who are the same in every way that we've measured, except that one visits more often. The number shows the difference that we'd expect in their spending. And add a warning. There may be things that we didn't measure. Now anyone can follow it.

Using analogies well

Analogies work well. Statistical power is like a telescope. A weak one can't see a small planet. Overfitting's like a student who memorises past exam papers, and can't handle a new question. A confounder is a hidden puppeteer. Try your explanation on a friend who's not in the field. If they can repeat it back, you've done it.

Summary