AI Hints
Home›Statistics & Math for ML Tutorial›Cross-Validation and Interviews
Statistics & Math for ML · Lesson 117 of 138

Interview Questions: Part 2

Math for ML.

⏱ 2 min read▶ Video at 2:19:31🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

Machine learning interviews also test the maths behind the models. Here are more questions, again with answers you could say in half a minute.

Interview Questions: Part 2
From the video: Interview Questions: Part 2
  • The maths behind models
  • More questions
  • Half-minute answers

Bias, variance, and regularization

Explain the bias variance trade off. Simple models underfit, with high bias. Complex models overfit, with high variance. We want the balance that does best on new data. What's the difference between L one and L two regularization? L two shrinks all weights smoothly. L one can push weights to zero, selecting features.

keyvalue
High biasunderfitting
High varianceoverfitting
L2 / ridgeshrinks weights
L1 / lassozeros some weights

Gradients and losses

How does gradient descent work? Compute the gradient of the loss, and step the weights in the opposite direction, scaled by the learning rate. Why use cross entropy for classification? It heavily punishes confident wrong answers, and gives good gradients with softmax. It's the negative log likelihood.

keyvalue
Gradient descentw ← w − lr × ∇loss
Too big lrdiverges
Cross-entropy−log p(correct)
Equalsnegative log likelihood

Metrics and PCA

When is accuracy misleading? When classes are imbalanced, like fraud. Use precision, recall, or AUC instead. What does PCA do? It finds new uncorrelated directions, ordered by how much variance they capture, using the eigenvectors of the covariance matrix, so you can keep fewer features.

keyvalue
Imbalanceddon't trust accuracy
Useprecision, recall, AUC
PCAdirections of max variance
Fromcovariance eigenvectors

Summary

Know bias and variance, L one versus L two, gradient descent, cross entropy, metrics for imbalanced data, and PCA. Next, three practice projects that put it all together.

Key takeaways
  • Bias-variance
  • Gradient descent
  • PCA
Next up: Practice: Exploring a Dataset — describe, plot, question