Log Transforms
Taming skewed data.
Incomes, prices, and website visits are often strongly skewed. A few huge values stretch the scale, and squash everything else together. Taking the logarithm is the most common fix in data science.

- Incomes, prices, visits
- A few huge values
- The log fix
Before and after
On the left are skewed incomes. On the right, the same incomes after NP dot log one P, which is the log of one plus each value, so zeros are safe. Run it. The long tail is gone, and the logged data looks close to a bell curve.
import numpy as np
import matplotlib.pyplot as plt
rng = np.random.default_rng(0)
income = rng.lognormal(10, 0.8, 5000)
fig, ax = plt.subplots(1, 2, figsize=(9, 4.5))
ax[0].hist(income, bins=60); ax[0].set_title("Income")
ax[1].hist(np.log1p(income), bins=60); ax[1].set_title("log(1 + income)")
plt.tight_layout(); plt.savefig("log.png")Thinking in ratios
A log turns multiplying into adding. A pay rise from ten thousand to twenty thousand, and from a hundred thousand to two hundred thousand, are the same step on a log scale, because both are doubling. That often matches how the world works. To get back to the original units, use exp, or exp m one.
| key | value |
|---|---|
| np.log1p | log(1 + x), safe at 0 |
| Undo | np.expm1 |
| Effect | pulls in long tails |
| Meaning | equal steps are equal ratios |
Summary
Log transforms pull in long right tails, and turn ratios into steps. Next, looking at the correlations between many features at once.
Key takeaways- np.log1p
- Skewed to symmetric
- np.expm1 to undo