Seeing Distributions

If I say that the average customer visits four times a month, what picture comes to mind? Probably lots of people visiting about four times. It might be very different. Perhaps most people come once, and a few come every day. Averages hide the shape. So always look at the shape.

• An average of four visits
• What's the real shape?
• Averages hide it

Histograms and box plots

Describe gives a quick summary. The mean is above the median, and that's a hint that a few heavy users pull it up. A histogram shows the shape directly. And a box plot shows the middle half, and any outliers. Run it. Look at the left chart. It's skewed, with a long tail on the right. So a typical customer visits less often than the average suggests. Nearly all business data has that shape.

import pandas as pd
import matplotlib.pyplot as plt

df = pd.read_csv("customers.csv").drop_duplicates()

print(df["visits_per_month"].describe().round(2).to_string())

fig, ax = plt.subplots(1, 2, figsize=(10, 4))
ax[0].hist(df["visits_per_month"], bins=40); ax[0].set_title("Visits per month")
ax[1].boxplot(df["avg_spend"], vert=False); ax[1].set_title("Average spend")
plt.tight_layout(); plt.savefig("shape.png", dpi=110)

count    6000.00
mean        4.36
std         2.91
min         0.10
25%         2.20
50%         3.80
75%         5.90
max        29.80

Summary