Logistic Regression

Good news. You have been promoted. HomeWorth is partnering with a clinic, which wants help with a screening tool, to flag tumours that need a closer look. This is a different type of question. The answer is not a number, it is yes or no. And a wrong answer could really hurt someone, so we need to be very careful. Welcome to classification.

• A new mission: medical screening
• Not a number: yes or no
• Mistakes really matter here

Squashing a line into a chance

Could we use a straight line? The trouble is that a line runs off to infinity, and gives answers like two hundred percent. We want a chance between zero and one. So we take our usual weighted sum, and push it through a function called the sigmoid. It is an S shaped curve that squashes any number into the range zero to one. Very negative numbers become nearly zero, and very positive ones become nearly one. Then we draw a line at point five. Above it, say yes. Below it, say no. That is logistic regression, and despite the name, it classifies.

Our first classifier

Here is a real medical dataset, with measurements from tumour images. We split it with stratify, which keeps the same balance of classes on both sides. The model is a pipeline, scale and then logistic regression. Accuracy is about ninety six percent. But look at the next line. Predict proba gives the chance itself, not only the decision. Those chances are often more useful than the yes or no, because they show how confident the model is.

from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X, y = load_breast_cancer(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(
    X, y, test_size=0.25, stratify=y, random_state=0)

model = make_pipeline(StandardScaler(), LogisticRegression())
model.fit(X_tr, y_tr)

print("Accuracy:", round(model.score(X_te, y_te), 3))
print("Class 1 chances:", model.predict_proba(X_te[:4])[:, 1].round(3))
print("Decisions:      ", model.predict(X_te[:4]))

Accuracy: 0.958
Class 1 chances: [0.996 0.    0.    1.   ]
Decisions:       [1 0 0 1]

Summary