Faces and Pose

Vision systems that look at people are powerful, and they need great care. They help drivers stay awake, help patients with physiotherapy, and keep workers safe. They can also be used for surveillance. Let's learn how they work, and when you shouldn't use them.

• Powerful, and sensitive
• Safety, health, and fitness
• Also surveillance risks

Finding a face

In two thousand and one, Viola and Jones built a face detector so fast that it ran inside cheap cameras. OpenCV still includes it. Run it. It finds the face, and draws a box. It's fast and small, but it struggles with angles and bad light. Modern deep detectors are far better, though this one still runs on tiny devices.

import cv2
import matplotlib.pyplot as plt
from skimage import data

image = data.astronaut()
gray = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)

path = cv2.data.haarcascades + "haarcascade_frontalface_default.xml"
faces = cv2.CascadeClassifier(path).detectMultiScale(gray, 1.2, 5)

out = image.copy()
for x, y, w, h in faces:
    cv2.rectangle(out, (x, y), (x + w, y + h), (0, 255, 0), 3)
plt.figure(figsize=(5, 5)); plt.imshow(out); plt.axis("off")
plt.tight_layout(); plt.savefig("face.png", dpi=110)
print("Faces found:", len(faces))

Faces found: 1

Body pose

Pose estimation finds a person's joints. Seventeen keypoints, such as the nose, shoulders, elbows, and knees. YOLO has a pose model for this. Run it. It draws a skeleton over the person, and gives the coordinates of each joint. Now a computer can reason about how someone's moving.

import cv2
import matplotlib.pyplot as plt
from skimage import data
from ultralytics import YOLO

model = YOLO("yolo11n-pose.pt")
image = cv2.cvtColor(data.astronaut(), cv2.COLOR_RGB2BGR)
r = model(image, verbose=False)[0]

points = r.keypoints.xy[0]
print("People:", len(r.boxes), " keypoints each:", len(points))
print("Nose at:", points[0].int().tolist())
print("Left shoulder:", points[5].int().tolist())

plt.figure(figsize=(5, 5)); plt.imshow(r.plot()[:, :, ::-1]); plt.axis("off")
plt.tight_layout(); plt.savefig("pose.png", dpi=110)

People: 1  keypoints each: 17
Nose at: [220, 120]
Left shoulder: [311, 249]

From joints to meaning

Once you've got joints, simple geometry gives you real insights. Three points make an angle. We compute an elbow angle, then check a worker's back while lifting. Run it. Orchard Vision can warn workers about risky lifting, with no sensors. Fitness apps count your repetitions in the same way.

import math

def angle(a, b, c):
    ba = (a[0] - b[0], a[1] - b[1])
    bc = (c[0] - b[0], c[1] - b[1])
    dot = ba[0] * bc[0] + ba[1] * bc[1]
    cos = dot / (math.hypot(*ba) * math.hypot(*bc))
    return math.degrees(math.acos(cos))

shoulder, elbow, wrist = (100, 80), (120, 150), (180, 170)
print(f"Elbow angle: {angle(shoulder, elbow, wrist):.0f} degrees")

back = angle((100, 60), (110, 150), (112, 240))
print("Lifting posture:", "safe" if back > 150 else "bending, risk of injury")

Elbow angle: 124 degrees
Lifting posture: safe

Using this responsibly

Face recognition, which identifies who someone is, is very different from detection, which only finds a face. It's banned or limited in many places. Accuracy can differ between groups. Always ask, do people know this is happening? Have they agreed? Could this hurt someone? And if you can solve the task without identifying a person, do that. Blur faces wherever you can.

Summary