AI Hints
Statistics & Math for ML · Lesson 77 of 138

Vector Arithmetic

Add and scale.

⏱ 2 min read▶ Video at 1:30:59🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

You can add two vectors, and multiply a vector by a number. These two simple operations are the building blocks of almost everything in linear algebra.

Vector Arithmetic
From the video: Vector Arithmetic
  • Add vectors
  • Multiply by a number
  • The building blocks

Tip to tail

Adding vectors adds them position by position. As arrows, it's like walking along one arrow, then the other. Multiplying by a number, called a scalar, stretches the arrow. Run it. The cyan arrow is V plus W, and it ends exactly where W ends, if you start W at the tip of V.

add.pypython
import numpy as np
import matplotlib.pyplot as plt

v, w = np.array([3, 1]), np.array([1, 2])
print("v + w =", v + w, "  2v =", 2 * v)

for vec, c in [(v, "#e879f9"), (w, "#fbbf24"), (v + w, "#22d3ee")]:
    plt.quiver(0, 0, *vec, color=c, angles="xy", scale_units="xy", scale=1)
plt.quiver(*v, *w, color="#fbbf24", alpha=0.4, angles="xy", scale_units="xy", scale=1)
plt.xlim(0, 5); plt.ylim(0, 4); plt.title("v, w and v + w")
plt.savefig("add.png")
Output
v + w = [4 3]   2v = [6 2]

Why it matters

In machine learning, gradient descent updates weights by subtracting a scaled vector. New weights equal old weights, minus a learning rate times the gradient. And the average of many data points is just their vector sum, divided by how many there are. Simple operations, used billions of times.

keyvalue
v + wadd position by position
c × vstretch or shrink
−vflip direction
ML updatew − lr × gradient

Summary

Vectors add position by position, and scalars stretch them. Next, the most important vector operation in machine learning, the dot product.

Key takeaways
  • v + w
  • 2 × v
  • Tip to tail
Next up: The Dot Product — multiply and sum