Skip to content
Model foundations · 1 of 4 · about 15 minutes

Vector dot product

A model cannot read words. It turns each word, sentence or document into a list of numbers, called a vector. The most basic way to compare two vectors is the dot product: multiply the numbers that sit in the same position, then add the results. Two vectors that are large in the same positions, with the same signs, get a large total. Most of what a model computes is built from this one operation.

Learn it

Words to know

vector
A list of numbers, such as [1, 2, 3]. Each position is called a dimension.
dot product
One number made from two vectors: multiply the numbers at the same position, then add the products.

How it works

  1. Line the two vectors up: position 0 with position 0, position 1 with position 1, and so on.
  2. Multiply each pair: a[i] * b[i].
  3. Add all the products. With no positions at all, the sum is 0.

Worked example

Input
a = [1, 2, 3]
b = [4, 5, 6]

Output
32
Positionaba × bRunning total
01444
1251014
2361832
012
a123
b456
a × b41018
  • added together
Each column is one position. The bottom row holds the products, and the dot product is their sum: 4 + 10 + 18 = 32.

The answer is one number, 32, however long the vectors are. The biggest product comes from position 2, where both vectors are large.

Why it matters

The next lesson divides a dot product by two lengths to get cosine similarity, which is how a search system decides which document is closest to a question.

Your turn

Return the dot product of two vectors with the same number of positions.

a
a list of numbers, possibly empty; values can be negative
b
a list with as many numbers as a
Returns
One number: the sum of a[i] * b[i] over every position. For two empty lists, 0.

What JavaScript gives you

a.length
how many numbers the list holds
for (let i = 0; i < a.length; i++)
visits every position

What Python gives you

len(a)
how many numbers the list holds
for i in range(len(a)):
visits every position

Example 1

Input
a = [1, 2, 3]
b = [4, 5, 6]

Output
32

4 + 10 + 18 = 32, as in the worked example above.

Example 2

Input
a = [-2, 3]
b = [4, -1]

Output
-11

The products are -8 and -3. A negative product pulls the total down, so the answer is -11.

Example 3

Input
a = []
b = []

Output
0

With nothing to multiply, the sum stays at its starting value, 0.

The checks also try

  • a vector of zeros
  • decimal values
  • one position
  • the same number twice in one vector

Hints

Each hint gives away a little more. Open one only when you need it.

  1. Hint 1: Where to start

    You need one running total that starts at 0, and one loop over the positions.

  2. Hint 2: The key step

    On each pass, multiply the two numbers at the same position, a[i] and b[i], and add the product to the total.

  3. Hint 3: The outline

    Set total to 0. For each position i, from 0 to the last one, add a[i] * b[i] to total. After the loop, return total.

Solution

From here on the page gives the method away. If you are still solving, go back to the hints.

Step by step

  1. Start the total at zero. Two empty vectors never enter the loop, so the starting value is also the answer for that case.
  2. Add the product of the two numbers at each position. Each position contributes one product. A position where both numbers are large raises the total the most.
  3. The total is the dot product. After the last position nothing is left to add, so the total is returned as it is.

The same numbered steps are the comments in the solution.

JavaScript
function solve(a, b) {
  // 1. Start the total at zero
  let total = 0;
  for (let i = 0; i < a.length; i++) {
    // 2. Add the product of the two numbers at each position
    total += a[i] * b[i];
  }
  // 3. The total is the dot product
  return total;
}
Python
def solve(a, b):
    # 1. Start the total at zero
    total = 0
    for i in range(len(a)):
        # 2. Add the product of the two numbers at each position
        total += a[i] * b[i]
    # 3. The total is the dot product
    return total

Watch out

  • Do not add the two numbers at a position. The dot product multiplies each pair first, and only then adds.
  • Do not start the total at 1, as you would for a running product. Two empty vectors must give 0.

Check yourself

What does a large positive dot product tell you about two vectors?

They are large in the same positions, with matching signs. Length counts too: making either vector ten times longer makes the dot product ten times larger.

Why is the dot product of two empty vectors 0?

A sum with nothing to add stays at its starting value, and a sum starts at 0.