Vector dot product
A model cannot read words. It turns each word, sentence or document into a list of numbers, called a vector. The most basic way to compare two vectors is the dot product: multiply the numbers that sit in the same position, then add the results. Two vectors that are large in the same positions, with the same signs, get a large total. Most of what a model computes is built from this one operation.
Learn it
Words to know
- vector
- A list of numbers, such as
[1, 2, 3]. Each position is called a dimension. - dot product
- One number made from two vectors: multiply the numbers at the same position, then add the products.
How it works
- Line the two vectors up: position 0 with position 0, position 1 with position 1, and so on.
- Multiply each pair:
a[i] * b[i]. - Add all the products. With no positions at all, the sum is 0.
Worked example
Input a = [1, 2, 3] b = [4, 5, 6] Output 32
| Position | a | b | a × b | Running total |
|---|---|---|---|---|
| 0 | 1 | 4 | 4 | 4 |
| 1 | 2 | 5 | 10 | 14 |
| 2 | 3 | 6 | 18 | 32 |
| 0 | 1 | 2 | |
|---|---|---|---|
| a | 1 | 2 | 3 |
| b | 4 | 5 | 6 |
| a × b | 4 | 10 | 18 |
- added together
The answer is one number, 32, however long the vectors are. The biggest product comes from position 2, where both vectors are large.
Why it matters
The next lesson divides a dot product by two lengths to get cosine similarity, which is how a search system decides which document is closest to a question.
Your turn
Return the dot product of two vectors with the same number of positions.
a- a list of numbers, possibly empty; values can be negative
b- a list with as many numbers as
a - Returns
- One number: the sum of
a[i] * b[i]over every position. For two empty lists, 0.
What JavaScript gives you
a.length- how many numbers the list holds
for (let i = 0; i < a.length; i++)- visits every position
What Python gives you
len(a)- how many numbers the list holds
for i in range(len(a)):- visits every position
Example 1
Input a = [1, 2, 3] b = [4, 5, 6] Output 32
4 + 10 + 18 = 32, as in the worked example above.
Example 2
Input a = [-2, 3] b = [4, -1] Output -11
The products are -8 and -3. A negative product pulls the total down, so the answer is -11.
Example 3
Input a = [] b = [] Output 0
With nothing to multiply, the sum stays at its starting value, 0.
The checks also try
- a vector of zeros
- decimal values
- one position
- the same number twice in one vector
Hints
Each hint gives away a little more. Open one only when you need it.
Hint 1: Where to start
You need one running total that starts at 0, and one loop over the positions.
Hint 2: The key step
On each pass, multiply the two numbers at the same position,
a[i]andb[i], and add the product to the total.Hint 3: The outline
Set
totalto 0. For each positioni, from 0 to the last one, adda[i] * b[i]tototal. After the loop, returntotal.
Solution
From here on the page gives the method away. If you are still solving, go back to the hints.
Step by step
- Start the total at zero. Two empty vectors never enter the loop, so the starting value is also the answer for that case.
- Add the product of the two numbers at each position. Each position contributes one product. A position where both numbers are large raises the total the most.
- The total is the dot product. After the last position nothing is left to add, so the total is returned as it is.
The same numbered steps are the comments in the solution.
JavaScript
function solve(a, b) {
// 1. Start the total at zero
let total = 0;
for (let i = 0; i < a.length; i++) {
// 2. Add the product of the two numbers at each position
total += a[i] * b[i];
}
// 3. The total is the dot product
return total;
}Python
def solve(a, b):
# 1. Start the total at zero
total = 0
for i in range(len(a)):
# 2. Add the product of the two numbers at each position
total += a[i] * b[i]
# 3. The total is the dot product
return total
Watch out
- Do not add the two numbers at a position. The dot product multiplies each pair first, and only then adds.
- Do not start the total at 1, as you would for a running product. Two empty vectors must give 0.
Check yourself
What does a large positive dot product tell you about two vectors?
They are large in the same positions, with matching signs. Length counts too: making either vector ten times longer makes the dot product ten times larger.
Why is the dot product of two empty vectors 0?
A sum with nothing to add stays at its starting value, and a sum starts at 0.