Predict with a linear model

Learn it

Words to know

feature

One of the numbers that describe an example, such as one pixel of an image or one word count.

weight

A number in the model that an input is multiplied by. Training changes the weights.

bias

A number added after the weighted sum. It lets the result be something other than 0 when every input is 0.

dot product

Multiply two lists of numbers position by position and add up the results. It gives one number.

How it works

1. Take one row of X. Multiply each feature by the weight in the same position.
2. Add those products up. This is the dot product of the row and the weights.
3. Add the bias. That is the prediction for this row.
4. Repeat for every row. The weights and the bias stay the same.

Worked example

Input
X = [[1, 2], [3, 4]]
weights = [2, -1]
bias = 1

Output
[1, 3]

Two rows in, two predictions out. The weight -1 means the second feature pulls the prediction down.

Why it matters

A neuron in chapter 2 is this sum followed by an activation. In PyTorch the whole table is done in one step, written X @ W + b.

Your turn

Predict one number for each row of a table with a linear model.

X

a table: a list of rows, each a list of features; may have no rows

weights

one weight per feature

bias

a number added to every prediction

Returns

A list with one prediction per row: the dot product of the row and weights, plus bias.

Input
X = [[1, 2], [3, 4]]
weights = [2, -1]
bias = 1

Output
[1, 3]

Input
X = [[1, 1], [2, 0]]
weights = [3, 1]
bias = -1

Output
[3, 5]

Input
X = []
weights = [1]
bias = 0

Output
[]

Hints

Each hint gives away a little more. Open one only when you need it.

1. Hint 1: Where to start

   The answer has one number per row, so loop over the rows of X.

2. Hint 2: The key step

   For one row, pair each feature with its weight using zip, multiply each pair, and add the products.

3. Hint 3: The outline

   For each row in X: start total at 0, add value * weight for each pair from zip(row, weights), then append total + bias to the result.

Solution

From here on the page gives the method away. If you are still solving, go back to the hints.

Step by step

1. Multiply each feature by its weight and add the products. zip pairs the first feature with the first weight, the second with the second, and so on. Their products added together are the dot product.
2. The bias shifts the prediction up or down. Without a bias, a row of zeros could only ever predict 0.

The same numbered steps are the comments in the solution.

def solve(X, weights, bias):
    predictions = []
    for row in X:
        # 1. Multiply each feature by its weight and add the products
        total = 0
        for value, weight in zip(row, weights):
            total += value * weight
        # 2. The bias shifts the prediction up or down
        predictions.append(total + bias)
    return predictions

Watch out

• Add the bias once per row, after the sum. Adding it inside the inner loop adds it once per feature.
• Start the total again at 0 for every row. A total kept across rows mixes one example into the next.

Check yourself