Measure token cost

Learn it

Words to know

token

The unit a model reads and writes: a word, a piece of a word, or a punctuation mark. Limits and prices are counted in tokens.

input and output tokens

Input tokens are what you send to the model; output tokens are what it writes back. They are counted and priced separately.

How it works

1. Input cost: inputTokens * inputRate. The rate is in dollars per million tokens, so for now this number is a million times too large.
2. Output cost: outputTokens * outputRate, in the same way.
3. Add the two, then divide by one million to get dollars.

Worked example

Input
inputTokens = 1000
outputTokens = 500
inputRate = 2
outputRate = 8

Output
0.006

The reply is half as long as the request and still costs twice as much, because output tokens are priced at four times the input rate here.

Why it matters

The fifth milestone asks for cost per request beside speed and quality. The prices in this lesson are made up; real ones change, and cached input or a model on your own machine is billed differently.

Your turn

Compute the cost in dollars of one request to a hosted model.

inputTokens

tokens sent to the model; a whole number of 0 or more

outputTokens

tokens the model wrote; a whole number of 0 or more

inputRate

dollars per million input tokens

outputRate

dollars per million output tokens

Returns

The total cost in dollars, not rounded.

Input
inputTokens = 1000
outputTokens = 500
inputRate = 2
outputRate = 8

Output
0.006

Input
inputTokens = 1000000
outputTokens = 0
inputRate = 3
outputRate = 10

Output
3

Input
inputTokens = 0
outputTokens = 1000
inputRate = 1
outputRate = 5

Output
0.005

The checks also try

• an empty request
• rates of zero
• a long conversation
• rates that are fractions of a dollar
• a very small request

Hints

Each hint gives away a little more. Open one only when you need it.

1. Hint 1: Where to start

   Work out the two parts separately, each with its own rate, and add them.

2. Hint 2: The key step

   A rate is for a million tokens. So tokens times rate is a million times too large, and one division at the end fixes both parts.

3. Hint 3: The outline

   Multiply inputTokens by inputRate. Multiply outputTokens by outputRate. Add the two products, divide the sum by 1_000_000, which is one million, and return the result without rounding.

Solution

From here on the page gives the method away. If you are still solving, go back to the hints.

Step by step

1. Each part at its own rate. Input and output have different prices, so the token counts cannot be added first and multiplied once.
2. Rates are per million tokens. Dividing once at the end converts both parts to dollars. A missing or extra zero here is the usual mistake.

The same numbered steps are the comments in the solution.

function solve(inputTokens, outputTokens, inputRate, outputRate) {
  // 1. Each part at its own rate
  const input = inputTokens * inputRate;
  const output = outputTokens * outputRate;
  // 2. Rates are per million tokens
  return (input + output) / 1_000_000;
}

def solve(inputTokens, outputTokens, inputRate, outputRate):
    # 1. Each part at its own rate
    input_cost = inputTokens * inputRate
    output_cost = outputTokens * outputRate
    # 2. Rates are per million tokens
    return (input_cost + output_cost) / 1_000_000

Watch out

• Do not add the token counts and use one rate. Output tokens usually cost several times more than input tokens.
• Do not round. Most requests cost less than a cent, and rounding to two decimals turns 0.002 dollars into 0 and 0.006 into 0.01.
• This is the plain token bill. Cached input has its own rates, and hosting, or the hardware and electricity for a local model, is a separate cost.

Check yourself