VereskLearn

Module 9 · Running it for real

Lesson 9.3

Tracking cost

The cost of a request is its token counts times the price per token, summed over all requests.

Lesson 56 of 622 min read

The problem

Why it matters

Without it

The bill arrives at the end of the month. Nobody knows which feature or request caused the spending.

With it

Each request records its tokens, so the cost per request and per day is known as the system runs.

How it works

  • Prices are per million tokens

    The sample uses 2 for input tokens and 8 for output tokens, per million tokens. These prices are invented for the example. Real prices come from the current price list that a model provider publishes.

    Token typePrice per million
    input2
    output8
    The example prices used in the sample, per million tokens.

  • The cost of one request

    Each token count is divided by one million and then multiplied by its price. The 1,200 input tokens cost 0.0024, and the 150 output tokens cost 0.0012. Together they cost 0.0036.

    The two parts of the cost of request 1, in dollars.
    const price = { input: 2, output: 8 };const usage = { input: 1200, output: 150 };const cost = (usage.input / 1_000_000) * price.input  + (usage.output / 1_000_000) * price.output;console.log(cost.toFixed(6));

    Output

    0.003600

  • The day total adds the requests

    The day total adds up the cost of every request. The three example requests cost 0.0036, 0.00232, and 0.0112. Their sum is 0.01712, which prints as 0.017120 with six decimal places.

    Cost of each request, in dollars.

  • Rounding changes only the display

    Floating-point math can leave tiny errors, because a computer stores decimals in binary. The toFixed call rounds the value only for printing. The stored number keeps its full precision.

    const total = 0.1 + 0.2;console.log(total);console.log(total.toFixed(6));

    Output

    0.30000000000000004
0.300000

Full code

Read the program

See the full code

Common mistakes

What goes wrong

  • Using a price from memory. Prices change, so read the current price list each time you set them.
  • Ignoring output tokens. Long answers can cost more than the question that caused them.
  • Looking only at one day. Compare weeks as well, because a slow rise is easy to miss.

Use it when and skip it when

Is it the right tool?

Use it when

  • Spending must be tracked per feature or per user.
  • You plan a budget for a new feature.

Skip it when

  • The system runs on a fixed monthly plan with no charge per token.

Words to know

Terms in this lesson

Input token
A token sent to the model in the request.
Output token
A token that the model writes back.
Price per million
The cost of one million tokens, the unit that prices are often given in.
Budget
The amount you allow for spending in one period.
Full code