VereskLearn

Module 1 · Talking to a model

Lesson 1.2

Tokens and cost

A token is a small piece of text that a model counts, and providers charge by the number of tokens.

Lesson 4 of 622 min read

The problem

Why it matters

Without it

A team guesses what each request costs. The bill arrives at the end of the month with no clear cause.

With it

The program counts the tokens in each request and reply. It estimates the cost before a change goes live.

How it works

  • A token is a piece of text

    A token is a piece of text that the model reads as one unit. It is often a short word or part of a long word. This lesson uses a rough rule: one token per 4 characters, rounded up, for each word. The word "refund" has 6 characters, so it counts as 2 tokens. Real tokenizers learn their pieces from data, so their counts differ.

    • Refunds2 tokens
    • take1 token
    • 51 token
    • to1 token
    • 71 token
    • business2 tokens
    • days.2 tokens
    The reply, one word per chip, with its token count. The total is 10 tokens.
    // One token per 4 characters of each word, rounded up.function countTokens(text: string): number {  const words = text.split(" ").filter((w) => w.length > 0);  return words.reduce((n, w) => n + Math.ceil(w.length / 4), 0);}console.log("refund ->", countTokens("refund"));const reply = "Refunds take 5 to 7 business days.";console.log("reply ->", countTokens(reply));

    Output

    refund -> 2
reply -> 10

  • Input and output are counted apart

    Input tokens cover everything sent to the model: the instructions, the help article, and the question. Output tokens cover the reply. Many providers charge more for each output token. In this example, the input is 1,208 tokens and the reply is 10.

    Tokens for one request. The long input and the short reply differ by a factor of over 100.
    // A stand-in article of 1,200 words, plus the question.const article = "text ".repeat(1200);const question = "How long does a refund take?";function countTokens(text: string): number {  const words = text.split(" ").filter((w) => w.length > 0);  return words.reduce((n, w) => n + Math.ceil(w.length / 4), 0);}const inputTokens = countTokens(article) + countTokens(question);console.log("Input tokens ->", inputTokens);

    Output

    Input tokens -> 1208

  • Prices are set per million tokens

    Providers publish prices per million tokens. This example uses 2 dollars per million input tokens and 8 dollars per million output tokens. A request costs its input tokens times the input price, plus its output tokens times the output price. The total is divided by one million.

    PartTokensPrice per millionCost
    Input1,208$2$0.002416
    Output10$8$0.00008
    Total1,218$0.002496
    The cost of one request. The total is 0.002496 dollars.
    // Made-up prices, in dollars per million tokens.const INPUT_PRICE = 2;const OUTPUT_PRICE = 8;function requestCost(input: number, output: number): number {  return (input * INPUT_PRICE + output * OUTPUT_PRICE) / 1000000;}console.log("One request ->", requestCost(1208, 10).toFixed(6));

    Output

    One request -> 0.002496

  • Many requests add up

    A daily bill is the cost of one request times the number of requests. At 10,000 requests a day, this example costs about 25 dollars. Sending the article with every question is the largest cost. Without it, the same 10,000 requests cost about 1 dollar.

    Cost of 10,000 requests a day, in dollars.
    // The same prices: 2 and 8 dollars per million tokens.function requestCost(input: number, output: number): number {  return (input * 2 + output * 8) / 1000000;}const perDay = (input: number) => requestCost(input, 10) * 10000;console.log("With article ->", perDay(1208).toFixed(2));console.log("Question only ->", perDay(8).toFixed(2));

    Output

    With article -> 24.96
Question only -> 0.96

Full code

Read the program

See the full code

Common mistakes

What goes wrong

  • Counting characters instead of tokens. The two differ, so a character count can be far off.
  • Ignoring the reply. Long replies add cost, so set a maximum reply length.
  • Sending a whole manual with every question. The cost grows on every call, even when the answer is short.

Use it when and skip it when

Is it the right tool?

Use it when

  • You need to estimate the monthly cost of a feature before you launch it.
  • You want to find out why one request costs more than another.

Skip it when

  • You run only a few test requests by hand. The bill is small, so a rough check is enough.
  • Your provider shows a usage report that you already trust.

Words to know

Terms in this lesson

Token
A piece of text that a model reads and counts.
Input tokens
Tokens sent to the model, including the instructions and the question.
Output tokens
Tokens the model writes back as its reply.
Price per million
The charge for one million tokens, the usual unit on a price list.
Full code