ThisCalculator

Everyday

Why Token Rates Don’t Predict Your AI API Bill

Updated · Standard formula, shown below

The calculator for this page is switched off while it is checked. The formula is below, and the worked example in the article shows how to use it by hand.

Token rates—the price per 1,000 tokens quoted by AI API providers—are the most visible cost metric, but they rarely match what you actually pay. Your bill depends on a mix of input/output pricing, caching, batching, retries, and hidden overhead like system prompts and tool definitions. This article explains why the sticker price is only the starting point and how to estimate your real costs.

Formula

The basic cost calculation is:

Monthly Cost = (Input Tokens × Input Price per Token) + (Output Tokens × Output Price per Token)

Where:

  • Input Tokens = tokens in your prompt (including system messages, examples, and tool definitions) per request × requests per day × days per month
  • Output Tokens = tokens generated by the model per request × requests per day × days per month
  • Input Price per Token = provider’s price per 1,000,000 input tokens divided by 1,000,000
  • Output Price per Token = provider’s price per 1,000,000 output tokens divided by 1,000,000

Most providers quote prices per million tokens, so you divide by 1,000,000 to get the per-token rate. The formula above uses that directly.

Worked Example

Using the default values from the calculator:

  • Input tokens per request: 1,000
  • Output tokens per request: 500
  • Requests per day: 1,000
  • Days per month: 30
  • Input price: $3.00 per 1M tokens
  • Output price: $15.00 per 1M tokens

Step 1: Calculate total input tokens per month:

1,000 tokens × 1,000 requests × 30 days = 30,000,000 tokens

Step 2: Calculate total output tokens per month:

500 tokens × 1,000 requests × 30 days = 15,000,000 tokens

Step 3: Calculate input cost:

30,000,000 × ($3.00 / 1,000,000) = $90.00

Step 4: Calculate output cost:

15,000,000 × ($15.00 / 1,000,000) = $225.00

Step 5: Add them:

$90.00 + $225.00 = $315.00

The calculator returns $315.00, matching this manual calculation.

Why Token Counts Vary More Than You Think

Tokenization is not uniform. A token is roughly four characters in English, but that varies by language, code, and even formatting. For example, the word “token” is one token, but “tokenization” might be two. Punctuation and spaces often attach to adjacent words, so a small change in formatting can add dozens of tokens per request.

More importantly, the same prompt can produce different token counts across models because each model uses its own tokenizer. A prompt that is 100 tokens with one model might be 120 with another. This means the price per token alone doesn’t tell you the cost per request—you also need to know how the model counts tokens.

Also, most providers count tokens differently for input and output. Input includes the full conversation history if you’re using a chat model, so a long back-and-forth can balloon your input tokens far beyond the latest user message. Output tokens are generated one at a time, and the model can stop early or run to the max, making the final count unpredictable.

Hidden Costs: System Prompts, Tools, and Caching

Your prompt is rarely just the user’s message. System prompts, few-shot examples, and tool/function definitions are all counted as input tokens. A complex tool schema can add 500–2,000 tokens per request, even if the user types only ten words. If you call the API thousands of times, those extra tokens dominate your bill.

Providers also offer prompt caching, where repeated prefixes are billed at a lower rate. But caching only helps if you structure your prompts with a stable prefix (e.g., system message + tools) and a variable suffix (the user message). If you change the system message frequently, the cache misses and you pay full price.

Batching and retries add another layer. Some APIs let you send multiple requests in a batch at a discount, but that requires waiting for all responses. Retries—automatic or manual—duplicate the input cost for the failed attempt, and you still pay for the output that was generated before the error.

How to Actually Estimate Your Bill

To get a realistic estimate, you need to measure your actual usage, not just the token rate. Here’s a practical approach:

  1. Log every request: capture the exact token counts from the API response (most providers include usage data).
  2. Separate input and output tokens, and note whether caching was applied.
  3. Add overhead: system prompts, tool definitions, and conversation history.
  4. Multiply by the number of requests per day and the price per million tokens.
  5. Add any extra fees: data transfer, storage, or per-request charges (if not token-based).

Use the calculator above as a quick starting point, but replace the default token counts with your real numbers. If you don’t have logs yet, run a small pilot with a few hundred requests and measure the average tokens per request.

Common Mistakes That Inflate Your Bill

  • Ignoring system prompts: A 500-token system prompt adds 500 tokens to every single request. With 10,000 requests/day, that’s 5M extra input tokens.
  • Not counting conversation history: Chat completions often send the entire message history, so the input grows with each turn. A 10-turn conversation can be 10× the size of a single message.
  • Assuming output is capped: If you set max_tokens high, the model may generate more than you expect. Conversely, if you set it too low, you might need retries.
  • Forgetting retries: A 5% retry rate adds 5% to your input costs, and possibly to output if the failed request still generated tokens.
  • Mixing up price units: Some providers quote per 1K tokens, others per 1M. A $0.003/1K rate is $3.00/1M—easy to misread.

Why the Sticker Price Can Be Misleading

Providers often advertise a “low” input price, but output is typically 3–5× more expensive. If your application generates long responses (e.g., summaries, code), output tokens can be 80% of your bill even if the input is larger in volume. Also, some models have different pricing for different context lengths—e.g., a higher price for inputs over 128K tokens. That means a single long prompt can cost more than the per-token rate suggests.

Additionally, some providers charge for “speculative decoding” or “thinking” tokens in reasoning models—tokens the model generates internally before answering. These are billed as output tokens but hidden from the final response. Always check the documentation for such extras.

How to Choose a Provider Based on Cost

When comparing providers, don’t just look at the price per million tokens. Consider:

  • Tokenization efficiency: a model that uses fewer tokens for the same text effectively costs less.
  • Cache pricing and hit rate: a provider with aggressive cache discounts can be cheaper for high-volume, stable-prefix workloads.
  • Output speed and max tokens: faster models may allow lower max_tokens, reducing wasted output.
  • Free tier or credits: many providers offer initial credits that can cover testing.
  • Hidden fees: data transfer out, per-request charges, or minimum monthly commitments.

Run a benchmark with your own prompts on each provider, measure the actual token counts, and compute the cost per 1,000 requests. That’s the only reliable comparison.

Frequently Asked Questions

Why is my bill higher than the token rate suggests?

Because the token rate is only the price per token. Your bill also depends on how many tokens you actually use, which includes system prompts, conversation history, tool definitions, retries, and possibly caching misses. If you don’t track those, the rate alone will understate your costs.

Do all tokens cost the same?

No. Input tokens are usually cheaper than output tokens, and some providers charge different rates for cached vs. uncached input. Also, some models have tiered pricing based on context length—e.g., a higher rate for prompts over a certain token count.

How do I reduce my AI API bill?

Optimize your prompts: trim system messages, use shorter tool descriptions, and limit conversation history. Use prompt caching where available, and set reasonable max_tokens to avoid over-generation. Also, batch requests if your provider offers a discount, and monitor retry rates.

What is a token exactly?

A token is a chunk of text that the model processes. It’s often a word or part of a word, but it can be a punctuation mark or a space. For English, one token is roughly four characters, but that varies by language and model. You can check token counts using the provider’s tokenizer tool.

Is it cheaper to use a smaller model?

Usually yes, but not always. Smaller models have lower per-token prices, but they might need more retries or longer prompts to achieve the same quality. Also, some small models have the same tokenization as larger ones, so the token count is identical—only the price per token differs.

Can I predict my bill exactly?

No, because token generation is probabilistic. The model can stop early or run to the max, and retries are unpredictable. But you can get a close estimate by logging actual usage over a few days and extrapolating. The calculator above gives a baseline, but real-world variance can be 10–20%.

Related

Formula used
in t | Input tokens per request | tokens | 1000 | 0
in o | Output tokens per request | tokens | 500 | 0
in r | Requests per day | - | 1000 | 0
in d | Days per month | - | 30 | 0
in pi | Price per 1M input tokens | USD | 3.00 | 0
in po | Price per 1M output tokens | USD | 15.00 | 0
out cost | Estimated monthly cost | USD | (t * pi + o * po) / 1e6 * r * d | 2
out total_in | Total input tokens | tokens | t * r * d | 0
out total_out | Total output tokens | tokens | o * r * d | 0

Each line is one input (in, sel, list) or one result (out: label, unit, formula, decimals). How the calculators work

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.