AI API Pricing Explained: Cache, Batch, Long Context & Effective Cost

Summary

AI API pricing looks simple when a model page shows two numbers: input dollars per million tokens and output dollars per million tokens. Production bills are rarely that simple.

The real cost of a workload can change because of prompt caching, cache writes, Batch or Flex processing, long-context thresholds, cache storage, tool calls, regional processing, retries and human review. Two models with similar headline prices can therefore have very different unit economics once the full request path is included.

This guide explains the pricing mechanics using current first-party documentation checked on September 21, 2026. The goal is not to rank providers. It is to give you a reusable way to calculate effective cost per accepted task.

Quick Take

• Start with input and output token cost, but do not stop there.
• Repeated prompt prefixes can become dramatically cheaper when cache hits are reliable.
• Batch processing commonly trades latency for lower token rates.
• Some providers change the rate when prompts cross a long-context threshold.
• Context caching can include a separate storage charge.
• Server-side tools can add per-call fees on top of model tokens.
• The best business metric is cost per accepted result, not dollars per million tokens.

The effective-cost formula

A practical cost model is:

Effective task cost = uncached input + cache writes + cache reads + output + processing-tier modifiers + long-context modifiers + tools + storage + regional premiums + retries

For a production system, add one more layer:

Effective accepted-result cost = total task cost ÷ acceptance rate

That last step matters. A cheaper request is not cheaper if it needs more retries, corrections or human review.

1. Base input and output tokens

The familiar headline rate is still the starting point.

If a model costs $2 per million input tokens and $12 per million output tokens, then a request with 50,000 input tokens and 2,000 output tokens costs:

• input: 0.05 × $2 = $0.10
• output: 0.002 × $12 = $0.024
• base token cost = $0.124

But that calculation assumes every input token is uncached, the request stays inside the standard context tier, no tool is called and the default processing tier is used.

Those assumptions often stop being true in real applications.

2. Prompt caching: write once, read many times

Caching matters when requests repeatedly contain the same large prefix: a system prompt, product catalog, policy manual, codebase context or long conversation history.

Providers do not all price caching in exactly the same way.

OpenAI

For GPT-5.6 Terra, OpenAI currently lists:

• standard input: $2 / MTok
• cached input: $0.20 / MTok
• cache write: 1.25× the uncached input rate
• output: $12 / MTok

That means a Terra cache write is effectively $2.50 per million cached input tokens, while a later cache read is $0.20 per million.

Anthropic

Anthropic documents explicit cache multipliers:

• 5-minute cache write: 1.25× base input
• 1-hour cache write: 2× base input
• normal cache hit/read: 0.1× base input
• Fable 5.1 and Mythos 5.1 cache reads: 0.025× base input

The write is more expensive than ordinary input, but reuse can recover that premium quickly.

Mistral

Mistral documents cached prompt tokens at 10% of the standard input token price. Its pricing interface describes cached input as a 90% input-cost reduction for repeated prompts.

Google Gemini

Gemini pricing is more explicit about cache storage. The pricing tables include both cached-token processing rates and a separate per-token-per-hour storage charge for supported models. That means the economics depend on how much context is stored and for how long, not only how often it is read.

Example: a 100K-token prefix reused 10 times

Assume:

• 100,000 stable input tokens are reused across 10 requests
• each request produces 5,000 output tokens
• no variable suffix is included in this simplified example

GPT-5.6 Terra without caching

Across 10 calls:

• total input = 1,000,000 tokens → $2.00
• total output = 50,000 tokens → $0.60
• total = $2.60

GPT-5.6 Terra with one cache write and nine cache reads

• first 100K cache write at $2.50 / MTok → $0.25
• nine 100K cache reads = 900K at $0.20 / MTok → $0.18
• output remains $0.60
• total = $1.03

In this simplified scenario, caching reduces the token bill by about 60%.

For Claude Sonnet 5 at its current $2/$10 base rates, the same 5-minute-cache pattern produces a similar result:

• no cache: $2.50
• one 1.25× write + nine 0.1× reads + outputs: $0.93
• reduction: about 63%

These are illustrative calculations. Real savings depend on how much of the prompt actually hits cache, cache lifetime, tokenization and the amount of changing context.

3. Batch and Flex: latency has a price

If a task does not need an immediate answer, asynchronous or lower-priority processing can materially reduce cost.

OpenAI

OpenAI's Batch API currently provides a 50% discount versus synchronous APIs. Current frontier-model documentation also states that Batch and Flex are priced at 50% of Standard rates for supported models. Flex trades lower cost for slower responses and occasional resource unavailability.

Anthropic

Anthropic's Batch API provides a 50% discount on both input and output tokens.

Mistral

Mistral describes Batch processing as half price for high-volume work.

Gemini

Google publishes separate Standard, Batch, Flex and Priority tables. The exact numbers are model-specific, so do not assume one universal multiplier across every Gemini model and date. Current Gemini 3.x examples show Batch/Flex rates below Standard and separate cache-storage pricing.

The architecture lesson is straightforward:

Do not pay interactive latency prices for work that can finish later.

Evaluation runs, nightly classification, document enrichment, embeddings pipelines and back-office summarization are typical candidates for asynchronous tiers.

4. Long context can change the entire request price

A large context window does not guarantee one flat price across the full window.

OpenAI's current 1.05M-context GPT-6 / GPT-5.6 family has a material threshold: prompts above 272K input tokens are priced at 2× input and 1.5× output for the full request.

For GPT-5.6 Terra that changes the standard rate from:

• $2 input / $12 output

to:

• $4 input / $18 output

for a request over the threshold.

Anthropic documents a different policy for Claude 4.6-and-later models and Mythos Preview: the full 1M context window remains at standard per-token pricing, with cache and Batch discounts still applicable.

This is why a single "$/MTok" column cannot fully describe long-document economics.

Example: 300K input + 5K output

Consider a request with:

• 300,000 input tokens
• 5,000 output tokens

GPT-5.6 Terra — Standard

Because 300K is above the 272K threshold, use $4 input and $18 output per million:

• input: 0.3 × $4 = $1.20
• output: 0.005 × $18 = $0.09
• total = $1.29

At the current 50%-of-Standard Batch/Flex rate for supported processing:

• effective token cost ≈ $0.645

Claude Sonnet 5 — Standard

Using current $2 input / $10 output pricing with standard pricing across the full 1M window:

• input: 0.3 × $2 = $0.60
• output: 0.005 × $10 = $0.05
• total = $0.65

With Anthropic Batch:

• effective token cost ≈ $0.325

This is not a model-quality comparison. It demonstrates how pricing rules change the cost of the exact same token shape.

5. Cache storage can matter

Some caching systems price not only the tokens read from cache, but also how long cached context is stored.

Google's current Gemini pricing tables include context-cache storage per million tokens per hour for supported models. A long-lived 500K-token cache therefore has a time component even before the next request reads it.

When forecasting cache economics, use:

cache cost = creation/write + cache reads + storage duration

Caching is most attractive when:

• the reusable prefix is large,
• the prefix is reused many times,
• reuse happens within the cache lifetime,
• changing suffixes are relatively small.

Caching is less attractive when the prompt changes substantially every call or a large cache sits idle.

6. Tools can become a separate meter

Token prices do not always include the full cost of tool use.

OpenAI model documentation notes that tool-specific services such as search and computer use can have separate fees. Anthropic similarly states that server-side tools can add usage-based pricing, while ordinary client-side tool schemas also consume model input/output tokens.

A tool-heavy agent therefore has at least two cost layers:

1. model tokens required to decide and process the tool call,
2. any separate provider fee for the server-side tool itself.

External APIs, browsers, databases and SaaS services can add a third layer outside the AI provider bill.

This is one reason agent costs should be measured per completed workflow, not per model turn.

7. Region and data-residency choices can add a premium

Infrastructure requirements can modify the same workload again.

Mistral's current pricing interface shows a 10% regional inference uplift for supported regional processing. OpenAI's pricing documentation also applies a 10% uplift to eligible regional-processing endpoints for newer supported models. Anthropic documents geography-related pricing modifiers for some deployment paths.

These premiums may be completely rational when data-residency or contractual requirements demand them. They still belong in the unit-economics model.

8. Retries are the hidden multiplier

If one request costs $0.20 but the workflow succeeds only 70% of the time, the effective model cost per successful result is already higher than $0.20.

A simplified estimate is:

cost per accepted result = average request cost × average attempts per accepted result

Then add:

• human review minutes,
• downstream API calls,
• failed tool executions,
• duplicated retrieval,
• generated artifacts,
• storage and observability.

This is why a more expensive model can sometimes be cheaper in production — and why a cheaper model can remain cheaper when its success rate is good enough. You need matched workload data to know.

A practical pricing checklist

Before estimating monthly spend, record these fields for every candidate model:

If even one high-volume row is missing, the monthly forecast can be materially wrong.

How AI World Scope calculates workloads

Use the AI World Scope Pricing Calculator for model-level scenario modeling, then treat it as the token layer of your budget.

For broader routing decisions, combine it with How to Choose an AI Model for Your API Workload in 2026 and Cheapest AI APIs in 2026: Price vs Capability.

The next useful step for any production team is to log actual usage by request class:

• input tokens,
• cached tokens,
• output tokens,
• tool calls,
• processing tier,
• latency,
• success/failure,
• review outcome.

That turns a pricing page into real unit economics.

AI World Scope take

The "$X input / $Y output" label is useful for discovery, but it is not enough for budgeting.

A robust AI cost model should answer five questions:

1. How much context is actually reused?
2. Which traffic can move to Batch or Flex?
3. Does long context trigger a higher rate?
4. Which extra meters — tools, storage, region — apply?
5. How much does one accepted business result cost after retries?

That is the number worth optimizing.

Headline price finds candidates. Effective cost chooses the architecture.