Prompt Caching Explained: Cut LLM Costs Without Changing Your Prompts

Prompt caching reuses the processed prefix of a repeated prompt, so later requests cost far less. How it works, what it costs, and the mistakes to avoid.

By Galactic News Editorial · Published 2026-11-10T04:29:00+00:00 · Updated 2026-11-10T04:29:00+00:00

Prompt Caching Explained: Cut LLM Costs Without Changing Your Prompts

Prompt caching cuts LLM costs by reusing work the model already did. When two requests start with the same text, the provider stores the processed prefix from the first request and reuses it on the second. You pay a small premium to write the cache, then a large discount to read from it. The prompts stay the same. The bill drops.

The mechanism is simple. A language model processes a prompt from the start. If the first thousand tokens are identical to a recent request, reprocessing them is wasted work. Prompt caching skips that step. It works best for long system prompts, tool definitions, retrieved documents, and multi-turn conversations where history grows each turn.

Prompt caching explained in plain terms

Think of it as a receipt for work already done. The first request pays full price and gets a receipt. The second request shows the receipt and pays only for the new part.

Every major hosted LLM provider offers some form of it. Anthropic's Claude API, OpenAI's API, Google Gemini, Amazon Bedrock, and DeepSeek all document prompt caching. The details differ, but the shape is the same: mark a stable prefix, pay to store it, then read it back at a discount.

The prefix must match exactly. Change one word at the start and the cache misses. Change a timestamp inside the cached section and the cache misses. This is the most common mistake, and it is covered in detail below.

Prompt caching is not the same as semantic caching. Semantic caching skips the provider call entirely when a similar question was asked before. Prompt caching still makes the call. It just makes the call cheaper.

How it works on the Claude API

Anthropic's documentation describes two ways to enable caching. Automatic caching adds a single cache_control field at the top level of the request. The system applies the breakpoint to the last cacheable block and moves it forward as the conversation grows. This is the simplest way to start, and it fits multi-turn conversations.

Explicit breakpoints give finer control. You place cache_control directly on individual content blocks. This lets you cache sections that change at different rates, such as tool definitions that rarely change and context that updates daily. You can define up to four cache breakpoints per request.

Cache prefixes are built in a fixed order: tools first, then system, then messages. Each level builds on the ones before it. A breakpoint writes exactly one cache entry: a hash of the prefix ending at that block. Because the hash is cumulative, changing any block at or before the breakpoint produces a different hash on the next request.

When a request arrives, the system checks whether the prefix hash at the breakpoint matches a stored entry. If not, it walks backward one block at a time, checking up to 20 positions. This is the lookback window. If nothing matches within 20 blocks, there is no cache hit. A second breakpoint can start a second lookback window, which helps in long conversations.

The default cache lifetime is 5 minutes. Each use of the cached content refreshes it at no extra cost. The lifetime is measured from the start of the request that writes or reads the entry, not from the end of its response. A 1-hour lifetime is available at a higher write price.

Short prompts cannot be cached at all. The minimum cacheable length depends on the model: 512 tokens for many current Claude models, 1,024 tokens for Claude Sonnet 5, Sonnet 4.6, Sonnet 4.5, and Opus 4.8, 2,048 tokens for Opus 4.7, and 4,096 tokens for Opus 4.6 and Opus 4.5. Requests below the minimum are processed without caching, and no error is returned.

What it costs: the write premium and the read discount

Caching has two prices. Writing to the cache costs more than a normal input. Reading from the cache costs far less. The table below uses Anthropic's published per-million-token rates.

The pattern is constant across models. A 5-minute cache write costs 1.25 times the base input price. A 1-hour write costs 2 times. A cache read costs 0.1 times the base input price on most models, 0.05 times on Opus 5.5 and Sonnet 5.5, and 0.025 times on Fable 5.1, Mythos 5.1, Fable 5.2, and Mythos 5.2.

OpenAI follows a similar structure on its newest models. GPT-5.6 and later charge 1.25 times the input rate for a cache write and 0.1 times for a read. Older OpenAI models write to cache for free and read at a per-model discount of up to 90%.

Do the break-even math on the standard rates. A 5-minute write at 1.25x plus one read at 0.1x costs 1.35x total. Two uncached requests cost 2.0x. The second use of a cached prefix already saves money. The break-even point is about 1.28 reuses. In practice, Anthropic reported cost reductions of up to 90% and latency reductions of up to 85% for long repeated prompts at launch.

How to set it up in five steps

1. Find the stable prefix. Identify the part of the prompt that never changes between requests: the system prompt, tool definitions, few-shot examples, or a reference document.
2. Put stable content first. Order the request so the unchanging material comes before anything that varies. The cache prefix is built from the start, so a varying block early in the prompt breaks everything after it.
3. Place the breakpoint at the end of the stable prefix. Add cache_control to the last block that stays identical across requests. Do not put it on a block that contains a timestamp or the user's new message.
4. Start with automatic caching. Add one top-level cache_control field and let the system move the breakpoint as the conversation grows. Switch to explicit breakpoints only when you need separate lifetimes for different sections.
5. Check the token counts. Confirm the cached prefix is above the model's minimum cacheable length. Then watch the usage response: cache_read_input_tokens should be non-zero on the second and later requests.

Common mistakes that kill the cache

The first mistake is caching a changing block. A system prompt with a timestamp, a date, or a per-request ID in it will never hit. The prefix hash changes every time. Move the varying content after the breakpoint.

The second mistake is caching too little. A 300-token system prompt is below the minimum on every model. It will never be cached. Combine stable material into one prefix that clears the threshold.

The third mistake is letting the conversation grow past the lookback window. If the breakpoint moves more than 20 blocks past the last cache write, the system stops looking and misses. Add a second breakpoint at a stable earlier position to start a fresh window.

The fourth mistake is toggling caching on and off mid-conversation. Each change breaks the chain. Set it once and leave it for the whole conversation.

The fifth mistake is paying the write premium for one-off prompts. If a prefix is used once and never again, the 25% write premium is pure cost. Caching only pays when the same prefix repeats.

When prompt caching pays off most

Long multi-turn conversations are the best case. Each turn re-sends the whole history, and the history only grows. Automatic caching turns that growth from a cost problem into a small one.

Agent loops are the second best case. An agent that calls the model dozens of times with the same system prompt and tool list saves on nearly every call after the first.

Retrieval pipelines are the third. When the same documents are attached to many questions, caching the document block cuts the input cost of every question.

Short, one-shot prompts gain nothing. Neither do prompts that change completely each time. Check the cache read rate in your usage data before assuming savings. If reads are near zero, the breakpoints are in the wrong place.

Bottom line

Prompt caching is the cheapest performance win in LLM development because it changes nothing about the prompt. Mark the stable prefix, pay 1.25 times to write it once, then read it back at a tenth of the price. The second reuse already saves money. Start with automatic caching, keep varying content after the breakpoint, and check that cache reads actually appear in your usage. If they do, the savings compound with every request.