If you're sending Claude the same 30,000-token knowledge base on every single request, you're paying for it every single time. A support assistant that reads your returns policy before answering, a code agent that re-sends the same project context on every turn, a RAG pipeline that prepends the same instructions to every query: they all re-process an identical prefix over and over, and the bill scales with that waste.
Prompt caching with the Claude API fixes this. You mark the stable part of your prompt with one field, and Claude caches the processed prefix. The next request that shares that prefix reads it back at roughly one-tenth the price instead of paying full freight. In this article we'll build a real example (a customer support assistant grounded in a policy document), measure the savings against the live API, and walk through the silent bug that makes caching look like it's working when it isn't.
What prompt caching actually does
Prompt caching is a prefix match. The Claude API hashes the exact bytes of your prompt up to a marked breakpoint. The first request with that prefix processes it normally and writes the result to a cache. Any later request that starts with the identical prefix reads the processed result instead of recomputing it.
That's the whole model, and one rule follows from it: any byte change anywhere in the prefix invalidates everything after it. Get the ordering right and caching mostly works for free. Get it wrong and no amount of configuration helps.
The economics are the reason to bother. A cache read costs about 0.1× the normal input price. A cache write costs 1.25× for the default 5-minute cache (or 2× for the 1-hour option). So with the default cache, you break even after two requests and save heavily after that. That's exactly the shape of a chatbot, an agent loop, or any pipeline with a fixed preamble.
Need this built into your product properly? Get in touch. We ship production Claude integrations for a living.
The setup: a support assistant that re-reads everything
Here's the naive version. We have a returns policy of about 26,000 tokens of structured rules, and we answer customer questions against it. Every question re-sends the whole document:
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
// loadReturnsPolicy() is a placeholder for your own loader (a file, a
// database, or a CMS). It returns the ~26K-token policy text.
const policyDocument = await loadReturnsPolicy();
async function ask(question: string): Promise<Anthropic.Message> {
return client.messages.create({
model: "claude-opus-4-8",
max_tokens: 256,
system: policyDocument,
messages: [{ role: "user", content: question }],
});
}
Functionally fine. But the policyDocument is identical on every call, and we pay full input price for all 26,000 tokens every time a customer asks a question. Over thousands of questions, that's the dominant cost in the request.
Turning on caching with cache_control
To cache the policy, we move the system prompt from a plain string to an array of content blocks and add a single cache_control marker to the stable block:
async function ask(question: string): Promise<Anthropic.Message> {
return client.messages.create({
model: "claude-opus-4-8",
max_tokens: 256,
system: [
{
type: "text",
text: policyDocument,
cache_control: { type: "ephemeral" },
},
],
messages: [{ role: "user", content: question }],
});
}
{ type: "ephemeral" } is the cache directive. The default time-to-live is 5 minutes; pass { type: "ephemeral", ttl: "1h" } for a 1-hour cache when your traffic is burstier. The marker can go on system text blocks, tool definitions, or message content blocks, and you get up to four breakpoints per request.
If you don't need fine-grained placement, the simplest option is top-level auto-caching: pass cache_control: { type: "ephemeral" } directly on messages.create() and the SDK caches the last cacheable block for you. We use the explicit block form throughout this article because it makes the breakpoint visible, which is what we want while you're learning where it goes.
The placement matters because of the prefix rule. The API renders your prompt in a fixed order (tools, then system, then messages), so a breakpoint on the last system block caches your tools and system prompt together. The question itself lives in messages, after the breakpoint, so it varies freely without disturbing the cache.
These snippets stay focused on caching and leave out error handling, retries, and instrumentation; the In production section at the end covers what to add before you ship.
Proving it works: read the usage fields
This is the part people skip, and it's the part that matters. Every response carries a usage object with three fields that tell you exactly what caching did:
cache_creation_input_tokens: tokens written to the cache this request (you paid the ~1.25× write premium)cache_read_input_tokens: tokens served from the cache this request (you paid ~0.1×)input_tokens: tokens processed at full price (the uncached remainder)
Here's a runnable harness that asks two questions in a row and logs those fields so you can watch the cache work:
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const policyDocument = await loadReturnsPolicy(); // large, stable prefix
// max_tokens is small here only because we care about the usage numbers,
// not the answers. A real assistant would set a generous limit.
async function askAndLogUsage(question: string): Promise<void> {
const response = await client.messages.create({
model: "claude-opus-4-8",
max_tokens: 256,
system: [
{ type: "text", text: policyDocument, cache_control: { type: "ephemeral" } },
],
messages: [{ role: "user", content: question }],
});
const { input_tokens, cache_creation_input_tokens, cache_read_input_tokens } =
response.usage;
console.log({ input_tokens, cache_creation_input_tokens, cache_read_input_tokens });
}
await askAndLogUsage("How long is the return window for category 1?");
await askAndLogUsage("Do I need the original packaging?");
Run against the live API with a ~26K-token policy, this prints:
{ input_tokens: 15, cache_creation_input_tokens: 26302, cache_read_input_tokens: 0 }
{ input_tokens: 14, cache_creation_input_tokens: 0, cache_read_input_tokens: 26302 }
Read those two lines carefully, because they're the entire point:
- First call wrote 26,302 tokens to the cache (
cache_creation_input_tokens).input_tokensis 15, just the question. - Second call read all 26,302 tokens back from the cache (
cache_read_input_tokens) and wrote nothing. The big document cost ~0.1× this time instead of full price.
One subtlety worth internalizing: input_tokens is the uncached remainder only. Your total prompt size is input_tokens + cache_creation_input_tokens + cache_read_input_tokens. If you ever see a tiny input_tokens and panic that the model isn't seeing your context, check the sum. The rest was served from cache.
The minimum prefix, and why short prompts silently won't cache
Caching only kicks in above a model-specific minimum prefix length. Below it, the marker is ignored, with no error: just cache_creation_input_tokens: 0 and no savings. The current thresholds:
- Opus 4.8, Opus 4.7, Opus 4.6, Opus 4.5, Haiku 4.5: 4096 tokens
- Claude Fable 5, Sonnet 4.6: 2048 tokens
- Sonnet 4.5: 1024 tokens
So a 3,000-token prompt caches on Sonnet 4.6 but silently won't on Opus 4.8. This is the first thing to check if you've added cache_control and see no reads: your prefix may simply be too short to qualify.
The silent invalidator that breaks caching for real
Here's the bug that bites everyone. Suppose you helpfully stamp the current time into your system prompt:
system: [
{
type: "text",
// The trap: a timestamp at the FRONT of the prefix
text: `Current time: ${new Date().toISOString()}\n\n${policyDocument}`,
cache_control: { type: "ephemeral" },
},
],
Looks harmless. It is catastrophic for caching. The timestamp changes on every request, so every prefix is unique, so nothing is ever read back. Running the two-question test with this change against the live API:
{ cache_creation_input_tokens: 26324, cache_read_input_tokens: 0 }
{ cache_creation_input_tokens: 26324, cache_read_input_tokens: 0 }
Both requests write the cache and neither reads it. You're now paying the 1.25× write premium on every single call, which is strictly worse than not caching at all. And nothing errors; the only signal is cache_read_input_tokens sitting at zero across requests that should share a prefix.
The same trap hides in several common places. Grep your prompt-building code for these before shipping:
new Date()/Date.now()anywhere in the prefixcrypto.randomUUID()or request IDs near the front of the contentJSON.stringify(obj)on an object whose key order isn't stable (re-serialize with sorted keys)- A user ID or session ID interpolated into the system prompt (this also kills cross-user sharing)
- Conditional system sections (
if (flag) system += ...), where every combination is a distinct prefix
The fix is always the same: keep the stable content first, and move anything volatile after the last breakpoint, into a later message rather than the system prompt. A timestamp in a user turn at the end of the conversation invalidates nothing before it.
Caching in a multi-turn agent
The payoff compounds in conversations and agent loops, where the prefix grows but its front stays fixed. Put a breakpoint on the last content block of the most recent turn, and each request reuses the entire prior conversation:
// Return a new messages array with a breakpoint on the latest turn,
// without mutating the conversation we were handed.
function withCacheBreakpoint(
conversation: Anthropic.MessageParam[],
): Anthropic.MessageParam[] {
const lastIndex = conversation.length - 1;
return conversation.map((message, index) => {
if (index !== lastIndex || !Array.isArray(message.content)) {
return message;
}
const lastBlockIndex = message.content.length - 1;
const content = message.content.map((block, blockIndex) =>
blockIndex === lastBlockIndex
? { ...block, cache_control: { type: "ephemeral" as const } }
: block,
);
return { ...message, content };
});
}
const response = await client.messages.create({
model: "claude-opus-4-8",
max_tokens: 1024,
system: [{ type: "text", text: systemPrompt, cache_control: { type: "ephemeral" } }],
messages: withCacheBreakpoint(conversation),
});
A few notes on that snippet. conversation and systemPrompt are your own values: the prior turns and your stable system string. withCacheBreakpoint returns a new array and never mutates conversation, which matters because that conversation is shared state your loop reuses across turns. The as const keeps "ephemeral" a literal type so the object matches the SDK's cache_control union instead of widening to string.
The harder problem in an agent is keeping the front of the prefix stable as the loop runs, and "don't touch it" isn't the only answer:
- Changing the tool set invalidates everything, because tools render at the very front of the prefix. Rather than swapping tools for different "modes", use the tool search tool, which appends tool schemas instead of replacing them and preserves the cached prefix.
- Switching models invalidates the cache too, since caches are scoped per model. If a sub-task wants a cheaper model, run it in a subagent and keep the main loop on one model.
- Injecting volatile context (a mode flag, fresh state) into the system prompt invalidates the whole conversation. Append it as a mid-conversation
{ role: "system", ... }message instead. It sits after the cached history and leaves the prefix intact.
If you're building agent loops, the same caching discipline applies to the patterns in our guide on building a production AI agent with the Claude Agent SDK, and it pairs naturally with the validation approach in structured outputs with Claude, where the schema is part of the stable prefix you want cached.
When not to cache
Caching isn't free, so don't reach for it reflexively. Skip it when:
- The prefix changes every request. If the first 1,000 tokens differ each time, there's no reusable prefix and you'd only pay the write premium for nothing.
- The prefix is below the cacheable minimum. A 1,500-token prompt won't cache on Opus 4.8 at all, and the cold-processing cost is already negligible.
- You're firing many requests with distinct prefixes speculatively. Each is a 1.25× write; the cost can exceed the latency you hoped to save.
The sweet spot is a large, stable prefix reused across many requests within the cache window, which describes almost every chatbot, support assistant, and agent in production.
In production
The snippets above are deliberately focused on caching, so they skip the error handling, retries, and instrumentation a real deployment needs. Here's what to add before you ship a cached prompt to production.
What you must do:
- Wrap API calls in error handling with retries. A cache miss is cheap; an unhandled
429in the request path is an outage. Use the SDK's typed exceptions (Anthropic.RateLimitError,Anthropic.APIError) and let the SDK's built-in retry/backoff do its job. Don't string-match error messages. - Instrument the cache, don't assume it. Log
cache_read_input_tokensandcache_creation_input_tokensper request and chart the read/write ratio. A caching regression is silent by nature: nothing errors, the bill just creeps up. A dashboard catches a newly introduced invalidator the same day it ships. - Pin the model. Caches are scoped per model, so a model change silently cold-starts every cache. Treat the model ID as configuration you deploy deliberately, not a default you drift on.
What to avoid:
- Per-user data in the cached prefix. A user ID or session detail baked into the system prompt gives every user a unique prefix that's written once and never read, so you pay the write premium for zero benefit. Keep the prefix identical across users; put per-user context after the last breakpoint.
- More than four breakpoints, or breakpoints on growing turns. You get four
cache_controlmarkers per request. In a long agent loop, move the breakpoint forward to the latest turn rather than accumulating markers on every turn. - Letting a single turn add more than 20 content blocks. Each breakpoint only looks back about 20 blocks to find a prior cache entry. A tool-heavy agent turn that adds dozens of tool-call and tool-result blocks can push the previous entry out of range, so the next request silently misses. Place an intermediate breakpoint every ~15 blocks in long turns.
- Caching below the minimum. If your prefix is under the model's threshold (4096 tokens on Opus 4.8), the marker is a no-op. Verify with
cache_read_input_tokensbefore assuming you're saving anything.
Cost and failure behavior under real traffic:
- Concurrent requests don't share a write. A cache entry is only readable after the first request begins streaming. Fire 50 identical requests at once on a cold cache and all 50 pay the write price. For fan-out, send one request, await its first token, then release the rest. They'll read the cache it just wrote.
- The cache is best-effort, not guaranteed. Entries can expire early under load. Your code must stay correct (just more expensive) on a miss, so never assume a read will hit.
- TTL math drives the cost. The 5-minute cache breaks even at two requests; the 1-hour cache costs 2× to write and needs at least three requests total (one write plus two reads) to pay off. Match the TTL to your real traffic gaps, not to a guess.
Wrapping up
Prompt caching is one cache_control field, but the leverage is real: a large shared prefix drops from full price to ~0.1× on every repeat request, and the only discipline required is keeping volatile data out of the prefix and watching cache_read_input_tokens to confirm it's actually hitting. Add the marker, run two identical-prefix requests, and check that the second one reads what the first one wrote. If it reads zero, you have a silent invalidator, and now you know exactly where to look.
If you're building a Claude-powered product and want the caching, cost, and reliability work done right the first time, we can help.
