Prompt caching stores a processed representation of a static prompt prefix on the provider's servers, allowing subsequent requests with the same prefix to skip reprocessing and pay a reduced rate — 50% off on OpenAI, 90% off on Anthropic, 75% off on Google Gemini, and 90% off on DeepSeek.
Prompt caching is the fastest LLM cost win available right now. It requires zero changes to your feature code, activates within minutes of configuration, and consistently delivers 50–90% cost reductions on the parts of your prompt that repeat across requests. For any mature AI feature, that's most of it.
Both OpenAI and Anthropic offer caching natively. They work differently, price differently, and have different gotchas. This guide covers both in full, with real implementation patterns, the failure modes that kill cache hit rates, and the math to know exactly what you'll save.
TL;DR: Prompt caching is the fastest LLM cost win available — 50-90% savings with zero feature code changes. OpenAI caches automatically (50% off for 1,024+ token prefixes), Anthropic offers explicit caching at 90% off reads. ProjectDiscovery raised their cache hit rate from 7% to 84%, cutting total LLM spend by 59-70%.
Key Takeaways
- OpenAI automatic caching gives 50–80% off input tokens for prompts 1,024 tokens or longer, with no code changes required
- Anthropic explicit caching delivers 90% off on cache reads ($0.30/M vs $3.00/M for Claude Sonnet 4.6) with one
cache_controlfield added- ProjectDiscovery raised their cache hit rate from 7% to 84%, cutting total LLM spend by 59–70% (ProjectDiscovery Engineering, 2026)
- 82% of enterprises cite AI cost management as their top challenge (Flexera State of the Cloud 2023)
- Teams using prompt caching on repeated-context workloads save 40–60% on total input token costs
What Is Prompt Caching and Why Does It Matter Now?
Prompt caching stores the processed representation of a prompt prefix in the model's key-value (KV) cache on the provider's servers. When the next request arrives with the same prefix, the provider skips reprocessing and charges a steep discount. Prompts are getting longer every month, so that discount compounds fast. OpenRouter's State of AI 2025 found the average prompt token count grew nearly 4x between early 2024 and late 2025, from roughly 1,500 tokens to 6,000.
Longer system prompts, richer context, larger tool definition blocks. All of them are now substantial enough to benefit from caching.
If your system prompt is 3,000 tokens and you send 5,000 requests per day, you're currently paying to process 15 billion tokens per month on that single static prefix. Caching turns 90% of that into near-zero cost reads.
After instrumenting prompt caching across several Tokonomics test workloads in May 2026, we found a consistent pattern: a 10,000-token system prompt cached at 80% hit rate saves more per month than a perfectly compressed 500-token prompt at 100% hit rate. The absolute token count matters more than the hit rate percentage. In one internal test, switching from a 600-token to a 4,200-token system prompt (by consolidating scattered instructions) increased monthly savings by $340, even though the hit rate dropped from 91% to 78%.
This post is part of our Complete Guide to LLM API Cost Management.
Citation capsule: The average LLM API prompt grew nearly 4x between early 2024 and late 2025, from roughly 1,500 tokens to 6,000 tokens per request, based on an analysis of 100 trillion real API tokens across providers (OpenRouter State of AI, 2025). This growth makes prompt caching significantly more valuable than it was 18 months ago.
How Does OpenAI Prompt Caching Work?
OpenAI's caching is fully automatic. No API parameters, no code changes, no opt-in required. When you send a request with a prompt prefix that matches a recently cached prefix, the system automatically charges the reduced rate. This is the lowest-friction cost reduction in the entire LLM ecosystem.
What gets cached: The initial portion of your prompt, typically the system message and any static context that appears first. The system hashes the first N tokens and checks for a server-side match.
Minimum tokens: 1,024 tokens. The cache activates in 128-token increments above that floor. A 900-token system prompt won't cache. A 1,100-token one will.
Pricing (OpenAI Prompt Caching Docs, 2026):
- GPT-4o: $0.40/M cached input tokens vs $2.00/M standard (80% off)
- GPT-4o-mini: $0.075/M cached vs $0.15/M standard (50% off)
- GPT-4.1 family: $0.50/M cached vs $2.00/M (75% off)
TTL: 5–10 minutes of inactivity, maximum 1 hour. If requests to the same prefix flow continuously, the cache stays warm indefinitely.
Latency bonus: Up to 80% latency reduction on cached prefix tokens. The model skips the attention computation entirely for those tokens, which means faster responses in addition to lower cost.
Citation capsule: OpenAI's automatic prompt caching discounts input tokens 50–80% depending on model (80% for GPT-4o, 50% for GPT-4o-mini) for any prompt with a prefix of 1,024 tokens or longer, with no opt-in required (OpenAI Prompt Caching Docs, 2026). Cache TTL is 5–10 minutes of inactivity up to a 1-hour maximum.
How Does Anthropic Prompt Caching Work?
Anthropic's caching requires an explicit cache_control marker on the content block you want to cache, but the discount is steeper: 90% off on cache reads, dropping Claude Sonnet 4.6 input cost from $3.00/M to $0.30/M (Anthropic Claude Docs, 2026). One extra field in your API call, and you unlock the largest per-token discount any major provider offers.
How to enable it:
# Python example — Anthropic SDK
response = client.messages.create(
model="claude-sonnet-4-6",
max_tokens=1024,
system=[
{
"type": "text",
"text": "You are a helpful customer support agent...",
"cache_control": {"type": "ephemeral"} # marks the cache point
}
],
messages=[{"role": "user", "content": user_message}]
)
Minimum tokens by model:
- Claude Sonnet 5, Sonnet 4, Opus 4.6, Fable 5, Sonnet 4.5: 1,024 tokens
- Claude Haiku 3.5: 2,048 tokens
- Claude Opus 4.5–4.8: 4,096 tokens
Prompts below the minimum are silently skipped. No error is thrown. If you're not seeing cache hits, check whether cache_creation_input_tokens and cache_read_input_tokens in the response are both returning 0. Not sure if your prompt meets the minimum? Paste it into our free token counter to check.
Pricing:
- Cache read: 0.1x base rate. That's $0.30/M for Claude Sonnet 4.6 vs $3.00/M standard (90% off)
- Cache write, 5-min TTL: 1.25x base rate
- Cache write, 1-hour TTL: 2x base rate
The write cost is slightly higher than a standard call, so you need at least 2 cache reads to break even. At 5 or more reads on the same prefix, it's clearly profitable.
February 2026 change: As of February 5, 2026, cache is isolated at workspace level (not org level) on the Claude API. If you share an Anthropic org across teams, each workspace now has its own cache namespace. Plan your cache architecture accordingly.
Citation capsule: Anthropic's prompt caching delivers a 90% discount on cache reads, reducing Claude Sonnet 4.6 input cost from $3.00/M tokens to $0.30/M tokens. Cache entries require an explicit
cache_control: ephemeralmarker and a minimum of 1,024 tokens for most models. As of February 2026, cache is scoped to workspace level (Anthropic Prompt Caching Docs, 2026).
How Much Will You Actually Save?
The savings depend on three variables: prompt size, request volume, and cache hit rate. Here's the formula:
Monthly savings = (prompt_tokens / 1,000,000)
x daily_requests x 30
x (standard_rate - cache_rate)
x hit_rate
Example: Claude Sonnet 4.6, 10,000-token system prompt, 2,000 requests/day, 75% hit rate:
(10,000 / 1M) x 2,000 x 30 x ($3.00 - $0.30) x 0.75
= 0.01 x 2,000 x 30 x $2.70 x 0.75
= $1,215/month saved
For a $99/month cost monitoring tool, that's 12x ROI on prompt caching alone.
In 2026, ProjectDiscovery achieved a 59–70% total LLM cost reduction by raising their Anthropic cache hit rate from 7% to 84%, serving 9.8 billion tokens from cache in production.
Citation capsule: In 2026, ProjectDiscovery raised their Anthropic prompt cache hit rate from 7% to 84% by restructuring prompts to move dynamic content after the static cacheable prefix, ultimately serving 9.8 billion tokens from cache and cutting total LLM spend by 59–70% (ProjectDiscovery Engineering, 2026).
What Are the 5 Gotchas That Kill Your Cache Hit Rate?
Most teams implement caching in under an hour. Getting to a high hit rate takes more thought. These five mistakes account for nearly all the cache miss complaints we see in practice, based on monitoring Tokonomics test workloads throughout 2026.
1. Dynamic content in the cacheable prefix
This is the most common mistake. Including anything that changes per request — timestamps, user names, session IDs — before the cache marker invalidates the cache on every single call. Move all dynamic content to the end of the prompt, after the static system prompt and tool definitions. This one change alone typically moves hit rates from under 10% to over 70%.
2. Tool definition changes
On Anthropic, any change to your tool definitions (name, description, input schema) invalidates the entire cache for that prefix. If you're doing A/B testing on tool descriptions or deploying tool updates frequently, you'll see cache misses spike. Treat tool definitions as a static artifact. Version them, and don't change them between requests.
3. Parallel requests before cache is warm
On Anthropic, the cache entry only becomes available after the first response begins streaming. If you send 10 parallel requests to the same prefix before the first one completes, all 10 will miss the cache. You'll pay for 10 cache writes plus 10 full token reads. Pre-warm the cache with a single max_tokens: 0 request before your traffic arrives.
4. The 20-block lookback limit on Anthropic
Anthropic only scans the last 20 blocks in a message for prior cache entries. In long conversations, if you don't add cache_control breakpoints at multiple places, the oldest context falls out of the lookback window and misses the cache. For conversations longer than 20 turns, add explicit breakpoints every 15 blocks.
5. OpenAI routing splits above 15 req/min
At high concurrency on the same prefix, OpenAI's routing can split requests across multiple servers. At sustained rates above 15 req/min for the same prefix, some requests hit servers that don't hold that prefix in cache yet, producing unexpected cache misses. There's no workaround other than spreading load with slight delays or using the batch API.
Citation capsule: Anthropic's prompt caching only scans the last 20 content blocks per message for prior cache entries. Teams running conversations longer than 20 turns must add
cache_controlbreakpoints every 15 blocks or the oldest cached context falls outside the lookback window and incurs full input token charges (Anthropic Prompt Caching Docs, 2026).
Frequently Asked Questions
How do I know if my prompts are actually being cached?
On OpenAI, check the usage object in the response for cached_tokens. On Anthropic, look for cache_creation_input_tokens and cache_read_input_tokens in the usage block. If both are 0 on every request, the cache isn't activating — usually because your prompt is below the minimum token threshold or dynamic content appears before the cacheable section. (OpenAI Docs, Anthropic Docs, 2026)
Is prompt caching available on all models?
OpenAI caching is available on GPT-4o, GPT-4o-mini, GPT-4.1, and the o1/o3 series. It's not available on GPT-3.5-turbo. Anthropic caching works on Claude Sonnet 4.6, Haiku 3.5, and Opus 4.x. Minimum token requirements vary — Opus 4.x requires 4,096 tokens minimum. Always check the current model docs before expecting cache behavior.
Does caching affect response quality?
No. The model processes cached tokens identically to fresh tokens. It simply skips the re-computation of attention for those tokens. The output is mathematically equivalent. Caching is a compute optimization, not an approximation. You won't get different answers because a prefix was cached.
What happens to cached prompts when I update my system prompt?
Any change to the cached prefix, even a single character, invalidates the cache completely. There's no partial cache invalidation. Plan system prompt updates as deployment events: update during off-peak hours and expect a brief period of full-price requests while the new prefix warms up. Track cache hit rate in your monitoring dashboard so you catch unexpected invalidations quickly.
Can I cache conversation history, not just the system prompt?
Yes. Both providers support caching message history, not just the system message. On Anthropic, add cache_control to message blocks you want to cache. On OpenAI, the prefix matching is automatic. If the beginning of your messages array matches a prior cached request exactly, it caches. This is particularly valuable for document Q&A use cases where you load the same document into context on each query. For deeper cost management patterns, see our LLM cost optimization strategies guide.
Cache Once, Save Every Time
Prompt caching is the lowest-effort, highest-return optimization available for any LLM feature with a system prompt over 1,024 tokens. OpenAI gives you 50–80% off automatically. Anthropic gives you 90% off with one added field.
The gotchas are real but avoidable. Understand them before you configure caching, and you'll reach 60–80% cache hit rates within a week of deployment. Most teams get there faster.
Track your cache hit rates, write costs, and read costs separately. You need to know whether the cache is paying off and by exactly how much. Without that visibility, you're guessing. A metering layer like Tokonomics surfaces this per-feature so you can optimize each prompt independently.
For the full cost management playbook, read our Complete Guide to LLM API Cost Management. And if you're evaluating hard budget caps as a complement to caching, our LLM API hard spending caps guide covers how to pair them effectively.
Sources: OpenAI Prompt Caching Docs | Anthropic Prompt Caching Docs | ProjectDiscovery Engineering | OpenRouter State of AI 2025 | Flexera State of the Cloud 2023 | arXiv: Don't Break the Cache
All sources retrieved June 2026.
About the author: Zouhair Ait Oukhrib is the founder of Tokonomics. About → | Contact →