Prompt caching: the 90% discount most teams leave on the table

Updated August 21, 2026

Every request you send an LLM API re-sends everything the model needs: the system prompt, your few-shot examples, your tool definitions, maybe a whole document. Most of that is identical on every call — and every major provider now bills that repeated prefix at roughly a tenth of the normal input price once it's cached.

What the discount looks like per vendor

Mechanics differ — OpenAI and Google cache automatically on repeated prefixes, Anthropic uses explicit cache_control breakpoints with a short TTL, DeepSeek caches transparently on disk — but the billing shape is the same: prefix repeated → prefix nearly free.

Worked example: a support chatbot

Say each request carries a 3,000-token system prompt + tools block, 500 tokens of fresh conversation, and produces 300 output tokens, at 30,000 requests/month on Claude Sonnet 5 ($2 in / $10 out, $0.20 cached).

Same model, same quality, 54% off — from one architectural decision. The bigger and more stable your prefix, the closer you get to the full 90%.

Patterns that maximize hit rate

Put the stable stuff first. Caches match prefixes, so order your prompt: system → tools → few-shot examples → documents → the user's fresh input last. One dynamic token early in the prompt (a timestamp, a request ID) invalidates everything after it.

Freeze your prefix bytes. Even whitespace changes break the match. Template it once; never string-interpolate into the cached region.

Batch conversations, not just jobs. Multi-turn chats naturally re-send the whole history — caching turns each turn's history into cheap tokens, which is why long conversations benefit most.

Watch the TTL. Caches expire in minutes on some vendors. Steady traffic keeps them warm; a trickle of requests spaced hours apart may never hit.

Caching vs batch — and stacking them

The Batch API is the other big lever: ~50% off input and output for async workloads. Caching cuts repeated input; batch cuts everything but requires waiting. Bulk jobs with a shared prompt prefix can sometimes stack both — vendor rules differ, so check docs before modeling it into your unit economics.

Prices last verified August 21, 2026. Caching mechanics summarized from vendor docs; behavior may change — consult official documentation for implementation details.