Next.js · AI costs ·

Prompt caching not working? Check it in OpenAI and Anthropic (Next.js)

You turned on prompt caching, or you were told it's on by default, and the usage numbers say nothing was cached. cached_tokens is 0 on OpenAI, or cache_read_input_tokens is 0 on Anthropic, and the bill hasn't moved. If you searched for prompt caching not working, this post covers how to check whether a request actually hit the cache, the usual reasons it didn't, and how to price cached reads and writes correctly once it does.

This guide pairs with Spend Caps Kit, and if you want the next step, read OpenAI and Anthropic spend limits per user in Next.js.

Everything here is for a Next.js App Router route using the Vercel AI SDK (ai 7.0.143, @ai-sdk/anthropic 4.0.82, @ai-sdk/openai 4.0.96). Limits and prices come from the OpenAI, Anthropic and AI SDK docs, checked on 17 October 2026.

What we did and didn't test. We haven't shown a live cache hit. We didn't call the OpenAI or Anthropic APIs with a real key, so every hit and usage number below comes from the providers' docs. Our code type-checks with tsc --strict against the package versions above and has 16 unit tests. Some of those tests run the AI SDK against a fake fetch, which lets us check the request the SDK would send and how it reads a usage block copied from the docs. That's all it proves. We aren't affiliated with or endorsed by OpenAI, Anthropic or Vercel; their product and model names are their trademarks, used here only to identify them.

The short checklist

If your cache reads are 0, check these in order:

  1. Is the cached part of the prompt at or above the model's minimum length?
  2. Is anything that changes between requests (a date, a user name, a request id, tool order) placed before the cache point?
  3. Was the next request sent before the cache expired?
  4. On Anthropic, is there a cache breakpoint, and is it on a block that stays identical between requests?
  5. In the AI SDK, did the cache option reach the provider, and are you reading the right usage field?
  6. Are you looking at cost, which can go up on the first request, instead of cache reads?

The rest of this post goes through each one.

How to tell if a request hit the cache

Both providers report cache use in the response's usage object, but they count input tokens differently. That difference is a common reason dashboards and home-made cost maths disagree.

Anthropic. The prompt caching docs list three fields:

  • cache_read_input_tokens: tokens read from the cache on this request.
  • cache_creation_input_tokens: tokens written to the cache on this request.
  • input_tokens: only the tokens after the last cache breakpoint, which can't be cached.

So the total input is the sum of all three. The docs' own example is 100,000 tokens read from the cache, 0 written and 50 in the user message, for a total of 100,050 input tokens. The docs also say that "if both cache_creation_input_tokens and cache_read_input_tokens are 0, the prompt was not cached". If you use the 1-hour cache, a cache_creation object splits writes into ephemeral_5m_input_tokens and ephemeral_1h_input_tokens.

OpenAI. The prompt caching guide tells you to track cached_tokens and cache_write_tokens, which sit inside input_tokens_details. Here input_tokens already includes the cached and written tokens. The guide's own cost example works out ordinary input as input_tokens minus cached_tokens minus cache_write_tokens. On models before GPT-5.6, the guide says reported cached_tokens is rounded down to the nearest multiple of 128.

AI SDK. The SDK maps both providers onto one shape. The Anthropic provider page and the OpenAI provider page both say to read cacheReadTokens and cacheWriteTokens under usage.inputTokenDetails. In the SDK's LanguageModelUsage type, inputTokens is the total input and noCacheTokens is the uncached part. Our tests confirm that for both providers: the Anthropic docs example comes out as 100,050 total and 50 uncached.

Here's a small reader that turns any of the three shapes into one report. "unknown" means the provider didn't send the fields at all, which isn't the same as a miss.

// lib/cache-usage.ts (excerpt)
import type { LanguageModelUsage } from 'ai';

export type CacheStatus = 'hit' | 'write' | 'none' | 'unknown';

export type CacheReport = {
  totalInput: number;
  uncached: number;
  cacheRead: number;
  cacheWrite5m: number;
  cacheWrite1h: number;
  status: CacheStatus;
};

function statusOf(read: number | undefined, write: number | undefined): CacheStatus {
  if (read === undefined && write === undefined) return 'unknown';
  if ((read ?? 0) > 0) return 'hit';
  if ((write ?? 0) > 0) return 'write';
  return 'none';
}

// AI SDK usage: inputTokens is the total, cache reads and writes included.
export function fromAiSdk(usage: LanguageModelUsage, writeTtl: '5m' | '1h' = '5m'): CacheReport {
  const read = usage.inputTokenDetails.cacheReadTokens;
  const write = usage.inputTokenDetails.cacheWriteTokens;
  const total = usage.inputTokens ?? 0;
  const uncached = usage.inputTokenDetails.noCacheTokens ?? total - (read ?? 0) - (write ?? 0);
  return {
    totalInput: total,
    uncached,
    cacheRead: read ?? 0,
    cacheWrite5m: writeTtl === '5m' ? write ?? 0 : 0,
    cacheWrite1h: writeTtl === '1h' ? write ?? 0 : 0,
    status: statusOf(read, write),
  };
}

// Raw Anthropic usage: input_tokens is only the part after the last breakpoint.
export type AnthropicUsage = {
  input_tokens: number;
  cache_read_input_tokens?: number | null;
  cache_creation_input_tokens?: number | null;
  cache_creation?: { ephemeral_5m_input_tokens?: number; ephemeral_1h_input_tokens?: number } | null;
};

export function fromAnthropic(u: AnthropicUsage): CacheReport {
  const read = u.cache_read_input_tokens ?? 0;
  const write = u.cache_creation_input_tokens ?? 0;
  const w1h = u.cache_creation?.ephemeral_1h_input_tokens ?? 0;
  const w5m = u.cache_creation ? u.cache_creation.ephemeral_5m_input_tokens ?? 0 : write;
  return {
    totalInput: u.input_tokens + read + write,
    uncached: u.input_tokens,
    cacheRead: read,
    cacheWrite5m: w5m,
    cacheWrite1h: w1h,
    status: statusOf(read, write),
  };
}

The full file, which also has a reader for raw OpenAI Responses usage, follows the same pattern. The AI SDK's normalised usage doesn't say whether an Anthropic write was 5-minute or 1-hour, so fromAiSdk takes the TTL you asked for.

Reason 1: the prompt is below the model's minimum

Neither provider caches short prompts, and Anthropic doesn't return an error when this happens. From Anthropic's cache limitations, on the Claude API the minimum cacheable prompt length is:

  • 512 tokens for Claude Fable 5.1, Mythos 5.1, Opus 5.5, Opus 5, Sonnet 5.5, Fable 5, Mythos 5 and Haiku 5.5.
  • 1,024 tokens for Claude Opus 4.8, Sonnet 5, Sonnet 4.6 and Sonnet 4.5, among others.
  • 2,048 tokens for Claude Mythos Preview and Opus 4.7.
  • 4,096 tokens for Claude Opus 4.6, Opus 4.5 and Haiku 4.5.

The docs add: "Any requests to cache fewer than this number of tokens will be processed without caching, and no error is returned."

OpenAI's guide gives a minimum of 1,024 visible input tokens for GPT-5.6 and later. For earlier models it "varies with request settings, including tools, images, output schemas, reasoning effort, and verbosity". Tokens in OpenAI's hidden system content don't count toward the minimum.

One thing to watch: the minimum-length list on the AI SDK's Anthropic page is older than Anthropic's own and doesn't list the newer models. Go by Anthropic's page.

Reason 2: something changes before the cached part

Caching only works on a prefix that is exactly the same from one request to the next. Anthropic's docs say "Cache hits require 100% identical prompt segments", and the cache covers tools, then system, then messages, in that order. OpenAI's guide says "Cache reuse requires the entire rendered prefix to match."

The usual causes:

  • A date or user detail in the system prompt. "Today is 17 October" or "The user's name is Sam" at the top changes the prefix on every request. Both providers say to put stable content first and changing content last.
  • Tool changes. On Anthropic, changing tool definitions (names, descriptions or parameters) invalidates the whole cache. Changing tool_choice, or adding or removing images, invalidates the messages part. On OpenAI, changing tool names, descriptions, schemas or order changes the prefix.
  • Changing the active tools in the AI SDK. The AI SDK's OpenAI page says activeTools removes tools from the request "and invalidates the cache whenever the allow-list changes". It suggests the allowedTools provider option instead, which keeps the full tool list in the request (Responses API only).
  • Other settings. Anthropic lists speed mode, thinking settings and effort among the things that invalidate parts of the cache. OpenAI lists reasoning.effort, text.verbosity and structured output format.
  • Rewriting history. OpenAI's guide says summarising, compaction or truncating earlier turns can reset cache reuse. Append new messages instead of editing old ones.

A cheap way to catch this is to log a hash of the part you expect to be cached next to each usage report. If the hash changes between two requests, a miss is expected.

// lib/prefix.ts
import { createHash } from 'node:crypto';

export function prefixFingerprint(...parts: string[]): string {
  return createHash('sha256').update(parts.join('\u0000')).digest('hex').slice(0, 12);
}

Reason 3: the cache expired

Anthropic: the default lifetime is 5 minutes, and it's "refreshed for no additional cost each time the cached content is used". The lifetime counts from the start of the request, so a response that takes 4 minutes to stream leaves about 1 minute for the follow-up. You can ask for a 1-hour cache by adding ttl: "1h" to the cache setting, at a higher write price (see the pricing section).

OpenAI: for GPT-5.6 and later, prompt_cache_options.ttl supports one value, 30m, which is also the default. The guide says a cached prefix stays reusable "for 30 minutes after its most recent write or reuse, though OpenAI may retain it longer." For earlier models, prompt_cache_retention takes in_memory (entries typically stay for 5 to 10 minutes of inactivity, up to one hour) or 24h (up to 24 hours, on the models the guide lists). GPT-5.5 and GPT-5.5 Pro support 24h only. Organisations without Zero Data Retention default to 24h, and those with it default to in_memory.

Two more timing traps from the docs:

  • Parallel requests. Anthropic says "a cache entry only becomes available after the first response begins". If you fire 10 identical requests at once, they won't share a cache. Wait for the first response to start.
  • High traffic on OpenAI. Cached state lives on individual machines, and "traffic above 15 requests per minute can lead to overflow routing" to machines without your cache. On models before GPT-5.6, the guide recommends a stable prompt_cache_key for requests that share a prefix.

Reason 4 (Anthropic): no breakpoint, or the wrong one

On Anthropic you opt in. You either add one cache_control field at the top level of the request (automatic caching, which moves the cache point forward as a conversation grows) or put cache_control on specific blocks. The docs allow up to 4 block-level breakpoints.

Writes happen only at a breakpoint. If the breakpoint sits on a block that changes every time, such as the user's latest message with a timestamp in it, the cached entry never matches. The docs' advice is to place the breakpoint "on the last block that stays identical across requests". Automatic caching has the same trap: it puts the breakpoint on the last cacheable block, so if that block changes every request, the docs suggest an explicit breakpoint at the end of the stable part instead.

In long chats, there's also a 20-block lookback: the system checks at most 20 positions back from each breakpoint for an earlier cache entry. The docs say a breakpoint on the final block works as long as each turn adds fewer than 20 blocks. If one turn adds more, the earlier entry falls outside the window, and the docs suggest a second breakpoint further back.

Anthropic also has a cache diagnostics feature that compares consecutive requests and reports where the prompt diverged. OpenAI has a Prompt Cache Diagnostics tool for the same job.

Reason 5 (AI SDK): the option never reached the provider

In the AI SDK, cache settings travel through providerOptions. For Anthropic, that's cacheControl with a type of ephemeral, set on a message, a message part or a tool. For OpenAI, the provider page lists promptCacheKey, promptCacheOptions (with mode and ttl, for GPT-5.6 and later), promptCacheRetention for earlier models, and promptCacheBreakpoint on a content part.

Three things we found while testing against ai 7.0.143 with a fake fetch (no live API calls):

  • A system message inside messages now throws. In ai 7, a message with the system role in messages fails with "System messages are not allowed in the prompt or messages fields. Use the instructions option instead." unless you set allowSystemInMessages. Older examples, including the system-message example on the AI SDK's Anthropic page, put it in messages. In version 7, put the cached system prompt in instructions, which accepts a system message with providerOptions. Our test confirms the SDK then sends cache_control on the system block.
  • More than 4 breakpoints are dropped, not rejected. With 5 cacheControl parts, the Anthropic provider sent 4 and added a warning to warnings saying the breakpoint limit was exceeded. Log warnings so you see it.
  • Read the standard usage object. Older code read cache counts from provider metadata. The provider pages now point to usage.inputTokenDetails.

Here's the Anthropic route with logging. It assumes a long, stable prompt in SUPPORT_PROMPT.

// app/api/chat/route.ts
import { anthropic } from '@ai-sdk/anthropic';
import { streamText, type ModelMessage } from 'ai';
import { fromAiSdk } from '@/lib/cache-usage';
import { costUsd } from '@/lib/cache-cost';
import { prefixFingerprint } from '@/lib/prefix';

// Stable text only: no dates, user names or request ids in here.
const INSTRUCTIONS = process.env.SUPPORT_PROMPT ?? '';
const MODEL = 'claude-haiku-4-5';

export async function POST(req: Request): Promise<Response> {
  const { messages } = (await req.json()) as { messages: ModelMessage[] };

  const result = streamText({
    model: anthropic(MODEL),
    instructions: {
      role: 'system',
      content: INSTRUCTIONS,
      providerOptions: { anthropic: { cacheControl: { type: 'ephemeral' } } },
    },
    messages,
    onEnd: ({ usage, warnings }) => {
      const cache = fromAiSdk(usage);
      console.log(JSON.stringify({
        model: MODEL,
        prefix: prefixFingerprint(INSTRUCTIONS),
        ...cache,
        usd: costUsd(MODEL, cache, usage.outputTokens ?? 0),
        warnings: warnings?.length ?? 0,
      }));
    },
  });

  return result.toTextStreamResponse();
}

Claude Haiku 4.5 needs at least 4,096 tokens before anything is cached, so a short SUPPORT_PROMPT will log none every time. That's Reason 1, not a bug in the route. In a real app, check the user is signed in before calling the model.

For OpenAI on GPT-5.6 or later, the same idea uses an explicit breakpoint on the instructions (the SDK sends them as a developer message, as OpenAI's guide recommends) and a per-user cache key:

// app/api/chat-openai/route.ts (excerpt)
const result = streamText({
  model: openai('gpt-5.6-terra'),
  providerOptions: {
    openai: {
      // userId from your auth session, never the request body
      promptCacheKey: `support-v1:${userId}`,
      promptCacheOptions: { mode: 'explicit', ttl: '30m' },
    } satisfies OpenAILanguageModelResponsesOptions,
  },
  instructions: {
    role: 'system',
    content: INSTRUCTIONS,
    providerOptions: { openai: { promptCacheBreakpoint: { mode: 'explicit' } } },
  },
  messages,
  onEnd: ({ usage }) => console.log(JSON.stringify(fromAiSdk(usage))),
});

OpenAI's guide says that in explicit mode, "When no explicit breakpoints are placed, the request does not use prompt caching". On GPT-5.6 and later the key isn't needed for routing, but the guide says separate keys keep cache accounting separate per user and help prevent cache-hit probing across users.

Reason 6: you're reading cost, not cache reads

Writing to the cache costs more than plain input on both providers' newer models, so the first request in a series can cost more than it would without caching. Only later reads are cheaper. If you check the bill after a handful of one-off requests, caching can look like it made things worse.

From Anthropic's pricing table:

  • 5-minute cache writes cost 1.25 times the base input price.
  • 1-hour cache writes cost 2 times the base input price.
  • Cache reads cost 0.1 times the base input price, except 0.05 times on Claude Opus 5.5 and Sonnet 5.5, and 0.025 times on Claude Fable 5.1 and Mythos 5.1.

For Claude Haiku 4.5, that's $1 per million base input tokens, $1.25 for 5-minute writes, $2 for 1-hour writes, $0.10 for cache reads and $5 for output.

From OpenAI's guide, for GPT-5.6 and later, writes cost 1.25 times the uncached input rate, and reads cost 0.1 times that rate (0.05 times on GPT-6.1 Sol). Earlier models have no extra cache-write charge. The pricing page lists gpt-5.6-terra at $2.00 input, $0.20 cached input, $2.50 cache writes and $12.00 output per million tokens (short-context, standard rates). The guide's cost formula counts each input token once, at the uncached, cached or cache-write rate, so a write isn't charged on top of ordinary input.

The guide gives its own worked figure: with a 0.1 times read rate, writing a prefix once and reusing it once costs 1.35 times its ordinary input cost, against 2 times for sending it twice without caching. Your figure depends on how often a prefix is reused before it expires.

Pricing cached tokens correctly

If you log cost per request, or cap or bill users by spend, price each kind of input token at its own rate. Here's the cost function we tested, with the prices above. The OpenAI row stores its single write rate in both write fields.

// lib/cache-cost.ts (excerpt: two of the five price rows)
import type { CacheReport } from './cache-usage';

// USD per million tokens, from the providers' pricing pages on 17 Oct 2026.
export type Price = { input: number; cacheRead: number; cacheWrite5m: number; cacheWrite1h: number; output: number };

export const PRICES: Record<string, Price> = {
  'claude-haiku-4-5': { input: 1, cacheRead: 0.1, cacheWrite5m: 1.25, cacheWrite1h: 2, output: 5 },
  'gpt-5.6-terra': { input: 2, cacheRead: 0.2, cacheWrite5m: 2.5, cacheWrite1h: 2.5, output: 12 },
};

export function costUsd(model: string, r: CacheReport, outputTokens: number): number {
  const p = PRICES[model];
  if (!p) throw new Error(`No price for model ${model}`);
  return (
    r.uncached * p.input +
    r.cacheRead * p.cacheRead +
    r.cacheWrite5m * p.cacheWrite5m +
    r.cacheWrite1h * p.cacheWrite1h +
    outputTokens * p.output
  ) / 1_000_000;
}

The tests check it against hand-worked numbers. For example, on Claude Haiku 4.5, 10,000 cache reads, 2,000 5-minute writes, 500 uncached tokens and 300 output tokens come to $0.0055. For OpenAI, the tests check it against the formula in OpenAI's guide. Unknown models throw rather than being priced at $0. That matters because a missing price quietly turns into free usage.

One more pricing detail: OpenAI's FAQ says cached input tokens still count toward tokens-per-minute rate limits. Anthropic's 1-hour cache section says cache hits are not deducted against your rate limit.

Test results

$ npx tsc -p . --strict
exit 0

$ node --import tsx --test test/cache.test.ts
ℹ tests 16
ℹ pass 16
ℹ fail 0

The tests cover the Anthropic and OpenAI docs' usage examples, the hit, write, none and unknown statuses, the cost maths for 5-minute and 1-hour writes, and four offline AI SDK round trips. Those round trips use a fake fetch that records the request and returns a usage block copied from the docs:

  • Anthropic cache_control from instructions.
  • The ai 7 error for system messages inside messages.
  • The dropped fifth breakpoint.
  • OpenAI cache key, options and breakpoint.

None of this proves a cache hit against a live API.

Checklist

  • The cached part is at or above the model's minimum length, checked on the provider's own page.
  • Nothing that changes per request comes before the cache point.
  • Tools, tool order and settings that affect the prefix stay the same between requests.
  • Follow-up requests arrive within the cache lifetime, and parallel requests wait for the first response to start.
  • On Anthropic there's a breakpoint, and it's on a block that stays the same.
  • In AI SDK 7, the cached system prompt is in instructions, and warnings is logged.
  • Cache reads and writes are read from usage.inputTokenDetails and priced at their own rates.

Related

Disclosure: this is our own paid product. Spend Caps Kit ($19 plus VAT where applicable) is copy-paste TypeScript and one Supabase migration for Next.js. It adds per-user rate limits and a monthly dollar cap per user, priced from each response's token usage with cached tokens and cache writes included, and a price table for 16 OpenAI and Anthropic models. The kit doesn't make the cache hit; it counts and caps the cost.

Sources (checked 17 October 2026):

Related posts