Claude Prompt Caching in Node.js: Cut Repeat Costs

Claude prompt caching in Node.js lets you reuse a cached prefix of your prompt across multiple API calls, which can cut repeat call costs by up to 90% for the cached portion. You implement it by adding a cache_control block to a prompt block in the Messages API, and the cache automatically expires after 5 minutes of inactivity. This article explains how caching works, how to use it in Node.js, and how to decide if it’s right for your application.

Formula

The calculator estimates your hourly cost savings when using prompt caching. The formula is:

Hourly savings = (Prompt tokens / 1,000,000) × Calls per hour × (Base input cost − (Cache read cost + Cache write cost))

Where:

• Prompt tokens – the number of tokens in the cached prefix of your prompt (the part that stays the same across calls).
• Calls per hour – how many API requests you make in an hour.
• Base input cost – the standard price per million input tokens (e.g., $3.00 for Claude 3.5 Sonnet).
• Cache read cost – the price per million tokens read from the cache (e.g., $0.30 for Sonnet).
• Cache write cost – the price per million tokens written to the cache (e.g., $1.25 for Sonnet).

This assumes every call uses the same cached prefix and that the cache is warm (i.e., you call within the 5-minute TTL).

Worked Example

Using the default values: 1,000 prompt tokens, 60 calls per hour, cache read $0.30/M, cache write $1.25/M, base input $3.00/M.

Without caching, hourly cost = (1,000 / 1,000,000) × 60 × $3.00 = $0.18.

With caching, each call costs (1,000 / 1,000,000) × ($0.30 + $1.25) = $0.00155 per call, so hourly = $0.093.

Hourly savings = $0.18 − $0.093 = $0.087. The calculator shows $0.09 when rounded to two decimals.

What Is Prompt Caching?

Prompt caching is a feature of the Anthropic API that stores a portion of your prompt (a prefix) on Anthropic’s servers for a short time. When you send a new request with the same prefix, the API reuses the cached computation instead of reprocessing those tokens from scratch. This reduces latency and cost because cached tokens are cheaper to process than fresh ones.

For Node.js developers, this is particularly useful for applications that send the same system prompt, few-shot examples, or large context documents with every request—like a chatbot with a long instruction set or a code assistant with a repository overview.

How to Enable Prompt Caching in Node.js

To use prompt caching, you add a cache_control parameter to the last content block of the prefix you want cached. Here’s a minimal example using the official Anthropic SDK:

const Anthropic = require('@anthropic-ai/sdk');
const client = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

async function chat() {
  const response = await client.messages.create({
    model: 'claude-3-5-sonnet-20241022',
    max_tokens: 1024,
    messages: [
      {
        role: 'user',
        content: [
          {
            type: 'text',
            text: 'You are a helpful assistant.\n\n' + longContext,
            cache_control: { type: 'ephemeral' }
          }
        ]
      },
      { role: 'user', content: 'What is the capital of France?' }
    ]
  });
  console.log(response.content[0].text);
}

Key points:

• The cache_control block must be on a content block, not on the message object itself.
• Only the prefix up to and including that block is cached.
• The cache is ephemeral—it expires after 5 minutes of no use.
• You can have multiple cache breakpoints, but each adds cost (write cost per breakpoint).

Cost Structure and When It Pays Off

Anthropic charges a separate rate for cache writes and cache reads. For Claude 3.5 Sonnet, the typical rates are:

• Base input: $3.00 per million tokens
• Cache write: $1.25 per million tokens (25% more than base)
• Cache read: $0.30 per million tokens (90% cheaper than base)

Caching becomes cost-effective when you make multiple calls with the same prefix within the 5-minute window. The break-even point is roughly 2–3 calls per cache TTL. If you only call once, caching is more expensive because you pay the write cost without any read benefit.

For example, if you have a 10,000-token system prompt and you call the API 100 times per hour, your savings are substantial: without caching you pay 10,000 × 100 = 1M tokens at $3 = $3.00. With caching, you pay 1M tokens at $0.30 = $0.30, plus the initial write of 10,000 tokens at $1.25 = $0.0125, so total ~$0.31—a 90% reduction.

Common Pitfalls and Misconceptions

One common mistake is thinking that the entire prompt is cached automatically. In reality, you must explicitly mark the cache breakpoint. If you don’t, no caching occurs.

Another pitfall is using caching for dynamic content. If the prefix changes between calls (e.g., you include a timestamp or user-specific data), the cache is invalidated and you pay the write cost every time, which can be more expensive than not caching.

Also, the cache is shared across your API key, not per user. So if two users trigger the same prefix, the cache is reused.

Finally, the 5-minute TTL is a sliding window—each call resets the timer. If your application has bursts of activity with gaps longer than 5 minutes, the cache may expire and you’ll pay write costs again.

How to Choose Cache Breakpoints

Choosing where to place cache_control is a trade-off between cache size and flexibility. Here are some guidelines:

• Cache the largest static prefix you have—usually the system prompt and any few-shot examples.
• Keep dynamic parts (like user input) outside the cached prefix.
• If you have multiple static sections, you can place breakpoints at each, but each breakpoint adds a write cost. Use only one or two breakpoints unless you have a strong reason.
• Monitor your cache hit rate via the API response’s usage object, which shows cache_read_input_tokens and cache_creation_input_tokens.

Alternatives and Related Features

If prompt caching isn’t suitable, consider other cost-reduction strategies:

• Model selection – Use a cheaper/faster model like Claude Haiku for simple tasks.
• Prompt compression – Reduce the number of tokens by summarizing or trimming context.
• Batching – Use the Messages API’s batch endpoint to process multiple requests together at a 50% discount (though results are delayed).
• Fine-tuning – For very repetitive tasks, a fine-tuned model may need fewer prompt tokens.

Each has its own trade-offs in latency, cost, and complexity.

Frequently Asked Questions

Does prompt caching work with streaming responses?

Yes, caching works with streaming. You still set the cache_control block, and the cache is used during the stream. The response includes usage information, but you may need to parse the stream events to see cache tokens.

Can I cache a tool definition or a system message?

Yes. The cache_control can be added to any content block, including system messages and tool definitions. In the SDK, you set it on the last block of the system parameter or on a tool’s input schema.

How do I know if my cache is being hit?

Check the usage field in the API response. It contains cache_read_input_tokens (tokens read from cache) and cache_creation_input_tokens (tokens written to cache). If cache_read_input_tokens is greater than zero, you got a hit.

What happens if my cached prefix is longer than the model’s context limit?

The cache prefix still counts toward the context window. So you cannot cache more tokens than the model’s maximum context (e.g., 200k for Claude 3.5 Sonnet). The cache is just a way to avoid reprocessing, not to exceed the limit.

Is prompt caching available for all Claude models?

As of late 2024, caching is available for Claude 3.5 Sonnet and Claude 3.5 Haiku, with plans for others. Check the Anthropic documentation for the latest list.

Can I use caching with the AWS Bedrock or Google Vertex AI versions of Claude?

Yes, prompt caching is supported on those platforms as well, but the implementation may differ slightly (e.g., using cachePoint in Bedrock). Refer to the respective provider’s docs.

Related

• Wi-Fi Access Point: What It Is and How It Works
• LoL Worlds Scores & Team Status: Where to Find Them
• Sandpaper Grit Numbers Explained

in t | Prompt tokens | tokens | 1000 | 0
in c | Calls per hour | calls | 60 | 0
in r | Cache read cost per 1M tokens | USD | 0.30 | 0
in w | Cache write cost per 1M tokens | USD | 1.25 | 0
in b | Base input cost per 1M tokens | USD | 3.00 | 0
out saving | Estimated hourly savings | USD | (t/1000000)*c*(b - (r + w)) | 2
out cached_cost | Hourly cost with caching | USD | (t/1000000)*c*(r + w) | 2
out no_cache_cost | Hourly cost without caching | USD | (t/1000000)*c*b | 2