ICP Internet Cache Protocol: How Web Caches Find Shared Content

The Internet Cache Protocol (ICP) is a lightweight UDP-based protocol that lets web caches query each other to see if a nearby cache holds a requested object. When your cache receives a miss, it sends an ICP query to neighboring caches; if one replies that it has the content, your cache fetches it from there instead of going all the way to the origin server. This article explains how ICP works, why it was created, and where it still fits in modern content delivery.

Background: Why ICP Was Created

In the mid-1990s, the web was growing fast and slow links made every request painful. The first web caches were standalone proxies that stored copies of popular pages, but a single cache could only hold so much. Administrators wanted to link caches together so that a cache in one school or office could borrow content from a cache in another. The challenge was figuring out which cache had the content without transferring the whole object to check.

ICP was designed in 1994 by Duane Wessels and others for the Harvest project, which later became the Squid cache. It was published as RFC 2186 in 1997. ICP is a simple request/response protocol that runs over UDP, usually on port 3130. The key idea is to ask a small question—”do you have this URL?”—and get a tiny answer, rather than downloading the object to find out.

How ICP Works: Messages and Flow

ICP defines two main message types: ICP_QUERY and ICP_HIT. There is also ICP_MISS, ICP_ERR, and a few others for maintenance. The flow is straightforward:

1. A client requests a URL from its local cache (the “parent” or “sibling”).
2. The local cache does not have the object, so it sends an ICP_QUERY to one or more neighboring caches.
3. Each neighbor checks its own store. If it has the object, it replies with ICP_HIT. If not, it replies with ICP_MISS.
4. The local cache waits a short time (usually 2 seconds) for replies. If any HIT arrives, it fetches the object from that neighbor. If all replies are MISS or no reply comes, it goes to the origin server.

ICP messages are small—the query contains the URL and a request ID, and the reply contains the same ID plus a result code. Because UDP is connectionless, there is no handshake, so the overhead is tiny. However, UDP does not guarantee delivery, so a cache may never receive a reply. That is why caches use a timeout and fall back to the origin server.

ICP also supports a “multicast” mode where a query is sent to a group of caches at once, but in practice most deployments used unicast to a list of known neighbors.

How Caches Choose Neighbors: Siblings vs. Parents

ICP does not itself decide which neighbor to use; that is left to the cache’s configuration. There are two common roles:

• Sibling – a peer cache that is at the same level in a hierarchy. You ask it for content, and if it has it, you fetch it. If it misses, you do not ask it again for that URL.
• Parent – a cache that is closer to the origin servers (or has a bigger store). You send it ICP queries, and if it hits, you fetch from it. If it misses, you may still send the request to the parent because the parent might fetch it from the origin and cache it for you.

In Squid, you configure this with the cache_peer directive, specifying whether the peer is a sibling or a parent. ICP is then used to ask that peer if it has the object. The decision to use a sibling or parent is often based on cost: fetching from a sibling is cheaper than going to the origin, but if the sibling misses, you wasted a round trip. Some caches use a “ICP timeout” and if no HIT arrives, they go direct to the origin.

ICP vs. Cache Digests and Modern Alternatives

ICP was effective for small networks, but it had limitations. The most significant is that it requires a round trip to every neighbor before you can fetch the object. That adds latency, especially if neighbors are far away. Also, ICP messages are not cached—every miss triggers new queries.

Cache digests were developed as an alternative. A cache digest is a compact summary (like a Bloom filter) of the URLs a cache holds. Instead of sending a query, a cache downloads the digest periodically and checks locally whether a neighbor likely has the object. This avoids the round trip for each request. Squid supports cache digests as an option.

Modern content delivery networks (CDNs) use a different approach: they use DNS routing or anycast to send a user to the closest cache, and caches within a CDN talk to each other using proprietary protocols (like Akamai’s or Cloudflare’s) that are more efficient than ICP. The HTTP Cache-Control headers and conditional requests (If-Modified-Since, ETag) also reduce the need for inter-cache queries.

Common Problems and Misconceptions

One common misconception is that ICP is a security protocol or that it encrypts traffic. It does not. ICP is plaintext and has no authentication, which means an attacker could spoof ICP_HIT messages to redirect traffic to a malicious server. That is one reason ICP was never widely deployed on the open internet; it was intended for trusted networks.

Another issue is that ICP can cause loops. If two caches are configured as siblings and both miss, they could keep querying each other. To prevent this, caches use a “hop count” in the ICP message, and they do not forward queries to the same neighbor that sent the query. Still, misconfiguration can cause broadcast storms on large networks.

A practical problem is that ICP only tells you whether a cache has the object, not whether it is fresh. A cache might have a stale copy, but ICP_HIT does not include a timestamp. The requesting cache must still send a conditional request (If-Modified-Since) to the neighbor to validate. This adds another round trip, partially negating the benefit.

Configuring ICP in Squid (Example)

Squid is the most famous cache that supports ICP. To enable ICP, you set the icp_port directive (default 3130) and add cache_peer lines. For example:

icp_port 3130
cache_peer cache1.example.com sibling 3128 3130
cache_peer cache2.example.com parent 3128 3130

This tells Squid to listen for ICP on port 3130 and to treat cache1 as a sibling and cache2 as a parent. When a miss occurs, Squid sends an ICP query to both. If cache1 replies HIT, Squid fetches from cache1. If cache2 replies HIT, it fetches from cache2. If both miss, Squid sends the request to the parent (cache2) anyway, because the parent might fetch and cache it.

You can also set icp_query_timeout (default 2000 ms) and icp_hit_stale to control behavior. Note that ICP is only useful if the neighboring caches are close and fast; otherwise the query latency may exceed the benefit of a cache hit.

Is ICP Still Used Today?

ICP is largely obsolete in public internet infrastructure. Most CDNs and large caching systems use proprietary protocols or HTTP-based forwarding. However, ICP is still present in some legacy enterprise and academic networks, especially where Squid is still running. It is also a good learning example for how distributed caching coordination works.

If you are building a new system, you would likely use HTTP itself for cache-to-cache communication (e.g., using the Cache-Control header and forwarding requests) or a modern protocol like Redis or Memcached for shared object stores. ICP’s simplicity is its charm, but its lack of security and efficiency makes it unsuitable for modern scale.

Frequently Asked Questions

Does ICP work over TCP?

No, ICP is defined to run over UDP. UDP is chosen because it is lightweight and fast, but that also means messages can be lost. Caches handle this by using timeouts and retries (or just falling back to the origin).

What port does ICP use?

The default port is 3130. You can configure a different port in Squid with the icp_port directive. The port must be the same on all peers that want to communicate.

Is ICP secure?

No. ICP has no authentication or encryption. An attacker who can spoof UDP packets could send a fake ICP_HIT and redirect traffic to a malicious server. Therefore, ICP should only be used on trusted networks, and modern systems use secure alternatives.

How is ICP different from HTTP caching?

HTTP caching is between a client and a cache (or between caches) and uses HTTP headers like Cache-Control and ETag. ICP is a separate protocol that caches use to ask each other if they hold a URL. HTTP caching does not require a separate query; the client simply sends a request and the cache responds with a hit or miss.

Can I use ICP with a CDN?

No, CDNs use their own internal protocols. ICP is not used in modern CDNs because it is inefficient and insecure. If you are using a CDN, you do not need to configure ICP; the CDN handles cache routing automatically.

Related

• Excel Assign Points Based on Late Time: How-To
• File-Based Routing in Next.js and Expo Router
• Top-Down vs Bottom-Up Design: How to Choose