Caching wards off Web access worries

The World Wide Web's rapid growth, coupled with the increased use of Web technology in corporate intranets, has ushered in an era of unprecedented levels of user frustration and ever-growing WAN costs.

Why? Because the Web's underlying infrastructure wasn't designed to support the demands now being placed on it. And it's only getting worse.

Web caching is the most popular approach to solving these problems. A cache acts as the middleman between the end user's browser and the Web server housing the content. When a browser requests a Web page, the request is initially directed to a local cache.

By creating a distributed system that stores frequently accessed content close to users, response times are improved while WAN traffic is reduced because Web requests are serviced locally without having to traverse the Internet.

There are two basic modes of cache operation: proxy caching and transparent proxy caching.

With proxy caching, or the nontransparent mode, each client browser must be manually configured to send all Web requests to the correct cache. There are major drawbacks to this approach.

First, each client can only hit one cache server, even if multiple caches have been deployed for scalability. Second, because the proxy caches generally cannot redirect requests for content to other caches, all pages must be stored on all caches, leading to unnecessary duplication of data.

Third, additional user delay and data duplication occurs because the cache must service all Web traffic, including intranet traffic. Finally, if a user's browser points to a cache that is down, then the user has no Web access until his browser is reconfigured or the cache returns to service.

Client administration is a major disadvantage of proxy caching because every browser must be configured to point to the cache. The amount of administration needed on the browsers increases the overall administrative burden because there are many more browsers than there are caches or intermediate systems.

The other mode of Web cache operation - transparent proxy mode - fixes many of these problems but introduces several new ones. In transparent proxy mode, the cache sits in the data path examining all packets bound for the Internet, intercepting and servicing the Web traffic.

While this eliminates one of the big disadvantages of proxy caching - the need to configure the individual browsers - clients still generally hit only one cache because there is just one active path between each client and the Internet. Moreover, network availability can be compromised because the cache sits in the data path. If the cache fails, users lose all Internet connectivity and possibly some intranet connectivity.

Transparent proxy mode can slow down all kinds of communication to the Internet. In this mode, caches must process every packet to filter out Web-related traffic while passing the rest along. This requires that caches be configured as next-hop routers or as the default gateway for clients with two network interfaces, one connected to the client side of the network and the other connected to a path to the Internet. The cache must be able to route between the interfaces so non-Web traffic that needs to go to the Internet can get there. Unfortunately, the processing architecture that works best for caches is not well-suited for examining high volumes of data packets.

Web-cache redirection is a new technique that helps solve the deployment and administrative problems associated with Web caching. Web-cache redirection removes the cache from the data path and improves cache performance by off-loading many CPU-intensive network functions they must perform.

Web-cache redirection is implemented on specialized LAN switches or routers already in the data path, providing connectivity to servers and caches. Unlike traditional network devices, Web-cache redirectors distribute processing capacity to ensure that packet processing is maximized and not limited by a single centralized CPU.

With Web-cache redirection, HTTP traffic is intercepted, the appropriate packet manipulation is performed, and the traffic is redirected to the cache housing the requested content. Meanwhile, all other traffic, such as Simple Mail Transfer Protocol or File Transfer Protocol, is switched at Layer 2 to the Internet. The cache is moved out of the data path, and the network functions performed by the cache are off-loaded to the redirector or, in some cases, eliminated entirely.

By removing packet examination and manipulation from the cache it frees up CPU cycles because the cache doesn't have to be configured to perform Network Address Translation or routing. Web-cache redirection brings caching as close to zero administration as possible.

Williams is director of business development at switch maker Alteon Networks, Inc. He can be reached at (408) 360-5500.

Bandwidth hunger creates a Cache-22:
Network World, 2/23/98

More than you'll need: With their latest proxy servers, Netscape and Microsoft go beyond what's required and deliver configuration and management headaches to boot.
Network World, 2/9/98

A protocol for clogged intranet arteries:
Network World, 10/6/97