By Thomas Nolle
Network World, 2/10/97

Your network is running slow. Users are nodding off, staring at screens that remain nearly static. Managers are screaming about lost productivity. Everything that isn't timing out is getting discarded due to congestion. The sky is falling or - worse - your salary is. What can you do?

To millions of network buyers, the answer is ''Switch!'' Many believe if they replace their routers with switches, all their networking problems will evaporate. Switching, we are told, is faster than routing. It will increase performance, reduce response time, and even let you run voice or video on your LANs. Buyers may believe in ATM switching or LAN switching, but they believe in switching.

But should they? The resellers who install LAN switches and have broad experience with them have reported some disquieting problems with packet discards on switch-to-server and switch-to-switch trunks. These problems aren't enough to cause us to question whether switching is useful in a general sense, but they are enough to warrant an exploration of how switching really works in a live network context. From that, we can draw some conclusions on just where it is - and isn't - likely to be useful to a given network buyer.

What's in a switch?

To meet that definition, a switch should generally have at its core a true switching matrix that provides an electronic pathway between any two points on demand, or a fast-switching bus that allows any port to inject information for another port to extract. It should also have port adapters that provide whatever electronic or protocol translation might be required to convert the attached device's protocol to the form acceptable by the switch core.

Applying this to the familiar area of LAN switching, the switch core forms a kind of ultrafast virtual LAN media. An attached data source presents a LAN message to its local port adapter, and the adapter translates the media access control (MAC)-layer address of the message to a switch port address. The adapter then presents the message to the switch core, addressing it to the port.

In the case of a multicast message, the switch addresses it to each member of the originator's VLAN. There's an assumption in LAN switching that the core network is too fast to be saturated by combined traffic. Remember that assumption; it's important.

ATM switching is a bit different. With ATM (or frame relay) switching, the assumption is there is a capacity limit to the ATM network. That means the network has to meter the applications it agrees to support to ensure that data gets through without congestion. To do this, ATM switches set up an enduring relationship, dubbed a ''call'' or a ''connection.'' The setup provides the network with the resources the call will need, and the network deducts this level from its pool of resources. If there aren't enough resources to support the call, the network rejects it. If there are, the network polices the data to ensure the caller doesn't send more than the agreed upon amount. This policing is done at the port adapter where the user first connects.

In most switch designs, there's a chance that the switch will be busy when data is presented at a given port. To prevent data from being lost, switches often provide buffers to hold the data until it can be processed. Buffers are simply memory spaces where data that has been accepted but not yet delivered can be stored. If a switch has buffers, it can absorb accidental traffic peaks long enough to keep from losing information, without the cost of making the switch fast enough to support the full-speed constant operation of all ports.

Switch builders have generally agreed that the best place to buffer information is at the input port. The user then feeds data to the switch at whatever rate the application happens to pick, and the switch collects it in a buffer.

When there's capacity to forward the data to the destination port, the switch pulls the next data element out of the buffer and passes it along. Frame relay network switches, for example, almost always provide large port buffers.

But buffers aren't always a good thing. If information builds up in buffers, it is being delayed. Delay, you'll remember, is one of the things we got switches to eliminate. This point explains why there is a difference in ATM switches and LAN switches.

With ATM switches, the assumption is that multiple switches form a network and are linked by limited-capacity trunks. The ATM switches need to ensure the fastest low-loss connection across the whole network. Thus, ATM switches often provide large buffers. With LAN switches, particularly those at the workgroup level, the virtual media can be made fast enough to reduce or eliminate the risk of congestion. LAN switches often have much smaller buffers and largely ignore traffic management issues.

What happens when switches that are not designed for buffering and traffic management run into resource congestion? You may be surprised to find that your switched network, under some conditions, will run slower - or not at all.

Exit point collision

It may be a bad one. There is also a question of whether the destination or exit port on the switch can keep up. In fact, it is the possibility that traffic can collide at the exit port that makes limited-speed carrier trunks a problem. If we have 20M bit/sec of traffic directed to a 10M bit/sec port, not all of it will be carried. In switches with a lot of port buffering, the excess will be held in the hope that the burst of traffic will subside and all the data can still be delivered. If the amount of buffering is small or the amount of colliding data is large, the switch will have to discard the information.

Hey, so what? Shared-media Ethernet LANs have collisions all the time and nobody gets upset. ATM networks are known to discard cells and nobody gets upset. Why should discarding due to exit port collision be bad?

The reason is simple: In shared-media Ethernet LANs, when two stations collide in their attempt to transmit information, each will sense the collision. That's what the ''CD'' in CSMA/CD stands for - collision detection. The stations will back off a random number of microseconds and then retry. The randomness of the back-off period for each will make it unlikely their second attempts will collide. But with most LAN switches, the sender's port adapter has already accepted the message by the time the exit collision occurs. The sending station goes happily about its business of preparing the next message, but the first message collides with other traffic at the destination switch port and is discarded.

That doesn't mean the data is lost, though. If the exit port adapter drops a message, the higher level protocol - if there is one - will recover it. TCP/IP will work in such a situation because TCP will detect the loss and retransmit after a significant delay. Datagram protocols such as those used for voice or video typically don't provide end-to-end error recovery. For those protocols, the data discarded at the exit port is lost forever. Worse yet, many protocols (such as TCP) will reset the flow control window size to the minimum value to prevent further discards. This significantly reduces performance until the window adaptively ''grows,'' which only happens if nothing else is discarded.

Exit port collision isn't all that unlikely either. There are two situations where it occurs in typical switched network applications: on server trunks and on switch-to-switch trunks. The slower the exit trunk, the more likely exit port collision because the chances are that a slow trunk will be tied up sending something when new data arrives. In both cases, shared destination ports are just like a shared-media LAN, which is ironic because that's what switching is supposed to be eliminating.

How bad does this get? One switched LAN study we did for an end user showed that eight active switched source ports running client/server application software could so overload a switch-to-switch trunk port that the effective throughput of the connection fell from 100M to 18M bit/sec. A similar degradation was reported to us by a LAN switch value-added reseller.

So the answer is ATM? No, because ATM in many of its current implementations has exactly the same problem. ATM LAN Emulation (LANE) and Multi-Protocol over ATM (MPOA) today rely on an ATM class of service called unspecified bit rate. With UBR service, the sender does not provide any description of the traffic flow that will begenerated, and the network doesn't make any resource reservations to carry it, police it or limit traffic at the source. Information flows into the switch at the rate the sender port can support; a 155M bit/sec ATM port could be fully utilized by a single UBR flow. When multiple flows converge on the same exit port (at the same speed or less), traffic collides at random - as a connectionless LAN flow would collide - and is discarded as readily.

During a recent road show, we talked to the user of a popular ATM switch who had exactly that problem when connecting two switches over a campus T-3 link. The client and server systems were running at 155M bit/sec, and the exit port collision at the T-3 trunk was so severe that the applications timed out and failed - which means zero throughput.

What can be done?

If exit port collision can ruin your application, what can you do to prevent it? There are two basic answers to the problem, each represented by some specific vendor implementations. You need to be sure what (if any) approach your vendor uses and what the ramifications of each are for performance.

The first approach is buffering. If a switch has sufficient buffers to ride out a period of collision, overall switch delay will increase (because of the buffering delay), but no data will be discarded. The question here is just what constitutes sufficient buffers. If there is no mechanism for constraining the senders of information, a long burst will eventually overflow any buffer if the combined data targeted to the port arrives faster than the port can send.

ATM switches tend to use buffering as their solution to the problem, but as the example of the VAR showed, multiple flows of UBR ATM originating in fast ports can overflow even ATM buffers. Where buffers are part of a central pool, it may be impossible to target available buffer capacity for specific congested output ports. And remember, no matter how big buffers are, it's still possible to overflow them if you can't shut up the senders.

The second strategy is to link switching and exit port processing so the sender of a message ''sees'' competition for the destination's port while data is still flowing. This allows the sender's port to create a kind of virtual collision at the MAC layer that forces the sender to back off and retry. In other words, this feature mimics the collision detection feature of shared-media LANs. Obviously, this approach works only with collision-detection MAC layers such as Ethernet. With token ring or FDDI, there is no MAC-layer flow control available.

To make flow control work, the switch must sense that the output port is busy when a data message arrives and generate a false collision to stop the sender from continuing to provide data. This is usually done based on buffer levels; Madge Networks, Inc. and 3Com Corp. are two vendors that offer this type of flow management on at least some of their products.

ATM promises per-station flow control through the Available Bit Rate (ABR) class of service, but ABR isn't fully deployed yet and most switch vendors offer it only on ATM-equipped desktops. Traffic originating at legacy adapters and entering an ATM switch using LANE or MPOA don't use ABR. Some within the ATM Forum have been promoting a change to the LANE and MPOA standards to allow ABR as well as UBR connections.

Some ATM vendors, including IBM, provide some traffic shaping and resource allocation even for LANE/UBR users. Most vendors will eventually adopt ABR as an option for LANE and MPOA when (and if) the standards provide for it. If you're planning on using ATM on the premises, find out how your prospective vendors handle this before you make the final selection.

Even if your switch vendor has one or both these features, you may need to quantify just how much their solution will help you. Some benchmark tests may provide a hint of what to expect from a given switch in terms of exit port collision risk. Look for tests where six or more input ports are used to drive a single output port. The more input ports, the better the test will mimic real multiclient server or switch trunk performance.

Performing the tests yourself is also possible, providing you have a LAN analyzer and can generate enough traffic to overload the server or switch-to-switch trunk.

What if none of the vendors you like can eliminate exit port collision? For all switch types, there is a third approach that buyers should seriously consider: making the exit port much faster than the source ports that will feed it data. This will reduce collisions to the point where they can be ignored. Even for LAN switches that gracefully handle exit port collision through buffering or flow control, faster shared ports/trunks will reduce switch delay and increase application performance.

Before you turn up your nose at this brute force strategy, you should note that it's the one most vendors recommend. Even buyers and resellers report that when 10M bit/sec Ethernet client systems are used with 100M bit/sec server or switch trunks, collisions are so rare that they can be ignored in most applications.

Resellers, in particular, like this kind of configuration because it reduces support problems, and they are recommending the configuration in increasing numbers. That would suggest that the biggest benefit of Gigabit Ethernet is its ability to oversupply these logical shared ports with bandwidth to prevent collision and performance loss, even when 100M bit/sec client systems are in use.

How much oversupply is needed? Testing in this area apparently isn't exhaustive, but interviews with users and resellers indicates that the trunk port should be at least five to six times as fast as the source ports. That ratio may have to increase if there are very large numbers of sources ganging up on a single destination. The each-generation-is-10-times-faster strategy of Ethernet meets this requirement. For ATM, using ATM25 client ports and 155M bit/sec trunks and server connections would work. Faster clients will require 622M bit/sec ATM servers and trunks.

Conclusion

If you've got fiber installed in your building, or access to dark fiber from place to place, the benefits of switching can extend as far as you can extend that glass. As soon as you hit a carrier service, though, you may find your bandwidth so limited by cost that switching won't provide any noticeable benefit. Don't throw your routers away, particularly the ones that support your WAN connections. You'll need them until the speed of those connections rises substantially - probably at least to the OC-3 level.

Speed of trunking is the ultimate answer to switch performance. If you can't provide it, be sure your vendor can handle exit point collision gracefully. Finally, if you're an ATM fan, grill your vendor on the use of ATM ABR service with all forms of LAN-over-ATM connectivity. If you don't address these points in your switch purchasing, you risk major performance problems now and in the future.