VoIP should be an easy fit for wireless LANs, but mixing the two technologies today is difficult. Despite VoIP's low-bandwidth profile, even a small amount of data traffic on the same network can lead to seriously degraded audio quality and dropped calls, even with QoS features enabled.

That's the major conclusion of our first-ever assessment of VoIP capability in WLAN systems. Over the course of three months we tested WLAN switches and access points from Aruba Wireless Networks, Chantry Networks (now Siemens), Cisco and Colubris Networks in terms of audio quality, QoS enforcement, roaming capabilities, and system features. Other vendors, including Airespace, Meru Networks and Trapeze Networks, declined to participate (see "How we did it").

Among our major findings:

•  With QoS enforcement enabled, the products delivered near-toll-quality audio, provided only voice traffic is active. This is fine as long as the wireless network carries voice traffic only, but that's not likely as companies move toward converged voice-data networks.

•  When voice traffic had to contend for bandwidth (even with a little data traffic), dropped calls were common and audio quality on the remaining calls was poor in many cases - and this was with QoS enforcement enabled.

•  With data traffic present, roaming from one access point to another took anywhere from 0.5 to 10 seconds - in cases where roaming succeeded at all. These long delays and dropped calls made roaming practically impossible with some vendors' gear.

While some products struggled mightily in our tests, Aruba's A2400 and A800 switches and A61 access points were consistently strong performers. The Aruba products posted generally excellent numbers, regardless of how much voice or data traffic was thrown at them. Aruba's gear just worked, earning it the Clear Choice Award.

Two issues confounded other vendors. First, when handling voice and data traffic on the same network, vendors need to pay attention to metrics such as delay and jitter rather than forwarding rates.

Many vendors are only just beginning to tune their products for voice/data convergence, even though some have touted that capability for 18 months or more. However, it's still relatively early days for VoIP over WLANs. Test tools that accurately measure these metrics on WLANs (such as the VeriWave instruments we used) are only just beginning to appear, and this test is among the first to measure audio quality, delay and jitter in a methodical way.

Second, the emerging 802.11e standard for QoS on WLANs might bring some relief. The 802.11e specification wasn't yet ratified when we began this project, so by definition all QoS methods were nonstandard. Companies might want to wait until the new 802.11e specification and products based on it are more mature and fully tested.

Click to see: Voice quality with and without QoS

Measuring voice quality over wireless

Our tests sought to answer a simple question: How does a VoIP over WLAN system sound?

To find out, we worked with VeriWave, a start-up that makes WLAN test and measurement equipment. VeriWave developed a new application, the VoIP over WLAN Analysis Test Suite, especially for our test.

In addition to collecting delay and jitter statistics, VeriWave's test suite and TestPoint hardware let us measure R-value, an ITU specification (G.107) for determining call quality. R-value is an objective measurement, computed directly from measurements of packet loss, jitter and delay. While R-value is objective, it has a strong correlation to the subjective Mean opinion score method in ITU standard P.80 (see R-value ratings).

We measured voice call quality with up to 14 handsets and an H.323 call server from SpectraLink, a maker of 802.11 handsets. We measured audio quality with up to seven concurrent calls, and in some events configured the VeriWave TestPoint boxes to offer background data. For each system tested, we checked call quality with QoS disabled, then enabled.

Audio quality without QoS

With QoS disabled, we started by routing all calls through one access point. Because all the vendors recommend enabling QoS for voice traffic, this baseline test gave us a "before" picture to demonstrate the need for voice traffic prioritization.

With QoS turned off, all four systems tested did fine with only a single call active, with R-values hovering around 78. That is about as good as it gets with VoIP over wireless. The threshold for near-toll-quality voice is generally considered to be around 75, meaning the systems delivered good audio quality for a single call.

Performance for all systems changed across the board when we placed six or seven calls through a single access point and switch, especially when data traffic was active. Yet even without background data, we could not test the Colubris system with seven calls active and QoS disabled - all the calls dropped.

When we configured the TestPoints to offer background data (a stream of User Datagram Protocol [UDP] packets at 1M bit/sec), the results were positively awful without QoS. With only six concurrent calls, R-values for all systems (except Aruba) were generally at or below the point where voice signals were unintelligible or calls were dropped. Sound quality through Aruba's system remained high, roughly the same as with no data, even without QoS.

The Chantry and Colubris systems could not perform the background data test with seven calls (QoS disabled). All calls failed as soon as the VeriWave box began offering background data.

All vendors recommend the use of QoS mechanisms for handling voice traffic, even when no data traffic exists. QoS is a must when handling VoIP traffic over a WLAN.