Can you hacker-proof your IP telephony network? The short answer - as demonstrated in the first-ever public test on this topic - is: Yes, pretty much. But it strongly depends on whose IP PBX you use and more importantly, whether you're willing to spend the dollars and the time it takes in terms of network security planning, network and personnel resources, and extra security gear.

In our tests, we developed a plan for realistically assessing how secure vendors' IP telephony packages are - or aren't - against a determined, malicious attacker. While we invited the top five vendors by VoIP market share to participate, only Cisco and Avaya stepped up to the challenge.

Cisco's "maximum-security" VoIP configuration - a midsize CallManager-based system, with call control, voice mail, gateway; a Catalyst 4500- and 6500-based Layer 2/Layer 3 infrastructure; a copious supply of intrusion-detection system (IDS) and PIX firewall security add-ons; plus a half-dozen Cisco security gurus supporting the test - earned our most Secure rating (see rating criteria, below). Our attack team couldn't disrupt, or even disturb, Cisco's phone operations after three days of trying.

Avaya submitted two configurations: A no-frills, out-of-the-box Avaya IP telephony deployment with no extra-priced security options; and a maximum-security alternative - featuring the same VoIP gear, but with an added firewall and Layer 2/Layer 3 infrastructure switches from Extreme Networks. Security weaknesses earned the basic Avaya configuration a so-so Vulnerable rating, while the hardened package fared better with an overall Resistant rating.

The ground rules (see below) imposed some limitations on the four-member assault team. For example, only hacker tools and attacks that were available on the Internet could be used. Attacks had to be launched via an end-user data port or IP phone connection, as if the hacker had access to a standard office cube; attackers could not disassemble or dissect the vendor's IP phone - and so on.

The objective was to disrupt phone communications. Via the data and IP phone connections, the attack team used scanning tools and other techniques to see and learn what they could of the topology. The attack team was told nothing of the vendor's configuration beforehand. After discerning and identifying "targets," the hackers then systematically launched dozens of attacks, at times in combinations concurrently.

Given the limits set by our ground rules and the duration of the tests, it is important to note that the attacks launched against these products are not as severe as those that could be encountered in an actual deployment. We consulted with a half-dozen security experts regarding these attacks, and they concluded that the attacks were of moderate intensity.

We will not disclose in this story complete details of vendors' specific vulnerabilities uncovered and exploited, so as not to put customers using these products at risk. These exploits are therefore discussed in general terms.

Like a rock

Cisco proved it could build a VoIP network that a sophisticated hacker assault team could not break or even noticeably disturb. The elaborate IP-telephony package - with underlying Layer 2 and Layer 3 infrastructure and assorted security add-ons (see "Cisco maximum-security topology") - is the most secure that Cisco's collective network security expertise could muster, and employs every defensive weapon in the Cisco arsenal.

Click to see: Cisco maximum-security VoIP topology

The Cisco topology tested certainly represents more security options and stricter security settings than most users currently employ, but all are available today for a price. The optional components included: two stand-alone PIX firewalls (about $8,000 each); another firewall on a blade in the backbone Catalyst 6500 (about $35,000); an IDS blade also in the 6500 (about $30,000); an entirely separate, out-of-band management subnet and various security-management applications. The price for the firewall and IDS pieces came to slightly more than $80,000. Cisco says, though, that it threw in systems that it could readily get its hands on, and that the same job could be done with less-expensive firewall and IDS models from Cisco.

The firewalls brought some very useful, high-level security features to the table. One is the notion of trusted vs. untrusted sides - and the untrusted interfaces were always pointed toward our hackers. Another is a stateful understanding of protocols, so that only specific VoIP protocols required for VoIP were allowed, with requests and responses passing only in the appropriate directions. Other firewall features that came into play during this test included:

•  Stateful inspection of VoIP call control, and the ability to network address translation and tunnel call control through the firewall.

•  TCP intercept, which makes sure TCP connections are completed. This can prevent certain denial-of-service (DoS) assaults on the CallManager.

•  Secure Skinny Call-Control Protocol (Secure SCCP) support. This is the newer, more secure form of Cisco's proprietary SCCP that the company used in this VoIP network. Secure SCCP uses a TCP connection rather than User Datagram Protocol (UDP) and encrypts call control information.

Enter CallManager

Version 4.0 of CallManager, which handles call control and is the heart of Cisco's IP telephony package, includes some new security-related features. Key among them is the company's first VoIP encryption implementation. At this time voice-stream (Real-time Transfer Protocol [RTP]) encryption is supported only on Cisco's newer 7970 IP phone sets. The latest CallManager also has been additionally hardened, along with the underlying Windows 2000 operating system, according to Cisco. For our tests, this meant that open ports were closed and unnecessary services disabled.

An impressive array of network self-defense features is included in the Catalyst IOS versions tested. Specifically, we had IOS 12.2(17b)sxa on a core Catalyst 6500, and IOS 12.1(20)ew on an access Catalyst 4500. These capabilities did more to thwart our assaults than any other component in the Cisco topology because they were the first line of defense. They include:

•  Traffic policing and committed access rate, which were very successful in fending off our DoS assaults.

•  Layer 2 port security, which restricts the number of media access control (MAC) addresses on a port.

•  Layer 2 Dynamic Host Configuration Protocol snooping, which prevents dynamic host configuration protocol exhaustion attacks.

•  Dynamic Address Resolution Protocol inspection, which stops ARP poisoning and ARP spoofing attacks. This, too, frustrated a number of our attack team's more insidious assaults.

•  IP Source Guard, which prevents impersonation attacks.

•  Virtual LAN (VLAN) access control lists, which restrict the traffic that can reach IP phones.

Cisco Security Agent (CSA) is a host-based intrusion-prevention system (IPS), and is now an integral security component in CallManager IP telephony servers. It was also on Cisco's Unity voice mail server and all other Win 2000 servers (seven CSA agents in all) deployed throughout Cisco's network topology. The CSA agent runs automatically and unattended, and provides some powerful safeguards at the server, including:

•  Buffer overflow protection, which protects the server's protocol stack from attacks involving malformed data packets.

•  Network worm and Trojan prevention (not tested).

•  Prevention of unauthorized application from running.

•  Protection against syn flood attacks - a family of DoS attacks against the server's TCP processing.

•  Detection of port scans, which all hackers employ to determine vulnerabilities based on a server's responses to specific services and port numbers.

Whitepapers

• Best Practices for HP Servers and HP Enterprise Virtual Array in a Microsoft Exchange
• Best Practices for WAN Optimization with Data Replication
• Demonstrating the Business Value of Mobile Device Management in the Enterprise
• Managing Mobility: An IT Perspective
• High Definition The Evolution of Video Conferencing
• Seeing is believing IDC White Paper

View more VOIP & CONVERGENCE whitepapers

Webcasts

• Economical Disaster Recovery through Virtualization- Thurs., July 10, 2008
• Free BlackBerry Technical Seminar, July 9
• Hosted VoIP Services. Is it for you?
• High Availability for SQL Server 2005 Using Array-Based Replication and Host-Based Mirroring Technologies
• Technical Overview of Exchange Server 2007 with HP Servers and Storage
• Bringing Order and Security to your Mobile Workforce: Corporate Mobility Policy and Device Management On-Demand

View more VOIP & CONVERGENCE webcasts

Special Reports

• The Wireless Explosion

View more VOIP & CONVERGENCE special reports