Review: Intrusion-detection products grow up
 
|
|||
 | |||
|
Last year, intrusion-detection systems were difficult to install, performance was lackluster and management was raw.
Things have changed.
IDS has evolved significantly, with products now containing new features, richer management applications and much better performance.
We tested five network-based IDS products and their associated management consoles for this review. We did not include host-based IDS, to ensure a true apples-to-apples comparison. We tested products from Cisco, Computer Associates, Enterasys Networks, Intrusion.com and Internet Security Systems (ISS).
Two products share the Blue Ribbon Award: Cisco's Secure IDS and ISS's RealSecure. Enterasys' IDS Dragon had stellar performance, but difficulties with installation and configuration, as well as some rough edges on the management, kept it in second.
CA's eTrust products made a strong showing in management and features. Its management application was intuitive and logical, but it entailed too many separate applications, which limited its ease of use. Intrusion.com's SecureNet Pro was the easiest product to install; it was running within 15 minutes. But the product had some performance problems, lowering its overall score.
Intrusion battleground evolves
How we did it
NetResults
Attacks we performed and performance of IDS products tested
Features overview
Interactive Buyer's Guide
What they offer
The IDS products from Cisco, Enterasys and Intrusion.com are appliances, while CA and ISS provided software-based systems to test. ISS also offers an IDS appliance through a partnership with Nokia, but we did not test that.
The network-based IDS products consisted of separate "sensor" and management functions. The sensors watch passing traffic and detect the attacks, and also perform functions related to intrusion detection. The system management function, which includes event management and reporting, was included on a separate console.
A variety of sensor platforms are employed, including Windows, Unix, Red Hat Linux and Solaris. With the exception of Enterasys, which uses Web browser access for management, the other products employ a Windows-based management server and/or client software. In addition to its Secure Policy Manager application, Cisco Secure IDS is also supported by Cisco Secure Director, which we did not test. This alternative management software runs on Hewlett-Packard's OpenView.
Performance
Several tests were conducted to measure performance. First, we measured how well the product could detect a random sample of commonly recognized intrusion attacks, such as ping floods, Jolt2 attacks, SYN floods, finger bombs and others. These were tested initially under no background traffic load. To achieve a passing score, the IDS had to correctly identify the attack within 5 minutes of the attack's launch. We tallied whether the intrusion was recorded, if it was correctly identified and the approximate time it took to recognize the attack.
Next, we ran stress tests to see how the products would work as background traffic load increased from 40M to 60M bit/sec, then up to 90M bit/sec. If we determined that an IDS could not detect an attack under the "no load" condition, we eliminated that attack from the stress tests. A third test determined whether the products could detect attacks specifically designed to avoid traditional IDS systems.
Performance was scored on three things: number of attacks detected, ability to detect attacks under load (the stress tests) and fault tolerance.
Enterasys' IDS Dragon took the gold in performance. In addition to its excellent showing in the first two performance tests, Dragon also beat the competition by detecting attacks that are specifically designed to avoid traditional IDS systems. IDS Dragon also performed with near bulletproof reliability - demonstrating minimal performance degradation under traffic load and solid system stability all during the tests.
The IDS Dragon missed only three out of 27 random attacks and detected 24 out of the resulting 24 attacks sent to it under the 40M and 60M bit/sec traffic load. With the 90M bit/sec traffic, IDS Dragon correctly detected 21 out of 24 attacks - very good under maximum load.
No other product performed as well with the basic intrusion-detection and stress tests, although Cisco Secure IDS performed well under load, detecting 19 out of 21 attacks sent to it under 40M-, 60M- and 90-M bit/sec loads. The ISS RealSecure performed well under 40M and 60M bit/sec loads, detecting 22 out of 24 attacks, but fell down to 17 attacks out of 24 when the traffic load went to 90M bit/sec.
Intrusion.com's SecureNet Pro had the hardest time under heavy background traffic loads. After a strong start - detecting 24 out of 27 attacks with no load - performance steadily declined as load subsequently increased. The SecureNet detected only four out of 27 attacks under the 90M-bit/sec load. Curiously, SecureNet detected the highest number of attacks (25) under no load, but supported the smallest database of known attack signatures of the products tested.
All the products tested did well in detecting certain attacks - including Whisker (various types), Targa3 and Bind - that are specifically designed to evade network-based IDS products. Cisco, CA, Enterasys and Intrusion.com detected 16 out of 17 attacks, while ISS got them all.
While CA's eTrust IDS performed adequately in our stress tests, it did not perform consistently under high (90M bit/sec) loads. It appeared that the longer we let the background traffic stream run (up to 10 minutes or more) the less consistently the eTrust could detect the attacks. It was for this reason that we rated eTrust's performance a 3 out of 5.
With few exceptions, the products tested were otherwise stable. While the ISS RealSecure was generally stable, its performance was affected in one instance. When we sent 90M bit/sec of traffic over 10 minutes, then launched an extended ping flood for several minutes, RealSecure could not detect any of the Internet Information Server exploits we tried. We also noticed that running a Jolt2 attack seemed to easily blind Intrusion.com's SecureNet for a brief time after the attack subsided.
Managing all this
All the vendors tested made huge strides in their management applications. They all performed well in generating reports, and they all exhibited their ability to adequately manage events and large deployments of IDS sensors.
Managing a large network of sensors is typically achieved through a three-tiered architecture: a central management console, sensors and an event collector that off-loads processing from the management console, but reports back to it. Under this arrangement, one event collector manages, for example, up to 50 sensors, but each management console supports multiple event collectors, thus facilitating scalability. All the vendors except CA have embraced this model. CA doesn't use the event collector, just the sensor and management console.
Cisco and ISS tied for top honors in this category. Cisco's Secure Policy Manager, which runs on Windows NT/98/2000, supports the best event management along with a highly intuitive, logically designed interface that was a breeze to use. Items were color-coded and easily sorted, and we could configure which fields we wanted displayed, easily viewing more (or less) detail, as we specified. The Secure Policy Manager also has excellent reporting and statistics, featuring easy-to-use templates for generating reports. Functions were well integrated into Secure Policy Manager, which is slated to eventually become the single manager platform for all of Cisco's security devices, including its PIX firewall.
In addition to Secure Policy Manager, Cisco supports Unix-based management and an HP OpenView-based platform, running the Cisco Secure Director plug-in, which was not tested for this review. Our only issue was its management of multiple sensors. While powerful, it was a bit cumbersome for us to set up and maintain.
The ISS RealSecure Manager, which resides on Win 2000/NT or Solaris platforms, is on par with Secure Policy Manager, supporting excellent event management, good reporting and the best integration of applications. One of the earlier IDS vendors, ISS has had more time to better integrate functions. Incorporation of a three-tired architecture facilitates management of multiple devices, and sensors are easily deployed.
CA, Enterasys and Intrusion.com were a step below, but were still good in this category. CA's eTrust Intrusion Detection Management, which runs on Win 98/NT/
2000 and Millennium Edition platforms, delivered the best statistics reporting of all five products tested. They were comprehensive and complete. But eTrust's was limited by the use of several different applications that should be integrated. For example, there were separate applications for real-time statistics and detailed monitoring. They would have been more useful if combined. With that, eTrust's management would be on par with Cisco and ISS.
While Enterasys' Web-browser based Dragon Policy Manager had good reports (we especially liked one that let us pick the most common attack and observe it closely) and statistics, its event management wasn't as robust as the other products. Events were displayed but couldn't be sorted, and we couldn't designate which fields to display. We could filter events, but we found this a tedious process. We also found the Dragon Policy Manager difficult to navigate.
However, it included a good forensics tool that let us drill down into the precise details of attacks and analyze them. While all the other products supported a similar feature, Enterasys did it best. Those familiar with Unix will like IDS Dragon's Unix-based command-line interface, which is traditional and familiar. But like CA, Enterasys needs to better integrate its varied application tools to make its management more effective and efficient.
Intrusion.com offers two management applications: the SecureNet Pro, an X-windows management application for Linux-based platforms, and the SecureNet Provider Win 2000-based application, which was developed for better management of larger deployments of sensors. The Windows application is well suited for
larger environments because it employs a three-tired architecture while the Linux application does not. While both are good, they offer distinctly different feature sets, which we found problematic.
Installation and ease of use
All products tested were easier to install and configure than in our previous tests. Overall, the appliances were easier to install than the software-based products because the software-based products also require "hardening" of the platform's operating systems, which turns off unnecessary servers (such as telnet or FTP) that could affect security or performance.
Intrusion.com's SecureNet PDS appliance was the easiest to install. Within 15 minutes, we were up and running with minimal tweaks. The Cisco Secure IDS, an appliance, was also easy to install, but because the product supports so many advanced settings and configurations, it was easy to get lost trying to find things. Also, Cisco doesn't always make good use of screen space. For example, on Secure Policy Manager's General Signatures screen, there's a small window for scrolling through signatures that the user can't expand, although there was lots of available and unused gray space around the window.
Installing CA's eTrust was intuitive and logical, but, as noted, it had too many separate applications, which limited its ease of use. CA says it is addressing this issue for future releases.
Enterasys' initial screen gives the user five options, such as policy configurations and real-time monitoring, but once you drill down into any one area, it's not easy to get back out to another. And because we were working with a Web browser, we had to repeatedly hit the "back" key to return to the main home page and then go forward to another. While it was possible to open multiple windows and switch between them, we found that cumbersome. Unix users will find that setting up the IDS Dragon Sensor through the Unix-based command-line interface fairly straightforward, although Windows users might get frustrated.
ISS's RealSecure software installation took the longest, but still took less than 30 minutes (minus hardening the server). RealSecure had the most logical and intuitive graphical user interface - almost everything we needed to see to install the system was visible on one screen so it was possible to manage devices and events without navigating through different screens. However, we found the RealSecure's "wizards" the least intuitive feature. Instead of stepping us through a process from beginning to end, the wizard presented a list of options for further selection. The wizards would have been more helpful if they were more focused.
Features
All the products supported a full complement of IDS features. With few exceptions, the products supported nearly all the specific features we looked for during our analysis.
Cisco Secure ID supported the largest database of known attack signatures, while Intrusion.com's was the smallest (which didn't prevent the SecureNet from recognizing the highest number of attacks in our random attack tests). Enterasys supported the most granular attack database, providing more details about attacks than the other products. However, some end users just want to know they've been attacked, so it's up to the individual to decide how much detail is a good thing.
Intrusion.com's SecureNet was the only product that did not support automatic update of attack signatures. It requires a manual download and installation of signatures, which forces the administrator to update each remote sensor individually - a tedious process and the main reason SecureNet scored a 3 out of 5 for features.
All the products supported a detailed explanation of attacks, including the Common Vulnerability and Exposures database of known vulnerabilities, and Bugtraq IDs, a Web site on which most security professionals release an exploit once it is discovered (http://www.securityfocus.com/).
All the products also supported a stealth-mode monitoring interface, which lets a network interface card (NIC) see all the traffic so the IDS can analyze it. NICs typically look at media access control addresses and listen only to traffic that is directed to it; software on the IDS puts the NIC into stealth (or "promiscuous") mode so that it can see everything.
Vendors have based their features development on addressing previously known vulnerabilities, such as Unicode detection and TCP reset prevention mechanisms, into their products. Unicode is a uniform coding scheme that allows communication between diverse texts (the current version is Unicode 3.1.1; see http://www.unicode.org/ for specifics). Previously, IDSes could not read Unicode, and attackers took advantage of that vulnerability. A TCP reset lets the IDS send out a spoof packet that terminates a TCP connection instead of telling a firewall to stop all packets.
All the IDSes also now support shunning, or prevention, of attacks, but it is turned off by default, and it's up to the user whether to use it.
Configuration
The Cisco Secure IDS was the bulkiest of the appliances (it is a 4-U appliance; the others were 1-U appliances), and it reported power status and disk activity, but had no link-level LEDs. It was the only product that was not offered as a stand-alone software product (as well as a stand-alone hardware/software appliance).
CA's eTrust, one of the two software-based products tested, had low minimum-system requirements, which might not support enough horsepower for high-speed-link monitoring. CA was the only vendor that did not offer an appliance-based IDS product - software only.
Enterasys offers appliance and software-versions of IDS products. We tested the appliance version, which supports LEDs that were small, recessed, not clearly labeled and hard to read. The IDS Dragon is well architected, though, incorporating a multitiered architecture for improved scalability. However, the event collector runs on the management console rather than on a separate platform, which would better conserve management resources.
Intrusion.com had the most well-designed appliance, with LEDs that were especially useful and easy to read, although there was no power button on the front of the device, which would have been a plus.
The ISS RealSecure, which we tested in software, has the advantage in scalability. RealSecure has distinctly different event collectors, and management is run on a separate machine, leaving it free to use all its resources for management.
Wrapping it up
IDS products have undergone a metamorphosis during the past year, blossoming into sophisticated machines that are easier to install, and incorporate better management and performance. But they could still use improvements in event correlation and management, and their ability to support speeds beyond 100M bit/sec.
Related links
Yocom is senior editor, Brown is lab test engineer and Van Derveer is lab technician at Miercom, a network test lab and consultancy in Princeton Junction, N.J. They can be reached at byocom@mier.com, kbrown @mier.com and dvanderveer@mier.com.
Intrusion battleground evolvesThe future of intrusion detection is hybrids of network- and server-based products, faster speeds and improved event correlation and analysis. But prices won't fall. How we did it
Our testing methods explained. Interactive Buyer's Guide
Use our buyer's guide to find the intrusion-detection tool that's right for you. We've got specs on 42 hardware and software-based tools. Scorecard and NetResults
