We registered for a customer account for each of the seven services tested. We used whatever Web-based tools were available from each service to set up an e-mail alert profile and, whenever possible, established Short Message Service text alerts.

We first focused on Windows alerts and then expanded our alert profile to cover the full list of products sitting on our test network. This list included Windows (2000 Server and Workstation, 2003 Enterprise Server and XP Professional), Red Hat (Fedora, Red Hat 9 and Enterprise Server 3.0), FreeBSD, Apache 1.3 and 2, IIS 5 and 6, Snap Server, proftpd, sendmail, postfix, samba, MS Office (XP and 2003), WebLogic application server, Sophos anti-virus, NetScreen Technologies' firewall, F5 Networks load balancer, Cisco routers, Cisco VPN Concentrator 3000, Network Appliance storage appliance, Brocade Fibre Channel switch, Snort, SQL Server 2000, Oracle 9i and OpenSSH.

For focused alert testing, we selected five announcements ranging from mundane to obscure that had occurred at some point during the 45 days we tested these services. For each announcement, we reviewed the classification of the alert, the detailed information provided in the alert, the time of delivery of the alert, and the general format and layout of the alert.

The alerts selected for detailed testing include the Internet Explorer IFRAME vulnerability (CAN-2004-1050), Apache mod_include vulnerability (CAN-2004-0940), Linux kernel IPTables initialization failure (CAN-2004-0986), Cisco ISO DHCP Denial of Service (CVE-2004-1111), and the Solaris Samba buffer overflow (CAN-2004-0686).