Performance testing these Web server packages was a little simpler, logistically speaking, than analyzing their user-appreciation attributes (although, we had problems with some products not working with our test). We architected a test that represented medium traffic on a typical Web server. Such a system serves HTML pages, text files, and images, rather than supporting video-streaming, CGI programs, and so forth. This approach made the tests easier and faster to run and represents how some people will use the packages. Our test characterizes average performance from each server package and demonstrated differences in the software.
We ported Silicon Graphics' WebStone 2.0 to Windows NT, tweaked it a little, and ran it on a four-system setup in the Windows NT Magazine Lab, all on an isolated 10 Base T Ethernet network. The setup consisted of a Primary Domain Controller (PDC), the Web server, and two client systems dividing the simulated user access load. We used an NEC RISC-Server 2200 as the PDC, configured with a single 200MHz MIPS R4400, 64MB of RAM, and a 2GB SCSI disk (look for a review of this server in the October issue). We installed NT 4.0 beta 2 running DNS.
The Web server (on which we ran all the Web software packages) was an Intergraph InterServe Web-300, with a 150MHz Pentium Pro CPU, 64MB of RAM, and two 1GB fast SCSI-2 drives (for a review of this server, see Joel Sloss, "Serving with Style," on page 45). This system ran NT Server 3.51 with Service Pack 4.
A Micron Millennia and a Canon Innova Pro 5400ST were the clients. We configured each system with NT Workstation 3.51 and Service Pack 4, and installed Remote Shell and Executive services.
We configured the tests with runs of 20, 25, 30, and 35 client sessions distributed between the two workstation systems. Each run lasted 10 minutes, and we repeated each three times. The clients retrieved a 5120-byte file (which they used 50% of the time), a 500-byte file (35% of the time), and a 51,200-byte file (15% of the time) from the Web server. Each test took two hours to complete.
To ensure that the test ran properly and that nothing influenced the server packages or other containable factors, we reinstalled NT Server for each run by booting the Web server to NT Workstation and copying a clean version of Server back to the root directory. This way, we restored the Registry and system files to their original state for each package.
We extracted some meaningful results from the test by evaluating several factors for each server package: server connection rate (connections per second), server throughput (Mbits per secondMbps), average response time (in seconds), average client throughput (Kbits per secondKbps), and total number of pages read. Although we attempted to test all 15 Web servers, only 10 servers worked within our test criteria. Table A shows the statistical results and Figure A shows the connection rates for the server packages we tested. We relied on server connection rate for performance characteristics to test each package's capacity under this test. The other values show what Web users can expect from your system when you run each package.
Microsoft's IIS 2.0 edged out Netscape's SuiteSpot (and FastTrack Server 2.0) and Internet Factory's Commerce Builder Pro 1.51. Tight integration with NT contributed to this winning performance.
Design also had a lot of impact on how each package performed during testing. Software such as IIS 2.0 showed little effect on the Web server's CPU utilization during the test (with barely 40% average utilization), whereas other packages such as WebSTAR NT/95 from Quarterdeck, maxed out the CPU. Quarterdeck pulled WebSTAR from the market before we finished the review, which is why it was not included.