Executive Summary:
As the established leader in the enterprise virtualization market, VMware’s ESX Server sets the bar for Microsoft’s new Hyper-V technology. Find out whether Microsoft’s Hyper-V matches the performance levels set by VMware’s ESX Server.

Editor's Note: The version of Hyper-V that was tested in this review was a beta. Performance optimization for Hyper-V occurred after the beta that was tested. You can read Microsoft's letter in response to the testing here.

As the established leader in the enterprise virtualization market, VMware’s ESX Server sets the bar for Microsoft’s new Hyper-V technology. In “Virtualization Shootout, Part 1” (June 2008, InstantDoc ID 98879), I examined the two products’ features, setup, basic management, and price. I found the products to be roughly equal—even though the Hyper- V version I tested was the beta version that Microsoft released with the initial Windows Server 2008 launch, Hyper-V was definitely in the same ballpark as the more mature ESX Server 3.5. I did find ESX Server easier to install and get running, and its management console is more polished and professional. Although Hyper-V was a bit more difficult to set up, its feature set was directly comparable to ESX Server’s. Both products are capable of running Windows and Linux OSs, and both support highly scalable 64-bit hosts and guests with up to 64GB of RAM per virtual machine (VM). Hyper-V looks good on paper, but the only way to truly know if it’s roadworthy is to take it out for a test drive and see if it can match the performance levels set by ESX Server.

Start Your Engines
I originally attempted to conduct the Hyper-V testing using the Server Core version. However, I ran into a lot of trouble getting Hyper-V set up and running on Server Core. I had no problem installing the OS or implementing the virtualization role, but the remote management capabilities for Hyper-V on Server Core were MIA. Remote management is mandatory for Server Core because Server Core doesn’t provide a graphical interface. I’m sure much of the problem stemmed from the code’s early release state. For more information about Hyper-V’s remote management capabilities on Server Core, see the sidebar “The Good, the Bad, and the Ugly." Because of time constraints, I ended up running Hyper-V on a full Windows Server 2008, Enterprise Edition installation—which theoretically shouldn’t make much difference because the Hyper-V hypervisor runs beneath the OS and the Windows Server management partition runs in a VM by default. However, using the full Server 2008 OS installation increases the overall system memory requirements, as well as increases the system’s attack surface and the need for patching.

Before beginning the performance testing, I needed to create a set of VMs on each platform. I started my testing with ESX Server and used the Virtual Infrastructure Client to create eight VMs. Using eight VMs is representative of a small-to-midsized business’s requirements. In addition, using eight VMs takes advantage of Server 2008 Enterprise’s licensing model, which allows four active Windows instances running in VMs with no additional licensing. Using eight VMs requires two Server 2008 Enterprise licenses. For more information about licensing costs for my test scenario, see “Virtualization Shootout, Part 1” (June 2008, InstantDoc ID 98879).

ESX Server 3.5
PROS: Excellent performance; easy installation; a polished management console
CONS: Somewhat limited hardware support
RATING: 5 out of 5
PRICE: $1,640 for Virtual Infrastructure Foundation, with 1 year of support; $2,640 with 3 years of support
RECOMMENDATION: ESX Server is great for medium and large businesses looking for performance and manageability.
CONTACT: VMware • 877-486-9273 • www.vmware.com

The Virtual Infrastructure Client connected immediately and was responsive, although a bit of delay sometimes occurred between status refreshes. The embedded console allowed immediate connection to the VMs and was also quite responsive. The hot-key sequence was the familiar Ctrl+Alt and Ctrl+Alt+Ins key combinations. Changing the boot source was quick and easy using the VM settings. One nice feature was the ability to connect the VM to either the local client’s CD-ROM/DVD drive or the host’s CD-ROM/DVD drive. Another nice feature of the Virtual Infrastructure Client was the activity log at the bottom of the console, which you can see in Figure 1. Finally, clicking the Performance tab showed a graphical representation of the host and guest VMs, with the option to select different sets of performance counters.

A couple of issues that I ran into with ESX Server were remotely connecting to the native console and copying VMs between servers. ESX Server doesn’t have Remote Desktop like Windows Server, and there’s no built-in File Explorer or FTP server. However, a couple of useful and free add-ins for ESX Server address these issues. For more information about these tools, see the Webexclusive sidebar “Free ESX Server Management Add-ons” (www.windowsitpro.com, InstantDoc ID 99250).

The basic VM creation and management using Hyper-V Manager was straightforward and easy. However, I found Hyper-V Manager to be less functional and less well thought out than the VMware management console. Like comparing a Toyota to a Ferrari, it seemed that Hyper-V Manager was forced to fit inside a generic Microsoft Management Console (MMC) 3.0 framework, whereas the VMware Virtual Infrastructure Client was specifically designed for VM server management. Hyper-V’s Snapshot pane in particular seemed poorly positioned. The VMware Virtual Infrastructure Client had a professional feel that Hyper-V Manager just doesn’t possess. Hyper-V Manager had minimal server performance information. Apparently in an effort to keep IT pros on their toes, the hot-key sequence inside the console was also changed from the old Virtual Server and Virtual PC standards to a new Ctrl+Alt+End and Ctrl+Alt+Left Arrow. I also ran into problems installing the Virtual Machine Integration Services (formerly called Virtual Machine Additions); the different VMs had conflicts sharing the vmguest.iso file that is used to install Integration Services.

This is certainly an early-code problem, but it caused me trouble nevertheless. Hyper-V’s new VM settings, which you can see in Figure 2, are better and more useful than Microsoft Virtual Server 2005’s settings. One very welcome new feature, which is also available in ESX Server, is Hyper-V’s ability to expand an existing virtual hard disk.

Pedal to the Metal
Performance is where the rubber meets the road. I tested the two virtualization products using an HP ProLiant ML370 G5 that is a rack-mounted 4U server. The HP ProLiant ML370 G5 can run Server 2008 with Hyper-V as well as VMware’s ESX Server. My test unit was equipped with two Intel quadcore Xeon processors running at 1.86GHz on a 1,066MHz front-side bus. The ML370 G5 came equipped with 8GB of RAM and eight 72GB 15,000rpm drives configured as a RAID array. Overall, the performance level provided by this system exceeded the test requirements, leaving headroom for additional scalability.

PROS: Very good performance; great price
CONS: Bad remote management experience; needs a better management console
RATING: 4 out of 5
PRICE: N/A (included with Server 2008)
RECOMMENDATION: Because of its low cost, Hyper-V is great for midsized businesses planning to adopt Server 2008.
CONTACT: Microsoft • 800-642-7676 • www.microsoft.com

To compare the performance of ESX Server 3.5 and Server 2008’s Hyper-V, I configured the eight x64 VMs with 512MB of RAM and accepted the default settings for new virtual hard drive configuration. I used external networking that linked the VMs’ virtual network adapters to the host, letting me test the connections from external client systems—which is how most organizations use production servers in a server consolidation environment. All the VMs were configured with Server 2008 Enterprise.

To simulate a mixed workload, I set up six of the VMs to function as file servers and two of the VMs as database servers running SQL Server 2005 Enterprise Edition SP2. To test the file server performance, I used a routine that copied a set of 10 files totaling about 130MB from the file server to the local client’s hard drive. Then the files were copied back to another directory on the server and deleted. I used a three-second think time between all the operations. This routine was repeated 10 times. To test the SQL Server workload, I used 27 different queries running against the sample AdventureWorks database. Although the bulk of the workload was data retrieval, the batch also contained a couple of loops and four SELECT INTO statements to add some computational and data modification operations. A think time of three seconds was inserted between each database interaction. The clients used SQLCMD to launch the workload against the two SQL Server systems. Each client ran a test against one of the files servers and two tests with the two SQL Server systems—for a total of 10 jobs in each test batch. The entire series was repeated three times and the results were averaged. The order in which each client ran jobs against the VM hosts was varied for each run.

The final performance measurement that I compared was the total time required to run all the workloads to completion. I compared both the total aggregate time for all workloads and the individual time for the workload run by each VM. Figure 3, shows a graph comparing the performance of the two platforms.

I measured the results in seconds down to a hundredth of a second. Perhaps surprisingly, Hyper-V consistently delivered better performance in the file server portion of the combined tests, consistently edging out ESX Server by a range of 1 percent to 5 percent depending on the VM. However, just as surprisingly, VMware’s ESX Server delivered notably better performance than Hyper-V in the SQL Server database tests. For the database performance, ESX Server averaged 36 percent to 39 percent better than Hyper-V. These results were somewhat surprising because I had recently conducted a set of SQL Server–only tests for SQL Server Magazine, in which ESX Server averaged just 4 percent better performance than Hyper- V. However, the workload was different in these tests. This time, I was performing a set of mixed file serving and database serving. Closer analysis of the results showed that one database client was slower using SQL Server under Hyper-V than ESX Server. The cause of the difference wasn’t clear. The client’s file system tests were the same for both platforms. There was no change on the client, and the results were consistent over all three test sets and both database servers. Rerunning the tests produced similar results. Discounting that client’s test results, the difference was about 1 percent in favor of ESX Server. Figure 4 shows a summary of the combined test results.

Combining the total of all workloads and all tests, ESX Server finished with a 4 percent advantage over Hyper-V. Although ESX Server held the performance advantage, the two platforms were definitely comparable even under load. Hyper-V did show an advantage in certain parts of the tests. And because my testing was done with the early release Hyper-V code, the final release might change these test results.

The Victory Lap
Overall, my editor’s choice and the checkered flag go to ESX Server. Here’s a breakdown of my take on some of the important differentiating features of each product.

  • Installation and Setup—ESX Server took this category hands down. Installation was much faster and setup was much easier. No rebooting was required. Remote management setup was dropdead simple.
  • Virtual Server Host Management— Hyper-V took this category. I used the full installation of Server 2008 Enterprise, so all the Windows Server management tools and capabilities were available, complete with a graphical interface.
  • VM Management—ESX Server was a clear winner in this category. After working with ESX Server 3.5 and the Virtual Infrastructure Client, I must admit that I really liked it. It was manageable, predictable, and polished. Hyper-V Manager seemed a bit clunky in comparison.
  • Performance—ESX Server showed a slight advantage here but Hyper-V really held its own in the performance department.
  • Price—Hyper-V held a clear advantage in this category. Most organizations will eventually move to Server 2008, which includes Hyper-V.

Both Microsoft’s Hyper-V and VMware’s ESX Server provide excellent levels of performance. For midsized and large businesses, ESX Server 3.5 coupled with the mature Virtual Infrastructure 3 management suite provides a more feature-rich—albeit more costly—platform for enterprises and midsized businesses. In addition, ESX Server has a mature eco-structure of support products that can help organizations with a variety of management operations, including backup, disaster recovery, and security. However, I found Hyper-V to be very compelling for midsized businesses—especially those organizations just getting into virtualization and those making the move to Server 2008. The fact that Hyper-V is bundled in with Server 2008 makes the price point very attractive. In addition, the product offered good performance and easy management.