Virtualization Rematch

Executive Summary: Since our head-to-to-head comparison of Microsoft’s new Hyper-V virtualization platform and VMware’s market-leading ESX Server, a lot has changed in the virtualization market. It’s time to retest these products’ management and performance aspects.

Earlier this year, Windows IT Pro published my head-tohead comparison of Microsoft’s Hyper-V virtualization platform and VMware’s market-leading ESX Server. In that two-part series, I compared the feature sets, licensing, and performance of both products. I found ESX Server to have a notably better installation and management story, as well as a slight performance edge. However, for a pre-release product, Hyper-V fared well and proved itself to be a viable virtualization platform.

Since that first round of reviews, much has changed in the virtualization market. First, the RTM version of Hyper-V is now available: Microsoft has made its final performance enhancements to the product. Second, Microsoft has released a standalone version of Hyper-V called Hyper-V Server 2008. For more information about this incarnation, see the web-exclusive sidebar “The Standalone Hyper-V Server 2008” (www.windowsitpro.com, InstantDoc ID 100574). Third, both companies have altered the licensing for their respective products. Hyper-V Server 2008 and VMware ESXi are free downloads. For more information about VMware ESXi, see the webexclusive sidebar “ESXi vs. ESX Server” (InstantDoc ID 100575).

Considering these changes, I’ve decided to retest these products’ management and performance aspects, as well as address some particular concerns about each product that readers—and Microsoft and VMware representatives—have brought up since my first tests. Now, let’s jump back into the ring with ESX Server and Hyper-V.

Hypervisor Differentiation
Both ESX Server and Hyper-V are hypervisor-based, but not all hypervisors are created equal. The architectures of these products differ significantly. And many IT pros have been confused by Hyper-V, mistakenly assuming that because it’s shipped with Windows Server 2008, it’s a hosted virtualization product that runs on top of the Server 2008 OS. That’s not the case. Like ESX Server, hypervisor-based Hyper-V runs directly on the system hardware.

Figure 1 provides an architecture comparison. As you can see, one of the biggest differences between the products is the way each handles hardware device drivers. ESX Server implements the drivers as a part of the hypervisor itself—a method that results in a comparatively large hypervisor. This approach also adds third-party code to the hypervisor. VMware tests and certifies these drivers, but they’re developed by system hardware vendors. (For a list of systems that support ESX Server 3.5 and ESXi, see the Learning Path.) Hyper-V implements the drivers in the parent partition, outside the hypervisor. Table 1 provides the pros and cons of each approach.

The implementations of the hypervisor itself also differ. The ESX Server hypervisor uses a 32-bit kernel, allowing it to run on both 32-bit and 64-bit systems. However, that doesn’t limit it to running only 32-bit guests; ESX Server also supports 64-bit guests if it’s running on a 64-bit hardware platform. With its next ESX Server release, VMware plans to move to a 64-bit hypervisor. By contrast, Hyper-V already uses a 64-bit hypervisor, which promises improved performance and scalability. Also, Hyper-V requires that the system you install it on possess processor-assisted virtualization (e.g., AMD processors that support AMD-V, Intel processors that support Intel-VT). Hyper-V requires that the processor have either AMD’s No Execute (NX) or Intel’s Execute Disable (XD) features, and the system needs to offer BIOS support for virtualization. These features are standard in most of today’s server systems, but they aren’t in all systems.

Guest Support
To some extent, the differences between the products’ hypervisor implementations are academic. Both products have proven to be good performers and scale well with multiple workloads. However, the difference in guest OS support is much clearer. In this respect, ESX Server is a much more mature product: VMware supports a wide array of guest OSs. Web Table 1 provides a list of guest OSs that ESX Server supports. (For a complete list of the guest OSs that the product supports, see the Learning Path.)

As you might expect, the list of guest OSs that Hyper-V supports includes all the recent Microsoft OSs but few others. Web Table 2 provides a list of guest OSs that Hyper-V supports. (For a complete list of the guest OSs that the product supports, see the Learning Path.) The list of OSs that Hyper-V supports is dominated by Microsoft, with the exception of SUSE Linux—but this Linux implementation is limited to a single virtual CPU, far short of the Linux support that ESX Server offers. Microsoft marketing states that Hyper-V runs other OSs, such as multiple Linux distributions, but actually Hyper-V doesn’t support any distribution other than SUSE, for which Microsoft has an agreement with Novell. Microsoft has made the code for the Linux Hyper-V integration components available but has left its adoption to other vendors—a significant development because the VMBus-aware drivers that provide the best Hyper-V performance are installed as a part of the integration components. Without them, the guest must run in slower legacy-emulation mode. Currently, no integration components are available for other Linux implementations, but you can run other Linux distributions as unsupported legacy guests.

Built-In Management
This review is focused solely on the virtualization platforms themselves, and I won’t touch on the management suites that either vendor provides as separate products. The distinction can be confusing: Many VMware-supporting readers have opined that VMware’s VMotion is the single biggest difference between the products; however, although VMotion is an important feature, it’s not a part of ESX Server but rather a component of the VMware Infrastructure 3 (VI3) management suite. A forthcoming Windows IT Pro article will compare VI3 and Microsoft’s management suite, System Center Virtual Machine Manager (SCVMM). Let’s take a look at the products’ inherent management functionality.

ESX Server. You use the Virtual Infrastructure Client to manage ESX Server. To download the client to your local system, you simply point your web browser to your ESX Server system, then click the Download VMware Infrastruture Client link. The entire process takes a couple minutes. The Virtual Infrastructure Client offers a fullfeatured, functional interface for managing multiple VMware virtual machines (VMs) for one ESX Server host. You can create and control VMs, and you can control a number of host settings, such as the configuration of virtual switches, the host time, the DNS server, and VMs’ automatic start and stop actions. Also, you can use the Virtual Infrastructure Client to set up users and groups, along with their associated permissions.

The most noticeable missing feature is the ability to easily copy VMs among hosts. There’s no built-in Windows Explorer, and no connections to remote hosts. However, free third-party tools such as Veeam and WinSCP can fill this gap. One of the best features of the client is its ability to track performance data at both the host and the VM level. It provides a storage summary, as well as CPU, memory, network, and disk usage. Figure 2 shows the Virtual Infrastructure Client’s Performance tab.

Although the Virtual Infrastructure Client provides a good management interface in the absence of the VI3 management suite, it’s limited. For example, it doesn’t provide the ability to import and convert VMs, as the other VMware virtualization products do. And it doesn’t let you copy or clone VMs. These options are present only if VI3 and VirtualCenter Server are available. Finally, I’ve found that I often need to drop back into the Linux management console to perform many tasks. For example, if I copy a VM to ESX Server, I don’t get a graphical option to register the VM—I need to use the Vmwarecmd command.

Hyper-V. In the arena of management, Hyper-V stumbles. Management for Hyper-V with a full Server 2008 installation is a good experience: When you install the Hyper-V role, the Hyper-V Manager is present and you can use it from the full Server 2008 installation to manage Hyper-V. Such is not the case for the Server Core version. Server Core has no built-in GUI and requires remote management. However, unlike ESX Server, the remote client is a separate download, and I had difficulty getting it connected. I used Server 2008 and a Vista client. I first tried it in a workgroup, then in a domain. Although it eventually worked, it wasn’t a good experience— certainly not on par with the easy Virtual Infrastructure Client installation and connection. During my first round of testing, I attributed the difficulty to Hyper-V’s prerelease code. Unfortunately, I was dismayed to find that the problem remains unresolved in the final release version.

Continue on Page 2

The core of the problem seems to be that the Hyper-V Manager doesn’t provide a mechanism for passing authentication information to the Hyper-V host. This omission requires you to embark on a painstaking multistep manual process to configure the client—and the server—that you want to use. You have to repeat the process for all the clients that you want to use to remotely manage Hyper-V. Adding insult to injury, the steps aren’t documented with the product; you need to search Microsoft’s blogs to find them. This situation represents a big hurdle to running Hyper-V on Server Core—particularly for SMBs looking to get started with virtualization. Until this problem is resolved, if you want to run Hyper-V, I’d go with the full Hyper-V and Server 2008 installation.

Personally, I’m surprised Microsoft didn’t do a better job with this aspect. After all, running Hyper-V on Server Core lets you have less overhead and a more secure implementation. Plus, VMware has already shown how to do it correctly. All that being said, using the full Server 2008 installation has little effect on performance but makes the management of Hyper-V much easier.

The Hyper-V Manager provides a basic management interface that lets you manage a VM on one or more Hyper-V servers. You can create VMs and control them, create VLANs through the new virtual switching feature, set up automatic VM start and stop attributes, and set VM resource allocations. The Hyper-V Manager is functional but doesn’t provide any of the advanced features (e.g., performance monitoring) that the Virtual Infrastructure Client provides. Figure 3 shows the Hyper-V Manager’s Resource Allocation dialog box. Web Table 3 provides a summary of the management features that each product provides.

Performance Testing
I ran two sets of tests on an HP ProLiant ML370 G4, with two Intel quad-core Xeon processors running at 1.86GHz on a 1066MHz frontside bus. The system comes with 8GB of RAM and eight 72GB 15,000rpm drives configured as a RAID array. My tests in the previous articles were based on timed Windows Shell scripts. For this set of follow-up tests, I converted my test scripts to PowerShell, which enabled better program control as well as the ability to use ADO.NET as my SQL Server data-access mechanism.

Hyper-V PROS: Very good performance; great value CONS: Poor remote-management experience; needs a better management console RATING: 4 out of 5 RECOMMENDATION: Recommended for small businesses planning to adopt Windows Server 2008. CONTACT: Microsoft • www.microsoft.com

The first set. First, I repeated the set of tests that I ran in the original articles. During those first tests, Hyper-V was in a pre-release state. For the final version, Microsoft has added some performance tweaks to the release code. To simulate a production server– consolidation scenario, I set up eight VMs on the host (each configured with 512MB of RAM) and I used the default settings for new virtual hard drive configuration. I used external networking, which linked the VMs’ virtual network adaptors to the host. For this first round of tests, all the VMs were configured with the 64-bit Server 2008 Enterprise Edition. For the Hyper-V portion of the tests, the integration components were loaded onto all the guests. And yes, the Hyper-V VMs were all using the high-performance VMBus device drivers. For the ESX Server tests, the VMware Tools were installed.

To create a mixed workload, I configured six of the VMs to function as file servers and two as database servers running Microsoft SQL Server 2005 Enterprise Edition SP2. To test the file-server performance, I used a routine that copied a set of 10 files (totaling about 130MB) from the file server to the local client’s hard disk. Then, the routine copied the files back to another directory on the server and deleted them. I used a three-second think time between all the operations. This routine repeated 20 times.

To test the SQL Server workload, I used 27 queries running against the sample AdventureWorks database. Although the bulk of the workload was data retrieval, the batch also contained a couple CPUintensive queries, a 5,000-row insert function, and four SELECT INTO statements to add some data-modification operations. I inserted a three-second think time between each database interaction.

As you can see in Figure 4, ESX Server and Hyper-V provided similar performance under these test conditions. The bars in the graph indicate the total average time to complete the test suite. ESX Server demonstrated a 4 percent edge over Hyper- V in the test’s file-server portion. However, Hyper-V beat ESX Server in the database testing by 1 percent. The combined results were the totals for both the file-server and database tests. This set of tests ran for about 20 minutes. Overall, ESX Server won the combined results by Figure 3: Hyper-V Manager providing 3 percent better performance than Hyper-V. Although ESX Server finished the test suites faster than Hyper-V, the 3 percent difference is small.

The second set. One reader comment about my first round of testing was that Server 2008 is optimized to run under Hyper-V and that other OSs might not deliver the same levels of performance. Microsoft confirmed this optimization and explained that certain guest OSs (e.g., Server 2008) are considered enlightened. Further, there are two types of guest enlightenment. The first is a basic level of driver enlightenment, which means that the guest OS can take advantage of Hyper-V’s high-performance VMBus architecture. Vista and Server 2008 possess a second level of enlightenment called kernel enlightenment. Kernel enlightenments improve processor and memory performance to further optimize the guest OS for running in a VM. For more information about the Hyper-V architecture, see the Learning Path.

To determine whether Server 2008 offered any advantages while running under Hyper-V, I re-ran a second set of tests, following the same pattern as the tests in the first set. However, the second test set used 32-bit Windows Server 2003 Enterprise Edition SP2 as the guest OS for all the VMs. Again, for Hyper- V, the integration components were loaded and the VMs were using the VMBus drivers. For ESX Server, the VMware Tools were installed.

As you can see in Figure 5, the results were even closer than the first set of tests. In a surprise turnabout, Hyper-V posted a 1 percent edge over ESX Server in the fileserver portion of the tests, whereas ESX Server posted a 2 percent advantage over Hyper-V in the database tests. Overall, ESX Server held a slight 1 percent edge in the combined performance results. Considering that the results with Windows 2003 were even closer than the results with Server 2008, it’s fair to conclude that under these test conditions, Server 2008 showed no significant performance benefit by running under Hyper-V as opposed to ESX Server.

At this scalability level, ESX Server had a slight lead in both the 64-bit and 32-bit tests, but it’s clear that both virtualization platforms deliver close levels of performance. That said, ESX Server’s support for larger system configurations enable it to have greater overall scalability than Hyper-V.

Continue on Page 3

Virtual Reality Check
Both products deliver excellent virtualization performance, but Hyper-V is hamstrung by substandard remote management and limited support for non-Microsoft guest OSs. VMware’s superior remote management and broader guest support characterize the more mature ESX Server.

At this point, for midsized-to-large business and enterprises, the more manageable ESX Server is the better choice, particularly if you want to support a mix of Windows and Linux guests. Remote management for Hyper-V is still too problematic. However, Hyper-V is a good choice for smaller businesses running Server 2008 that primarily want to virtualize Windows servers. The product’s inclusion with Windows makes it simpler to use and adopt: You don’t need to learn the unfamiliar commands necessary to deal with ESX Server’s Linux-based management console. However, because of the aforementioned remote-management difficulties, I can’t recommend Hyper-V on Server Core at this time. That being said, running Hyper-V on a full Server 2008 installation works well.

ESX Server 3.5 PROS: Excellent performance; easy installation; polished management console CONS: Somewhat limited hardware support RATING: 5 out of 5 RECOMMENDATION: Recommended for midsized-to-large businesses looking for performance and manageability. CONTACT: VMware • 877-486-9273 • www.vmware.com

Virtualization is fast becoming an important business commodity, with both Microsoft and VMware essentially providing free virtualization products. However, raw virtualization is only half the story. The other half is management—which is where both vendors are looking to make their money. VMware’s VI3 management suite has a big head start in this area, but Microsoft’s SCVMM, with its ties to the System Center family of products, offers unique advantages. An upcoming issue of Windows IT Pro will compare Microsoft’s and VMware’s virtualization-management suites.