Last week, we benchmarked VMware Fusion 5 and found that it offered a consistent performance increase over its predecessor, Fusion 4. Just a few days later, Parallels Desktop 8, the primary competitor to Fusion in the consumer OS X virtualization market, launched. Armed with the latest software from both companies, we can now take a look at how well Mac users can expect to run Windows in the coming year.
Both applications can virtualize a myriad of operating systems, including OS X, nearly all versions of Windows, and multiple Linux varieties. However, most Mac consumers will use the software to virtualize a recent version of Windows, so we have focused our benchmarking efforts on that configuration. If there is high demand for benchmarking other configurations, please let us know and we will do our best to accomodate requests.
Hardware & Software
Our tests were performed on a 2011 27-inch iMac at 3.4 GHz with 16 GB of RAM running OS X 10.8.1. We compared Parallels Desktop 8.0.181 with Fusion 5.0.1 and measured their ability to run the 64-bit versions of Windows 7 Professional and Windows 8 Pro.
Testing Methodology
Both virtualization applications were configured to use four of the iMac’s eight logical processors and 4 GB of RAM. The configuration options don’t completely translate between applications, but each was set up to maximize the performance of the virtual machine over that of the host OS.
The virtual machines were all stored on an external Pegasus R4 Thunderbolt RAID array with four 3 TB mechanical hard drives in a RAID 5 configuration. This was to ensure that there would be plenty of space for all the virtual machines, as well as to eliminate any potential drive or interface bottlenecks.
Except where otherwise noted, all tests were performed three times, and the results, as long as they were within 5 percent of each other, were averaged. Some tests were not possible in certain configurations, and those are noted below.
Parallels 8 provides “experimental” support for DirectX 10, Microsoft’s set of multimedia APIs, while Fusion only supports up to DirectX 9. Unsure as to exactly how Parallels was implementing DirectX 10, and curious about whether its implementation would cause performance variations outside of applications that called for DirectX 10, we ran separate tests for Parallels in both DirectX 9 and 10 configurations. We also tested Windows 8 performance in both applications, as is noted in the graphs below.
Futuremark’s PCMark benchmark attempts to evaluate overall system performance, and includes tests for computational tasks, image and video playback and manipulation, web browsing, gaming, and storage speed.
On Windows 7, Parallels in both DirectX configurations enjoys a slight performance lead over Fusion. One area where Parallels is significantly ahead of Fusion is in the Entertainment section of the benchmark, although it should be noted that this is due to Parallel’s ability to run DirectX 10 benchmarks that are included as part of the test. If we focus on Parallels in DirectX 9 mode, the green bar, we see that its performance, while still ahead of Fusion, is greatly reduced.
An anomaly that is clear from the results is Windows 8 performance on the creativity and computation tests in both Fusion and Parallels. The result, far greater than its Windows 7 counterparts, was verified in multiple tests. It even exceeds brief testing done natively in Boot Camp, and so we chalk its large result up to the way that Windows 8 is virtualized. In other words, synthetic benchmarks can sometimes provide misleading results, and users who want to upgrade their virtual machine to Windows 8 should not expect their system to suddenly make huge improvements in gaming and computational performance.
Cinebench is a multi-platform benchmarking utility that is based on Maxon’s Cinema 4D rendering software. It tests OpenGL graphics performance and multi- and single-CPU rendering capabilities.
OpenGL performance was good for all configurations, although Parallels performed about 20 percent better in the Windows 7 tests. A disadvantage for Parallels, however, was its inability to run the OpenGL test while virtualizing Windows 8. We tried multiple configurations and settings, and even created a whole new virtual machine but were unable to start the test. Because Windows 8 in Fusion was able to run the test without issue, we believe the problem may be on Parallels’ end, and should be correctable via software updates to the graphics drivers.
Windows 8 on Parallels made up for its inability to run the OpenGL test by scoring the highest on the single- and multi-core rendering tests. Parallels overall scored higher than Fusion, but by less than five percent.
Futuremark’s 3DMark06 is a DirectX 9 gaming benchmark that attempts to stress a system’s GPU and CPU the same way that a highly detailed game would. It’s a bit old at this point but it provides one of the best ways to test DirectX 9 performance, for which both Parallels and Fusion offer full support.
In terms of DirectX 9 performance, Fusion falls about eight to ten percent short of Parallels in all areas of the benchmark’s measurement. The good news, however, is that both applications performed very well, reaching over 100 frames per second at points of the benchmark. Parallels may be faster, but both are fast enough to run many older DirectX 9 titles.
Geekbench is a multi-platform tool for measuring a system’s computational and memory performance. It does not test graphical or storage capabilities, but is useful in that it scales from systems as small as an iPhone to those as large as dozen-processor workstations.
The pattern of Parallels holding a slight, but consistent lead over Fusion continues with Geekbench. However, as was mentioned above, both applications have reached near-native processing performance, and virtual performance in all categories is less than five percent off of native Geekbench results on OS X.
Boot Time
Both Parallels and Fusion have worked hard to decrease boot times, and new operating systems like Windows 8 boast boot times of under 10 seconds. To see how well the virtualization apps can handle cold booting a virtual machine, we used a stop watch and timed five reboots of each configuration. In each test, the clock started when we clicked the power button to start a virtual machine from within each application. For Windows 7, the clock stopped when all notification items had loaded in the system tray and for Windows 8 the clock stopped when the Start screen had finished loading. We then averaged the results.
As you can see above, Windows 8 has a significant advantage over Windows 7 in boot times, and Windows 8 via Parallels recorded an amazingly fast 9 second boot. Loading Windows 8 on Fusion took about five seconds longer, and Windows 7 on Fusion took the longest time, 22 seconds.
Readers should note that all operating systems were fresh installs with non-essential software disabled. Virtual machines that are in actual productive use will likely have software configured to launch at startup and will necessarily experience longer boot times. Further, the speed of the drive on which the virtual machine is stored also plays an important role in boot times, and those using virtual machines on a single mechanical drive will have to wait a few seconds longer to jump into Windows.
Gaming: Crysis
Once the ultimate measure of a gaming PC’s performance, Crysis can now be played at sufficient frame rates from within a virtual machine. Using the Medium and High settings, we tested three common resolutions, with each test performed three times.
Performance is remarkably similar between platforms and versions of Windows, with the exception of Parallels in the DirectX 9 configuration on High settings. The benchmark was run exclusively in DirectX 9 mode, so Parallel’s reduced performance while in a DirectX 10 configuration underscores Parallels’ classification of the feature as “experimental,” and gamers looking for the best performance from their DirectX 9 titles should keep the software configured accordingly.
Conclusions
From a purely performance-based perspective, Parallels has won this year’s competition of virtualization software on OS X. The good news for consumers, however, is that Fusion is not far behind in most categories and, while Parallels is indeed faster, Fusion is often fast enough for common tasks.
Mac users who need to access relatively lightweight productivity applications or test software on Windows will be well suited by either Parallels or Fusion, although gamers who need the absolute best performance will be better off with Parallels.
Regardless, this latest round of benchmarking shows just how far virtualization capabilities have come in the six years since Parallels was first released. Performance in many areas is just a few points shy of native capabilities and the outlook gets better every year.
Consumers may not be happy with the yearly paid update cycles, but the fierce competition between Parallels and Fusion has led the market to a mature and capable state and consumers of both products will likely be satisfied with their performance.
Are there any additional benchmarks that you’d like to see? Let us know in the comments and we’ll do our best to get the results to you as quickly as possible.