(Updated to include links to results)
I’ve had a few questions lately about “the best” blade server to use for virtualization – specifically VMware virtualization. While the obvious answer is “it depends”, I thought it would be an interesting approach to identify the blade servers that ranked in the top 5 in VMware’s VMmark benchmark. Before I begin, let me explain what the VMmark testing is about. VMmark enables equipment manufacturers, software vendors, system integrators and other organizations to:
- Measure virtual machine performance accurately and reliably
- Determine the performance of different hardware and virtualization platforms
- Make appropriate hardware decisions for your virtual infrastructure
VMware developed VMmark as a standard methodology for comparing virtualized systems. According to VMware’s VMmark website, the benchmark system in VMmark is comprised of a series of “sub-tests” that are derived from commonly used load-generation tools, as well as from benchmarks developed by the Standard Performance Evaluation Corporation (SPEC®). In parallel to VMmark, VMware is a member of the SPEC Virtualization subcommittee and is working with other SPEC members to create the next generation virtualization benchmark.
In testing the terms, a “tile” is simply a collection of virtual machines (VM’s) that are executing a set of diverse workloads designed to represent a natural work environment. The total number of tiles that a server can handle provides a detailed measurement of that server’s consolidation capacity. The more tiles, the better. The faster the performance, the better.
THE RESULTS (as of 6/2/2010)…click on the link to open the details
24 Cores (4 Sockets)
HP ProLiant BL685c G6 (Six Core AMD Opteron 8435) running VMware ESX v4.0 – 29.19@20 tiles (published 7/14/2009)
HP ProLiant BL680c G5 (Six Core Intel Xeon E7458) running VMware ESX v3.5.0 Update 3 – 18.64@14 tiles (published 3/30/2009)
16 Cores (4 Sockets)
Dell PowerEdge M905 (Four Core AMD Opteron 8393 SE) running VMware ESX v4.0 – 22.90@17 tiles (published 6/19/2009)
HP ProLiant BL685 G6 (Four Core AMD Opteron 8389) running VMware ESX v4.0 – 20.87@14 tiles (published 4/24/2009)
12 Cores (2 Sockets)
Cisco UCS B250 M2 (Six Core Intel Xeon X5680) running VMware ESX v4.0 Update 1 – 35.83@26 tiles (published 4/6/2010)
Fujitsu BX922 S2 (Six Core Intel Xeon X5680) running VMware ESX v4.0 Update 1 – 32.89@24 tiles (published 4/6/2010)
8 Cores (2 Sockets)
Fujitsu BX922 S2 (Four Core Intel Xeon X5677) running VMware ESX v4.0 Update 1 – 27.99@18tiles(published 5/10/2010)
HP ProLiant BL490c G6 (Four Core Intel Xeon X5570) runningVMware ESX v4.0 – 25.27@17tiles (published 4/20/2010)
THE WINNER IS…
Cisco UCS B250 M2 running VMware ESX v4.0 Update 1 – 35.83 with 26 tiles
Cisco’s Winning Configuration
So – how did Cisco reach the top server spot? Here’s the configuration:
server config:
- 2 x Intel Xeon X5680 Processors
- 192GB of RAM (48 x 4GB)
- 1 x Converged Network Adapter (Cisco UCS VIC M81KR)
storage config:
- EMC CX4-240
- Cisco MDS 9134
- 1173.48GB Used Disk Space
- 1024MB Array Cache
- 50 disks used on 5 enclosures/shelves (1 with 14 disk, 4 with 9 disks)
- 55 LUNs used
*21 at 38GB (file server + mail server) over 20 x 73GB SSDs
*5 at 38GB (file server + mail server) over 20 x 73GB SSDs
*21 at 15GB (database) + 2 LUNs at 400GB (Standby, Webserver, Javaserver) over 16 x 450GB 15k disks
*5 at 15GB (database) over 16 x 450GB 15k disks
* 1 LUN at 20GB (boot) over 5 x 300GB 15k disks - RAID 0 for VMs, RAID 5 for VMware ESX 4.0 O/S
As you can see from the information above, the Cisco UCS B250 M2 is the clear winner above all of the blade server offerings. As you can see, none of the Xeon 7500 blade servers have yet to be tested but when they do, I’ll be sure to let you know.
It's strange comparing. In some pairs they use different versions of VMware ESX. And why all benchmarks between two servers? And what about storage? Does all servers worked with identical storage systems?
Pingback: Tweets that mention Blades Made Simple » Blog Archive » Best Blade Server for VMware Is… -- Topsy.com
How does AMD stand against Intel in this test?
My question is if AMD could be a good choice when you looking at Price/Performance? What would you recommend, is AMD a choice at all?
#AMD doesn't compare to #Intel on the #VMware benchmarks. I've updated the blog post to show which of the servers are AMD and which are Intel to make it easier to understand. While there are more servers with AMD CPUs, the Intel Xeon 5600 CPU achieves the highest marks and even exceeds the 24 core AMD performance. It will be interesting to see how the Intel Xeon 7500 fares, but we won't have that info for a few more weeks (if not months.) Thanks for reading, and thanks for the comments!
It appears on the #vmware VMmark tests there are different versions of ESX tested because they were done at different times. One result shows 3.5 the other shows ESX 4.0 (once it was released.) Regarding your question about the storage – I've updated the post to include the links to the results so you can go through and compare the storage used on each of these. The good thing about the results is they go into details about how the storage was set up (# of drives / LUNs, etc.) Thanks for reading, and thanks for the comment!
Don't know what set of results you were looking at, but Cisco has a 4-socket Xeon 7500 result published, hitting 73.82 @ 50 tiles on a Cisco UCS C460 M1 (see the top of the 32-core results section).
Sm@sher, your question is spot on. Cisco uses SSDs in the I/O intensive portions for all their VMmark tests. No one would use SSDs for mail and file servers in the real world, and Cisco is the only vendor that does.
20 73GB SSDs = ~$220,000.00
20 72GB SAS = ~$ 5,000.00
VMware should implement a $/VMmark to make this type of game playing more apparent.
The #Cisco UCS C460 M1 is a RACK server. I covered the results for blade servers only. Thanks for comment, and for reading.
Great observation that #cisco used Solid State Drives to achieve higher #VMmark scores, however HP, IBM and Dell have the option to do this as well. Wonder why they don't. Would love to see the comparison. Thanks for the comment and for reading!
bladeguy-
“No one would use SSD's” for “I/O intensive” workloads? Huh? Are you sure about that? I would think a customer would want know what a system is capable of if they chose to use SSD's for their I/O intensive workloads, and many customers are doing just that.
Last time I checked, VMmark was a Performance Benchmark test, No?
Ok, it's up for customer or integrator what kind of disks to use. But it's not “fair” to compare VMware performance on servers with different types of disks.
The #vmware vmmark “test” is just another way to gauge server capabilities. As with all benchmarks, there will be ways to design the outcomes so it favors your product. I give Cisco credit for their creativity. All of the vendors have the option to design a solution that will give them the edge, but the question is, does it really matter?
As it has been pointed out several times in the comments on this post and previous posts – there is no “one size fits all” when it comes to designing virtual solutions. One customer may need the cheapest solution while another may need the solution that will give them the best performance (regardless of the cost.) With that in mind, isn't going to be a right way. In some situations Cisco will be a better choice. In others, HP, IBM or Dell may be better choices. The point of this post was to highlight the often overlooked VMmark benchmark and to spark some discussion. Looks like it's working ;) Thanks for your continued support of this blog. I really appreciate your thoughts, feedback and comments.
Not exactly true, Kevin. Results are all over the board with AMD and Intel systems. to be fair, benchmarking a 6000-series Opteron system would be most appropriate vs. the Westmere products. ;)
Pingback: Kevin Houston
Pingback: M. Sean McGee
Pingback: jeremywaldrop
Pingback: Kevin Houston
Just curious where the IBM System X blades are in this comparison? In particular, the IBM HX5 blade that was released.
Pingback: Kevin Goodman
#VMware 's VMmark scores do not yet include any scores from the #IBM HX5 server. In fact, I haven't seen any blade server scores using the Intel Xeon 6500 / 7500 CPUs but I'm sure they'll be coming soon. Thanks for the comment and for reading! I appreciate your support.
Pingback: Cisco España
Pingback: Robert Guzman
lol..Interesting that you didnt test any of the new Dell blade servers including M610, M710 or the latest M910. Using the M905 which leverages very old AMD technology is very misleading when you compare this to the Cisco Westmere and Fujitsu Westmere processors with the QPI and NUMA based architecture. Crappy test to be honest. Antony Collins.
Not at the moment…Westmere and Nehalem EX have the lead on AMD currently…will be interesting to see magny cour though..Antony Collins.
To further clarify….understand these results are VMware VMmark tests but it’s quite misleading espousing a clear winner when all tests aren’t equivalent in technology use. There needs to be a clear baseline as otherwise it makes this article very irrelevant when you dig into the detail…
Actually #DELL did not test any new Blade servers with the #VMware VMmark test. I didn't do any testing. I simply posted the results from the VMware VMmark website. Typically each vendor does their own testing, so I encourage you to go to your brand managers and see if they'll do some VMmark testing on the new products and get the results posted. I appreciate your comment.
My article was to show what the “best blade server for #VMware” would be based on the VMware VMmark scores. Each vendor has the ability to use whatever hardware they want to help them achieve the top scores. That being said, I do agree with you and several others on the comments, that there needs to be a more standardized hardware requirement to help insure the scores are a reflection of the servers' abilities and not the results of who came up with the most ingenious way to get top results. Great point! Thanks for reading!
The opening sentence reads, The 8 core R710 and BL685 G6 outperformed all the 16 core VMmark results. That’s half the sockets and half the cores for the same amount of tiles (as the highest 16 core result) for a slightly higher VMmark score..example.
Dell Dell PowerEdge R710
VMware ESX v4.0 VMmark v1.1.1
24.27@17tiles
View Disclosure 2 sockets
8 total cores
16 total threads 09/08/09
My point is there needs to be a little more clarity up front. On face value, you could determine that out of all platforms suitable for virtualisation, the Cisco UCS platform provides the best performance due to its higher VMmark score and tile count. Reality is, devil is in the detail. It makes this article relevant specific to the actual test and the vendor equipment used at a point in time…but this cant be easily translated or have much specificity in the real world in terms of real world platform virtualisation capability – like any benchmark – this is just an indication.
Good job with the site though. Good to see some points / counter points being raised.
Kevin – agree…standards would help here. Believe they are in train via SPEC??
The 6000 series would be positioned toward Nehalem EX AND Westmere. Its a single platform capable of 2 socket (DP) and 4 socket (MP) scalable processors….
Agree with Kevin – since Intel threw out north bridge they've wreaked havoc on AMD since they released back March 09. AMD will catch up once the benchmarks are readily available.
The #Dell R710 you mentioned is a rack server. My blog post discussed the blade servers that were tested in the #vmware VMmark test. The Cisco server achieved more tiles with fewer cores than the HP BL685 G6. I appreciate the feedback, though and for the comments. Always appreciated.
Yep, same architecture but get your point..not enough blades thrown into the VMmark tests. Interesting IBM hasnt yet tested their HX5 blade where they tout better memory scalability and 1066Mhz speeds regardless of processor.
Its a useless comparison until Nehalem-EX numbers are posted…which will blow the crap out of all of the above.
Sorry for the shameless plug, but the the boots we've developed have really taken off and we couldn't be more proud of the feedback we've had from our customers.
Andrew
Pingback: Kevin Houston
Pingback: unix player
Pingback: Kevin Houston
Pingback: Kevin Houston