I once was a huge fan of HP’s Virtual Connect Flex-10 10Gb Ethernet Modules but with the new enhancements to VMware vSphere 5, I don’t think I would recommend for virtual environments anymore. The ability to divide the two onboard network cards up to 8 NICS was a great feature and still is, if you have to do physical deployments of servers. I do realize that there is the HP Virtual Connect FlexFabric 10Gb/24-port Module but I live in the land of iSCSI and NFS so that is off the table for me.

With vSphere 5.0, VMware improved on its VMware’s Virtual Distributed Switch (VDS) functionality and overall networking ability, so now it’s time to recoup some of that money on the hardware side. The way I see is most people with a chassis full of blade servers probably already have VMware Enterprise Plus licenses, so they are already entitled to VDS, however what you may not have known is that customers with VMware View Premier licenses are also entitled to use VDS. Some of the newest features found in VMware VDS 5 are:

The last feature is the one that makes me think I don’t need to use HP’s Flex-10 anymore. Network I/O control (NIOC) allows you to assign shares to your network interfaces set priority, limits control congestions all in a dynamic fashion. What I particularly like about NIOC as compared to Flex-10 is the wasted bandwidth with hard limits. In the VDI world, the workload becomes very bursty. One example can be seen when using vMotion. When I’m performing maintenance work in a virtual environment I think it sure would be nice to have more than 2 GB/s a link to move the desktops off – however when you have to move 50+ desktops per blade you have to sit there and wait awhile. Of course, when this is your design, you wait because you wouldn’t want to suffer performance problems during the day by lack of bandwidth on services.

A typical Flex-10 configuration may break down the on board nic (LOM) something like this

To get a similar setup with NIOC it may look something like this.

Total shares from above would be: 5 + 50 + 40 + 20 = 115

In this example FT, iSCSI and Replication don’t have to be counted as they will not be used. The shares only kick if there is contention. The shares are also only applied if the traffic type exists on the link. I think it would best practice to limit vMotion traffic as multiple vMotions kicking off could easily exceed the bandwidth. I think 8000 Mbps would be reasonable limit with this sort of setup.

Management: 5 shares; (5/115) X 10 Gb = 434.78 Mbps

NFS: 50 shares; (50/115) X 10 Gb = 4347.83Mbps

Virtual Machine: 40 shares; (40/115) * 10 Gb = 3478.26Mbps

vMotion: 20 shares; (20/115) X 10 Gb = 1739.13Mbps

I think the benefits plus the cost saving is worth moving ahead with a 10GB design with NIOC. Below are some list prices taken on November 28, 2011. Which one are you going to choose?

Flex-10

HP_6120G-XG

Dwayne is the newest Contributor to BladesMadeSimple.com and is the author of IT Blood Pressure (http://itbloodpressure.com/) where he provides tips on Virtual Desktops and gives advice on best practices in the IT industry with a particular focus in Healthcare.  In his day job, Dwayne is an Infrastructure Specialist in the Healthcare and Energy Sector in Western Canada.

I always find it interesting to hear how customers are using blade server technology “in the real world.”   Brickworks, a $1.7 billion Australian brick company with over 60 locations and 1,000 employees, recently provided The Australian with details about how Citrix XenApp Enterprise combined with VMware vSphere and Dell rack and blade servers helped develop a more efficient workplace for its employees.  Read the full article at:

http://www.theaustralian.com.au/australian-it/brickworks-builds-solid-system-using-citrix/story-e6frgakx-1226129943367

Kevin Houston is the founder of BladesMadeSimple.com.  He has over 14 plus years of experience in the x86 server marketplace.  Since 1997 Kevin has worked at several resellers in the Atlanta area, and has a vast array of competitive x86 server knowledge and certifications as well as an in-depth understanding of VMware and Citrix virtualization.    Kevin works for Dell as a Server Sales Engineer covering the Global 500 market. 

#IWorkForDell

Since my last post, I heard lots of rumors that VMware would make some changes in their new licensing model. Well, this week VMware made an official announcement of changes and posted here.

VMware made very clear that all the changes were motivated by the feedbacks they have received:

“These changes generated debate in the blogosphere, across the VMware communities, and in conversations with customers and partners.  Some of the discussion had to do with confusion around the changes.  We have been watching the commentaries on the blogs very carefully, and we have been listening in the customer conversations very intently. We got a ton of feedback that probed the impact of the new licensing model on every possible use case and scenario, and equally important, reflected our customers’ intense passion for VMware.” Said the Product management VP, Bogomil Balkansky.

This shows how important feedback is! We can say that changes come from us.

So, let´s see the new comparison of vSphere 4.x licensing vs the vSphere 5 model:

To see the original document, click here.

The main changes:

• Increased vRAM entitlements for all vSphere editions, including the doubling of the entitlements for vSphere Enterprise and Enterprise Plus.
• Capped the amount of vRAM  in any given VM by 96GB.
• Compliance policies will not be measured by high watermark. Instead, a 12 months rolling average of daily high watermark. That´s very important for testing and developing environments.

Plus: An official monitoring tool will replace all the scripts that have been circulating around the community to calculate how much “vRAM” customers are using now.

VMware confirmed that the new licensing model was made to keep up with the market changes and that changes are unlike, but necessary.

So, to compare apples to apples, let’s use the same example of last post: an IBM HX5 blade server with MAX5 maxed out with 640GB of memory with the NEW (I mean the newest) licensing model a user would need 7 VMware vSphere 5 Enterprise Plus Licenses to be “legal” vs 14 of the previous vSphere 5 licensing vs 2 with the current VMware vSphere 4.x licensing model.

That´s more reasonable. Especially if keep in mind that customers would not use all the physical RAM in the vRAM pool. They must have some free physical RAM to be able to do a vMotion for example.

We have to give VMware some credit to change so quickly and listen the community. Would any other vendor do something like this? I don´t know.

About the Author
Thales Osterne is a contributor for BladesMadeSimple.com.  He has over six years in the IT field with four years of experience in IBM BladeCenter and System X.  When he is not blogging, Thales works as a product manager for IBM System X & Bladecenter at Lanlink Informatica, a major business partner in Brazil.  He is fluent in Portuguese and English.

As you no doubt heard by now, VMware has announced a new version of vSphere along with some new or improved features however, this post will not highlight these features.  In this post, I want to talk about what did not improve – the licensing. 
 
With the announcement of the VMware vSphere 5, the biggest thing that startles me is the way that VMware has found to a way to collect more revenue from users with new server technologies. This seems to be totally going against the flow of x86 market since the main message of the major server manufacturers (in the past) has been how they can save users $$ on VMware licensing by offering the ability to stack your servers full of memory.  

For example – take a look at the IBM HX5 blade server combined with the MAX 5 which provides users with a total of 40 memory DIMM slots and a current maximum of 640GB using only two processors.  Taking a look at the comparison, with the previous vSphere 4.x licensing, you would only need two VMware vSphere 4.x Enterprise Plus licenses. (With 640GB of RAM, one can only imagine how many virtual machines were possible with just 2 licenses.)
 
Since more users were moving to at the architecture of larger 2 socket systems with A LOT of memory, I believe that VMware realized the vSphere licensing  opportunities were than they expected so they decided to change their licensing method  to  a “per CPU and memory” licensing model creating the concept of a processor and vRAM pooling of entitlements.
 
Here is a look at a comparison of vSphere 4.x licensing vs the vSphere 5 model:

 
So, if we use the example of a the IBM HX5 blade with MAX5 mentioned above maxed out with 640GB of memory with the NEW licensing model a user would need 14 VMware vSphere 5 Enterprise Plus Licenses to be “legal” vs 2 with the current VMware vSphere 4.x licensing model. 
 
According to VMware, the good news is that there is “no limit to the number of cores per processor”, but in reality  – I don’t see that that as a big deal. vSphere 4.X already supported 12 cores per processor with vSphere Enterprise Plus licenses which is the maximum of the current market  – even the next generation Intel 2 socket CPU will only be 8 cores – so there is no advantage to changing the licensing.  This is why many people are calling this new licensing model “vRAM Tax“.
 
We all agree that VMware has the best products and is the market leader for virtualization, BUT this new licensing model is giving ammo to Microsoft and Citrix to strike at the cost issue (which was already an issue in VS 4.x).

About the Author
Thales Osterne is a contributor for BladesMadeSimple.com.  He has over six years in the IT field with four years of experience in IBM BladeCenter and System X.  When he is not blogging, Thales works as a product manager for IBM System X & Bladecenter at Lanlink Informatica, a major business partner in Brazil.  He is fluent in Portugese and English.

Over the past couple of weeks, I’ve been researching what are the “ideal” configurations for a home VMware vSphere lab.  Ideally, it would be small enough to run a few virtual machines and demonstrate high availability features like vMotion, DRS, HA or even FT.  The problem that I’ve run into is that I haven’t found a really good site that has a detailed list of parts and pieces to buy.  The problem that I’ve run into is that some people recommend rack servers, others recommend you use tower servers and all of the designs includes external switching, connectivity to storage.  As you can imagine, this leads to a mess. 

Sure, it’s probably okay to have a pile of cables, switches and servers generating noise and heat in your home office, but what if you didn’t have to?  One of the big value propositions that blade servers offer is the ability to reduce your cable, management and power complexity – so what if there was a blade infrastructure that would be cost efficient for a home lab and was robust enough to server your virtual needs.  I’m working on finding a blade solution to be able to write about that will solve your lab needs, but this is where I need your help.  I need to know what your ideal lab environment would look like.  Take a few minutes and help me out by answering the following questions (either via comments below, or email me at kevin AT bladesmadesimple.com):

• What’s the MINIMUM amount of servers would you need for your lab?
• What’s the MINIMUM memory you’d want for your lab?
• What’s the MINIMUM storage you’d need for your lab?
• What’s the MAXIMUM price you’d pay for a bundled blade solution (servers/storage/networking)?

Thanks for taking the time to answer – feel free to comment about what your needs would be for a Citrix or Microsoft lab as well (no bias here.)

As VMworld 2010 is right around the corner, I wanted to take a few minutes to make a plea to all attendees. 

This year, IF you receive a bag or backpack that you just don’t want, please don’t throw it away, but instead take it home, go to the dollar store and fill  the backpack with pencils, crayons, paper and erasers and donate it to your local school system.  You would be AMAZED to find out the numbers of children who don’t get backpacks and whose familes can not afford the costly school supplies that are required each year.  You will be making some family happy and you’ll get the name “VMware” marketed throughout the schools, getting the next generation of techno geeks ready to learn all about virtualization.

Thanks for the consideration!

(Updated to include links to results)

I’ve had a few questions lately about “the best” blade server to use for virtualization – specifically VMware virtualization. While the obvious answer is “it depends”, I thought it would be an interesting approach to identify the blade servers that ranked in the top 5 in VMware’s VMmark benchmark.  Before I begin, let me explain what the VMmark testing is about.   VMmark enables equipment manufacturers, software vendors, system integrators and other organizations to:

• Measure virtual machine performance accurately and reliably
• Determine the performance of different hardware and virtualization platforms
• Make appropriate hardware decisions for your virtual infrastructure

VMware developed VMmark as a standard methodology for comparing virtualized systems. According to VMware’s VMmark website, the benchmark system in VMmark is comprised of a series of “sub-tests” that are derived from commonly used load-generation tools, as well as from benchmarks developed by the Standard Performance Evaluation Corporation (SPEC®). In parallel to VMmark, VMware is a member of the SPEC Virtualization subcommittee and is working with other SPEC members to create the next generation virtualization benchmark.

In testing the terms, a “tile” is simply a collection of virtual machines (VM’s) that are executing a set of diverse workloads designed to represent a natural work environment.   The total number of tiles that a server can handle provides a detailed measurement of that server’s consolidation capacity.  The more tiles, the better.  The faster the performance, the better.

THE RESULTS (as of 6/2/2010)…click on the link to open the details

24 Cores (4 Sockets)
HP ProLiant BL685c G6 (Six Core AMD Opteron 8435) running VMware ESX v4.0 – 29.19@20 tiles (published 7/14/2009)
HP ProLiant BL680c G5 (Six Core Intel Xeon E7458) running VMware ESX v3.5.0 Update 3 - 18.64@14 tiles (published 3/30/2009)

16 Cores (4 Sockets)
Dell PowerEdge M905 (Four Core AMD Opteron 8393 SE) running VMware ESX v4.0 – 22.90@17 tiles (published 6/19/2009)
HP ProLiant BL685 G6 (Four Core AMD Opteron 8389) running VMware ESX v4.0 – 20.87@14 tiles (published 4/24/2009)

12 Cores (2 Sockets)
Cisco UCS B250 M2 (Six Core Intel Xeon X5680) running VMware ESX v4.0 Update 1 – 35.83@26 tiles (published 4/6/2010)
Fujitsu BX922 S2 (Six Core Intel Xeon X5680) running VMware ESX v4.0 Update 1 – 32.89@24 tiles (published 4/6/2010)

8 Cores (2 Sockets)
Fujitsu BX922 S2 (Four Core Intel Xeon X5677) running VMware ESX v4.0 Update 1 – 27.99@18tiles(published 5/10/2010)
HP ProLiant BL490c G6 (Four Core Intel Xeon X5570) runningVMware ESX v4.0 – 25.27@17tiles (published 4/20/2010)

THE WINNER IS…
Cisco UCS B250 M2 running VMware ESX v4.0 Update 1 – 35.83 with 26 tiles

Cisco’s Winning Configuration
So – how did Cisco reach the top server spot?  Here’s the configuration:

server config:

• 2 x Intel Xeon X5680 Processors
• 192GB of RAM (48 x 4GB)
• 1 x Converged Network Adapter (Cisco UCS VIC M81KR)

storage config:

• EMC CX4-240
• Cisco MDS 9134
• 1173.48GB Used Disk Space
• 1024MB Array Cache
• 50 disks used on 5 enclosures/shelves (1 with 14 disk, 4 with 9 disks)
• 55 LUNs used
  *21 at 38GB (file server + mail server) over 20 x 73GB SSDs
  *5 at 38GB (file server + mail server) over 20 x 73GB SSDs
  *21 at 15GB (database) + 2 LUNs at 400GB (Standby, Webserver, Javaserver) over 16 x 450GB 15k disks
  *5 at 15GB (database)  over 16 x 450GB 15k disks
  * 1 LUN at 20GB (boot) over 5 x 300GB 15k disks
• RAID 0 for VMs, RAID 5 for VMware ESX 4.0 O/S

As you can see from the information above, the Cisco UCS B250 M2 is the clear winner above all of the blade server offerings.  As you can see, none of the Xeon 7500 blade servers have yet to be tested but when they do, I’ll be sure to let you know.

Let’s face it.  Virtualization is everywhere.  

Odds are there is something virtualized in your data center.  If not, it soon will be.  As more workloads become virtualized, chances are you are going to run out of “capacity” on your virtualization host.  When a host’s capacity is exhausted, 99% of the time it is because the host ran out of memory, not CPU.  Typically you would have to add another ESX host server when you run out of capacity.  When you do this, you are adding more hardware cost AND more virtualization licensing costs.  But what if you could simply add memory when you need it instead of buying more hardware.  Now you can with Dell’s FlexMem Bridge.

Background
You may recall that I mentioned the FlexMem Bridge technology in a previous post, but I don’t think I did it justice.  Before I describe what the FlexMem Bridge technology, let me provide some background.  With the Intel Xeon 7500 CPU (and in fact with all Intel Nehalem architectures), the memory is controlled by a memory controller located on the CPU.  Therefore you have to have a CPU in place to access the associated memory DIMMs…up until now.  Dell’s innovative approach removed the necessity to have a CPU in order to access the memory.

Introducing  Dell FlexMem Bridge
Dell’s FlexMem Bridge sits in CPU sockets #3 and #4 and connects a memory controller from CPU 1 to the memory DIMMs associated to CPU socket #3 and CPU 2 to the memory associated to CPU Socket #4.  

The FlexMem Bridge does two things:

1. It extends the Scalable Memory Interconnects (SMI) from CPU 1 and CPU 2 to the memory subsystem of CPU 3 and CPU 4.
2. It reroutes and terminates the 2nd Quick Path Interconnect (QPI) inter-processor communications links to provide optimal performance which would otherwise be disconnected in a 2 CPU configuration.

Sometimes it’s easier to view pictures than read descriptions, so take a look at the picture below for a diagram on how this works.

(A special thanks to Mike Roberts from Dell for assistance with the above info.)

Saving 50% on Virtualization Licensing
So how does this technology from Dell help you save money on virtualization licenses?  Simple - with Dell’s FlexMem Bridge technology, you only have to add memory, not more servers, when you need more capacity for VMs.  When you add only memory, you’re not increasing your CPU count, therefore your virtualization licensing stays the same.  No more buying extra servers just for the memory and no more buying more virtualization licenses.  In the future, if you find you have run out of CPU resources for your VM’s, you can remove the FlexMem bridges and replace with CPUs (for models with the Intel Xeon 7500 CPU only.) 

Dell FlexMem Bridge is available in the Dell PowerEdge 11G R810, R910 and M910 servers running the Intel Xeon 7500 and 6500 CPUs.

Cisco is getting some (more) recognition with their UCS blade server product, as they recently achieved the top position for “8 Core Server” on VMware’s VMmark benchmark tool.  VMmark is the industry’s first (and only credible) virtualization benchmark for x86-based computers.  According to the VMmark website, the Cisco UCS B200 blade server reached a score of 25.06 @ 17 tiles.  A “tile” is simple a collection of virtual machines (VM’s) that are executing a set of diverse workloads designed to represent a natural work environment.   The total number of tiles that a server can handle provides a detailed measurement of that server’s consolidation capacity.

Cisco’s Winning Configuration
So – how did Cisco reach the top server spot?  Here’s the configuration:

server config:

• 2 x Intel Xeon X5570 Processors
• 96GB of RAM (16 x 8GB)
• 1 x Converged Network Adapter (Cisco UCS M71KR-Q)

storage config:

• EMC CX4-240
• Cisco MDS 9130
• 1154.27GB Used Disk Space
• 1024MB Array Cache
• 41 disks used on 4 enclosures/shelves (1 with 14 disk, 3 with 9 disks)
• 37 LUNs used
  *17 at 38GB (file server + mail server) over 20 x 73GB SSDs
  *17 at 15GB (database) + 2 LUNs at 400GB (Misc) over 16 x 450GB 15k disks
  * 1 LUN at 20GB (boot) over 5 x 300GB 15k disks
• RAID 0 for VMs, RAID 5 for VMware ESX 4.0 O/S

While first place on the VMmark page (8 cores) shows Fujitsu’s RX300, it’s important to note that it was reached using Intel’s W5590 processor – a processor that is designed for “workstations” – not servers.  Second place, of server processors, currently shows HP’s BL490 with 24.54 (@ 17 tiles)

Thanks to Omar Sultan (@omarsultan) for Tweeting about this and to Harris Sussman for blogging about it.