As VMworld 2010 is right around the corner, I wanted to take a few minutes to make a plea to all attendees. 

This year, IF you receive a bag or backpack that you just don’t want, please don’t throw it away, but instead take it home, go to the dollar store and fill  the backpack with pencils, crayons, paper and erasers and donate it to your local school system.  You would be AMAZED to find out the numbers of children who don’t get backpacks and whose familes can not afford the costly school supplies that are required each year.  You will be making some family happy and you’ll get the name “VMware” marketed throughout the schools, getting the next generation of techno geeks ready to learn all about virtualization.

Thanks for the consideration!

(Updated to include links to results)

I’ve had a few questions lately about “the best” blade server to use for virtualization – specifically VMware virtualization. While the obvious answer is “it depends”, I thought it would be an interesting approach to identify the blade servers that ranked in the top 5 in VMware’s VMmark benchmark.  Before I begin, let me explain what the VMmark testing is about.   VMmark enables equipment manufacturers, software vendors, system integrators and other organizations to:

• Measure virtual machine performance accurately and reliably
• Determine the performance of different hardware and virtualization platforms
• Make appropriate hardware decisions for your virtual infrastructure

VMware developed VMmark as a standard methodology for comparing virtualized systems. According to VMware’s VMmark website, the benchmark system in VMmark is comprised of a series of “sub-tests” that are derived from commonly used load-generation tools, as well as from benchmarks developed by the Standard Performance Evaluation Corporation (SPEC®). In parallel to VMmark, VMware is a member of the SPEC Virtualization subcommittee and is working with other SPEC members to create the next generation virtualization benchmark.

In testing the terms, a “tile” is simply a collection of virtual machines (VM’s) that are executing a set of diverse workloads designed to represent a natural work environment.   The total number of tiles that a server can handle provides a detailed measurement of that server’s consolidation capacity.  The more tiles, the better.  The faster the performance, the better.

THE RESULTS (as of 6/2/2010)…click on the link to open the details

24 Cores (4 Sockets)
HP ProLiant BL685c G6 (Six Core AMD Opteron 8435) running VMware ESX v4.0 – 29.19@20 tiles (published 7/14/2009)
HP ProLiant BL680c G5 (Six Core Intel Xeon E7458) running VMware ESX v3.5.0 Update 3 - 18.64@14 tiles (published 3/30/2009)

16 Cores (4 Sockets)
Dell PowerEdge M905 (Four Core AMD Opteron 8393 SE) running VMware ESX v4.0 – 22.90@17 tiles (published 6/19/2009)
HP ProLiant BL685 G6 (Four Core AMD Opteron 8389) running VMware ESX v4.0 – 20.87@14 tiles (published 4/24/2009)

12 Cores (2 Sockets)
Cisco UCS B250 M2 (Six Core Intel Xeon X5680) running VMware ESX v4.0 Update 1 – 35.83@26 tiles (published 4/6/2010)
Fujitsu BX922 S2 (Six Core Intel Xeon X5680) running VMware ESX v4.0 Update 1 – 32.89@24 tiles (published 4/6/2010)

8 Cores (2 Sockets)
Fujitsu BX922 S2 (Four Core Intel Xeon X5677) running VMware ESX v4.0 Update 1 – 27.99@18tiles(published 5/10/2010)
HP ProLiant BL490c G6 (Four Core Intel Xeon X5570) runningVMware ESX v4.0 – 25.27@17tiles (published 4/20/2010)

THE WINNER IS…
Cisco UCS B250 M2 running VMware ESX v4.0 Update 1 – 35.83 with 26 tiles

Cisco’s Winning Configuration
So – how did Cisco reach the top server spot?  Here’s the configuration:

server config:

• 2 x Intel Xeon X5680 Processors
• 192GB of RAM (48 x 4GB)
• 1 x Converged Network Adapter (Cisco UCS VIC M81KR)

storage config:

• EMC CX4-240
• Cisco MDS 9134
• 1173.48GB Used Disk Space
• 1024MB Array Cache
• 50 disks used on 5 enclosures/shelves (1 with 14 disk, 4 with 9 disks)
• 55 LUNs used
  *21 at 38GB (file server + mail server) over 20 x 73GB SSDs
  *5 at 38GB (file server + mail server) over 20 x 73GB SSDs
  *21 at 15GB (database) + 2 LUNs at 400GB (Standby, Webserver, Javaserver) over 16 x 450GB 15k disks
  *5 at 15GB (database)  over 16 x 450GB 15k disks
  * 1 LUN at 20GB (boot) over 5 x 300GB 15k disks
• RAID 0 for VMs, RAID 5 for VMware ESX 4.0 O/S

As you can see from the information above, the Cisco UCS B250 M2 is the clear winner above all of the blade server offerings.  As you can see, none of the Xeon 7500 blade servers have yet to be tested but when they do, I’ll be sure to let you know.

Let’s face it.  Virtualization is everywhere.  

Odds are there is something virtualized in your data center.  If not, it soon will be.  As more workloads become virtualized, chances are you are going to run out of “capacity” on your virtualization host.  When a host’s capacity is exhausted, 99% of the time it is because the host ran out of memory, not CPU.  Typically you would have to add another ESX host server when you run out of capacity.  When you do this, you are adding more hardware cost AND more virtualization licensing costs.  But what if you could simply add memory when you need it instead of buying more hardware.  Now you can with Dell’s FlexMem Bridge.

Background
You may recall that I mentioned the FlexMem Bridge technology in a previous post, but I don’t think I did it justice.  Before I describe what the FlexMem Bridge technology, let me provide some background.  With the Intel Xeon 7500 CPU (and in fact with all Intel Nehalem architectures), the memory is controlled by a memory controller located on the CPU.  Therefore you have to have a CPU in place to access the associated memory DIMMs…up until now.  Dell’s innovative approach removed the necessity to have a CPU in order to access the memory.

Introducing  Dell FlexMem Bridge
Dell’s FlexMem Bridge sits in CPU sockets #3 and #4 and connects a memory controller from CPU 1 to the memory DIMMs associated to CPU socket #3 and CPU 2 to the memory associated to CPU Socket #4.  

The FlexMem Bridge does two things:

1. It extends the Scalable Memory Interconnects (SMI) from CPU 1 and CPU 2 to the memory subsystem of CPU 3 and CPU 4.
2. It reroutes and terminates the 2nd Quick Path Interconnect (QPI) inter-processor communications links to provide optimal performance which would otherwise be disconnected in a 2 CPU configuration.

Sometimes it’s easier to view pictures than read descriptions, so take a look at the picture below for a diagram on how this works.

(A special thanks to Mike Roberts from Dell for assistance with the above info.)

Saving 50% on Virtualization Licensing
So how does this technology from Dell help you save money on virtualization licenses?  Simple - with Dell’s FlexMem Bridge technology, you only have to add memory, not more servers, when you need more capacity for VMs.  When you add only memory, you’re not increasing your CPU count, therefore your virtualization licensing stays the same.  No more buying extra servers just for the memory and no more buying more virtualization licenses.  In the future, if you find you have run out of CPU resources for your VM’s, you can remove the FlexMem bridges and replace with CPUs (for models with the Intel Xeon 7500 CPU only.) 

Dell FlexMem Bridge is available in the Dell PowerEdge 11G R810, R910 and M910 servers running the Intel Xeon 7500 and 6500 CPUs.

Cisco is getting some (more) recognition with their UCS blade server product, as they recently achieved the top position for “8 Core Server” on VMware’s VMmark benchmark tool.  VMmark is the industry’s first (and only credible) virtualization benchmark for x86-based computers.  According to the VMmark website, the Cisco UCS B200 blade server reached a score of 25.06 @ 17 tiles.  A “tile” is simple a collection of virtual machines (VM’s) that are executing a set of diverse workloads designed to represent a natural work environment.   The total number of tiles that a server can handle provides a detailed measurement of that server’s consolidation capacity.

Cisco’s Winning Configuration
So – how did Cisco reach the top server spot?  Here’s the configuration:

server config:

• 2 x Intel Xeon X5570 Processors
• 96GB of RAM (16 x 8GB)
• 1 x Converged Network Adapter (Cisco UCS M71KR-Q)

storage config:

• EMC CX4-240
• Cisco MDS 9130
• 1154.27GB Used Disk Space
• 1024MB Array Cache
• 41 disks used on 4 enclosures/shelves (1 with 14 disk, 3 with 9 disks)
• 37 LUNs used
  *17 at 38GB (file server + mail server) over 20 x 73GB SSDs
  *17 at 15GB (database) + 2 LUNs at 400GB (Misc) over 16 x 450GB 15k disks
  * 1 LUN at 20GB (boot) over 5 x 300GB 15k disks
• RAID 0 for VMs, RAID 5 for VMware ESX 4.0 O/S

While first place on the VMmark page (8 cores) shows Fujitsu’s RX300, it’s important to note that it was reached using Intel’s W5590 processor – a processor that is designed for “workstations” – not servers.  Second place, of server processors, currently shows HP’s BL490 with 24.54 (@ 17 tiles)

Thanks to Omar Sultan (@omarsultan) for Tweeting about this and to Harris Sussman for blogging about it.

Unless you’ve been hiding in a cave in Eastern Europe, you know by now that the New Orleans Saints are headed to the Super Bowl.  According to IBM, this is all due to the Saints having an IBM BladeCenter S running their business.  Okay, well, I’m sure there’s other reasons, like having stellar tallent, but let’s take a look at what IBM did for the Saints.

Other than the obvious threat of having to relocate or evacuate due to the weather, the Saints’ constant travel required them to search for a portable IT solution that would make it easier to quickly set up operations in another city.  The Saints were a long-time IBM customer, so they looked at the IBM BladeCenter S for this solution, and it worked great.  (I’m going to review the BladeCenter S below, so keep reading.)  The Saints consolidated 20 physical servers onto the BladeCenter S, virtualizing the environment with VMware.   Although the specific configuration of their blade environment is not disclosed, IBM reports that the Saints are using 1 terabyte of built-in storage, which enables the Saints to go on the road with the essential files (scouting reports, financial apps, player stats, etc) and tools the coaches and the staff need.  In fact, in the IBM Case Study video, the Assistant Director of IT for the New Orleans Saints, Jody Barbier, says, “The Blade Center S definitely can make the trip with us if we go to the Super Bowl.”  I guess we’ll see.  Be looking for the IBM Marketing engine to jump on this bandwagon in the next few days.

A Look at the IBM BladeCenter S
The IBM BladeCenter S is a 7u high (click image on left for larger view of details) chassis that has the ability to hold 6 blade servers and up to 12 disk drives held in Disk Storage Modules located on the left and right of the blade server bays.  The chassis has the option to either segment the disk drives to an individual blade server, or the option to create a RAID volume and allow all of the servers to access the data.  As of this writing, the drive options for the Disk Storage Module are: 146GB, 300GB, 450GB SAS, 750GB and 1TB Near-Line SAS and 750GB and 1TB SATA.  Depending on your application needs, you could have up to 12TB of local storage for 6 servers.  That’s pretty impressive, but wait, there’s more!  As I reported a few weeks ago, there’s is a substantial rumour that there is a forthcoming option to use 2.5″ drives.  This would enable the ability to have up to 24 drives (12 per Disk Storage Module.)  Although that would provide more spindles, the current capacities of 2.5″ drives aren’t quite to the capacities of the 3.5″ drives.  Again, that’s just “rumour” – IBM has not disclosed whether that option is coming (but it is…)

IBM BladeCenter – Rear View
I love pictures – so I’ve attached an image of the BladeCenter S, as seen from the back.  A few key points to make note of:
110v Capable – yes, this can run on the average office power.  That’s the idea behind it.  If you have a small closet or an area near a desk, you can plug this bad boy in.   That being said, I always recommend calculating the power with IBM’s Power Configurator to make sure your design doesn’t exceed what 110v can handle.  Yes, this box will run on 220v as well.  Also, the power supplies are auto-sensing so there’s no worry about having to buy different power supplies based on your needs.

I/O Modules – if you are familar with the IBM BladeCenter or IBM BladeCenter H I/O architecture, you’ll know that the design is redundant, with dual paths.  With the IBM BladeCenter S, this isn’t the case.   As you can see below, the onboard network adapters (NICs) both are mapped to the I/O module in Bay #1.  The expansion card is mapped to Bay #3 and 4 and the high speed card slot (CFF-h) is mapped to I/O Bay 2.  Yes, this design put I/O Bays 1 and 2 as single points of failure (since both paths connect intothe module bay), however when you look at the typical small office or branch office environment that the IBM BladeCenter S is designed for, you’ll realize that very rarely do they have redundant network fabrics – so this is no different.

Another key point here is that I/O Bays 3 and 4 are connected to the Disk Storage Modules mentioned above.  In order for a blade server to access the external disks in the Disk Storage Module bays, the blade server must:

a) have a SAS Expansion or Connectivity card installed in the expansion card slot
b) have 1 or 2 SAS Connectivity or RAID modules attached in Bays 3 and 4

This means that there is currently no way to use the local drives (in the Disk Storage Modules) and have external access to a fibre storage array.

BladeCenter S Office Enablement Kit
Finally – I wanted to show you the optional Office Enablement Kit.  This is an 11U enclosure that is based on IBM’s NetBay 11.  It has security doors and special acoustics and air filtration to suit office environements.  The Kit features:
 *an acoustical module (to lower the sound of the environment)  Check out this YouTube video for details.
 * a locking door
 * 4U of extra space (for other devices)
 * wheels

There is also an optional Air Contaminant Filter that is available that assists in keeping the IBM BladeCenter S functional in a dusty environment (i.e. shops or production floors) using air filters.

If the BladeCenter S is going to be used in an environment without a rack (i.e. broom closet) or in a mobile environment (i.e. going to the Super Bowl) the Office Enablement Kit is a necessary addition.

So, hopefully, you can now see the value that the New Orleans Saints saw in the IBM BladeCenter S for their flexible, mobile IT needs.  Good luck in the Super Bowl, Saints.  I know that IBM will be rooting for you.

(updated 1/13/2010 – see bottom of blog for updates)

Eric Gray at www.vcritical.com blogged today about the benefits of using a flash based device, like an SD card, for loading VMware ESXi, so I thought I would take a few minutes to touch on the topic.

As Eric mentions, probably the biggest benefit of using VMware ESXi on an embedded device is that you don’t need local drives, which lowers the power and cooling of your blade server.  While he mentions HP in his blog, both HP and Dell offer SD slots in their blade servers – so let’s take a look:

HP
HP currently offers these SD slots in their BL460 G6 and BL490 G6 blade servers.  As you can see from the picture on the left (thanks again to Eric at vCritical.com) HP allows for you to access the SD slot from the top of the blade server.  This makes it fairly convenient to access, although once the image is installed on the SD card, it’s probably not ever coming out.  HP’s QuickSpecs for the BL460 G6 state offer up an “HP 4GB SD Flash Media” that has a current list price of $70, however I have been unable to find any documentation that says you MUST use this SD card, so if you want to try and use it with your own personal SD card first, good luck.  It is important to note that HP does not currently offer VMware ESXi, or any other virtualization vendor’s software, pre-installed on an SD card, unlike Dell.

Dell
Dell has been offering SD slots on select servers for quite a while.  In fact, I can remember seeing it at VMworld 2008.  Everyone else was showing “embedded hypervisors” on USB keys while Dell was using an SD card.  I don’t know that I have a personal preference of USB vs SD, but the point is that Dell was ahead of the game on this one.

Dell currently only offers their SD slot on their M805 and M905 blade servers.  These are full-height servers, which could be considered good candidates for a virtualization server due to its redundant connectivity, high memory offering and high I/O (but that’s for another blog post.)

Dell chose to place the SD slots on the bottom rear of their blade servers.  I’m not sure I agree with the placement, because if you needed to access the card, for whatever reason, you have to pull the server completely out of the chassis to service.  It’s a small thing, but it adds time and complexity to the serviceability of the server.  

An advantage that Dell has over HP is they offer to have VMware ESXi 4 PRE-LOADED on the SD key upon delivery.  Per the Dell website, an SD card with ESXi 4 (basic, not Standard or Enterprise) is available for $99.  It’s listed as “VMware ESXi v4.0 with VI4, 4CPU, Embedded, Trial, No Subsc, SD,NoMedia“.  Yes, it’s considered a “trial” and it’s the basic version with no bells or whistles, however it is pre-loaded which equals time savings.  There are additional options to upgrade the ESXi to either Standard or Enterprise as well (for additional cost, of course.)

It is important to note that this discussion was only about SD slots.  All of the blade server vendors, including IBM, have incorporated USB slots internally to their blade servers, so whereas a specific server may not have an SD slot, there is still the ability to load the hypervisor onto an USB key (where supported.)

1/13/2010 UPDATE -SD slots are also available on the BL 280G6 and BL 685 G6.

There is also an HP Advisory discouraging use of an internal USB key for embedded virtualization.  Check it out at:

http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?objectID=c01957637&lang=en&cc=us&taskId=101&prodSeriesId=3948609&prodTypeId=3709945

In the wake of the Cisco, EMC and VMware announcement, HP today is formally announcing the HP Converged Infrastructure.  You can take a look at the full details of this design on HP’s Website, but I wanted to try and simplify:

The HP Converged Infrastrcture is comprised of four core areas:

• HP Infrastructure Operating Environment
• HP FlexFabric
• HP Virtual Resource Pools
• HP Data Center Smart Grid

According to HP, achieving the benefits of a “converged infrastructure” requires the following core attributes:

1. Virtualized pools of servers, storage, networking
2. Resiliency built into the hardware, software, and operating environment
3. Orchestration through highly automated resources to deliver an application aligned according to policies
4. Optimized to support widely changing workloads and different applications and usage models
5. Modular components built on open standards to more easily upgrade systems and scale capacity

Let’s take a peak into each of the core areas that makes up the HP Converged Infrastructure.

HP Infrastructure Operating Environment
This element of the converged infrastructure provides a shared services management engine that adapts and provisions the infrastructure.  The goal of this core area is to expedite delivery and provisioning of the datacenter’s infrastructure. 

The HP Infrastructure Operating Environment is comprised of HP Dynamics - a command center that enables you to continuously analyze and optimize your infrastructure; and HP Insight Control- HP’s existing server management software.

HP FlexFabric
HP defines this core area as  a “next-generation, highly scalable data center fabric architecture and a technology layer in the HP Converged Infrastructure.”  The goal of the HP FlexFabric is to create a highly scalable, flat network domain that enables administrators to easily provision networks as needed and on-demand to meet the virtual machines requirements. 

HP’s FlexFabric is made up of HP’s ProCurve line and their VirtualConnect technologies.  Beyond the familiar network components, the HP Procurve Data Center Connection Manager is also included as a fundamental component offering up automated network provisioning.

HP Virtual Resource Pools
This core area is designed to allow for a virtualized collection of storage, servers and networking that can be shared, repurposed and provisioned as needed.

Most of HP’s Enterprise products fit into this core area.  The HP 9000 and HP Integrity servers use HP Global Workload Managerto provision workloads; HP Proliant servers can use VMware or Microsoft’s virtualization technologies and the HP StorageWorks SAN Virtualization Services
Platform(SVSP) enables network-based (SAN) virtualization of heterogeneous disk arrays.

HP Data Center Smart Grid
The goal of this last core area of the HP Converged Infrastructure is to “create an intelligent, energy-aware environment across IT and facilities to optimize and reduce energy use, reclaiming facility capacity and reducing energy costs.”

HP approaches this core area with a few different products.  The Proliant G6 server lines offer a “sea of sensors” that aid with the consumption of power and cooling.  HP also offers a Performance Optimized Datacenter (POD)- a container based datacenter that optimize power and cooling.    HP also uses the HP Insight Control software to manage the HP Thermal Logic technologies and control peaks and valleys of power management on servers.

Summary
In summary, HP’s Converged Infrastructure follows suit with what many other vendors are doing – taking their existing products and technologies and re-marketing them to closely align and reflect a more coherent messaging.  Only time will tell as to if this approach will be successful in growing HP’s business.

By now I’m sure you’ve read, heard or seen Tweeted the announcement that Cisco, EMC and VMware have come together and created the Virtual Computing Environment coalition .   So what does this announcement really mean?  Here are my thoughts:

Greater Cooperation and Compatibility
Since these 3 top IT giants are working together, I expect to see greater cooperation between all three vendors, which will lead to understanding between what each vendor is offering.  More important, though, is we’ll be able to have reference architecturethat can be a starting point to designing a robust datacenter.  This will help to validate that an “optimized datacenter” is a solution that every customer should consider.

Technology Validation
With the introduction of the Xeon 5500 processor from Intel earlier this year and the announcement of the Nehalem EX coming early in Q1 2010, the ability to add more and more virtual machines onto a single host server is becoming more prevalent.  No longer is the processor or memory the bottleneck – now it’s the I/O.  With the introduction of Converged Network Adapters (CNAs), servers now have access to  Converged Enhanced Ethernet (CEE) or DataCenter Ethernet (DCE) providing up to 10Gb of bandwidth running at 80% efficiency with lossless packets.  With this lossless ethernet, I/O is no longer the bottleneck.

VMware offers the top selling virtualization software, so it makes sense they would be a good fit for this solution.

Cisco has a Unified Computing System that offers up the ability to combine a server running a CNA to a Interconnect switch that allows the data to be split out into ethernet and storage traffic.  It also has a building block design to allow for ease of adding new servers – a key messaging in the Coalition announcement.

EMCoffers a storage platform that will enable the storage traffic from the Cisco UCS 6120XP Interconnect Switch and they have a vested interest in VMware and Cisco, so this marriage of the 3 top IT vendors is a great fit.

Announcement of Vblock™ Infrastructure Packages
According to the announcement, the Vblock Infrastructure Package “will provide customers with a fundamentally better approach to streamlining and optimizing IT strategies around private clouds.”  The packages will be fully integrated, tested, validated, and that combine best-in-class virtualization, networking, computing, storage, security, and management technologies from Cisco, EMC and VMware with end-to-end vendor accountability.  My thought on these packages is that they are really nothing new.  Cisco’s UCS has been around, VMware vSphere has been around and EMC’s storage has been around.  The biggest message from this announcement is that there will soon be  “bundles” that will simplify customers solutions.  Will that take away from Solution Providers’ abilities to implement unique solutions?  I don’t think so.  Although this new announcement does not provide any new product, it does mark the beginning of an interesting relationship between 3 top IT giants and I think this announcement will definitely be an industry change – it will be interesting to see what follows.

UPDATE – click here check out a 3D model of the vBlocks Architecture.

According to IBM’s System x and BladeCenter x86 Server Blog, the IBM BladeCenter HS22 server has posted the best SPECweb2005 score ever from a blade server.  With a SPECweb2005 supermetric score of 75,155, IBM has reached a benchmark seen by no other blade yet to-date.  The SPECweb2005 benchmark is designed to be a neutral, equal benchmark for evaluting the peformance of web servers.  According to the IBM blog, the score is derived from three different workloads measured:

• SPECweb2005_Banking – 109,200 simultaneous sessions
• SPECweb2005_Ecommerce – 134,472 simultaneous sessions
• SPECweb2005_Support – 64,064 simultaneous sessions

The HS22 achieved these results using two Quad-Core Intel Xeon Processor X5570 (2.93GHz with 256KB L2 cache per core and 8MB L3 cache per processor—2 processors/8 cores/8 threads). The HS22 was also configured with 96GB of memory, the Red Hat Enterprise Linux® 5.4 operating system, IBM J9 Java® Virtual Machine, 64-bit Accoria Rock Web Server 1.4.9 (x86_64) HTTPS software, and Accoria Rock JSP/Servlet Container 1.3.2 (x86_64).

It’s important to note that these results have not yet been “approved” by SPEC, the group who posts the results, but as soon as they are, they’ll be published at at http://www.spec.org/osg/web2005

The IBM HS22 is IBM’s most popular blade server with the following specs:

• up to  2 x Intel 5500 Processors
• 12 memory slots for a current maximum of 96Gb of RAM
• 2 hot swap hard drive slots capable of running RAID 1 (SAS or SATA)
• 2 PCI Express connectors for I/O expansion cards (NICs, Fibre HBAs, 10Gb Ethernet, CNA, etc)
• Internal USB slot for running VMware ESXi
• Remote management
• Redundant connectivity