Revised with corrections  3/1/2011 10:29 a.m. (EST)
Almost a year ago, I wrote an article highlighting the 4 socket blade server offerings.  At that time, the offerings were very slim, but over the past 11 months, that blog post has received the most hits, so I figured it’s time to revise the article.  In today’s post, I’ll review the 4 socket Intel and AMD blade servers that are currently on the market.  Yes, I know I’ll have to revise this again in a few weeks, but I’ll cross that bridge when I get to it. 

Cisco

Cisco released their first 4 socket blade offering in April 2010  known as the “UCS B440 M1“.

 This blade is a full-width blade that offers:

• 4 x Intel Xeon 7500 CPUs (6 to 8 cores)
• Max of 512GB (across 32 DIMM slots)
• 2 x I/O mezzanine expansion card slots
• 4 x hot-swap drive bays

The Cisco UCS B440 M1 blade server also has 2 x mezzanine expansion card slots that are capable of holding a variety of converged network adapters (CNAs) with a minimum of 10GbE connectivity.  Each Cisco UCS 5108 chassis can hold 4 x UCS B440 M1 for a total of 28 UCS B440 M1′s per 42u rack.  To save you from doing the math, that’s a max of 896  CPU cores and 14.3TB of memory per 42u rack. 

You can read more about the Cisco UCS B440 M1 on Cisco.com.

Dell
Dell has two 4 socket blade offerings, the PowerEdge M905 and the PowerEdge M910.  Anytime you see a “5″ at the end of a Dell server number, you can be assured that it has an AMD processor whereas the servers ending in “0″ are Intel processors.

The PowerEdge M905 is a full-height blade that offers:

• 4 x AMD Opteron 8300 or 8400 (up to 6 cores)
• 4 x 1Gb Ethernet LOM (LAN on Motherboard)
• Max of 192GB (across 24 DIMM slots)
• 4 x I/O mezzanine expansion card slots
• 2 x hot-swap drive bays

The PowerEdge M910 is a full-height blade that offers:

• 4 x Intel Xeon 7500 (up to 8 cores)
• 4 x 1Gb Ethernet LOM (LAN on Motherboard)
• Max of 512GB (across 32 DIMM slots)
• 4 x I/O mezzanine expansion card slots
• 2 x hot-swap drive bays

Each Dell M1000e chassis can hold 8 x PowerEdge M905 or PowerEdge M910′s for a total of 32 Dell PowerEdge M905/M910 per 42u rack.  To save you from doing the math, for the M905 that’s a max of 768 CPU cores and 6.1TB of memory per 42u rack; for the M910 that’s a max of 1024 CPU cores and 16.3TB of memory per 42u rack.

You can find out more about the Dell PowerEdge M905 and PowerEdge M910 on Dell.com.

HP

HP has been in the 4 socket blade offering space for the longest, so they have the largest collection of servers to review.  Keep in mind, the “Gx” at the end of the server signifies the family generation of the product, so the G5 is older than the G7 (yes, common sense, but if I don’t spell it out, someone will comment on it…)

The BL680c G5 is a full-height blade that offers:

• 4 x Intel Xeon 7400 CPUs (up to 6 cores each)
• 4 x 1Gb Ethernet LOM (LAN on Motherboard)
• Max of 128GB (across 16 DIMM slots)
• 4 x I/O mezzanine expansion card slots
• 2 x hot-swap drive bays

(You may ask – why am I listing this older generation blade server?  Quite simply because HP still mentions it on the HP.com website.)

The BL680c G7  is the next generation to the BL680 G5 and it was first mentioned June 2010.  The BL680 G7 is a double-width (2 x full-height) blade that offers:

• 4 x Intel Xeon 7500 CPUs (up to 8 cores each)
• 6 x 10Gb FlexFabric NICs LOM (LAN on Motherboard)
• Max of 1TB (across 64 DIMM slots)
• 7 x I/O mezzanine expansion card slots
• 4 x hot-swap drive bays

HP uses a naming schema that is fairly easy to understand.  Anytime you see a “5″ at the end of a HP server number, you can be assured that it has an AMD processor whereas the servers ending in “0″ are Intel processors.   HP offers a pair of AMD servers that parity the  2 Intel blade servers mentioned above.

The BL685c G6  is a full-height blade that offers:

• 4 x AMD Opteron 8300 or 8400 (up to 6 cores)
• 4 x 10Gb Flex-10 NICs LOM (LAN on Motherboard)
• Max of 256GB (across 32 DIMM slots)
• 4 x I/O mezzanine expansion card slots
• 2 x hot-swap drive bays

The BL685c G7  is the newest AMD offering in a full-height blade that comes with:

• 4 x AMD Opteron 6100 (up to 12 cores)
• 4 x 10Gb FlexFabric NICs LOM (LAN on Motherboard)
• Max of 512 GB (across 32 DIMM slots)
• 3 x I/O mezzanine expansion card slots
• 2 x hot-swap drive bays

I’ll be talking about FlexFabric in the near future, but for now think of it as HP’s version of the Converged Network Adapter.  You can read more  at www.hp.com/go/flexfabric.

Here are the totals (sorry, too much info for fancy summarizing:)

BL680c G5

BL680c G7

BL685c G6

BL685c G7

You can find out more about HP’s blade servers at HP.com.

IBM

The IBM HX5 was announced March 2010.   The HX5 is a 2 CPU blade server that uses a modular approach allowing users to add a memory blade (called the MAX5) or an 2nd HX5 to create a 4 CPU offering.  While I covered the technology back in March (check it out here) I’m looking at the 2 x HX5 combination as IBM’s 4 socket offering.

The IBM HX5  is a double-wide (2 x 30mm) blade that comes with:

• 4 x Intel Xeon 7500 CPUs (up to 8 cores each)
• 4 x 1Gb Ethernet NICs LOM (LAN on Motherboard)
• Max of 256GB (across 32 DIMM slots)
• 4 x I/O mezzanine expansion card slots
• 4 x internal drive bays

Each IBM BladeCenter H chassis can hold 7 x HX5 blade servers for a total of 28 HX5 blade servers per 42u rack.  To save you from doing the math, that’s a max of 896 CPU cores and 7.1TB of memory per 42u rack.

Summary
Here’s a summary of all of the current 4 socket blade servers (click to enlarge):

Here’s a summary of the top offerings (shown in yellow on the chart):

Most CPU Cores in a Blade Server:  HP BL685c G7
Most I/O Expansion in a Blade Server: HP BL680c G7
Most Memory in a Blade Server: HP BL680c G7

Most Memory in a 42u Rack: (tie) Dell M910, HP BL680c G7, HP BL685c G7
Most CPU Cores in a 42u Rack: HP BL685c G7

Yes, I know that there are slim chances that anyone would fill up a rack with 4 socket servers, however I thought this would be good comparison to make. What are your thoughts? Let me know in the comments below.

Chassis Overview

Dell’s blade platform relies on the M1000e Blade Chassis.  Like Cisco, Dell offers a single chassis design for its blade portfolio.  The chassis is 10 rack units tall (17.5″) and holds 16 half height or 8 full height servers in any combination.  The front of the chassis provides two USB Keyboard/Mouse connections and one Video connection (requiring the optional integrated Avocent iKVM switch to enable these ports) for local front “crash cart” console connections that can be switched between blades. 

Also standard is a LCD control panel with interactive Graphical LCD that offers an initial configuration wizard and provides local server blade, enclosure, and module information and troubleshooting.  The chassis also contains a power button that gives the user the ability to shut power off to the entire enclosure. 

Taking a look at the rear of the chassis, we see that the Dell M1000e offers up to 6 redundant 2350 – 2700 watt hot plug power supplies and 9 redundant fan modules.  The 2700 watt power supplies are part of Dell’s newest addition to the chassis offering a higher efficiency, according to a recent study performed by Dell.  

The M1000e allows for up to six total I/O modules to be installed providing three redundant fabrics.  I’ll cover the I/O module offerings later on this post.

The chassis comes standard with a Chassis Management Controller (CMC) that gives a single management window to view the inventory, configurations, monitoring and alerting for the chassis and all components.  There is also a slot for an optional secondary CMC for redundancy.  The CMC can be connected to another Dell chassis’ CMC to provide consolidation and reduction of port consumption on external switches. 

Between the primary and secondary CMC slots is a slot to add the optional Integrated KVM (iKVM) module.    The iKVM provides local keyboard, video and mouse connectivity into the blade chassis and associated blade servers.  The iKVM also contains a dedicated RJ45 port with an Analog Console Interface (ACI) that is compatible with most Avocent switches letting one port on an external Avocent KVM switch for all 16 blade servers within a Dell M1000e chassis.

Server Review

Dell offers a full range of servers available in half height and full height form factors with a  full height server taking up 2 half height server bays.  Dell offers both AMD and Intel in their portfolio, with AMD based blades ending in “5″ and Intel based blades ending in “0″. 

Looking across the entire spectrum of blade server offerings, the M910 offers the most advantages with the highest CPU, core and memory within the Dell blade server family.  This blade server also currently has the #1 VMware VMmark score for a 16 core blade server, although I’m sure it will get trumped soon.  An advantage that I think the Dell blade server have is they provide 4 I/O card slots on their full height servers.  This is a big deal to me because it gives redundant mezzanine (daughter) cards for each fabric (for more on I/O connectivity see below.)  While that leads to twice the mezzanine card cost compared to their competitors, it can provide a piece of mind for that user looking for high redundancy.  At the same time, having 4 I/O card slots gives users the ability to gain more I/O ports per server.

In the list of Dell blade server offerings are a few specialized servers – the M610x and the M710HD.  The M610x offers local PCI-express expansion on the server (not via an expansion blade) while the M710HD offers greater memory density along with a unique modular Lan-On-Motherboard expansion card called “Network Daughter Card”.  I’ve written about these servers in a previous post, so I encourage you to take a few minutes to read that post if you are interested.

I/O Card Options

Dell offers a wide variety of daughter cards, aka “mezzanine cards”, for their blade servers.  Let’s take a quick look at what is offered across the three major I/O categories – Ethernet, Fibre and Infiniband. 

Ethernet Cards
(click on the link to read full details about the card)

• Broadcom NetXtreme II 5709 Dual Port Ethernet Mezzanine Card with TOE and iSCSI Offload
• Broadcom NetXtreme II 5709 Quad Port Ethernet  Mezzanine Card
• Intel Gigabit ET Quad Port Mezzanine Card with Virtualization technology and iSCSI Acceleration
• Broadcom NetXtreme II 57711 Dual Port 10Gb Ethernet Mezzanine Card with TOE and iSCSI Offload
• Intel Ethernet X520 10Gb Dual Port –x/k Mezzanine Card 
• Mellanox ConnectX-2 VPI Dual Port 10Gb Low Latency Ethernet -x/k Mezzanine Card 
• Emulex OCM10102FM 10 Gb Fibre Channel over Ethernet Mezz Card 
• Qlogic QME8142 10Gb Fibre Channel over Ethernet Adapter

Fibre Cards
(click on the link to read full details about the card)

• Emulex LPe1105-M 4Gb Fibre Channel I/O Card
• QLogic QME2472 4Gb Fibre Channel I/O Card
• Emulex LPe1205-M 8Gb Fibre Channel I/O Card
• QLogic QME2572 8Gb Fibre Channel I/O Card

Infiniband Cards
(click on the link to read full details about the card)

• Mellanox QDR & DDR ConnectX-2 Dual-Port Quad & Dual Data Rate (QDR & DDR) InfiniBand Host Channel Adapters (HCA)
• Mellanox ConnectX-2 VPI Dual Port 10Gb Low Latency Ethernet -x/k Mezzanine Card 

A key point to realize is that each mezzanine card requires an I/O module to connect to.  Each card contains at least two ports – one goes to I/O module in one bay and the other port goes to the I/O module in another bay.  (More on this below.)  Certain mezzanine cards work with certain switch modules, so make sure to review the details of each card to understand what switch is compatible with the I/O card you want to use.

Chassis I/O Switch Options

One of the most challenging components of any blade server architecture is understanding how the I/O modules, or switches work within a blade infrastructure.  The concept is quite simple – in order for an I/O port on a blade server to get outside of the chassis into the network or storage fabric there must be a module in the chassis that correlates to that specific card.   It is important to understand that each I/O port is hardwired inside the M1000e chassis to connect to an I/O bay.  On two port cards, port 0 would go to I/O Module Bay 1 and port 1 would go to I/O Module Bay 2.  On four port cards, the even # ports (0 and 2) would go to I/O Module Bay 1 and the odd # ports (1 and 3) go to I/O Module Bay 2.  This is an important point to understand.  If you have a dual port card, but only put an I/O module in one of the two I/O Bays, you only get 1 of the 2 ports on the card lit up AND you have no redundant path, so it’s always best practice to put I/O modules in both bays of the I/O fabric.

For example, the NICs that reside on the blade motherboard need to have Ethernet modules in I/O bays A1 and A2.  (Click on each image to enlarge for better viewing.)

 The mezzanine cards in slots 1 and 3 need to have a related I/O module in I/O bays A2, so if you put in an Ethernet card in Mezzanine Slot 1 and/or 3, you’d have to have an Ethernet module in I/O bay B1 and B2.

 As you can imagine, the same applies for the mezzanine cards in slots 2 and 4, which map to C1 and C2.

The images shown above reflect Dell’s full height server offerings.  In the event that half height servers are used, then only mezzanine slots 1 and 2 would be connected.

When we review the I/O module offerings from Dell we see there is quite a list of Ethernet, Fibre and Infiniband devices available to work with the mezzanine cards listed above.  I/O modules come in two offerings: switches and pass-thru modules.  Dell M1000e Blade Chassis I/O Switches function the same way as external  switch modules – they provide a consolidated switched connection into a fabric.  For a fully loaded blade chassis, you could use a single connection per I/O module to connect your servers into each fabric.  With a switch you often have fewer uplinks than internal connections, very similar to an external I/O switch.  

In comparison, a pass-thru module provides no switching, only a direct one-for-one connection from the port on the blade server to the port on an external switch.   For example, if you had the Dell M1000e populated with 16 servers and you wanted to get network connectivity and redundancy for the NICs on the motherboard of each blade server and  you put in an Ethernet pass-thru module in I/O Bay A1 and A2, you would need 32 ports available on your external network switch fabric (16 from A1 and 16 from A2).  In summary, switches = fewer cable connections to your external fabric.

Here’s a list of what Dell offers:

Ethernet Modules
(click on the link to read full details about the module)

• Dell 16 Port Gigabit Ethernet Pass-Through
• Cisco Catalyst Blade Switch 3032
• Cisco Catalyst Blade Switch 3130G – Stackable Managed Gigabit Ethernet Switch
• Cisco Catalyst Blade Switch 3130X – StackableManaged 1Gb/10Gb Uplink Ethernet Switch
• Dell PowerConnect M6220 – High Performance Layer 3, Managed Gigabit Ethernet Switch with resilient stacking and 10Gigabit Ethernet capabilities
• Dell PowerConnect M6348 – 48 Port Gigabit Ethernet
• Dell 10Gb Ethernet Pass-Through I/O Module
• Dell PowerConnect M8024 - All 10Gb Ethernet performance and bandwidth Layer 3 Managed Switch

Fibre Modules
(click on the link to read full details about the module) 

• 4Gb Fibre Channel Pass-Through Module
• Dell 8/4Gbps FC SAN Module
• Brocade®  M5424 – 8Gb Fibre Channel Module

Infiniband Modules
(click on the link to read full details about the module) 

• Mellanox M2401G – Dual Data Rate (DDR) InfiniBand Switch
• Mellanox M3601Q – Quad Data Rate (QDR) InfiniBand Switch

A few things to point out about these offerings. 

a) The Dell PowerConnect M6348 is classified as a “48 Port Gigabit Ethernet switch” providing 32 internal GbE ports (2 per blade) and 16 external fixed 10/100/1000Mb Ethernet ports ONLY when using the quad port GbE mezzanine cards (Broadcom 5709 or Intel ET 82572).  If Dual port GbE cards are used only half of the switch’s internal ports will be used.   It can also be used in Fabric A when using M710HD with its quad port Broadcom 5709C NDC.  (thanks to Andreas Erson for this point.)

b) If you are connecting to a converged fabric, i.e. to a Cisco Nexus 5000, use the Emulex OCM10102FM 10 Gb Fibre Channel over Ethernet Mezz Card or the Qlogic QME8142 10Gb Fibre Channel over Ethernet Adapter  along with the Dell 10Gb Ethernet Pass-Through I/O Module.

New addition to previous post- c) The Infiniband QDR Mellanox M3601Q is a dual-width I/O Module occupying both B- and C-fabric slots.

Server Management

Dell’s chassis management is controlled by the onboard Chassis Management Controller (CMC).  The CMC provides multiple systems management functions for the Dell M1000e, including the enclosure’s network and security settings, I/O module and iDRAC network settings, and power redundancy and power ceiling settings.  There is a ton of features that are available in the Dell CMC console, so I thought I would put together a video of the different screen shots.  This video includes all areas of the console, based on version 3.0 and was taken from my work lab so I’ve blacked out some data. 

A Review in Pictures of the Dell Chassis Management Controller (CMC)

Here’s the YouTube link for those of you without Flash: http://www.youtube.com/watch?v=4uCGHw8jK3M&hd=1

A Review in Pictures of the Dell blade iDRAC

Once you get beyond the CMC you also have the ability to access the onboard management console on the individual blades, called the iDRAC (short for “Integrated Dell Remote Access Controller”).  The features found here are unique to management / monitoring of the individual blade, but it also is the gateway to launching a remote console session – which is in the next session.

Here’s the YouTube link for those of you without Flash: http://www.youtube.com/watch?v=gjOsPorqhcw&hd=1

A Review in Pictures of the Dell blade Remote Console

Finally – if you were to launch a remote console session from the iDRAC, you would have complete remote access.  Think of an RDP session, but for blade servers.  Take a look, it’s pretty interesting.  With newer blade servers (PowerEdge 11G models), you can launch the Remote Console from the CMC – a nice addition.  My videos were used with older blade servers (M600′s) so I didn’t have any screen shot, but my friend from Dell, Scott Hanson (@DellServerGeek), gave me some photos to point to:

Here’s a couple screenshots :

From the Chassis Overview – http://www.twitpic.com/33vv87
From the Server Overview – http://www.twitpic.com/33vvj2
From the Individual Server Overview – http://www.twitpic.com/33vvz0

Here’s the YouTube link for those of you without Flash: http://www.youtube.com/watch?v=h6S9nuPv7XE&hd=1

So that’s it.  For those of you who have been waiting the past few months for me to finish, let me know what you think.  Is there anything I’m missing – anything else you would like to see on this?  Let me know in the comments below.  Make sure to keep an eye on this site as I’ll be posting information on HP and IBM in the following weeks (months?).

IMPORTANT NOTE – I updated this blog post on Feb. 28, 2011 with better details.  To view the updated blog post, please go to:

http://bladesmadesimple.com/2011/02/4-socket-blade-servers-density-vendor-comparison-2011/

Original Post (March 10, 2010):

As the Intel Nehalem EX processor is a couple of weeks away, I wonder what impact it will have in the blade server market.  I’ve been talking about IBM’s HX5 blade server for several months now, so it is very clear that the blade server vendors will be developing blades that will have some iteration of the Xeon 7500 processor.  In fact, I’ve had several people confirm on Twitter that HP, Dell and even Cisco will be offering a 4 socket blade after Intel officially announces it on March 30.  For today’s post, I wanted to take a look at how the 4 socket blade space will impact the overall capacity of a blade server environment.  NOTE: this is purely speculation, I have no definitive information from any of these vendors that is not already public.

The Cisco UCS 5108 chassis holds 8 “half-width” B-200 blade servers or 4 “full-width” B-250 blade servers, so when we guess at what design Cisco will use for a 4 socket Intel Xeon 7500 (Nehalem EX) architecture, I have to place my bet on the full-width form factor.  Why?  Simply because there is more real estate.  The Cisco B250 M1 blade server is known for its large memory capacity, however Cisco could sacrifice some of that extra memory space for a 4 socket, “Cisco B350“ blade.  This would provide a bit of an issue for customers wanting to implement a complete rack full of these servers, as it would only allow for a total of 28 servers in a 42U rack (7 chassis x 4 servers per chassis.)

Estimated Cisco B300 with 4 CPUs

On the other hand, Cisco is in a unique position in that their half-width form factor also has extra real estate because they don’t have 2 daughter card slots like their competitors.  Perhaps Cisco would create a half-width blade with 4 CPUs (a B300?)  With a 42U rack, and using a half-width design, you would be able to get a maximum of 56 blade servers (7 chassis x 8 servers per chassis.)

Dell
The 10U M1000e chassis from Dell can currently handle 16 “half-height” blade servers or 8 “full height” blade servers.  I don’t forsee any way that Dell would be able to put 4 CPUs into a half-height blade.  There just isn’t enough room.  To do this, they would have to sacrifice something, like memory slots or a daughter card expansion slot, which just doesn’t seem like it is worth it.  Therefore, I predict that Dell’s 4 socket blade will be a full-height blade server, probably named a PowerEdge M910.  With this assumption, you would be able to get 32 blade servers in a 42u rack (4 chassis x 8 blades.) 

HP
Similar to Dell, HP’s 10U BladeSystem c7000 chassis can currently handle 16 “half-height” blade servers or 8 “full height” blade servers.  I don’t forsee any way that HP would be able to put 4 CPUs into a half-height blade.  There just isn’t enough room.  To do this, they would have to sacrifice something, like memory slots or a daughter card expansion slot, which just doesn’t seem like it is worth it.  Therefore, I predict that HP’s 4 socket blade will be a full-height blade server, probably named a Proliant BL680 G7 (yes, they’ll skip G6.)  With this assumption, you would be able to get 32 blade servers in a 42u rack (4 chassis x 8 blades.) 

IBM
Finally, IBM’s 9U BladeCenter H chassis offers up 14 servers.  IBM has one size server, called a “single wide.”  IBM will also have the ability to combine servers together to form a “double-wide”, which is what is needed for the newly announced IBM BladeCenter HX5.  A double-width blade server reduces the IBM BladeCenter’s capacity to 7 servers per chassis.  This means that you would be able to put 28 x 4 socket IBM HX5 blade servers into a 42u rack (4 chassis x 7 servers each.)

Summary
In a tie for 1st place, at 32 blade servers in a 42u rack, Dell and HP would have the most blade server density based on their existing full-height blade server design.  IBM and Cisco would come in at 3rd place with 28 blade servers in a 42u rack..  However IF Cisco (or HP and Dell for that matter) were able to magically re-design their half-height servers to hold 4 CPUs, then they would be able to take 1st place for blade density with 56 servers. 

Yes, I know that there are slim chances that anyone would fill up a rack with 4 socket servers, however I thought this would be good comparison to make.  What are your thoughts?  Let me know in the comments below.