Many people get confused as to why so many I/O modules are needed within a given blade chassis.  The basic concept is simple (in most cases) – for each port you need on a given blade server, you need to have a corresponding I/O module.  For example, if you need 4 NICs, you’re going to need 4 Ethernet modules (in most cases.)  In today’s post, I thought I would keep it simple and publish the I/O diagrams of Cisco, Dell, HP and IBM chassis.  Of course, I am human and “have been known to make mistakes – from time to time” so please feel free to correct me on any errors you see.  Enjoy.

(Updated 8/3/2011 – fixed Dell M1000e Full Height I/O Diagram)

Since the dawn of mankind (or more realistically, the early 2000’s) people have debated whether blade servers or rack servers were best for their datacenter environment.   It may surprise you that I personally don’t believe that blade servers fit everywhere.  In fact, if you aren’t running out of power/cooling, you don’t have space constraints, you need a lot of local I/O expansion or you don’t have management issues with your physical server environment, you may be better off with rack servers.  That being said, there are a lot of benefits to a blade server design as well.

Blade servers offer flexibility.  From the ability to cable / power an enclosure ONE TIME to the ability to consolidate I/O connectivity, the design of a bladed server gives you the flexibility to do what you need to do in a smaller, consolidated space. 

Blade servers are more efficient.  Blade servers share the same space, share the same cooling, share the same power, share the same connectivity.  Sharing of resources mean you utilize your power and cooling more efficiently.  A blade server infrastructure gives you the ability to manage all of your data center resources in one location, helping you manage your datacenter more efficiently as well.

Blade servers offer better performance.  Now I’m sure I’ll get some debate on this one.  The argument is that some rack servers can hold more memory, hold more CPUs, hold more I/O and use the same chipsets as blade servers, so how can blade servers have better performance?  I think blade servers (in the same chassis) can offer better performance because they share the same mid-plane.  In most blade server designs, if the blade server in slot 1 needs to communicate to the blade server in slot 6, the communication never leaves the chassis.  The two servers can communicate at line speed.  Compared to a rack server environment, the communications would have to travel to the top of rack or edge network device before it can travel back.  This may not always be the case, but it’s an advantage that only blade servers can offer.

A few weeks ago I had the pleasure of recording an Infosmack Deepdive podcast with Nigel Poulton, Bob Plankers, and Rick Vanover to discuss rack servers vs blade servers.  It was a lively discussion that touches on the advantages and disadvantages of both rack servers and blade servers.   We even talk about FCoTR.   The podcast is now available on http://www.theregister.co.uk/ (look for the link under top stories “Infosmack podcasts”.)

Let me know what you think – your thoughts are valuable.

Revised with corrections  3/1/2011 10:29 a.m. (EST)
Almost a year ago, I wrote an article highlighting the 4 socket blade server offerings.  At that time, the offerings were very slim, but over the past 11 months, that blog post has received the most hits, so I figured it’s time to revise the article.  In today’s post, I’ll review the 4 socket Intel and AMD blade servers that are currently on the market.  Yes, I know I’ll have to revise this again in a few weeks, but I’ll cross that bridge when I get to it. 

Cisco

Cisco released their first 4 socket blade offering in April 2010  known as the “UCS B440 M1“.

 This blade is a full-width blade that offers:

• 4 x Intel Xeon 7500 CPUs (6 to 8 cores)
• Max of 512GB (across 32 DIMM slots)
• 2 x I/O mezzanine expansion card slots
• 4 x hot-swap drive bays

The Cisco UCS B440 M1 blade server also has 2 x mezzanine expansion card slots that are capable of holding a variety of converged network adapters (CNAs) with a minimum of 10GbE connectivity.  Each Cisco UCS 5108 chassis can hold 4 x UCS B440 M1 for a total of 28 UCS B440 M1′s per 42u rack.  To save you from doing the math, that’s a max of 896  CPU cores and 14.3TB of memory per 42u rack. 

You can read more about the Cisco UCS B440 M1 on Cisco.com.

Dell
Dell has two 4 socket blade offerings, the PowerEdge M905 and the PowerEdge M910.  Anytime you see a “5″ at the end of a Dell server number, you can be assured that it has an AMD processor whereas the servers ending in “0″ are Intel processors.

The PowerEdge M905 is a full-height blade that offers:

• 4 x AMD Opteron 8300 or 8400 (up to 6 cores)
• 4 x 1Gb Ethernet LOM (LAN on Motherboard)
• Max of 192GB (across 24 DIMM slots)
• 4 x I/O mezzanine expansion card slots
• 2 x hot-swap drive bays

The PowerEdge M910 is a full-height blade that offers:

• 4 x Intel Xeon 7500 (up to 8 cores)
• 4 x 1Gb Ethernet LOM (LAN on Motherboard)
• Max of 512GB (across 32 DIMM slots)
• 4 x I/O mezzanine expansion card slots
• 2 x hot-swap drive bays

Each Dell M1000e chassis can hold 8 x PowerEdge M905 or PowerEdge M910′s for a total of 32 Dell PowerEdge M905/M910 per 42u rack.  To save you from doing the math, for the M905 that’s a max of 768 CPU cores and 6.1TB of memory per 42u rack; for the M910 that’s a max of 1024 CPU cores and 16.3TB of memory per 42u rack.

You can find out more about the Dell PowerEdge M905 and PowerEdge M910 on Dell.com.

HP

HP has been in the 4 socket blade offering space for the longest, so they have the largest collection of servers to review.  Keep in mind, the “Gx” at the end of the server signifies the family generation of the product, so the G5 is older than the G7 (yes, common sense, but if I don’t spell it out, someone will comment on it…)

The BL680c G5 is a full-height blade that offers:

• 4 x Intel Xeon 7400 CPUs (up to 6 cores each)
• 4 x 1Gb Ethernet LOM (LAN on Motherboard)
• Max of 128GB (across 16 DIMM slots)
• 4 x I/O mezzanine expansion card slots
• 2 x hot-swap drive bays

(You may ask – why am I listing this older generation blade server?  Quite simply because HP still mentions it on the HP.com website.)

The BL680c G7  is the next generation to the BL680 G5 and it was first mentioned June 2010.  The BL680 G7 is a double-width (2 x full-height) blade that offers:

• 4 x Intel Xeon 7500 CPUs (up to 8 cores each)
• 6 x 10Gb FlexFabric NICs LOM (LAN on Motherboard)
• Max of 1TB (across 64 DIMM slots)
• 7 x I/O mezzanine expansion card slots
• 4 x hot-swap drive bays

HP uses a naming schema that is fairly easy to understand.  Anytime you see a “5″ at the end of a HP server number, you can be assured that it has an AMD processor whereas the servers ending in “0″ are Intel processors.   HP offers a pair of AMD servers that parity the  2 Intel blade servers mentioned above.

The BL685c G6  is a full-height blade that offers:

• 4 x AMD Opteron 8300 or 8400 (up to 6 cores)
• 4 x 10Gb Flex-10 NICs LOM (LAN on Motherboard)
• Max of 256GB (across 32 DIMM slots)
• 4 x I/O mezzanine expansion card slots
• 2 x hot-swap drive bays

The BL685c G7  is the newest AMD offering in a full-height blade that comes with:

• 4 x AMD Opteron 6100 (up to 12 cores)
• 4 x 10Gb FlexFabric NICs LOM (LAN on Motherboard)
• Max of 512 GB (across 32 DIMM slots)
• 3 x I/O mezzanine expansion card slots
• 2 x hot-swap drive bays

I’ll be talking about FlexFabric in the near future, but for now think of it as HP’s version of the Converged Network Adapter.  You can read more  at www.hp.com/go/flexfabric.

Here are the totals (sorry, too much info for fancy summarizing:)

BL680c G5

BL680c G7

BL685c G6

BL685c G7

You can find out more about HP’s blade servers at HP.com.

IBM

The IBM HX5 was announced March 2010.   The HX5 is a 2 CPU blade server that uses a modular approach allowing users to add a memory blade (called the MAX5) or an 2nd HX5 to create a 4 CPU offering.  While I covered the technology back in March (check it out here) I’m looking at the 2 x HX5 combination as IBM’s 4 socket offering.

The IBM HX5  is a double-wide (2 x 30mm) blade that comes with:

• 4 x Intel Xeon 7500 CPUs (up to 8 cores each)
• 4 x 1Gb Ethernet NICs LOM (LAN on Motherboard)
• Max of 256GB (across 32 DIMM slots)
• 4 x I/O mezzanine expansion card slots
• 4 x internal drive bays

Each IBM BladeCenter H chassis can hold 7 x HX5 blade servers for a total of 28 HX5 blade servers per 42u rack.  To save you from doing the math, that’s a max of 896 CPU cores and 7.1TB of memory per 42u rack.

Summary
Here’s a summary of all of the current 4 socket blade servers (click to enlarge):

Here’s a summary of the top offerings (shown in yellow on the chart):

Most CPU Cores in a Blade Server:  HP BL685c G7
Most I/O Expansion in a Blade Server: HP BL680c G7
Most Memory in a Blade Server: HP BL680c G7

Most Memory in a 42u Rack: (tie) Dell M910, HP BL680c G7, HP BL685c G7
Most CPU Cores in a 42u Rack: HP BL685c G7

Yes, I know that there are slim chances that anyone would fill up a rack with 4 socket servers, however I thought this would be good comparison to make. What are your thoughts? Let me know in the comments below.

A white paper released today by Dell shows that the Dell M1000e blade chassis infrastructure offers significant power savings compared to equivalent HP and IBM blade environments. In fact, the results were audited by an outside source, the Enterprise Management Associates (http://www.enterprisemanagement.com). After the controversy with the Tolly Group report discussing HP vs Cisco, I decided to take the time to investigate these findings a bit deeper.

The Dell Technical White Paper titled, “Power Efficiency Comparison of Enterprise-Class Blade Servers and Enclosures” was written by the Dell Server Performance Analysis Team. This team is designed to run competitive comparisons for internal use, however the findings of this report were decided to be published external to Dell since the results were unexpected. The team used an industry standard SPECpower_ssj2008 benchmark to compare the power draw and performance per watt of blade solutions from Dell, HP and IBM. SPECpower_ssj2008 is the first industry-standard benchmark created by the Standard Performance Evaluation Corporation (SPEC) that evaluates the power and performance characteristics of volume server class and multi-node class computers. According to the white paper, the purpose of using this benchmark was to establish a level playing field to examine the true power efficiency of the Tier 1 blade server providers using identical configurations.

What Was Tested

Each blade chassis was fully populated with blade servers running a pair of Intel Xeon X5670 CPUs. In the Dell configuration, 16 x M610 blade servers were used, in the HP configuration, 16 x BL460c G6 blade servers were used and in the IBM configuration, 14 x HS22 blade servers was used since the IBM BladeCenter H holds a x maximum of 14 servers. Each server was configured with 6 x 4GB (24GB total) and 2 x 73GB 15k SAS drives, running Microsoft Windows Server 2008 Enterprise R2. Each chassis used the maximum amount of power supplies – Dell: 6, HP: 6 and IBM: 4 and was populated with a pair of Ethernet Pass-thru modules in the first two I/O bays.

Summary of the Findings

I don’t want to re-write the 48 page technical white paper, so I’ll summarize the results.

• While running the CPUs at 40 – 60% utilization, Dell’s chassis used 13 – 17% less power than the HP C7000 with 16 x BL460c G6 servers
• While running the CPUs at 40 – 60% utilization, Dell’s chassis used 19 – 20% less power than the IBM BladeCenter H with 14 x HS22s
• At idle power, Dell’s chassis used 24% less power than the HP C7000 with 16 x BL460c G6 servers
• At idle power, Dell’s chassis used 63.6% less power than the IBM BladeCenter H with 14 x HS22s

Following a review of the findings I had the opportunity to interview Dell’s Senior Product Manager for Blade Marketing, Robert Bradfield, , where I asked some questions about the study.

Question – “Why wasn’t Cisco’s UCS included in this test?”

Answer – The Dell testing team didn’t have the right servers. They do have a Cisco UCS, but they don’t have the UCS blade servers that would equal the BL460 G6 or the HS22’s.

Question – “Why did you use pass-thru modules for the design, and why only two?”

Answer – Dell wanted to create a level playing field. Each vendor has similar network switches, but there are differences. Dell did not want for those differences to impact the testing at all, so they chose to go with pass-thru modules. Same reason as to why they didn’t use more than 2. With Dell having 6 I/O bays, HP having 8 I/O bays and IBM having 8 I/O bays, it would have been challenging to create an equal environment to measure the power accurately.

Question – “How long did it take to run these tests?”

Answer – It took a few weeks. Dell placed all 3 blade chassis side-by-side but they only ran the tests on one chassis at a time. They wanted to give the test in progress absolute focus. In fact, the two chassis that were not being tested were not running at all (no power) because the testing team wanted to ensure there were no thermal variations.

Question – “Were the systems on a bench, or did you have them racked?”

Answer – All 3 chassis were racked – in their own rack. They were properly cooled with perforated doors with vented floor panels under the floor. In fact, the temperatures never varied by 1 degree between all enclosures.

Question – “Why do you think the Dell design offered the lowest power in these tests?”

Answer – There are three contributing factors to the success of Dell’s M1000e chassis offering a lower power draw over HP and IBM. The first is the 2700W Platinum certified power supply. It offers greater energy efficiency over previous power supplies and they are shipping as a standard power supply in the M1000e chassis now. However, truth be told, the difference in “Platinum” certified and “Gold” certified is only 2 – 3%, so this adds very little to the power savings seen in the white paper. Second is the technology of the Dell M1000e fans. Dell has patent pending fan control algorithms that help provide better fan efficiency. From what I understand this patent helps to ensure that at no point in time does the fan rev up to “high”. (If you are interested in reading about the patent pending fan control technology, pour yourself a cup of coffee and read all about it at the U.S. Patent Office website – application number 20100087965). Another interesting fact is that the fans used in the Dell M1000e are balanced by the manufacturer to ensure proper rotation. It is a similar process to the way your car tires are balanced – there is one or two small weights on each fan. (This is something you can validate if you own a Dell M1000e). Overall, it really comes down to the overall architecture of the Dell M1000e chassis being designed for efficient laminar airflow. In fact (per Robert Bradfield) when you look at the Dell M1000e as tested in this technical white paper versus the IBM BladeCenter H, the savings in power realized in a one year period would be enough power saved to power a single U.S. home for one year.

I encourage you, the reader, to review this Technical White Paper (Power Efficiency Comparison of Enterprise-Class Blade Servers and Enclosures) for yourself and see what your thoughts are. I’ve looked for things like use of solid state drives or power efficient memory DIMMs, but this seems to be legit. However I know there will be critics, so voice your thoughts in the comments below. I promise you Dell is watching to see what you think…

The Washington Post has a nice article today from Logan G. Harbaugh of PC World about blade servers. In the article, Logan takes the discussion of blade servers back to the basics. The piece covers the pro’s and con’s of blade servers as well as some good comparisons between blade and rack servers.

While I don’t agree with all of his statements, it is a good read so I encourage you to take 5 minutes and check it out.

Here is the link:
http://www.washingtonpost.com/wp-dyn/content/article/2010/09/02/AR2010090200060.html

Cisco’s Unified Computing System (UCS) is a bit unique in that the chassis is a small component of the overall offering. Cisco’s UCS is a “system” of components that consists of blade servers, blade chassis, fabric extenders and fabric interconnects. The blade chassis is called the UCS 5100. It is a 6 rack unit (6u) tall chassis that can hold anywhere from 4 to 8 blade servers (dependent upon the blade form factor). The chassis comes with 4 front-accessible 2500W single-phase, hot-swappable power supplies that are 92 percent efficient and can be configured to support non-redundant, N+1 redundant, and grid-redundant configurations.

The rear of the UCS5100 chassis offers 4 hot-swap blowers, 4 power plug connectors requiring 15.5A, 220-240V AC. There are also a pair of redundant fabric extenders. This is where Cisco’s design differs from everyone else. These “fabric extenders”, known as UCS 2104XP Fabric Extender simply extend the reach of the onboard 10Gb or Converged Network Adapters (CNAs) I/O fabric from the blade server bays to the management console, known as the fabric interconnect. I previously blogged that there were rumours at one time that there would be an 8 port version of the fabric extender, however to date, I have not seen any proof of this. The UCS 2104XP Fabric Extender provides 4 x 10Gb uplinks, so if you have 8 blade servers, you theoretically would be looking at a 2:1 ratio (8 blade servers to 4 uplinks.) There have been several comments and blog posts on the functionality of the fabric extender, including the infamous Tolly Report that received several comments from the Tolly Group, Cisco employees and HP employees – but in summary, the 4 x 10Gb uplinks are adequate for handling all the I/O that the max 8 blade servers can throw at it. Yes, you can put two in for redundant pathways as well. The Fabric Extenders connect in to the brains of the solution – the fabric interconnect. 

The function of the UCS 6100 fabric interconnect is to connect ALL of the UCS 5100 chassis to the network and storage fabrics.The Cisco UCS 6100 series fabric interconnect currently comes in two flavors – a 20 port (UCS 6120XP) and a 40 port (UCS 6140XP). A 20 port could connect 5 x UCS5100 chassis’ fabric extenders (4 ports x 5 chassis = 20) all the way up to 20 x UCS5100 (1 port per fabric extender). This last example doesn’t seem to be ideal, as you would be running up to 8 x blade servers’ 10Gb I/O traffic up a single 10Gb uplink – but, who knows – I’m not a networking guy, so I’ll have to leave those comments to the experts. Personally, I think that it’s a bunch of marketing fluff… 

When we look at the sheer capacity of the quantity of blade servers that you can fit into a 42u rack, we see that Cisco can offer a maximum of 7 chassis into a rack (6u tall). The stats below provide a good comparison between the different server offerings from Cisco. 

A few things to point out:

• there are no AMD options (Intel only)
• half-width blade servers have a 1 x I/O card whereas full-width blade servers have 2 x I/O cards    

I/O Card Options
Cisco offers 4 different I/O Network Card options for their blade servers:

• Cisco UCS 82598KR-CI 10 Gigabit Ethernet Adapter  – based on the Intel 82598 10 Gigabit Ethernet controller, which is designed for efficient high-performance Ethernet transport.
• Cisco UCS M71KR-E Emulex Converged Network Adapter – uses an Intel 82598 10 Gigabit Ethernet controller for network traffic and an Emulex 4-Gbps Fibre Channel controller for Fibre Channel traffic all on the same mezzanine card.
• Cisco UCS M71KR-Q QLogic Converged Network Adapter – uses an Intel 82598 10 Gigabit Ethernet controller for network traffic and a QLogic 4-Gbps Fibre Channel controller for Fibre Channel traffic, all on the same mezzanine card.
• Cisco UCS M81KR Virtual Interface Card -  a dual-port 10 Gigabit Ethernet mezzanine card that supports up to 128 virtual interfaces that can be dynamically configured so that both their interface type (network interface card [NIC] or host bus adapter [HBA]) and identity (MAC address and worldwide name [WWN]) are established using just-in-time provisioning.

Fellow blogger, Sean McGee, has written up a nice post on the Cisco UCS B-Series I/O Card Options.  I recommend you go read it (after you finish this post).  You can find Sean’s article here.

  (For more details on these card options, please visit http://www.cisco.com/en/US/products/ps10280/products_data_sheets_list.html)

 As you may notice, there are no card options with fibre-channel only or Infiniband.  This is part of Cisco’s UCS strategy – the network and the storage traffic travel over the same cable from the blade server though the fabric extender to the fabric interconnect where the traffic is separated into network fabrics and storage fabrics.  This design allows for Cisco to require a maximum of 8 cables (4 from each fabric extender) per blade chassis and as few as 2 cables (1 per fabric extender).  Compared to a traditional server environment using multiple 1Gb Ethernet and 4Gb fibre connections per connection, there is a huge savings in cables.

Chassis Switch Options

 As I have previously mentioned, the architecture of Cisco’s UCS blade environment takes an approach of “extending” the I/O connectivity from the blades to the fabric interconnect.  With this design, there are no “switches”, therefore there are no switch options.

Server Management

Cisco’s blade infrastructure management lies within the Cisco UCS 6100 fabric interconnect.  The base management software, called UCS Manager, is the central point of management for the entire UCS environment.  It manages the UCS system, including the blades, the chassis, and the network (both LAN and SAN) – configuration, environmentals, etc.  Take a few minutes to look at the UCS Manager software in this short video:

While the UCS Manager is rich in features, it does have the following limitations:

•(Hardware) Templates can NOT be shared across systems
•Available MAC addresses are scoped per UCS Manager instance (not across the Enterprise)
•Available WWN addresses are scoped per UCS Manager instance (not across the Enterprise)
•Available UUIDs are scoped per UCS Manager instance (not across the Enterprise)

Bottom line – the UCS Manager is limited to each UCS chassis and most of the features are manual steps.  Never fear, however.  Cisco offers BMC’s BladeLogic which adds the following:

•UCS template creation and editing
•Cross-UCS template management
•Cross-UCS MAC and WWN Management
•Local disk provisioning for UCS
•SAN provisioning for UCS
•ESX provisioning for UCS
•Consolidated UCS operator and management action
•Manages UCS resources, VMs, guest OS, and business applications

The catch, however, is that the BMC BladeLogic is an extra cost.  How much – I’m not sure, but it adds something…  Cisco has a really good simulator that highlights what you can do with the UCS Manager software, so if you are interested, take a few minutes to watch.  There is no narration, just a walk-through of the UCS Manager.  I also recommend you view in full screen:

So let me know what you think.  Is there anything I’m missing – anything else you would like to see on this?  Let me know in the comments below.  Make sure to keep an eye on this site as I’ll be posting information on Dell, HP and IBM in the following weeks.

“Oo-rah” (that’s for @jonisick) 

Updated 5/24/2010 – I’ve received some comments about expandability and I’ve received a correction about the speed of Dell’s memory, so I’ve updated this post.  You’ll find the corrections / additions below in GREEN.

Since I’ve received a lot of comments from my post on the Dell FlexMem Bridge technology, I thought I would  do an unbiased comparison between Dell’s FlexMem Bridge technology (via the PowerEdge 11G M910 blade server) vs IBM’s MAX5 + HX5 blade server offering.  In summary both offerings provide the Intel Xeon 7500 CPU plus the ability to add “extended memory“ offering value for virtualization, databases and any other workloads that benefit from large amounts of memory.

The Contenders

IBM
IBM’s extended memory solution is a two part solution consisting of the HX5 blade server PLUS the MAX5 memory blade.

• HX5 Blade Server
  I’ve spent considerable time on previous blogs detailing the IBM HX5, so please jump over to those links to dig into the specifics, but at a high level, the HX5 is IBM’s 2 CPU blade server that offers the Intel Xeon 7500 CPU.   The HX5 is a 30mm, ”single wide” blade server therefore you can fit up to 14 in an IBM BladeCenter H blade chassis. 
• MAX5
  The MAX 5 offering from IBM can be thought of as a “memory expansion blade.”  Offering an additional 24 memory DIMM slots, the MAX5 when coupled with the HX5 blade server, provides a total of 40 memory DIMMs.    The MAX5 is a standard “single wide”, 30mm form factor so when used with a single HX5 two IBM BladeCenter H server bays are required in the chassis.

DELL
Dell’s approach to extended memory is a bit different.  Instead of relying on a memory blade, Dell starts with the M910 blade server and allows users to use 2 CPUs plus their FlexMem Bridge to access the memory DIMMs of the 3rd and 4th CPU sockets.  For details on the FlexMem Bridge, check out my previous post.

• PowerEdge 11G M910 Blade Server
  The M910 is a 4 CPU capable blade server with 32 memory DIMMs.  This blade server is a full-height server therefore you can fit 8 servers inside the Dell M1000e blade chassis.

The Face-Off

ROUND 1 – Memory Capacity
When we compare the memory DIMMs available on each, we see that Dell’s offering comes up with 32 DIMMs vs IBM’s 40 DIMMs.  However, IBM’s solution of using the HX5 blade server + the MAX 5 memory expansion has a current maximum memory size is 8Gb whereas Dell offers a max memory size of 16Gb.  While this may change in the future, as of today, Dell has the edge so I have to claim: 

Round 1 Winner:  Dell

ROUND 2 – Memory Performance
As many comments came across on my posting of the Dell FlexMem Bridge technology the other day, several people pointed out that the memory performance is something that needs to be considered when comparing technologies.  Dell’s FlexMem Bridge offering reportedly runs at a maximum memory speed of 833Mhz,  runs at a max of 1066Ghz, but is dependent upon the speed of the processor.  A processor that has a 6.4GT QPI supports memory @ 1066Ghz ; a processor that supports 5.8GT/s QPI supports memory at 978Mhz, and a processor with a QPI speed of 4.8GT runs memory at 800Mhz.  This is a component of Intel’s Xeon 7500 architecture so it should be the same regardless of the server vendor.  Looking at IBM, we see the  HX5 blade server memory runs at a maximum of 978Mhz.    However, when you attach the MAX5 to the HX5 for the additional memory slots, however, the memory runs at speed of 1066Mhz, regardless of the speed of the CPU installed.  While this appears to be black magic, it’s really the results of IBM’s proprietary eXa scaling – something that I’ll cover in detail at a later date.   Although the HX5 blade server memory, when used by itself, does not have the ability to achieve 1066Ghz, this comparison is based on the Dell PowerEdge 11G M910 vs the IBM HX5+MAX5.  With that in mind, the ability to run the expanded memory at 1066Mhz gives IBM the edge in this round.

Round 2 Winner:  IBM

ROUND 3 – Server Density
This one is pretty straight forward.  IBM’s HX5 + MAX5 offering takes up 2 server bays, so in the IBM BladeCenter H, you can only fit 7 systems.  You can only fit 4 BladeCenter H chassis in a 42u rack, therefore you can fit a max of 28  IBM HX5 + MAX5 systems into a rack.

The Dell PowerEdge 11G M910 blade server is a full height server, so you can fit 8 servers into the Dell M1000e chassis.  4 Dell chassis will fit in a 42u rack, so you can get 32 Dell M910′s into a rack.

Round 3 Winner:  Dell

(NEW) ROUND 4 – Expandability
It was mentioned several times in the comments that expandability should have been reviewed as well.  When we look at Dell’s design, we see there two expansion options: run the Dell PowerEdge 11G M910 blade with 2 processors and the FlexMem Bridge, or run them with 4 processors and remove the FlexMem Bridge.

The modular design of the IBM eX5 architecture allows for a user to add memory (MAX5), add processors (2nd HX5) or both (2 x HX5 + 2 x MAX5).  This provide users with a lot of flexibility to choose a design that meets their workload.

Choosing a winner for this round is tough, as there a different ways to look at this:

Maximum CPUs in a server: TIE – both IBM and Dell can scale to 4 CPUs. 
Maximum CPU density in a 42u rack:  Dell wins with 32 x 4 CPU servers vs IBM’s 12.
Maximum Memory in a server: IBM with 640Gb using 2 x HX5 and 2 x MAX5
Max Memory density in a 42u Rack: Dell wins with 16Tb

Round 4 Winner:  TIE
Summary
While the fight was close, with a 2 to 1 win, it is clear the overall winner is Dell.  For this comparison, I tried to keep it focused on the memory aspect of the offerings. 

On a final note, at the time of this writing, the IBM MAX 5 memory expansion has not been released for general availability, while Dell is shipping their M910 blade server. 

There may be other advantages relative to processors that were not considered for this comparison, however I welcome any thoughts or comments you have. 

Cisco recently announced their first blade offering with the Intel Xeon 7500 processor, known as the ”Cisco UCS B440-M1 High-Performance Blade Server.”  This new blade is a full-width blade that offers 2 – 4 Xeon 7500 processors and 32 memory slots, for up to 256GB RAM, as well as 4 hot-swap drive bays.  Since the server is a full-width blade, it will have the capability to handle 2 dual-port mezzanine cards for up to 40 Gbps I/O per blade. 

Each Cisco UCS 5108 Blade Server Chassis can house up to four B440 M1 servers (maximum 160 per Unified Computing System). 

How Does It Compare to the Competition?
Since I like to talk about all of the major blade server vendors, I thought I’d take a look at how the new Cisco B440 M1 compares to IBM and Dell.  (HP has not yet announced their Intel Xeon 7500 offering.)

Processor Offering
Both Cisco and Dell offer models with 2 – 4 Xeon 7500 CPUs as standard.  They each have variations on speeds – Dell has 9 processor speed offerings; Cisco hasn’t released their speeds and IBM’s BladeCenter HX5 blade server will have 5 processor speed offerings initially.  With all 3 vendors’ blades, however, IBM’s blade server is the only one that is designed to scale from 2 CPUs to 4 CPUs by connecting 2 x HX5 blade servers.  Along with this comes their “FlexNode” technology that enables users to have the 4 processor blade system to split back into 2 x 2 processor systems at specific points during the day.  Although not announced, and purely my speculation, IBM’s design also leads to a possible future capability of connecting 4 x 2 processor HX5′s for an 8-way design.  Since each of the vendors offer up to 4 x Xeon 7500′s, I’m going to give the advantage in this category to IBM.  WINNER: IBM

Memory Capacity
Both IBM and Cisco are offering 32 DIMM slots with their blade solutions, however they are not certifying the use of 16GB DIMMs – only 4GB and 8GB DIMMs, therefore their offering only scales to 256GB of RAM.  Dell claims to offers 512GB DIMM capacity on their the PowerEdge 11G M910 blade server, however that is using 16GB DIMMs.  REalistically, I think the M910 would only be used with 8GB DIMMs, so Dell’s design would equal IBM and Cisco’s.  I’m not sure who has the money to buy 16GB DIMMs, but if they do – WINNER: Dell (or a TIE)

Server Density
As previously mentioned, Cisco’s B440-M1 blade server is a “full-width” blade so 4 will fit into a 6U high UCS5100 chassis.  Theoretically, you could fit 7 x UCS5100 blade chassis into a rack, which would equal a total of 28 x B440-M1′s per 42U rack.Overall, Cisco’s new offering is a nice addition to their existing blade portfolio.  While IBM has some interesting innovation in CPU scalability and Dell appears to have the overall advantage from a server density, Cisco leads the management front. 

Dell’s PowerEdge 11G M910 blade server is a “full-height” blade, so 8 will fit into a 10u high M1000e chassis.  This means that 4 x M1000e chassis would fit into a 42u rack, so 32 x Dell PowerEdge M910 blade servers should fit into a 42u rack.

IBM’s BladeCenter HX5 blade server is a single slot blade server, however to make it a 4 processor blade, it would take up 2 server slots.  The BladeCenter H has 14 server slots, so that makes the IBM solution capable of holding 7 x 4 processor HX5 blade servers per chassis.  Since the chassis is a 9u high chassis, you can only fit 4 into a 42u rack, therefore you would be able to fit a total of 28 IBM HX5 (4 processor) servers into a 42u rack.
WINNER: Dell

Management
The final category I’ll look at is the management.  Both Dell and IBM have management controllers built into their chassis, so management of a lot of chassis as described above in the maximum server / rack scenarios could add some additional burden.  Cisco’s design, however, allows for the management to be performed through the UCS 6100 Fabric Interconnect modules.  In fact, up to 40 chassis could be managed by 1 pair of 6100′s.  There are additional features this design offers, but for the sake of this discussion, I’m calling WINNER: Cisco.

Cisco’s UCS B440 M1 is expected to ship in the June time frame.  Pricing is not yet available.  For more information, please visit Cisco’s UCS web site at http://www.cisco.com/en/US/products/ps10921/index.html.

InfoWorld.com posted on 3/22/2010 the results of a blade server shoot-out between Dell, HP, IBM and Super Micro. I’ll save you some time and help summarize the results of Dell, HP and IBM.

The Contenders
Dell, HP and IBM each provided blade servers with the Intel Xeon X5670 2.93GHz CPUs and at least 24GB of RAM in each blade.

The Tests
InfoWorld designed a custom suite VMware tests as well as several real-world performance metric tests. The VMware tests were composed of:

• a single large-scale custom LAMP application
• a load-balancer running Nginx
• four Apache Web servers
• two MySQL servers

InfoWorld designed the VMware workloads to mimic a real-world Web app usage model that included a weighted mix of static and dynamic content, randomized database updates, inserts, and deletes with the load generated at specific concurrency levels, starting at 50 concurrent connections and ramping up to 200.  InfoWorld’s started off with the VMware tests first on one blade server, then across two blades. Each blade being tested were running VMware ESX 4 and controlled by a dedicated vCenter instance.

The other real-world tests included serveral tests of common single-threaded tasks run simultaneously at levels that met and eclipsed the logical CPU count on each blade, running all the way up to an 8x oversubscription of physical cores. These tests included:

• LAME MP3 conversions of 155MB WAV files
• MP4-to-FLV video conversions of 155MB video files
• gzip and bzip2 compression tests
• MD5 sum tests

The ResultsDell
Dell did very well, coming in at 2nd in overall scoring.  The blades used in this test were Dell PowerEdge M610 units, each with two 2.93GHz Intel Westmere X5670 CPUs, 24GB of DDR3 RAM, and two Intel 10G interfaces to two Dell PowerConnect 8024 10G switches in the I/O slots on the back of the chassis

Some key points made in the article about Dell:

• Dell does not offer a lot of “blade options.”  There are several models available, but they are the same type of blades with different CPUs.  Dell does not currently offer any storage blades or virtualization-centric blades.
• Dell’s 10Gb design does not offer any virtualized network I/O. The 10G pipe to each blade is just that, a raw 10G interface.  No virtual NICs.
• The new CMC (chassis management controller) is a highly functional and attractive management tool offering new tasks like pusing actions to multiple blades at once such as BIOS updates and RAID controller firmware updates.
• Dell has implemented more efficient dynamic power and cooling features in the M1000e chassis. Such features include the ability to shut down power supplies when the power isn’t needed, or ramping the fans up and down depending on load and the location of that load.

According to the article, “Dell offers lots of punch in the M1000e and has really brushed up the embedded management tools. As the lowest-priced solution…the M1000e has the best price/performance ratio and is a great value.”

HP
Coming in at 1st place, HP continues to shine in blade leadership.  HP’s testing equipment consisted of a c7000 nine BL460c blades, each running two 2.93GHz Intel Xeon X5670 (Westmere-EP) CPUs and 96GB of RAM as well as embedded 10G NICs with a dual 1G mezzanine card.  As an important note, HP was the only server vendor with 10G NICs on the motherboard.  Some key points made in the article about HP:

•  With the 10G NICs standard on the newest blade server models, InfoWorld says ”it’s clear that HP sees 10G as the rule now, not the exception.”
• HP’s embedded Onboard Administrator offers detailed information on all chassis components from end to end.  For example, HP’s management console can provide exact temperatures of every chassis or blade component.
• HP’s console can not offer  global BIOS and firmware updates (unlike Dell’s CMC) or the ability to powering up or down more than one blade at a time.
• HP offers “multichassis management” – the ability to daisy-chain several chassis together and log into any of them from the same screen as well as manage them.  This appears to be a unique feature to HP.
• The HP c7000 chassis also has power controlling features like dynamic power saving options that will automatically turn off power supplies when the system energy requirements are low or increasing the fan airflow to only those blades that need it.

InfoWorld’s final thoughts on HP: “the HP c7000 isn’t perfect, but it is a strong mix of reasonable price and high performance, and it easily has the most options among the blade system we reviewed.”

IBM
Finally, IBM’s came in at 3rd place, missing a tie with Dell by a small fraction.  Surprisingly, I was unable to find the details on what the configuration was for IBM’s testing.  Not sure if I’m just missing it, or if InfoWorld left out the information, but I know IBM’s blade server had the same Intel Xeon X5670 CPUs as Dell and HP used.   Some of the points that InfoWorld mentioned about IBM’s BladeCenter H offering:

• IBM’s pricing is higher.
• IBM’s chassis only holds 14 servers whereas HP can hold 32 servers (using BL2x220c servers) and Dell holds 16 servers.
• IBM’s chassis doesn’t offer a heads-up display (like HP and Dell.)
• IBM had the only redundant internal power and I/O connectors on each blade.  It is important to note the lack of redundant power and I/O connectors is why HP and Dell’s densities are higher.  If you want redundant connections on each blade with HP and Dell, you’ll need to use their “full-height” servers, which decrease HP and Dell’s overall capacity to 8.
• IBM’s Management Module is lacking graphical features – there’s no graphical representation of the chassis or any images.  From personal experience, IBM’s management module looks like it’s stuck in the ’90s – very text based.
• The IBM BladeCenter H lacks dynamic power and cooling capabilities.  Instead of using smaller independent regional fans for cooling, IBM uses two blowers.  Because of this, the ability to reduce cooling in specific areas, like Dell and HP offer are lacking.

InfoWorld summarizes the IBM results saying, “ if you don’t mind losing two blade slots per chassis but need some extra redundancy, then the IBM BladeCenter H might be just the ticket.”

Overall, each vendor has their own pro’s and con’s.  InfoWorld does a great job summarizing the benefits of each offering below.  Please make sure to visit the InfoWorld article and read all of the details of their blade server shoot-out.

IBM BladeCenter H

Cisco UCS 5108

Server Density and Server Offerings: winner in this category is IBM. IBM’s BladeCenter E and BladeCenter H chassis offer up to 14 blade servers with servers using Intel, AMD and Power PC processors. In comparison, Cisco’s 5808 chassis offers up to 8 server slots and currently offers servers with Intel Xeon processors. As an honorable mention Cisco does offer a “full width” blade (Cisco UCS B250 server) that provides up to 384Gb of RAM in a single blade server across 48 memory slots offering up the ability to get to higher memory at a lower price point.

 Management / Scalability: winner in this category is Cisco.
This is where Cisco is changing the blade server game. The traditional blade server infrastructure calls for each blade chassis to have its own dedicated management module to gain access to the chassis’ environmentals and to remote control the blade servers. As you grow your blade chassis environment, you begin to manage multiple servers. Beyond the ease of managing , the management software that the Cisco 6100 series offers provides users with the ability to manage server service profiles that consists of things like MAC Addresses, NIC Firmware, BIOS Firmware, WWN Addresses, HBA Firmware (just to name a few.)

Cisco UCS 6100 Series Fabric Interconnect

With Cisco’s UCS 6100 Series Fabric Interconnects, you are able to manage up to 40 blade chassis with a single pair of redundant UCS 6140XP (consisting of 40 ports.)

If you are familiar with the Cisco Nexus 5000 product, then understanding the role of the Cisco UCS 6100 Fabric Interconnect should be easy. The UCS 6100 Series Fabric Interconnect do for the Cisco UCS servers what Nexus does for other servers: unifies the fabric. HOWEVER, it’s important to note the UCS 6100 Series Fabric Interconnect is NOT a Cisco Nexus 5000. The UCS 6100 Series Fabric Interconnect is only compatible with the UCS servers.

Cisco UCS I/O Connectivity Diagram (UCS 5108 Chassis with 2 x 6120 Fabric Interconnects)

If you have other servers, with CNAs, then you’ll need to use the Cisco Nexus 5000.

The diagram on the right shows a single connection from the FEX to the UCS 6120XP, however the FEX has 4 uplinks, so if you want (need) more throughput, you can have it. This design provides each half-wide Cisco B200 server with the ability to have 2

CNA ports with redundant pathways. If you are satisified with using a single FEX connection per chassis, then you have the ability to scale up to 20 x blade chassis with a Cisco UCS 6120 Fabric Interconnect, or 40 chassis with the Cisco UCS 6140 Fabric Interconnect. As hinted in the previous section, the management software for the all connected UCS chassis resides in the redundant Cisco UCS 6100 Series Fabric Interconnects. This design offers a highly scaleable infrastructure that enables you to scale simply by dropping in a chassis and connecting the FEX to the 6100 switch. (Kind of like Lego blocks.)

On the flip side, while this architecture is simple, it’s also limited. There is currently no way to add additional I/O to an individual server. You get 2 x CNA ports per Cisco B200 server or 4 x CNA ports per Cisco B250 server.

As previously mentioned, IBM has a strategy that is VERY similar to the Cisco UCS strategy using the Cisco Nexus 5000 product line with pass-thru modules. IBM’s solution consists of:

• IBM BladeCenter H Chassis
• 10Gb Pass-Thru Module
• CNA’s on the blade servers

Even though IBM and Cisco designed the Cisco Nexus 4001i  switch that integrates into the IBM BladeCenter H chassis, using a 10Gb pass-thru module ”may” be the best option to get true DataCenter Ethernet (or Converged Enhanced Ethernet) from the server to the Nexus switch – especially for users looking for the lowest cost. The performance for the IBM solution should equal the Cisco UCS design, since it’s just passing the signal through, however the connectivity is going to be more with the IBM solution. Passing signals through means NO cable

BladeCenter H Diagram with Nexus 5010 (using 10Gb Passthru Modules)

consolidation – for every server you’re going to need a connection to the Nexus 5000. For a fully populated IBM BladeCenter H chassis, you’ll need 14 connections to the Cisco Nexus 5000. If you are using the Cisco 5010 (20 ports) you’ll eat up all but 6 ports. Add a 2nd IBM BladeCenter chassis and you’re buying more Cisco Nexus switches. Not quite the scaleable design that the Cisco UCS offers.

IBM also offers a 10Gb Ethernet Switch Option from BNT (Blade Networks) that will work with converged switches like the Nexus 5000, but at this time that upgrade is not available. Once it does become available, it would reduce the connectivity requirements down to a single cable, but, adding a switch between the blade chassis and the Nexus switch could bring additional management complications. Let me know your thoughts on this.

IBM’s BladeCenter H (BCH) does offer something that Cisco doesn’t – additional I/O expansion. Since this solution uses two of the high speed bays in the BCH, bays 1, 2, 3 & 4 remain available. Bays 1 & 2 are mapped to the onboard NICs on each server, and bays 3&4 are mapped to the 1st expansion card on each server. This means that 2 additional NICs and 2 additional HBAs (or NICs) could be added in conjunction with the 2 CNAs on each server. Based on this, IBM potentially offers more I/O scalability.

And the Winner Is…

It depends. I love the concept of the Cisco UCS platform. Servers are seen as processors and memory – building blocks that are centrally managed. Easy to scale, easy to size. However, is it for the average datacenter who only needs 5 servers with high I/O? Probably not. I see the Cisco UCS as a great platform for datacenters with more than 14 servers needing high I/O bandwidth (like a virtualization server or database server.) If your datacenter doesn’t need that type of scalability, then perhaps going with IBM’s BladeCenter solution is the choice for you. Going the IBM route gives you flexibility to choose from multiple processor types and gives you the ability to scale into a unified solution in the future. While ideal for scalability, the IBM solution is currently more complex and potentially more expensive than the Cisco UCS solution.

Let me know what you think. I welcome any comments.