Monthly Archives: September 2009

Cisco Announces Nexus 4000 Switch for Blade Chassis

Cisco is announcing today the release of the Nexus 4000 switch.  It will be designed to work in “other” blade vendors’ chassis, although they aren’t announcing which blade vendors.  My gut is that Dell and IBM will OEM it, but HP will stick with their ProCurve line announced a few weeks ago.  Here’s what I know about the Nexus 4000 switch:

1) It will aggregate 1GB links to 10Gb uplink.  To me, this means that it will not be compatible with Converged Network Adapters (CNAs).  From this description, it seems to be just the Cisco Nexus 2000 in a blade form factor.  It’s simply a “fabric extender” allowing all of the traffic to flow into the Nexus 5000 Switch.

2) It will run on the Nexus O/S (NX-OS)  This is key because it allows users to have a seamless environment for their server and their Nexus switch infrastructure.

3) Cisco Nexus 4000 will provide “cost effective transition from multiple 1GbE links to a lossless 10GbE for virtualized environments”  This statement confuses me.  Does it mean that the Cisco Nexus 4000 switch will be capable of working with 1Gb NICs as well as 10Gb CNAs, or is it just stating that the traditional 1Gb NICs will be able to connect into a lossless unified fabric??

Cisco is having a live broadcast at 10 am PST today, but I just reviewed the slide deck and they talk at a VERY high level on this new announcement.  I suppose maybe they are going to let each vendor (Dell and IBM) provide details once they officially announce their switches.  When they do, I’ll post details here.

IBM Announces 4 Socket Intel Blade Server-UPDATED

IBM announced last week they will be launching a new blade server modeled with the upcoming 4 socket Intel Nehalem EX.  While details have not yet been provided on this new server, I wanted to provide an estimation of what this server could look like, based on previous IBM models.  I’ve drawn up what I think it will look like below, but first let me describe it.

“New Server Name”
IBM’s naming schema is pretty straight forward: Intel blades are “HS”, AMD blades are “LS”, Power blades are “JS”.  Knowing this, I the new server will most likely be called a “HS42“.  IBM previously had an HS40 and HS41, so calling it an HS42 would make the most sense. 

“Size
With the amount of memory that each CPU will have access to, I don’t see any way for IBM to create a 4 socket blade that wasn’t a “double-wide” form factor.  A “double-wide” design means the server is 2 server slots wide, so in a single IBM BladeCenter H chassis, customers would be limited to 7  x HS42’s per chassis.

“Memory”
The Intel Nehalem EX will tentatively support 16 memory slots PER CPU, across 4 memory channels, so a 4 socket server will have 64 memory slots.  Each memory channel can hold up to 4 DIMMs each.  This is great, but this is the MAX for an upcoming Intel Nehalem EX server.  I do not expect for any blade server vendor to achieve 64 memory slots with 4 CPUs.  Since this is the maximum, it makes sense that vendors, like IBM, will be able to use less memory.  I expect for these new servers to have 12 memory slots per CPUs (or 3 DIMMs per memory channel).  This will still provide 48 memory dimms per” HS42″ blade server; and with 16Gb DIMMs, that would equal 768Gb per blade server.

“CPU”
The “HS42” would have up to 4 x Intel Nehalem EX CPU’s, each with 8 cores, for a total of 32 CPU cores per “HS42” server.  HOWEVER, Intel is offering Hyperthreading with this CPU so an 8 core CPU now looks like 16 CPUs.

“Internal Drive Capacity”
I don’t see any way for IBM to have hot-swap drives in this server.  There is just not enough real estate.  So, I believe they would consider putting in Solid State drives (SSD’s) toward the front of the server.  Will they put it on both sides of the server, probably not.  The role of these drives would be just to provide space for your boot O/S.  The data will sit on a storage area network. 

“I/O Expansion”
I don’t think that IBM will re-design their existing I/O architecture for the blade servers.  Therefore, I expect for each side of the double-wide “HS42” to have a single CIOv and a CFF-h daughter card expansion slot, so a single HS42 would have 4 expansion slots.  This is assuming that IBM designs connector pins that interconnect the two halves of the server together that don’t interfere with the card slots (presumably at the upper half of the connections.)HS42 Estimation

As we come closer to the release date of the Intel Nehalem EX processor later in Q4 of 2009, I expect to hear more definitive details on the announced 4 socket IBM Blade server, so make sure to check back here later this year.

UPDATE (10/6/09):   I’m hearing rumors that IBM’s Nehalem EX processor offerings (aka “X5″ offerings” will be shipping in Q2 of 2010.)  Once that is confirmed by IBM, I’ll post a new post.

Cisco UCS vs IBM BladeCenter H

News Flash: Cisco is now selling servers!

Okay – perhaps this isn’t news anymore, but the reality is Cisco has been getting a lot of press lately – from their overwhelming presence at VMworld 2009 to their ongoing cat fight with HP.  Since I work for a Solutions Provider that sells HP, IBM and now Cisco blade servers, I figured it might be good to “try” and put together a comparison between the Cisco and IBM.  Why IBM?  Simply because at this time, they are the only blade vendor who offers a Converged Network Adapter (CNA) that will work with the Cisco Nexus 5000 line.  At this time Dell and HP do not offer a CNA for their blade server line so IBM is the closest we can come to Cisco’s offering.  I don’t plan on spending time educating you on blades, because if you are interested in this topic, you’ve probably already done your homework.  My goal with this post is to show the pros (+) and cons (-) that each vendor has with their blade offering – based on my personal, neutral observation

Chassis Variety / Choice: winner in this category is IBM.
IBM currently offers 5 types of blade chassis: BladeCenter S, BladeCenter E, BladeCenter H, BladeCenter T and BladeCenter HT.   Each of the IBM blade chassis have unique offerings, such as the BladeCenter S is designed for small or remote offices with local storage capabilities, whereas the BladeCenter HT is designed for Telco environments with options for NEBS compliant features including DC power.  At this time, Cisco only offers a single blade chassis offering (the 5808).

IBM BladeCenter H

IBM BladeCenter H

Cisco UCS 5108

Cisco UCS 5108

Server Density and Server Offerings: winner in this category is IBM.  IBM’s BladeCenter E and BladeCenter H chassis offer up to 14 blade servers with servers using Intel, AMD and Power PC processors.  In comparison, Cisco’s 5808 chassis offers up to 8 server slots and currently offers servers with Intel Xeon processors.  As an honorable mention Cisco does offer a “full width” blade (Cisco UCS B250 server)  that provides up to 384Gb of RAM in a single blade server across 48 memory slots offering up the ability to get to higher memory at a lower price point.  

 Management / Scalability: winner in this category is Cisco.
This is where Cisco is changing the blade server game.  The traditional blade server infrastructure calls for each blade chassis to have its own dedicated management module to gain access to the chassis’ environmentals and to remote control the blade servers.  As you grow your blade chassis environment, you begin to manage multiple servers. 
Beyond the ease of managing , the management software that the Cisco 6100 series offers provides users with the ability to manage server service profiles that consists of things like MAC Addresses, NIC Firmware, BIOS Firmware, WWN Addresses, HBA Firmware (just to name a few.) 

Cisco UCS 6100 Series Fabric Interconnect

Cisco UCS 6100 Series Fabric Interconnect

With Cisco’s UCS 6100 Series Fabric Interconnects, you are able to manage up to 40 blade chassis with a single pair of redundant UCS 6140XP (consisting of 40 ports.)

If you are familiar with the Cisco Nexus 5000 product, then understanding the role of the  Cisco UCS 6100 Fabric Interconnect should be easy.  The UCS 6100 Series Fabric Interconnect do for the Cisco UCS servers what Nexus does for other servers: unifies the fabric.   HOWEVER, it’s important to note the UCS 6100 Series Fabric Interconnect is NOT a Cisco Nexus 5000.  The UCS 6100 Series Fabric Interconnect is only compatible with the UCS servers. 

UCS Diagram

Cisco UCS I/O Connectivity Diagram (UCS 5108 Chassis with 2 x 6120 Fabric Interconnects)

If you have other servers, with CNAs, then you’ll need to use the Cisco Nexus 5000.

The diagram on the right shows a single connection from the FEX to the UCS 6120XP, however the FEX has 4 uplinks, so if you want (need) more throughput, you can have it.  This design provides each half-wide Cisco B200 server with the ability to have 2

CNA ports with redundant pathways.  If you are satisified with using a single FEX connection per chassis, then you have the ability to scale up to 20 x blade chassis with a Cisco UCS 6120 Fabric Interconnect, or 40 chassis with the Cisco UCS 6140 Fabric Interconnect.  As hinted in the previous section, the management software for the all connected UCS chassis resides in the redundant Cisco UCS 6100 Series Fabric Interconnects.   This design offers a highly scaleable infrastructure that enables you to scale simply by dropping in a chassis and connecting the FEX to the 6100 switch.  (Kind of like Lego blocks.)

On the flip side, while this architecture is simple, it’s also limited.  There is currently no way to add additional I/O to an individual server.  You get 2 x CNA ports per Cisco B200 server or 4 x CNA ports per Cisco B250 server. 

As previously mentioned, IBM has a strategy that is VERY similar to the Cisco UCS strategy using the Cisco Nexus 5000 product line.  IBM’s solution consists of:

  • IBM BladeCenter H Chassis
  • 10Gb Pass-Thru Module
  • CNA’s on the blade servers

Until IBM and Cisco design a Cisco Nexus switch that integrates into the IBM BladeCenter H chassis, using a 10Gb pass-thru module is the best option to get true DataCenter Ethernet (or Converged Enhanced Ethernet) from the server to the Nexus switch.  The performance for the IBM solution should equal the Cisco UCS design, since it’s just passing the signal through, however the connectivity is going to be more with the IBM solution.  Passing signals through means NO cable 

BladeCenter H Diagram with Nexus 5010 (using 10Gb Passthru Modules)

BladeCenter H Diagram with Nexus 5010 (using 10Gb Passthru Modules)

consolidation –  for every server you’re going to need a connection to the Nexus 5000.  For a fully populated IBM BladeCenter H chassis, you’ll need 14 connections to the Cisco Nexus 5000.  If you are using the Cisco 5010 (20 ports) you’ll eat up all but 6 ports.  Add a 2nd IBM BladeCenter chassis and you’re buying more Cisco Nexus switches.  Not quite the scaleable design that the Cisco UCS offers.

IBM offers a 10Gb Ethernet Switch Option from BNT (Blade Networks) that will work with converged switches like the Nexus 5000, but at this time that upgrade is not available.  Once it does become available, it would reduce the connectivity requirements down to a single cable, but, adding a switch between the blade chassis and the Nexus switch could bring additional management complications.  That is yet to be seen.

IBM’s BladeCenter H (BCH) does offer something that Cisco doesn’t – additional I/O expansion.  Since this solution uses two of the high speed bays in the BCH, bays 1, 2, 3 & 4 remain available.  Bays 1 & 2 are mapped to the onboard NICs on each server, and bays 3&4 are mapped to the 1st expansion card on each server.  This means that 2 additional NICs and 2 additional HBAs (or NICs) could be added in conjunction with the 2 CNAs on each server.  Based on this, IBM potentially offers more I/O scalability.

And the Winner Is…

It depends.  I love the concept of the Cisco UCS platform.  Servers are seen as processors and memory – building blocks that are centrally managed.  Easy to scale, easy to size.  However, is it for the average datacenter who only needs 5 servers with high I/O?  Probably not.  I see the Cisco UCS as a great platform for datacenters with more than 14 servers needing high I/O bandwidth (like a virtualization server or database server.)  If your datacenter doesn’t need that type of scalability, then perhaps going with IBM’s BladeCenter solution is the choice for you.  Going the IBM route gives you flexibility to choose from multiple processor types and gives you the ability to scale into a unified solution in the future.  While ideal for scalability, the IBM solution is currently more complex and potentially more expensive than the Cisco UCS solution.

Let me know what you think.  I welcome any comments.