384GB RAM in a Single Blade Server? How Cisco Is Making it Happen (UPDATED 1-22-10)

UPDATED 1/22/2010 with new pictures 
Cisco UCS B250 M1 Extended Memory Blade Server
Cisco UCS B250 M1 Extended Memory Blade Server

 Cisco’s UCS server line is already getting lots of press, but one of the biggest interests is their upcoming Cisco UCS B250 M1 Blade Server.  This server is a full-width server occupying two of the 8 server slots available in a single Cisco UCS 5108 blade chassis.  The server can hold up to 2 x Intel Xeon 5500 Series processors, 2 x dual-port mezzanine cards, but the magic is in the memory – it has 48 memory slots.  

This means it can hold 384GB of RAM using 8GB DIMMS.  This is huge for the virtualization marketplace, as everyone knows that virtual machines LOVE memory.  No other vendor in the marketplace is able to provide a blade server (or any 2 socket Intel Xeon 5500 server for that matter) that can achieve 384GB of RAM. 

 

So what’s Cisco’s secret?  First, let’s look at what Intel’s Xeon 5500 architecture looks like.

 
 

intel ram

 

As you can see above, each Intel Xeon 5500 CPU has its own memory controller, which in turn has 3 memory channels.  Intel’s design limitation is 3 memory DIMMs (DDR3 RDIMM) per channel, so the most a traditional server can have is 18 memory slots or 144GB RAM with 8GB DDR3 RDIMM. 

With the UCS B-250 M1 blade server, Cisco adds an additional 15 memory slots per CPU, or 30 slots per server for a total of 48 memory slots which leads to 384GB RAM with 8GB DDR3 RDIMM. 

 

b250-ram

How do they do it?  Simple – they put in 5 more memory DIMM slots then they present all 24 memory DIMMs across all 3 channels to an ASIC that sits between the memory controller and the memory channels.  The ASIC presents the 24 memory DIMMs as 1 x 32GB DIMM to the memory controller.  For each 8 memory DIMMs, there’s an ASIC.  3 x ASICs per CPU that represents 192GB RAM (or 384GB in a dual CPU config.) 

It’s quite an ingenious approach, but don’t get caught up in thinking about 384GB of RAM – think about 48 memory slots.  In the picture below I’ve grouped off the 8 DIMMs with each ASIC in a green square (click to enlarge.)

Cisco UCS B250 ASICS Grouped with 8 Memory DIMMs

Cisco UCS B250 ASICS Grouped with 8 Memory DIMMs

With that many slots, you can get to 192GB of RAM using 4GB DDR3 RDIMMs– which currently cost about 1/5th of the 8GB DIMMs.  That’s the real value in this server.

Cisco has published a white paper on this patented technology at http://www.cisco.com/en/US/prod/collateral/ps10265/ps10280/ps10300/white_paper_c11-525300.html so if you want to get more details, I encourage you to check it out.

13 thoughts on “384GB RAM in a Single Blade Server? How Cisco Is Making it Happen (UPDATED 1-22-10)

  1. Pingback: Twitter Trackbacks for 384GB RAM in a Single Blade Server? How Cisco’s Making it Happen « Kevin Houston's Blade Server Blog [kevinbladeguy.wordpress.com] on Topsy.com

  2. Aaron Delp

    What I really like about the Cisco blade is you can populate the slots and still keep the memory bus at 1066Mhz. Because the use the concept of a 4 to 1 vDIMM, the CPU only sees the 48 as 12.

    Other vendors, (say the HP 490 blade) have 18 slots but the bus speed drops to 800Mhz. This can be a significant impact on performance if you are memory performance bound.

    Nice article!

  3. Omar Sultan

    Kevin:

    Nice write-up. BTW, we just announced the same technology available in a rack form-factor for folks with a rack server environment.

    Omar Sultan
    Cisco

  4. kevinbladeguy

    Thanks, Omar. Yes, I’ve known about the Cisco C-Class Servers for quite a while and am debating blogging about them. I’m trying to keep my thoughts on this blog to just blade-related servers. Hope you keep my blog bookmarked, as I’m going to keep digging into blade details from HP, Dell, IBM and Cisco as I get them.

  5. Pingback: Tweets that mention Blades Made Simple · 384GB RAM in a Single Blade Server? How Cisco Is Making it Happen (UPDATED 1-22-10) -- Topsy.com

  6. Pingback: Kevin Houston

  7. Pingback: Kevin

  8. Pingback: unix player

  9. Pingback: Blades Made Simple · 384GB RAM in a Single Blade Server? How Cisco … | Drakz Free Online Service

  10. Pingback: Blades Made Simple » Blog Archive » More Blade Server Rumours

  11. Pingback: Blades Made Simple™ » Blog Archive » HP’s New 32GB DIMMs Too Expensive?

  12. Pingback: Blades Made Simple™ » Blog Archive » What Cisco Has to Do to Win the Blade Server Market

Comments are closed.