If you are new to blade servers, you may find there are quite a few options to consider in regards to managing your Ethernet traffic. Some vendors promote the traditional integrated switching, while others promote extending the fabric to a Top of Rack (ToR) device. Each method has its own benefits, so let me explain what those are. Before I get started, although I work for Dell, this blog post is designed to be an un-biased review of the network options available for many blade server vendors.
NOTE: The diagrams shown below only show 8 servers in an effort to simply the explanation. It is not reflective of any one vendor design.
Pass-Thru Modules
Pass-thru modules provide blade servers with point-to-point connectivity very much like you have in rack servers. If an Ethernet cable is not plugged into the corresponding port on the pass-thru module, the server’s port is not active. In the example to the left, port 0 on blades 1, 2 and 3 are active as well as port 1 on blade 4.
Pass-thru modules are ideal for environments that have small budgets (they are cheaper), or environments where the Networking group does not want to manage networking outside of the traditional networking fabric. Since pass-thru modules require a 1:1 server-to-external switch connection, it is the easiest to manage, although it offers no cable consolidation benefits. Dell, IBM and HP have Pass-thru options in their portfolio.
Fabric Extender (FEX) Modules
Fabric Extender, or FEX, modules extend the blade I/O fabric up to a top of rack Cisco Nexus 5xxx or UCS 6xxx device. Unlike the pass-thru module, FEX modules give you the ability to connect all servers to the Ethernet fabric with as little as one cable. The purpose of the FEX module is to allow simplified connectivity into a Cisco fabric allowing the Top of Rack device to see the FEX module as an extension of the fabric. Cisco, Dell, IBM and HP all have FEX options in their portfolio.
Aggregator or Bridge Modules
Aggregator, or Bridge modules are designed to aggregate Ethernet and offer consolidated Ethernet connectivity much like the FEX but unlike the FEX, it has Layer 2 switching at the chassis level which offers intra-chassis communication for the blade servers. Aggregator modules can also connect into a Cisco Nexus 5xxx fabric to take advantage of FCoE Transit and Data Center Bridging (DCB). Dell is currently the only vendor offering Aggregator modules in their portfolio.
Switch Modules
The last category of choices is the integrated switch module. This category has been around since the creation of blade servers. These modules are nearly identical to the traditional top of rack switches but redesigned for use within a blade chassis. Switch modules offer advantages similar to the Aggregator modules listed above but offer additional advantages such as Layer 3 switching, Ethernet speeds up to 40GbE and stacking (a way to connect multiple switches together to offer a single virtual fabric for management.) While switch modules are the most expensive of all networking options, it also offers the most bandwidth, connectivity and performance. Dell, IBM and HP have switch module options in their portfolio.
Hopefully now you understand what your networking options are for blade servers. I’m interested in your thoughts, so feel free to leave your comments below.
Kevin Houston is the founder and Editor-in-Chief of BladesMadeSimple.com. He has over 15 years of experience in the x86 server marketplace. Since 1997 Kevin has worked at several resellers in the Atlanta area, and has a vast array of competitive x86 server knowledge and certifications as well as an in-depth understanding of VMware and Citrix virtualization. Kevin works for Dell as a Server Sales Engineer covering the Global Enterprise market.
Disclaimer: The views presented in this blog are personal views and may or may not reflect any of the contributors’ employer’s positions. Furthermore, the content is not reviewed, approved or published by any employer.
So does it mean that with Aggregator (or Bridge) modules we will have at the same time both intra-chassis communication and no hop count increase? Or it will increase hop count?
Thanks for the question. It does not increase hop count becuase it has local connectivity. Traffic from blade to blade in the same chassis will go to the aggregator and down to the blade. It does NOT have to go to the aggregator, then to the top of rack, then to the aggregator, then to the blade as would the FEX I/O Module. I appreciate you reading.
Where are the additional options like Virtual Connect? That has intra chassis communication option and deliver cable count reduction etc.
While arguments could be made that HP’s VirtualConnect modules are unique, when you review the physical characteristics, under the covers they would be classified as an Ethernet switch. I wanted to focus on the basics on this post. If I focused on every vendor’s switch capabilities like network port virtualization, it would be a much longer post. Thanks for the comment!
Please explain intra chassis and inter communication will take place in case of passthru module
Great ask. Since pass thru modules are a point-to-point connection, there is no communication between servers inside the chassis. All LOM (LAN on Motherboard) data must go out of the chassis to the external switch and back to the chassis if server to server communication is required. Same applies if a server needs to communicate to another server in another chassis. Thanks for the comment.