Data Center Cabling Solutions
The Evolution of Ethernet and Fibre Channel Structured Cabling in the Data Center

The Evolution of Ethernet and Fibre Channel Structured Cabling in the Data Center

  • Feb 13, 2018
  • data center architecture, data center cabling, copper cables, fiber cables, hyperconvergence, cabling solutions, Spine and Leaf, singlemode, multimode, multi-path, fiber optic cabling, storage area network

Managing a hybrid connectivity data center is complex, but can be done effectively with a proper structured cabling solution.

By: Dave Fredricks | Data Center Infrastructure Architect

In this decade we have seen a shift in structured data center cabling from copper to fiber and a hybrid mix of both. It’s important to note that managing a hybrid connectivity data center is complex, but can be done effectively with a proper structured cabling solution.

So how did the copper and fiber relationship get started? Structured copper cabling in the data center began running from speeds of 1GE to 10GE and now perform at even faster speeds, ranging from 25GE, 40GE to 100GE.

Most data centers built at the beginning of the decade ran mostly copper cabling to satisfy 1GE speeds for Network Ethernet protocols, whereas Storage or Fibre Channel used fiber optic cabling to support the need for 2GFC and 4GFC speeds. As a result, two different cabling plants were created to support both media.

Trouble in paradise

The use of structured network copper cabling infrastructure quickly became a massive bulk with thick bundles that were being measured in feet. Typically, data center operators ran Cat5 or Cat5e copper cabling to meet the distance verse speed requirements to support 1GE+ speeds.

Overhead or underfloor conveyance became overpopulated with copper strands creating cross talk issues and other cable management challenges.

The Storage or Fibre Channel cabling infrastructure was running on multimode glass with an orange jacket as OM1 -62.5 microns and OM2 -50 microns. This media easily supported Generation 3 and 4 Fibre Channel speeds for long distances to meet most data center requirements.

However, during this time period, there were some issues matching OM1 and OM2 glass cores together because of the different size between the two 12.5 microns.

Product innovations lead to changes…but not necessarily solutions

Product innovations then led to changes for both the copper and fiber cabling that were being used in the data center but didn’t necessarily offer the best solution.

For network copper, Cat6 became the most commonly used category. Cat6 could effectively support a 10GE signal to about 50 meters. Cat6A was also being used when 10GE distances up to 100 meters were required.

However, some equipment vendors actually specified the use of Cat6A even if the distance requirement wasn’t met. The problem was that Cat6A had a thicker jacket than the previous categories and tended to be stiffer and more difficult to route.

This, in turn, had most Cat6A installations terminated on site with higher labor costs instead of using a pre-terminated cabling solution. Despite the challenges, both copper and fiber optic cabling advanced. The multimode glass in fiber optics became laser optimized.

Laser optimization means that it can better support the newer technology for transceivers or optics. Technology for optics went from an LED light source to a VCSEL light source.

A VCSEL light source operates best at 850nm and the new laser optimized glass was made to support that product set. This led to the use of new OM3 and OM4 aqua jacket fiber optic cabling. OM3 and OM4 glass can handle speeds of 10G to 100G+ to standards distances of 100 meters and up to 400 meters with extended reach optics.

Fiber rules, but hybrid solutions still needed

That brings us to the connectivity infrastructure in today’s data center. The bulk of longer, 10 meters or more, copper connections has moved to fiber as the preferred media type. There are several reasons for this change.

First and foremost, there are limitations with copper from speed to distance aspect. While Cat6A is designed for 10G speeds, there are still issues with installation and termination. New Cat8 cabling is made for 40G speeds, but above that number, there isn’t a copper solution. With the demand for 100GE, copper is not looking like the best solution. Copper today is typically used for in-row, in-pod or in-cabinet connectivity.

The limitations of copper have led to increased use of structured fiber cabling solutions. Using OM4 multimode fiber satisfies most all current standards-based products coming into the market as well as next-generation products. OM5 multimode fiber has been introduced and can gain a slight distance increase over OM4, but still has not been written into many vendor specifications.

Multimode fiber also supports speeds of 100G to distances of 150 meters. If the distance is greater than that, then singlemode fiber can be used.

One cost-effective solution is the use of PSM-4 optics, as its distance reaches up to 500 meters.

Enter SDN products

Another reason for the use of fiber is the adoption of Software Defined Networking (SDN) in the data center. This technology helped produce Software Defined Storage (SDS) and Ethernet-based storage protocols. These newer solutions can operate on one cabling plant verse separate Ethernet and Fibre Channel cabling systems. This, in turn, has helped the Spine and Leaf topology design for networking to become a popular choice in the marketplace

Solid State Drives (SSD) have recently become equal in cost to Hard Disk Drives (HDD) and are typically a quarter of the weight and much smaller than HDD.

This allows equipment vendors to: 

  • Design compute systems with storage capabilities in each pod
  • Compute cabinets or 1u devices 
  • Reduces the need for large Storage Area Network (SAN) Director class switches and storage ports to every compute cabinet

As new SDN products enter the market, most data center operators still need to have the ability to utilize Fibre Channel SAN connections and separate Ethernet Network connections. They may also have to connect with Hyperconvergence machines, which are growing in popularity.

The challenge here is to try and effectively manage and plan for a hybrid structured cabling plant that still offers the ability to grow with additional equipment that meets increased speed optics.

Principles of a structured cabling solution

Managing a hybrid connectivity data center is complex, but can be done effectively with a proper structured cabling solution. While every structured cabling data center solution is different, there are some basic principles that apply.

For example, when choosing between multimode or single-mode fiber, it’s important to keep in mind single-mode optics have decreased significantly in price and should work well when connecting to a Spine and Leaf system, which is becoming more of a standing in the marketplace.

Regardless, CABLExpress architects are experienced in providing structured cabling solutions to meet your data center needs, whether it be a hybrid solution or strictly a fiber design.

Related content: