The Race to 400G Ethernet Part 2

The Race to 400G Ethernet Part 2

By: Dave Fredricks | Data Center Infrastructure Architect

 

This post is part two of a series meant to track the technology advancements in the move from 100G to 400G Ethernet speeds and their effects on structured cabling. You can read the first part here.

The first quarter of 2017 has accelerated the move away from three-layer data center topologies toward spine-and-leaf cabling designs. These new spine-and-leaf designs are typically running 100G connections using SR-4 optics.

Spine-and-leaf cabling connectors

In connector terms, they are running a 12-fiber MPO/MTP® connector into each transceiver. The 100G SR-4 specifications were released in April 2015 by the IEEE 802.3bm standard. This move from 10G or 40G from spine to leaf has changed the types of enclosures above the spine switch and near the leaf switches.

No longer are MTP®-LC cassettes used on each end of the link with an MTP® to MTP® trunk in between. MTP® coupler panels are used on each end. The shift from cassettes to MTP® coupler panels has also changed the jumpers into the active optics. The link now uses 12-fiber MTP® to MTP® jumpers instead of duplex LC to LC jumpers.

A typical spine-and-leaf cabling design

A typical design is shown below. This is a Cisco 9000 design with 100G links between spine (N9504) and leaf (N93180) using SR-4 optics.

The standard for 200Gb/s and 400Gb/s from IEEE, named 802.3bs, is planned for release in late 2017 to early 2018. The 400G link is planned to run up to 100 meters on multi-mode fiber and 500 meters to 10 kilometers on single-mode glass.

The groups involved in spine-and-leaf cabling specifications

A group involved in the new IEEE specifications is the CDFP multi-source agreement (CDFP MSA). This group is recommending a 400G solution using 16 lanes or fibers running 25G per lane. This technology has competition from the Octal Small Form Factor Pluggable (OSFP) Multi Source Agreement (MSA), which is supporting 8 lanes or fibers running 50G per lane.

Driven by Arista Networks, the OSFP MSA group released their new specifications document on March 17, 2017. This quote from the release explains the group’s goal:

“Established in November 2016, the OSFP MSA group has developed a high-performance pluggable optics module form factor that is capable of supporting the full range of 400G optics technologies for datacenter and metro applications. In addition, the OSFP is designed to support the next generation of 800G optics modules that will use eight lanes of 100Gbp...”

The OSFP optic differs in one aspect from the QSFP (where “Q” stands for “Quad”) optic in that the height is shortened and width widened to accommodate 32 transceivers into a 1U space. This new 400G standard currently has multi-mode running at a distance up to 50 meters and in-cabinet connections on copper up to three meters.

CABLExpress is a member of the OSFP MSA group and will offer connectivity products to work with these new optics. The new standard can be found at www.osfpmsa.org.

CABLExpress offers structured fiber optic cabling designs and products to work with 10G to 400G applications. Available solutions to replicate 32-, 36-, 48- and 64-port line cards and blades that use LC and MPO/MTP® connectors.

Also available are 8-, 12-, 16- and 24-port MTP® connectors for use with the next-generation optics along with required light path polarity. As new switching equipment comes into the market, new port replication solutions will be provided to best match and mirror the machines’ port counts.

In the first several months of 2017, the move to 400G has seen the release from the OSFP MSA group of 50Gbs per fiber. This surpasses the CDFP MSA group’s 25Gbs per fiber. There has been talk that a 100G Bi-Directional (BiDi) optic will be available soon to the market, and I believe that should happen sometime this year.

The 100G BiDi will utilize an LC connector running on duplex signals. The unknown is the distance it will run on OM3/OM4 glass. Look for my next update later this year.


Learn more about cabling for spine-and-leaf architecture in our white paper, Cabling Designs for Hyperconvergence.