The true legacy of networking is brightly emerging in today’s COVID-19 world. It is through this hyper-connected world that people are continuing some semblance of normalcy. Friends and family are staying in touch through social media and different video tools and applications. Our students are continuing their education online. Many are working remotely from home. People are shopping online, while others are seeking escape through online video streaming and game playing. Life is continuing, though in a much different manner than before.
The importance of bandwidth in our daily lives has become apparent, and it is not clear if things will go back to where we were before this pandemic, settle at this new “normal”, or find some new middle-ground. During this pandemic reported traffic numbers around the world are showing increases ranging from 30% to almost 110%. At the same time download speeds, in general, appear to be mostly decreasing, and in some instances significantly, as decreases up to 35% have been reported. Furthermore, steps are being taken by some to reduce traffic on networks, such as Netflix, which reduced its traffic by 25% to assist European networks.
The IEEE 802.3 Ethernet Working Group, which defines the Ethernet standards that enable today’s wired infrastructure, was nearing completion of its second Ethernet Bandwidth Assessment. Initiated soon after the ratification of the 400 Gigabit Ethernet (GbE) standard, this assessment gathered a wealth of information related to help assess the future bandwidth needs of a diverse group of applications. In addition to observed bandwidth data, the Ethernet Bandwidth Assessment (publicly available here) also addressed three key factors: 1) the number of users; 2) increased access rates; and 3) increased services. it is felt that the reader will agree that the forecasted growth of these factors all point to a continuing bandwidth explosion.
Figure 1 shows a summary forecast from the Ethernet Bandwidth Assessment. In this figure traffic for a given application is normalized to its 2017 value. By 2025 the projected growth of the various applications is expected to vary anywhere from 2.3x to 55.4x the values of 2017. From an observation perspective, based on the standardization of 400 GbE in 2017, 800 GbE would only represent a 2x increase and 1.6 Terabit Ethernet (TbE) would only represent a 4x increase. These projected growths for these two potential Ethernet rates would lag nearly all of the observed growth rates for the different application spaces studied.
Figure 1- Ethernet Bandwidth Assessment Forecast
The path forward
The path to the next higher speed of Ethernet will consist of the same series of tradeoff that have been made in the past. Consider that there are a number of physical layer specifications that make up today’s high speed Ethernet family that services 100 Gigabit Ethernet and beyond: chip-to-chip and chip-to-module electrical interfaces, backplane solutions, twin-ax copper cable solutions, duplex and parallel fiber approaches for both multi-mode and single mode optical fiber, as well as the latest family member - DWDM systems. This family provides Ethernet connectivity ranging from centimeters inside a system box to distances reaching up to 80km.
This wide array of physical layer specifications has led to an array of different approaches to solving the “fat pipe” problem for Ethernet. Four basic approaches can be considered: 1) increase the baud or symbol rate; 2) increase the number of bits per baud or symbol, i.e., higher orders of modulation; 3) employ multiple channels of fiber or conductors, i.e. special multiplexing; and 4) employ multiple optical wavelengths over a given fiber, i.e. wavelength-division multiplexing. The rate per lane in gigabit per second is the product of the baud rate and the modulation scheme’s bit per symbol approach, while the “interface width” is based on spatial multiplexing or the product of spatial and wavelength division multiplexing. In addition, these approaches can be combined to provide the proverbial “fat pipe.”
The manner in which the Ethernet community has repeatedly employed this approach over the past 20 years for the various Ethernet families going back to 10 Gigabit Ethernet is illustrated in Figure 2. Curves representing each of the Ethernet families from 10 GbE to 400 GbE, plus 800 GbE and 1 6 TbE, are shown. The first generation of a new Ethernet rate tends to be wider and operates at a lower rate per lane. Take for example the 100 Gigabit Ethernet family. The first 100 Gigabit Ethernet Attachment Unit Interface (CAUI) for chip-to- chip was based on 10 lanes of 10 Gb/s, based on NRZ signaling. However, currently there is development activity within IEEE 802.3 to reduce the lane width to a single lane of 100 Gb/s, based on PAM4 signaling. The new lane count of 100 GbE will be 1/10 the width of the original interface, with the baud rate increasing 5.15x (10.3125 to 53.125 GBd) while the number of bits per baud has increased from 1 to 2. Taking this a step further, the new 100GBASE-ZR physical layer specification is driven by a DP-DQPSK modulation format, with a 27.9525 GBd that represents 4 bits per symbol. This simple diagram allows one to quickly ascertain the interface width / rate per lane combination to determine the various Ethernet solutions.
Figure 2- The Basic Math of Ethernet
It is important to note that, ultimately, it is the economics of the application that will dictate the solution. There is an underlying balancing act between cost, power, and achievable density for a given supported bandwidth.
The questions that will be asked
There is an activity underway within the IEEE 802.3 New Ethernet Application Ad hoc to begin development of a Call-For-Interest to request formation of a “Beyond 400 GbE” Study Group. The formation of a study group within IEEE 802.3 is one of the first steps in defining a new project. The role of the study group is to essentially define and justify the proposed project. This anticipated future study group will have some key questions to consider. Some of the key questions include:
- What is the next rate of Ethernet?
There is already a considerable amount of debate as to what the next rate of Ethernet would be. Given the doubling in lane rate signaling over the past decade, the two most likely candidate rates are 800 GbE and 1.6 TbE. However, given the observed exponential growth of data, will 800 GbE represent an acceptable investment for the industry? This is countered by the other perspective which asks does the industry have a technically feasible, cost effective solution for 1.6 TbE? Perhaps the right solution will be to develop both rates?
- What application spaces need physical layer specifications Beyond 400 GbE first?
The Ethernet Bandwidth Assessment clearly highlighted the growing bandwidth demand of both the data center and mobile application spaces. It is not surprising that data centers are a target application space, given their role as a primary driver for Ethernet over the past decade. Ethernet solutions targeting physical layer specifications 2km and below would be likely candidates. “Mobile” is also one of the leading bandwidth drivers, and with the deployment of 5G underway, the core of 5G networks looks likely to be another potential application space.
There are other application spaces that may emerge. Artificial intelligence is one topic area typically mentioned, given that the demand in processing power for training has been observed to be doubling every 3 to 4 months, compared to the observed doubling in processing power every 24 months observed with Moore’s Law. The full impact on networking is being explored.
- What are technically feasible solutions to address “Beyond 400 GbE” physical layer specifications?
This is one of the key questions that a future Beyond 400 GbE Study Group will need to consider. This will be challenging given that a number of physical layer specifications, supporting 100 Gb/s signaling are currently in the standardization process. The only effort underway targeting a higher signaling rate than 100 Gb/s per lane is the IEEE P802.3cw project that is defining 400GBASE-ZR, but this is a coherent optics solution targeting 80km. It is not clear if this will help in the development of solutions targeting more cost-sensitive, lower reach solutions. Therefore, technologies targeting electrical signaling and direct detect optical solutions will need exploration.
The need to start
American scholar, Albert Bartlett, said “The greatest shortcoming of the human race is our inability to understand the exponential function.” The Ethernet Bandwidth Assessment highlighted the coming exponential bandwidth growth. The anticipated future IEEE 802.3 Beyond 400GbE Study Growth will have some incredibly significant issues to start considering. Given the noted bandwidth forecast, it should be evident that any delay in getting started will add a critical timing aspect to the equation that will need to be factored into the equation of defining the next rate of Ethernet.
Written by John D’Ambrosia, Advisor, EPIC, Distinguished Engineer, Futurewei Technologies
Back to Blog
