Advances in data center technology and Internet usage have pushed enterprises to 10 Gigabit Ethernet (Gbps) links and search engines, carriers, ISPs to multiple 10-Gbps links. Projected growth mandates the need for higher speed Ethernet (HSE) connections. An IEEE Task Force has been established with the objective of standardizing 40 Gbps within the data center and 100 Gbps between major Internet nodes. If HSE is to be a success it must take advantage of existing data center copper and longer-reach fiber cables.

The Need for Speed
The amount of backbone Internet bandwidth maintained by major Internet carriers has been expanding at 75 to125% per year , driven by the explosion of broadband Internet users and growing use of bandwidth-hungry applications. Video-centric applications are the main contributors, including high-definition IPTV, video distributors such as YouTube, and video conferencing services. Other significant bandwidth gobblers include peer"to-peer sharing, virtual private networks, wireless backhaul, gaming, and various forms of online collaboration. Figure 1² illustrates the changing nature of Internet usage from 2006 to 2012 (estimated).

Advances in computer and networking technologies have likewise led to an increased demand for bandwidth within the data center. Multicore processing and virtualization have expanded the capacity and utility of individual servers.

Network attached storage (NAS) is rapidly replacing local disks, allowing access to larger amounts of data, centralized maintenance, and improved disk usage. Reduced costs are realized by consolidating servers and by employing NAS, as well as through the reduction of the number of network interfaces required both on servers and on network switches. With this consolidation, however, comes higher network data volume on a per-server basis. Most data centers have switched from 1 Gigabit to 10-Gbps technologies to meet their bandwidth needs. A growing number of the largest Internet services--Google, Yahoo, MySpace, Facebook, AOL, for example--have a near-term expected need for 40 Gbps in the data center and 100 Gbps between data centers, as illustrated in Figure 2 (courtesy of Google).

With enterprise migration from frame relay and other private networks to IPSec and Layer 2-3 VPNs, ISPs have found it necessary to offer 10 Gbps customer connections. ISPs traditionally require 4 to 10 times the bandwidth of their largest customer connection in order to provide the proper levels of service.

Different Uses--Different Speeds
Data center network usage is driven by Moore's law, doubling every 24 months. While 10 Gbps appears to be sufficient for most data centers today, 40 Gbps is expected to meet data center needs through 2014. Aggregated Internet traffic, however, is related to general Internet and telecom growth, doubling every 12 to18 months. A higher bandwidth solution is required. According to the Ethernet Alliance :

"The increasing trend of Ethernet being an end-to-end network solution and the understanding that a 40G interface will not be sufficient to address the aggregation demand has resulted in the 100 Gbps rate being identified as the next high-speed network operator interface. The core network needs to evolve to a higher rate, and operators are looking to Ethernet as the new 'fat pipe' for the core of their backbone; positioning 100 Gbps to complete the transition to an end-to-end all-Ethernet infrastructure."

Based on the current rate of backbone bandwidth expansion, 100 Gbps should meet the needs of carriers and ISPs through 2015.

The Costly Alternative
Lacking alternatives to 10-Gbps connections, carriers and enterprises have resorted to using multiple 10 Gbps connections to satisfy their aggregated bandwidth requirements. Each new 10 Gbps bandwidth step comes with additional switch and/or computer interfaces, both of which are expensive.

The complexity of such shear numbers can also be daunting, as shown in the metro network depicted in Figure 3. The benefits from a 100 Gbps network infrastructure are immediately visible.

Figure 3. Network Complexity and Benefits of 100 Gbps Infrastructure

There is further complexity associated with splitting up large aggregated flows resulting from server consolidation into 10 Gbps links. Bundling multiple links is usually handled though the use of link aggregation groups (LAGs), as specified in IEEE standard 802.3ad. Shorter data flows, such as VoIP packets, lend themselves to easy distribution across multiple links and result in efficient packing due to their small size and even flow rate. The single data flows associated with VPNs, streaming, and long-term data storage protocols are bursty in nature and cannot be split across multiple connections. Such data flows can generate bursts of up to 5 Gbps in traffic rate, resulting in high latency. ISPs often reserve bandwidth to handle maximum traffic rates without SLA penalties, resulting in low bandwidth usage.

Misallocation of bandwidth can lead to dropped packets and low quality of real-time applications such as video transmissions.

Larger 40 and 100 Gbps pipes will serve to average out multiple bursty flows, resulting in higher bandwidth efficiency and lower per-Gbps costs.

Recent Developments and Events
In July of 2006, the IEEE established the high-speed study group (HSSG) to look into the three- to seven-year requirements for Ethernet speeds beyond 10 Gbps. The HSSG reached the conclusion that both 40 and 100 Gbps solutions should be pursued, and forwarded a project authorization request (PAR) for a higher speed Ethernet amendment to the IEEE 802.3 standard.

With approved objectives and PAR, the IEEE 802.3ba Task Force was formed in December of 2007. The task force has been meeting regularly to discuss technology proposals that are nearing finalization (expected in early 2010). The approved task force objectives were:

• Exclusive support full-duplex operation
• Preserve the 802.3 Ethernet frame format using the 802.3 MAC
• Preserve the current 802.3 standard minimum and maximum frame sizes
• Support bit error rates better than or equal to 10-12 at the MAC/PLS service interface
• Provide appropriate support for optical transport networks (OTN)
• Support a MAC data rate of 100 Gbps, with physical layer specifications that support operation over:
  o At least 40 km on SMF fiber
  o At least 10 km on SMF fiber
  o At least 100 m on OM3 MMF
  o At least 10 m over a copper cable assemblies
• Support a MAC data rate of 40 Gbps, with physical layer specifications that support operation over:
  o At least 10 km on single-mode fiber (SMF) fiber
  o At least 100 m on OM3 multi-mode (MMF) fiber
  o At least 10 m over a copper cable assemblies
  o At least 1 m over a backplane