Ethernet interface speeds have increased tenfold every four years or so since 1995. But only recently has attention turned to the power efficiency of copper based Ethernet links.
The rapid rise in the power demands of data centres crammed with IT equipment is one factor that has brought the issue to the fore. Another is the high power consumption of 10GBase-T, the 10Gbit/s copper Ethernet standard.
"When the first [10GBase-T] products came out, they were power hungry," said Jason Rock, senior product marketing manager at Vitesse Semiconductor. "A lot of people realised that process technology alone would not lead to the standard growing in volume."
Indeed, power consumption has become an integral part of network ic design. "Traditionally, when designing systems, it was a tradeoff between performance and cost," said Nick Ilyadis, cto of Broadcom's infrastructure and networking group. "Now energy has become a third variable."
Work addressing the power consumption of copper transceivers has resulted in Energy Efficient Ethernet (EEE); the IEEE 802.3az physical layer standard that exploits the bursty nature of traffic, saving power when the network is idle. Now attention is turning from the physical layer to the networking layer, which promises further power savings.
At the physical layer, chip vendors have developed several power saving techniques alongside EEE. These include cutting power when a link is down, matching the power to the copper cable length and optimising power based on link speed. Of the four, only EEE is a standard as the physical layer devices (PHYs) at the link's ends must agree configuration parameters; having a standard ensures PHYs from different vendors interoperate.
Vitesse's ActiPHY saves power when there is no link: when a link is unplugged, when a cable is connected but the remote end is not attached, or when the remote end is off.
The PHY detects the absence of a link when no energy is received. "We have a basic energy detector: think of a voltage comparator and some hysteresis to make sure there is no noise on the link," said Rock. "Once no energy is detected for some time, we turn off the transmitter." Even the receiver, which includes a dsp, is shut down. To avoid the link being asleep when data is sent, Vitesse uses wake up timers to activate the circuitry periodically.
Meanwhile, PerfectReach is Vitesse's technology that modifies power based on cable length. Ethernet copper PHYs are specified to drive 100m cables or receive a signal that has been attenuated over 100m of cable, but many links are only 6m or less. "With a PHY, you have to drive 100m of copper," said Rock. "That is a known voltage and a known impedance and there is only so much you can do [to save power]." But determining cable length using the PHY's dsp allows an algorithm to tailor the drivers to the cable length.
Broadcom has its technology variants that save power when no link is present and which take cable length into account.
Paying attention to power consumption
Until recently, little attention was paid to the power consumption of a 1Gbit/s transceiver when reverting to the 10 and 100Mbit/s Ethernet modes. According to Rock, the power consumed at lower speeds has been comparable to 1Gbit/s. Vitesse's latest 1Gbit/s
designs consume power comparable to its standalone 10/100Mbit/s PHYs when operating at the lower speeds.
With an Ethernet network, the busiest times are typically when staff arrive and download emails, at lunchtime when people go online and later in the day when staff are working. "Beyond that, the network is pretty quiet," said Rock. The IEEE 802.3az standard, ratified in October 2010, saves power during such low traffic conditions. As such, EEE is viewed as a complement to ActiPHY, which saves power when no link is present.
Traditionally, frames are sent across an Ethernet link even when the network is idle. With EEE, the drivers are turned off when there is no traffic, while refresh signals are sent periodically to maintain synchronisation with the remote end. "There are enough refreshes to maintain frequency lock, but the majority of time – if there is no traffic – is spent in the off state," said Ilyadis. Parameters are shared between the end devices to determine how long the drivers are in the off state, the refresh burst duration and how the link is brought back from the low power idle mode.
The EEE standard also allows additional flexibility. The two devices can exchange parameters at link up that allows additional time to power up subsystems deeper in the system. "With 10GBase-T, it takes 8µs to wake up the link for just the PHY," said Ilyadis. "But you can request an additional 20µs to power down products like FIFOs behind it."
IEEE 802.3az also allows for higher level control mechanisms once the link is established. Ilyadis cites the example of waiting for 10 packets to arrive before turning on the link, rather than sending each packet on arrival, unless the packet is high priority and cannot be delayed.
Such an in system control policy, part of what Broadcom calls Ethernet Efficient Networking, allows for the tuning of how frequently system functions come out of the low power idle states. There is also an end to end control policy between systems and here factors such as time of day are considered. After midnight, when equipment like printers or IP phones are unlikely to be used, a link between the switch and the end devices can be put in idle for longer.
"EEE saves 75% at the PHY, but in a switch, there are a lot of other things using power," said Ilyadis. "With Ethernet Efficient Networking, we are driving power savings deeper into the system."
An example is an Ethernet switch linked to a network interface card (NIC) within a server. The switch can inform the NIC there is no traffic and the link will be idle for 3ms as it buffers traffic. The controller can then switch off its PCI Express bus and other subsystems and put itself into a deep sleep, knowing no data will be sent for the next 3ms. "If you start doing that 50 times a second, the power savings add up," said Ilyadis.
Broadcom says such coordination between the network and its end points results in much richer power savings than that of the PHYs alone. "But EEN is very much in its infancy," said Ilyadis.
Broadcom has shown EEE compliant PHYs with non compliant EEE switches can achieve a 17% power saving. Broadcom's PHYs 'fool' the non compliant EEE switch into thinking they are always on, even when they are idle and buffering traffic. When both switches are EEE compliant, power savings rise to 37% between the conditions of no traffic and full traffic.
Vitesse says all its 1Gbit/s PHYs support ActiPHY and PerfectReach, while all its products introduced in 2010 are EEE compliant. It adds that a combination of the techniques brings an overall halving in power consumption. Broadcom has implemented EEE as part of its 10/100/1000Base-T and 10GBase-T PHYs, and 1 and 10Gbit Ethernet controllers as well as its switches.
