How cool is that? Liquid cooling

Put a billion transistors on a chip and what do you get? Staggering performance, yes, making possible applications otherwise impossible. Unfortunately, you also get something else: enormous amounts of heat. As a result, many in the electronics industry now believe its most important challenge is far better cooling. Without it, we could see the end of Moore's Law. The situation is already critical. For example, the power required to cool a typical large data centre is now equal to the power consumed by the servers themselves. In 2009, data centres consumed an estimated 330TWhr of energy – around 2% of global electricity production. But cooling isn't cool – or hasn't been until now, as IBM Research's Gerhard Meijer says. "50% of electricity consumption goes towards cooling infrastructure, but while Moore's Law is widely known and often cited, increasingly critical physical laws of thermodynamics receive little popular attention." Today, most cooling uses air, at room temperature for the typical pc, chilled for data centres, supported by elements such as heat sinks with long fins to enhance the convective heat transfer. But air cooling cannot be the future because it is far less efficient than liquid cooling – water, for example, can capture heat about 4000 times more efficiently than air. There are major challenges to using liquids, but many are convinced it has to happen – and in fact, it already has, as Meijer points out. "Critics of liquid cooling contend it comes at the price of increased mechanical complexity. True, but this can be managed, as computers were once equipped with liquid cooling when the power density of bipolar based computer systems reached its peak during the 1980s. For example, the Cray-2 supercomputer used liquid immersion cooling." Also, chilled liquid cooling has been recently reintroduced into high end mainframes and densely packed servers. But this is just a start. "Liquid cooling can be taken further," Meijer says. "Microchannel heat sinks can be designed such that the thermal resistance between the transistor and the fluid is reduced to the extent that even cooling water temperatures of 60 to 70°C ensure no overheating of the microprocessors." IBM believes such an approach has major benefits. For a start, it could cut data centre energy consumption by up to 50%. Also, it would be possible to use the collected thermal energy either for district heating or industrial applications, significantly improving the system's green credentials. Again, this is now happening. At the beginning of May, with its partner the Swiss Federal Institute of Technology (ETH) Zurich, IBM introduced a new supercomputer, the Aquasar, an entirely new, water cooled machine that will also supply heat to university buildings. Aquasar consumes up to 40% less energy than a comparable air cooled machine and through direct use of waste heat will cut its carbon footprint by up to 85%. Aquasar is now operating at ETH Zurich. Using special water cooled IBM BladeCenter Servers, it achieves a performance of 6Tflops and consumes 20kW, compared with an average of 257kW for a conventional supercomputer. Each blade has a liquid cooler per processor, as well as input and output pipeline networks and connections. Cooling with a water temperature of approximately 60°C keeps the chips at operating temperatures, well below the maximum allowable temperature of 85°C. The pipelines from the individual blades link to the larger network of the server rack, which in turn are connected to the main water transportation network. The whole system is a closed circuit: cooling water is heated by the chips and then cooled to the required temperature as it passes through a passive heat exchanger, delivering the removed heat directly to the building heating system at ETH Zurich. Cooling conventional chip systems is going to be a major challenge. But what about even more tightly integrated devices – notably 3d assemblies, which many think will be vital to take Moore's Law past 2020? To tackle this, IBM Research has formed a four year project with ETH Zurich and the Ecole Polytechnique Federale de Lausanne (EPFL). Called CMOSAIC, it will study cooling techniques needed to support a 3d chip architecture of multiple cores with an interconnect density of from 100 to 10,000 connections per mm2. IBM believes the use of hair thin, liquid cooling microchannels, measuring only 50µm in diameter between the active chips, will be vital for future 3d chip systems. "Data centre electricity consumption is doubling every five years so, at this rate, a supercomputer in 2050 will require the entire production of the US energy grid," says Professor John Thome, professor of heat and mass transfer at EPFL and CMOSAIC project coordinator. "3d chip stacks with interlayer cooling not only yield higher performance, but also allow systems with a much higher efficiency." The CMOSAIC team faces major challenges, but recent progress has been made, for example in the fabrication of through silicon vias. This has opened new avenues for high density area array interconnects between stacked processor and memory chips. Prof Thome is in no doubt as to how fundamentally challenging the CMOSAIC project is. "It is a genuine opportunity to contribute to the realisation of arguably the most complicated system that mankind has ever assembled: a 3d stack of computer chips with a functionality per unit volume that nearly parallels the functional density of a human brain. CMOSAIC's aggressive goal is to provide the necessarily 3d integrated cooling system that is the key to compressing almost 10exp12 nanometre sized functional units into 1cm³ with a 10 to 100 fold higher connectivity than otherwise possible. "Even the most advanced air cooling methods are inadequate for high performance 3d ic systems, where the main challenge is to remove the heat produced by multiple stacked dies in a volume of 1 to 3cm3, each layer dissipating 100 to 150W/cm². CMOSAIC aims to develop the engineering science base that will enable a new state of the art in high density electronics cooling." Cooling is so important for future chip technology that it has became a major research field in its own right, in both academia and the electronics industry. Intel, for example, is working on thermoelectric chillers, which pump heat out when current flows through them, known as the Peltier effect. A prototype brings together two breakthroughs discovered during the past decade: the realisation that nanoscale layers of thermoelectric material make for much more efficient cooling devices; and that using thermoelectric heat pumps for cooling the hottest spots on a chip is much more energy efficient than trying to cool the whole ic. The Intel group says it is the first to demonstrate both concepts in a working chip. They attached a small gallium arsenide substrate to a conventional heat spreader used to cool chips by convection. On the substrate, the researchers grew a 100µm thick layered structure, called a superlattice, containing bismuth, tellurium, antimony, and selenium. The structure pumps heat from the back side of the chip to the heat spreader. The researchers made a hot spot on the chip with a heat flux of about 1300W/cm2, much higher than usually found on chips. When 3A were sent through the thermoelectric cooler, the total temperature was reduced by nearly 15^°C. Another potential approach to cooling is to avoid the need for it in the first place – by finding a superconductor like material in which heat is not generated. Physicists at the US National Accelerator Laboratory and Stanford University Institute are aiming to do just that with bismuth telluride, a material that not only allows electrons on its surface to travel with no loss of energy at room temperatures, but which can also be fabricated using existing semiconductor technologies. The work has shown that bismuth telluride shows clear signs of being a 'topological insulator', which stems from a quantum phenomenon known as the Hall effect. The result is a material which electrons can flow across with no resistance, hence no heat dissipation. While topological insulators are different to conventional superconductors in that they can only carry small currents, they could pave the way for a major shift in microchip development. "This could lead to new applications of spintronics, or using the electron spin to carry information," says Xiaoliang Qi, assistant Professor of Physics at Stanford. "Whether or not it can build better wires, I'm optimistic it can lead to new devices, transistors and spintronics devices." Even better, bismuth telluride is relatively simple to grow and to work with. "It's a three dimensional material, so it's easy to fabricate with the current mature semiconductor technology. It's also easy to dope – you can tune the properties relatively easily," Qi says. Another leading research centre into cooling is Purdue University, which has developed a technique based on two cooling methods, microchannels and microjets to produce a significant advance, able to cool chips producing more than 1kW/cm2. The system comprises channels less than a millimetre wide on top of a chip covered with a metal plate containing tiny holes. The coolant is pumped through the holes in microjets and liquid then flows along channels to cool the chip. As the liquid is heated, it bubbles and momentarily becomes a vapour, facilitating the cooling process. In past research using just microchannels, the temperature of the coolant varied depending on its location as it moved through the channel, because it heated up along the way. The microjets make it possible to achieve uniform cooling because the liquid is supplied everywhere along the length of each channel. Santa Clara based Ventiva – a spin off company from Purdue – is working on an innovative technique called 'ionic wind' cooling. In this approach, a high intensity electrical field is created to produce ions and pump air molecules, a method of creating air flow known as the Corona Wind. The aim is to produce solid state fans, which would be completely silent and much smaller than comparable rotary fans. They would also have a flexible form factor. Ventiva says its technology could be built directly into a chip's heat sink to achieve a cooling rate similar to that of water. The aim is to produce the air flow right at the wall of the heat sink – whereas blowing it down from above using a fan creates a cushion of stationary air that impedes the heat dissipation. The system being developed by Ventiva consists of 300 electrodes that ionise and then pump up the air molecules across the surface. Today, we are all too well aware that it's not only computer chips that are getting hotter – the planet itself is warming. Keeping temperatures down has never been – well – cooler.