The problem facing service providers hosting language models is that they need to evaluate each word or fragment of an image or sound against tens or even hundreds of billions of parameters, one at a time. Multiply that by millions of users around the world and it’s clear why the power demands of data centres are spiralling upward.
Optical interconnection provides one way to slow down the rise in electricity bills. Companies are scrambling to bring photonics from the top-of-rack routers into the servers.
Photonic interfaces potentially deliver the bandwidth needed to support hundreds of accelerators running in parallel as they push and pull terabytes of neuron weights and data in and out. Could photonics go further and attack the huge power demand that comes from the highly parallelised floating-point accelerators themselves?
Photonics can handle some linear arithmetic operations at high speed. Some companies have been working in this space for some time, such as UK-based Optalysys, with its use of free-space optics for scientific computing. The team behind the newer startup Lumai has employed similar optics, adding some newly found techniques for training neural networks in the optical domain, as part of a plan to build lower-energy AI accelerators.
Computers based on free-space optics use micromirror displays and spatial light modulators (SLMs) to steer light onto a sensor. Multiplication takes place through constructive and destructive interference when beams are steered onto the same sensor elements by the liquid crystals in the SLM. These free-space systems have the advantage of making it easy to perform operations such as Fourier transforms by passing the light through lenses. They also provide enormous bandwidth thanks to the ability to have many beams operate in parallel through the SLMs.
A versatile optical component
Others have focused on the mixing of light in the Mach-Zehnder inferometer (MZI), an arrangement of optical couplers and phase shifters. Where paths meet, the interference performs part of a matrix multiplication in the analogue domain. A 4x4 matrix operation requires just four inputs that feed into six of the coupling elements: the four output ports provide results at the rate at which pulses pass through the array.
The MZI is a remarkably versatile optical component. By tuning the phase of the light beams that pass in and out, the structure can act as a logic gate, supporting not just on and off states but intermediate states where some light is coupled out through both output ports. This is one reason researchers such as Bogaerts favour the MZI as a key building block in the new domain of programmable photonics.
A photonic AI accelerator or similar arithmetic engine arranges the MZIs in a triangular structure that operates in one direction: light coming from one side passes to a final sum on the opposite side.
Researchers such as Wim Bogaerts, a professor at the University of Ghent, have proposed using honeycomb-like arrays of MZIs that let the light snake its way round a matrix programmed with the help of photodetectors. Those photodetectors form part of a feedback loop to tune the phase for maximum or minimum cancellation on one of the output ports, acting as a full gate. In the honeycomb, this switching action steers photons either to the left or the right port.
Other MZIs that do not use the feedback element may then be used as compute elements to process data. Just by altering the controls, MZIs can flip between the two modes.
There are, however, drawbacks with programmable optical processing. In principle, reducing the many logic gates needed for electronic calculations in AI accelerators to a much smaller number of MZIs looks to be more efficient. In practice, programming optical elements is itself quite hungry both in terms of power and size.
The most reliable way to alter the phase of a lightwave passing through a silicon waveguide is to use a heating element. The demand quickly scales up to the point where several hundred optical elements operating at gigahertz rates might need a kilowatt of power, not including any air-conditioning around the unit.
Above: Honeycomb-like arrays of MZIs let the light snake its way round a matrix programmed with the help of photodetectors
Programmable photonic systems
The systems available today operate on a much smaller scale. One of the first programmable photonics systems to appear on the market is a unit made by Spanish company iPronics. The chip itself contains a set of interlinked rings that can be configured as around 20 MZIs or seven microring resonators. As it is mainly intended to support prototyping, the company supplied it with accompanying lasers and heater support in a rack mount unit.
The heating issue has focused attention on other ways of implementing phase shifters. None of them have moved far from the lab as yet, but they could slash energy demands if they become practical.
One option for disposing of heating elements lies in electro-optic polymers such as those developed by Lightwave Logic, which are now reaching the point of commercialisation. An option that is some way further out lies in phase-change materials, such as antimony selenide, that are fully transparent in both their states, but which exhibit large changes in refractive index as they shift from amorphous to crystalline forms. They still need heaters to force the change in state but that only needs brief pulses. Once cooled into either state, they have the same non-volatile advantage as phase-change memories.
Already used in SLMs, liquid crystals provide another route to integrated photonic circuits, though they suffer from the issue of being liquid. This means cells that incorporate the crystals need to be filled and sealed, which leads to quite different manufacturing processes that those used in most semiconductor fabs used for silicon photonics. However, inkjet printing looks to be a viable option. On the plus side, liquid crystals could deliver the kinds of change in refractive index needed and a faster response than phase-change materials.
Integrating phase-change or similar materials may help with AI applications. Though MZIs and resonators are good for linear operations, they fare less well with the highly non-linear activation functions used in neural networks. Novel materials would avoid the need to go through energy-intensive electro-optical conversions to implement these functions.
Above: iPronics chip contains a set of interlinked rings that can be configured as around 20 MZIs or seven microring resonators
Photonic AI developments
In the meantime, though more photonics AI developments are coming out of academia, some companies already in business have moved into parallel markets.
Optalysys now focuses on using its SLM-based architecture for accelerating algorithms that work on encrypted data. US-based Luminous had originally planned to build its own photonic AI system, looking at both spiking neural networks and matrix-arithmetic accelerators. But a little over a year ago, the company changed direction.
Management decided to focus on integrating a fully electronic accelerator with photonics for the interconnect on the basis that this architecture would deliver better prospects for relieving the bottlenecks in AI training using today’s technology.
Though data-centre applications continue to attract venture-capital funding, they may not provide the best outlet for novel materials. One driver behind the development of optical computing in the 1990s lay in routers. The increasing use of machine learning for classifying network packets and for coordinating 5G basestations, which are today interconnected by fibre, provides an opportunity for photonic AI.
Similarly, photonic processors could find uses as high-bandwidth preprocessors for optical sensors and cameras.
A key issue with deploying the new materials lies in obtaining access to manufacturing to even prototype with them.
Researchers have found it difficult to find foundries who will work with novel phase-change and similar materials. Initiative like the EU’s PhotonixFab network of pilot lines may close the gap by providing stronger links between commercial foundries and research institutes with lab-scale equipment.
But photonics has a way to go before it becomes a staple of computing rather than communications.
