Recreating the Big Bang

It is well known as the largest, most expensive and potentially most important scientific experiment the world has ever staged. What is not so widely recognised is that CERN's Large Hadron Collider (LHC) is also arguably the most demanding test there has ever been for electronics. Astonishing statistics come naturally to the LHC – appropriately, perhaps, since it is trying to discover ultimate truths about reality. To take just a few: the superconducting cabling it uses would go round the equator 6.8 times. The vacuum in the LHC is comparable to outer space. The equipment for one of the main experiments, ATLAS, is around 45m long and 25m high, and weighs 7000tonne. Protons are accelerated in the LHC to 0.999997828 the speed of light and travel round its 27km ring more than 11,000 times a second, producing 600million collisions per second. The LHC's ring houses 1232 dipole magnets, each 15m long, which bend the beams, and 392 quadrupole magnets, each 5 to 7m long, which focus them, all cooled to -271°C by 60tonne of liquid helium. Ultimately, two proton beams travelling in opposite directions will reach a collision energy of 14TeV, seven times greater than the previous most powerful particle accelerator. This will, the experimenters hope, create subatomic particles that have not been seen since the Big Bang, the origin of the universe. A total of six experiments are planned for the LHC; the four main ones being ATLAS, CMS, ALICE, and LHCb. ATLAS will look for signs of new physics, including the origins of mass, believed to be linked with an as yet unseen particle – the Higgs boson. CMS will do this in a different way, looking for clues to the nature of dark matter. ALICE will study a quark-gluon plasma, a form of matter believed to have existed shortly after the Big Bang, while LHCb will try to discover why the universe is made mostly of matter, rather than its opposite, antimatter. The LHC will also try to create mini black holes, which scientists believe are created constantly when cosmic rays hit the atmosphere, but which the LHC detectors should be able to study with precision. The volumes of data generated by the LHC are extraordinary. Even though massive data pruning takes place, the experiments will still produce 15 petabytes (1015) of information every year that has to be processed. This will be done by the Worldwide LHC Computing Grid, which comprises around 65 sites, including seven primary data centres that are connected via 10Gbit/s fibre optic links. Some 5000 scientists will participate. And where does this data originate from? For the most part, electronic detectors; many of which have to be sited deep within the experimental apparatus. This means electronics is fundamental to the LHC. "For each experiment, the value of the electronics is estimated to be around 25% of the total cost," says Philippe Farthouat, a physicist working on the ATLAS experiment. As he explains, each LHC experiment contains several sub detectors, each having a large number of channels – close to 100m for pixel detectors, and several hundred thousand for the calorimeters (which measure the energies of charged and neutral particles) and the muon spectrometers (which measure the paths of muon particles to determine their momenta). The typical readout chain of a detector follows the same pattern: amplification, shaping, optional digitisation, local storage and data extraction. "Shaping is basically filtering, for several purposes," Farthouat says. "These include noise optimisation by removing some frequency ranges of the signal and the noise included in these ranges, or time occupancy optimisation – a signal can be longer than we want and we need to avoid 'pile up' – signals from different particles being summed. The shaper reduces the time occupancy and hence the probability of pile up." One crucial aspect of much of the electronics used by the LHC is that the only way to get data from the detectors is to have the readout electronics on the detector itself. "There is no way to extract the analogue signals with cables, and so this leads to the need for custom ics. The connection between the electronics on the detector and the rest of the readout electronics is done with fast links, analogue or digital, which in most cases are optical to allow long distance, high speed and minimal electromagnetic problems." It is the need for electronics to work in the extraordinary environment of the LHC that poses such special challenges. "The key demands in terms of environment come from radiation. Electronic devices do not like ionising doses of radiation, or being hit by particles like protons or neutrons, which can damage the crystalline network. In addition, electronic devices are subject to single event upsets (SEUs) which can cause bits to flip so that a 0 becomes a 1, or the other way around, or even devices to be destroyed because of latch-up or gate rupture. "So special technologies or lay out techniques had to be used. To deal with radiation, we have used radiation hard technologies developed for space or military applications. To counter SEUs, we apply techniques such as triple redundancy: a register is replicated three times and a voting mechanism decides which value is correct – we assume an SEU would change only one register and the two others are still OK. "In some places, the temperature has to be extremely low and in many places the magnetic field is high. Some power supplies installed in the experimental caverns have to work with a magnetic field as high as 1000Gauss and we have to use special magnetic materials that prevent them being saturated by the field. "The front end electronics, the electronics directly attached to the detector, is fully custom. For most of the systems, asics have been needed to provide the electrical performance required in terms of noise levels and dynamic range, density – about 80m channels for a pixel detector – low power, or radiation hardness. The electronics outside of the detector use both custom design and commercial devices such as industrial modular electronics systems like VME, and commercial single board computers. "In general, the electronics for the pixel detectors have required the development of large asics in deep submicron technology, each handling thousands of channels in a uniform way, capable of sustaining very high radiation levels and using non standard bump bonding technology to connect to the detector. Every single point here is a challenge." One example of a major LHC electronic system is the silicon microstrip tracker readout system for the CMS experiment. This has more than 9m channels and is an analogue readout system, implemented using cmos asics and linear semiconductor laser transmitters, which send pulse height data off-detector for digitisation and the first level of data processing. The electronic readout, timing and control system of the CMS tracker is based upon three major elements: cmos ics, fibre optic transmission with customised optical transmitters and receivers in miniaturised packages, and large off-detector digital electronic boards relying on large fpgas. Ironically, almost none of the components eventually deployed in the system could have been procured when the system was originally conceived and its realisation was possible only because of technology evolution during the lifetime of the project. This was partly because of the high level of radiation tolerance required for all particle tracking systems at the LHC, but this was not the only factor. Requirements to be met were extremely challenging in terms of numbers of channels, noise performance, and power to be dissipated. A specific example of the kind of specialised asic required by the LHC has been developed for ATLAS by the Southern Methodist University (SMU) in Dallas, a 'link on chip' – or LOC – serialiser circuit. "The serialiser enables us to reduce the size of the cables needed to transmit signals from one place to the other," explains Farthouat. "In the particular application, we have 32bit words being produced every 25ns on the detector and we want them in the off-detector electronics 100m away. We could use cables with 32 lines but that would occupy a large volume. So we serialise the data and use a single optical fibre to transfer it. "In the current detector, we have 1.6Gbit/s serialisers and 1600 fibres to do the job. In the future, we want to increase the number of words to be read out by almost a factor of 100, but we don't want 100 times more fibres. So we'll use faster serialisers to transfer this data over fewer fibres. The LOC is one of the steps towards that goal." SMU says its cmos loc serialiser, made using silicon on sapphire, is the fastest such device yet developed, with a transmission rate of 5.8Gbit/s. Designed for the LHC's high radiation environment, as well as for high data bandwidth, low power dissipation and extremely high reliability, it can also operate at cryogenic temperatures and has been tested at –210^°C. The LOC serialiser will transmit data for the optical link readout system of the ATLAS Liquid Argon Calorimeter, which measures the energies of electrons and photons. Again, because the electronic readout components are in the centre of the ATLAS detector, they are essentially inaccessible for routine maintenance, so reliability is paramount, according to physicist Jingbo Ye, an associate professor of physics who led development of the chip at SMU. SMU is already planning to increase the chip's data rate to 10Gbit/s and, in the next few years, hopes to increase the total speed by a factor of 62 more than what is currently installed in ATLAS. The next generation of the Calorimeter's optical link system, based on the new LOC serialiser, will reach 152.4Tbit/s for the whole system. Ye said SMU's new LOC serialiser is also critical for the SuperLHC, the upgrade to the LHC which is planned to go online in 2017. "The original ATLAS design used a commercial serialiser from Agilent Technologies," Ye said. "But, for the sLHC, there is no commercial device that can meet the requirements." No one can be certain that the LHC and its electronic detection systems will make ground breaking discoveries – that is the nature of science. But it is off to a good start: just days after ATLAS started, it detected particles called W bosons, which the Higgs boson is thought to decay into. Collisions are now taking place at 7TeV. By the time it reaches its maximum of 14TeV, we may have found the origin of mass itself.