The challenges of taking COTS to top speed

That latter factor has been addressed with the development of open standards, such as VPX. But the COTS approach is beginning to falter as data rates increase. When the original VPX specification was developed, it was intended to handle data rates of 3.125Gbit/s. Today, data rates have pushed beyond 10Gbit/s towards 20Gbit/s and technology developers are looking to offer equipment that supports 40Gbit/s. Looking to support this migration to ever higher data rates, Curtiss Wright (CW) has developed the Fabric40 programme, said to be the first end to end system approach that allows 40Gbit/s fabrics to be integrated into high performance applications. David Jedynak, chief technology officer with Curtiss Wright Controls Defense Solutions (CWCDS), explained the approach. "The idea is, fundamentally, to take the elements you need to build a high performance computing system and make sure they can work together at 40Gbit/s. We will have done a lot of pre integration to make sure it all works." Where did the idea come from? "As customers look to COTS to get more modularity in terms of form factor, computing elements and so on, it takes a lot of their risk away and allows them to focus more on the 'hard stuff'. But their next area of effort is to get COTS boards to work together, so Fabric40 is a 'value add' from CW, providing a family of products that is integrated and benchmarked. We see it as COTS plus – something like a reference design platform. "At the end of the day, when the customer starts to build their application, they will know that we've addressed the signal integrity, interoperability and high speed connection issues, along with the different layers of middleware and drivers." He said there are basically two ways to move data at 40Gbit/s: Ethernet or Infiniband. "Both have their pros and cons and are useful in different ways. With Fabric40, we've decided to be agnostic, so we'll be making both approaches available." Bob Sullivan, cto with CW's engineered electronic packaging division, explained some of the issues: "We have been involved with signal integrity on the backplane for some time now but we got some CW business units together to go through a signal integrity analysis before we embarked on Fabric40. We already understood that passing 10Gbit/s through a VPX system was non trivial, so we wanted to make sure the 'end to end' was optimised and worked reliably when put together, with low bit error rate and good data fidelity. "In the course of the analysis, we 'peeled the onion' and looked at a range of effect, including pcb skin effects. We found out how bad it was and what we had to do to make things work at higher data rates." Sullivan noted that part of the work done to ensure reliable operation has resulted in patent application for novel optimisation of backplane design. CWCDS has recently announced the latest addition to the Fabric40 portfolio. The Hybricon6U six slot Gen3 OpenVPX backplane is designed to support end to end transmission of extremely high speed data and, with a bandwidth of 40Gbit/s, is claimed to be the first such device to provide support for Gen3 OpenVPX. By supporting full speed distribution of data over 40Gbit/s Ethernet or Infiniband fabrics, the backplane is said to enable embedded subsystems to deliver previously unobtainable levels of performance. CWCDS says OpenVPX systems built using the Hybricon backplane and featuring other Fabric40 system elements, such as dsp boards and single board computers, will more than double the performance of previous generation Serial RapidIO Gen 2 based systems and four times the performance of 10Gbit Ethernet based systems. Jedynak outlined some of the benefits of Fabric40. "We're making sure the 40G interface on the single board computer, which might be the overall manager of the system or hosting a high speed a/d converter, is fully interoperable at this speed and has all the optimisations. We're making sure all the critical paths are operating and managing them within the context that everything needs to work together." Sullivan pointed to one issue that affects the move to higher data rates. "When we're moving from Gen 2 (10Gbit/s) to Gen 3, we're really pushing connectors to the edge of their abilities. It's the connector footprint, rather than the connector itself." According to Sullivan, issues include return loss, crosstalk, weave skew and resonances. "We've done around 10,000 simulations where we got into the 'corners' and examined what happened with various permutations of modules. You can't do anything about the geometries – how close they are together. Some combinations bring big resonances with process variations, so what you can do is look at the worst cases and work out the things you can do to improve margins. Some of these things are part of our patent application. Overall, we expect an overall improvement in noise margin of 6dB." CWCDS says that assuming low loss laminate materials would mitigate these problems isn't the solution. The problem is highly complex, it says, and 'aggressive design rules' need to be applied to modules and the backplane in order to avoid 'problem permutations'. But how proprietary is Fabric40? Jedynak explained: "The standards we use in Fabric40 are all open, but there is some proprietary technology which comes in 'under the hood'. Signal integrity at 40Gbit/s isn't something everyone can do and getting data to pass reliably across backplanes and a mix of cards is a big challenge. Our proprietary technology – what's under the hood – gives us the performance." The first Fabric40 element to be announced was the CHAMP-AV9 dsp engine, said to be the first such device to provide a 40Gbit/s Ethernet or Infiniband data plane fabric. Using Intel's Advanced Vector Extensions 2.0 instruction set, CHAMP-AV9 offers 664GFLOPS, doubling the performance of previous dsp solutions. The dsp engine shows the importance of algorithms in high performance embedded computing systems. Jedynak said algorithms might be the first part of system development, but getting them to run on a system within given data flows becomes a challenge. "Developing high end algorithms doing high end tasks is complex. We are offering the ability to marshal the number of processors, dsps and so on early in the process and customers know that when they are using it, the basic interconnect and so on is there. "We're going to the type of people working on these apps and saying we've got it pre integrated – this is what you can get straight away. When they start implementing algorithms, it reduces the guesswork because they know what they are going to get. They can quickly choose their system, benchmark it and know the 'plumbing' works." Meanwhile, Sullivan noted: "Over the next few months, there will be a revision to Open VPX that adds Gen 3 speeds into the various profiles. But having profiles and products that work are two different things," he concluded.