The web based survey attracted almost 3000 responses from embedded systems designers around the world. After qualification, this number was reduced to 2452, with 33% of responses coming from engineers based in Europe. Not surprisingly, the top four market segments were automotive, industrial automation, defence and medical
“If the respondents didn’t design, they weren’t included in the results,” said Michael Barr, chief technology officer. “And neither were they included if they couldn’t describe their current project.”
Andrew Girson, Barr’s CEO, said one of the survey questions asked respondents what was the worst thing that could happen if their system failed. “And 22% said their system could kill. That’s a big number; there’s a lot of people involved in industry who say that, if something goes wrong, it will kill people.”
What the survey highlighted, said Girson, is that safety risks are abundant. “A lot of people are designing systems which have safety implications. If something goes wrong with them, something bad will happen. But what are these designers doing to protect those who use their systems? We’d like to see best practice being followed, but that doesn’t appear to be the case.”
Of the 543 who said their system could kill or injure if it went wrong, 16% admitted they were creating software without working to coding standards, 40% said they weren’t undertaking code reviews and 30% said they weren’t using static analysis. “These are well know industry practices,” Girson said. “They are available and understandable.”
Regression testing
When asked about test plans, only 59% of respondents said they were using regression tests, with more than 80% saying they used system level tests and 68% applying unit tests. “Regression testing should be close to, if not, 100%,” Girson observed.
Barr continued with the analysis of the 543 respondents. “They came from the auto, medical, defence and industrial sectors,” he said. “These are where you find embedded systems that can kill.
“But if your design could cause someone to be killed, are you designing it to meet safety standards – MISRA software, for example? When asked, about two thirds said yes. Interestingly, 22% said no and 11% didn’t know. Either there are no standards for these people to design to or they don’t care. In my experience, if you’re working on a project, you should know whether there are such standards, which suggests they aren’t.”
According to the survey, the automotive sector is a big offender. “Automotive designs are more likely to risk lives,” Barr said, “but designers are less likely to follow standards.”
Barr attempted to explain why this might be, noting that some industries have what he called ‘voluntary’ standards. “The US FDA and FAA have their own policies which address potentially dangerous software.
“But what we don’t see – which may account for some of the data, including automotive designers not following safety standards – is an insistence on enforcing safety standards that apply specifically to the automotive industry.”
Risk, in Barr’s opinion, should dictate the process. “The worse the risk,” he said, “the greater the need for safety.”
Barr said that, if you read standards, they generally talk about safety integrity levels, or SILs. “These look at the worst thing that could happen and require designers to apply the appropriate SIL to that product. On that basis,” he continued, “standards dictate what things you need to do. For example, MISRA software requires a code review at SIL 2 and above, withautomated static analysis at SIL 3 and above.
“What it seems we are seeing here is that people who are designing at these SILs are not letting the risk dictate the process.
“Safety must be ‘baked in’,” he asserted. “It can’t be a ‘bolt on’ feature.”
Security issues
The discussion then turned to security, with a focus on communications. The survey produced little in the way of a shock when it found that internet connectivity is becoming more popular. According to the findings, 43% of respondents said their designs would be online all the time, with another 19% answering ‘sometimes’. “There are implications for security here,” said Barr. “If a product that could kill someone is put online – even sometimes – it opens a new attack surface.”
More than 50% of respondents are using wireless communications, while 92% said they were designing in one or more wired connections.
The survey also found that half of respondents were upgrading an existing product. “If you’ve built a garage door opener, for example, it’s a big deal when you decide to write software that connects it to a home Wi-Fi system,” said Barr.
“All of a sudden, a door opener connected to the internet is an opportunity for a hack that you didn’t have to worry about before.”
Nevertheless, designers have security on their agendas. Of those responding, 61% said security was a design requirement and 50% were making their latest project more secure than the previous one. But what kind of security did they have in mind?
The survey found a wide range of technologies in use. For example, 25% of respondents’ designs have more than four processors. They also implement different communications approaches and different operating systems – while 35% use an RTOS, 21% use Linux. “How do you secure all these?,” Barr asked. “They all have different attack surfaces. What it tells us is that every project is different and needs its own security. But the problem isn’t easy to solve; you can’t download a security package and there is no ‘one size fits all’ solution to security.”
Asked what other security issues they were addressing, designers gave a number of responses, but those at the top of the list were product tampering (52%), data theft (40%) and IP theft (37%). “Only 28% of respondents were worried about security failures resulting in injuries or death,” Barr pointed out. “There was a split between hacks that affect manufacturers and those which affect users and it appears from the survey that more designers care about their own interests than those of their customers.”
A subset of the results showed 194 respondents were working on devices that could kill and which were linked to the web. Of these, of these 15% aren’t applying coding standards, 17% don’t do coding reviews and
36% don’t do static analysis,” Girson noted. “These are practices which are well known as being effective and which are cost effective. But people are designing systems which could kill and which are on the web.”
And a worrying statistic emerging from the survey is that 20% of those designers whose systems could kill and which are online say security doesn’t matter. “That makes no sense,” Barr concluded.
