Ten ways to make the debugging process more difficult

1. Don't design in debug ports It is useful to have a Jtag port, a spare uart or maybe even an Ethernet interface on the development version of the hardware, but these are luxuries; it is much better to use guesswork and inspiration. Plus, you will have to remove those debug interfaces from the production hardware. 2. Mix logging and tracing techniques By using printf() liberally, you can affect the time behaviour of your system and hide bugs that will only cause problems when printf() statements are removed. Stack overflows will add to the fun of debugging by causing unpredictable behaviour. Without a logging mechanism that can be enabled or disabled selectively, you will have more irrelevant output to look at, which makes finding what you are looking for more challenging. 3. Don't use off the shelf debugging tools Good development tools make the job too easy, they bloat the code, increase execution time by as much as 2% and it takes time to learn how to use them; time better spent sticking to printf() and guesswork. 4. Don't use hardware based debugging tools If you are tracking down a software problem, you don't want to connect hardware based debugging tools to your target. That Jtag probe is bound to be hard to use. Don't even think about using a trace box, oscilloscope or logic analyser; you might have to ask a hardware engineer for help and that is an admission of defeat. 5. Ignore your configuration management system Real programmers don't need a configuration management system. If you have one, the last thing you want to do is use it to help you during debugging – the ability to revert to previous, working, versions of the code and find the specific change that caused a problem makes the process easier than it has any right to be. Don't test changes one at a time, reverting to the baseline version after each test. Make several changes at once, then test them together. This will be the quickest way to solve the problem and it doesn't matter which change fixed it; all changes have improved the software. 6. Assume there is a bug in the compiler Mostly, this is not the case. However, it is a good reason not to look more carefully in the right places for the cause of a problem. 7. Don't keep notes Keeping notes will just slow the process of trying new things and stop you being 'in the zone'. People will say they find it helpful to review their notes to see whether they have overlooked anything, but they are kidding themselves; you really have to be working with your head in the code to make progress. 8. Ignore 'one off' behaviour If a problem cannot be reproduced, assume it no longer exists. If your system behaves oddly during one test, but not when you run the same test again, you must have imagined it. Of course, there could be marginal timing on a hardware interface, but if it only happened once, don't worry about it. 9. Never ask for help You know what is really going on. Having to explain the problem to someone else will be a waste of time; it will also mean you have to think about your assumptions. This is one of those times when you don't need a second opinion, especially if it sends you along a path you have not considered. 10. Make last minute changes It is always a good idea to make one or two last minute changes before software is released; change the compiler's optimisation level or remove your tracing/logging code. Even if these changes unearth some race conditions, they will only cause problems once in a few thousand times. But seriously … This list of 'tips', created for fun, is based on experience. Debugging an embedded system is a serious business – it is usually the most stressful part of the development process. As engineers, we are often under commercial pressure and working to tight deadlines. A lot can rest on our shoulders and being able to think clearly is a luxury that is rarely available. Author profile: Ian Willats is managing director of Pebble Bay Consulting.