Testing efficiently an increasing challenge for design engineers

Three interacting groups The processes associated with testing fall into three interacting groups: organisation; test management; and test. Organisation When addressing test, a company must first focus on the organisational aspects, implementing a test policy and an overall test strategy. This should cover the drafting, agreement and publishing of the test specification. Once released, the specification's use should be monitored and controlled and a future review date set. Test management Once the organisational process is implemented, the second key need is the test management process. This involves the creation of a test strategy and the design of a test plan, the monitoring and control of testing and the production of a report once testing is complete. The test plan should include identification and mitigation of risks, staff requirements and the schedule. The test management process interacts with the organisational process via use of and feedback to the organisational test documentation. Test The test plan is passed to the test process, which includes the design and implementation of test cases. ISO12119 defines a test case as 'a documented instruction for the tester that specifies how a function or a combination of functions shall or should be tested'. IEEE610.12 goes further, stating that a test case is 'a set of test inputs, execution conditions and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement'. The test cases come together to form the test specification. In reality, this could be an internationally recognised document or one that has been produced internally. The test process also includes the test execution and the feedback of test metrics to the monitoring part of the test management process, allowing for the reporting of test status. Product life cycle phases Once these three processes have been put in place, we can take a bird's eye view of testing from when a project has been given the green light. There are three phases of a product's life cycle. Development Testing during the development phase includes the testing of software and hardware. Software testing can be broken into three parts; unit testing, component testing and integration testing. Unit testing, the most granular level, can be considered as I/O verification. Here, a series of valid, invalid and null inputs are fed into a method and the outputs verified against expected results. They exercise the method to check for failures, such as crashes or uncaught exceptions, or invalid results. Moving up in complexity is component testing; still at the code structure level. Here, the testing of a particular method is performed in the context of the application, rather than in isolation, usually using test hooks within the code. Integration testing focuses on interfaces and interactions, examining the end to end scenarios, verifying the behaviour during correct and incorrect user actions. It is worthwhile performing precompliance tests during the development phase – and preferably whilst the product is still in prototype. This provides confidence that the product conforms to given technology recommendations or certification requirements. For example, a Bluetooth interface would be tested to the rf and protocol requirements needed for the logo programme. It also helps a product to meet regulatory requirements for its target markets. Some tests can be performed in house – for example creepage and clearance distances between components – while other tests may require expensive equipment or specialised expertise. Performing as much precompliance testing as possible will reduce the possibility of product redesigns and launch delays. Regulatory and market requirements Once the product has moved through the development stage, the next tests ensure it is ready for market. The first stage here is system testing; functional and non functional testing of the complete product. This is best performed independently by someone with knowledge of the technologies supported. The legal barrier to product launch is covered by regulatory testing; the legal aspects for the target countries. Some areas of test apply to all products: for example, emc testing ensures that emissions are within limits and the product is immune to electromagnetic interference. Safety testing is another important aspect. Other areas are product specific, for example radio requirements for an intentional transmitter and telecoms testing for connection to a phone network. Some or all of these tests could be performed at a local test lab, but for some markets these have to be performed in country. A final aspect is logo testing, the requirements that must be met by before displaying an industry recognised logo, for example Wi-Fi or ZigBee. These tests are normally performed at test houses audited and accredited by the industry body. Robustness and maintenance Testing doesn't end once a product meets market requirements; it is important that a robust product enters the marketplace without problems which could lead to expensive recalls. Field trials ensure a product has undergone tests under real world conditions, perhaps beta testing with trial users. If a product is not designed to be part of a proprietary system, it will likely have to interact with other elements, so interoperability testing becomes essential. Some of this testing is best performed during the development phase, either with partners or at interoperability events such as plugfests. Interoperability testing also provides the opportunity for feedback to Standards Development Organisations to help make standards error free and to remove ambiguities. Before releasing an upgrade or update to a product, regression testing is necessary. If the test processes described above have been implemented correctly and the test tools designed well, then a lot of time can be saved at this stage. For example, unit, component and integration test suites will ensure that firmware updates will not cause issues elsewhere. Although each part of an organisation is usually focused and delivers within their separate areas, it is essential there is an oversight function in place. This needs to address the entire product life cycle, coordinating efforts using the organisational and test management processes. In effect, this function ensures the efficient use of the limited and distributed resources helping to organise, prioritise and document all areas of testing minimising delays to a product launch and reducing problems in the field.