Prioritization and parallel execution of test cases for certification testing of embedded systems

Abstract

Testing for certification of embedded systems is common in consumer electronics domain. Application vendors provide test suites that have to be repeatedly executed for each product. Multiple test stations can be used in parallel for test execution. Test execution tasks must be prioritized and scheduled such that the overall test execution time is minimized. We present heuristics and an integer linear programming model for this purpose. Many of the existing heuristics and models assume a sequential test execution. Others fall short either in scalability or in finding the optimal solution. We perform an industrial case study to evaluate and compare solution approaches. We measure effectiveness in terms of the rate of fault detection while taking parallel execution and varying test execution times into account. We use certification test suites of 3 Smart TV applications applied on 3 projects as real experimental objects. We compare our results with respect to those obtained with the state-of-the-practice and 3 other previously proposed approaches. Results show that the overall test execution time can be reduced by up to 16% even when only 3 test stations are available. Test effectiveness is also improved by up to 40.7% as a result of optimal scheduling of test cases.