Valk, Max (2022) Test selection, minimization and prioritization. Master's Thesis / Essay, Artificial Intelligence.
|
Text
Thesis_Valk_MV_Final.pdf Download (2MB) | Preview |
|
![[img]](https://fse.studenttheses.ub.rug.nl/style/images/fileicons/text.png) |
Text
toestemming.pdf Restricted to Registered users only Download (148kB) |
Abstract
A crucial component of the software development process is the performance of regression testing, which ensures that a piece of software remains functional under changes. However, for large or frequently changing software projects, the volume of required testing can outpace the resources and/or time available, resulting in the need for more efficient testing practices. In this work, we explored test runtime forecasting, outcome correlation and predictive test selection. We find that predictive test selection can also be efficiently used for test prioritization, outperforming several heuristics common in the literature. In addition, we propose improvements to predictive test selection by implementing an asymmetric loss function and pre-selection of tests based on historical runtimes, which lead to a test-time reduction of 31.1% whilst still maintaining a recall of above 0.9. We also found that runtime based forecasting at the level of test suites, rather than test cases, performed well and led to a reduction in test execution time of 24.3% while obtaining a precision of over 0.98. We have evaluated correlation-based minimization on the level of suites, and obtained a time save of 7.89% whilst maintaining a precision of 0.99%. Lastly, we propose methodological improvements to correlation-based minimization and runtime based prediction by suggesting the use of a validation set. It remains to be shown how much time these systems can save in total when deployed at the same time.
| Item Type: | Thesis (Master's Thesis / Essay) |
|---|---|
| Supervisor name: | Gaydadjiev, G. |
| Degree programme: | Artificial Intelligence |
| Thesis type: | Master's Thesis / Essay |
| Language: | English |
| Date Deposited: | 14 Oct 2022 13:44 |
| Last Modified: | 14 Oct 2022 13:44 |
| URI: | https://fse.studenttheses.ub.rug.nl/id/eprint/28760 |
Actions (login required)
 |
View Item |