Towards practical application of mutation testing in industry — Traditional versus extreme mutation testing

Maik Betka, Stefan Wagner

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

Mutation testing is a technique that changes code instructions to assess the quality of automated software tests. Industry has not broadly adopted the technique because execution and analysis times are too long and not considered worth the effort. To change this, a variation called “extreme mutation testing” emerged, which mutates whole methods instead of instructions. The extreme variant trades accuracy for speed gains and also provides pre-analyzed results. In this study, we aim to analyze both techniques on their granularity levels, look for benefits when combining them, and find motivations when a developer considers killing mutants. For that, we conducted a case study in a company from the semiconductor industry. We mutated a large Java software project which is tested by more than 11,000 unit tests, analyzed the results, manually inspected more than 1000 mutants, and conducted a focus group with five developers of the software. Among other results, we provide the distribution of traditional across extreme mutants as well as qualitative coding results of our mutant inspection and focus group transcript. We conclude that the traditional approach can be similarly strategically applied as the extreme one and that motivations of developers to target mutants are mostly not code related.

Original languageEnglish
Article numbere2450
JournalJournal of Software: Evolution and Process
Volume34
Issue number11
DOIs
StatePublished - Nov 2022
Externally publishedYes

Keywords

  • code coverage
  • java programming
  • mutation testing
  • software engineering
  • software quality

Fingerprint

Dive into the research topics of 'Towards practical application of mutation testing in industry — Traditional versus extreme mutation testing'. Together they form a unique fingerprint.

Cite this