How to measure test quality
Use mutation testing to benchmark unit test quality
Last updated
Was this helpful?
Use mutation testing to benchmark unit test quality
Last updated
Was this helpful?
Traditional test coverage (i.e line, statement, branch, etc.) measures what code is executed by your tests. It does not check that your tests are actually able to detect faults in the executed code.
Mutation testing fills this gap. Faults (or mutations) are seeded into your code, then your tests are run. If your tests fail then the mutation is killed, if your tests pass then the mutation lived. The quality of your tests can be gauged from the percentage of mutations killed.
Source code is changed or mutated in three ways – , and .
Change a small value to a larger value or vice versa.
Change logical or arithmetic operators to detect errors.
Delete a statement or replace it with some other statement
Brings a good level of error detection in the program
Discovers ambiguities in the source code
Helps testers write better tests and measure effectiveness of tests written
Makes source code more efficient
Add this plugin to your pom.xml
:
Then run: mvn test-compile org.pitest:pitest-maven:mutationCoverage
, which will output an HTML report to target/pit-reports/<YYYYMMDDHHMI>
.
Add this plugin to your build.gradle
:
Run gradle pitest pitestReportAggregate
task to have an aggregated report be produced at ${PROJECT_DIR}/build/reports/pitest
.
(recommended)
|
For multi-module projects, please see .
|