Test Coverage Optimizations

Diffblue Cover can create many tests for a given package, class or even method, but how does Cover optimize the tests that it provides? And what options are there to help make sure that Diffblue Cover provides you the test coverage levels that you prefer?

---

Test Class Coverage Optimization (Default)

By default, Diffblue Cover will optimize test output by only providing tests which increase coverage within the target test class.

What does Test Class Coverage Optimization look like?

• For a source class Foo

• Cover will create a test class called FooDiffblueTest

• Cover will start to create test methods that exercise the contents of Foo and insert them into FooDiffblueTest

• When a new test candidate is found by Cover, and its coverage does not increase upon the coverage in FooDiffblueTest, then it will be discarded.

Through this methodology, Cover will aim to provide the minimum number of test methods to deliver the maximum coverage, in the test class.

What does Test Class Coverage Optimization not do?

• It does not consider coverage provided by test classes outside of the target test class.

• It does not guarantee that each test provides unique coverage, in fact it is highly likely that there will still be coverage overlap between test methods within the same test class, just as there would be with manual testing. However each test will provide some unique coverage, within the test class.

Disabling Test Class Coverage Optimization (ignore existing coverage)

Disabling test class coverage optimization will allow Diffblue Cover to provide the maximum number of tests, including tests which provide duplicated coverage. This may increase test strength and mutation coveage.

This can be achieved in the CLI and Plugin through use of the --ignore-existing-coverage options. See CLI: Commands & Arguments Plugin: Cover Plugin settings

Currently Test class coverage optimisation only support the JUnit Jupiter platform. For JUnit 4 / TestNG environments test class coverage optimisation will not be applied.

---

Test Suite Coverage Optimization (new jacoco coverage)

Optionally, Cover can optimize coverage across the entire test suite using the CLI option --new-jacoco-coverage. This will take into account the coverage provided by existing tests before starting to create Diffblue Tests. Tests created by Diffblue Cover are added if they increase JaCoCo coverage.

Notable effects:

• This has the ability to greatly reduce the number of DiffblueTests produced by Diffblue Cover, depending on existing test coverage levels.

• This feature will reduce the overlap between Diffblue and Manual test cases, however not all overlap will be eliminated, while maintaining the same overall Coverage levels.

• This feature may cause many Diffblue tests to be discarded that may be better quality than existing tests, or test other values or edge cases. This will likely reduce the overall strength and mutation coveage provided of the test suite.

• If used in conjunction with --ignore-existing-coverage then Diffblue Cover may still provide redundant coverage within its own test classes.

For more information see Commands & Arguments

The --new-jacoco-coverage option can be valuable for high quality, trusted, test suites, allowing Diffblue Cover to focus on adding new coverage to round out the test suite without adding code bloat.

However, if you are looking to modernise your test suite, code or are at all unsure of the veracity of your test suite, then we would encourage you to take advantage of Covers ability to generate high quality, high strength tests, at scale, by NOT using this option. Once you have your Cover augmented test suite, consider the coverage and test strength improvements Cover has provided and balance that against any coverage or test duplication, you may find that deleting flakey or poorly understood manual tests is the correct approach for you.