Diffblue Cover allows users to filter generated tests based on a set of tags associated with each test, in both the IDE plugins and the Web UI.
There are some scenarios where Diffblue Cover is unable to produce a valid assertion for the test:
- Void methods
- Methods that do not exhibit side effects
- Tests where only
assertNotNullis used as an assertion
If the test uses
thrown.expect in order to check that an exception has been thrown, this is counted as a valid assertion.
Diffblue Cover will write tests which cause an exception to be thrown in line with expected behavior of the code. This tag allows the user to explicitly filter tests based on whether they expect to throw exceptions or not.
We calculate the heuristic probability that each block of a control flow will be touched by any given walk through the program. The test is then rewarded for reaching more complicated paths e.g. inner loops.
If Diffblue Cover does not have access to dependent code for a method at analysis time, then it will mock the behaviour of that dependency. This may also include parts of the JDK library which have yet to be modelled by Diffblue Cover.
The following classes will always be mocked:
"java.io.File", "java.io.FileInputStream", "java.io.FileReader", "java.io.InputStreamReader", "java.io.PrintWriter", "java.lang.Process", "java.lang.ProcessBuilder", "java.lang.System", "java.nio.file.Files", "java.nio.file.Paths", "java.security.KeyFactory", "java.security.Signature", "java.security.SecureRandom", "java.util.Date", "java.util.Random"
Inputs are measured according to various criteria:
- String inputs are checked to ensure that their length is within preferred bounds set in the analysis configuration options, and that all characters appear in the following list:
- Character inputs are checked against the same character set as string inputs.
- Integral (int, long) inputs are checked to be non-zero and inside a specified range.
- Floating-point (float, double) inputs are checked to be non-zero, non-infinite, non-NaN and inside a specified range.
- Array and collection lengths are checked to be within a specified range. The members of arrays and collections are also evaluated (treated as separate inputs).
- Tests are penalised for any null inputs.
Phase n (where n is the phase number)
If a method is not fully covered by tests generated by Diffblue Cover, the method may be analyzed again in a subsequent "phase". The next phase will use different options in order to try to cover the method fully.
Diffblue Cover may generate tests that use reflection as a way to access parts of the code that are protected. Below are some of the reasons why reflection might be used:
- Inaccessible private fields or methods
- Overly complex code
- A lack of setters and getters
Test star ranking
The star ranking system has been developed to tag tests based on their readability, the complexity of the method under test and the quality of the inputs.
1 - 2 stars (★ - ★★)
Generated tests that have been ranked with 1 or 2 stars will include one or more of the following traits (as defined above) which may hamper readability:
- No assertion
3 - 5 stars (★★★ - ★★★★★)
Generated tests that have been ranked with 3 to 5 stars are considered to have a good level of readability, and only make use of mocking and reflection where necessary. Higher levels of ranking are awarded if the method under test is sufficiently complex and/or the inputs for that test are of high enough quality (e.g. the input values use human-readable strings).