Test filtering

Diffblue Cover allows users to filter generated tests based on a set of tags associated with each test, in both the IDE plugins and the Web UI.

Tags

Assertions/No assertions

There are some scenarios where Diffblue Cover is unable to produce a valid assertion for the test:

  • Void methods
  • Methods that do not exhibit side effects
  • Tests where only assertNotNull is used as an assertion

If the test uses thrown.expect in order to check that an exception has been thrown, this is counted as a valid assertion.

Exceptions/No exceptions

Diffblue Cover will write tests which cause an exception to be thrown in line with expected behavior of the code. This tag allows the user to explicitly filter tests based on whether they expect to throw exceptions or not.

High complexity

We calculate the heuristic probability that each block of a control flow will be touched by any given walk through the program. The test is then rewarded for reaching more complicated paths e.g. inner loops.

Mocking/No mocking

If Diffblue Cover does not have access to dependent code for a method at analysis time, then it will mock the behaviour of that dependency. This may also include parts of the JDK library which have yet to be modelled by Diffblue Cover.

The following classes will always be mocked:

"java.io.File",
"java.io.FileInputStream",
"java.io.FileReader",
"java.io.InputStreamReader",
"java.io.PrintWriter",
"java.lang.Process",
"java.lang.ProcessBuilder",
"java.lang.System",
"java.nio.file.Files",
"java.nio.file.Paths",
"java.security.KeyFactory",
"java.security.Signature",
"java.security.SecureRandom",
"java.util.Date",
"java.util.Random"

Natural inputs

Inputs are measured according to various criteria:

  • String inputs are checked to ensure that their length is within preferred bounds set in the analysis configuration options, and that all characters appear in the following list:
0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!\"#$%&'()*+,-./:;<=>@[\\]^_`{|}~ \t\n
  • Character inputs are checked against the same character set as string inputs.
  • Integral (int, long) inputs are checked to be non-zero and inside a specified range.
  • Floating-point (float, double) inputs are checked to be non-zero, non-infinite, non-NaN and inside a specified range.
  • Array and collection lengths are checked to be within a specified range. The members of arrays and collections are also evaluated (treated as separate inputs).
  • Tests are penalised for any null inputs.

Phase n (where n is the phase number)

If a method is not fully covered by tests generated by Diffblue Cover, the method may be analyzed again in a subsequent "phase". The next phase will use different options in order to try to cover the method fully.

Reflection/No reflection

Diffblue Cover may generate tests that use reflection as a way to access parts of the code that are protected. Below are some of the reasons why reflection might be used:

  • Inaccessible private fields or methods
  • Overly complex code
  • A lack of setters and getters

Test star ranking

The star ranking system has been developed to tag tests based on their readability, the complexity of the method under test and the quality of the inputs.

1 - 2 stars (★ - ★★)

Generated tests that have been ranked with 1 or 2 stars will include one or more of the following traits (as defined above) which may hamper readability:

  • No assertion
  • Mocking
  • Reflection

3 - 5 stars (★★★ - ★★★★★)

Generated tests that have been ranked with 3 to 5 stars are considered to have a good level of readability, and only make use of mocking and reflection where necessary. Higher levels of ranking are awarded if the method under test is sufficiently complex and/or the inputs for that test are of high enough quality (e.g. the input values use human-readable strings).

results matching ""

    No results matching ""