How to improve code coverage

Introduction

Diffblue Cover is designed to create tests that provide good coverage for your code “out of the box” without needing any configuration. That said, the amount of coverage you get is also dependent on the testability of your code, the build configuration of the project, and also the specific domain or environment that your code operates within. In this section we’ll give you several ways to increase your code coverage.

Completely compile your project

This might sound obvious, but sometimes getting a complete compilation of a project can be a high bar! If the project is not completely compiled, Cover won’t be able to write a full set of tests because it works on compiled bytecode.

Run the command to build the entire project, which is for example something like mvn install -DskipTests for Maven projects - the docs for the project should tell you. See Compiling your project successfully for further troubleshooting.

IntelliJ-specific compilation advice

If you're using IntelliJ, note that sometimes it isn’t obvious that the project didn’t completely compile due to the interaction between IntelliJ and the underlying build system. Compiling the project externally by running a Maven command from the command line or a run configuration is not sufficient.

IntelliJ needs to import the build configuration which can be achieved by selecting Maven or Gradle from the right side bar and clicking the Reload All Projects button (two circular arrows symbol). The project must compile correctly using the Build Project button (green hammer symbol).

Fix missing/broken dependencies

Diffblue Cover works by running your code, and so it needs all the dependencies to be correct. Crucially, it requires the dependencies listed in the build system to match the actual dependencies in the code. In our experience, it's common to see discrepancies between the POM file (Maven) dependencies and the code due to the complexity of dependency management and Java’s dynamic class loading.

If your project is fully compiled and you are seeing class loading errors in the output of Cover, it’s usually a sign there’s a missing dependency, or an incorrect version of a dependency.

• Code that is deployed on an application server such as Tomcat may make use of provided-scope dependencies. Such dependencies are assumed to be provided by the application server, but Cover does not have access to these dependencies.

• Dependencies-of-dependencies can also be a problem: they are not required for building the project because none of their classes are directly referenced, but they are required for running your code because they are referenced from a class in your dependencies. For example, interfaces and implementations are often packaged separately; whereas your code compiles with just the interface (often called api) dependency, it does not execute without having the dependency containing the interface implementations on the classpath too. Similarly, mock implementations of the API required for unit testing may be packaged separately and need to be added to test scope.

In either situation, you have to make sure that these dependencies are indeed part of the compile and test classpaths. Cover outputs the classpath that it receives from Maven or Gradle in its log file. If required dependencies are missing there, then change the scope of these dependencies in the build system configuration.

Allocate more memory / CPU

Cover requires at least 8GB of memory and 2 cores to function, but the more memory and CPU you can give it, the faster it goes. When Cover conducts a search for the best test for your code, a slow machine with insufficient memory can result in the search timing out, producing fewer tests. Ensure that sufficient memory is available to Cover by following the advice in Memory management.

You may also see warnings when your machine is heavily loaded, e.g. because you have multiple applications running that take up most of the cores of your machines (browser, video conferencing, IDE, compilation). In such cases, there is not enough CPU available to run Cover and it will slow down too much to operate properly. You have to reduce the load on your machine or make additional cores/memory available to Cover. If you are running Cover inside a VM or a container, change the settings to allocate more vCPUs and/or more memory to the VM and restart.

Disable sandboxing

By default, Cover runs with the Java sandbox turned on because we do not want to damage your system environment by wiping your filesystem, sending sensitive data on the network, or deleting your database. Also, if that code writes to the file system or does network I/O then we run the risk of non-deterministic behavior. We try to mock out these kind of I/O operations, but there are situations where they are beyond our control.

The net: disabling the sandbox when you're running in a safe environment could allow Diffblue Cover to get more coverage for this code, and then you can look at refining mocking controls and other behavior.

Use the --disable-sandbox argument to disable the Diffblue sandbox - see Diffblue Sandbox and Commands & Arguments for details.

Provide hints on test inputs

The hardest part about writing unit tests is creating the right inputs for your method under test. Cover is pretty smart about constructing inputs "out of the box", using a variety of techniques and heuristics. But when there are highly specific requirements on test inputs, for example specifically formatted strings, Cover may not be able to create the right input. We provide a couple of ways how you can provide hints for Cover about how to create the most useful inputs when writing tests for your project.

Annotations

Java Annotations are a powerful method for adding metadata to Java code. We provide annotations that enable you to tell Cover which specific values you want to see in your tests for a particular method or which objects to mock.

For example the following method is annotated with some genuine examples of song titles that can be used to achieve coverage:

public static boolean isDayRelatedSongTitle(@InTestsUseStrings({"I Don't Like Mondays", "Here Comes The Weekend"}) String title) {
    return Stream.of(DayOfWeek.values())
            .map(DayOfWeek::name)
            .map(String::toLowerCase)
            .anyMatch(title.toLowerCase()::contains);
}

See Cover Annotations to find out more.

Rules

More complex rules for creating test inputs can by codified via a file. For example, you can tell Diffblue Cover to use a particular factory method whenever creating an object of a particular type as shown below.

Consider that you are working with a library project and need to deal with International Standard Book Numbers in the form of an org.example.ISBN class objects, created by parsing ISBN formatted strings. On it's own, Cover may not be able to discover valid ISBN string formats. To tell Cover what to do in this specific situation, you can create a DiffblueRules.yml file in the root of the module containing the following factory rules that specify three valid ISBN strings that can be used to create test objects of type org.example.ISBN:

com.example.ISBN:
  - factory:
      method: com.example.ISBN.parse:(Ljava/lang/String;)Lcom/example/ISBN;
      params: [ "1-56619-909-3" ] # An 'old' style ISBN-10 formatted identifier
  - factory:
      method: com.example.ISBN.parse:(Ljava/lang/String;)Lcom/example/ISBN;
      params:
        - "978-1-56619-909-4" # A 'new' style ISBN-13 formatted identifier
  - factory:
      method: com.example.Platform.getBean:(Ljava/lang/Class;)Ljava/lang/Object;
      params: [ com.example.ISBN ]

See Customizing test inputs to find out more.