To write tests using Cover Plugin for IntelliJ:

In general, if the Write Tests option is available then Cover Plugin can write tests for your methods or classes. However:

• If it is not possible for Cover to write a complete test, instead a partial test may be created - see Creating partial tests.

• If you don't want to write a complete test, Cover Plugin can help set up your tests by creating a skeleton test instead - see Creating skeleton tests.

Cover Plugin provides several icons in the "gutter" area next to methods and classes in the IntelliJ editor panel - click the icon displayed to write, update, or delete tests.

Action icons

Status icons

Disable the icons

To disable the icons in IntelliJ:

1. Go to File -> Settings.

2. Select Editor -> General -> Gutter Icons.

3. In the Diffblue Cover section, uncheck the boxes corresponding to the icons you want to disable.

The Write Tests menu option can be used throughout IntelliJ:

• Right click in the Project Tool Window.

• Right click in the Structure Tool Window.

• Right click in the Text Editor.

• Use the Diffblue Menu Bar.

The Write Tests menu option will create tests for your methods and classes but each menu option works slightly differently to produce tests for different parts of your code.

Project Tool Window

When navigating through your project, you'll likely use the project tool window in IntelliJ. Right-click on any of the elements and select Write Tests. Diffblue Cover will attempt to write tests for everything it can find within that package (license limitations may apply).

You can also select multiple classes with Shift or CTRL (CMD on MacOS). Cover will attempt to write tests for every class within your selection.

Structure Tool Window

Similar to the Project Tool Window, IntelliJ provides a Structure menu to help you navigate around your codebase. In particular, the structure menu will show classes, methods and other elements in the currently open file. As with the Project Tool Window, right-click on any method in the Structure Tool Window and select Write Tests, and you can also select multiple methods with Shift or CTRL (CMD on MacOS).

Text Editor

In the IntelliJ editor panel, you can right-click a method or class to display the Write Tests menu option - an alternative to the gutter icon, but also allows you to right-click anywhere within a class or method.

The menu option is also available when making one or more editor selections.

Diffblue Menu Bar

The Write Tests menu option is also available from the Diffblue menu bar. This will perform the exact same function as the menu option in the text editor.

Run configurations allow you to configure the environment variables and system properties that are used when Cover Plugin creates tests. It also allows you to manually specify the method, class, package, or prefix for writing tests.

Dialog

The IntelliJ run configuration dialog for Diffblue Cover contains the following elements:

• Name - Provide a useful name for the run configuration.

• Store as project file - If selected, the run configuration will be stored as a project file.

• Test creation target - Select All in package, Class, Method, or Prefixes from the drop down to set the scope of items you're writing tests for. Enter the target details as appropriate.

• Write skeleton tests - If selected, Cover Plugin will write skeleton tests instead of creating complete tests. See Creating skeleton tests for more information.

• Environment Variables - Provide a list of key/value pairs to set environment variables.

To access the system properties setting, click the Modify options button, and select Add VM Options. The VM option text box is added to the Run Configuration options - use this to add JVM system properties in the standard format, for example -Dkey1=value1.

Creating a run configuration

Automatic creation

When you write tests with Cover Plugin a run configuration is created for your selection. For example, suppose we have the sample below:

package com.example;

public class StringUtils {

  public static boolean isPalindrome(final String s) {
    for (int i = 0; i < s.length() / 2; i++) {
      char first = s.charAt(i);
      char second = s.charAt(s.length() - 1 - i);
      if (first != second) {
        return false;
      }
    }
    return true;
  }
}

If you write tests for the method, the run configuration that is created will look like the following (to display the dialog box, go to Run > Edit Configurations and select the appropriate configuration from the list):

You will notice there's a split between the class name and the method name, and that the method includes the signature of the method you've selected – this is the (Ljava/lang/String;)Z string after the colon.

Manual creation

To create a run configuration "manually", go to Run > Edit Configurations, click Add New Configuration (the + button) and select Diffblue Cover.

First select the scope, then use the Browse action on the class/package (and method, if appropriate) to make your selection. If you've selected the method scope, the signature will be populated for you.

Template

If you have a particularly large list of system properties or environment variables that you would like to use every time you use Cover Plugin to write tests, you can specify a template. (File > New Projects Setup > Run Configuration Templates > Diffblue Cover). The values you specify will be used for any future run configurations.

Prefixes

If required, you can specify what packages, classes, or methods you want to restrict these operations to, by specifying one or more prefixes in the run configuration.

Example: io.corebanking.Account

Package example

Specify a package entry point using the standard Java naming convention, plus an optional trailing dot.

Syntax: <path>[.]

Trailing dot: <path>.

Example: com.a.

Description: Write tests for all accessible methods within package <path> (in our example, com.a) and any sub-packages. For example, tests could be written for sub-packages com.a.B and com.a.b.C

No trailing dot: <path>

Example: com.a

Description: Write tests for all accessible methods within any package that begins with <path> (in our example, com.a) and any sub-packages. For example, tests could be written for com.a, com.apple, com.apricot, com.apricot.pip, etc.

Class example

Specify a class entry point using the standard Java naming convention, plus an optional trailing dot.

Syntax: <path>[.]

Trailing dot: <path>.

Example: io.diffblue.corebanking.account.Account.

Description: Write tests for all accessible methods within class <path> only - in our example, io.diffblue.corebanking.account.Account.

No trailing dot: <path>

Example: io.diffblue.corebanking.account.Account

Description: Write tests for all accessible methods within any class that begins with <path> - in our example, this could include ...Account and ...AccountException.

Method example

Specify a method entry point using the standard Java naming convention, plus an optional method descriptor.

Syntax: <path>:[<methodDescriptor>]

Example: io.diffblue.corebanking.account.Account.addToBalance:

Description: Write tests for the specified method. If the optional method descriptor is omitted, all parameter and return types will be included.

A method descriptor is a list of type descriptors that describe the parameter types and the return type of a method, in a single string. For more information on method descriptors, see section 2.1.4 of the ASM 4.0 A Java bytecode engineering library.

When writing tests, the Cover Plugin tool window is displayed. The Overview panel provides summary progress and status information for each stage when writing tests - click an item to review detailed status and results in the Details panel, including preflight and environment check results, a Diff Viewer for reviewing tests written, test links, output codes, and general information.

Preparing

During this stage, Cover Plugin runs initial checks to ensure it can write tests. If there are any issues, the Overview panel will tell you which check failed, along with an explaination to help you resolve it. Example issues may include problems with licensing or compiling the project.

Checking environment

Next, Cover Plugin will check your project's environment, to make sure it is set up correctly. This checks your project and its dependencies against Diffblue specifications and requirements. If Cover Plugin detects something outside of these requirements you may see warnings or errors in the Details panel - these can be expanded to display more information to help you resolve the issue and some may have automatic fixes (see Fix issues below).

Fix issues

Certain errors and warnings reported during the Checking environment stage may have automated fixes available - click the Fix Issue button in the Details panel and Cover will attempt to automatically apply the fix to your project.

Writing tests

Finally, Cover Plugin will write your tests. The Overview panel displays the progress for each method (grouped by class) - select an item to review created tests, as well as to view details such as test links and general information for your tests once they have been reviewed.

Cancel tests

If you need to cancel test creation at any point, click the red Stop button.

When you click Stop, any remaining methods will be marked as cancelled and tests will not be created for them. Any tests that have already been written will be retained for review and can be merged into your codebase once accepted.

Note that cancelling test creation may not happen immediately. Cover will attempt to stop gracefully, so you may have to wait for its current progress to complete before the process is halted.

Restart test generation

Once test generation has finished, you can click on the restart button to rerun the test generation process.

When you click Restart, the tool window will be cleared and Cover will attempt to write tests on the exact same configuration of methods or classes.

Open logs

Please see accessing logs for more information.

Open Support request

If you need to raise a support request, click the Open support request button (not available for all users, please see Cover Editions for more details).

Test Review allows you to edit, analyse and manage each of the tests Diffblue Cover creates for you, as soon as those tests are available.

When enabled, you are given the opportunity to review and change the tests before they get added to your codebase.

Beginning a review

To perform a review, simply write tests as normal. The number of tests available for review is shown in the tool panel. Once the first test is created the Review Now button will become active; clicking this will take you to the first test waiting to be reviewed.

Reviewing a test

To help you review your tests, we display them inside a Diff Viewer. When comparing side-by-side, on the left you'll see your original test file (the file your tests will be inserted into). On the right, you'll see Diffblue Cover's test suggestion. This includes the test that has been created, as well as any other modifications to the test class required for the test to run.

You can use any of the features of the Diff Viewer to help you review your test, you can even edit the test inside the Diff Viewer if you wish.

Accepting a test

Once you're happy with the test suggestion, you must accept it for the test to be added to your test file. If you have edited the test, these changes will be included when you accept.

Accepting all tests

You might wish to inspect each test, and accept them all in one action. In which case, when you are happy with your tests, you can use the dropdown menu of the accept button and click "Accept all".

Rejecting a test

If you do not wish to keep a test suggestion, for whatever reason, you can reject it. All tests can be rejected in one click using the "Reject all" option within the dropdown menu of the reject button.

Navigating through tests

Once you have accepted or rejected a test, you will be automatically taken to the next test requiring your review.

However, If you wish to inspect multiple tests before taking an action, or you wish to review a specific test, you can navigate through the tests to find what you want. The tree on the left displays all methods under test. If a method has any test requiring review, it will display a icon. Once all tests for the method have been reviewed, this icon will be replaced according to the status of the tests.

There may be situations where Diffblue Cover creates multiple tests for a single method. In this case, a list of test methods will appear on the left. You can click through the items in this list to review the tests you're most interested in. This list can be resized or collapsed at any point while it is open.

Alternatively, you can navigate through these tests with the left/right arrow icons below the Diff Viewer.

Enabling/Disabling Test Review

From Diffblue Cover IntelliJ Plugin 2025.03.01, Test Review is enabled by default - but it can be turned on or off at any time. When Test Review is disabled, tests will automatically be accepted and merged into your codebase.

To enable or disable Test Review, follow these steps:

1. Open the Diffblue Settings menu via Diffblue -> Change Settings

2. Find the section titled "Test Review".

3. Check or uncheck the Test Review checkbox.

4. Apply the settings.

In this topic we show both the input source code, and the tests written by Diffblue Cover, for code of varied complexity, accompanied by a detailed narrative to help you further understand tests created by Diffblue Cover.

Basic assertions

This is a simple example of Spring Service source code with a trivial getter. Cover can write a Spring Boot test for the getter in a service provider by inlining Arrange, Act, and Assert.

Source

import org.springframework.stereotype.Service;

@Service
public class SimpleService
{
  public String getValue() {
    return "a really simple service";
  }
}

Test written by Diffblue Cover

@SpringBootTest
@RunWith(org.springframework.test.context.junit4.SpringRunner.class)
public class SimpleServiceDiffblueTest {
  @Autowired
  private SimpleService simpleService;
  @Test
  public void diffbluetestGetValue() {
    // Arrange, Act and Assert
    assertEquals("a really simple service", this.simpleService.getValue());
  }
}

Mocking

This is a class that contains two methods to upload and download a file to/from an Amazon S3 bucket. As the Amazon S3 bucket is a cloud storage mechanism, the test for this class/methods requires dependency injection. Cover can mock these types of dependencies and tests downloadFileFromBucket method by asserting the expected S3Object and the downloaded S3Object using the method.

Source

@Service
public class AmazonService {

  @Autowired
  private AmazonS3 s3client;

  public PutObjectResult uploadFileToBucket(String bucketName, String key, File file) {
    return s3client.putObject(bucketName, key, file);
  }

  public S3Object downloadFileFromBucket(String bucketName, String key) {
    return s3client.getObject(bucketName, key);
  }
}

Tests written by Diffblue Cover

@SpringBootTest
public class AmazonServiceDiffblueTest {
  @MockBean
  private AmazonS3Client amazonS3Client;
  @Autowired
  private AmazonService amazonService;
  @Test
  public void diffbluetestUploadFileToBucket() {
    // Arrange
    PutObjectResult putObjectResult = new PutObjectResult();
    putObjectResult.setContentMd5("file-hash");
    when(this.amazonS3Client.putObject(or(isA(String.class), isNull()), or(isA(String.class), isNull()),
        or(isA(File.class), isNull()))).thenReturn(putObjectResult);

    // Act and Assert
    assertSame(putObjectResult, this.amazonService.uploadFileToBucket("foo", "foo",
        Paths.get(System.getProperty("java.io.tmpdir"), "test.txt").toFile()));
  }
  @Test
  public void diffbluetestDownloadFileFromBucket() throws UnsupportedEncodingException {
    // Arrange
    StringInputStream objectContent = new StringInputStream("file-name");
    S3Object s3Object = new S3Object();
    s3Object.setObjectContent(objectContent);
    when(this.amazonS3Client.getObject(or(isA(String.class), isNull()), or(isA(String.class), isNull())))
        .thenReturn(s3Object);

    // Act and Assert
    assertSame(s3Object, this.amazonService.downloadFileFromBucket("foo", "foo"));
  }
}

OS agnostic assertions

This example returns time in a different format. Cover writes a test to assert date and time that is not dependent on operating system or local time zones.

Source

public class TimeInAliceWonderland {

  private static final String DODO_DATETIME_FORMAT = "ss:mm:HH dd-MMM-yyyy";

  public static String reformatDodoDateTime(Date humanDate) {
    Map<String, DateFormat> dateFormatMap = new HashMap<>();
    dateFormatMap.put(DODO_DATETIME_FORMAT, new SimpleDateFormat(DODO_DATETIME_FORMAT));
    return dateFormatMap.get(DODO_DATETIME_FORMAT).format(humanDate);
  }
}

Test written by Diffblue Cover

@Test
  public void diffbluetestReformatDodoDateTime() {
    LocalDateTime atStartOfDayResult = LocalDate.of(1970, 1, 1).atStartOfDay();
    assertEquals("00:00:00 01-Jan-1970", TimeInAliceWonderland
        .reformatDodoDateTime(
            Date.from(atStartOfDayResult.atZone(ZoneId.systemDefault()).toInstant())));
  }

Example containing logic

In this example, Cover writes two tests to cover each pathway through the conditional - a test which adds 10 to the balance and asserts the new balance should be 20 and a second test where the account is closed.

Source

public class Account {
  private final long accountNumber;
  private final Client client;
  private long currentBalance;
  private String accountName;
  private AccountState accountState;

  public Account(final long accountNumber, final Client client, final long amount) {
    this.accountNumber = accountNumber;
    this.client = client;
    currentBalance = amount;
    accountName = "Current";
    accountState = AccountState.OPEN;
  }

  public long getCurrentBalance() {
    return currentBalance;
  }

  public void addToBalance(final long amount) throws AccountException {
    if (getAccountState() != AccountState.OPEN) {
      throw new AccountException("Cannot add to balance, account is closed.");
    }
    currentBalance += amount;
  }

Test written by Diffblue Cover

 @Test
  public void diffbluetestAddToBalance() throws AccountException {
    // Arrange
    Account account = new Account(1234567890L, new Client("Carlos Welch"), 10L);

    // Act
    account.addToBalance(10L);

    // Assert
    assertEquals(20L, account.getCurrentBalance());
  }

Example of complex logic

This example code has three branches - Cover covers 100% of branches and creates three tests for three cases using existing enum values.

Source

public class VirtualSnackCupboard {

  public static Snack whatCanISnackNow(int oClock) {
    if (oClock == 10) {
      return Snack.CHOCOLATE;
    } else if (oClock == 15) {
      return Snack.CAKES;
    } else {
      return Snack.VEGGIE;
    }
  }
  enum Snack {VEGGIE, CHOCOLATE, CAKES}
}

Tests written by Diffblue Cover

  public void diffbluetestPickSnack() {
    assertEquals(VirtualSnackCupboard.Snack.CHOCOLATE, VirtualSnackCupboard.whatCanISnackNow(10));
    assertEquals(VirtualSnackCupboard.Snack.CAKES, VirtualSnackCupboard.whatCanISnackNow(15));
    assertEquals(VirtualSnackCupboard.Snack.VEGGIE, VirtualSnackCupboard.whatCanISnackNow(0));
  }

Testing trivial methods

Diffblue Cover CLI deliberately does not test trivial methods such as getters, setters, constructors and factory methods to avoid creating large quantities of noisy tests that do not contribute to overall coverage. There isn't a benefit to be gained from testing these trivial methods directly as, for example, getters and setters simply access attributes so there is no actual logic to be tested. Similarly, constructors and factory methods also contain little or no logic. If Diffblue Cover CLI did test such methods directly, a minor coverage increase might be gained, but this "improvement" would be meaningless in terms of improving code quality.

Ultimately these trivial methods will be covered when Cover generates tests for other methods that call these trivial methods. Diffblue recommends improving the percentage of coverage using the many options within the product, as shown in the Commands & Arguments topic.

About partial tests

Normally, selecting or Write Tests will create complete tests for a method. A complete test consists of Arrange, Act, and Assert sections and passes when executed. However, sometimes Diffblue Cover is not (yet) able to write complete tests and will give the reason for not producing complete tests in the Diffblue Cover tool window. In these cases, Diffblue Cover will create partial tests - these tests may be incomplete in various aspects:

• The test does not have assertions.

• The test does not always pass when executed.

• The Arrange or Act sections produce an error when executed.

• The test may execute code that is potentially harmful to your system, leaks resources or times out.

Creating such tests is useful to developers for two reasons:

• Firstly, it will save developers time. Instead of writing tests from scratch, they'll be able to use the partial tests as a starting point.

• Secondly, it may help developers understand why Diffblue Cover was not able to complete the test automatically, especially in cases where the reason for not producing a test is lack of testability.

For example, for the method increment in the following class:

public class MyClass {
    private int x;
    public void increment() {
	  ++x;
    }
}

Diffblue Cover will create a partial test:

  @Test
  public void testIncrement() {
    // TODO: This test is incomplete.
    //   Reason: R002 Missing observers.
    //   Diffblue Cover was unable to create an assertion.
    //   Add getters for the following fields or make them package-private:
    //     MyClass.x

    // Arrange
    MyClass m = new MyClass();

    // Act
   m.increment();
  }

The problem is clearly that Diffblue Cover has no opportunity to assert on the side effect of the increment method on field x, which is marked private. The developer could add a getter to MyClass:

public class MyClass {
    private int x;
    public void increment() {
	  ++x;
    }
	public int getX() {
	  return x;
	}
}

Selecting or Write Tests again would now create a complete test.

Disabling the creation of partial tests

This feature is enabled by default. To disable the creation of partial tests, go to Diffblue > Change Settings > Partial Test Creation and uncheck the boxes for the relevant Allow writing tests ... options. This will disable the creation of partial tests unless no other tests could be created. If no other tests could be created when selecting or Write Tests then Diffblue Cover will create a skeleton test with simple inputs to get you started.

Skeleton tests simply consist of a call to the method under test, with all variables initialized to null or zero, and some comments reminding the user to better populate the "arrange" section and come up with some assertions. They are intended as a helpful starting point for users to write their own tests.

There are two ways of obtaining a skeleton test for a method in Diffblue Cover:

• Click or select Write Tests - Diffblue Cover will analyze the method and related classes to write one or more complete tests (or partial tests if this is enabled) exploring the coverage that can be accessed from each method under test. Sometimes this approach is unsuccessful and rather than give you nothing, Diffblue Cover falls back to offering a single skeleton test for the method instead. In this case, the skeleton test provided is disabled, just like any other partial test that is not expected to pass.

• Right-click and select Write Skeleton Tests - Diffblue Cover won't analyze your code to write complete tests but instead will only create a single skeleton test for the method. In this case, the skeleton test provided is not disabled as it's assumed that you will complete the test by hand immediately. This can be useful when you need to write a test reproducing a specific scenario and just want some help getting started.

Skeleton tests are intentionally written with all available comments to aid readability, and minimal inlining to ease editing, which means that all test formatting options will be superseded.

For example, consider the method in the following inner class in an outer class, DatabaseDao:

public class DatabaseDao {
    public static class Inner {
        private Inner() {
        }

        public void myMethod(Inner inner) {
        }
    }
}

Diffblue Cover is unable to test this by default as there doesn't appear to be a way of creating an instance of Inner on which to call myMethod(Inner), and so this results in an R008 output code with some explanation why a full test could not be written. Rather than provide just the output code, Diffblue Cover will offer a skeleton test for the method as follows:

/**
  * Method under test: {@link DatabaseDao.Inner#myMethod(DatabaseDao.Inner)}
  */
@Test
@Disabled("TODO: Complete this test")
void testInnerMyMethod() {
    // TODO: Complete this test.
    //   Reason: R008 Failed to instantiate class under test.
    //   Diffblue Cover was unable to construct an instance of DatabaseDao.Inner.
    //   Add a package-visible constructor or a factory method for the class under test.
    //   If such a method is already present but Diffblue Cover does not find it, it can
    //   be specified using custom rules for inputs:
    //   https://docs.diffblue.com/knowledge-base/cli/custom-inputs/
    //   This can happen because the factory method takes arguments, throws, returns null
    //   or returns a subtype.
    //   See https://diff.blue/R008

    // Arrange
    // TODO: Populate arranged inputs
    DatabaseDao.Inner inner = null;
    DatabaseDao.Inner inner1 = null;

    // Act
    inner.myMethod(inner1);

    // Assert
    // TODO: Add assertions on result
}

This skeleton test serves as a helpful illustration of the output code, allowing you to play with your code and better understand why testing the method is difficult, and what could be changed to make it easier. In other cases you will know a way of obtaining an instance to test with, in which case you are well placed to complete the test by hand, choosing useful values for the variables and adding relevant assertions. Alternatively, you may want to "teach" Diffblue Cover how to create useful instances via Custom Inputs.

Cover Plugin for IntelliJ can write tests which ensure that all possible values of an Enum type are covered by the test suite, when that Enum is passed to a method as an argument or is a return value of a method.

With and without "Cover all enum values"

To understand the difference that the Cover all enum values feature makes to created tests, we can write tests on a simple class which takes an Enum type as an argument and behaves in different ways depending on that argument's value.

public class ErrorFormatter {

  private static final String NON_FATAL_ERROR_TEMPLATE =
      "<p style=\"color:orange\"><em>non-fatal: %s</em></p>";

  private static final String FATAL_ERROR_TEMPLATE =
      "<p style=\"color:red\"><strong>FATAL! %s</strong></p>";

  public String asHtml(ErrorLevel level, String message) {
    switch (level) {
      case INFORMATIONAL:
      case WARNING:
        return String.format(NON_FATAL_ERROR_TEMPLATE, message);
      case FATAL:
        return String.format(FATAL_ERROR_TEMPLATE, message);
      default:
        throw new AssertionError(String.format("Unrecognised error level %s", level));
    }
  }
}

Note in particular that the method exhibits the same behavior when the level argument is INFORMATIONAL or WARNING. Two tests that exercise this method by passing those two values to its first argument will therefore cover the same lines of code.

When we write tests on this method with Cover all enum values disabled, then Cover Plugin creates these tests:

@Test
public void testAsHtml() {
  assertEquals("<p style=\"color:orange\"><em>non-fatal: Not all who wander are lost</em></p>",
      (new ErrorFormatter()).asHtml(ErrorLevel.INFORMATIONAL, "Not all who wander are lost"));
  assertEquals("<p style=\"color:red\"><strong>FATAL! Not all who wander are lost</strong></p>",
      (new ErrorFormatter()).asHtml(ErrorLevel.FATAL, "Not all who wander are lost"));
}

When we enable Cover all enum values and write tests again, Cover Plugin creates these tests:

@Test
public void testAsHtml() {
  assertEquals("<p style=\"color:orange\"><em>non-fatal: Not all who wander are lost</em></p>",
      (new ErrorFormatter()).asHtml(ErrorLevel.INFORMATIONAL, "Not all who wander are lost"));
  assertEquals("<p style=\"color:orange\"><em>non-fatal: Not all who wander are lost</em></p>",
      (new ErrorFormatter()).asHtml(ErrorLevel.WARNING, "Not all who wander are lost"));
  assertEquals("<p style=\"color:red\"><strong>FATAL! Not all who wander are lost</strong></p>",
      (new ErrorFormatter()).asHtml(ErrorLevel.FATAL, "Not all who wander are lost"));
}

With Cover all enum values enabled the Cover Plugin creates an extra test, to ensure that both the INFORMATION and WARNING levels are tested, even though the extra test may not increase the test suite's measured coverage.

Disable "Cover all enum values"

The Cover all enum values feature is enabled by default. To disable, go to Diffblue > Change Settings > Test Creation in IntelliJ, and uncheck the Allow writing tests for all enum values option.

Created test methods are inserted into a test class in the same order that the methods under test appear in the source class (Diffblue Cover does not add a duplicate test if exactly the same test already exists).

Source Class

public class MyClass {
    public void xyz() {
    }
    public void abc() {
    }
}

Test Class

public class MyClassDiffblueTest {
    public void testXyz() {
    }
    public void testAbc() {
    }
}

As illustrated above, the test methods testAbc and testXyz are not inserted alphabetically into the test class, but instead according to how the methods under test are ordered. Test methods which test the same method are inserted in the order in which they are generated by Cover Plugin.

Diffblue Cover writes unit tests by running your code thousands of times as it searches for the best tests that achieve maximum coverage and regression detection. By default, your code is run inside a sandbox which blocks calls with potentially disruptive side-effects such as network, file system, or system changes. However, this can also limit the number of tests that Diffblue Cover can write. Disabling the Diffblue sandbox will allow Cover to run all of your code regardless of side-effects, thereby maximizing coverage. Disabling the sandbox must be done with caution as your code may behave in ways you don't expect (e.g. file system modifications).

Note that if you receive the output code R011 when using Diffblue Cover, this means the method tested performs operations that violate Diffblue Cover’s sandbox policy.

Sandbox Policy

Disabling the Diffblue Sandbox

The sandbox policy can be disabled to allow almost all of the operations outlined above (except calling System.exit). However, we recommend that you fully familiarize yourself with the information contained in this topic in order to avoid any undesirable effects by allowing these previously denied operations.

To disable the sandbox policy, go to Diffblue > Change Settings > Sandboxed Environment in IntelliJ and uncheck Enable sandboxing.

Note that if you just want to allow a small set of JNI libraries, use the JNI allow list (a comma separated list of JNI library name prefixes) - go to Diffblue > Change Settings > Sandboxed Environment > Allowed JNI prefixes.

Also, you may want to consider simply refactoring parts of your code. For example, if you have a method that executes a denied operation but also contains business logic that needs to be verified, you could refactor your code so that any logic that can be executed within the sandbox is contained in a separate, unit-testable method - this ensures the business logic is tested and consequently increases code coverage.

Harmful Example

Let's look at an example of a denied operation and what could happen if it were allowed. One of the operations prevented by default is accessing files outside of the temp directory (specified by java.io.tmpdir). We further restrict this by allowing only writing to newly created files. The root parameter may end up pointing to a real directory on your system (it could be given any value, such as /, C:\, /home/User/), and then be executed, removing all the files contained within that directory along with the directory itself. With the default policy in place, trying to write tests for this method would result in an R011 output code.

public class Harmful {
    public void deleteRecursively(final Path root) {
        final File[] contents = directoryToBeDeleted.listFiles();
        if (contents != null) {
            for (final File file : contents) {
                deleteRecursively(file);
            }
        }
        return directoryToBeDeleted.delete();
    }
}

Operations

The tables below outline the denied operations (Diffblue sandbox enabled) and explains what can happen (in the worst case) if they are allowed (Diffblue sandbox disabled). Note that:

• The exact nature of the risk is heavily dependent on the application being analyzed and can't be predicted ahead of time. If the application being analyzed is a web-based application in an automated pipeline that has no access to production systems, the risk of experiencing many of the adverse effects mentioned below is relatively low.

• The impact of the Diffblue sandbox policy is not restricted to just the method, but methods that call these methods, and so on.

I/O Operation

• Data loss is likely to occur if methods (like a hypothetical delete method in a typical web application) are invoked and happen to have access to live credentials (e.g. production or test environments).

• Likewise, unexpected network traffic can occur if the application makes raw network requests to servers.

• Disk space may be consumed because the application may not clean up temporary files.

Java Libraries

• Trying to write unit tests for methods that invoke GUIs is difficult. In those cases, the java.awt.headless property is usually set to true to disable those GUI components. If you require tests for the logic behind the GUIs, it's better to refactor the code to allow that logic to be tested independently of the GUI.

• The sun.misc.* packages provide very low level access to fundamental system operations and do not form a part of the public interface. Because of the very low level access, we deem these operations unsafe due to the risk of unintended side effects.

• Furthermore, the following is taken from the published Oracle FAQ regarding the use of these packages.

The sun.* packages are not part of the supported, public interface.

A Java program that directly calls into sun.* packages is not guaranteed to work on all Java-compatible platforms. In fact, such a program is not guaranteed to work even in future versions on the same platform.

Each company that implements the Java platform will do so in their own private way. The classes in sun.* are present in the JDK to support Oracle's implementation of the Java platform: the sun.* classes are what make the Java platform classes work "under the covers" for Oracle's JDK. These classes will not in general be present on another vendor's Java platform. If your Java program asks for a class "sun.package.Foo" by name, it may fail with ClassNotFoundError, and you will have lost a major advantage of developing in Java.

Technically, nothing prevents your program from calling into sun.* by name. From one release to another, these classes may be removed, or they may be moved from one package to another, and it's fairly likely that their interface (method names and signatures) will change. (From Oracle's point of view, since we are committed to maintaining the Java platform, we need to be able to change sun.* to refine and enhance the platform.) In this case, even if you are willing to run only on Oracle's implementation, you run the risk of a new version of the implementation breaking your program.

In general, writing java programs that rely on sun.* is risky: those classes are not portable, and are not supported.

System Operations

* Note that if the Diffblue sandbox policy is blocking access to a JNI library that you believe to be safe to use, then you can add it to the JNI allow list (a comma separated list of JNI library name prefixes can be used) - go to Diffblue > Change Settings > Sandboxed Environment > Allowed JNI prefixes.

Peripherals

Generally, these operations require access to specific hardware which may or may not be present. Because this can't be determined beforehand, unit tests for methods using these APIs would be inherently unstable.

Security

A part of writing tests is to determine whether the environment is configured correctly. This environment check is performed immediately before writing tests and can take some time.

Since the environment does not change that often, Cover will cache the results of a check and reuse the results in subsequent write test operations. This helps to deliver tests faster.

If some of the properties of the environment change, the cache will be automatically invalidated and Cover will run a new check the next time tests are written.

Clearing the Environment Check Cache

It is possible that occasionally changes to the environment are not detected by Cover and the cache will be used when it shouldn't be.

In this case, an error will be shown when writing tests. To clear the cache and force Cover to check the environment either:

• Select the Clear Environment Check Cache option from the main Diffblue menu

• Press the Clear Environment Check Cache button in the Diffblue Cover tool panel

The next write tests operation will then check the environment for changes.