Cover Optimize & Gradle

Cover Optimize speeds up the time required to run Java unit tests by running only the tests in your project that are relevant to your code change. The Cover Optimize for Gradle plugin accepts a patch file, analyses the code change and determines which Java unit tests in your project need to be run in order to exercise the changes made by the patch.

To use the Cover Optimize for Gradle plugin, Cover CLI (release >= 2022.03.02) must be installed and activated with an appropriate license. Further installation details can be found here.

Overview

Diffblue Cover Optimize can be integrated into your CI with the help of the Cover Optimize for Gradle plugin. This plugin acts as a wrapper around the Cover CLI, invoking Diffblue Cover Optimize before the Gradle Test task. The plugin calculates the impact of the patch file on your codebase and applies filters to the Test task to ensure that Gradle only runs the tests relevant for the changes introduced.

To configure Diffblue Cover Optimize in your Gradle project you will need to update your build script (build.gradle or build.gradle.kts) to:

1. Install the Cover Optimize for Gradle plugin.

2. Configure the optimizeTests task

The optimizeTests task examines the code in your classpath to calculate the effects of your patch file. It passes the resulting test filters to the test task via an accompanying task updateTestFilter.

The following diagram shows the dependencies between these tasks required to add Diffblue Cover Optimize to your Gradle project:

Additionally optimizeTests applies an internal dependency on compileTestJava.

Configuring build system: Gradle

Install the Cover Optimize for Gradle plugin

The Cover Optimize for Gradle plugin is distributed as a JAR file, available from the Diffblue Public Maven Repository.

GROOVY

To install the plugin into your project you will need to make changes to your build script:

1. Add the Diffblue Public Maven Repository as a buildscript repository.

2. Add the plugin's JAR as a buildscript dependency.

3. Activate the Cover Optimize for Gradle plugin.

The following detail is from an example Gradle build script which installs the Cover Optimize for Gradle plugin:

buildscript {
  repositories {
    // Add the Diffblue Public Maven Repository
    maven {
      url "https://maven.diffblue.com/release"
    }
  }
  dependencies {
    // Add the Plugin's JAR
    // specify the version that corresponds with your installed version of Cover CLI
    classpath 'com.diffblue.cover:com.diffblue.cover.gradle.plugin:[diffblue-cover-version]'
  }
}
// Activate the Cover Plugin
apply plugin: 'com.diffblue.cover'

KOTLIN

To install the plugin into your project you will need to make changes to your build scripts:

1. Add the Diffblue Public Maven Repository as a pluginManagemnet repository.

2. Activate the Cover Plugin.

The following pluginManagement section in settings.gradle.kts allows the Cover Gradle Plugin to be installed from the Diffblue Public Maven Repository:

pluginManagement {
    repositories {
        // Add the Diffblue Public Maven Repository
        maven("https://maven.diffblue.com/release")
    }
}

The following applies the Cover Gradle Plugin into your Gradle project, and makes the CoverOptimizeTask available for configuration later:

import com.diffblue.cover.gradle.plugin.CoverOptimizeTask

plugins {
  id("com.diffblue.cover") version "[diffblue-cover-version]"
}

In the above example, you should replace [diffblue-cover-version] with the version number of Diffblue Cover you are using (e.g. 2022.03.02). Run dcover version to determine your current installed version.

For more documentation on this process please review the Gradle documentation: Applying plugins with the buildscript block.

Configure the optimizeTests Task

The Cover Optimize for Gradle plugin supplies the optimizeTests task which is run before the Gradle Test task and identifies the unit tests which are required to test your patch file.

You also need to configure a series of Gradle properties to control the optimizeTests task from the command line.

To configure the optimizeTests task you will again need to make changes to your build script:

1. Configure the optimizeTests task to calculate the impact of the patch file on your codebase and select relevant test filters.

2. Configure an accompanying task updateTestFilter to apply selected test filters.

3. Specify dependencies between the tasks to ensure they coordinate correctly with the Gradle Test task.

4. Also specify Gradle properties to control the behaviour of the optimizeTests from the command line.

The following snippet is from an example Gradle build script which configures the optimizeTests task:

GROOVY

// Define a Gradle property to control the execution of the optimizeTests task
// This will be useful for running the tasks without optimization (e.g. -PskipTestOptimizer=true)
def skipTestOptimizer = project.hasProperty('skipTestOptimizer') ? project.getProperty('skipTestOptimizer').toBoolean() : false

// Configure the optimizeTests task with required fields
optimizeTests {
  skip = skipTestOptimizer
  // REQUIRED: Calculate the runtime classpath used to execute tests
  classpath = sourceSets.test.runtimeClasspath;
}

// Configure the updateTestFilter task to set the fields of any Test tasks
task updateTestFilter() {
  doLast {
    if (!skipTestOptimizer && cover.allSelectedTests) {
      tasks.withType(Test) {
        // Set selected filters in the test task
        filter.setIncludePatterns cover.allSelectedTests
        // Specify that test tasks will not fail if the optimizer finds no tests to execute
        filter.setFailOnNoMatchingTests false
      }
    }
  }
}

// Specify dependencies to ensure the correct order of task execution
// optimizeTests -> updateTestFilter -> test
updateTestFilter.dependsOn optimizeTests
test.dependsOn updateTestFilter

KOTLIN

// Define a Gradle property to control the execution of the optimizeTests task
// This will be useful for running the tasks without optimization (e.g. -PskipTestOptimizer=true)
val skipTestOptimizer = project.properties["skipTestOptimizer"]?.toString()?.toBoolean() ?: false

tasks {
  // Configure the optimizeTests task with required fields
  optimizeTests {
    skip = skipTestOptimizer
    // REQUIRED: Calculate the runtime classpath used to execute tests
    classpath = sourceSets.test.get().runtimeClasspath
  }

  // Configure the updateTestFilter task to set the fields of any Test tasks
  register("updateTestFilter") {
    doLast {
      if (!skipTestOptimizer && cover.allSelectedTests != null) {
        withType<Test> {
          // Set selected filters in the test task
          filter.includePatterns.clear()
          filter.includePatterns += cover.allSelectedTests

          // Specify that test tasks will not fail if the optimizer finds no tests to execute
          filter.isFailOnNoMatchingTests = false
        }
      }
    }
  }

  // Specify dependencies to ensure the correct order of task execution
  // optimizeTests -> updateTestFilter -> test
  named("updateTestFilter").get().dependsOn("optimizeTests")
  test.get().dependsOn("updateTestFilter")
}

Environment variables

For the example build script above, we will use environment variables to pass the required absolute paths to the optimizeTests task:

• DIFFBLUE_COMMAND The absolute path to the installed dcover script.

• DIFFBLUE_PATCH The absolute path the patch file.

For example, in bash run export DIFFBLUE_PATCH=/path/to/a/changes.patch or in powershell run $env:DIFFBLUE_PATCH=/path/to/a/changes.patch.

As an alternative to using environment variables, you could choose to set these values directly in the optimizeTests task, or by using Grade properties to be specified from the command line (-Pcom.diffblue.cover.command=/path/to/dcover -Pcom.diffblue.cover.patch=/path/to/a/changes.patch):

GROOVY

optimizeTests {
  ...
  // The absolute path to your installed 'dcover' executable
  command = project.findProperty('com.diffblue.cover.command')
  // The absolute path to the patch file for which optimal tests will be calculated
  patch = project.findProperty('com.diffblue.cover.patch')
}

KOTLIN

tasks {
  optimizeTests {
    ...
    // The absolute path to your installed 'dcover' executable
    command = project.findProperty("com.diffblue.cover.command") as String?
    // The absolute path to the patch file for which optimal tests will be calculated
    patch = project.findProperty("com.diffblue.cover.patch") as String?
  }
}

The optimizeTest will prefer settings set directly in the task to those set via environment variables.

You can check the value of all the parameters used by the plugin by displaying the Gradle info output, e.g. using --info.

Multi-project builds

A Gradle multi-project build allows a series of Gradle subprojects to share common configuration.

If your multi-project build uses the subprojects {} or allprojects {} DSL constructs, then you may need configure the optimizeTests task within these blocks.

For more documentation on this process please review the Gradle documentation: Cross project configuration

Additional Test types

If you have introduced additional Test types into your project, for example to support Integration Tests, then you should add these tasks as dependencies of updateTestFilter.

For example, in the snippet below the user has added a custom Test type to test only the utils package using a test exclusion. They have registered an additional dependency on updateTestFilter.

GROOVY

tasks.register("utilsTest", Test) {
    useJUnitPlatform()
    include "**/utils/**"
}
// Ensure that `updateTestFilter` is always run before `utilsTest`
utilsTest.dependsOn updateTestFilter

KOTLIN

tasks {
  register("utilsTest", Test::class) {
    useJUnitPlatform()
    dependsOn("updateTestFilter")
    include("**/utils/**")
  }
}

For more on how to specify Test types, please review the Gradle documentation: Using additional test types Sample.

Notice in the example build script we use test filtering to replace all inclusion filters using filter.setIncludePatterns with a series of class names selected by the optimizer for execution. If your Test type uses test filter inclusions these may be overwritten. You may prefer to use test filter exclusions (filter.excludeTestsMatching) which take precedence over inclusions. Or use Test inclusion/exclusion by class file which is applied separately to test filters.

For more on test filtering, please review the Gradle documentation: Test filtering.

Installing Diffblue Cover in your CI environment

Before you begin the installation, please obtain the link to download Cover (from your product update email, or contact Diffblue). Please note you need version 2022.03.02 or above, installed and activated with an appropriate license.

For full details, please see: Installing Diffblue Cover in your CI environment.

Running for the first time: Diffblue Cover Optimize

Create a patch file

Create a patch file called for example changes.patch containing the changes you wish to run Diffblue Cover Optimize against. For examples on how to create a patch file from your changes using git, see Patch files.

Execute tests with Diffblue Cover Optimize

For the example build script above we are relying upon environment variables to pass the absolute paths of the command and patch files to the plugin. An execution on bash might looks like this:

export DIFFBLUE_COMMAND=/absolute/path/to/dcover
export DIFFBLUE_PATCH=/absolute/path/to/changes.patch
./gradlew test --info

Use the command gradle if not using gradlew. The --info option can useful for debugging Gradle scripts and viewing output from Cover Optimize.

Only the tests that are impacted by the changes in the .patch file will be run.

NOTE: Using a relative path such as ~/path/to/a/changes.patch will not work, the path must be absolute.

Executing all tests without optimization

In the example build script we have added the Gradle property skipTestOptimizer to allow test optimization to be skipped, if the user so requires. In which case Gradle test will execute without test optimization and without applying test filters:

./gradlew test --info -PskipTestOptimizer=true

All tests will be run without optimization.

For more on how to specify and use Gradle properties, please review the Gradle documentation: Gradle properties

Using the --tests option from the command line

The Gradle --tests option allows the user to specify additional test filters from the command line; these are applied in addition to and test inclusion/exclusion and test filtering specified within the build script:

When you use –tests, be aware that the inclusions declared in the build script are still honored. Test filtering.

As the test optimizer dynamically calculates test filters from your patch file, your --test option may try to include or exclude tests which have not been selected by the optimizer, therefore no tests will be run.

If you wish to specify additional test filters via the --tests option then it is recommended to skip test optimization when doing so:

./gradlew test --info -PskipTestOptimizer=true --tests "*.utils.*"

For more on test filtering, please review the Gradle documentation: Test filtering.

Defining custom rules

There might be cases where Diffblue Cover Optimize does not select tests on certain changes as expected. It is easily possible to add custom rules to the configuration of the optimizeTests task to instruct Diffblue Cover Optimize in these situations. For example:

GROOVY

optimizeTests {
  ...
  rules = [
    [filesChanged: "build.gradle", testsToRun: "**/*Test.java,**/*IT.java"],
    [filesChanged: "**/resources/**,**/projects/**", testsToRun: "***/*IT.java"]
  ]
}

KOTLIN

tasks {
  optimizeTests {
    ...
    rules += mapOf("filesChanged" to "build.gradle.kts", "testsToRun" to "**/*Test.java,**/*IT.java")
    rules += mapOf("filesChanged" to "**/resources/**,**/projects/**", "testsToRun" to "***/*IT.java")
  }
}

The rules above mean that:

• If there was a change to a pom.xml file then all tests with the suffixes Test and IT will be run.

• If there was a change to any file inside a resources or projects directory then all tests with suffixes IT will be run. The matcher syntax uses glob patterns as used in Gradle to specify include patterns, for instance.

Custom rules are available from release 2022.05.02.