Cover Optimize & Maven

Cover Optimize speeds up the time required to run Java unit tests by running only the tests in your project that are impacted by your code change. Our Cover Optimize for Maven plugin accepts a patch file, analyses the code change and determines which Java unit tests in your project need to be run in order to exercise the changes in the patch.

To use the Cover Optimize for Maven plugin, Cover CLI (release >= 2022.03.02) must be installed and activated with an appropriate license. Further installation details can be found here.

Overview

Cover Optimize can be integrated into your CI with the help of the Cover Optimize for Maven plugin. This plugin acts as a wrapper around the Cover CLI, invoking Cover Optimize during mvn test or mvn verify, and feeding its output into Maven's Surefire & Failsafe plugins in order to only run the tests exercised by the patch.

The below graphic illustrates how Diffblue Cover Optimize is invoked from cover-maven-plugin within a Maven project:

Configuring build system: Maven

Install the Cover Optimize for Maven plugin

Add the Diffblue Public Maven Repository to your project's root POM file in order to download the plugin:

<pluginRepositories>
    <pluginRepository>
       <id>maven-diffblue-repository</id>
       <name>Diffblue Public Maven Repository</name>
       <url>https://maven.diffblue.com/release</url>
       <snapshots>
            <enabled>false</enabled>
       </snapshots>
    </pluginRepository>
</pluginRepositories>

If you already have a <pluginRepositories> section in the POM file, add the Diffblue Repository at the bottom of the section.

Configure the Cover Optimize for Maven plugin to run on your project

To activate the plugin, please add the skipTestOptimizer and skipTests properties in the <properties> section of your project's POM file:

<properties>
    <skipTestOptimizer>false</skipTestOptimizer>
    <skipTests>false</skipTests>
</properties>

If either of these properties are set to true, then Cover Optimize will be skipped.

By default, Diffblue Cover Optimizer filters tests using the same default patterns as Surefire and Failsafe If your test naming scheme is different, you can specify these patterns as a comma separated list of standard globs as properties. For example,

<properties>
    ...
    <com.diffblue.cover.unitTestPattern>**/UnitTest*.java, **/*UnitTest.java</com.diffblue.cover.unitTestPattern>
    <com.diffblue.cover.integrationTestPattern>**/*IntTest*.java, **/*ITest.java</com.diffblue.cover.integrationTestPattern>
</properties>

These may also be specified directly in the call to mvn test

mvn test -Dcom.diffblue.cover.unitTestPattern="**/UnitTest*.java, **/*UnitTest.java" -Dcom.diffblue.cover.integrationTestPattern="**/*IntTest*.java, **/*ITest.java"

Next, add the Cover Optimize for Maven plugin to the <build><plugins> (or <build><pluginManagement><plugins> depending on how your project is configured) section of your project's POM file:

<build>
    <plugins>
        <plugin>
            <groupId>com.diffblue.cover</groupId>
            <artifactId>cover-maven-plugin</artifactId>
            <version>[diffblue-cover-version]</version>
            <executions>
                <execution>
                <phase>process-test-classes</phase>
                <goals>
                    <goal>optimize</goal>
                </goals>
                </execution>
            </executions>
            <configuration>
                <skip>${skipTestOptimizer}</skip>
                <skipTests>${skipTests}</skipTests>
                <failOnError>false</failOnError>
            </configuration>
        </plugin>
    </plugins>
</build>

In the above example, you should replace [diffblue-cover-version] with the version number of Diffblue Cover you are using e.g. <version>2022.03.02</version> for version 2022.03.02.

The plugin needs to have both the source and test class files compiled in order to perform its analysis, hence the use of the process-test-classes phase. The plugin then provides input to the subsequent phases for executing the tests.

In order to run Diffblue Cover Optimize against any unit tests, add a <failIfNoTests> and an <include> entry into the <configuration> section of maven-surefire-plugin (the full plugin declaration is displayed here for reference, if you already have the plugin configured, simply add the above two elements into your existing configuration):

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.0.0-M5</version>
    <configuration>
        <failIfNoTests>false</failIfNoTests>
        <includes>
            <include>${com.diffblue.selectedTests}</include>
        </includes>
    </configuration>
</plugin>

In order to run Diffblue Cover Optimize against any integration tests, add a <failIfNoTests> and an <include> entry into the <configuration> section of maven-failsafe-plugin (the full plugin declaration is displayed here for reference, if you already have the plugin configured, simply add the above two elements into your existing configuration):

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>3.0.0-M5</version>
        <executions>
            <execution>
                <goals>
                    <goal>integration-test</goal>
                    <goal>verify</goal>
                </goals>
            </execution>
        </executions>
    <configuration>
        <failIfNoTests>false</failIfNoTests>
        <includes>
            <include>${com.diffblue.selectedITs}</include>
        </includes>
    </configuration>
</plugin>

The com.diffblue.selectedTests and com.diffblue.selectedITs properties are dynamically set by the Diffblue Cover Maven plugin. Due to the way Maven initializes properties, these properties must not be defined anywhere else in your POM files.

The use of <failIfNoTests>false</failIfNoTests> prevents Surefire and/or Failsafe from failing when no tests have been selected for a module.

Running for the first time: Cover Optimize

Create a patch file

Create a patch file containing the changes you wish to run Cover Optimize against. For examples on how to create a patch file from your changes using git, see Patch files.

Execute tests with Diffblue Cover Optimize

Then run mvn test (for unit tests) or mvn verify (for integration tests) appending the following two arguments:

• -Dcom.diffblue.cover.command=/path/to/dcover, where /path/to/dcover is the absolute (not relative) path to Diffblue Cover CLI. Using a relative path such as ~/path/to/dcover will not work, the path must be absolute.

• -Dcom.diffblue.cover.patch=/path/to/changes.patch, where /path/to/changes.patch is the absolute (not relative) path to the patch file generated above. Using a relative path such as ~/path/to/a/changes.patch will not work, the path must be absolute. e.g. mvn test -Dcom.diffblue.cover.command=/path/to/dcover -Dcom.diffblue.cover.patch=/path/to/changes.patch

Alternatively, instead of using Maven properties, you can also define these two values in environment variables (again, both paths must be absolute):

• e.g. in bash run export DIFFBLUE_COMMAND=/path/to/dcover or in powershell run $env:DIFFBLUE_COMMAND=/path/to/dcover

• e.g. in bash run export DIFFBLUE_PATCH=/path/to/a/changes.patch or in powershell run $env:DIFFBLUE_PATCH=/path/to/a/changes.patch and then you can simply run mvn test/mvn verify on its own.

You can check the value of all the parameters used by the plugin by running Maven in debug mode, e.g. using -X.

Finally, to run Cover Optimize, run mvn test (for unit tests) or mvn verify (for unit & integration tests) as usual. Only the tests that are impacted by the changes in the .patch file will be run.

Adding Diffblue Cover Optimize into your CI system

When Diffblue Cover Optimize is set up for your project, it can be used in place of your test command to save time running tests in a CI environment.

Installing Diffblue Cover in your CI environment

Before you begin the installation, please obtain the link to download Cover (from your product update email, or contact Diffblue). Please note you need version 2022.03.02 or above, installed and activated with an appropriate license.

For full details, please see: Installing Diffblue Cover in your CI environment.

Example

An example using Jenkins and Maven is available here.

Defining custom rules

There might be cases where Diffblue Cover Optimize does not select tests on certain changes as expected. It is easily possible to add custom rules to the configuration of the cover-maven-plugin to instruct Cover Optimize in these situations. For example:

<configuration>
  ...
  <rules>
    <rule>
      <filesChanged>**/pom.xml</filesChanged>
      <testsToRun>**/*Test.java,**/*IT.java</testsToRun>
    </rule>
    <rule>
      <filesChanged>**/resources/**,**/projects/**</filesChanged>
      <testsToRun>**/*IT.java</testsToRun>
    </rule>
  <rules>
</configuration>

The rules above mean that:

• If there was a change to a pom.xml file then all tests with the suffixes Test and IT will be run.

• If there was a change to any file inside a resources or projects directory then all tests with suffixes IT will be run. The matcher syntax uses glob patterns as used by the Surefire plugin to specify includes, for instance.

Custom rules are available from release 2022.05.02.