What is Diffblue Cover?

Diffblue Cover is a reinforcement learning AI platform that automatically writes comprehensive, human-like Java unit tests for Java and Kotlin projects - saving developer time, increasing test coverage, and reducing regression risks. Cover is provided as an IntelliJ IDE plugin (Cover Plugin), a CLI application (Cover CLI), and a CI integration to provide fully autonomous operation (Cover Pipeline). Three additional components (Cover Reports, Cover Optimize, and Cover Refactor), for test management and analytics, complete the Diffblue Cover platform.

But what does that mean in practical terms? As an example, let's take a method for uploading a file to an Amazon S3 bucket. The method is relatively simple, just a few lines of code, but the required test is more complex and would take a developer approximately 30 minutes to write if they've never done it before. Diffblue Cover did this in 1.6 seconds - that's not a typo - 1.6 seconds, computationally perfect, human readable, every pathway tested.

Method

    public PutObjectResult uploadFileToBucket(String bucketName, String key, File file) {
        try {
            return s3client.putObject(bucketName, key, file);
        } catch (AmazonS3Exception e) {
            throw new IllegalStateException("Totally handled this exception", e);
        }
    }

Unit Test

@ContextConfiguration(classes = {AmazonService.class})
@ExtendWith(SpringExtension.class)
class AmazonServiceDiffblueTest {
    @MockBean
    private AmazonS3 amazonS3;

    @Autowired
    private AmazonService amazonService;

    /**
     * Method under test: {@link AmazonService#uploadFileToBucket(String, String, File)}
     */
    @Test
    void diffbluetestUploadFileToBucket() throws SdkClientException {
        // Arrange
        PutObjectResult putObjectResult = new PutObjectResult();
        putObjectResult.setContentMd5("MjdjN2NmNDAwMjI5MTAzZTAwYzZkODgzMDAyOWUyOWI=");
        when(amazonS3.putObject(Mockito.<String>any(), Mockito.<String>any(), Mockito.<File>any()))
                .thenReturn(putObjectResult);

        // Act and Assert
        assertSame(putObjectResult, amazonService.uploadFileToBucket("bucket-name", "object-key",
                Paths.get(System.getProperty("java.io.tmpdir"), "test.txt").toFile()));
        verify(amazonS3).putObject(Mockito.<String>any(), Mockito.<String>any(), Mockito.<File>any());
    }
}

Taking another real-world example from an existing customer we can illustrate how this is scaled up for an organization. Diffblue Cover was able to write 3,000 unit tests across an expansive code base for a single mission-critical application, right out of the box. A conservative estimate is that this would have taken a senior developer an estimated 268 developer days to write. Diffblue Cover achieved this in 8 hours, doubling test coverage, even before any refactoring.

Some basics

If you missed our short intro video on the website, take a look here - this captures the essentials of Diffblue Cover:

Key features

Diffblue Cover is the generative AI engine at the center of our platform. Cover automatically writes human-like Java unit tests that are easy for a developer to read and understand. AI-written unit tests are ready to use - they compile, run, and accurately validate the current behavior of your code.

Cover is fully autonomous. It can create an entire unit test suite for your whole application in a single execution, run locally within your environment, no cloud service required, no developer overhead. Cover also maintains this unit test suite as your software evolves, even on applications with millions of lines of code. Thanks to a deep understanding of how your code works, Diffblue Cover knows what tests need to be added and updated for every code change. It implements the necessary updates automatically, ensuring coverage doesn’t dip while your teams move faster.

IDE, CLI, CI Pipeline

Diffblue Cover is one AI, provided in three interaction models - an IDE plugin (Cover Plugin) and a CLI tool (Cover CLI) for developer use, and a CI Pipeline integration (Cover Pipeline) for DevOps teams. Use Cover Plugin, CLI, and Pipeline in isolation or in partnership as needed, depending on your preferred workflow.

Cover Plugin

• Cover Plugin for IntelliJ writes tests for your methods and classes directly within the IDE on the developer desktop.

• One click test creation, fully integrated UI.

Cover CLI

• Advanced users may prefer the Cover CLI tool.

• More powerful, scriptable, and can generate tests for entire projects with a single command.

• Works 100% autonomously, configuring itself from your Maven or Gradle environment.

• Provides access to a wider feature set such as running preflight environment checks, creating coverage report bundles for use with Cover Reports, restricting test creation to code changes only, and more.

Cover Pipeline

• Integrate the power of Cover CLI into your CI pipeline.

• Cover Pipeline creates and keeps your entire team's unit tests up-to-date within your GitHub, GitLab, Maven, Jenkins, Azure, AWS, or other CI pipelines.

• Each time a pull request is opened Cover can automatically create, execute, and update your unit test library at appropriate points in the process.

• The Cover Pipeline CI integration is the most scalable, reliable, repeatable option - unit testing becomes an effortless part of your workflow.

Report, Optimize, Refactor

Three additional features are provided to allow you to monitor your test coverage (Cover Reports), optimize test runs (Cover Optimize), and automatically refactor your code to improve testability (Cover Refactor).

Cover Reports

• Visualization tool for test coverage statistics, coverage risk, testability, and more.

• Get valuable insights about your Java codebase - pinpoint unique, actionable insights that improve quality and efficiency.

• Have a more holistic view of unit testing. Coverage data is combined with information on variables like code quality, testability. and complexity.

• Understand risk. Reports show you exactly what code isn’t tested, how important it is, and why tests might not have been written.

• Benchmark and track the effectiveness of your Java unit testing over time.

• Track how Diffblue Cover is being used across your organization.

Cover Optimize

• Selective unit test execution, integrated into Cover CLI and Cover Pipeline.

• Automatically selects only the unit tests required to fully validate that a code change hasn’t introduced regressions.

• Reduce developer waiting time, reduced cloud computing costs, optimized PR workflows.

Cover Refactor

• Automated code refactoring tool to improve testability and test coverage.

• Integrated into Cover CLI and Cover Pipeline.

• Cover Refactor can suggest and apply code refactorings that improve the observability of Java code and make it more testable.

• Better code, increased coverage, reduced regression risk.

Key benefits - in summary

It's difficult to summarize all the benefits of Diffblue Cover, but here's our top ten:

• Automated Java unit tests. This one's a given. Automatic, unbiased unit tests that exercise the actual behavior of your code, including obscure corner cases and scenarios developers may not know how to test.

• Eliminate developer overhead. Avoid researching tests, writing tests, maintaining tests, learning unfamiliar code source, context switching, waiting time, and more. Target your developer skills at improving your applications instead.

• Catch regressions early and at the most granular level. Avoid rework time, optimize workflows, and avoid outages in the field.

• Increase coverage. Get to 80% coverage in half the time! Diffblue Cover creates unit tests in bulk to help your team quickly assess the quality of your test suite.

• Untangle legacy code. Our tests document your code, making scary code changes easy. This helps you to quickly write tests for large legacy codebases, and identifies untestable code that should be refactored.

• Reduce development costs and increase productivity. With reduced or eliminated developer time, optimized test runs, CI/CD integration, and more, Diffblue Cover works 24/7.

• Document your code. Diffblue Cover unit tests describe every behavior of every method - effectively documenting your code to make future code changes quicker and reduce regressions.

• Monitor coverage and understand risk. Cover shows you exactly what code isn’t tested, how important it is, and why tests might not have been written. Target developer expertise more efficiently, either at areas of most risk or those where additional coverage can most easily be delivered.

• Accelerate modernization and cloud migration. Diffblue Cover can support the migration from applications to microservices by providing a documented code base and improved test coverage.

• Keep code private. Diffblue Cover’s generative AI operates entirely within your environment. No data ever leaves your organization.

How it works

• Diffblue Cover first ensures that your code is compiled and examines each method that Cover will create tests for, and any related methods.

• Cover then creates initial test candidates for each method, which are then evaluated and adjusted to maximize test coverage. This process is repeated until the set of tests that optimize coverage are selected and committed to your code base. The tests are computationally correct and human readable, every time.

• Diffblue Cover is intended to be used for unit regression testing. That means it creates tests that reflect the current behavior of your code. Later, when you make changes to the code, these tests may fail and therefore highlight regressions. Since tests are written for your entire application at the most granular level, this means the smallest of regressions are caught which can avoid the dangers of incubated regressions that can manifest and build over time.