Integrating Diffblue Cover into CI on pull requests

Overview

This topic will help you to create a Jenkins declarative pipeline that is triggered by a webhook to automatically write Diffblue tests for a pull request.

Prerequisites

This topic assumes that you have:

• Set up the Jenkins freestyle project to trigger Diffblue Cover.

• A Jenkinsfile in your project (see Quick Start - Jenkins or perhaps you have an existing Jenkinsfile).

and will wrap the Jenkins freestyle project that pushes tests to a branch in a Jenkinsfile for a Jenkins multi-branch pipeline that is triggered by a webhook for pull requests.

Setup a webhook

The Jenkins pipeline will be triggered by a webhook, so it is necessary to set up a webhook on the remote host if you do not already have one.

Setting up a webhook is well-documented on Github https://docs.github.com/en/developers/webhooks-and-events/webhooks/creating-webhooks

• The payload URL should be JENKINS_URL/multibranch-webhook-trigger/invoke?token=github-webhook, where JENKINS_URL might look like http://jenkins.your.org.com:8080/

• The type is json/application

• Select Let me select individual events and check Pull requests

The Jenkins pipeline will use the Multibranch Scan Webhook Trigger plugin in Jenkins which is triggered by the payload URL above. Since only pull requests trigger the webhook, the job will run when a pull request is made or updated.

In the Jenkins configuration, under Scan Repository Triggers select Scan by webhook with trigger token github-webhook, to tell Jenkins what to listen for.

Add stages to the Jenkinsfile

If you have done the quick start guide, you can replace the Use dcover create in Jenkins with a step that calls the Jenkins freestyle project difflue-cover-tool, or add a stage to your existing Jenkins file, as follows:

pipeline {
    agent any

    stages {
        stage('Use dcover create in Jenkins') {
            steps {
                script {
                   build job: 'diffblue-cover-tool', parameters: [string(name: 'HEAD_BRANCH', value: "$CHANGE_BRANCH"), string(name: 'BASE_BRANCH', value: "$CHANGE_TARGET"), string(name: 'MODULES', value: "module1 module2 module3")]
                }
            }
        }
    }

}

Replace the MODULES parameter value with the modules in your project. Though for testing purposes, you may wish to select a single module for now.

This will trigger the diffblue-cover-tool job on a pull request when the pull request is made or the branch is updated. However, since this job will push the tests to the pull request branch, you first need to skip this stage if the commit author is the user that pushes the Diffblue tests, which is done in the next section.

Skip Diffblue tests when bot user commits

Add the stage to get the author of the last commit before the Use dcover create in Jenkins stage.

stage('Get last commit author and bot name') {
    steps {
        script {
            env.GIT_AUTHOR = sh (script: 'git log --no-merges -1 --pretty=%cn', returnStdout: true).trim()
            env.DIFFBLUE_BOT_NAME = "db-ci-bot"
        }
        sh '''#!/bin/bash
            echo "Git author is $GIT_AUTHOR and the Diffblue bot name is $DIFFBLUE_BOT_NAME"
        '''
    }
}

then add a when statement to the Update diffblue tests stage

stages {
    stage('Use dcover create in Jenkins') {
        when {
            not { environment name: 'GIT_AUTHOR', value: "$env.DIFFBLUE_BOT_NAME" }
        }
        steps {
            script {
                build job: 'diffblue-cover-tool', parameters: [string(name: 'HEAD_BRANCH', value: "$CHANGE_BRANCH"), string(name: 'BASE_BRANCH', value: "$CHANGE_TARGET"), string(name: 'MODULES', value: "module1 module2 module3")]
            }
        }
    }
}

With this, the Use dcover create in Jenkins stage will only run on code changes by a user other than the Diffblue tests user, avoiding an unnecessary run. At this point, you may wish to add this to any other stages in your Jenkinsfile that require it.

Create a test branch and PR

In the repository you wish to write tests for, create a branch called diffblue-test-pr based on your main branch.

Make a simple change in a module that the diffblue-cover-tool job is set to run on and commit the change. Push this branch to the remote host and make a pull request.

You should see the job running and then a commit pushed to the PR branch. In this simple setup, the commit that pushes Diffblue tests will re-trigger CI, but the author check should prevent Diffblue Cover from re-running. When the PR is merged, the Diffblue tests will be merged with it.

What's next

With this, you can have Diffblue Cover writing tests automatically for your repository.

It is basic and there are efficiencies to make and errors to catch, but how this is done depends heavily on your specific pipeline and environment.

You can see how we at Diffblue have address some of these in our full example which includes:

1. Storing and running Diffblue tests separately to hand-written unit tests.

2. Dealing with non-compiling or failing tests using dcover validate before dcover create.

3. Running existing Diffblue tests against a pull request to catch possible regressions.

4. Parallelizing to speed up runs on PRs and dealing with concurrency in the remote host.

5. Stopping multiple builds on the same PR.

6. Adding a comment to the PR so the end user knows what is happening.