Integrating Diffblue Cover into CI on pull requests
Create a Jenkins pipeline triggered by a webhook that calls cover and pushes tests to a branch
This topic will help you to create a Jenkins declarative pipeline that is triggered by a webhook to automatically write Diffblue tests for a pull request.
This topic assumes that you have:
- A Jenkinsfile in your project (see Quick Start - Jenkins or perhaps you have an existing Jenkinsfile).
and will wrap the Jenkins freestyle project that pushes tests to a branch in a Jenkinsfile for a Jenkins multi-branch pipeline that is triggered by a webhook for pull requests.
The Jenkins pipeline will be triggered by a webhook, so it is necessary to set up a webhook on the remote host if you do not already have one.
Setting up a webhook is well-documented on Github https://docs.github.com/en/developers/webhooks-and-events/webhooks/creating-webhooks
- The payload URL should be
JENKINS_URL/multibranch-webhook-trigger/invoke?token=github-webhook
, whereJENKINS_URL
might look likehttp://jenkins.your.org.com:8080/
- The type is
json/application
- Select
Let me select individual events
and checkPull requests
The Jenkins pipeline will use the Multibranch Scan Webhook Trigger plugin in Jenkins which is triggered by the payload URL above. Since only pull requests trigger the webhook, the job will run when a pull request is made or updated.
In the Jenkins configuration, under
Scan Repository Triggers
select Scan by webhook
with trigger token github-webhook
, to tell Jenkins what to listen for.If you have done the quick start guide, you can replace the
Use dcover create in Jenkins
with a step that calls the Jenkins freestyle project difflue-cover-tool
, or add a stage to your existing Jenkins file, as follows:pipeline {
agent any
stages {
stage('Use dcover create in Jenkins') {
steps {
script {
build job: 'diffblue-cover-tool', parameters: [string(name: 'HEAD_BRANCH', value: "$CHANGE_BRANCH"), string(name: 'BASE_BRANCH', value: "$CHANGE_TARGET"), string(name: 'MODULES', value: "module1 module2 module3")]
}
}
}
}
}
Replace the
MODULES
parameter value with the modules in your project. Though for testing purposes, you may wish to select a single module for now.This will trigger the
diffblue-cover-tool
job on a pull request when the pull request is made or the branch is updated. However, since this job will push the tests to the pull request branch, you first need to skip this stage if the commit author is the user that pushes the Diffblue tests, which is done in the next section.Add the stage to get the author of the last commit before the
Use dcover create in Jenkins
stage.stage('Get last commit author and bot name') {
steps {
script {
env.GIT_AUTHOR = sh (script: 'git log --no-merges -1 --pretty=%cn', returnStdout: true).trim()
env.DIFFBLUE_BOT_NAME = "db-ci-bot"
}
sh '''#!/bin/bash
echo "Git author is $GIT_AUTHOR and the Diffblue bot name is $DIFFBLUE_BOT_NAME"
'''
}
}
then add a
when
statement to the Update diffblue tests
stagestages {
stage('Use dcover create in Jenkins') {
when {
not { environment name: 'GIT_AUTHOR', value: "$env.DIFFBLUE_BOT_NAME" }
}
steps {
script {
build job: 'diffblue-cover-tool', parameters: [string(name: 'HEAD_BRANCH', value: "$CHANGE_BRANCH"), string(name: 'BASE_BRANCH', value: "$CHANGE_TARGET"), string(name: 'MODULES', value: "module1 module2 module3")]
}
}
}
}
With this, the
Use dcover create in Jenkins
stage will only run on code changes by a user other than the Diffblue tests user, avoiding an unnecessary run. At this point, you may wish to add this to any other stages in your Jenkinsfile that require it.In the repository you wish to write tests for, create a branch called
diffblue-test-pr
based on your main branch.Make a simple change in a module that the
diffblue-cover-tool
job is set to run on and commit the change. Push this branch to the remote host and make a pull request.You should see the job running and then a commit pushed to the PR branch. In this simple setup, the commit that pushes Diffblue tests will re-trigger CI, but the author check should prevent Diffblue Cover from re-running. When the PR is merged, the Diffblue tests will be merged with it.
With this, you can have Diffblue Cover writing tests automatically for your repository.
It is basic and there are efficiencies to make and errors to catch, but how this is done depends heavily on your specific pipeline and environment.
- 1.Storing and running Diffblue tests separately to hand-written unit tests.
- 2.Dealing with non-compiling or failing tests using
dcover validate
beforedcover create
. - 3.Running existing Diffblue tests against a pull request to catch possible regressions.
- 4.Parallelizing to speed up runs on PRs and dealing with concurrency in the remote host.
- 5.Stopping multiple builds on the same PR.
- 6.Adding a comment to the PR so the end user knows what is happening.