Diffblue Logo
Go to diffblue.com

Get unit tests for your Java code automatically with our free IntelliJ plugin. Download now

Knowledge Base > DCover CLI > Custom Inputs

Customizing Test Inputs

Background

A key feature of Diffblue Cover is automatically identifying appropriate inputs necessary to write tests for a given method under test.

Sometimes Diffblue Cover is unable to find appropriate inputs and thus does not produce useful tests. One such example is with the open source XXL-JOB project where Cover fails to produce tests for the CronExpression class, resulting in an R013 output code:

com.xxl.job.admin.core.cron.CronExpression.<init>
  R013: No inputs found that don't throw a trivial exception
    Diffblue Cover tried to run the arrange/act section, but the method under
    test threw                                  
    java.lang.NullPointerException              
        at com.xxl.job.admin.core.cron.CronExpression.<init>(CronExpression.java:295)
    In order to prevent <init>(CronExpression)  
    from throwing NullPointerException, add constructors or factory
    methods that make it easier to construct fully initialized objects used in
    <init>(CronExpression).                     
    See https://diff.blue/R013 to resolve this issue.

Also, because the R013 occurs in the primary constructor for the class under test, Diffblue Cover is then unable to produce CronExpression instances for a further 28 methods and R008 output codes:

com.xxl.job.admin.core.cron.CronExpression.addToSet
  R008: Failed to instantiate class under test  
    Diffblue Cover was unable to construct an instance of CronExpression.
    Add a package-visible constructor or a factory method for testing which
    (ideally) takes no arguments, and does not throw, return null or return
    a subtype.                                  
    See https://diff.blue/R008

Experienced developers or system administrators will immediately recognize the concept of “cron expressions” and suppose that tests likely need input strings of a very specific format in order to successfully construct a CronExpression instance. In fact the javadoc in CronExpression.java devotes over 100 lines of comments to explaining the requirements for this format - helpfully providing an example of a valid cron expression string: "0 0 14-6 ? * FRI-MON". Developers can therefore produce a DiffblueRules.yml file in the repository with the following contents to provide the necessary hints:

java.lang.String:
  - immediate: "0 0 14-6 ? * FRI-MON"
    parameter: cronExpr

With this configuration file in place, when java.lang.String values are required, Diffblue cover will try to use the constant cron expression value (immediate: "0 0 14-6 ? * FRI-MON") if the parameter begins with cronExpr (parameter: cronExpr). Using just this 1 custom input allows this class to go from 54 tests with 21% line coverage to 194 tests with 59% line coverage.

Rules Location

Custom rules are loaded from the project working directory and any of its ancestor directories. Assuming you have your project checked out into ~/Projects/MyProject and are working with a module SomeModule within that, then rules will be read from each of the following locations in order, if they exist:

  • ~/Projects/MyProject/SomeModule/DiffblueRules.yml
  • ~/Projects/MyProject/DiffblueRules.yml
  • ~/Projects/DiffblueRules.yml
  • ~/DiffblueRules.yml
  • …
  • /DiffblueRules.yml

The intention here is that module and project level rules can be checked into the project source control system, but additional fallbacks into the user home directory are also possible. Rules are loaded from each discovered file in order, and so applicable rules local to the module will take precedence over rules defined at a wider scope.

Rules Format

Custom rules are defined in a YAML format with the structure shown here. Rules are grouped by the type of input produced, and so the top level keys are the type being produced, each with a list of rules at the next level down. For example the following would mean that the constant values "Diffblue", "Cover", "Custom", "Inputs" are all considered as candidate values to use whenever a String is required. Note that this does not guarantee that all of these constant values will be used, in practice the first candidate constant value may produce full coverage and so further candidate values are not needed.

java.lang.String:
  - immediate: "Diffblue"
  - immediate: "Cover"
  - immediate: "Custom"
  - immediate: "Inputs"

immediate: Rule

Currently the only supported mechanism for producing values is via immediate constant values. In practice this means that the only supported types at the moment are primitives and java.lang.String, and that every rule will have an immediate: <constant> component. For example the following configuration specifies a single unconditional default rule for each supported type:

boolean:
  - immediate: true

byte:
  - immediate: 65 # 0x41

char:
  - immediate: '*'

float:
  - immediate: 888.888f

double:
  - immediate: 12345.6789

int:
  - immediate: 7 # low values are recommended in case used for resource allocation

long:
  - immediate: -23

short:
  - immediate: 291 # 0x0123

java.lang.String:
  - immediate: "Lorem ipsum dolor sit amet"

Conditions

Typically a custom constant value is associated with one or more conditions for when this custom value should be used. When producing inputs for a particular method, supported conditions allow matching against the fully qualified class name, method name, or parameter name. For ease of use, matching is typically performed using case-insensitive substrings, but if finer control is required then the pattern can be wrapped in a ^ and $ and the pattern will be treated as a regular expression instead.

Consider the following example code under test and the numbered contexts where string inputs need to be produced:

package com.example.myapp;

class MyExampleClass {
  static MyExampleClass parse(String text /* Context #1 */) {
    ...
  }

  void doSomething(String firstParam /* Context #2 */, String secondParam /* Context #3 */) {
    ...
  }

  void doSomethingElse(String firstParam /* Context #4 */) {
    ...
  }
}

class SomeOtherClass {
  void doIt(String firstParam /* Context #5 */) {
    ...
  }
}

class: Condition

Class name conditions match against the fully qualified class name, com.example.myapp.MyExampleClass in context #1-4 above, or com.example.myapp.SomeOtherClass in context #5. For example class: "example.myapp" and class: "^.*Class$" could both be used to match all contexts above.

method: Condition

Method name conditions match against the name of the method, parse in context #1 above, doSomething in #2-3. For example method: "parse" would match context #1, whereas method: "^do.*$" would match contexts #2-5.

parameter: Condition

Parameter name conditions match against the name of the parameter, text in context #1 above, firstParam in contexts #2,4,5. For example parameter: "text" would match context #1 above, parameter: "^.*Param$" would match contexts #2-5.

Condition Combination

Up to one of each condition can be combined in a single rule, so for example the following rule can be used to offer the input “Hello World!” only in context #4 above:

java.lang.String:
  - immediate: "Hello World!"
    method: "doSomethingElse"
    parameter: "firstParam"