Module 6: Getting more from Cover

We will delve into advanced features of Cover that will allow you to write more tests and improve the testability of your codebase.

To start this module you must have completed Module 1 or Module 3.

For each group of methods that Cover analyzes, it either creates tests or it outputs a code that explains why it was not able to write a test. These output codes are the starting point for making Cover write more tests. -> Understanding output codes

In many cases Cover can output a partial test that shows how far Cover got with writing a test. The partial test allows you to better understand the output code in the context of your code. By debugging such a partial test, you can gather more information to identify what is required to make Cover write more tests. -> Methodology for Effective Debugging

Toolbox for tweaking Cover

The three main levers to influence the behavior of Cover are:

• Mocking

  • Cover automatically decides what to mock. However, sometimes it may not know at what level it should mock or try to instantiate classes that rather should be mocked. You can instruct Cover to try to mock certain classes.

• Custom inputs

  • Cover automatically figures out which input values to use. However, some tests may require specific inputs, i.e. specific domain knowledge to initialize objects correctly. You can give hints to Cover to use certain values.

• Custom setup

  • Some projects rely on static state or use custom application configuration mechanisms that need to be called before running any test. Cover will not be able to figure that out. You make Cover execute specfied custom setup so that it can build its tests upon it.

There are multiple ways how to actuate these levers:

Command line options:

• for mocking: --mock, --mock-static, --mock-construction

Cover Annotations:

• for mocking

• for custom inputs

Diffblue rules:

• for custom inputs

Test factories:

• for custom inputs of more complex types; usually in combination with annotations or Diffblue rules

Custom base class:

• for before/after methods and static setup

Handling R013

Cover did not find inputs that wouldn’t cause a trivial exception, e.g. NullPointerException.

Causes:

• Very specific input values are needed.

• A hard-to-initialize object is needed.

• Static (global) state needs to be initialized.

Possible resolutions:

• Provide custom inputs.

• Rather than providing a fully initialized instance, mock the methods called from the code under test.

• Initialize static state in a @Before method using a custom base class.

Similar output codes: R008, R081, R082, R083.

Handling R026

Cover automatically figures out what Spring test context to load. Sometimes this fails because Spring configurations are missing for loading a test context.

Resolution approach:

1. Use keep-partial-tests command line option --keep-partial-test

2. Run the created partial test and examine the obtained Spring exception

3. Manually fix the code until the test passes

4. Run Cover again

(3) may involve

• Adding an application-test.properties file.

• Adding a Spring @Configuration class providing @Bean definitions for testing purposes.

• Adding an additional class to the test context: use the --spring-configuration option, see spring-configuration.

If you decide that a Spring context should not be loaded, use the --no-spring-tests option, see no-spring-tests.

Similar output codes: R027

Handling R011

Cover runs snippets of your code.

• In order to protect your system, it runs them in a sandbox.

• The sandbox forbids potentially dangerous operations, e.g. network access.

Cause:

• Your code calls a method that performs a forbidden operation.

Possible resolutions:

• Mock the operation so that it doesn’t get called.

• Verify that the operation is harmless and run with --disable-sandbox. ⚠

Similar output codes: R012