Working with code V003
The output code V003 indicates that tests were discarded during the validation stage where the created tests are run in a realistic test environment. The possible causes of V003 differ depending upon the state of your test suite before running Diffblue Cover CLI Cover.
Test suite fails before created tests are added
Before creating tests using Cover, ensure that the existing test suite can be run and all tests pass via the Maven or Gradle test commands. If the test suite cannot be successfully run at this stage then it is probable that many or all tests generated by Cover will be discarded during validation. Resolve failing tests before running Cover on the project.
To ignore failures during validation and retain all tests that pass verification, run Cover with the
Test suite passes before created tests are added
To resolve the conflict causing this issue, it may be necessary to run Cover with the
--skip-test-validation option - this will ignore test failures in the real testing environment and output all tests which passed verification. Once all the generated tests have been output, run tests from your build tool (e.g.
mvn test or
./gradlew test) to discover which test classes are problematic.
Annotation or dependency conflicts
Created tests may be inserted into existing test files. Those existing files may contain dependencies or annotations which conflict with those required for the tests that Cover attempts to insert into the test file, which can result in failing tests.
Dependence upon shared mutable state
For tests that rely upon shared mutable state, it is possible for that state to get set in a different way in the true test environment compared to when the test is run by Cover’s internal checks. Please ensure that any shared mutable state upon which the classes used in the failing tests depend is correctly set in the arrange section of those tests.
Alternately, try running Cover, to generate tests for only the problematic classes with prefix filtering, e.g.
dcover create com.example.problematicpackage com.example.ProblematicClass. Sometimes this will allow the state to be set correctly in the generated tests.
Style checking/linting failures
Cover does not guarantee that tests it produces obey the required styles for the project. Run Cover with
--ignore-stylechecks, disable style checking/linting for Cover generated test files or manually edit the produced tests to align with your style rules.