Resolving code V003
- Test suite fails before generated tests are added
- Test suite passes before generated tests are added
The message V003 indicates that tests were discarded during the validation stage where the generated tests are run in a realistic test environment. The possible causes of V003 differ depending upon the state of your test suite before running Cover CLI 3.0.
Test suite fails before generated tests are added
Before generating tests using Cover CLI 3.0, ensure that the existing test suite can be run and all tests pass via the Maven or Gradle test commands. If the test suite cannot be successfully run at this stage then it is probable that many or all tests generated by Cover CLI 3.0 will be discarded during validation. Resolve failing tests before running Cover CLI 3.0 on the project.
To ignore failures during validation and retain all tests that pass verification, run Cover CLI 3.0 with the
Test suite passes before generated tests are added
To resolve the conflict causing this issue, it may be necessary to run Cover CLI 3.0 with the
--skip-test-validation option - this will ignore test failures in the real testing environment and output all tests which passed verification. Once all the generated tests have been output, run tests from your build tool (e.g.
mvn test or
./gradlew test) to discover which test classes are problematic.
Annotation or dependency conflicts
Generated tests may be inserted into existing test files. Those existing files may contain dependencies or annotations which conflict with those required for the tests that Cover CLI 3.0 attempts to insert into the test file, which can result in failing tests.
Dependence upon shared mutable state
For tests that rely upon shared mutable state, it is possible for that state to get set in a different way in the true test environment compared to when the test is run by Cover CLI 3.0’s internal checks. Please ensure that any shared mutable state upon which the classes used in the failing tests depend is correctly set in the arrange section of those tests.
Alternatively, try running Cover CLI 3.0 to generate tests for only the problematic classes with prefix filtering, e.g.
dcover create com.example.problematicpackage com.example.ProblematicClass. Sometimes this will allow the state to be set correctly in the generated tests.
Style checking/linting failures
Cover CLI 3.0 does not guarantee that tests it produces obey the required styles for the project. Run Cover CLI 3.0 with
--ignore-stylechecks, disable style checking/linting for Cover CLI 3.0 generated test files or manually edit the produced tests to align with your style rules.