I have more of a theoretical question. We have a project with complicated module hierarchy. The goal is to measure unit test coverage and in the future also integration test coverage.
In order to properly place the testing dependancies, plugins and configurations in the different project poms, I need to understand the interplay between the following plugins:
failsafe
surefire
jacoco
What I know is, failsafe generates integration test reports, surefire generates the unit tests reports and jacoco can use both to generate the test coverage report.
What I need to know is:
How jacoco interacts with both failsafe and surefire?
Do jacoco need failsafe and surefire?
Can you explain the jacoco configuration and particularly why does the problem with the argLine occur?
I found a lot of examples scattered, and got the jacoco plugin working, but my goal is to place all configs etc. where they are actually needed, so that they do not pollute the maven dependency-hierarchy.
Related
I have a build configuration that is running code coverage on unit tests using TeamCity's built-in dotCover tool. Then another runner runs code coverage on some API tests using a non-supported tool. I am currently reporting the coverage stats to TeamCity using the service messages:
Write-Host "##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='$totalLines']"
Write-Host "##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='$totalLinesCovered']"
Everything works great. However, the first runner reports other statistics (Methods, Classes, Branches) on the unit tests and coverage it ran that I want cleared (I don't want any stats from the unit tests). How do I clear a build statistic that is already reported?
I tried using a similar message as above and setting value='' or value='0'. The value='' threw an java error that it couldn't parse, while value='0' had no effect on removing the statistic.
How do I clear a build statistic? Or just make dotCover not report any code-coverage stats?
I'm just starting to mess about with continous integration. Therefore I wanted to set up Jenkins as also Sonarqube. While reading manuals/docs and tutorials I got a little bit confused.
For both systems, there are descriptions about how to set up unit test runners. So where should unit tests ideally be run? In Jenkins or in Sonarqube or in both systems? Where does it belong in theory/best practice?
We have configured Jenkins to launch the unit tests and the results are “forwarded” to Sonar to be interpreted as a post build action
The Best practice would be running the Unit test in Jenkins. This would ensure the Unit test cases are executed before we Build/Deploy.
SonarQube is normally used to ensure the quality of the code which will point out the bad codes, based on the guidelines/rules.It also gives the report on the Unit test coverage, Lines of code etc.
Usually it's done in Jenkins as you want to actually test your code before building the module.
Currently i am using these tools to run my tests,code coverage and documentation:
Unit testing:
jasmine
xUnit
Code Coverage:
Istanbul
dotCover
Documentation:
Typedoc
As i'm trying to do everything modular for both frontend and backend we have multiple bower components and nuget packages where of course each components runs different type of tests and documentation.
Now what i want to do is to have a dedicated site which grabs all test results and documentation and have a dedicated site where all developers etc. can use it as a point of reference.
Is there any plugin available that can help me achieve it?
if not do you have any idea from where can i start as i tried googling a bit but with no luck.
I'm using roughly the same technologies.
As a build server I use TeamCity.
In a nutshell: your build is composed by steps, e.g (simplified):
build .sln
gulp build
xUnit tests (*A: publishing coverage)
karma run
remap coverage from Javascript to Typescript (*B: publish coverage)
The only problem I had so far is with the coverage (*A + *B). The last data will overwrite the first one, (not average it all). So in that case I use custom reports page to display the istanbul generated html report and only use the xUnit coverage report.
You could have the coverage.json from istanbul as an artifact of your build, and a second build picks up and reports that coverage through teamcity. It would be simply a coverage report build (only 1 step, report code coverage). The trigger is a successful build generating the coverage.
For your generated documentation you can also use custom reports page.
About the unit tests execution (both jasmine (karma?) and xunit), both report its numbers and the final Test report will show them combined.
I am currently using jenkins to build a list of different modules for my project. I trigger the builds using Maven. I have sonarqube installed on the server and have set it up correctly so that when a module builds it is displayed on sonarqube and includes all of the basic details such as lines of code, technical debt etc. The modules all have Junit tests that run against them, and sonarqube displays this by saying that the Unit Test Sucess is 100% and it also says the number of tests that have been run in that module. However I cannot get the Unit tests coverage field to display anything and it is blank for all of the modules.
Here is an exert (one module) from my pom.xml
customer.sonar.projectBaseDir=.
customer.sonar.sources=D:/TFS/WorkSpace/DEV_2_HYBRID/APP_FO/application/customer/src/main/java
customer.sonar.Hybrid=Customer
customer.sonar.tests=D:/TFS/WorkSpace/DEV_2_HYBRID/APP_FO/application/customer/target/surefire-reports
customer.sonar.junit.reportsPath=D:/TFS/WorkSpace/DEV_2_HYBRID/APP_FO/application/customer/target/surefire-reports
The versions of the software I am using are as follows:
Sonarqube v.5.0,
Jenkins Sonarqube plugin v.2.1,
Maven v3.2.5
As I said at the beginning the unit test success rate does show successfully, so I believe it is only a small change needed that will get the unit test coverage field working.
Any help would be really appreciated!
You need to execute the coverage engine of your choice and provide the report to SonarQube via the appropriate property.
If you are using JaCoCo, the report importer is embeded in the java plugin, for other coverage engine (clover, cobertura...) you have to install the dedicated plugin.
For more information see the dedicated page of documentation.
I have written a REST server (in Java using RestEasy) with a unit test suite written in Scala. The test suite uses the mock server provided by RestEasy and runs with every Maven build.
I would like to create a second functional test suite that calls an actual tomcat server and exercises each REST service. I do not want this new suite to run with every build, but only on demand, perhaps controlled with a command line argument to Maven.
Is it possible to create multiple independent test suites in a Maven project and disable some from automatic running, or do I need to create a separate Maven project for this functional suite? How can I segregate the different functional suite code if these tests are in the same project with the unit tests (different directories)? How do I run a selected suite with command line arguments?
I never used it myself but I am aware of maven integration tests run by the Maven Failsafe plugin.
As the surefire plugin by default includes the tests named **/Test*.java, **/*Test.java, **/*TestCase.java the failsafe plugin runs the **/IT*.java, **/*IT.java, **/*ITCase.java tests.
Both test approaches have different intentions which seems to match part of your needs. It might be worth to have a look.....
Another approach would be to use maven profiles and specifiy different surefire includes for each profile.