I have a build configuration that is running code coverage on unit tests using TeamCity's built-in dotCover tool. Then another runner runs code coverage on some API tests using a non-supported tool. I am currently reporting the coverage stats to TeamCity using the service messages:
Write-Host "##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='$totalLines']"
Write-Host "##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='$totalLinesCovered']"
Everything works great. However, the first runner reports other statistics (Methods, Classes, Branches) on the unit tests and coverage it ran that I want cleared (I don't want any stats from the unit tests). How do I clear a build statistic that is already reported?
I tried using a similar message as above and setting value='' or value='0'. The value='' threw an java error that it couldn't parse, while value='0' had no effect on removing the statistic.
How do I clear a build statistic? Or just make dotCover not report any code-coverage stats?
Related
I have an ASP.net Web API project (.NET Framework v4.6.2) where I have written unit test cases using XUnit.
I have created a build pipeline in Azure DevOps with 'Run only impacted tests' set to checked. Also, my Visual Studio test version is set to *2.** in my test assemblies.
I have tried to run my mock tests (No data driven test cases) to check that only impacted test run but all the tests run.
I had hard luck finding the solution for It. Can anyone know how to do it right?
I have read this article and and done setting accordingly, but failed.
Currently i am using these tools to run my tests,code coverage and documentation:
Unit testing:
jasmine
xUnit
Code Coverage:
Istanbul
dotCover
Documentation:
Typedoc
As i'm trying to do everything modular for both frontend and backend we have multiple bower components and nuget packages where of course each components runs different type of tests and documentation.
Now what i want to do is to have a dedicated site which grabs all test results and documentation and have a dedicated site where all developers etc. can use it as a point of reference.
Is there any plugin available that can help me achieve it?
if not do you have any idea from where can i start as i tried googling a bit but with no luck.
I'm using roughly the same technologies.
As a build server I use TeamCity.
In a nutshell: your build is composed by steps, e.g (simplified):
build .sln
gulp build
xUnit tests (*A: publishing coverage)
karma run
remap coverage from Javascript to Typescript (*B: publish coverage)
The only problem I had so far is with the coverage (*A + *B). The last data will overwrite the first one, (not average it all). So in that case I use custom reports page to display the istanbul generated html report and only use the xUnit coverage report.
You could have the coverage.json from istanbul as an artifact of your build, and a second build picks up and reports that coverage through teamcity. It would be simply a coverage report build (only 1 step, report code coverage). The trigger is a successful build generating the coverage.
For your generated documentation you can also use custom reports page.
About the unit tests execution (both jasmine (karma?) and xunit), both report its numbers and the final Test report will show them combined.
I have more of a theoretical question. We have a project with complicated module hierarchy. The goal is to measure unit test coverage and in the future also integration test coverage.
In order to properly place the testing dependancies, plugins and configurations in the different project poms, I need to understand the interplay between the following plugins:
failsafe
surefire
jacoco
What I know is, failsafe generates integration test reports, surefire generates the unit tests reports and jacoco can use both to generate the test coverage report.
What I need to know is:
How jacoco interacts with both failsafe and surefire?
Do jacoco need failsafe and surefire?
Can you explain the jacoco configuration and particularly why does the problem with the argLine occur?
I found a lot of examples scattered, and got the jacoco plugin working, but my goal is to place all configs etc. where they are actually needed, so that they do not pollute the maven dependency-hierarchy.
I am currently using jenkins to build a list of different modules for my project. I trigger the builds using Maven. I have sonarqube installed on the server and have set it up correctly so that when a module builds it is displayed on sonarqube and includes all of the basic details such as lines of code, technical debt etc. The modules all have Junit tests that run against them, and sonarqube displays this by saying that the Unit Test Sucess is 100% and it also says the number of tests that have been run in that module. However I cannot get the Unit tests coverage field to display anything and it is blank for all of the modules.
Here is an exert (one module) from my pom.xml
customer.sonar.projectBaseDir=.
customer.sonar.sources=D:/TFS/WorkSpace/DEV_2_HYBRID/APP_FO/application/customer/src/main/java
customer.sonar.Hybrid=Customer
customer.sonar.tests=D:/TFS/WorkSpace/DEV_2_HYBRID/APP_FO/application/customer/target/surefire-reports
customer.sonar.junit.reportsPath=D:/TFS/WorkSpace/DEV_2_HYBRID/APP_FO/application/customer/target/surefire-reports
The versions of the software I am using are as follows:
Sonarqube v.5.0,
Jenkins Sonarqube plugin v.2.1,
Maven v3.2.5
As I said at the beginning the unit test success rate does show successfully, so I believe it is only a small change needed that will get the unit test coverage field working.
Any help would be really appreciated!
You need to execute the coverage engine of your choice and provide the report to SonarQube via the appropriate property.
If you are using JaCoCo, the report importer is embeded in the java plugin, for other coverage engine (clover, cobertura...) you have to install the dedicated plugin.
For more information see the dedicated page of documentation.
I’ve installed the SonarQube plugin in intellij and associated my project to our sonar server. The server tells me the branch coverage each class has and updates when I’ve submitted unit tests. However when I run a local analysis (right click on project –> analyze –> run inspection by name –> SonarQube), SonarQube tells me X more branches need to be covered by unit tests to reach the minimum threshold of 65.0% branch coverage for all of my classes and it doesn’t change locally even when I add more unit tests (but it does change on the server)
Any idea why this might be happening?
This is a known current limitation (that actually applies to both IntelliJ and Eclipse plugins): local analyses can't automatically execute unit tests, so they can't get the coverage results and give you the correct information.
The reason for this is that local analyses are just "standard" analyses that don't push data to the server. And by definition, SonarQube analyses don't execute any external tool, they just reuse previously generated reports if they need to.
If it's Maven-based project, you have a solution: just run mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent install -Dmaven.test.failure.ignore=true prior to launching a local analysis. This should generate the coverage report at the default location and should therefore work.