Currently i am using these tools to run my tests,code coverage and documentation:
Unit testing:
jasmine
xUnit
Code Coverage:
Istanbul
dotCover
Documentation:
Typedoc
As i'm trying to do everything modular for both frontend and backend we have multiple bower components and nuget packages where of course each components runs different type of tests and documentation.
Now what i want to do is to have a dedicated site which grabs all test results and documentation and have a dedicated site where all developers etc. can use it as a point of reference.
Is there any plugin available that can help me achieve it?
if not do you have any idea from where can i start as i tried googling a bit but with no luck.
I'm using roughly the same technologies.
As a build server I use TeamCity.
In a nutshell: your build is composed by steps, e.g (simplified):
build .sln
gulp build
xUnit tests (*A: publishing coverage)
karma run
remap coverage from Javascript to Typescript (*B: publish coverage)
The only problem I had so far is with the coverage (*A + *B). The last data will overwrite the first one, (not average it all). So in that case I use custom reports page to display the istanbul generated html report and only use the xUnit coverage report.
You could have the coverage.json from istanbul as an artifact of your build, and a second build picks up and reports that coverage through teamcity. It would be simply a coverage report build (only 1 step, report code coverage). The trigger is a successful build generating the coverage.
For your generated documentation you can also use custom reports page.
About the unit tests execution (both jasmine (karma?) and xunit), both report its numbers and the final Test report will show them combined.
Related
I had configured JaCoCo in WebSphere as JavaAgent (Refer: http://www.jacoco.org/jacoco/trunk/doc/agent.html).
Restarted the server, and ran a series of automated tests on the application (to give some load) and then stopped the server.
I can see the jacoco.exec getting generated in the Server (as configured to /tmp/ location).
Now, How do I generate the HTML report ?
Before voting down this question or marking it as duplicate, here is the reason why I'm posting this question. I went through the JaCoCo Documentation like http://www.jacoco.org/jacoco/trunk/doc/maven.html and also multiple StackOverflow questions but still I'm confused.
What I understood is that the Maven plugin allows us to run the Unit tests, Integration tests and then generate a report.
What I'm looking for is a report based on the load I had given to my application deployed in Websphere. I can see the jacoco.exec file generated, but not sure from the documentation on how to generate the HTML reports.
Thanks in advance.
You could use the jacoco:report-aggregate goal with Maven.
You could refer this http://www.eclemma.org/jacoco/trunk/doc/report-aggregate-mojo.html
P.S. : However, when i had the same issue, I used Sonar to read the exec file that was generated. It gives much more than just code coverage.
I was able to generate JaCoCo report as follows :
Configured JaCoCo as Java Agent
Restart the Server and do some transactions/give some load (in my case I ran a series of automated tests)
Stop the Server (This will actually generate the jacoco.exec file)
Create/Configure the Ant script and run it (This will read the jacoco.exec file and generate the html report). Refer : http://www.eclemma.org/jacoco/trunk/doc/ant.html
Even though my Project is a Maven project, I used ant script for the Report generation. I automated all the above steps using Bamboo and it made running and maintaining this Job easier.
I have a build configuration that is running code coverage on unit tests using TeamCity's built-in dotCover tool. Then another runner runs code coverage on some API tests using a non-supported tool. I am currently reporting the coverage stats to TeamCity using the service messages:
Write-Host "##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='$totalLines']"
Write-Host "##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='$totalLinesCovered']"
Everything works great. However, the first runner reports other statistics (Methods, Classes, Branches) on the unit tests and coverage it ran that I want cleared (I don't want any stats from the unit tests). How do I clear a build statistic that is already reported?
I tried using a similar message as above and setting value='' or value='0'. The value='' threw an java error that it couldn't parse, while value='0' had no effect on removing the statistic.
How do I clear a build statistic? Or just make dotCover not report any code-coverage stats?
I am attempting to set up Java code coverage for a fairly complex app that
combines multiple large modules, only one of which I need to check coverage on
uses a combination of ant and Maven for builds
cannot be run except as an installed application on a server, with configuration
the automated tests to be analyzed for coverage are not part of the application build and make use of API calls to the application server from a remote client
The examples given in the jacoco documentation and in the online sources I have found assume the app under test is not previously installed and the tests are unit/integration tests run as part of the build. The documentation does not cover the details of how the jacoco instrumentation is done or when the call is recorded to a particular line of code. If I use ant or maven to instrument a particular module, use that module to build the full app, install it on a server, and configure it, will my remote tests then generate the .exec file?
Any advice on how to achieve the end goal (knowing how much of our code is covered by the tests) is greatly appreciated, including better search terms than "jacoco for installed app" which as you can imagine is ... not very useful. My google-fu is humbled.
I am currently using jenkins to build a list of different modules for my project. I trigger the builds using Maven. I have sonarqube installed on the server and have set it up correctly so that when a module builds it is displayed on sonarqube and includes all of the basic details such as lines of code, technical debt etc. The modules all have Junit tests that run against them, and sonarqube displays this by saying that the Unit Test Sucess is 100% and it also says the number of tests that have been run in that module. However I cannot get the Unit tests coverage field to display anything and it is blank for all of the modules.
Here is an exert (one module) from my pom.xml
customer.sonar.projectBaseDir=.
customer.sonar.sources=D:/TFS/WorkSpace/DEV_2_HYBRID/APP_FO/application/customer/src/main/java
customer.sonar.Hybrid=Customer
customer.sonar.tests=D:/TFS/WorkSpace/DEV_2_HYBRID/APP_FO/application/customer/target/surefire-reports
customer.sonar.junit.reportsPath=D:/TFS/WorkSpace/DEV_2_HYBRID/APP_FO/application/customer/target/surefire-reports
The versions of the software I am using are as follows:
Sonarqube v.5.0,
Jenkins Sonarqube plugin v.2.1,
Maven v3.2.5
As I said at the beginning the unit test success rate does show successfully, so I believe it is only a small change needed that will get the unit test coverage field working.
Any help would be really appreciated!
You need to execute the coverage engine of your choice and provide the report to SonarQube via the appropriate property.
If you are using JaCoCo, the report importer is embeded in the java plugin, for other coverage engine (clover, cobertura...) you have to install the dedicated plugin.
For more information see the dedicated page of documentation.
I’ve installed the SonarQube plugin in intellij and associated my project to our sonar server. The server tells me the branch coverage each class has and updates when I’ve submitted unit tests. However when I run a local analysis (right click on project –> analyze –> run inspection by name –> SonarQube), SonarQube tells me X more branches need to be covered by unit tests to reach the minimum threshold of 65.0% branch coverage for all of my classes and it doesn’t change locally even when I add more unit tests (but it does change on the server)
Any idea why this might be happening?
This is a known current limitation (that actually applies to both IntelliJ and Eclipse plugins): local analyses can't automatically execute unit tests, so they can't get the coverage results and give you the correct information.
The reason for this is that local analyses are just "standard" analyses that don't push data to the server. And by definition, SonarQube analyses don't execute any external tool, they just reuse previously generated reports if they need to.
If it's Maven-based project, you have a solution: just run mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent install -Dmaven.test.failure.ignore=true prior to launching a local analysis. This should generate the coverage report at the default location and should therefore work.