Gradle jacoco code coverage - Then publish/show in Jenkins - unit-testing

I'm trying to setup code coverage for a Java application project.
Project name : NewApp
Project structure:
src/java/** (source code)
src/java-test (unit tests - Jnuit)
test/it-test (integration test)
test/at-tests (acceptance tests)
tomcat/* (contain tomcat start/stop scripts)
xx/.. etc folders which are required for a usual application.
Gradle version : 1.6
Environment : Linux
I have a running gradle build script that fetches application (NewApp) dependencies (i.e. service jars used by the app for build process) from a build artifact repository (artifactory/maven for ex), and builds the app.
Now at this point, I wanted to get code coverage using JaCoCo plugin for my NewApp application project.
I followed the documentation per Gradle/Jacoco but it doesn't seems to create any reports/... folder for jacoco etc where I can find what Jacoco coverage report did.
My questions:
1. For getting code coverage using Unit tests (Junit), I assume all I have to do is the following and it will NOT require me to start/stop the tomcat before running unit test (test task i.e. "gradle test") to get code coverage for/via using unit tests. Please advise/correct. The code (just for Gradle jacoco unit test part) - I'm using is:
apply plugin: 'jacoco'
test {
include 'src/java-test/**'
}
jacocoTestReport {
group = "reporting"
description = "Generate Jacoco coverage reports after running tests."
reports {
xml.enabled true
html.enabled true
csv.enabled false
}
//classDirectories = fileTree(dir: 'build/classes/main', include: 'com/thc/**')
//sourceDirectories = fileTree(dir: 'scr/java', include: 'com/thc/**')
additionalSourceDirs = files(sourceSets.main.allJava.srcDirs)
}
and for Integration tests:
task integrationTest(type: Test) {
include 'test/java/**'
}
As jacocoTestReport is depends upon test task(s), thus they will be called first and then finally jacocoTestReport will report what it found for the code coverage.
For getting code coverage for integration tests, I assume I must start tomcat first (i.e. before running / calling test target for integration tests), then call "gradle integrationTest" or "gradle test" task and then stop tomcat -- to get the code coverage report. From other blog posts I also found that one should setup JAVA_OPTS variable to assign jacoco agent before tomcat starts.
for ex: setting JAVA_OPTS variable like:
export JACOCO="-Xms256m -Xmx512m -XX:MaxPermSize=1024m -javaagent:/production/jenkinsAKS/jobs/NewApp/workspace/jacoco-0.6.3.201306030806/lib/jacocoagent.jar=destfile=/production/jenkinsAKS/jobs/NewApp/workspace/jacoco/jacoco.exec,append=true,includes=*"
export JAVA_OPTS="$JAVA_OPTS $JACOCO"
Being new to Gradle/groovy - I'm not sure what code should I write within build.gradle (build script) to get the above Integration/Unit tests working if it involves start/stop of tomcat. If someone can provide a sample script to do that, I'll try.
I'm not getting any code coverage right now, when I publish Jacoco code coverage in Jenkins (using Jenkins post build action for publishing Jacoco reports). Jenkins build dashboard shows 0% for code coverage (i.e. bars showing all red color, no green for actual code coverage).
Need your advice to get some traction on this.

Question : I assume that your unit tests doesn't depend on tomcat. In this case, you're right, you must not start tomcat upfront.
To create the coverage report you need to execute
gradle jacocoTestReport
without jacocoTestReport gradle won't trigger jacoco to generate the reports.
One additional thing, regarding to your snippet. I assume that you have changed the the default main sourceset to source/java. in this case you don't have to set the additionalSourceDirs.
Integration tests : Yes, you need to start tomcat first, or at least you have to ensure that tomcat is running. You should have a look into Gradle 1.7. It has a new task ordering rule called finalizedBy
With this you could do something like
task integrationtests(type: Test) {
dependsOn startTomcat
finalizedBy stopTomcat
}
where start/stopTomcat are custom tasks.If you have to stay on Gradle 1.6 you have to build a dependsOn chain:
stopTomcat -dependsOn-> integrationtests -dependsOn-> startTomcat
I assume that the blog article is right, I don't have any experience with that.
Starting/Stoping Tomcat : You could do it in a way like this
task startTomcat() << {
def tomcatStartScript = "${project.rootDir}/tomcat/startScript"
tomcatStartScript.execute()
}
The stop script can be written in a similiar way. (Some in from Groovy doc : Executing)
Jenkins & Jacoco : Should be fixed when executing jacocoTestReport

Got it working.
Gradle 1.7
- download the .zip which contains the binaries/src and documentation.
- Go to folder: if you unzip the above .zip at C:\gradle-1.7
C:\gradle-1.7\samples\testing\jacoco\quickstart
Run:
gradle build jacocoTestReport
You’ll see a new folder “build” after the build.
– folder jacoco gets created with classdumps and .exec if only build task is called.
– folder jacoco and jacocoHtml gets created – if both build jacocoTestReport is called
have fun.
I also saw that it’s better to include:
the following section in build.gradle
/////
tasks.withType(Compile) {
options.debug = true
options.compilerArgs = ["-g"]
}
////

Related

How to Have 2 Code Coverages in Gitlab Repo Badges

My team has a gitlab repo. It has two parts: an NPM package under projects folder and an angular application under src folder. So there are 2 projects in the angular.json file.
We currently have unit tests with coverage setup in our gitlab pipes. The issue is, since we have 2 projects in this repo, we really need to show the coverage for each project.
I noticed in demo image of the gitlab badges documentation (https://docs.gitlab.com/ee/user/project/badges.html), they have a 'JS Coverage' badge. This seems to be a custom badge (I can't find a list of given badges, but I'm not finding anything for 'JS coverage', so I'm assuming it's custom).
So I think I can do something like that to create 2 custom badges that has the code coverage of each project (1 for 'Pkg Coverage' and 1 for 'App Coverage'). But (TBH) the documentation around creating custom badges isn't great. I need to know how to store this custom value to use in the badge, and how to update in the gitlab pipe.
Does anyone know how to achieve this? If I could just figure out how that example is using 'JS Coverage' (and how to update the value in the pipe), then I could figure out what I need to do for my 2 custom badges. Any tips?
Some details, right now we have a gitlab job like this (it runs unit tests and updates the coverage values. Since 'ng test' runs the tests of both projects 1 by 1, the code coverage of the 1st project is saved to the 'coverage' value):
unit-tests:
stage: test
rules:
# Run unit tests, including when merge requests are merged to default branch (so coverage % is updated)
- when: on_success
image: trion/ng-cli-karma:$ANGULAR_VERSION
before_script:
- *angular-env-setup-script
coverage: '/Statements \W+: (\d+\.\d+)%.*/'
script:
- npm run build:ds-prod
- npm install dist/ds
- ng test --code-coverage --progress false --watch false
artifacts:
expose_as: "Coverage Report"
paths:
- coverage/
tags:
- kubernetes-runner

How do I include unit test and exclude integration test during Build and Unit test phase and vice versa after the deployment for Gradle project?

we are using Gradle. We are using Jenkins for our CI-CD. The Jenkins stages in our project is Checkout, Build and unit test, Sonar, Docker, Approve deployment, Dev deployment and Integration test.
How can I run only unit tests during Build and unit test step in Jenkins and run only Integration test after Dev Deployment Jenkins?
I have attached a screenshot wherein I have included Integration test and excluded Unit tests in build.gradle file, but it's not the correct way.
Below is my Jenkins Groovy file:
Map buildOptions(Map optionParams) {
return [
javaVersion: '11.0'
] + optionParams
}
void call(Map optionParams = [:]) {
buildIfNotRelease(buildOptions(optionParams))
}
boolean buildIfNotRelease(Map options = [:]) {
def buildCommand = options.buildSystem == 'maven' ? 'mvn clean package' : './gradlew clean build'
return sh(script: "export JAVA_VERSION=${options.javaVersion};. /etc/profile.d/jenkins.sh;${buildCommand}"
// returnStatus: true
) == 0
}
Separate your integration test into different sourceSet using the jvm-test-suite-plugin.
This way you have a dedicated tasks for test and integrationTest and you don't have to deal with include/exclude stuff.
If you want to have some common test code, then use the java-test-fixtures-plugin.

Publish code coverage not finding coverage file in Azure DevOps

I'm using Node 14.x and Jest 26.x. There is a npm test script in package.json file which contains the following:
cross-env NODE_ENV=test jest --coverage --forceExit
When I run it locally, it generates the code coverage report in ./coverage directory. The contents of ./coverage directory is as follows:
lcov-report (folder)
clover.xml
coverage-final.json
lcov.info
It looks like clover.xml contains code coverage report. There are more details which can be found in lcov-report folder.
I've setup the Azure DevOps pipeline as follows for code coverage:
...
- task: PublishTestResults#2
condition: succeededOrFailed()
inputs:
testRunner: JUnit
testResultsFiles: '**/junit.xml'
# Publish code coverage results
# Publish Cobertura or JaCoCo code coverage results from a build
- task: PublishCodeCoverageResults#1
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '$(System.DefaultWorkingDirectory)/coverage/clover.xml'
After running the pipeline, Azure DevOps doesn't seem to find the cover.xml file. I get the following error:
##[debug]Result: true
##[debug]Report directory: /home/vsts/work/_temp/cchtml
Reading code coverage summary from '/home/vsts/work/1/s/coverage/clover.xml'
##[warning]No coverage data found. Check the build errors/warnings for more details.
##[debug]Processed: ##vso[codecoverage.publish codecoveragetool=Cobertura;summaryfile=/home/vsts/work/1/s/coverage/clover.xml;reportdirectory=/home/vsts/work/_temp/cchtml;]
I also tried the following options for summaryFileLocation but all resulted in same error.
'**/coverage/clover.xml'
'$(System.DefaultWorkingDirectory)/**/coverage/clover.xml'
I understand that the format of clover.xml may not be the same as Cobertura or JaCoCo, but at least Azure should be able to locate the file.
What I'm missing?
I've found a way accidentally.
It looks like Jest can generate cobertula coverage report. It needs to be added in Jest configuration. I've added the following in jest.config.js:
coverageReporters: ['text', 'text-summary', 'clover', 'cobertura']
Jest generated the cobertura coverage file in:
./coverage/cobertura-coverage.xml
Next I modified Azure pipeline file as follows:
- task: PublishCodeCoverageResults#1
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '$(System.DefaultWorkingDirectory)/coverage/cobertura-coverage.xml'
After this changes, when I run the pipeline, Azure DevOps can find the file and show coverage report!

How can I add JavaScript code coverage to TeamCity?

I have Chutzpah running QUnit tests in TeamCity with the following command:
chutzpah.console.exe /path src /debug /teamcity /coverage /emma coverage.xml
The tests are executed and listed in the Tests tab for the build.
The file coverage.xml is created and imported using a build feature. TeamCity is reporting a successful import:
[Ant JUnit report watcher] Successfully parsed
[Successfully parsed] 1 report
[Successfully parsed] coverage.xml
I cannot get the coverage report to be displayed in TeamCity.
How can I add the code coverage report to TeamCity?
I found 2 main ways to post coverage information in Teamcity:
Teamcity Service Messages: Some coverge reporters came with Teamcity report capabilities. This is the best one in my opinion, because you can use Coverage Thresholds as build failure conditions: i.e make the build fail if coverage is below 60%
HTML reports: If your coverage reporter generates HTML reports, you can either store them as Build Artifacts, or you can create a custom build tab to display coverage information, you only need a index.html file for this: https://confluence.jetbrains.com/display/TCD9/Including+Third-Party+Reports+in+the+Build+Results

Jenkins tests reports analyzer integration with catch

I've recently started working with Jenkins to automatically build my c++ project and run my tests (I'm using catch.cpp).
I wanted some sort of a table of test run time and status and that led me to the "Test Results Analyzer" Plugin for Jenkins.
I have my builds run like this:
And you can see they actually run in the console output:
finally, my test results analyzer plugin shows nothing:
It looks like the plugin does not recognize that these are my tests. Which is reasonable since I've only told jenkins to execute these commands and i don't think it's smart enough to understand these are the tests to report. But i could not find how to tell "Test Reports Analyzer" what are the tests it needs to report.
My question is how do i get a table of tests like in the Plugins webpage:
Tests Reports Analyzer
Solution:
Jenkins needs a Junit format xml file of the test results.
specifically, in Catch.cpp this is achieved by the "-r junit" command line option.
after this i needed to configure jenkins to "Publish JUnit test result report" post-build action and git it a path to the output xml file i create with my "make test" command.
Solution provided by OP:
Jenkins needs a Junit format xml file of the test results.
specifically, in Catch.cpp this is achieved by the "-r junit" command line option.
after this i needed to configure jenkins to "Publish JUnit test result report" post-build action and git it a path to the output xml file i create with my "make test" command.