Does Jacoco provide code coverage for integration tests of APIs? That is, I have an instance of my application running locally and I have integration tests where I hit an api offered by my running application instance. In this scenario can I use Jacoco to get how many lines of my running application instance was covered when integration tests where ran?
I have already tried Jacoco's maven plugin's prepare-agent-integration and report-integration goals. But they give code coverage as 0. I think its because jacoco only measures code coverage of the currently ran instance and not the instance whose api is hit.
I had forgot to run the javaagent while running the service. Running the jar file with javaagent with output=tcpserver and then dumping the execution file using Jacoco:dump and creating report using Jacoco:report solved the issue.
java -javaagent:<path_to_agent>/org.jacoco.agent-0.7.9-runtime.jar=output=tcpserver,address=127.0.0.1 -jar myapp.jar
mvn clean verify -Pintegration-tests
mvn jacoco:report -DdataFile=./target/jacoco.exec
mvn jacoco:dump -Djacoco.address=localhost -Djacoco.destFile=./service/target/jacoco.exec
Related
Hi We are using TeamCity and OCtopus for CI/CD. We have several hundred test cases.
As set up we have build server (Machine A) and it gets deployed on server (Machine B)
We use TeamCity and and last step is Deploy step through OctopusDeploy.
We have several test case which gets executed as Pre Deploy Tests. Now I want to add few Performance test case which will run on server (Machine B). How can I do this ?
Thanks in advance
Octopus is already helping us with this , You can write a PowerShell step that can run your performane testing cases. So the actual process of performing the tests is pretty easy.Moreover if you wanna check the test results with display (viewing the test results) .In Octopus 2.0 you can “attach” files to a deployment via PowerShell which are then uploaded and available on the deployment page.
But there is a possibility where you can run the load tests from the Teamcity using TC JMeter plugin and you can Parameterise the target environment and run the load tests , Here below is the link which can help you.
https://devblog.xero.com/run-jmeter-performance-tests-on-teamcity-8315f7ccffc1
References and sources:
https://octopus.com/docs/deployment-examples/custom-scripts
I have inherited a rails app which has an extensive suite of rspec and cucumber tests, when I run these tests on a CI server such as Circle, I see the tests run (most pass, some fail).
The app connects to a back-office API & I see the code has mocked responses to various API calls, the app also uses elasticsearch.
When I run the test suite locally using bundle exec rspec - I immediacy see a whole bunch of "Elasticsearch::Transport::Transport::Errors::BadRequest" errors from RSpec and failed to connect to those backoffice API's which I mentioned, however none of these failures occur on the CI server (yes, I have tried starting ES locally, same issue)
What am I doing wrong here? How can I run my tests locally to mirror what the CI server is doing to get them to run?
Any help is appreciated! Thank you.
I want to get a code coverage report (or tracking controller flow of an application) when I execute the some scenario on live application i.e. while application running in server, so I have refereed the official Jacoco documentation and did as fallows
As a first step, I tried to bind the jacoco agent to the application running port
export MAVEN_OPTS=-javaagent:/home/user/.m2/repository/org/jacoco/org.jacoco.agent/0.7.4.201502262128/org.jacoco.agent-0.7.4.201502262128-runtime.jar=includes=*,output=tcpserver,port=6300,address=*,dumponexit=true
then I have started the server and executed some test scenarios by that time I have also requested jacoco to get the dump by executing following command
mvn jacoco:dump
when I executed the above command first it showed like java.net.BindException: Address already in use then it printed as fallows
[INFO] Connecting to localhost/127.0.0.1:6300
When I stopped the server jacoco.exec has been generated to a known location there after I have requested to generate a jacoco report by executing the following command
mvn jacoco:report
Finally I got the report but when I open the index.html page it showed 0% coverage.
you can reproduce my difficulty with following sample Spring MVC project
please download Spring MVC project from https://github.com/mkyong/spring4-mvc-ajax-example from the Github
In my case I am using WildFly 8.x server to run the project and add the following dependencies to downloaded projects pom.xml
maven-surefire-plugin of 2.19.1 version
jacoco-maven-plugin of 0.7.4.201502262128 version
junit of 4.12 version
Then clean and build the project by using following commands.
mvn clean install
once it is done get the .war file from target folder of the project and keep inside the wildfly serverwildfly-9.0.0.CR2/standalone/deployments/ folder
Once the .war is deployed go to /wildfly-9.0.0.CR2/bin/ folder and execute/run the ./standalone.sh file or depending on platform you can run the.dat/.sh file
once project is deployed and server is started execute the following command
mvn jacoco:dump
it displayed like listening/connected to localhost/127.0.0.1:6300
Then execute some test scenarios and execute following maven command
mvn jaococ:report
Finally report will be generated with 0% coverage!.
Downloaded project can be found in MKyong website
Please help me to get the code coverage report and any help will be appreciated.
JaCoCo requires the exact same class files for report generation that were used at execution time, so
if report is completely empty, then classes were not provided
if report contains classes but their coverage is 0%, then they don't match classes that were used at runtime - this is described along with other related information in JaCoCo documentation on page http://www.jacoco.org/jacoco/trunk/doc/classids.html
and in either case check existence of warnings in log.
Update for updated question
Here is what I did:
downloaded and unpacked JaCoCo 0.7.9 into /tmp/jacoco/jacoco-0.7.9
downloaded and unpacked Wildfly 9.0.0.CR2 into /tmp/jacoco/wildfly-9.0.0.CR2
cloned https://github.com/mkyong/spring4-mvc-ajax-example into /tmp/jacoco/spring4-mvc-ajax-example and built as mvn verify
copied /tmp/jacoco/spring4-mvc-ajax-example/spring4-mvc-maven-ajax-example-1.0-SNAPSHOT.war into /tmp/jacoco/wildfly-9.0.0.CR2/standalone/deployments
Wildfly started as JAVA_OPTS=-javaagent:/tmp/jacoco/jacoco-0.7.9/lib/jacocoagent.jar=output=tcpserver ./standalone.sh and got enough time to deploy application
in directory /tmp/jacoco/spring4-mvc-ajax-example executed mvn org.jacoco:jacoco-maven-plugin:0.7.9:dump org.jacoco:jacoco-maven-plugin:0.7.9:report (note that version of used agent matches version of jacoco-maven-plugin) so that it created /tmp/jacoco/spring4-mvc-ajax-example/jacoco.exec and report /tmp/jacoco/spring4-mvc-ajax-example/site/jacoco:
opened http://localhost:8080/spring4-mvc-maven-ajax-example-1.0-SNAPSHOT/ and did some actions
executed mvn org.jacoco:jacoco-maven-plugin:0.7.9:dump org.jacoco:jacoco-maven-plugin:0.7.9:report again to get an updated report:
I have some maven projects configured in Jenkins, and I execute them from a parent project (which have them as <modules> in its pom.xml, the children have this project defined as <parent>).
The unit tests execute successfully for each project in Jenkins and it shows the results, but then in Sonar all projects appear as having no unit tests.
The sonar configuration for each project is
sonar.projectKey=project:key
sonar.projectName=project_name
sonar.projectVersion=1.0
sources=src/main/java
tests=src/test/java
binaries=target/classes
Do I have to do anything else in order for Sonar to pick up the results of the unit tests being executed?
Thanks in advance.
From what I see, I guess you're trying to run Sonar analyses with the Sonar Jenkins plugin "on-the-fly" mode, where you specify properties.
You have to understand that this mode does not support running tests: it can only reuse test reports (if you specify "sonar.dynamicAnalysis=reuseReports").
But as you are using Maven, why don't you just run the Sonar post-build action? (see documentation for that)
I have a job in Jenkins that is run every night. The tasks executed during this build are: compilation, unit tests, integration tests (which are only JUnit tests which are longer than "real unit tests" to execute), and Sonar quality analysis.
When a test fails, the job is however considered as successfull and thus, no email is sent to notify this failure.
The Maven command used is mvn clean install sonar:sonar. Removing the install goal does not change anything.
What is wrong with that?
Is there a way to get the expected behavior (i.e. having an unstable build when a test failed) with only one Jenkins job, or should I create two jobs, one for the whole "Java part" (compile, unit test and integration tests), and one for the Sonar analysis?
We are using Maven 2.0.9, Java 1.6, Sonar 2.8, Jenkins 1.413.
Jenkins seems to set that property: Hudson build successful with unit test failures
With the property (-Dmaven.test.failure.ignore=false), when there is a test failure, the build stops.
There is a jenkins plugin for sonar:
That seems to analyze even if Tests fails: http://jira.codehaus.org/browse/SONARPLUGINS-461
In my sonar installation, I run the tests seperate from sonar and reuse the junit/surefire reports. That way I can control the tests independently from sonar.