I am attempting to set up Java code coverage for a fairly complex app that
combines multiple large modules, only one of which I need to check coverage on
uses a combination of ant and Maven for builds
cannot be run except as an installed application on a server, with configuration
the automated tests to be analyzed for coverage are not part of the application build and make use of API calls to the application server from a remote client
The examples given in the jacoco documentation and in the online sources I have found assume the app under test is not previously installed and the tests are unit/integration tests run as part of the build. The documentation does not cover the details of how the jacoco instrumentation is done or when the call is recorded to a particular line of code. If I use ant or maven to instrument a particular module, use that module to build the full app, install it on a server, and configure it, will my remote tests then generate the .exec file?
Any advice on how to achieve the end goal (knowing how much of our code is covered by the tests) is greatly appreciated, including better search terms than "jacoco for installed app" which as you can imagine is ... not very useful. My google-fu is humbled.
Related
I had configured JaCoCo in WebSphere as JavaAgent (Refer: http://www.jacoco.org/jacoco/trunk/doc/agent.html).
Restarted the server, and ran a series of automated tests on the application (to give some load) and then stopped the server.
I can see the jacoco.exec getting generated in the Server (as configured to /tmp/ location).
Now, How do I generate the HTML report ?
Before voting down this question or marking it as duplicate, here is the reason why I'm posting this question. I went through the JaCoCo Documentation like http://www.jacoco.org/jacoco/trunk/doc/maven.html and also multiple StackOverflow questions but still I'm confused.
What I understood is that the Maven plugin allows us to run the Unit tests, Integration tests and then generate a report.
What I'm looking for is a report based on the load I had given to my application deployed in Websphere. I can see the jacoco.exec file generated, but not sure from the documentation on how to generate the HTML reports.
Thanks in advance.
You could use the jacoco:report-aggregate goal with Maven.
You could refer this http://www.eclemma.org/jacoco/trunk/doc/report-aggregate-mojo.html
P.S. : However, when i had the same issue, I used Sonar to read the exec file that was generated. It gives much more than just code coverage.
I was able to generate JaCoCo report as follows :
Configured JaCoCo as Java Agent
Restart the Server and do some transactions/give some load (in my case I ran a series of automated tests)
Stop the Server (This will actually generate the jacoco.exec file)
Create/Configure the Ant script and run it (This will read the jacoco.exec file and generate the html report). Refer : http://www.eclemma.org/jacoco/trunk/doc/ant.html
Even though my Project is a Maven project, I used ant script for the Report generation. I automated all the above steps using Bamboo and it made running and maintaining this Job easier.
So after much hunting I failed to find a continuous testing tool for IntelliJ 14.
I stumbled across a post that references uses eclipse and Ant in order to simulate this. On save, Ant then runs the tests for any tests that were modified.
I've tried to replicate this but, alas! I've never used Ant before and am finding it extremely difficult. I've setup and configured a generic Ant build file in Intellij but simply cannot figure out how to achieve my task.
Any help, pointers in the right direction is very much appreciated. I've searched but only found information that needs to be decrypted first.
Eclipse has the builder feature, you create an AntBuilder for your project, see also https://stackoverflow.com/a/15075732/130683.
IntelliJ has a trigger feature that might serve the purpose.
Also Infinitest , which provides a Continous Testing Plugin for Eclipse and IntelliJ might be helpful.
Ant is a build tool. Although IntelliJ does that for you, you need IntelliJ to do this which means you can't distribute your application without IntelliJ.
Ant uses a dependency matrix for building. This is sometimes difficult for developers to understand, but it basically means that you define the steps, how the steps are dependent upon each other, and let the build tool figure out exactly how to do its job. Ant is for Java like Make is to C and C++ applications.
Ant uses targets which are the steps you specify to do. For example, you might have a target called package that will build your jar or war. That target might depend upon another target called compile to compile the code. That target might depend upon a code generation phases (like if you had WSDL files).
Each target is a set of tasks. For example, the compile target is likely to have the <javac> task in it. It might also need the <mkdir> task to create the work directories where you classfiles are stored.
There are plenty of books on Ant, and there's a tutorial on the Ant Website. You didn't explain the issues you were having, so it's hard to be more specific than this.
Ant can also run your unit tests too. There's a <junit> target which can run the tests, and you specify whether or not you want to run almost all of your <junit> tests via the <batchtest> sub-entity or if you have a program driver you specify via the <test> entity.
Once you get an Ant script that can build and run your tests outside of IntelliJ, you can now get a Continuous Integration tool like Jenkins. A continuous integration tool watches your repository for changes, and if a change occurs, will then build your application. It's a great way to catch errors early on.
What does this have to do with Continuous Testing? Well, if you have your Ant script able to run unit tests, the Continuous Integration engine not only can build your app, but then run the unit tests with each and every change that occurs.
Jenkins is nice because it's very simple to use. You download a jenkins.war and you can launch the Jenkins webpage via the java -jar jenkins.war command. This brings up a web server on port 8080 on your machine. Obviously, Jenkins can be configured to run on different ports and under Tomcat if you so desire. It can integrate with Windows Active Directory, LDAP, and many other user verification systems.
Jenkins will show you charts and graphs of your tests, let you know which tests failed or passed, and will notify you of any problems via email, tweets, IM, Jabber, and even Facebook posts. People have even setup a traffic light in their offices that turns red when builds or tests fail.
Take it one step at a time. Get a good book on Ant. Read the tutorial on the Ant website. Then try to get a working Ant script to just to build your app. If you are having specific issues, you can ask for help.
Once you have the build going, extend the script to run your unit tests. Once that is done, download Jenkins and try to get that up and running.
I am currently using jenkins to build a list of different modules for my project. I trigger the builds using Maven. I have sonarqube installed on the server and have set it up correctly so that when a module builds it is displayed on sonarqube and includes all of the basic details such as lines of code, technical debt etc. The modules all have Junit tests that run against them, and sonarqube displays this by saying that the Unit Test Sucess is 100% and it also says the number of tests that have been run in that module. However I cannot get the Unit tests coverage field to display anything and it is blank for all of the modules.
Here is an exert (one module) from my pom.xml
customer.sonar.projectBaseDir=.
customer.sonar.sources=D:/TFS/WorkSpace/DEV_2_HYBRID/APP_FO/application/customer/src/main/java
customer.sonar.Hybrid=Customer
customer.sonar.tests=D:/TFS/WorkSpace/DEV_2_HYBRID/APP_FO/application/customer/target/surefire-reports
customer.sonar.junit.reportsPath=D:/TFS/WorkSpace/DEV_2_HYBRID/APP_FO/application/customer/target/surefire-reports
The versions of the software I am using are as follows:
Sonarqube v.5.0,
Jenkins Sonarqube plugin v.2.1,
Maven v3.2.5
As I said at the beginning the unit test success rate does show successfully, so I believe it is only a small change needed that will get the unit test coverage field working.
Any help would be really appreciated!
You need to execute the coverage engine of your choice and provide the report to SonarQube via the appropriate property.
If you are using JaCoCo, the report importer is embeded in the java plugin, for other coverage engine (clover, cobertura...) you have to install the dedicated plugin.
For more information see the dedicated page of documentation.
I've a server running a proprietary language on which I'm able to run "unit tests" in this language. I cannot install a Hudson slave on this machine, but would like to have these tests results appearing in a job of hudson (to have at least a monitoring of the code quality for this server code).
I'm currently trying to use web services to get the results and store them in Hudson workspace, but I do fear it is not the right solution.
What solutions can you advice me ?
I finally have gotten through the web services path, although it was not easy.
There are some steps in this path
I created a maven mojo with groovy (see GMaven for more infos) which, using groovyws, called a web service that, from tests results, creates the junit report.
Armed with this mojo, I created a maven project that called the web service and stores the junit.xml file in an output folder
Finally, i created in hudson a maven job for this project and called it regularly. Thanks to junit reporting integration in maven builds, my tests results are visible as a graph in Hudson and user can drill down to failing tests.
Not sure if these are possible but...
Maybe one option is when the build job finished execute a second build target or script to scp the test results from the remote server to the local build server so they appear in hudson
Or if the platform allows
Map a directory on the remote machine to the local file system by using something like sshfs etc
karl
Yup, you can scp or whatever the results (in junit xml format) to the current workspace dir using a script task. Then have a "Publish JUnit test result report" post-build task & point it at the copied-in files.
Obviously if it's not in junit-compatible format you'll have to convert it.
Sounds like you're on the right path though
In a J2EE web application, how do people manage resources so that they are visible to both the web context and to unit/integration tests?
I find that often you end up having your source/resource folders configured a certain way during development (i.e., what Maven expects) and so your unit tests will run in your IDE. But once the web app is built and packaged into a WAR file (i.e., when your Continuous Integration server has done a build) your unit tests won't run anymore because the resources are located elsewhere.
Do you end up keeping resources in two different places and manually keeping them in sync?
We tried using unit tests in the container but gave up on it years ago. It's much better (for us at least) to make each unit test cover a single class and nothing else, mocking out the dependencies on other classes (see JMock or its many competitors). A good basic rule is that if it touches the database, network, or the filesystem, it isn't a unit test. (It may be useful for something else, but it isn't a unit test. See these unit testing rules for more on this.)
Unit tests written this way can be run anywhere, and they are blazingly fast (we have thousands and run them in under 60 seconds on medium-spec hardware.)
You may also want to run integration tests that check a subsystem or the whole application. We find that subsystem tests can also use mocking at their borders - for instance, we fake an external pricing feed - and that end-to-end tests work best with tools like Selenium or WebDriver, which let you deploy the whole application on a server and then hit it with a browser just like users do.
(By the way, our method of unit testing makes us mockists, rather than classicists, in Martin Fowler's taxonomy.)
Normally this is the reason for multi-module builds. The external services are in a separate build unit than the web application. So you build, package and run your integrations tests when you build that module.
Another module can contain your domain model and its unit tests, which are also run at build time.
It is quite common for a module that results in a WAR to not have any java code in it at all, but only web related artifacts. Although not necessary, this is often done because code that is in a war module cannot be included into another module.
The last special case is the module containing web-tests. This module may often need test-scoped artifacts from the other modules (because it is testing the application from the outside, but may need data from the inside). This can be solves by also packaging test-resources in jar files, creating a separate set of "test" jar files for each modules.
Multi module builds are the norm for maven projects, and are also easy to set up for other build systems like ant.
I won't package testing resources nor tests in a WAR file neither run unit tests from the WAR. Why you are trying to do so?