We have inherited a NetBeans Modules project (a.k.a. NetBeans Platform project) which builds using the NetBeans Ant tasks. We want to generate a code coverage report of the project to get an idea of what components are not being unit tested.
Generating a report is not part of the build process, it is simply something we want to do to understand the existing unit test coverage in the project.
We've decided to use JaCoCo to try and generate this. However the native support in NetBeans is for maven based projects. Unfortunately the TikiOne JaCoCoverage plugin currently does not support NetBeans Modules projects.
We are happy to just drive this via CLI to get an initial report. Does anyone have a success means of doing so under the given constraints?
Someone recently posted a related message to one of NetBeans mailing lists - https://netbeans.org/projects/platform/lists/dev/archive/2014-04/message/124
Short version: Cobertura plugin that does what you're looking for is not maintained now but you can build it and run it.
We have been successful in generating the report by doing the following:
Edited the project.properties file to run the JaCoCo agent when unit tests run. This is done by adding the following:
test.run.args=-javaagent:[yourpath/]jacocoagent.jar=destfile=[yourpath/]jacoco.exec
Use Netbeans to run the test task. This produces the jacoco.exec file.
Create a simple ant buildfile to generate reports from an .exec file (see this post)
Run the ant buildfile to generate the report.
Related
I am using Google test framework for C++ unit tests. We are building our projects using MsBuild (runs on teamcity). Now, I want sonarqube to parse the coverage info. We have cfamily plugin in sonarqube. The compatible reports with cfamily plugin are bullseye, vscoverage, gcov, llvm-cov. As per my knowledge, because we can't use gcc for compiling, llvm-cov and gcov are ruled out. Since we are using googletest and also wants to run this on teamcity, vscoverage isn't possible. We aren't using Bullseye. (It is more for functional automation I am told).
So I have decide to use OpenCppCoverage tool. This can generate coverage in cobertura format or generic format specified by sonarqube. I have tried generic format but sonarqube is ignoring the coverage for files but is parsing them successfully.
Exploring more, I tried to use C++ community plugin(cxx). But I wasn't able to disable cfamily plugin so that C++ community can be used.
So I want to know if I can do something else so that coverage of our C++ test projects can be parsed by sonarqube.
Did you try sonar-cxx plug-in from SonarOpenCommunity. This plugin adds C++ support to SonarQube. It's free plug-in and availble with LGPL-3.0 License.
I made the paths in coverage report generated by OpenCppCoverage using OCCSonarQube plugin. Now SonarQube is able to parse the report in generic format.
I have seen a lot of tutorials how to make vscode extensions. Like:
https://code.visualstudio.com/docs/extensions/testing-extensions
And there is many tutorials how to do coverage exports, there many ways how to do it, but I didn't seen good examples which would follow the examples from their docs and work with vscode extensions (they need the extensionHost instead of nodejs).
I have all the tests written in the mocha which is bundled in the vscode as proposed by their documents.
I tried to experiment and I'm stuck, are there any hints or directions where I could continue and get my process working again?
The answer which worked is in the comments to my original question. But to simplify and boil down what had to be done, and what steps I did personally. Not all steps are really essential and required, but these steps made it pretty convient:
installed istanbul, istanbul-coveralls, gulp, gulp-json-editor, coveralls to my package as the DEV dependencies
included the MS supplied istanbul test runner for vscode extension testing into my tests
modified the tests to run an istanbul runner instead of mocha runner directly (the tests itself could stay as they are).
attached reference to the istanbul runner now I had to read the coverconfig.json to know how to do the coverage itself (what parts to ignore, where is source, where to output results, what formats the results should be etc...).
added coverage to gitignore and to vscode project as so they will not annoy me in the UI and will not polute the repo.
added few visual studio launchers/tasks to make it easier for me to trigger them from the UI
installed vscode plugin to display the coverage in the gutters: https://marketplace.visualstudio.com/items?itemName=ryanluker.vscode-coverage-gutters and then added to my project settings the "lcov.path": [ "coverage/lcov.info" ] (or whatever location you will have the line coverage) so the plugin can read and show the coverage directly in the UI.
setup gulp to delete old coverage results and start new ones on each run
added npm scripts for regular tests and for the coverage variant
changed travis steps so the coverage variant is running now instead of the generic non-coverage one. Added script step so the coveralls module is run and broadcasts the results from the coverage/lcov.info to the https://coveralls.io/ website. (i think signing in and doing the first-time setup is required to make it work on their website)
Now on each commit travis will do the work by itself and I can use coveralls banner inside my readme to give quick glipse how well is my coverage. And you can use gulp to watch it for changes and build/test locally and have fast feedback in the UI what lines were covered by your tests.
The setup is similar to the listed links:
https://github.com/Microsoft/vscode-mssql
https://github.com/kenhowardpdx/vscode-gist/pull/10
In my build definition I have couple of Java & multiple MS build projects. Java projects are built using Ant. There are multiple ANT & MSbuild steps in build defination
I want to integrate my build with SonarQube. I referred to this link https://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Extension+for+VSTS-TFS to add one begin analysis and one end analysis tasks,these tasks are added almost in the beginning & end of the build defination. Hence there are only 2 Sonar task & in between there are number of ms build and Ant tasks. When I run my build then I get some weird results on SonarQube. it seems like it scanned only part of codebase not the entire codebase.
I have following questions
Does SonarQube VSTS integration supports multiple MSbuild projects inside a same build definition. If Yes, can anyone please give pointers how to do that.
I have couple of Ant tasks as well. Is it supported by Sonar task? (its name says MSBuild task, is it applicable for Ant as well)?
Atul Sureka
I have 80+ VS2010 solutions, each contains some unit test projects.
All solutions are merged into one big solution before build process.
All tests (or some test subset) are executed after the successful build.
I want to enable test code coverage for all tests to get exact code coverage for all output assemblies.
My question is: how to enable code coverage without using Visual Studio?
Note: I'm using TFS2010 (MSBuild) to build merged solution. VS 2010 Premium is installed on the build server. MSTest is used for test execution.
You can use JetBrain's TeamCity Professional. It is a CI server that supports executing unit tests and calculating code coverage. It is free for small installations.
I think you need to consider deploying a code coverage tool, see here for a comparison (provided you implement .net).We use NCover, which integrated in our TFS-Build in it's console variant and, although it's not trivial to set it up, we 're very satisfied with it.In this post I had briefly described how we inserted NCoverin our build, this might we useful to you even if you go with another tool.
If you create a Vsmdi file in your large solution (ms test will usually do this for you) you can use this to tell the build which assemblies you want to instrument.
This will only provide code coverage for assemblies that have tests run against them. If you're using testrun.config files to decide which tests you want to run, this should be all you need. The code coverage results should then be published to the build drop location
Edit:
This blog post looks like it covers setting up code coverage
I'm trying to work out how to run xUnit tests from TFS 2010.
I found some articles on how to achieve that with the older version (http://jonnekats.wordpress.com/2009/05/07/integrate-xunit-tests-into-your-daily-team-build/, http://weblogs.asp.net/mehfuzh/archive/2009/08/25/configuring-team-build-using-xunit.aspx).
These approaches don't work anymore with TFS 2010 because builds are no longer MSBuild files but instead are Workflows with different tasks.
What I would like to achieve is something similar to what I've got on my dev machine:
- build everything
- run the .xunit project file
- check the results
All that I can see on the web are custom build steps which I can't/won't use because I'll have to configure them for each single unit test assembly and they'll probably get completely messed up in TFS.
Any input would be appreciated.
Why dont use "InvokeProcess" in TFS Build workflow and invoke xUnit command line interface with the necessary xUnit project files as parameters?
You can capture the output of the command line by setting InvokeProcess->stdout to write a build message.
Take a look at the xUnit work item Support VSTS Test Runner on codeplex. You can find there a VS2010 extension in Beta version.