In my Android Studio project I used this example with its including gradle scripts to run on each of my modules and combine all results of all modules together into one single coverage report. See GitHub playground project which worked fine so far for API level 31 or lower. Since I updated the compiledSdkVersion to
compileSdkVersion = 32
the scripts are not working anymore. They run through 100% and finish successfully, but no JaCoCo results were generated!?
Related
I have seen a lot of tutorials how to make vscode extensions. Like:
https://code.visualstudio.com/docs/extensions/testing-extensions
And there is many tutorials how to do coverage exports, there many ways how to do it, but I didn't seen good examples which would follow the examples from their docs and work with vscode extensions (they need the extensionHost instead of nodejs).
I have all the tests written in the mocha which is bundled in the vscode as proposed by their documents.
I tried to experiment and I'm stuck, are there any hints or directions where I could continue and get my process working again?
The answer which worked is in the comments to my original question. But to simplify and boil down what had to be done, and what steps I did personally. Not all steps are really essential and required, but these steps made it pretty convient:
installed istanbul, istanbul-coveralls, gulp, gulp-json-editor, coveralls to my package as the DEV dependencies
included the MS supplied istanbul test runner for vscode extension testing into my tests
modified the tests to run an istanbul runner instead of mocha runner directly (the tests itself could stay as they are).
attached reference to the istanbul runner now I had to read the coverconfig.json to know how to do the coverage itself (what parts to ignore, where is source, where to output results, what formats the results should be etc...).
added coverage to gitignore and to vscode project as so they will not annoy me in the UI and will not polute the repo.
added few visual studio launchers/tasks to make it easier for me to trigger them from the UI
installed vscode plugin to display the coverage in the gutters: https://marketplace.visualstudio.com/items?itemName=ryanluker.vscode-coverage-gutters and then added to my project settings the "lcov.path": [ "coverage/lcov.info" ] (or whatever location you will have the line coverage) so the plugin can read and show the coverage directly in the UI.
setup gulp to delete old coverage results and start new ones on each run
added npm scripts for regular tests and for the coverage variant
changed travis steps so the coverage variant is running now instead of the generic non-coverage one. Added script step so the coveralls module is run and broadcasts the results from the coverage/lcov.info to the https://coveralls.io/ website. (i think signing in and doing the first-time setup is required to make it work on their website)
Now on each commit travis will do the work by itself and I can use coveralls banner inside my readme to give quick glipse how well is my coverage. And you can use gulp to watch it for changes and build/test locally and have fast feedback in the UI what lines were covered by your tests.
The setup is similar to the listed links:
https://github.com/Microsoft/vscode-mssql
https://github.com/kenhowardpdx/vscode-gist/pull/10
I'm not a Windows 8 developer so excuse any stupid questions.
I have a Windows 8.1 application built in Visual Studio 2013 via a .sln file. This contains a number of projects: there is an application that relies on a library, this library (call it datalib) is built as a Windows Runtime Component (not a DLL);
there is also a test project which runs tests on 'datalib'
All of this works fine in VS2013, i.e. the tests execute.
I've been tasked with creating a TeamCity build for this so I've create a TeamCity build that builds against the sln file but I can't get the tests to execute (they're MSTest tests).
In my configuration I've added the test.dll (in the 'include assembly files list') but when I add this on it's own then I get a bunch of errors around references to types in the 'datalib'
The 'datalib' is built into datalib.winmd (I'm assuming this is the binary) so I thought that including this in the assembly list would fix my problem, but it doesn't.
I assume I'm missing something simple, what is it?
I'm now able to do this and it was a multi step process.
Firstly you don't load/test the winmd file directly you have to load the appx that's created by the test project 1; you can't use the MSTest runner that comes with TeamCity, instead you have to install the Visual Studio Test Runner plugin [2]; you have to run the build agent in an interactive process; and finally you have to install the root certificate that the application was signed with.
You reference the appx like you would a DLL
To install the build agent and have it run interactively it wasn't good enough to use the service and mark 'Interact with Desktop' you need to have full interaction so the agent has to be run from the desktop, as admin, at startup. To do this I had to create a scheduled task that run at startup and was given full privileges.
I installed the cert by running the ps1 file in the same directory as the appx file (Add-AppDevPackage.ps1) I'm sure this is overkill but it seemed like the easiest option at the time
[1] http://msdn.microsoft.com/en-us/library/hh691189.aspx
[2] https://confluence.jetbrains.com/display/TW/VSTest.Console+Runner
We have inherited a NetBeans Modules project (a.k.a. NetBeans Platform project) which builds using the NetBeans Ant tasks. We want to generate a code coverage report of the project to get an idea of what components are not being unit tested.
Generating a report is not part of the build process, it is simply something we want to do to understand the existing unit test coverage in the project.
We've decided to use JaCoCo to try and generate this. However the native support in NetBeans is for maven based projects. Unfortunately the TikiOne JaCoCoverage plugin currently does not support NetBeans Modules projects.
We are happy to just drive this via CLI to get an initial report. Does anyone have a success means of doing so under the given constraints?
Someone recently posted a related message to one of NetBeans mailing lists - https://netbeans.org/projects/platform/lists/dev/archive/2014-04/message/124
Short version: Cobertura plugin that does what you're looking for is not maintained now but you can build it and run it.
We have been successful in generating the report by doing the following:
Edited the project.properties file to run the JaCoCo agent when unit tests run. This is done by adding the following:
test.run.args=-javaagent:[yourpath/]jacocoagent.jar=destfile=[yourpath/]jacoco.exec
Use Netbeans to run the test task. This produces the jacoco.exec file.
Create a simple ant buildfile to generate reports from an .exec file (see this post)
Run the ant buildfile to generate the report.
I'm trying to work out how to run xUnit tests from TFS 2010.
I found some articles on how to achieve that with the older version (http://jonnekats.wordpress.com/2009/05/07/integrate-xunit-tests-into-your-daily-team-build/, http://weblogs.asp.net/mehfuzh/archive/2009/08/25/configuring-team-build-using-xunit.aspx).
These approaches don't work anymore with TFS 2010 because builds are no longer MSBuild files but instead are Workflows with different tasks.
What I would like to achieve is something similar to what I've got on my dev machine:
- build everything
- run the .xunit project file
- check the results
All that I can see on the web are custom build steps which I can't/won't use because I'll have to configure them for each single unit test assembly and they'll probably get completely messed up in TFS.
Any input would be appreciated.
Why dont use "InvokeProcess" in TFS Build workflow and invoke xUnit command line interface with the necessary xUnit project files as parameters?
You can capture the output of the command line by setting InvokeProcess->stdout to write a build message.
Take a look at the xUnit work item Support VSTS Test Runner on codeplex. You can find there a VS2010 extension in Beta version.
I have looked at other questions similar to this and they all seem to be Pre-RTM of Visual Studio 2010 (or they don't have a real answer).
I have downloaded the latest version of Gallio and I am trying to run my MS Test Project using it.
I added Gallio to my References in my MS Test Project then did a full build and then opened the compiled dll in Gallio.
I then then tried to run my tests and they don't run. I don't get any errors, just that they fail.
It says 218 tests - 0 passed - 1 failed - 0 inconclusive - 0 skipped.
I checked the run time logs and it says that it needs Gallio.VisualStudio.Shell which is disabled. I checked the plugins and this is that that one says:
Visual Studio Integration Shell
Provides a framework for hosting Gallio plugins within Visual Studio.
Disabled: The plugin enable condition was not satisfied. Please note that this is the intended behavior for plugins that must be hosted inside third party applications in order to work. Enable condition: '${process:DEVENV.EXE} or ${process:VSTESTHOST.EXE} or ${process:QTAGENT.EXE} or ${process:QTAGENT32.EXE} or ${process:QTDCAGENT.EXE} or ${process:QTDCAGENT32.EXE} or ${process:MSTEST.EXE}'.
I am not even running inside visual studio, but it sees it as needed. I have devenv.exe running (both Gallio and Devenv.exe are running under my account).
Any ideas on how to get this working?
This is a known bug fixed in build 3.2.587 an later. It did prevent the Gallio test runners to properly execute MSTest-based tests outside the Visual Studio environment (e.g. from Echo or Icarus)
There is also a quick workaround (from Kevin Ortman) if you need to make it work right now. You might just delete the whole attribute enableCondition in the following plugin definition data files (at line 3 or 4 I believe):
Gallio.VisualStudio.Shell90.plugin
Gallio.VisualStudio.Tip90.plugin
Gallio.VisualStudio.Shell.plugin
Or if you are working with VS2010:
Gallio.VisualStudio.Shell100.plugin
Gallio.VisualStudio.Tip100.plugin
Gallio.VisualStudio.Shell.plugin