I am somewhat new to TeamCity. I have set it up for a Visual Studio solution.
I am using NUnit step to run unit tests and dotCover for code coverage. Problem is, there are particular parts of my code that I know are covered by unit tests but in the code coverage report, it shows them with zero coverage.
I know the DLLs are created just fine because I run a dir /s command and can see all the test DLLs.
The NUnit step uses MSIL 4.0 platform and the test path looks like this:
**/bin/Debug/*Tests.dll
There are no filters under code coverage.
Many tests are run. But there are some specific tests that are not present in the search report when I search.
Why is this happening and how do I fix it?
Can you please provide information about your NUnit step, especially about .Net Coverage (Filters).
Can you also confirm that tests are actually run?
Full build log (pasted in pastebin.com or such provider) would be also appreciated.
Stupid me. The DLL that was being ignored was spelled *Test.dll instead of *Tests.dll. I use *Tests.dll in my test path.
The issue got fixed once I changed it to *Test*.dll
Related
The code coverage tool in VS2017 enterprise no longer shows what application code is covered. Whether I run it on selected tests or all tests, it shows what code in the unit tests are covered, but shows the code actually being tested at 0%. When I first started using the tool it worked just fine and showed application code coverage. I only check coverage occasionally, so I'm not sure when it broke or what I might have done to cause the issue. I'm thinking that there is something in the Registry that is causing it since I haven't been able to find anything in Tools/Options/...
I installed JetBrains dotCoverage and it does the same thing even when I exclude the unit test project from the coverage session. I've tried to repair the VS installation. My next step is uninstall/reinstall. However, unless that cleans up the registry where I think the configuration error is, it will fail to address the problem.
So any suggestions on where I can look to figure this out would be appreciated.
I am trying to use NUnit for unit testing in my code base. When I try to run my test cases using ReSharper, I get:
"System.IO.FileNotFoundException: Could not load file or assembly
'ServicesTests.dll' or one of its dependencies.
ServicesTests is my class lib for my unit tests.
I have also tried to install the NUnit Test Adapter as described here but do not see my test cases in the Test Explorer.
Any ideas on what might be happening? Below are images of my class library from the solution explorer, the test explorer, ReSharper screen.
I was able to test using Nunit3 and VS2015 with Test Explorer after I changed the type of class library I was creating. Initially I was creating a Class Library (Package) and when I switched to Class Library it worked (see image below). I am still not able to use Resharper since I am using Nunit3 beta and that is currently unsupported by Resharper.
I have a similar issue also. The best answer I've come across to solve it is try and rebuild the solution (cleans and builds), or just resort to MS Test. It's a pain to convert the tests though...
Hope this helps.
EDIT:
My NUnit runs, but no tests "exist", despite having a project, classes, and methods all correct.
ReSharper does not support running unit tests in DNX projects https://youtrack.jetbrains.com/issue/RSRP-446679.
I know this is an old question but I had a similar issue recently in VS 2017. Use the Fusion Log Viewer (aka "Assembly Binding Log Viewer") to see what assembly is failing to load:
I have two projects in my solution which contains tests. I have tried clean build of solution but tests from only one project are appearing in the Test Explorer. I have also checked that Test->Test Settings->Default Processor Architecture is matching with that of project build settings (i.e. x86).
By default the Tests window will only show MsTest tests. If you have written your tests in Nunit you will need to install the 'Nunit Test Adapter' Visual studio extension for it to recognize those.
Have you tried "Kristian Kristensen's" answer to a similar question here:
How to get VS2010 to recognize my mstests generated by SpecFlow?
It worked for me.
When I set Project build settings to Any CPU it works but for any other settings (even when it matching with Test->Test Settings->Default) it fails to detect the unit tests.
I have 80+ VS2010 solutions, each contains some unit test projects.
All solutions are merged into one big solution before build process.
All tests (or some test subset) are executed after the successful build.
I want to enable test code coverage for all tests to get exact code coverage for all output assemblies.
My question is: how to enable code coverage without using Visual Studio?
Note: I'm using TFS2010 (MSBuild) to build merged solution. VS 2010 Premium is installed on the build server. MSTest is used for test execution.
You can use JetBrain's TeamCity Professional. It is a CI server that supports executing unit tests and calculating code coverage. It is free for small installations.
I think you need to consider deploying a code coverage tool, see here for a comparison (provided you implement .net).We use NCover, which integrated in our TFS-Build in it's console variant and, although it's not trivial to set it up, we 're very satisfied with it.In this post I had briefly described how we inserted NCoverin our build, this might we useful to you even if you go with another tool.
If you create a Vsmdi file in your large solution (ms test will usually do this for you) you can use this to tell the build which assemblies you want to instrument.
This will only provide code coverage for assemblies that have tests run against them. If you're using testrun.config files to decide which tests you want to run, this should be all you need. The code coverage results should then be published to the build drop location
Edit:
This blog post looks like it covers setting up code coverage
Is it possible to integrate Hudson with MS Test?
I am setting up a smaller CI server on my development machine with Hudson right now, just so that I can have some statistics (ie. FxCop and compiler warnings). Of course, it would also be nice if it could just run my unit tests and present their output.
Up to now, I have added the following batch task to Hudson, which makes it run the tests properly.
"%PROGRAMFILES%\Microsoft Visual Studio 9.0\Common7\IDE\MSTest.exe" /runconfig:LocalTestRun.testrunconfig /testcontainer:Tests\bin\Debug\Tests.dll
However, as far as I know, Hudson does not support analysis of MS Test results, yet. Does anyone know whether the TRX files generated by MSTest.exe can be transformed to the JUnit or NUnit result format (because those are supported by Hudson), or whether there is any other way to integrate MS Test unit tests with Hudson?
I've been meaning to write this as a guide and develop a plugin but I havent gotten around to it. I know this question is old but I'm SURE someone else out there wants the same thing so here it is.
In the project configuration on Hudson:
Execute Windows batch command
SET MSTest="C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\MSTest.exe"
SET XSLParser="C:\MsBuildNunit\msxsl.exe"
SET TestDLL=path-to-your-test-projects.dll
SET TestOutFILE=TestResults\some-unique-filename.trx
SET TransformedOutputFile=%TestOutFILE:.trx=%.xml
SET XSLFile=c:\MsBuildNunit\MSBuild-to-NUnit.xslt
MKDIR TestResults
%MSTest% "/testcontainer:%TestDLL%" /nologo /resultsfile:%TestOutFILE%
%XSLParser% %TestOutFILE% %XSLFile% -o %TransformedOutputFile%
SET ERRORLEVEL=0
Then check the box "Publish NUnit test result report" and for "Test report XMLs" enter
TestResults/*.xml
There is an XSLT in C:\MsBuildNunit as well as msxsl.exe which comes from Microsoft.
You can download the MSBuild-to-NUnit.xslt from here and get msxsl.exe from microsoft here or you can just get the zipped copy of my MsBuildNunit folder that contains the xslt and exe here
When run, it calls MSTest.exe which runs the tests and outputs the format in microsofts trx (xml) format. Then it calls msxsl.exe with the xslt and the trx and translates it to nunits xml format. At the end of the build, Hudson picks it up as any other Nunit test result and you are good to go.
Edited to add:
I forgot to mention, with this xslt we get full test results. We have multiple test projects and multiple dll's and we get great feedback with the ability to trend graph, view the tests by name, view the statuses of the tests, and if it errors we get the error message along with the stack trace. Basically almost everything that you would get with Nunit.
Edit (again): I just now added the test duration in the transform so it will show up in Hudson now! Seems to work great for our tests.
Edit: I tried the new MSTest plugin and it currently does not support parsing multiple TRX files, just 1, so currently this is your only solution if you are like us and have multiple test assemblies that you have to run through MSTest.
Hudson has a new plugin for MSTest. Just specify the location of the .trx file and the work is done for you. It wouldn't surprise me if the plugin used Allen's solution.
I've been able to use a variation of "hangy's" command line, and the MSTest plugin to successfully run and analyze/publish the test cases. The biggest change I made was to specify the output file for mstest.exe and fore the MSTest plugin to consume that file (no wildcards allowed... must be actual filename). For example, the following is my custom build step:
"%PROGRAMFILES%\Microsoft Visual Studio 9.0\Common7\IDE\MSTest.exe" /runconfig:LocalTestRun.testrunconfig /testcontainer:MyProject1.Test/bin/Debug/MyProject1.Test.dll /testcontainer: MyProject2.Test/bin/Debug/MyProject2.Test.dll /resultsfile:TestResults\HudsonJobTestResults.trx
exit 0
Notice that the "results file" is relative to the Job's workspace. Thus, the MSTest plugin's result file to parse is:
TestResults\HudsonJobTestResults.trx
And that's it!
Hudson has a Plot Plugin which can be used to plot generic data. It's not the easiest plugin to configure and use if you have multiple data points per graph, but if you can parse your MS Test output and generate input files for the plugin, you can at the very least plot the trends of failed, successful, and total tests.
I've not been able to use Hudson to perform analysis of MS Test results for historical purposes, but I've at least been able to figure out that if you use MSBuild and the Exec task, the Hudson build will properly be marked as "failed" if any of the tests fail.
<Exec Command=""C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\mstest.exe" /testcontainer:"MyAssembly.dll"" />