I am trying to use NUnit for unit testing in my code base. When I try to run my test cases using ReSharper, I get:
"System.IO.FileNotFoundException: Could not load file or assembly
'ServicesTests.dll' or one of its dependencies.
ServicesTests is my class lib for my unit tests.
I have also tried to install the NUnit Test Adapter as described here but do not see my test cases in the Test Explorer.
Any ideas on what might be happening? Below are images of my class library from the solution explorer, the test explorer, ReSharper screen.
I was able to test using Nunit3 and VS2015 with Test Explorer after I changed the type of class library I was creating. Initially I was creating a Class Library (Package) and when I switched to Class Library it worked (see image below). I am still not able to use Resharper since I am using Nunit3 beta and that is currently unsupported by Resharper.
I have a similar issue also. The best answer I've come across to solve it is try and rebuild the solution (cleans and builds), or just resort to MS Test. It's a pain to convert the tests though...
Hope this helps.
EDIT:
My NUnit runs, but no tests "exist", despite having a project, classes, and methods all correct.
ReSharper does not support running unit tests in DNX projects https://youtrack.jetbrains.com/issue/RSRP-446679.
I know this is an old question but I had a similar issue recently in VS 2017. Use the Fusion Log Viewer (aka "Assembly Binding Log Viewer") to see what assembly is failing to load:
EDIT 2016-10-19:
The original question was about an issue specific to VS2015 CTP6 with the XUnit test runner. It's clear from the answers that there is a much broader issue with unit test discovery in Visual Studio which may occur in many different situations. I have cleaned up my question to reflect that.
I have also included a script in my own answer that I still use to this day to solve similar problems when they appear.
Many other answers have also proven helpful in better understanding the intricacies of the VS test runner. I appreciate that people are still sharing their solutions!
Original question 2015-04-10:
Since yesterday, my Visual Studio Test Explorer won't discover tests for any of my projects. It does not show the green loading bar after building, either.
When I go to the Visual Studio Test Explorer and click "Run All", or when I right-click any test method and select "Run Tests", I get the following in my output window:
Could not load file or assembly 'Microsoft.VisualStudio.Web.ProjectSystem, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
I am running Visual Studio 2015 CTP 6 on Windows 10 Pro Technical Preview, build 10041. The .NET Framework version does not seem to matter - it happens on 4.0, 4.5.2 and 4.6.
I tried with the following testing frameworks and all of them give the same behavior:
Microsoft.VisualStudio.QualityTools.UnitTestFramework v14.0.22609.0
xunit v2.1.0-beta1-build2945 with xunit.runner.visualstudio v2.1.0-beta1-build1051
NUnit v2.6.4 with NUnitTestAdapter v2.0.0
I found an issue on GitHub (xunit) that appeared to be similar: Cannot get tests discovered #295, with this comment from the xunit team:
Be aware that Visual Studio 2015 CTP 5 has been reported to be broken
by many people with unit testing in general (not just xUnit.net), so
don't expect that to work.
Also, please make sure you've cleaned out Visual Studio's runner
cache. If it gets corrupted, Visual Studio will permanently misbehave
until it's deleted. To clear the cache, shut down all instances of
Visual Studio, then delete the folder
%TEMP%\VisualStudioTestExplorerExtensions (honestly, it probably
wouldn't hurt to delete everything in %TEMP% that can be deleted).
I tried their suggestion to delete the folder %TEMP%\VisualStudioTestExplorerExtensions. Unfortunately that did not fix the problem.
I noticed that ReSharper actually is able to discover some tests. It only works for the VS and NUnit tests, not for xunit.
There has to be some sort of temp or cache folder I need to clear, but I know Visual Studio has many of them and not all of them can be deleted without unwanted side-effects.
To my surprise, clearing temp files located in the %TEMP% directory resolved the issue for me.
Note: This path is generally at C:\Users\(yourusername)\AppData\Local\Temp
As #Warren-P included, you can navigate to the temp folder by putting in %temp% in Start Menu, or launch "File Explorer" and enter %temp% in the address bar.
It could be that your code is compiled with x64 therefore have to enable the Default Processor Architecture as X64.
Test > Test Settings > Default Processor Architecture > X64
Check out, if NUnit Test Adapter 2/3 is installed in VisualStudio.
(Tools>Extensions and Updates )
Make sure that correct processor architecture is chosen:
(Test>Test Settings>Default Processor Architecture)
EDIT 2016-10-19 (PowerShell script)
This issue still returns every now and then. I wrote a small PowerShell snippet to automate clearing the relevant cache/temp folder/files for me. I'm sharing it here for future readers:
#(
"$env:TEMP"
"$env:LOCALAPPDATA\Microsoft\UnitTest"
"$env:LOCALAPPDATA\Microsoft\VisualStudio\14.0\1033\SpecificFolderCache.xml"
"$env:LOCALAPPDATA\Microsoft\VisualStudio\14.0\1033\ProjectTemplateMRU.xml"
"$env:LOCALAPPDATA\Microsoft\VisualStudio\14.0\ComponentModelCache"
"$env:LOCALAPPDATA\Microsoft\VisualStudio\14.0\Designer\ShadowCache"
"$env:LOCALAPPDATA\Microsoft\VisualStudio\14.0\ImageLibrary\cache"
"$env:LOCALAPPDATA\Microsoft\VisualStudio Services\6.0\Cache"
"$env:LOCALAPPDATA\Microsoft\WebsiteCache"
"$env:LOCALAPPDATA\NuGet\Cache"
) |% { Remove-Item -Path $_ -Recurse -Force }
Make sure to close Visual Studio beforehand and it's probably a good idea to reboot afterwards.
Deleting the TEMP folder may not be necessary and may in some cases even be undesirable, so I would recommend trying without clearing the TEMP folder first. Just omit the "$env:TEMP".
Original answer 2015-04-12
The problem was "solved" after a thorough cleaning of Visual Studio-related temp/cache folders.
Since I did not have the time to go through everything one-by-one and then test in-between, I unfortunately don't know which one actually caused the problem.
These are the exact steps I've taken:
Closed Visual Studio
Used CCleaner to clear system and browser temp files/folders
Manually cleared/deleted the following files/folders:
%USERPROFILE%\AppData\Local\assembly
%USERPROFILE%\AppData\Local\Microsoft\UnitTest
%USERPROFILE%\AppData\Local\Microsoft\VisualStudio\14.0\1033\SpecificFolderCache.xml
%USERPROFILE%\AppData\Local\Microsoft\VisualStudio\14.0\1033\ProjectTemplateMRU.xml
%USERPROFILE%\AppData\Local\Microsoft\VisualStudio\14.0\ComponentModelCache
%USERPROFILE%\AppData\Local\Microsoft\VisualStudio\14.0\Designer\ShadowCache
%USERPROFILE%\AppData\Local\Microsoft\VisualStudio\14.0\ImageLibrary\cache
%USERPROFILE%\AppData\Local\Microsoft\VisualStudio Services\6.0\Cache
%USERPROFILE%\AppData\Local\Microsoft\WebsiteCache
%USERPROFILE%\AppData\Local\NuGet\Cache
%USERPROFILE%\AppData\Local\Temp
One reason for this problem is that your test class is not public. MSTest only discovers tests from public classes.
In Visual Studio 2015(Update 3) if you want to attach the tests in the test explorer then have to install the NUnit Test Adapter.Download the adapter from Tools->Extension And Updates->Online tab(you have to search for the adapter)->Download.
By restarting the Visual Studio you can see the change for the test framework.
I don't have a complete answer to this, but I have determined some things by playing with a test project:
The xunit.runner.aspnet : 2.0.0-aspnet-beta4 that appears to be part of the official beta4 aspnet5 release does not work in Visual Studio.
Instead, using "xunit": "2.1.0-*" and "xunit-runner.dnx": "2.1.0-*" packages DO work in Visual Studio.
In order for VS to discover the tests, your project MUST have a SINGLE command named "test" that runs "xunit.runner.dnx". Adding additional commands may break it.
If your Test Explorer window still ends up empty, REMOVE the "test" command from your project, then re-build the solution, then add the "test" command back to the project.json.
Clearing all your caches as per #Fred-Kleuver 's suggestion may help, but I haven't done all the steps in isolation, so I'm not sure.
This is current as per VS 2015 CTP 6, using the beta4 releases, not the dailies.
I had an instance where some tests would not be picked up because I had made them async like the following:
public async void This_IsMy_UnitTest()
The problem was I forgot to make them return a Task and not void when I did the switch-over. One would think this would cause an error or failed test but nope. The unit tests in that class were fully ignored and acted like they didn't exist.
It wasn't after about 3 clean and builds + restarting VS.NET that I saw the test run and fail indicating I forgot to add the Task return type:
public async Task This_IsMy_UnitTest()
After the update, the unit tests were found and worked correctly. This might be an edge case, but having async tests for using await within but not having the signature correct can cause this same issue and it's not the 1st time I've done this.
Go to Nuget package manager and download Nunit Adapter as follow.
I had the same pronlem but the folder "%TEMP%\VisualStudioTestExplorerExtensions" did not exist on my machine so as i read the posts i had the idea to create it and it works. The test explorer is now able to show all my tests.
Thanks.
Just restart Visual Studio and in Test Explorer do "Run All" ... All my tests are discovered then.
In my case (Visual Studio Enterprise 2015 14.0.25425.01 Update 3, Resharper 2016.2) I just needed to do a clean solution from the Build menu. Rebuilding the solution then cause the test explorer to "wake up" and find all the tests again.
The solution in my case was just to install the NUnit 3 Test Adapter extension to my Visual Studio 2015.
In my case, the problem was "between the chair and keyboard". I had switched to a configuration in the Configuration Manager which did not include my unit test projects on build. Switching back to a configuration (e.g. Debug) which includes all projects fixed the issue.
In my case, MSTest under VS 2015 was ignoring tests with test (i.e. method) names that were longer than 174 characters. Shortening the name allowed the test to be visible. This was determined via guess-and-check by manipulating the test name.
This probably won't help most people, but someone inexperienced at unit testing had written a test method that returned bool instead of void:
[TestMethod]
public bool TestSomething()
Changing the return type to void fixed the problem.
Ensure you have xunit.runner.visualstudio package in your test project packages.config and also that was correctly restored.
I know this was not the case of the original question however it could save time for someone like me.
I would just like to add that I found an entirely different solution to the ones above.
I had declared my test class as below:
[TestClass]
class ClassificationTests
{
//unit tests
}
As soon as I added the public modifier to the class, it worked as expected!
This topic is somewhat outdated, but my solution to the missing Test status in VS2015:
The task-status only shows up on the Debug buildconfiguration. Ofcourse this also makes it impossible to debug your test through the test-explorer.
Somehow my project was set to compile as a Static Library (.lib). After changing this to a Dynamic Library (.dll), tests where discovered correctly by Visual Studio 2012.
My Unit Test Project ->
Properties ->
Configuration Properties ->
General ->
Configuration Type
If you're targetting .NET Standard or .NET Core, you need to use the NuGet package for NUnit Test Adapter and not the extension.
It is recommended to install the adapter from NuGet if you are testing .NET Core or .NET Standard projects. The VSIX adapter does not, and will not, support .NET Core because VSIX packages cannot target multiple platforms.
Source: NUnit GitHub Wiki
.
Also check the FAQ there:
My tests aren't showing up in Visual Studio 2017?
Are you using the NuGet package?
Are you using version 3.8.0 or newer of the NuGet package?
Do your tests target .NET Core or the full .NET Framework? (see above)
Have you added a Package Reference to Microsoft.NET.Test.Sdk?
Have you restarted Visual Studio? It is still a bit tempermental.
Source: NUnit GitHub Wiki
This happened to me because my test project contained an app.config.
It was automatically added by NuGet packages for assembly redirection, but my tests seemed to run fine without it.
See: https://developercommunity.visualstudio.com/comments/42858/view.html.
I had the same problem. I just cleaned and rebuilt the project and I was able to see the tests that were missing.
Popping in to share my solution. I was on Windows 10, Visual Studio 2015, NUnit 3.5, NUnit Test Adapter 3.6 (via NuGet, not the VISX extension) and none of my tests were being discovered. My problem was that in the Tests project of my solution, somehow a shortcut to my "Documents" folder had been created within the project folder. I'm guessing the test adapter was seeing the shortcut and getting hung up trying to figure out what to do with it, resulting in the failure to display unit tests.
Deleting the file \AppData\Local\Microsoft\VisualStudio\14.0\1033\SpecificFolderCache.xml solved the issue for me.
I was also bitten by this wonderful little feature and nothing described here worked for me. It wasn't until I double-checked the build output and noticed that the pertinent projects weren't being built. A visit to configuration manager confirmed my suspicions.
Visual Studio 2015 had happily allowed me to add new projects but decided that it wasn't worth building them. Once I added the projects to the build it started playing nicely.
I resolved it by changing X64 to :
Right click on project -> Properties -> Build -> Platform target -> Any CPU
It was so easy for me to fix the issue as:
Select your Unit Test Project
Click on 'Show All Files' button in the Solution Explorer and new temporary files appeared in the file tree of the Solution Explorer within 'obj\x86\Debug'.
Delete these temporary files and rebuild project.
Retried to run tests and worked!.
We had the same problem. We have a big VS 2015 solution with multiple C# projects in it and even more test projects.
Resharper's test discovery worked just fine, but VS Test Explorer failed miserably.
Turns out that the projects didn't have the same version of MsTest TestFramework and TestAdapter, and that sometimes they used NuGets and other times good old references, and that is not supported apparently (so much for such an expensive IDE).
Removing all Microsoft.VisualStudio.Test* references and then adding / updating the two MSTest NuGets fixed the problem.
I solved this problem by realizing that the Target Framework for my test project was different than the project under test. Yes, I caused this problem by changing the target framework from the default (Project>Properties>Application), but failed to this this for the test project, which was created several weeks later. The mismatch did not cause a compiler error, but it did result in a warning in the Error List window. Once I selected the option to display warnings, the solution was obvious.
Another day, another problem with VS2012 and unit tests: Our solution has three test projects, together containing 335 different unit tests. All of these tests successfully run locally on different machines, but when checked in to our TFS 2012 build server, each time only 235 tests are being discovered.
There are 15 different test classes containing the unit tests, some of them using the new Fakes framework, others MOQ, some none or both. All of the classes reside in test projects (assemblies) ending in Test, the classes themselves are all ending in Test.
Some classes are not being discovered at all during the test run as part of the build process, in other classes e.g. 8 out of 12 tests are discovered and run.
Taking the "8 out of 12" example, the [ProjectNameTest].dll that's being generated shows all 12 tests using .NET Reflector 7.0, it's just so that some of the tests aren't run. I also cannot find any similarities (e.g. tests using Fakes / MOQ / ...) among the tests not run / discovered.
The build settings being used define the Test Sources Spec as **\*.dll (it used to be **\*test*.dll, but I changed this just to make sure that all files are taken into account).
For the Run Settings I initially started of with no Run Settings File and CodeCoverageEnabled. As part of trying to solve the problems I also changed this to a user defined UnitTest.runsettings file, unfortunately with the same result.
Does anybody know why there are all these problems? What might be the reasons? VS2010 and Moles worked a lot better, VS2012 has been causing a lot of trouble recently.
Update
Months later we finally updated the TFS installation to Update 3, but this made things even worse. Unit tests on the build server don't run anymore at all. Enabling dumps and analysing them showed that a StackOverflowException is thrown, but there is no indicator about the test that might be causing it. Again, locally on different developer systems the tests run without any problems.
We are considering opening a call at Microsoft as we've spent far too much time getting the whole thing to work (again). Moles were a whole lot better... I will keep this updated as soon as there is some kind of progress.
Update 2
Another few weeks have passed, and no real solution yet. As we have moved from VS2012 to VS2013 (Premium each time) now, we discovered that all tests are now run as part of the build process (again). Unfortunately, we still have problems getting all tests to pass - locally, everything runs fine on multiple systems, on the build server about ten percent of test fail. Investigation to be continued...
Usually, when tests pass locally but not on the build server, you probably need to include your assembly dependencies or if your tests are using databases you are connecting to the database properly.
Finally an answer was found - the problems with test failing randomly on the TFS build server were caused by using the Microsoft Fakes Framwork. On our local systems, everything worked as expected - we had Fakes defined in our different unit test projects (generally one test project for each "normal" project).
On the build server all the fakes were generated in the same directory, therefore depending on the build order of the projects (which in parts may be different for each build), the fakes generated first were overwritten by the fakes from the following project etc.
Instead of having the fake definitions as part of every project in our solution, we changed it so that we have a solution wide definition of fakes (e.g. System.Web.Fakes.dll) that we link to from each test project where we need it (Add existing item => Add as link). This solved the problems with running the unit tests on the build server, all tests pass again.
I am just starting to work with QuantLib project. There are quite a few BOOST unit tests defined, hence the question: is there an easy way to run these tests selectively from Visual Studio (and see results)?
I've only used unit testing in .Net languages so far, and there you can either use the built-in test runner if you use MsTest, or some add-on (I used Resharper and TestDriven.NET) for other frameworks. There you can choose which tests you run, and you can run them with the debugger attached (which is unfortunately useful sometimes). Are there similar add-ons for BOOST tests in unmanaged C++?
Yes you can!
If your boost unit test project is defined correctly, then you can run the entire test project from VS10 debugger by clicking
right click on project --> Debug --> Start new instance
This will run the tests inside VS framework which will give you all the debugger goodies.
For running specific test suites/cases, just edit the project command line arguments under
right click on project --> Properties --> General --> Debugging
and add the parameters according to the links in the other answers
you can run any test case or test suite via command line. running tests by name
it even allows wildcards.
ReSharper C++ supports discovery and selective running/debugging of Boost tests (with Boost 1.59 and later).
I'm using the Boost Unit Test Adapter. It's free and has a very nice output. It can also list the time the single tests consumed and you can start single tests and group of tests.
You can download it here:
https://visualstudiogallery.msdn.microsoft.com/5f4ae1bd-b769-410e-8238-fb30beda987f?SRC=VSIDE
In our solution we have several test projects. Most of them contain unit tests that run fast but some are for integration, acceptance and UI tests which take quite some time to finish.
Now I'd like to have a convenient way of running only the unit tests. Either by using the hotkey to run all or using the new feature to run tests after each build.
Unfortunately VS also runs all the slow tests as well. In VS10 I grouped the tests in Test View by project and selected only the ones containing unit tests. It's not as easy as I wished it to be but at least it was possible. Now there doesn't seem to be a way around running all tests in a solution.
The only alternative that I see at the moment is to remove all but the unit tests from the solution but this would limit the refactoring capabilities.
So does anyone know how to easily group and run only unit tests in the new VS?
In Visual Studio 2012 Update 1 CTP, Test Explorer has added the option to group tests by Project or Traits (Categories).
Please go to this link to see what's new there.
You can use the search filter in Unit Text Explorer.
In the new Test Exlorer shipped with 2012, you can group tests by 'Test Results', 'FileName',... etc but in RTM build group/search by 'TestCategory' is not present from Test Explorer. However there is a filter in commandline and TeamBuild based on TestCategory (Running selective unit tests in VS 2012 RC using TestCaseFilter) in VS2012.
From a member of VS dev Team, they understood the importance of this missing feature. It is in the backlog, and it should be available in furutre builds/updates.
The following blog provides more details: http://blogs.msdn.com/b/vikramagrawal/archive/2012/07/23/running-selective-unit-tests-in-vs-2012-rc-using-testcasefilter.aspx
As of Visual Studio 2012 Update 2 (currently on CTP) now we are able to run a subset of unit test using "Playlists" or based on "Lists & Classes". More information here: http://blogs.msdn.com/b/visualstudioalm/archive/2013/01/30/first-ctp-for-visual-studio-update-2.aspx#playlists
Original Answer:
AFAIK, The new unit testing framework in VS2012 "supports selective execution based on filtering condition through TestCaseFilter". Unfortunately, currently there is not support to do this through the user interface but it is in the product group's backlog. You can use this approach through Test Build or command line. Please refer to the following blog: http://blogs.msdn.com/b/vikramagrawal/archive/2012/07/23/running-selective-unit-tests-in-vs-2012-rc-using-testcasefilter.aspx
EDIT: Following the MSDN documentation to use test categories to group tests.
http://msdn.microsoft.com/en-us/library/dd286595.aspx
Also, refer to this question: How to exclude certain tests in the Visual Studio 2012 Test Runner?
It shows an example of filtering based on the test path.
I'd recommend using TestDriven.Net. Group all Unit Testing projects in one solution folder, and all other tests in another. Then just right click the "Unit Tests" folder and choose "Run All Tests". I always favor this great test runner of the VS native testing tools.
Another approach is using NCrunch - It runs all you tests in the background while you code. It handles integration tests pretty well, as they run in parallel, in the background so it doesn't block you (you also choose which test projects are run). One catch though - I believe it doesn't support VS2012 yet, but probably soon.
I was also missing this feature... (We did the same as you did: grouping by project or sometimes by namespace). But I found that I could somehow workaround this using the test explorer search capabilities:
https://msmvps.com/blogs/deborahk/archive/2012/09/16/searching-and-grouping-in-vs-2012-test-explorer.aspx
In my case I could identify the specific tests I needed to run using the search filter on FullName (use dropdown on search box, or type FullName:"UnitTest")
Its not the same, but it works for me