I am currently working on a C# WPF application and using MSTest to handle our unit tests. The unit tests are created in a separate project that is attached to the main project. until today all tests have been running with no issue. I performed a commit push on the previous working day, however when I opened the project today, when I run the tests, there are no results appearing. I had a colleague open the push I performed the previous day, and they are able to detect the tests and their results so I assume there is an issue in my config somewhere, I just don't know where to look.
EDIT:
we have uninstalled MSTest.TestFramework and MSTest.TestAdapter 1.1.18 and updated to 1.4.0 and the tests are being detected again.
I'm not entirely sure what the source of the problem is... but I too have seen this behavior sometimes. Try: (1) Build => Clean Solution, (2) Close & re-open VS2017, (3) Test => Run => Run All Tests
Related
Using CUITe testing framework in VS2012, I created a coded UI test. This test works as expected when right clicking the test, and hitting "run test" in VS2012.
We are in the process of migrating to VS2015, so I'm trying to go through our solutions to make sure they are behaving as expected. In this solutions case, we are running into an issue where when attempting to run a test, the output pane states:
There is no test with the specified Id (guid listed)
And no CUITe tests are run.
I have confirmed that normal coded UI tests are running in VS2015, it is specifically coded UI tests created using CUITe that are failing to run.
Don't know if anyone will have any workarounds for this, but I figured I'd give it a shot. I have additionally added all of the "Microsoft Testing" dlls to the solution in a separate folder, in order to ensure the solution is using the "specific versions" of the testing DLLs. This did not solve the problem.
The issue was due to CUITe not having been compiled/provided a build definition for VS2015.
In the current GitHub repo of CUITe (code plex URL from the original question no longer the "active link") provides a new build that addresses the issue.
Current URL for CUITe is:
https://github.com/icnocop/cuite
Issue was resolved here.
We noticed that there was a unit testing project within the latest Script# 0.8 code on the Git repository, but there is no documentation as to how to actually use it, and the example files don't seem to work correctly.
Having stepped through the source code, we can't actually see any test code within the unit test files.
Has anyone else managed to successfully run unit tests against Script# within the Visual Studio IDE?
The AroundMe sample at https://github.com/nikhilk/scriptsharp/tree/cc/samples/AroundMe demonstrates unit testing.
In short, there are two parts -
1. The Tests subnamespace within your project containing tests alongside code being tested. Test code gets split out by the compiler to generate a Foo.test.js alongside Foo.js and Foo.debug.js.
A test project which provides a driver to launch unit tests in browser and capture results of QUnit to surface into VS unit testing UI. This is optional... you could run the Foo.test.js manually using QUnit yourself if you've got your own pipeline for executing tests.
Hope that helps.
I have a Windows 8 Store App with two projects within the solution - one project exclusively for tests. I have added NUnit and NUnit Test Adapter (https://www.nuget.org/packages/NUnitTestAdapter/1.0.0) for this through nuget.
My tests are detected - but I cannot run or debug them. Looking at the test output window, I get the following error:
Could not find test executor with URI 'executor://nunittestexecutor/'. Make sure that the test executor is installed and supports .net runtime version 4.0.30319.34003.
I am not using resharper and I am using Visual Studio Professional 2012 Update 3 on Windows 8.1. References for the test project are as follows:
Based on my comments above, below is a workaround. Unfortunately at this stage, Windows 8.1 store Unit Test project types, using NUnit extension wouldn't work due to the different .NET targets. I tried with different Test Unit Adapters including an NUnitTestAdapterWithFramework.
It seems that the issue you haveing was occurring with standard .NET libraries targeting NUnit test adapter but the above NUnitTestAdapterWithFramework must have fixed those issue. See the Q & A section of the NUnitTestExtension
But unfortunatly it seems that this still of an issue that hasn't been fixed for Win8 Store App type Unit Testing. Pretty sure xUnit.NET also not compatible yet with different .NET target types (i,e WinRT)
So what are the options?
a. For your group, you can change them to use MSTest framework. Outcome - Problem Solved no issues.
b. Workaround "linked project". Outcome - Can't *guarantee** but this should also work.
With option 'b'
In your comment you mentioned.
but I'm still not sure what it does or how to implement a 'linked
project', do you have any more information on this? Also, as this is
for a group university project, I was hoping i wouldn't have to force
too many workarounds
When you think about it, it is not really hard work around. It is simple and I'm sure your group would be able to apply this workaround easily.
Please follow the below steps.
Create a separate class library in your solution (you can target .NET framework 4).
Then add NUnit assemblies and the NUnit test adapter as usual.
Right click on this project and select 'Add' then 'Existing Item'
Select the Win8 Store Unit Test project and locate the Unit Test file you want to add. When you add the file, make sure you select 'Add As a Link' button. Please see below.
Now rebuild the solution, close and re-open the UnitTest explorer and you should be able to run those tests.
*The reason I said can't guaranteed. I haven't really written Unit tests against Win8 App. So if your SUT (System Under Test) require special configuration it might cause issues. But I'm not sure.
Finally creating a link files are not that hard if everything works you can continue to do this until NUnit has the support for Win8 Unit Testing. Or the other option is simply change all your Unit Tests to use MSTest framework if possible.
Another day, another problem with VS2012 and unit tests: Our solution has three test projects, together containing 335 different unit tests. All of these tests successfully run locally on different machines, but when checked in to our TFS 2012 build server, each time only 235 tests are being discovered.
There are 15 different test classes containing the unit tests, some of them using the new Fakes framework, others MOQ, some none or both. All of the classes reside in test projects (assemblies) ending in Test, the classes themselves are all ending in Test.
Some classes are not being discovered at all during the test run as part of the build process, in other classes e.g. 8 out of 12 tests are discovered and run.
Taking the "8 out of 12" example, the [ProjectNameTest].dll that's being generated shows all 12 tests using .NET Reflector 7.0, it's just so that some of the tests aren't run. I also cannot find any similarities (e.g. tests using Fakes / MOQ / ...) among the tests not run / discovered.
The build settings being used define the Test Sources Spec as **\*.dll (it used to be **\*test*.dll, but I changed this just to make sure that all files are taken into account).
For the Run Settings I initially started of with no Run Settings File and CodeCoverageEnabled. As part of trying to solve the problems I also changed this to a user defined UnitTest.runsettings file, unfortunately with the same result.
Does anybody know why there are all these problems? What might be the reasons? VS2010 and Moles worked a lot better, VS2012 has been causing a lot of trouble recently.
Update
Months later we finally updated the TFS installation to Update 3, but this made things even worse. Unit tests on the build server don't run anymore at all. Enabling dumps and analysing them showed that a StackOverflowException is thrown, but there is no indicator about the test that might be causing it. Again, locally on different developer systems the tests run without any problems.
We are considering opening a call at Microsoft as we've spent far too much time getting the whole thing to work (again). Moles were a whole lot better... I will keep this updated as soon as there is some kind of progress.
Update 2
Another few weeks have passed, and no real solution yet. As we have moved from VS2012 to VS2013 (Premium each time) now, we discovered that all tests are now run as part of the build process (again). Unfortunately, we still have problems getting all tests to pass - locally, everything runs fine on multiple systems, on the build server about ten percent of test fail. Investigation to be continued...
Usually, when tests pass locally but not on the build server, you probably need to include your assembly dependencies or if your tests are using databases you are connecting to the database properly.
Finally an answer was found - the problems with test failing randomly on the TFS build server were caused by using the Microsoft Fakes Framwork. On our local systems, everything worked as expected - we had Fakes defined in our different unit test projects (generally one test project for each "normal" project).
On the build server all the fakes were generated in the same directory, therefore depending on the build order of the projects (which in parts may be different for each build), the fakes generated first were overwritten by the fakes from the following project etc.
Instead of having the fake definitions as part of every project in our solution, we changed it so that we have a solution wide definition of fakes (e.g. System.Web.Fakes.dll) that we link to from each test project where we need it (Add existing item => Add as link). This solved the problems with running the unit tests on the build server, all tests pass again.
I'm trying to get my continuous integration setup to run my unit tests for me, so I've set
<RunTest>true</RunTest>
And I've setup the tests to auto-discover:
<ItemGroup>
<TestContainer Include="$(OutDir)%2a.Test.dll" />
</ItemGroup>
Running the build on my development machine runs my unit tests, and reports that the Build succeeded; When run as an automated build, the tests run (according to the BuildLog), but I'm seeing this in the log:
Publishing results of test run tfs_service#TFS-BUILD 2012-04-13 14:13:59_Any CPU_Release to http://mycompany-tfs:8080/Build/v1.0/PublishTestResultsBuildService2.asmx...
.The drop share directory "\tfs-build\Builds\Test\MyProduct Test_20120413.2" could not be accessed.
The drop share directory "\tfs-build\Builds\Test\MyProduct Test_20120413.2" could not be accessed.
MSBUILD : warning MSB6006: "MSTest.exe" exited with code 1.
The previous error was converted to a warning because the task was called with ContinueOnError=true.
This is odd, as the binaries publish quite merrily.
I tried to diagnose the issue, by logging on to the console of the build server, running up a Visual Studio 2008 Command Prompt, navigating to the source location, and invoking MSBuild, it doesn't appear to even attempt to invoke the unit tests?
Ultimately, I'm more interested in playing "guess why the drop of the results failed" - could it be the embedded space in my build definition name? What else could I check, given the binaries publish OK?
The unit tests are running, the problem is that the results are not being published correctly. When unit tests are run, the results are published to the tfs server and then copied to the drop location. Because of this, the drop share needs to allow the TFS build service account to write to this folder.
http://blogs.msdn.com/b/vstsqualitytools/archive/2010/10/24/test-agent-test-controller-and-mstest-faq.aspx