I have a project which builds and runs tests in Visual Studio Online. I recently added SpecFlow the the project and wrote a bunch of scenarios, some of which run against a LocalDb database. I was half expecting the tests to fail on VSO, but when I pushed the code up it ran all of my tests and they all passed. When I checked the logs it seems that my Specflow tests were not run.
Is there something I have to do to get the specflow custom tool which generates the actual unit tests to run as part of the build, so that VSO will find the tests after it has built?
Can I run what are effectively integration tests in VSO, creating and destroying databases in my tests?
Ok so this was a bit of a D'oh moment. VSOnline wasn't running my tests because Specflow was building NUnit test. Changing it to build MSTest tests meant that the generated files containing the tests had the correct attributes and VSOnline recognized them and so happily built and ran them.
And using the LocalDb during the tests was fine as well, all worked like a dream.
Related
I use in my project SpecFlow to provide automation testing of web-site(with Selenium). I want create end-to-end test suite for provide full checking of project according to BDD. When I create tests playlist in test explorer in Visual Studio and start all tests in playlist all my tests launch according to their alphabetic name. But when I restart project and start this playlist newly all my tests launch in the wrong order. I think, that this problem may be because of rebuild playlist file, but I'm not sure. Could You help me, please?
TestRunner - MSTest
My code has NUnit tests. My colleague is able to run the tests from within Visual Studio on their desktop, however I can't run them on mine. When I try to use Test Explorer and "Run All", it just builds the project. Other options, such as Run Tests and Debug Tests also give the same result.
I have NUnit referenced in my project, why can't I run the tests from within Visual Studio?
Out of the box, Visual Studio doesn't have support for running tests built for the NUnit framework. As has been said in the the comments, your colleague probably has some kind of add-in like Resharper installed which adds in support for running Nunit tests.
If you don't have a license for whatever tool they're using, then you can still have visual studio integrated support for running/debugging NUnit tests by installing the NUnit Test Adapter. This is a free plugin that you can install into your solution via NuGet.
He may also be running the MS test adapter which you can get from the nunit site
I'm using TFS, it works fine. Now I want to add tests and code coverage. So I create a Local.testsettings. Running tests on my computer runs fine.
Configuration of the TFS Build is as follows:
- Automated test with test assembly file specification and the same test settings file (as recommended, instead of using a .vsmdi file)
When launching a build with TFS, my tests don't get executed :
Run MSTest for Test Assemblies
The MSTestActivity was invoked without a value for Platform or Flavor. The values Mixed Platforms and Debug were used.
C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe /nologo /usestderr /testSettings:"C:\Builds\1\MyProject\Sources\MyProject\Main\Source\MyProject\Local.testsettings" /searchpathroot:"C:\Builds\1\MyProject\Binaries" /resultsfileroot:"C:\Builds\1\MyProject\TestResults" /testcontainer:"C:\Builds\1\MyProject\Binaries\MyProject.Tests.Module1.dll" /maxpriority:"1" /minpriority:"1" /publish:"http://tfsserver:8080/tfs/Test" /publishbuild:"vstfs:///Build/Build/433" /teamproject:"MyProject" /platform:"Mixed Platforms" /flavor:"Debug"
Loading C:\Builds\1\MyProject\Sources\MyProject\Main\Source\MyProject\Local.testsettings...
Loading C:\Builds\1\MyProject\Binaries\MyProject.Tests.Module1.dll...
Starting execution...
No tests to execute.
There are no results to be published.
So it finds the assembly, but MSTest doesn't actually runs the tests.
Any hint is welcome.
Looking at this:
/maxpriority:"1" /minpriority:"1"
It looks like your priority criteria may be the problem.
I'm considering running unit tests for my Visual Studio 2010 projects on our build server at build time. The problem is that when I'm working locally, I want to test against DEV, when building for QA, I want the tests to run against QA, when building/promoting for UAT/PROD... you get the picture.
I think VS 2010 might have support for per-environment configs. If so, does this apply for test projects also? If not, what are some alternatives?
thanks,
Mark
I don't really have much experience with VS 2010, but there was no such functionality in VS 2008.
I usually have a msbuild script to build and run the tests of the solution. In your case I would have a step in the this script to set the correct configuration after the code was built and before the tests are run. I used XmlUpdate task from http://msbuildtasks.tigris.org/ last time.
I'm having a problem when a project is committed to SVN, which in turn is automatically picked up by our newly set-up CruiseControl.NET server, the UnitTests are failing.
The unit tests are written in the default Visual Studio Unit test framework - which is the problem. I spot that nUnit appears to be recommended as a good alternative. However, that doesn't change my current predicament - I would like a working build without having to re-write every single one of my Unit Tests in the nUnit Framework (Would converting a MS Unit Test project take any considerable amount of time - it looks the same to me?)
What is the best way for me to proceed now? We are unable to install Visual Studio on the box and our IT department won't allow us access to go fiddling in the registry.
I am not sure how to run unit tests without Visual Studio, but I had moved to NUnit from MSTests and its reasonably easy - as long as you don't use the test context from MSTest(can't remember the real name), all you have to do is to rename the tags [TestClass] -> [TestFixture], etc (there is always 1-1 mapping) and change the usings. Running NUnit tests on the build machine is trivial.
Also you can try to convince your company to buy you a copy of Resharper (preferred), or TestDriven.NET - so you can easily run NUnit tests from Visual Studio.