I'm considering running unit tests for my Visual Studio 2010 projects on our build server at build time. The problem is that when I'm working locally, I want to test against DEV, when building for QA, I want the tests to run against QA, when building/promoting for UAT/PROD... you get the picture.
I think VS 2010 might have support for per-environment configs. If so, does this apply for test projects also? If not, what are some alternatives?
thanks,
Mark
I don't really have much experience with VS 2010, but there was no such functionality in VS 2008.
I usually have a msbuild script to build and run the tests of the solution. In your case I would have a step in the this script to set the correct configuration after the code was built and before the tests are run. I used XmlUpdate task from http://msbuildtasks.tigris.org/ last time.
Related
Friends,
I have been looking at unit test code coverage tools for the last 2 weeks, evaluating them for our specific needs. The last one I would like to evaluate is Visual Studio's own code coverage for native C++ unit tests. In our solution, we have the following:
full Visual Studio 2013 development environment with large projects
of multi-100 k lines of code
900+ unit tests already written in GTEST (working and executing in
the build process) and working in this environment native C++ code
(unmanaged)
a dozen projects in the solution or so
What I am trying to do is figure out
If I can use the Visual Studio code coverage Analysis tool to report results on my gtest unit tests
get code coverage results generated each time the unit tests or run (our unit tests are automatically run in a post-build step)
I have looked around these forums, MSDN and other sites (like stackoverflow) and I can't find a good source of guidance to making this combination of tools work. As I say, I have the solution, projects and unit tests running and working well, but to add code coverage I am not seeing how to do so using Microsoft's own Visual Studio code coverage.
I have already evaluated a few other packages like Software Verify, Semantic Designs, OpenCPPCoverage and Bullseye. I was hoping to try our Visual Studio's code coverage component since it is already part of our dev environment, but I can't seem to figure out how or if that will work.
Any help?
Thanks for any feedback, info you can give.
After searching more, I found that what I was missing was an adapter to run the tests in VS 2013. While my tests were executing as part of the build process, they weren't generating results in the Test Explorer because it wasn't being interfaced with Visual Studio's test explorer. This can be done simply by installing an extension. The extension I found that works well for us is GoogleTest Runner, which is found in the extensions library here:
GoogleTest Runner
I am leaving this question and answer here in case others have this same issue and need some help
I have used Microsoft shims in my project for writing unit tests. These tests run fine on my local machine, however when build on server the tests fail as the ms test runner does not support shims. After searching alot i found that i have to specify vs test runner to run test but it can be specified only in default template. Is there any way to specify the test runner in build script file.
I am using TFS 2013 on TFS server with upgrade template.
The Upgrade template was provided only for backward compatibility for folks that has serious customization investment in 2005/2008 builds. I would not expect new functionality to work in legacy builds without serious rework and effort.
I would suggest that there will likely be less effort in moving your build to the modern build templates and gain access to new features effortlessly.
I have a project which builds and runs tests in Visual Studio Online. I recently added SpecFlow the the project and wrote a bunch of scenarios, some of which run against a LocalDb database. I was half expecting the tests to fail on VSO, but when I pushed the code up it ran all of my tests and they all passed. When I checked the logs it seems that my Specflow tests were not run.
Is there something I have to do to get the specflow custom tool which generates the actual unit tests to run as part of the build, so that VSO will find the tests after it has built?
Can I run what are effectively integration tests in VSO, creating and destroying databases in my tests?
Ok so this was a bit of a D'oh moment. VSOnline wasn't running my tests because Specflow was building NUnit test. Changing it to build MSTest tests meant that the generated files containing the tests had the correct attributes and VSOnline recognized them and so happily built and ran them.
And using the LocalDb during the tests was fine as well, all worked like a dream.
I'm having a problem when a project is committed to SVN, which in turn is automatically picked up by our newly set-up CruiseControl.NET server, the UnitTests are failing.
The unit tests are written in the default Visual Studio Unit test framework - which is the problem. I spot that nUnit appears to be recommended as a good alternative. However, that doesn't change my current predicament - I would like a working build without having to re-write every single one of my Unit Tests in the nUnit Framework (Would converting a MS Unit Test project take any considerable amount of time - it looks the same to me?)
What is the best way for me to proceed now? We are unable to install Visual Studio on the box and our IT department won't allow us access to go fiddling in the registry.
I am not sure how to run unit tests without Visual Studio, but I had moved to NUnit from MSTests and its reasonably easy - as long as you don't use the test context from MSTest(can't remember the real name), all you have to do is to rename the tags [TestClass] -> [TestFixture], etc (there is always 1-1 mapping) and change the usings. Running NUnit tests on the build machine is trivial.
Also you can try to convince your company to buy you a copy of Resharper (preferred), or TestDriven.NET - so you can easily run NUnit tests from Visual Studio.
I have developed a build system on MSBuild (NET 3.0) and cc.net to perform continuous integration builds of a Visual Studio 2008 application, however a developer on the team recently added a VSTS unit test project to the mix. Is there any SDK or add-in provided by microsoft to allow this to be compiled on a non-team system build environment?
You can run the tests from the command-line, but it requires that the Test Edition of Visual Studio be installed on your build server:
http://devlicio.us/blogs/derik_whittaker/archive/2008/09/25/clean-build-server-with-mstest-fail.aspx
http://aspadvice.com/blogs/ssmith/archive/2008/03/18/Continuous-Integration-Setup-with-MSTest.aspx
This person is doing MSTest with CC.NET, so it might be helpful:
http://testearly.findtechblogs.com/default.asp?item=630602
In general, this is a main reason why many people choose something like NUnit, xUnit.NET or MbUnit over MSTest.
If you use MSTEST You will have to put that edition on the build server. To cut a long story short youd be better off changing the testing framework the developer is using from MStest to Nunit.
Its a much lighter solution.