I have used Microsoft shims in my project for writing unit tests. These tests run fine on my local machine, however when build on server the tests fail as the ms test runner does not support shims. After searching alot i found that i have to specify vs test runner to run test but it can be specified only in default template. Is there any way to specify the test runner in build script file.
I am using TFS 2013 on TFS server with upgrade template.
The Upgrade template was provided only for backward compatibility for folks that has serious customization investment in 2005/2008 builds. I would not expect new functionality to work in legacy builds without serious rework and effort.
I would suggest that there will likely be less effort in moving your build to the modern build templates and gain access to new features effortlessly.
Related
We have Team Foundation Server 2013 installed with Visual Studio Professional 2015 and I've been able to get continuous integration builds working using the TfvcTemplate.12.xaml build process template and the "/tv:14.0" MSBuild argument. However, I can't seem to find a way to tell TFS to use the 2015 Unit Test suite.
This post Build VS 2015 Solution w/ Unit Tests on TFS 2013 Team Build worked around the problem by installing VS 2013 on the build server, but that means that the build server would use the 2013 testing framework rather than the version used by the developer.
Another option is to use an older build process template that uses MSTest, which supports the "ToolPath" parameter, but MSTest appears to be deprecated (MSTest V2). BTW, I tried using the MSTest V2 NuGet packages, but that didn't have any effect on the framework used by TFS, which made sense once I thought about it.
I assume that there should be some way to tie in a third-party testing framework, but the examples I've found use the older build process template.
Has anyone else found a solution for this?
If you need your test agent or test controller to work with TFS 2013,
use Agents for Microsoft Visual Studio 2013 Update 5 and
configure the test controller.
Source Link: Install and configure test agents
This is the official recommended practices, we don't suggest you to use some third-party testing framework for handling your situation (not sure if there were).
Besides just as MrHinsh commented, we encourage you to move to new web-based build system which is more excellent. For the detail benefit of VNext builds, you could refer to this article Why You Should Switch to Build VNext
We would like to measure code coverage for our own automated regression test system run over a fairly large native app. This is a sophisticated, scripted test system using the inbuilt scripting of our app. It has thousands of tests and is not going to be replaced by MSTest unit tests.
Whilst we're using VS2012 (Premium) as the IDE currently it is still compiled with the VS2010 compilers & libraries. That could change sooner if it was a prerequisite to getting code coverage going.
We can do separate builds for this - instrumenting is not a problem.
I'm just confused reading the MS documentation which seems to all start from an assumption you're running unit tests using their inbuilt test framework. That's when I'm not struggling to find stuff which actually talks about native support for ALM in the first place!
thanks
Visual Studio 2012's code coverage tool is entirely separate from the test execution system (full disclosure: I wrote it, but the team that inherited it after I left Microsoft removed some fairly useful functionality). It was rewritten from the ground up in VS 2012 to dynamically instrument native (x86 and x86-64) and managed code (.NET and Silverlight) when it loads into the process instead of modifying executables on disk.
You can find CodeCoverage.exe in "%ProgramFiles%\Microsoft Visual Studio 11.0\Team Tools\Dynamic Code Coverage Tools".
To collect data:
CodeCoverage.exe collect /output:foo.coverage foo.exe foos_args
A configuration file (there's a default one in that directory called CodeCoverage.config) can be specified to control collection.
To analyze the coverage data, you can open foo.coverage in Visual Studio 2012 or use the coverage tool itself to do the analysis:
CodeCoverage.exe analyze /output:results.xml foo.coverage
Note: for instrumentation to take place, .pdb files must be discovered for your modules. Since you are building with 2010, they may not work with 2012's DIA so you may have to rebuild with 2012's toolset. If you are not seeing the modules you expect in the coverage analysis, pass /include_skipped_modules to the analyze command; there will be a "reason" attribute telling you why the module was skipped (excluded, no debug information, etc.).
Edit: Also, unlike previous versions of Visual Studio, 2012's coverage file format is completely self-contained. The modules and .pdbs don't need to be present at analysis time.
I realize this is an old post, but I believe the answer still is relevant.
With all the things that I used to have at my disposal in C#, I didn't really like what I saw when I moved to Visual C++. Also, like you the MSTests only partially worked for me; I'm used to have my own test applications as well.
Basically what I wanted was the following:
Run MS tests or an EXE file
Get code coverage right in Visual Studio.
After doing some research, I noticed that VS Enterprise supports this feature today with test adapters.
If you're not on VSE, I noticed there are some other tools, each providing users with an independent UI. Personally I don't like that; I want my coverage right in Visual Studio, preferably in Visual Studio Community edition.
So I decided to build this addin myself and - while it's not as sophisticated as VSE - it does the trick for me.
I wrote a VSIX code coverage tool on https://github.com/atlaste/CPPCoverage . Basically it manages the highlighting in Visual Studio, generates a clickable report, and integrates in solution explorer.
For the coverage measurements themselves, I used to use https://opencppcoverage.codeplex.com/ . Basically that allows you to perform code coverage tests on any debuggable (native) executable. Nowadays, I'm using my own code coverage measuring tools (which are open sourced above as well).
I'm considering running unit tests for my Visual Studio 2010 projects on our build server at build time. The problem is that when I'm working locally, I want to test against DEV, when building for QA, I want the tests to run against QA, when building/promoting for UAT/PROD... you get the picture.
I think VS 2010 might have support for per-environment configs. If so, does this apply for test projects also? If not, what are some alternatives?
thanks,
Mark
I don't really have much experience with VS 2010, but there was no such functionality in VS 2008.
I usually have a msbuild script to build and run the tests of the solution. In your case I would have a step in the this script to set the correct configuration after the code was built and before the tests are run. I used XmlUpdate task from http://msbuildtasks.tigris.org/ last time.
I'm having a problem when a project is committed to SVN, which in turn is automatically picked up by our newly set-up CruiseControl.NET server, the UnitTests are failing.
The unit tests are written in the default Visual Studio Unit test framework - which is the problem. I spot that nUnit appears to be recommended as a good alternative. However, that doesn't change my current predicament - I would like a working build without having to re-write every single one of my Unit Tests in the nUnit Framework (Would converting a MS Unit Test project take any considerable amount of time - it looks the same to me?)
What is the best way for me to proceed now? We are unable to install Visual Studio on the box and our IT department won't allow us access to go fiddling in the registry.
I am not sure how to run unit tests without Visual Studio, but I had moved to NUnit from MSTests and its reasonably easy - as long as you don't use the test context from MSTest(can't remember the real name), all you have to do is to rename the tags [TestClass] -> [TestFixture], etc (there is always 1-1 mapping) and change the usings. Running NUnit tests on the build machine is trivial.
Also you can try to convince your company to buy you a copy of Resharper (preferred), or TestDriven.NET - so you can easily run NUnit tests from Visual Studio.
I have developed a build system on MSBuild (NET 3.0) and cc.net to perform continuous integration builds of a Visual Studio 2008 application, however a developer on the team recently added a VSTS unit test project to the mix. Is there any SDK or add-in provided by microsoft to allow this to be compiled on a non-team system build environment?
You can run the tests from the command-line, but it requires that the Test Edition of Visual Studio be installed on your build server:
http://devlicio.us/blogs/derik_whittaker/archive/2008/09/25/clean-build-server-with-mstest-fail.aspx
http://aspadvice.com/blogs/ssmith/archive/2008/03/18/Continuous-Integration-Setup-with-MSTest.aspx
This person is doing MSTest with CC.NET, so it might be helpful:
http://testearly.findtechblogs.com/default.asp?item=630602
In general, this is a main reason why many people choose something like NUnit, xUnit.NET or MbUnit over MSTest.
If you use MSTEST You will have to put that edition on the build server. To cut a long story short youd be better off changing the testing framework the developer is using from MStest to Nunit.
Its a much lighter solution.