We're using JIRA with Bamboo as build server for continuous integration.
But I have a native C++ project (using Visual Studio 2010), and apparently Microsoft doesn't support unit tests for native C++ code. There's some tools like WinUnit or cfix that seem to do the job, but neither do I know about their compatibility to Bamboo, nor do I know which tool is the best / easiest to use / has the most features / has best VS compatibility ...
Does someone have experience with that?
Just as a heads up, my current project is using c++ with bamboo for CI. We use googletest for our testing framework. If you have the build run with --gtest_output=xml:{file or Directory} it will produce xml output that can parsed by Bamboo's JUnit Parse task.
I played a bit with CppUnit. The way it works is that you create an executable project which you fill with your test cases. When run, it runs your tests. I think cfix does that but I've never used it for user mode testing. To run the tests as part of your build process, put in a post build step for your test project that runs the test exe. It'll run when you build your Visual Studio solution.
Related
We have a C++ Linux project in Visual Studio that we build and run remotely on a Docker container. I am beginning to write unit tests for the project using Google Test (not tied to this framework) which I also want to run remotely. Unfortunately the tests are run locally which doesn't suit what I am trying to achieve. Does anyone know if this can be done with any test framework for C++ and if so could you point me to relevant docs etc?
I know I can create another standard Linux project containing the tests which I can run remotely but then I lose the Test Explorer functionality built-in to VS which I would prefer to be able to use.
Any help here would be greatly appreciated.
I am just starting to work with QuantLib project. There are quite a few BOOST unit tests defined, hence the question: is there an easy way to run these tests selectively from Visual Studio (and see results)?
I've only used unit testing in .Net languages so far, and there you can either use the built-in test runner if you use MsTest, or some add-on (I used Resharper and TestDriven.NET) for other frameworks. There you can choose which tests you run, and you can run them with the debugger attached (which is unfortunately useful sometimes). Are there similar add-ons for BOOST tests in unmanaged C++?
Yes you can!
If your boost unit test project is defined correctly, then you can run the entire test project from VS10 debugger by clicking
right click on project --> Debug --> Start new instance
This will run the tests inside VS framework which will give you all the debugger goodies.
For running specific test suites/cases, just edit the project command line arguments under
right click on project --> Properties --> General --> Debugging
and add the parameters according to the links in the other answers
you can run any test case or test suite via command line. running tests by name
it even allows wildcards.
ReSharper C++ supports discovery and selective running/debugging of Boost tests (with Boost 1.59 and later).
I'm using the Boost Unit Test Adapter. It's free and has a very nice output. It can also list the time the single tests consumed and you can start single tests and group of tests.
You can download it here:
https://visualstudiogallery.msdn.microsoft.com/5f4ae1bd-b769-410e-8238-fb30beda987f?SRC=VSIDE
I have 80+ VS2010 solutions, each contains some unit test projects.
All solutions are merged into one big solution before build process.
All tests (or some test subset) are executed after the successful build.
I want to enable test code coverage for all tests to get exact code coverage for all output assemblies.
My question is: how to enable code coverage without using Visual Studio?
Note: I'm using TFS2010 (MSBuild) to build merged solution. VS 2010 Premium is installed on the build server. MSTest is used for test execution.
You can use JetBrain's TeamCity Professional. It is a CI server that supports executing unit tests and calculating code coverage. It is free for small installations.
I think you need to consider deploying a code coverage tool, see here for a comparison (provided you implement .net).We use NCover, which integrated in our TFS-Build in it's console variant and, although it's not trivial to set it up, we 're very satisfied with it.In this post I had briefly described how we inserted NCoverin our build, this might we useful to you even if you go with another tool.
If you create a Vsmdi file in your large solution (ms test will usually do this for you) you can use this to tell the build which assemblies you want to instrument.
This will only provide code coverage for assemblies that have tests run against them. If you're using testrun.config files to decide which tests you want to run, this should be all you need. The code coverage results should then be published to the build drop location
Edit:
This blog post looks like it covers setting up code coverage
I'm trying to work out how to run xUnit tests from TFS 2010.
I found some articles on how to achieve that with the older version (http://jonnekats.wordpress.com/2009/05/07/integrate-xunit-tests-into-your-daily-team-build/, http://weblogs.asp.net/mehfuzh/archive/2009/08/25/configuring-team-build-using-xunit.aspx).
These approaches don't work anymore with TFS 2010 because builds are no longer MSBuild files but instead are Workflows with different tasks.
What I would like to achieve is something similar to what I've got on my dev machine:
- build everything
- run the .xunit project file
- check the results
All that I can see on the web are custom build steps which I can't/won't use because I'll have to configure them for each single unit test assembly and they'll probably get completely messed up in TFS.
Any input would be appreciated.
Why dont use "InvokeProcess" in TFS Build workflow and invoke xUnit command line interface with the necessary xUnit project files as parameters?
You can capture the output of the command line by setting InvokeProcess->stdout to write a build message.
Take a look at the xUnit work item Support VSTS Test Runner on codeplex. You can find there a VS2010 extension in Beta version.
From reasons I won't get into, all our unit tests are using the VSTS test framework. I now want to create an MSBuild script that runs the tests, but I don't want to use mstest.exe from various reasons (it's slower, requires Visual Studio installation everywhere, I need to maintain testrunconfig, etc.)
I've seen that TestDriven.net and TeamCity are able to run VSTS tests 'NUnit style', without using mstest.exe. Are you aware of any standalone command line utility that does this?
You can execute Team System Tests (MSTest) in NUnit if you use a special NUnit Addin that recognizes the MS Test Attributes (TestClass, etc).
Exact Magic Software has an open-source "test-adapter" that can do this.
UPDATE: I've reworked Exact Magic's Msts NUnit Adapter for NUnit 2.5.2.
Download here: http://snippetware.googlecode.com/files/ExactMagic.MstsAdapter.zip
Read more about it here: http://www.bryancook.net/2009/10/mstest-nunit-adapter.html
It seems like TeamCity is simply leveraging Gallio to run VS tests. Gallio appears to have msbuild integration and sounds perfect but after a closer look it seems that it would require a VS install just like MSTest as it appears to depend on MS exes:
The plugin enable condition was not satisfied: '${process:DEVENV.EXE} or
${process:VSTESTHOST.EXE} or
${process:QTAGENT.EXE} or
${process:QTAGENT32.EXE} or
${process:QTDCAGENT.EXE} or
${process:QTDCAGENT32.EXE}'.
Host process exited with code: 0
That being said it sounds like at least one person has got it working:
Christoph De Baene - Running MSTest without Visual Studio
It is possible to run MSTests without installing Visual Studio. See how-do-i-use-mstest-without-visual-studio.
I did this so that I could run my tests as part of my CI process. (I am using CC.NET for my CI solution).
I am in a similar situation as you, in that I want to use TestDriven.NET to get code coverage stats. But, I am running into problems. My first problem is that I am using AssemblyInitialize attributes to initialize a database connection. This isn't supported by NUnit so about half of my tests fail whereas they run fine under MSTest.
So, it seems that translating tests from one test framework to another has pitfalls. If you are aware of that, then go forth, but it might be better to try and keep consistent on one test framework.
We run VSTS tests using msbuild TestToolsTask on a Cruise Control server. This does not use the MSTEST executable -- the condition you ask for -- but does use a variety of TFS dependencies.
Note that we are migrating tests off of the VSTS test framework for NUnit, mostly because we can create extensions for NUnit to perform useful tasks.