Is there a way you can run your UnitTest project against a compiled DLL outside of the IDE? Basically we have testing procedures to validate code before moving into production.
I would not want to run the tests inside the IDE. I would want to have the compiled code ready to move to production and be able to run a final test against the .dll before the final copy.
Is there some kind of Command-Line utility that could do this? Just supply both .dlls and get a "all good" report of some kind.
I think you're probably looking for [MSTest](http://msdn.microsoft.com/en-us/library/ms182486(VS.80).aspx)
To run all tests in a specific container, you could do something like the following at a command line:
MSTest /testcontainer:MyTestProject\bin\debug\MyTestProject.dll
[MSTest Command Line Options for .NET 2.0](http://msdn.microsoft.com/en-us/library/ms182489(VS.80).aspx)
MSTest Command Line Options for .NET 3.5
Other Stack Overflow related MSTest Questions
Related
Currently how I run my unit tests:
I have a visual studio project, which builds a C++ google test (unit test) executable.
I then copy this exe to my Virtual Machine with QNX and run it manually, to get the unit test results.
Can I use resharper to automate this?
Resharper is able to detect and list tests from test file. But on running them, it throws error as it is not able to execute the tests in Windows.
Below is the exact error, I get, on running any of the tests.
ReSharper Ultimate – Error while running unit tests:
Invalid path
Can I set up resharper and Visual studio environment, so that, I can run tests directly from visual studio, which will copy the exe to the Virtual Machine and give me the execution results?
This will avoid me manually copy pasting the exe and running command to execute the tests?
I searched a lot regarding this in google. But couldn't find much help.
I'm new to google test and resharper. So any help would be greatly appreciated.
Thanks in advance.
No, I'm afraid ReSharper cannot do that out of the box.
Though it is quite powerful, so if you manage to write a script that does the VM specific part, ReSharper should be able to cope with it. What that script needs to do is:
It must be run on the host (Windows).
Copy the test binary to the QNX system in the VM.
Execute the test.
Capture the test's output and redirect it to the host's stdout.
The ReSharper options can be set here:
Menu ReSharper > Options...
In the left pane scroll down to Tools > Unit Testing > C++ Tests
In the text field Command: you need to enter the path to the script.
See https://www.jetbrains.com/help/resharper/Reference_Options_Tools_Unit_Testing_CPP_Tests.html for more details.
I think you'll find that the msg box showing "ReSharper Ultimate – Error while running unit tests: Invalid path" has been caused by the last update to VS 2017, which has broken things. I usually run my google tests through resharper c++, and until a few days ago, it worked. Now, I see this exact same error.
I created a test framework to test a sample application. This basically required creating a Solution for my sample application and then adding a Native Test Project, which can test the application.
The test project generates an .exe, but if I were to execute the .exe on it's own, it does nothing.
Is there a way to get information from the test programme so it can be used for other purposes outside of the VS environment?
Yes, you can run unit tests without VS2013.
Use Run automated tests from the command line using MSTest source to do so.
In general it says:
Open a Visual Studio command prompt.
Either change directory to your solution folder or, when you run the MSTest.exe program in step 3, specify a full or relative path to the metadata file or to the test container.
Run the MSTest.exe program.
I hope it will solve your issue.
It took a couple of days, but I've found that DeJaVo's answer is incorrect. Because I am running a Native Test Project, MSTest will not work with it. To run a Native Test from the command line, use VSTest.Console.
I am just starting to work with QuantLib project. There are quite a few BOOST unit tests defined, hence the question: is there an easy way to run these tests selectively from Visual Studio (and see results)?
I've only used unit testing in .Net languages so far, and there you can either use the built-in test runner if you use MsTest, or some add-on (I used Resharper and TestDriven.NET) for other frameworks. There you can choose which tests you run, and you can run them with the debugger attached (which is unfortunately useful sometimes). Are there similar add-ons for BOOST tests in unmanaged C++?
Yes you can!
If your boost unit test project is defined correctly, then you can run the entire test project from VS10 debugger by clicking
right click on project --> Debug --> Start new instance
This will run the tests inside VS framework which will give you all the debugger goodies.
For running specific test suites/cases, just edit the project command line arguments under
right click on project --> Properties --> General --> Debugging
and add the parameters according to the links in the other answers
you can run any test case or test suite via command line. running tests by name
it even allows wildcards.
ReSharper C++ supports discovery and selective running/debugging of Boost tests (with Boost 1.59 and later).
I'm using the Boost Unit Test Adapter. It's free and has a very nice output. It can also list the time the single tests consumed and you can start single tests and group of tests.
You can download it here:
https://visualstudiogallery.msdn.microsoft.com/5f4ae1bd-b769-410e-8238-fb30beda987f?SRC=VSIDE
Since running Metro apps headlessly is still a gray area: Running a metro app headlessly, I've recently decided to add a native unit test project to my Windows Metro app in hopes that I can find a way to run these unit tests in an automated fashion on the build server. Basically, I'm looking for something similar to MSTest.exe - a utility which is great for running tests from batch files and/or scripts.
In fact, I've tried using the new version of MSTest.exe that comes with VS11 on a generated test .dll, but it fails with the error:
"Unable to load the test container 'test.dll' or one of its dependencies... Error details: Could not load file or assembly file://test.dll' or one of its dependencies. The Module was expected to contain an assembly manifest."
Does MSTest.exe work with test containers that contain WinRT code? If not, is there a utility that will do what I want?
Edit: I just found out that MSTest does not support running tests on a Metro style app. Found here: http://msdn.microsoft.com/en-us/library/ms253138%28v=vs.110%29.aspx This really is too bad. I'm still hoping there's a utility out there that will work.
After blindly digging through the VS folders, I happened to find a new test runner under:
C:\Program Files\Microsoft Visual Studio
11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe
This utility allows you to execute WinRT unit tests from the command line. It's very similar to MSTest.exe.
There doesn't seem to be any documentation out there for this yet, but at least a help command exists.
If you are executing vstest.console.exe <filename>.dll then your tests do not get executed in appcontainer mode. You need to give <filename>.appx to execute tests in appcontainer mode. More info on how to execute tests for Windows Metro style apps from command line can be found at Running Unit Tests for Windows Metro style apps from Command Line
There is a documentation about (among other things) running unit tests for Windows Phone:
http://msdn.microsoft.com/en-us/library/windowsphone/develop/dn168930(v=vs.105).aspx
It describes also the command line way of doing it using vstest.console.exe.
It also gives a comparison of supported features between unit tests for Windows Phone and WinRT.
One important detail is that unit tests for WinRT cannot be run on a device. This is a pity and relevant to question How to automate non-interactive tests on Microsoft Surface
I have 80+ VS2010 solutions, each contains some unit test projects.
All solutions are merged into one big solution before build process.
All tests (or some test subset) are executed after the successful build.
I want to enable test code coverage for all tests to get exact code coverage for all output assemblies.
My question is: how to enable code coverage without using Visual Studio?
Note: I'm using TFS2010 (MSBuild) to build merged solution. VS 2010 Premium is installed on the build server. MSTest is used for test execution.
You can use JetBrain's TeamCity Professional. It is a CI server that supports executing unit tests and calculating code coverage. It is free for small installations.
I think you need to consider deploying a code coverage tool, see here for a comparison (provided you implement .net).We use NCover, which integrated in our TFS-Build in it's console variant and, although it's not trivial to set it up, we 're very satisfied with it.In this post I had briefly described how we inserted NCoverin our build, this might we useful to you even if you go with another tool.
If you create a Vsmdi file in your large solution (ms test will usually do this for you) you can use this to tell the build which assemblies you want to instrument.
This will only provide code coverage for assemblies that have tests run against them. If you're using testrun.config files to decide which tests you want to run, this should be all you need. The code coverage results should then be published to the build drop location
Edit:
This blog post looks like it covers setting up code coverage