I have test cases that are executable (*.exe files). There is no user interface involved.
How do I use Team center /visual studio online to run these test cases on server.
For now, either on demand running or scheduled running will work for me.
(Currently I have no test case that runs on server. So you may mention the basic setup. )
I have written some test cases (they are exe files). I can run them locally line any other exe file.
My code is in C++.
My test cases are in C++.
You could run them as part of your build. Just configure a build in VSO for your solution, and then modify the msbuild project file to call your tests and send the output to the build folder so it gets uploaded as part of the drop. If you are using VS, you would get a better experience using the VS unit testing support (i.e., get results in VS): http://www.visualstudio.com/get-started/run-tests-with-builds-vs.
Related
We have a C++ Linux project in Visual Studio that we build and run remotely on a Docker container. I am beginning to write unit tests for the project using Google Test (not tied to this framework) which I also want to run remotely. Unfortunately the tests are run locally which doesn't suit what I am trying to achieve. Does anyone know if this can be done with any test framework for C++ and if so could you point me to relevant docs etc?
I know I can create another standard Linux project containing the tests which I can run remotely but then I lose the Test Explorer functionality built-in to VS which I would prefer to be able to use.
Any help here would be greatly appreciated.
Currently how I run my unit tests:
I have a visual studio project, which builds a C++ google test (unit test) executable.
I then copy this exe to my Virtual Machine with QNX and run it manually, to get the unit test results.
Can I use resharper to automate this?
Resharper is able to detect and list tests from test file. But on running them, it throws error as it is not able to execute the tests in Windows.
Below is the exact error, I get, on running any of the tests.
ReSharper Ultimate – Error while running unit tests:
Invalid path
Can I set up resharper and Visual studio environment, so that, I can run tests directly from visual studio, which will copy the exe to the Virtual Machine and give me the execution results?
This will avoid me manually copy pasting the exe and running command to execute the tests?
I searched a lot regarding this in google. But couldn't find much help.
I'm new to google test and resharper. So any help would be greatly appreciated.
Thanks in advance.
No, I'm afraid ReSharper cannot do that out of the box.
Though it is quite powerful, so if you manage to write a script that does the VM specific part, ReSharper should be able to cope with it. What that script needs to do is:
It must be run on the host (Windows).
Copy the test binary to the QNX system in the VM.
Execute the test.
Capture the test's output and redirect it to the host's stdout.
The ReSharper options can be set here:
Menu ReSharper > Options...
In the left pane scroll down to Tools > Unit Testing > C++ Tests
In the text field Command: you need to enter the path to the script.
See https://www.jetbrains.com/help/resharper/Reference_Options_Tools_Unit_Testing_CPP_Tests.html for more details.
I think you'll find that the msg box showing "ReSharper Ultimate – Error while running unit tests: Invalid path" has been caused by the last update to VS 2017, which has broken things. I usually run my google tests through resharper c++, and until a few days ago, it worked. Now, I see this exact same error.
I created a test framework to test a sample application. This basically required creating a Solution for my sample application and then adding a Native Test Project, which can test the application.
The test project generates an .exe, but if I were to execute the .exe on it's own, it does nothing.
Is there a way to get information from the test programme so it can be used for other purposes outside of the VS environment?
Yes, you can run unit tests without VS2013.
Use Run automated tests from the command line using MSTest source to do so.
In general it says:
Open a Visual Studio command prompt.
Either change directory to your solution folder or, when you run the MSTest.exe program in step 3, specify a full or relative path to the metadata file or to the test container.
Run the MSTest.exe program.
I hope it will solve your issue.
It took a couple of days, but I've found that DeJaVo's answer is incorrect. Because I am running a Native Test Project, MSTest will not work with it. To run a Native Test from the command line, use VSTest.Console.
I'm not a Windows 8 developer so excuse any stupid questions.
I have a Windows 8.1 application built in Visual Studio 2013 via a .sln file. This contains a number of projects: there is an application that relies on a library, this library (call it datalib) is built as a Windows Runtime Component (not a DLL);
there is also a test project which runs tests on 'datalib'
All of this works fine in VS2013, i.e. the tests execute.
I've been tasked with creating a TeamCity build for this so I've create a TeamCity build that builds against the sln file but I can't get the tests to execute (they're MSTest tests).
In my configuration I've added the test.dll (in the 'include assembly files list') but when I add this on it's own then I get a bunch of errors around references to types in the 'datalib'
The 'datalib' is built into datalib.winmd (I'm assuming this is the binary) so I thought that including this in the assembly list would fix my problem, but it doesn't.
I assume I'm missing something simple, what is it?
I'm now able to do this and it was a multi step process.
Firstly you don't load/test the winmd file directly you have to load the appx that's created by the test project 1; you can't use the MSTest runner that comes with TeamCity, instead you have to install the Visual Studio Test Runner plugin [2]; you have to run the build agent in an interactive process; and finally you have to install the root certificate that the application was signed with.
You reference the appx like you would a DLL
To install the build agent and have it run interactively it wasn't good enough to use the service and mark 'Interact with Desktop' you need to have full interaction so the agent has to be run from the desktop, as admin, at startup. To do this I had to create a scheduled task that run at startup and was given full privileges.
I installed the cert by running the ps1 file in the same directory as the appx file (Add-AppDevPackage.ps1) I'm sure this is overkill but it seemed like the easiest option at the time
[1] http://msdn.microsoft.com/en-us/library/hh691189.aspx
[2] https://confluence.jetbrains.com/display/TW/VSTest.Console+Runner
Since running Metro apps headlessly is still a gray area: Running a metro app headlessly, I've recently decided to add a native unit test project to my Windows Metro app in hopes that I can find a way to run these unit tests in an automated fashion on the build server. Basically, I'm looking for something similar to MSTest.exe - a utility which is great for running tests from batch files and/or scripts.
In fact, I've tried using the new version of MSTest.exe that comes with VS11 on a generated test .dll, but it fails with the error:
"Unable to load the test container 'test.dll' or one of its dependencies... Error details: Could not load file or assembly file://test.dll' or one of its dependencies. The Module was expected to contain an assembly manifest."
Does MSTest.exe work with test containers that contain WinRT code? If not, is there a utility that will do what I want?
Edit: I just found out that MSTest does not support running tests on a Metro style app. Found here: http://msdn.microsoft.com/en-us/library/ms253138%28v=vs.110%29.aspx This really is too bad. I'm still hoping there's a utility out there that will work.
After blindly digging through the VS folders, I happened to find a new test runner under:
C:\Program Files\Microsoft Visual Studio
11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe
This utility allows you to execute WinRT unit tests from the command line. It's very similar to MSTest.exe.
There doesn't seem to be any documentation out there for this yet, but at least a help command exists.
If you are executing vstest.console.exe <filename>.dll then your tests do not get executed in appcontainer mode. You need to give <filename>.appx to execute tests in appcontainer mode. More info on how to execute tests for Windows Metro style apps from command line can be found at Running Unit Tests for Windows Metro style apps from Command Line
There is a documentation about (among other things) running unit tests for Windows Phone:
http://msdn.microsoft.com/en-us/library/windowsphone/develop/dn168930(v=vs.105).aspx
It describes also the command line way of doing it using vstest.console.exe.
It also gives a comparison of supported features between unit tests for Windows Phone and WinRT.
One important detail is that unit tests for WinRT cannot be run on a device. This is a pity and relevant to question How to automate non-interactive tests on Microsoft Surface