This is weird.
Firstly, loading the solution doesn't detect the two unit tests - I have to modify the unit test and do a rebuild for the tests to appear in test explorer.
Once I've done that, I can run a unit test ONCE. After that, I get:
Message: Failed to set up the execution context to run the test
How did it run the test the first time and not the subsequent times? Using depends.exe I can see there's one file missing: Microsoft.VisualStudio.TestTools.CppUnitTestFramework.x64.dll. I tried copying this file to the output directory but it made no difference.
The output directory contains all the files required by the main application to run, so all I've done is place the unit test DLL in that same directory. The test runs once then all subsequent runs die.
Found a solution. On the Test menu, Test Settings turn OFF Keep Test Execution Engine Running. Now I can run any test as much as I need to.
Related
I am using gtest in VS2019. I have one hundred tests in ten test suites. When I run all tests, and some tests fail, then some test suites are skipped in entirety.
I did not mark any test as skipped.
I feel that this happens when some of the fails are memory issues(invalid pointer etc). When I fix the errors and rerun, everything runs.
Why does this happen? How do I make sure every test runs when I hit "run all"?
you need to change the way you run the test,
for that, what I suggest is create a test project file then, paste your files there and then click the build and run button the tests would open up as terminal and all tests would run, just as gtest works on linux; terminal based
I have a TFS 2015 build which builds one of our applications (it's an ASP.NET Web API application). As part of the build process it runs our unit tests.
I have a Visual Studio Test build step which runs these unit tests and they all pass okay.
I then run dotCover from this same build to determine code coverage (no we don't use the built-in code coverage as we don't have enterprise licences). However, when run from dotCover all the same unit tests fail.
I use a script step to run a batch file which invokes dotCover as follows.
E:\JetBrains\Installations\dotCover05\dotCover.exe analyse coverage.xml /LogFile=dotcover.log
The dotCover log file doesn't seem to give any indications as to why the unit tests have failed.
Any ideas why the unit tests pass when run from the Visual Studio Test build step and then fail when run from dotCover?
It seems that the problem is related to the fact that my build uses an XML file to hold certain data values. This XML file is found by VSTest when run under TFS but not by dotCover.
When dotCover runs it creates a TestResults folder which it then copies all the necessary files to required to run the unit tests. All files are copied except the XML file. I have set the file to "Copy always" so can't understand why this file isn't copied. I tried copying the file manually as a batch file but the folder structure is created by dotCover so it doesn't exist until I actually run the code coverage.
The solution is to decorate my test classes with the DeploymentItem() attribute.
[TestClass]
[DeploymentItem("File.xml")]
This has resolved my problem.
Make sure your build service account have enough permission to run the dotcove.exe. According to your E:\JetBrains\Installations\dotCover05\dotCover.exe Seems you didn't install for all users on the build agent. Which should installed under %ProgramFiles(x86)% not %LOCALAPPDATA%\JetBrains\Installations.
Try to use CoreInstructionSet parameter in your dotCover as a workaround for your situation. Details see below picture.
After doing this try to run the build again.
You can also try this method if it is failed by the file copying: Shadow-copying in dotCover: if your NUnit tests fail during continuous testing .
I used work on Java. I was using Ant/Maven for making the build process easier.
I used to write the unit tests in the JUnit framework. When I say build all unit test area lso used to run.
So there means, Build includes the compiling the source code and running the unit test against that compiled source code.
Now I started working with C++ on new project in Visual C++. Here, When I say build, Only source is compiled and linked with libs. But unit test are not ran.
So, now I got confused with actual definition of build process.
Does build process includes running tests also? Or it is just compiling and linking of source code?
Ant and Maven "know" about unit tests. If an executable is marked as test, it will be run during the build process.
Visual Studio, AFAIK, does not have this concept. Even if an executable is written using some *Unit framework, Visual Studio will build it but not run it (by default). It is possible to run tests during the build process, but you need to add a post-build step to the test program's project to run the executable.
Disclaimer: The last version I used was VS 2003.NET; things might have improved (or gotten worse) since then.
Nowadays, I use the build tool with the greatest flexibility: make. In the project I'm currently working on, there's a build target named 'all' which builds every executable and test program, a target named 'check' which runs every test program (and builds them if needed), a target named 'coverage' which computes code coverage, and a target named 'run' which does all of the above. This way I get to choose what to run.
I hope that you can see from this that "what does the build process include" depends on how the build system is set up. There isn't a universally defined concept of "build process".
I'm using Xcode 3.2.2 and the built in OCUnit test stuff. One problem I'm running into is that every time I do a build my unit tests are run, even if the build failed. Let's say I make a syntax error in one of my tests. The test fails to compile and the last successful compilation of the unit tests are run. The same thing happens if one of the dependent targets fail to build - the tests are still run. Which is obviously not what I want.
How can I prevent the tests from running if the build fails? If this is not possible then I'd rather have the tests never run automatically, is that possible? Sorry if this is obvious, I'm an Xcode noob. Should I be using a better unit testing framework?
The answer is to dump OCUnit and use GHUnit which is about a million times better:
http://github.com/gabriel/gh-unit
All you need to do is make the script that runs the unit tests dependent on your test bundle having been built. To do this:
In your Targets group expand your unit test bundle and Get Info on the Run Script.
On the general tab click the + button for the Input Files and enter:
$(BUILT_PRODUCTS_DIR)/$(EXECUTABLE_PATH)
What are the steps to get Team Foundation Server running unit tests when a given build runs?
What are the caveats / pitfalls / workarounds a dev or sysadmin should be aware of when setting up a TFS server to do this for the first time?
What are common troubleshooting steps for unit test problems during builds?
it depends on which version of TFS you are running, so I will assume it is 2008.
Firstly, you must have Team Edition for Testers installed on the computer that will act as your build agent, as stated in How To: Create a Build Definition
There are a couple of ways to tell Team Build to run tests for your build.
Unit tests can be run from a defined Test List within the Solution being built. This list is referenced by the build definition and all tests within the chosen list(s) are executed. More info here
WildCard test exectution is also available by defining a wildcard mask (i.e. Test*.dll) that instructs Team Build to run any tests present in assemblies that match the mask. This is done when defining the build definition as well.
Things to note:
If you intend to use the wildcard method and want to enable code coverage for your test configuration, you must add the following to your build definition file to enable it.
<RunConfigFile>$(SolutionRoot)\TestRunConfig.testrunconfig</RunConfigFile>
See my previous question on this for more info here
If you don't want to use test configs (A Pain in the ass to manage) just run all the test in a .dll by adding this to the build config:
<ItemGroup>
<TestContainerInOutput Include="MyProject.UnitTests.dll" />
</ItemGroup>
The whole process is smooth and fairly simple. You can inspect the unit tests that filaed on the build server by opening the test result file locally (a bit of a pain) but generally you'll just run the unit tests locally and you can see the results immediately.
If you are used to NUnit, you may opt to sort the tests by classname, it gives a similar view.
Careful with code coverage, it makes complete copies of your binaries on compile. If your binaries are sufficiently large and you compile often, it will eat through drive space quickly.
http://msdn.microsoft.com/en-us/library/cc981972(v=vs.90).aspx
I like this defination as it gives you a complete 'walkthrough' from
Creating the Project
Creating the Unit Test Project
To configuring Team build to use it Unit Test