I am using gtest in VS2019. I have one hundred tests in ten test suites. When I run all tests, and some tests fail, then some test suites are skipped in entirety.
I did not mark any test as skipped.
I feel that this happens when some of the fails are memory issues(invalid pointer etc). When I fix the errors and rerun, everything runs.
Why does this happen? How do I make sure every test runs when I hit "run all"?
you need to change the way you run the test,
for that, what I suggest is create a test project file then, paste your files there and then click the build and run button the tests would open up as terminal and all tests would run, just as gtest works on linux; terminal based
Related
How do I find the Test Coverage of my code during the build process automatically, and ensure that if coverage is below 80%, build should fail.
So, is it possible that as part of my gradle build process when the test case runs, there should also be another testcase, which should find the test-coverage percentage and apply assertions to ensure coverage %age > 80?
Can we get this through JUnits only. I don;t want to externally check the coverage in eclipse.
Yes you can, please have a look at this project:
https://docs.gradle.org/current/userguide/jacoco_plugin.html
This is weird.
Firstly, loading the solution doesn't detect the two unit tests - I have to modify the unit test and do a rebuild for the tests to appear in test explorer.
Once I've done that, I can run a unit test ONCE. After that, I get:
Message: Failed to set up the execution context to run the test
How did it run the test the first time and not the subsequent times? Using depends.exe I can see there's one file missing: Microsoft.VisualStudio.TestTools.CppUnitTestFramework.x64.dll. I tried copying this file to the output directory but it made no difference.
The output directory contains all the files required by the main application to run, so all I've done is place the unit test DLL in that same directory. The test runs once then all subsequent runs die.
Found a solution. On the Test menu, Test Settings turn OFF Keep Test Execution Engine Running. Now I can run any test as much as I need to.
I have a TFS 2015 build which builds one of our applications (it's an ASP.NET Web API application). As part of the build process it runs our unit tests.
I have a Visual Studio Test build step which runs these unit tests and they all pass okay.
I then run dotCover from this same build to determine code coverage (no we don't use the built-in code coverage as we don't have enterprise licences). However, when run from dotCover all the same unit tests fail.
I use a script step to run a batch file which invokes dotCover as follows.
E:\JetBrains\Installations\dotCover05\dotCover.exe analyse coverage.xml /LogFile=dotcover.log
The dotCover log file doesn't seem to give any indications as to why the unit tests have failed.
Any ideas why the unit tests pass when run from the Visual Studio Test build step and then fail when run from dotCover?
It seems that the problem is related to the fact that my build uses an XML file to hold certain data values. This XML file is found by VSTest when run under TFS but not by dotCover.
When dotCover runs it creates a TestResults folder which it then copies all the necessary files to required to run the unit tests. All files are copied except the XML file. I have set the file to "Copy always" so can't understand why this file isn't copied. I tried copying the file manually as a batch file but the folder structure is created by dotCover so it doesn't exist until I actually run the code coverage.
The solution is to decorate my test classes with the DeploymentItem() attribute.
[TestClass]
[DeploymentItem("File.xml")]
This has resolved my problem.
Make sure your build service account have enough permission to run the dotcove.exe. According to your E:\JetBrains\Installations\dotCover05\dotCover.exe Seems you didn't install for all users on the build agent. Which should installed under %ProgramFiles(x86)% not %LOCALAPPDATA%\JetBrains\Installations.
Try to use CoreInstructionSet parameter in your dotCover as a workaround for your situation. Details see below picture.
After doing this try to run the build again.
You can also try this method if it is failed by the file copying: Shadow-copying in dotCover: if your NUnit tests fail during continuous testing .
I'm using Xcode 3.2.2 and the built in OCUnit test stuff. One problem I'm running into is that every time I do a build my unit tests are run, even if the build failed. Let's say I make a syntax error in one of my tests. The test fails to compile and the last successful compilation of the unit tests are run. The same thing happens if one of the dependent targets fail to build - the tests are still run. Which is obviously not what I want.
How can I prevent the tests from running if the build fails? If this is not possible then I'd rather have the tests never run automatically, is that possible? Sorry if this is obvious, I'm an Xcode noob. Should I be using a better unit testing framework?
The answer is to dump OCUnit and use GHUnit which is about a million times better:
http://github.com/gabriel/gh-unit
All you need to do is make the script that runs the unit tests dependent on your test bundle having been built. To do this:
In your Targets group expand your unit test bundle and Get Info on the Run Script.
On the general tab click the + button for the Input Files and enter:
$(BUILT_PRODUCTS_DIR)/$(EXECUTABLE_PATH)
What are the steps to get Team Foundation Server running unit tests when a given build runs?
What are the caveats / pitfalls / workarounds a dev or sysadmin should be aware of when setting up a TFS server to do this for the first time?
What are common troubleshooting steps for unit test problems during builds?
it depends on which version of TFS you are running, so I will assume it is 2008.
Firstly, you must have Team Edition for Testers installed on the computer that will act as your build agent, as stated in How To: Create a Build Definition
There are a couple of ways to tell Team Build to run tests for your build.
Unit tests can be run from a defined Test List within the Solution being built. This list is referenced by the build definition and all tests within the chosen list(s) are executed. More info here
WildCard test exectution is also available by defining a wildcard mask (i.e. Test*.dll) that instructs Team Build to run any tests present in assemblies that match the mask. This is done when defining the build definition as well.
Things to note:
If you intend to use the wildcard method and want to enable code coverage for your test configuration, you must add the following to your build definition file to enable it.
<RunConfigFile>$(SolutionRoot)\TestRunConfig.testrunconfig</RunConfigFile>
See my previous question on this for more info here
If you don't want to use test configs (A Pain in the ass to manage) just run all the test in a .dll by adding this to the build config:
<ItemGroup>
<TestContainerInOutput Include="MyProject.UnitTests.dll" />
</ItemGroup>
The whole process is smooth and fairly simple. You can inspect the unit tests that filaed on the build server by opening the test result file locally (a bit of a pain) but generally you'll just run the unit tests locally and you can see the results immediately.
If you are used to NUnit, you may opt to sort the tests by classname, it gives a similar view.
Careful with code coverage, it makes complete copies of your binaries on compile. If your binaries are sufficiently large and you compile often, it will eat through drive space quickly.
http://msdn.microsoft.com/en-us/library/cc981972(v=vs.90).aspx
I like this defination as it gives you a complete 'walkthrough' from
Creating the Project
Creating the Unit Test Project
To configuring Team build to use it Unit Test