[I am very much new to the visual studio environment. After working with visual studio for 2-3 days I was able to get some knowledge about it.]
Well. I am working in a testing team. I just wanna setup a code coverage for cpp code developed in visual studio 2010. There are merely 2 doubts I wanna ask
1.My company has purchased visual studio 2010 ultimate edition and using that I was able to generate the code coverage for a sample test project (.dll) created using Test Project template in visual studio.
The problem here is that developers had written a test project using
cppunit as an .exe. So will I be able to get the code coverage for
an .exe too (with PROFILER enabled) using the usual method to get
the code coverage in visual studio ultimate edition. Or should I ask
them to write the unit tests using the Microsoft's unit test
framework(ie .dll). I just cant simply say do it, I need to present
them the genuine reason to do so.
2.The second thought I had was, there are lot more open source tools to get code coverage for cpp compiled using gcc or g++. So I thought of compiling the cpp code using g++ and use the open source tools to get code coverage. Is this a good idea?
When I shared this idea with a senior developer he said there are lot
more cpp code designed to be MSDN specific, you cant do by this way.If
you wanna do so you should make the code cross platform compatible or
portable[something I couldn't understand].
So here is the final thing, I am trying to present them a immediate solution of getting code coverage using the visual studio ultimate edition which we have. And later on search for the best way to achieve this without purchasing any paid tools like ncover, squish coco, bullseye coverage, etc only so I opted for the above mentioned second option.
My ultimate aim is to display the final report in the Jenkins so that everyone is able to see it without any manual intervention after the build process [which is not possible using the visual studio I guess,].
There is a plugin called MSTestRunner plugin for jenkins which can generate unit tests report but not the coverage report.
I think I made it clear. If this is a duplicate question please mention me the link to.Thanks in advance
I forgot to say this I was also able to find some tool called vcxproj2cmake I think it is clear from the name itself. So if i use this tool to generate a cmake corresponding to the actual vcxproj will i be able to the my second choice?
NOTE:
As for as now, I created a test project (lets say, SampleTestProject) as said in method 1 which has
only one test method (lest say, TestMethod) which in turn triggers all the unit tests written
using cppunit by the below mentioned method.
I asked the devs to make the .exe (which they have already with cppunit unit tests written) as .dll in a specific format like it
should contain a class called TestInitiator which contains the
CppUnit::TextUI::TestRunner (the runner instance which triggers the
cppunit tests, ideally this method should be present in order to trigger cppunit tests) with all the test methods defined in a suite.
This method is called by the above mentioned TestMethod in the SampleTestProject and thereby obtaining the code coverage
By this way the code coverage was obtained with very less change done to the code.Hope you like it!
Related
We've just switched to using google test for unit testing due to deficiencies in our previous framework.
In Visual Studio, I have a solution with multiple projects. Each project contains unit tests using google test for a specific class (or collection of linked classes) within our application. We only use VS for managing and running google tests and reference the test classes found elsewhere.
When I want to run all tests, I set startup projects to multiple, then press ctrl-f5. This pops ups 1 console window per project with the standard google test output. This is great! I have to click a key to close each of the output windows, though. It sure would be nice to integrate into "Test Explorer" within VS2013 express.
I've found this script runner:
https://visualstudiogallery.msdn.microsoft.com/9dd47c21-97a6-4369-b326-c562678066f0.
It appears to be exactly what I want, but apparently express editions don't allow extensions? (I'm new the land of visual studio.) Anyway, we're using express instead of a different version because we're commercial. My fallback solution is spending the money on a full version of VS. However, I'd much rather find a free solution so I don't have to justify purchase, then deal with licensing visual studio for every new person that joins the group.
Has anyone figured out a solution to this?
Our team also tried using "Google Test Runner" extension. The
integration with VS Test Explorer was great but we did not like
the fact that we could not see gmock warnings. This means that
some tests can pass with warnings and you will simply let that
slip by and create problems in the future. The same problem
exists when trying to use XML output generated by gmock.
Since viewing output in console is a poor option, we've decided to
parse the output of executable that runs the tests and display them
in a browser. To this end, we've modified the project so that it directs
its output to a file ( "> test_output.txt" in Debugging -> Command
Arguments) and modified the main function to parse the output,
insert it into a html file template, save this html file and then open
it in default browser using:
ShellExecute(NULL, NULL, "test_output.html", "", "", SW_SHOW);
This also enabled us to make some customizations to the way results
are displayed using javascript.
You can use this approach in you case by creating a simple project which
will run last in the list of multiple startup projects, parse the output
of other executables and display it in a browser. I know this solution has
its drawbacks, but it is (in my opinion) better than ignoring warnings.
I am trying to use NUnit for unit testing in my code base. When I try to run my test cases using ReSharper, I get:
"System.IO.FileNotFoundException: Could not load file or assembly
'ServicesTests.dll' or one of its dependencies.
ServicesTests is my class lib for my unit tests.
I have also tried to install the NUnit Test Adapter as described here but do not see my test cases in the Test Explorer.
Any ideas on what might be happening? Below are images of my class library from the solution explorer, the test explorer, ReSharper screen.
I was able to test using Nunit3 and VS2015 with Test Explorer after I changed the type of class library I was creating. Initially I was creating a Class Library (Package) and when I switched to Class Library it worked (see image below). I am still not able to use Resharper since I am using Nunit3 beta and that is currently unsupported by Resharper.
I have a similar issue also. The best answer I've come across to solve it is try and rebuild the solution (cleans and builds), or just resort to MS Test. It's a pain to convert the tests though...
Hope this helps.
EDIT:
My NUnit runs, but no tests "exist", despite having a project, classes, and methods all correct.
ReSharper does not support running unit tests in DNX projects https://youtrack.jetbrains.com/issue/RSRP-446679.
I know this is an old question but I had a similar issue recently in VS 2017. Use the Fusion Log Viewer (aka "Assembly Binding Log Viewer") to see what assembly is failing to load:
I have recently started to try to set up unit testing for my project. However, it is not working properly for classes.
I set it up using the directions from the question Unresolved externals when compiling unit tests for Visual C++ 2012.
What will happen is that the tests will start fine, but then there will be a memory access error (C0000005). This does not happen all of the time though. It will generally fail every two or three attempts, but can fail as often as one in every 20. It does not seem to matter what other functions are in the code.
When I try to debug the test, the first time a variable is accessed or used, the test will fail.
I am trying out two different testing frameworks, Boost.test (using Boost Unit Test Adapter) and Native Unit Tests. Both of them are having the same problems.
I know that the dll and the test application both use the same calling convention (__cdecl).
What is bizarre is that the test will work for a while, then suddenly begin to fail.
The project is available here.
Reinstalling Visual Studios did not help.
Whatever is going on, it's almost certainly in your code as many, many people use Boost.Test successfully with Visual Studio. Instead of a 100+ MB zip file on dropbox that noone is going to download, try to simplify your problem to the smallest possible bit of code that reproduces the problem and post it in your question.
Here's a more constructive answer. Try working through the 5-part tutorial I wrote on test-driven development with Boost.Test and Visual Studio:
C++ Unit Tests With Boost.Test, Part 1
C++ Unit Tests With Boost.Test, Part 2
C++ Unit Tests With Boost.Test, Part 3
C++ Unit Tests With Boost.Test, Part 4
C++ Unit Tests With Boost.Test, Part 5
You can also look at a workshop that I presented at C++ Now! 2014, which covers the use of the Turtle Mock library for mocking collaborators. I used CMake for the build system in the workshop as it was unreasonable to assume Visual Studio for that audience.
Working through either (or both) of those examples should give you enough step-by-step instructions that you should be able to identify what you are doing wrong in your project.
I was able to execute the following code in VS 2010 but in 2012 the following does not work. Am i doing something wrong here.
[TestMethod]
public void UTestMethod()
{
TestContext.BeginTimer("t1");
Thread.Sleep(100);
TestContext.EndTimer("t1");
}
it throws NoSupportedException on method BeginTimer.
According to the following Microsoft blog, http://blogs.msdn.com/b/geoffgr/archive/2014/08/05/adding-timers-to-coded-ui-tests-running-in-load-tests.aspx, the ability to use timers in a Unit test was removed from Visual Stuido 2012. The blog states that the timer still works when a unit test is placed inside a load test, but standalone, the unit test will throw the System.NotSupportedException. The blog states that a workaround is to use the following line of code just before the beginTimer() call:
if (TestContext.Properties.Contains("$LoadTestUserContext")) //running as load
testContextInstance.BeginTimer("MyTimerName");
This approach is working on my installation of VS2012, and allows me to have the timers in my unit test and to run it as a single user without putting it in a load test. Removing the functionality in VS 2012 was NOT the best decision by Microsoft. I have found some cases where a timer in unit tests still does not work in VS 2013, but then sometimes it will work in 2013. In any case, in VS 2012, "unit test timers" are consistently broken but the above line of code provides a workaround.
IMHO, Microsoft should a) (preferred) restore the timer functionality to the way it was in VS 2010 or b) at least update the tooltip in intellisense to state that the timer functionality only works in a load test, and update the IDE to show a squiggly warning unless timers are associated with a $LoadTestUserContext variable as shown here. The change in VS 2012 gives Visual Studio load test scripting a confusing and irritating disadvantage in comparison with industry standard tools such as LoadRunner's Vugen.
The primary use of TestContext.BeginTimer is within loadtests. However I think you are running this within a standard unit test. So in this case yes this is a bug.
The workaround here is to use a .testsettings file while running your tests to allow you to get the functionality that you are familiar with.
Hope that helps
I have 80+ VS2010 solutions, each contains some unit test projects.
All solutions are merged into one big solution before build process.
All tests (or some test subset) are executed after the successful build.
I want to enable test code coverage for all tests to get exact code coverage for all output assemblies.
My question is: how to enable code coverage without using Visual Studio?
Note: I'm using TFS2010 (MSBuild) to build merged solution. VS 2010 Premium is installed on the build server. MSTest is used for test execution.
You can use JetBrain's TeamCity Professional. It is a CI server that supports executing unit tests and calculating code coverage. It is free for small installations.
I think you need to consider deploying a code coverage tool, see here for a comparison (provided you implement .net).We use NCover, which integrated in our TFS-Build in it's console variant and, although it's not trivial to set it up, we 're very satisfied with it.In this post I had briefly described how we inserted NCoverin our build, this might we useful to you even if you go with another tool.
If you create a Vsmdi file in your large solution (ms test will usually do this for you) you can use this to tell the build which assemblies you want to instrument.
This will only provide code coverage for assemblies that have tests run against them. If you're using testrun.config files to decide which tests you want to run, this should be all you need. The code coverage results should then be published to the build drop location
Edit:
This blog post looks like it covers setting up code coverage