I have just recently started carrying out unit testing for my software written in Visual Studio 2005 using Google Test Framework. Alongside, I have come across this tool to carry out code coverage known as: Coverage Validator. I am intending to monitor my unit tests using this Coverage Validator software but looks like it does not support Google Test Framework yet. It can carry out the monitoring using CPP unit but I was unable to find any documentation or way to do the same via Google Test. Is there a way through which I could achieve this?
I don't know Coverage Validator, but in general I don't see a reason why ANY coverage tool for CPP would need to officially 'support' ANY decent unit-test framework- it should just work...
Related
Currently, I am using jacoco report tool to know how much coverage for my system. But this system is quite old and has many existing Fitnesse tests. I am newbie on this system and want to learn it by reading its Fitnesse test. My problem is I dont know what Fitnesse test is for what specific class.Beside that When I see a class is covered and I dont know which Fitnesse test covered this class. My system has more than 500 Fitnesse Tests.
JaCoCo does not provide this out of the box.
There are some third-party JaCoCo integrations such as SonarQube that allow to obtain information about relation between JUnit or TestNG test to code. Don't know much about Fitnesse, so can't tell whether this will work with it or not.
However referring to the same link: general principle to obtain such information using JaCoCo (and that's how SonarQube does) - is to measure coverage separately for each test and save data into place dedicated to this test.
Also IDEs are able to show coverage (there are EclEmma plugin for Eclipse based on JaCoCo and other tools for other IDEs), so if you're able to run Fitneese test in IDE, then this principle can be applied even manually by running one test after another within IDE.
Also have a look at another code coverage tool for Java that is commercial and named Clover.
I'm writing unit tests using NUnit.Framework for a project I have up live. I have read few things about Ncover and it looks like an alternative of NUnit. Could someone please explain what it really is and what it does or post a link where I could find my answer.
NCover is a code-coverage tool that allows you to assess how much of your source code is "covered" by unit tests. The idea is that while you're running your unit tests with nUnit, nCover will be operating in the background, keeping track of each line of code that's being executed. After the tests are finished, nCover will show a visual report showing exactly which parts of your source code are and are not covered by existing unit tests.
It's a tool to help you determine if you have any functionality that isn't assessed by unit tests. While it's not perfect, it does give a good visual representation if you're missing out on some potential tests.
The company I'm working at uses xUnit to write Integration tests. xUnit works perfectly for us but we will like to extract more statistical information out of runs. For example - "How many times did this specific Test-Case has been failing in the last month", and maybe even sort it out nicely on a pie chart.
Since we have a Microsoft Test Manager license, I did some research on it, and it seems like it does support more detailed reports. I also like the coupling between Manual Test-Cases to Automation Test-Cases, and the fact you easily identify how much of your Test-Cases are automated.
Sadly enough, Test Manager only support MSTest integration out of the box. I did noticed however that the MSTest.ext alternative - VSTest.exe is able to run xUnit tests, and even output TRX result file. Is there any way to integrate xUnit (or nUnit) to the Test Manager somehow? Has anyone done so in the past? we prefer to use Test Manager, but I'm interesting to know if there is an alternative that support a couple Test-Cases with Automated Test-Cases and a way to get statistical information about multiple-runs.
Thank you.
This tool will allow you to associate NUnit and xUnit test cases with Microsoft Test Manager.
https://github.com/JakeGinnivan/TestCaseAutomationAssigner
After some research on the Microsoft Forum (and personally speaking with Microsoft representative), It seems like it's not possible to use xUnit with MS Test Manager.
We decided not to use MS Test Manager and handle all our test runs using VSTest.exe and xUnit categories.
Edit: It's now possible to use an external tool for that called "TestCaseAutomationAssigner". See Jeff's answer for more information.
I am working on a project in where I decided to use unit tests. This was new to me but after researching I feel pretty confident I am doing it correctly creating mock objects and testing that the correct methods are called. This is working great but now I would like to actually run some tests that use the actual database and external components. How should I go about testing the actual execution of code? I do not want these tests to run when I run all tests. Is there a way to accomplish this using built in testing in vs2012?
Not that I'm aware of.
What you could do is create a separate project for your integration tests and then in the test explorer, separate your tests by class. Or some other logical seperation.
the current implementation that exists seems to specifically have unit testing mainly in mind, but that should change with 2013 and a stronger push towards TDD and Agile development.
Have a look here :http://msdn.microsoft.com/en-us/library/ms243147(v=vs.80).aspx
Scroll down to "Attributes for Identifying and Sorting Tests". There might be something there that's useful that you can use.
Which do you prefer?
What are the advantages and disadvantages of each for Unit Testing?
EDIT: I will admit that Team System offers a lot more than just Unit Testing, such as performance and load testing of applications and databases. This question was centering around writing unit tests and which do you prefer.
Nunit:
Advantages:
Free
Very similar to team system in attributs and methods for assertion, some names are even the same
Disadvantages:
Tests must be run via console or external application ( this can be seen as an advantage, but not from my point of view).
Team System testing
Advantages:
A part of VS, you can run tests in a test window.
If you run a team system server you can run tests more easily as a part of the automated build
Disadvantages:
Expensive
Still isn't as stable as NUnit
A comparison between team system and Nunit
We use team system 2008 as we are gold certified partners to microsoft, but earlier used Nunit due to bug related issues in VS 2005. I prefer the VS solution.
Both are good solutions for your work, look also out for other free solutions like:
Good alternatives to Team System
One very specific reason, is that NUnit won't tie you to the professional edition of the visual studio.
Update: Here is a link about unit testing support on Professional edition in vs 2008: http://msdn.microsoft.com/en-us/library/bb385902.aspx
One other advantage of NUnit is that it doesn't require that you add anything to your test classes. MSTest requires the presence of a TestContext property. We started out with MSTest but converted to NUnit. I also find NUnit to be significantly faster and I prefer ReSharper's test runner UI.
Currently NUnit has test categories that allow you to run unit tests separately from slower integration tests.
MS Tests has no such built-in mechanism.
When using MS Tests, you can use CHESS:
CHESS is a tool for systematically testing multithreaded code. Given a concurrent test, CHESS systematically drives the test along all possible thread interleavings.
Also, I found a nice comparison here that claims MS Tests are a little slower than NUnit, but I didn't check it myself.
Doesn't Visual Studio 2008 allow you to use other testing frameworks when you create the test project? I vaguely remember this from watching the old MVC Framework videos back when Hanselman was doing the preview 2 or 3 videos.
This would allow you to use any testing framework you like and still be able to use it in your VS2008 IDE.
What about testing private methods.
Team System create automatically shadow accessors using reflections - does NUnit same?
There is always tools like ReSharper and TestDriven.NET. They will let you run tests from Visual Studio