I have installed DevPartner and I am using visual studio 2010.
However, when I launch 'Start with coverage analysis', nothing happens.
In the videos, they show that in the under Options->DevPartner->Code review, there should be a list of projects. But I do not see such list.
It will be great if you could please help me with it.
Many thanks.
Chintan
Coverage analysis ans Code review are 2 separate products in the Devpartner Suite.
Coverage Analysis is to used to determine which lines of code are executed in a run of your project. You can also merge several runs to get an overall idea of how much code you have exercised. Coverage analysis works for both Native (C, C++) and managed languages.
When you launch your application with coverage analysis you will only see a small toolbar appear that allows you to take snapshots or stop recording coverage information. After your program exits a final session will be collected and displaying in Visual Studio. Unless something is wrong with your license. Another problem would be when launching Visual Studio and opening your project you may see an error unable similar to Devpartner start session control service.
Code Review is used as static analysis of Managed (.Net) code. It does not run your project but analysis the code in your projects and determines if any of that code violates the rule set you selected to run against.
Related
I am trying to get the code coverage for a Windows service written in C++ (let's call it hello.exe) with Visual Studio 2012 Premium tools. So far I have attempted to use the method outlined in the answer to this post: How to use MS code coverage tool in command line?
For your convenience, the steps given were to run:
vsinstr /coverage hello.exe
start vsperfmon /coverage /output:mytestrun.coverage
hello.exe
vsperfcmd /shutdown
I am able to instrument, test and get the .coverage file for hello.exe when I start it as an executable in the command prompt no problem. But if I instead try to start it as a service in step 3 (either through services.msc manually or win32serviceutil.StartService in Python), nothing registers and the coverage file is empty. Is there something i'm missing here?
Extra info: In an effort to solve this I came across what seems to be another code coverage utility called CodeCoverage.exe (Microsoft Visual Studio 11.0\Team Tools\Dynamic Code Coverage Tools). Is that what I'm supposed to be using for this task? Also, I found something seemingly related here https://msdn.microsoft.com/en-us/library/dd255408.aspx, but (forgive me for my ignorance) am not sure whether a .NET service is the same thing as a Windows service.
Answering my own question here, after a lot of trying and pouring through documents I found this:
https://blogs.msdn.microsoft.com/hilliao/2008/08/19/code-coverage-using-visual-studio-2008s-code-coverage-tools/
Basically, a windows Service is different from a process so we have to enable the cross session flag and specify the user when monitoring.
start vsperfmon /coverage /output:mytestrun.coverage /cs /user:”Everyone”
So just instrument the exe as usual, start monitoring with above command, start the service, do testing, stop the service, and shutdown the monitoring.
Hope it helps someone in the future.
We've just switched to using google test for unit testing due to deficiencies in our previous framework.
In Visual Studio, I have a solution with multiple projects. Each project contains unit tests using google test for a specific class (or collection of linked classes) within our application. We only use VS for managing and running google tests and reference the test classes found elsewhere.
When I want to run all tests, I set startup projects to multiple, then press ctrl-f5. This pops ups 1 console window per project with the standard google test output. This is great! I have to click a key to close each of the output windows, though. It sure would be nice to integrate into "Test Explorer" within VS2013 express.
I've found this script runner:
https://visualstudiogallery.msdn.microsoft.com/9dd47c21-97a6-4369-b326-c562678066f0.
It appears to be exactly what I want, but apparently express editions don't allow extensions? (I'm new the land of visual studio.) Anyway, we're using express instead of a different version because we're commercial. My fallback solution is spending the money on a full version of VS. However, I'd much rather find a free solution so I don't have to justify purchase, then deal with licensing visual studio for every new person that joins the group.
Has anyone figured out a solution to this?
Our team also tried using "Google Test Runner" extension. The
integration with VS Test Explorer was great but we did not like
the fact that we could not see gmock warnings. This means that
some tests can pass with warnings and you will simply let that
slip by and create problems in the future. The same problem
exists when trying to use XML output generated by gmock.
Since viewing output in console is a poor option, we've decided to
parse the output of executable that runs the tests and display them
in a browser. To this end, we've modified the project so that it directs
its output to a file ( "> test_output.txt" in Debugging -> Command
Arguments) and modified the main function to parse the output,
insert it into a html file template, save this html file and then open
it in default browser using:
ShellExecute(NULL, NULL, "test_output.html", "", "", SW_SHOW);
This also enabled us to make some customizations to the way results
are displayed using javascript.
You can use this approach in you case by creating a simple project which
will run last in the list of multiple startup projects, parse the output
of other executables and display it in a browser. I know this solution has
its drawbacks, but it is (in my opinion) better than ignoring warnings.
[I am very much new to the visual studio environment. After working with visual studio for 2-3 days I was able to get some knowledge about it.]
Well. I am working in a testing team. I just wanna setup a code coverage for cpp code developed in visual studio 2010. There are merely 2 doubts I wanna ask
1.My company has purchased visual studio 2010 ultimate edition and using that I was able to generate the code coverage for a sample test project (.dll) created using Test Project template in visual studio.
The problem here is that developers had written a test project using
cppunit as an .exe. So will I be able to get the code coverage for
an .exe too (with PROFILER enabled) using the usual method to get
the code coverage in visual studio ultimate edition. Or should I ask
them to write the unit tests using the Microsoft's unit test
framework(ie .dll). I just cant simply say do it, I need to present
them the genuine reason to do so.
2.The second thought I had was, there are lot more open source tools to get code coverage for cpp compiled using gcc or g++. So I thought of compiling the cpp code using g++ and use the open source tools to get code coverage. Is this a good idea?
When I shared this idea with a senior developer he said there are lot
more cpp code designed to be MSDN specific, you cant do by this way.If
you wanna do so you should make the code cross platform compatible or
portable[something I couldn't understand].
So here is the final thing, I am trying to present them a immediate solution of getting code coverage using the visual studio ultimate edition which we have. And later on search for the best way to achieve this without purchasing any paid tools like ncover, squish coco, bullseye coverage, etc only so I opted for the above mentioned second option.
My ultimate aim is to display the final report in the Jenkins so that everyone is able to see it without any manual intervention after the build process [which is not possible using the visual studio I guess,].
There is a plugin called MSTestRunner plugin for jenkins which can generate unit tests report but not the coverage report.
I think I made it clear. If this is a duplicate question please mention me the link to.Thanks in advance
I forgot to say this I was also able to find some tool called vcxproj2cmake I think it is clear from the name itself. So if i use this tool to generate a cmake corresponding to the actual vcxproj will i be able to the my second choice?
NOTE:
As for as now, I created a test project (lets say, SampleTestProject) as said in method 1 which has
only one test method (lest say, TestMethod) which in turn triggers all the unit tests written
using cppunit by the below mentioned method.
I asked the devs to make the .exe (which they have already with cppunit unit tests written) as .dll in a specific format like it
should contain a class called TestInitiator which contains the
CppUnit::TextUI::TestRunner (the runner instance which triggers the
cppunit tests, ideally this method should be present in order to trigger cppunit tests) with all the test methods defined in a suite.
This method is called by the above mentioned TestMethod in the SampleTestProject and thereby obtaining the code coverage
By this way the code coverage was obtained with very less change done to the code.Hope you like it!
"local continuous integration system" may not be the correct term, but what I'm hoping to find is an continuous integration system that can be configured to monitor changes to local files (C++ files in particular) and 1) try to compile the affected object files (stopping on first failure), and if successful and no new source file changes 2) link the affected binaries, and if successful and no new source file changes 3) run affected tests.
By monitor changes to local files, I do not mean monitor commits to a revision control system, but the state of local files as they are saved. Ideally the system would be provide integrations into source editors so it could monitor changes in the editor that haven't even been saved to disk yet.
Ideally it would also provide a graphical indication (preferably on Windows 7) of current and recent status that quickly allows drilling into failures when desired.
The closest thing I found was nose as described here but that only covers running Python tests not building C++ files.
The closest thing to what you are looking for is cdash and the Boost test bench; I think that a tool like the one you are looking for will never exist for C++ because compiling each project after editing a single file it's only a waste of time in a productive C++ workflow.
Continues Integration is a rising concept today, so you are not alone here.
Assuming you are developing on Windows, if you are working with Microsoft Visual Studio
you may consider Microsoft's Visual Studio Team Foundation Server (TFS)
(formerly Visual Studio Team System).
That will give you Source-Control AND Build-Automation in one package,
with great integration to Microsoft products, of course
(I think there is a free version for MSDN users).
If not keen on Microsoft products, or just looking for build-automation,
I would recommend a great Open-Source Continues Integration tool:
Jenkins CI.
Good luck!
I would look at Jenkins CI - it is a good tool, works on any platform, and can be configured to do almost anything. I used it to run Python Code that talked to a mobile phone, made calls and recorded those calls (and tested the "quality" of the call, although my project never got the £xxxx real quality software, as we were just showing a concept), and then Jenkins would produce graphs of "how well it worked".
You can also do what you describe of "chaining" - so it would discover that your source has changed, try to build it [generally this is done using make, so it would automatically stop at the first errored file (although it could be hundreds of errors in one file!)]. Compile and build success then chains to running tests. Not entirely sure how you determine what is "relevant". If your test cycle isn't enormous, I'd run them all!
I have 80+ VS2010 solutions, each contains some unit test projects.
All solutions are merged into one big solution before build process.
All tests (or some test subset) are executed after the successful build.
I want to enable test code coverage for all tests to get exact code coverage for all output assemblies.
My question is: how to enable code coverage without using Visual Studio?
Note: I'm using TFS2010 (MSBuild) to build merged solution. VS 2010 Premium is installed on the build server. MSTest is used for test execution.
You can use JetBrain's TeamCity Professional. It is a CI server that supports executing unit tests and calculating code coverage. It is free for small installations.
I think you need to consider deploying a code coverage tool, see here for a comparison (provided you implement .net).We use NCover, which integrated in our TFS-Build in it's console variant and, although it's not trivial to set it up, we 're very satisfied with it.In this post I had briefly described how we inserted NCoverin our build, this might we useful to you even if you go with another tool.
If you create a Vsmdi file in your large solution (ms test will usually do this for you) you can use this to tell the build which assemblies you want to instrument.
This will only provide code coverage for assemblies that have tests run against them. If you're using testrun.config files to decide which tests you want to run, this should be all you need. The code coverage results should then be published to the build drop location
Edit:
This blog post looks like it covers setting up code coverage