I am now working on a visual studio package project, currently all our tests just simulates the way we interact with visual studio with windows open and close, menus selection, button click and so on.I think these tests can be classified as integrated test and they highly depends on the devenv.exe process. For pure unit test, I just want to test one of my business logic and mock other dependencies.Although visual studio SDK contains a test library under the path VisualStudioIntegration\Common\Source\CSharp\UnitTest in SDK folder, I found it a too simple project, too young to be used in my project. When I tried to write unit test from scratch, I got a lot of problems with mocking VSX interfaces. Also there is so little resources about VSX unit test, Can anyone give me some hints or links about this? Any help is appreciated.
In my case, I needed to launch and control Visual Studio from a control program to run some developer studies.
Basically, why not just have your tests run your plugin in the experimental hive on a programmatic instance?
This was a helpful resource for launching an instance of Visual Studio through COM:
http://blogs.msdn.com/b/kirillosenkov/archive/2009/03/03/how-to-start-visual-studio-programmatically.aspx
Related
I use in my project SpecFlow to provide automation testing of web-site(with Selenium). I want create end-to-end test suite for provide full checking of project according to BDD. When I create tests playlist in test explorer in Visual Studio and start all tests in playlist all my tests launch according to their alphabetic name. But when I restart project and start this playlist newly all my tests launch in the wrong order. I think, that this problem may be because of rebuild playlist file, but I'm not sure. Could You help me, please?
TestRunner - MSTest
Friends,
I have been looking at unit test code coverage tools for the last 2 weeks, evaluating them for our specific needs. The last one I would like to evaluate is Visual Studio's own code coverage for native C++ unit tests. In our solution, we have the following:
full Visual Studio 2013 development environment with large projects
of multi-100 k lines of code
900+ unit tests already written in GTEST (working and executing in
the build process) and working in this environment native C++ code
(unmanaged)
a dozen projects in the solution or so
What I am trying to do is figure out
If I can use the Visual Studio code coverage Analysis tool to report results on my gtest unit tests
get code coverage results generated each time the unit tests or run (our unit tests are automatically run in a post-build step)
I have looked around these forums, MSDN and other sites (like stackoverflow) and I can't find a good source of guidance to making this combination of tools work. As I say, I have the solution, projects and unit tests running and working well, but to add code coverage I am not seeing how to do so using Microsoft's own Visual Studio code coverage.
I have already evaluated a few other packages like Software Verify, Semantic Designs, OpenCPPCoverage and Bullseye. I was hoping to try our Visual Studio's code coverage component since it is already part of our dev environment, but I can't seem to figure out how or if that will work.
Any help?
Thanks for any feedback, info you can give.
After searching more, I found that what I was missing was an adapter to run the tests in VS 2013. While my tests were executing as part of the build process, they weren't generating results in the Test Explorer because it wasn't being interfaced with Visual Studio's test explorer. This can be done simply by installing an extension. The extension I found that works well for us is GoogleTest Runner, which is found in the extensions library here:
GoogleTest Runner
I am leaving this question and answer here in case others have this same issue and need some help
Using the C++ unit testing framework with Visual Studio 2013 Professional, one can write unit tests and run them from within the IDE, but in order to generate any coverage reports, apparently, one needs to have the Premium or Ultimate edition of Visual Studio.
Is it possible to get code coverage reports with the Professional edition, preferably without installing any third party tools? If not, what alternate options exist for people who are not using the more expensive editions of the IDE?
Please note that it's possible to generate the coverage data by using command line tools, but I am unable to find a way to look at the results. For the sake of reference, here are the steps for command line generation of the coverage statistics:
Build the code to test with with /PROFILE linker switch
Run vsinstr /coverage <binaryName> to instrument the code; make sure that you are inside VS 2013 command prompt
Run start vsperfmon -coverage -output:results to run the profiler
Run your unit tests
Stop the profiler by running vsperfcmd -shutdown
The above will give you a results.coverage file, with no way to view it without the Premium or Ultimate editions as far as I know.
With VS2013 Professional you are out of luck if you want to do it without third party tools Requirements: Visual Studio Ultimate, Visual Studio Premium (http://msdn.microsoft.com/en-us/library/dd537628.aspx). From what I understand you already managed to generate your *.coverage file and you are having problems opening it. Visual Coverage (https://github.com/jsargiot/visual-coverage) tool can help you with that, its very simple to use and it is opensource. If you would like to find more alternatives, see another SO thread: Viewing Code Coverage Results outside of Visual studio. The tools are meant for C# coverage files but from what I understand there should not be any difference.
If you are ready to use third-party libraries, you can use OpenCPPCoverage. It works for me like a command line app. But I couldn't run it like a visual studio plugin at Professional Visual Studio 2013.
One possible way you could get coverage data is by running an instrumentation toolkit on your Unit Test program. The instrumentation tool will tell you how much was covered during execution and you can easily relate this data back to how much coverage your unit tests give you. I did this with an XCode project and OpenPAT but you could do the same with any Visual Studio friendly toolkit.
I am developing a Windows Phone 7 project and wish to test the app's business logic (i.e. just the vanilla C# code) within stand-alone unit tests, run on Windows from the command line (from Jenkins) rather than on a device.
What's the best way of achieving this using the free Express tools only?
Currently I am thinking of:
Developing the main WP7 project using VS Express WP as usual
Having a build script copy the WP7 code and generate a VC# console project file containing each source file
Create unit test code in this second project using Visual C# 2010 Express
Compile test code for Windows and run unit tests using NUnit, Express Unit or similar.
Sounds like a whole lot of hassle, does anyone have experience of a nicer option than this?
Have a look at this question: State of unit testing for Windows Phone
It has some nice ideas and hints and I am afraid that it isn't much better at this time.
I would like to run my Silverlight Unit Tests from Visual Studio instead of opening a new instance of my Silverlight Test App in the browser.
Apparently it is possible (http://www.jeff.wilcox.name/2008/09/rc0-new-test-features/ - search for "Visual Studio Team Test log provider output")....but I don't understand how you enable this, does anyone know how?
Would be great if these tests would run from the "Run All Tests in Solution" button (Ctrl+R, A).
The blog post you referenced clearly states that this is output only and does not integrate Silverlight UT with Visual Studio Team Testing System. It will simply output into a TRX format that can then be opened in that window within Visual Studio. You still cannot run your Silverlight tests from a vsmdi, you will need to run them from within the web browser test harness.
I agree it would be great if the tests would run from the "Run All Tests in Solution" button, but that is not possible with the current tools today.
I have written a blog post about how to get the Silverlight Testing Framework to output the Visual Studio "TestResults.trx" log files. I couldn't find anything about how to do this anywhere else.