We have a testing environment using Visual Studio 2013 and MSTest where we need to run a setup script (.cmd-file) before starting certain test runs. Currently we are using .testsettings files for this purpose, which works great.
However a problem arose when we started using MS Fakes and shims, because debugging tests using the ShimsContext throws a UnitTestIsolationException with the error message Failed to resolve profiler path from COR_PROFILER_PATH and COR_PROFILER environment variables.
The way to resolve this appears to be to change to a .runsettings file instead, to use the Visual Studio testrunner instead of MSTest. But how can we specify a setup script to be run once before the entire test run using this file format?
You can call your *.cmd scripts from initialize and cleanup methods.
Check this: How to create Startup and Cleanup script for Visual Studio Test Project?
Related
I can't find the place where VS 2017 keeps the test result file. I have a lot of Specflow tests which are running fine but i want to find the place where VS is keeping the results. I am using a Library called Pickles which generates documentation from specflow, so it was be accessed via an HTML page. This library needs the VS test results file to add info about which tests are passing etc.
Since SpecFlow is just generating unit tests under the hood, if you run those tests from visual studio 2017 it does not generate test result files.
Instead, you need to use the vstest.console.exe command line interface to run the tests.
I have been creating test automation using Visual Studio Test. My Code is stored on Visual Studio Online and it uses GitHub to store the source code.
The problem is as of this morning, I started seeing the error "'Text' is an invalid XmlNodeType' when I try to run my tests from Visual Studio' Test Explorer.
I have been adding new test methods to the source and updating the test.runsettings file with additional parameters. The test.runsettings file is getting large, as I have about 152 tests.
I can't post the runsettings file, as it contains server names ad infinitum.
And the server tests code is standardized MS unit tests.
Has anyone seen this issue pop up?
I don't have an answer as to why the project can't be run in Visual Studio 2017, however I found a workaround. I'm using VSTest.Console.exe on the command line to test my automation.
vstest.console.exe myTests.dll /Settings:test.runsettings /Tests:TestMethod1 /Diag:Info.txt
For the life of me I can't get unit testing working in Visual Studio 2017 from the new msbuild-based netcoreapp1.0 xunit project template.
The requirement is for unit tests to work both inside Visual Studio (for the devs) and from dotnet test on the CLI for the automated build process however, I can't get either working consistently.
Here is what I have tried:
In an existing solution, create a new project and select .NET Core > xUnit Test Project.
Build project from Visual Studio, default test appears and runs successfully, now run dotnet test from powershell prompt, get:
> dotnet test
Test run for D:\...\bin\Debug\netcoreapp1.0\MyProj.dll(.NETCoreApp,Version=v1.0)
dotnet exec needs a managed .dll or .exe extension. The application specified was 'C:\Program'
Or dotnet test with csproj file:
> dotnet test MyProject.csproj
(same error as above)
> dotnet test ..\MySolution.sln
Couldn't find a project to run test from. Ensure a project exists in D:\...
Or pass the path to the project
If I add the xunit.runner.console or xunit.runner.msbuild nuget packages, it stops the unit tests working from inside Visual Studio.
How do I get both working at the same time?
Thanks!
The bug you're hitting is present in Preview 3 and fixed in Preview 4. They didn't escape the command line when executing it, and since dotnet.exe is installed into C:\Program Files\dotnet by default, it always fails.
If you want to continue to use Preview 3, the simplest work-around is to edit your system PATH environment variable, and replace C:\Program Files\dotnet with C:\Progra~1\dotnet.
I know this isn't a very good answer, but dotnet-test-xunit only support project.json files. VS 2017 forces you to switch to csproj files.
I found this on xunit twitter feed:
If you're trying use #xunit in VS2017 RC w/ .NET Core, remove dotnet-test-xunit and use xunit.runner.visualstudio 2.2 beta 4 instead.
With the latest RC.3 I was having issues with the tests not being discovered, and found out that when you run the built-in Test Explorer it says in the output that Microsoft.DotNet.InternalAbstractions 1.0.0 is missing. This was also issue in the previous versions of .NET Core, and the solution is the same, install the package from Nuget.
My c++ unit tests use gtest framework so every test is a separate executable. I'm running the CodeCoverage.exe tool to get the raw coverage data of each executable run. When I open the results in Visual Studio, each executable run appears in a separate branch and the results can't be merged together.
Is there any way to make this work?
One thing I've tried is to specify /session:uniq in the command line for both runs but that didn't have any effect.
Does it help you to capture via vsperfcmd.exe?
Build your gtest with /profile flag specified to the linker
Instrument your gtest exe with Visual Studio coverage instrumentation
vsinstr.exe gtest.exe /COVERAGE
Launch the code coverage capture tool
vsperfmon.exe /COVERAGE /OUTPUT:gtest.coverage
Run the gtest
Stop the capture tool
vsperfcmd.exe -shutdown
Launch the output file (gtest.coverage) to see the output in Visual Studio
OK, this should have been a comment, but I don't have the permissions yet.
You can run all your unit tests with the macro:
RUN_ALL_TESTS();
I have a project upgraded from Visual Studio 2010 to 2012 and the .testrunconfig file was included in the upgrade process.
I noticed that it was possible to click "Analyze code coverage" on any of the unit tests that I had run and it would correctly display the result. However, my test run configuration (originally from VS 2010) had code coverage disabled.
After doing a bit of research I learned that the VS 2010 configuration files have been deprecated and replaced by .runsettings files. It would appear that VS 2012 enforces assembly instrumentation by default which has a massive overhead associated with it.
Therefore, I would like to know how I can disable code coverage in VS 2012. Based upon my current findings it does not seem to be a trival task. One recent article I read had me creating an XML file manually and naming it "MYCONFIGURATION.runsettings" and manually manipulating XML attribute values.
Does anyone know how this should be done?
This is what I understand from your post:
You have a Test project with .testsettings file. You have not enabled code coverage in the test settings.
Code coverage instrumentation is not enabled by default in your scenario. Binaries will be instrumented if you do 'analyze code coverage' from VS.
Additional Info:
You can confirm that .coverage file is not generated by running the following command from visual studio developer command prompt:
vstest.console.exe /Settings:<your test settings file> test.dll
A coverage file will only get generated if you have enabled coverage in test settings.
Code coverage is only enabled through the Test Explorer using data driven adapters. The metadata for tests ran through the test explorer is almost completely different than that of tests ran straight from the unit test session window. Have you tried simply running it straight from the code (the MSTest gui bubbles) or from the unit test session window?