I am working on a a test solution through the CodedUITest framework on visual studio 2017 enterprise.
I am using a runsettings file as detailed here : https://learn.microsoft.com/fr-fr/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file?view=vs-2019#example-runsettings-file
I set the mediarecorder in order to get the video even when the test passed:
<DataCollector uri="datacollector://microsoft/VideoRecorder/1.0" assemblyQualifiedName="Microsoft.VisualStudio.TestTools.DataCollection.VideoRecorder.VideoRecorderDataCollector, Microsoft.VisualStudio.TestTools.DataCollection.VideoRecorder, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" friendlyName="Screen and Voice Recorder">
<!--Video data collector was introduced in Visual Studio 2017 version 15.5 -->
<Configuration>
<!-- Set "sendRecordedMediaForPassedTestCase" to "false" to add video attachments to failed tests only -->
<MediaRecorder sendRecordedMediaForPassedTestCase="true" xmlns="">
<ScreenCaptureVideo bitRate="512" frameRate="2" quality="20" />
</MediaRecorder>
</Configuration>
</DataCollector>
But I did not find where the video file is stored.
Any idea where I can find it? Is it possible to specify a folder path where we want to store the record?
Thx,
Check further up in the runsettings file: Recordings are stored underneath the ResultsDirectory
<RunSettings>
<RunConfiguration>
...
<!-- Path relative to directory that contains .runsettings file-->
<ResultsDirectory>.\TestResults</ResultsDirectory>
Related
I have just recently started using Continuous Integration with VSTS and I have set up a pipeline which includes the "Visual Studio Test" Task.
Within the task, there's an option to execute a code coverage scan as part of the test.
To ensure that my code coverage only covers MY code and have a) created a .RunSettings file to include only the assemblies I generate and b) there are some parts of the code with the [ExcludeFromCodeCoverage] attribute.
Now, when execute Analyze Code Coverage from Visual Studio (2017 Enterprise, 15.7.4), everything works as I would expect, only my assembly gets analyzed and the code I have excluded is, errr, excluded.
However, when the VSTS pipeline is run, no such limitations are applied and ALL assemblies are tested and ALL code, including the specifically excluded code, which then results in a dramatic drop on the code-coverage %age.
Not sure where to go from here so hoping the S/O community can help.
Update - RunSettings file in use
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- Configurations for data collectors -->
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
<Configuration>
<CodeCoverage>
<ModulePaths>
<Include>
<ModulePath>.*\.dll$</ModulePath>
</Include>
<Exclude>
<ModulePath>.*Tests.dll$</ModulePath>
<ModulePath>.*moq.dll$</ModulePath>
</Exclude>
</ModulePaths>
<UseVerifiableInstrumentation>False</UseVerifiableInstrumentation>
</CodeCoverage>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
There is an issue here: RunSettings file not used in TFS VsTest task, you can go and check the discussion about this issue.
As a workaround you can use below .RunSettings formats (add the UseVerifiableInstrumentation = False)
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
<Configuration>
<CodeCoverage>
<UseVerifiableInstrumentation>False</UseVerifiableInstrumentation>
<ModulePaths>
<Include>
<ModulePath>.*\\MyProjectName.*\.dll$</ModulePath>
</Include>
<Exclude>
<ModulePath>.*Tests.dll$</ModulePath>
</Exclude>
</ModulePaths>
</CodeCoverage>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
</RunSettings>
Just check acesiddhu's explanation:
you are using visual studio test tools installer task in your
definition. in case that task is used useverifiable property needs to
be set to false because in xcopy mode we don't GAC the above dll
'Microsoft.VisualStudio.CodeCoverage.Shim (this is a dependency which
needs to be loaded when useverifiable is set to true)
marking this property false ensures it doesn't use this particular
dll.
If I start a new web api project, the appsettings files are grouped together. However, I'm creating a working project from the console app template and when I create the appsettings files manually, the do not group together. I think back in older versions, there was something I'd put in the csproj file, but I don't know how to do it in .net core and I'm not seeing anything in properties or configurations
In the project file of your solution you can edit or add an <ItemGroup> element within the <Project> element. This worked for me:
<ItemGroup>
<Content Include="appsettings.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>
<Content Include="appsettings.*.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<DependentUpon>appsettings.json</DependentUpon>
</Content>
</ItemGroup>
Please Note that my console project targets .Net Core 2.0 and is running in Visual Studio Pro 2017 Version 15.7.5.
Also, if your Solution Explorer doesn't immediately refresh try unloading and reloading the project.
Using an <ItemGroup> with <Content> as suggested gave me an error (in Visual Studio 2019) about "Duplicate 'Content' items included". It turns out the .NET SDK includes 'Content' items from your project directory by default. Setting the EnableDefaultContentItems property to false seems a bit rigid, so now I include the items as <None>.
<ItemGroup>
<!-- Group AppSettings in Console project. Use None to prevent "Duplicate 'Content' items were included" when using (default) EnableDefaultContentItems=true -->
<None Include="appsettings.*.json">
<DependentUpon>appsettings.json</DependentUpon>
</None>
</ItemGroup>
This does show the files grouped, but shows their properties with Build Action 'None' and 'Do Not Copy' in the Solution-explorer, so I guess that's the price for wanting them to group?
FWIW: a file-nesting rule as suggested in appsettings-json-not-in-hierarchy
will not show the files as grouped/nested, but it will make it collapse if the solution-explorer collapse-button is pressed.
You just need to click the File Nesting icon, and choose "Web"
In my C++ project Visual Studio (2017) fails to detect changes in a property value when I trigger a "build" (e.g. via F5 requesting a debugger start).
I want to be able to use the property pages dialog in visual studio to specify a path variable to another library.
The simplified versions are here. First I use a .targets file:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<PropertyPageSchema Include="$(MSBuildThisFileDirectory)\overrideSettings.xml" />
</ItemGroup>
<PropertyGroup Condition="'$(myDir)' == ''">
<__my_IncludeDir>somewhere\include\</__my_IncludeDir>
</PropertyGroup>
<PropertyGroup Condition="'$(myDir)' != ''">
<__my_IncludeDir>somewhere_else\include\</__my_IncludeDir>
</PropertyGroup>
<ItemDefinitionGroup>
<ClCompile>
<AdditionalIncludeDirectories>$(__my_IncludeDir);%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
<ResourceCompile>
<AdditionalIncludeDirectories>$(__my_IncludeDir);%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ResourceCompile>
</ItemDefinitionGroup>
</Project>
And an .xml file for the settings:
<?xml version="1.0" encoding="utf-8"?>
<ProjectSchemaDefinitions xmlns="clr-namespace:Microsoft.Build.Framework.XamlTypes;assembly=Microsoft.Build.Framework">
<Rule Name="ProjectSettings_NugetDevOverride" PageTemplate="tool" DisplayName="Nuget Development Override" SwitchPrefix="/" Order="1">
<Rule.Categories>
<Category Name="myTest" DisplayName="myTest" />
</Rule.Categories>
<Rule.DataSource>
<DataSource Persistence="UserFile" ItemType="" />
</Rule.DataSource>
<StringProperty Name="myDir" Category="myTest" />
</Rule>
</ProjectSchemaDefinitions>
Now, this does work to some extent.
However, Visual Studio fails to correctly detect changes in the property variable defined on the property page.
I build the project -> 1 succeeded
I build the project again (F5) -> 1 up-to-date
I change the variable myDir using the property page; making it empty or setting a value, so that the other property group in the .targets file is triggered.
I build the project again (F5) -> 1 up-to-date This is wrong!
I re-build the project -> 1 succeeded with correctly used new property value.
Where is the problem in my setup? Can I explicitly add myDir as property to be checked before the build is marked as up-to-date?
I found one workaround: DisableFastUpToDateCheck
https://stackoverflow.com/a/36004494/552373
But this is horrible, since my real project is very large and the fast-up-to-date-check really helps.
Update 20171128:
I now also tried the following:
I build the project -> 1 succeeded
I build the project again (F5) -> 1 up-to-date
I change the variable myDir using the property page; making it empty or setting a value, so that the other property group in the .targets file is triggered.
I close Visual Studio and open it again. No file-save dialog popped up.
I build the project again (F5) -> 1 up-to-date This is wrong!
The fact, that even this does not work, points at a real problem with the FastUpToDateCheck in Visual Studio.
Does anyone have a further idea?
Goal
I want to be able to run my unit tests remotely on another machine since they interact with the UI of another application. For clarity these tests are not Coded UI Tests, they are tests methods that use FlaUI to interact with the desktop.
Problem
I can't get the Visual Studio Test Controller and Test Agent to work with MSTest V2. When I set the .runsettings file to use the .testsettings file and to ForcedLegacyMode like the documentation says here I get the following warnings and no tests are loaded into the test explorer.
[11/22/2017 9:54:12 AM Warning] Index was outside the bounds of the array.
[11/22/2017 9:54:13 AM Warning] Index was outside the bounds of the array.
[11/22/2017 9:54:13 AM Warning] Index was outside the bounds of the array.
[11/22/2017 9:54:14 AM Warning] Warning : A testsettings file or a runsettings with a ForcedLegacyMode set to true is not supported with the MSTest V2 Adapter.
I am hoping I am just missing some setting I can put into my .runsettings file that will allow me to specify the url for my Test Controller.
Settings
Here are my .runsettings and .testsettings files for reference. These settings successfully connect to the machine but when I build my test runner no longer finds and tests to run.
.runSettings
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- Configurations that affect the Test Framework -->
<RunConfiguration>
<MaxCpuCount>1</MaxCpuCount>
<!-- Path relative to solution directory -->
<ResultsDirectory>.\TestResults</ResultsDirectory>
<!-- [x86] | x64
- You can also change it from menu Test, Test Settings, Default Processor Architecture -->
<TargetPlatform>x86</TargetPlatform>
<!-- Framework35 | [Framework40] | Framework45 -->
<TargetFrameworkVersion>Framework40</TargetFrameworkVersion>
</RunConfiguration>
<!-- Configurations for data collectors -->
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
<Configuration>
<CodeCoverage>
<ModulePaths>
<Exclude>
<ModulePath>.*CPPUnitTestFramework.*</ModulePath>
</Exclude>
</ModulePaths>
<!-- We recommend you do not change the following values: -->
<UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
<AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
<CollectFromChildProcesses>True</CollectFromChildProcesses>
<CollectAspDotNet>False</CollectAspDotNet>
</CodeCoverage>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
<!-- Parameters used by tests at runtime -->
<TestRunParameters>
</TestRunParameters>
<!-- Adapter Specific sections -->
<!-- MSTest adapter -->
<MSTest>
<MapInconclusiveToFailed>True</MapInconclusiveToFailed>
<CaptureTraceOutput>false</CaptureTraceOutput>
<DeleteDeploymentDirectoryAfterTestRunIsComplete>False</DeleteDeploymentDirectoryAfterTestRunIsComplete>
<DeploymentEnabled>False</DeploymentEnabled>
</MSTest>
</RunSettings>
.testsettings
<?xml version="1.0" encoding="UTF-8"?>
<TestSettings name="rmoqa01" id="076be28c-d18b-46bf-ad20-4d43ec821ea4" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<Description>These are default test settings for a local test run.</Description>
<RemoteController name="10.2.0.101" />
<Execution location="Remote">
<Hosts skipUnhostableTests="false">
<VSSDKTestHostRunConfig name="VS IDE" HiveKind="DevEnv" HiveName="15.0_c9b36733" xmlns="http://microsoft.com/schemas/VisualStudio/SDK/Tools/IdeHostAdapter/2006/06" />
</Hosts>
<TestTypeSpecific>
<UnitTestRunConfig testTypeId="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b">
<AssemblyResolution>
<TestDirectory useLoadContext="true" />
</AssemblyResolution>
</UnitTestRunConfig>
</TestTypeSpecific>
<AgentRule name="AllAgentsDefaultRole">
</AgentRule>
</Execution>
<Properties />
</TestSettings>
I opened a issue on the mstest github page and after looking at the source code generating my warning I am seeing it looks like there is probably no work around to this. Here is the source code I was looking at in the MSTestDiscover.cs.
// Scenarios that include testsettings or forcing a run via the legacy adapter are currently not supported in MSTestAdapter.
if (MSTestSettings.IsLegacyScenario(logger))
{
return;
}
EDIT: 10/28/2018
The issue I linked to above was updated with the following response.
There are no plans to have remote execution capability through MSTVest2. You can use the test task in VSTS which supports distributed execution on multiple agents. https://learn.microsoft.com/en-us/vsts/pipelines/test/set-up-continuous-test-environments-builds?view=vsts
I am trying log some entries in a log file (Log.Trace / Log.Debug) while my VS unit test runs. I have also copied the file NLog.config to out directory through DeploymentItem attribute over class. Still my log file is not created. Any help as to how can we log entries in a file same as we do for normal web application.
I've just found a solution to this problem:
Add the App.config file to yout unit test project with the following contents:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<configSections>
<section name="nlog" type="NLog.Config.ConfigSectionHandler, NLog"/>
</configSections>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<targets>
<target name="debugLog" xsi:type="Console" />
</targets>
<rules>
<logger name="*" minlevel="Debug" writeTo="debugLog"></logger>
</rules>
</nlog>
</configuration>
You may put any configuration you wish as with NLog.config file.
Unfortunately that is the only XML solution at the time. It is not only about Unit Testing, though. NLog.config does not work with just any Console Application.
Do not know if it is by design or just an oversight. Anyway I would not say that moving NLog.config to App.config section is any way satisfactory =/
Maybe it is worth to notice that there is a possibility of configuring nlog directly from code, what could be helpful in some scenarios.
One could be also glad to find nlog debug option, that will log the whole process of processing configuration file, registering targets and so on...
To turn it on, just add internalLogFile and internalLogLevel attributes to nlog element:
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" internalLogFile="C:/logs/nlog.txt" internalLogLevel="Debug">
or from code:
InternalLogger.LogFile = #"c:\logs\nlog.txt";
InternalLogger.LogLevel = LogLevel.Trace;