We use Jenkins as our CI, and im looking to have Jenkins run visual studio tests. Ive figured out how to have Jenkins use vstest.console.exe for desktop applications, but im not sure how to get that to work for windows phone 8.1 tests. I'm running vstest.console.exe against my unit test project's appx file, but im getting:
Error: App package ...appx does not has test executor entry point. For
running unit tests for Windows Store apps, create app package using
Windows Store app Unit Test Library project.
Any help getting the program to work thorugh command line would be of big help, but if there is a short cut to run it through jenkins that would be even better.
I found an answer to my problem. Make sure to use /InIsolation, and use /Settings:{settingsFile}. For some reason vstest.console.exe cant run the unit tests for windows phone without specifically setting a settingsfile. In jenkins/vsTestrunner there is an option to set the settings file. You will likley still get an error regarding running the test in non-interactive way on jenkins, but there are a few stackoverflow posts about this already.
Related
I have a C++ project that I am attempting to build and run using Azure Pipelines.
The build is working fine but I can't seem to get the unit tests to run in VSTest task using GoogleTest.
It was my understanding Microsoft.VisualStudio.Component.VC.TestAdapterForGoogleTest is installed (as per https://github.com/actions/virtual-environments/blob/main/images/win/Windows2019-Readme.md) so it can't be that.
Looking at the log I see:
No test is available in D:\a\1\s\Release\UnitTests.exe. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
I also attempted a Command Line Script task but running the UnitTests.exe just yields:
'UnitTests.exe' is not recognized as an internal or external command,
operable program or batch file.
I solved this in the end by installing the Nuget package "GoogleTestAdapter" by Christian Soltenborn. All now seems to work fine but still very puzzled why this workaround is needed
My Universal Windows Application now is able to deploy and I try to associate my gitlab CI service to it,But failed to execute the test part automatically with MSTest commandline, which is separated from UWP and work as an Unit Test Application.
I had found so many solutions,telling me such as: Package my app Firstly before using the vstest.console command line to test(Cannot continuous integrate I guess), Making my Test APP and use the project's .dllfiles ,as the testcontainer, to test(Actually I cannot found out the .dll file)
What should I do next If I want to integrate APP with automatically testing part?
Here is the environment:
Windows 10 OS
Visual Stdio 2017(Includes Windows_10_14393 SDK)
GitLab + Windows Runner(same environment as PC)
Test Execution Command Line Tool Version 15.0.27309.0
And my DOM in Solution:
--+-- APP Solution
------+---APP1(University Windows Application)
------+---TestAPP1(Unit Test Application)
You have to just build the test pan p in Release mode and use the .appxreciepe as argument to run vstest.console. no need to build an appx package
It is not really an issue I would like to ask today but I search best practices to unit testing a UWP application with Jenkins.
First, I created a Unit Test App for my main application and, on my development machine, I generated appx package using MSBuild command line tool. It works well and I can also execute unit tests with the VS Test console.
Then I tried to automate it with Jenkins on my build server. But the MSBuild command does not work. I have no issue but it produces no output. After lot of research (without any success), my question is the following:
What is the best way to make UWP unit testing with a result dashboard using Jenkins?
Thanks
In order to make MSBuild work correctly, I had to create two subsequent build steps, one with command line argument:
-t:restore
in order to make it restore all the nuget dependencies, and a second one with command line arguments:
/t:Rebuild /p:Configuration=Release
/p:UapAppxPackageBuildMode=StoreUpload
for the real release compilation and the creation of both sideload and storeUpload msix files
I'm getting the following error on a TFS2013 build:
TF900547: The directory containing the assemblies for the Visual Studio Test Runner is not valid ''.
Looking here, it would seem that the answer to this is to change the test runner from VS test runner to MSTest. However, when I try to edit the Test Run in the build definition, the test runner option is set to Visual Studio Test Runner and greyed out. What determined what is shown in this box, and how can I add MSTest?
I have installed VS2012 agents for handling tests on the server.
I fixed this by installing VS2013 on the build agent
We had the same problem as the OP but, because the TFS server is considered a production box (other departments use it for other things), we were not permitted to install VS2013 on the server. The same applied to any third party tools needed for builds, they were a pain to integrate into the build.
I eventually worked round this by using another dev box to act as a build server for TFS. By installing the Build Server component of TFS on the dev box and setting up a controller and build agents on that box, TFS could pass the responsibility for the actual build to the dev box, on which we could install anything, without compromising the TFS server itself.
While it is not perfect - an extra box and VS2013 license needed - it is by far the easiest way to avoid poluting a production TFS box. Alternatively, nominate a developer's own box and install the build agent there - the environment is ready built, it will simply slow down a little during a build.
Look at section Define Multiple Test Runs of this article. It is very informative and explains different configuration in dept. Hope this helps !!!
I'm wondering if anyone has any best practices for automating the testing of installers on various machines with potentially different hardware / software profiles and by specifying various options to the installer. The idea would be that I could write "unit test like" code to set up a machine, run the installer, then test that certain things are true. Tests might look similar to:
Test:
Boot Machine without IIS
Run Installer
Assert Installer Had Errors
Test:
Boot Machine with IIS
Run Installer
Assert Installer Ran
Test_Fixture:
SetUp:
Boot Machine with IIS
Test:
Run Installer without IIS install
Assert Website Not Installed
Test:
Run Installer with IIS install
Assert Website Installed
I know I could create lots of VMs, but waiting for a VM to boot for each functional test sounds like way more work than I want. What I really want is a way to virtualize the installer environment. Any suggestions?
We have created a set of VMs and find it is very easy to manage. We run the tests for 13 different Windows installers over night. The VMs we have created our very bare bones, so it is possible to run a number of tests in parallel.
If you have the installer runnable from the command line, it's easy to have a script to call it automatically.
Then you can use a web app testing tool to see it the install was successful, like this one http://seleniumhq.org/ For this you will need an unique way to test a new install - like a page with the current version.