I'm looking for a solution to start my tests from command line.
I created a UnitTest Procjet in VisualStudio2017 for my .NET solution.
Added TestStack.White NuGet package to the project.
The test are running fluently when I start from the VisualStudio2017.
I would like to start it from Jenkins also. I think it is the easiest to do it from command line, so I add it to my pipeline configuration (Jenkinsfile)
stage('Run UI Tests') {
steps {
bat('"C:\\PATH_TO_MSTEST\\mstest" /testcontainer:PATH_TO_MY_TEST_PROJECT\\bin\\Debug\\MyTests.dll')
}
}
When I try to start it from cmd like I would do with with regular Unit Tests, it is not working.
It says:
Starting execution...
No tests to execute.
I build the project before I start 'Run UI Tests' stage.
Any ideas how to make it work? Could really find it on stackoverflow, github issues of TestStack nor other glory places on the web
Found a solution.
On my local developer machine it was working, the mstest version was 14
On the build agent machine the mstest version was 15, that was not working somehow (it had nothing to do with TestStack White, simply the unit tests were not working)
What I do is, calling vstest.console.exe instead of the mstest.
C:\Program Files (x86)\Microsoft Visual Studio\2017\TestAgent\Common7\IDE\Extensions\TestPlatform\vstest.console.exe
So, instead of
stage('Run UI Tests') {
steps {
bat('"C:\\PATH_TO_MSTEST\\mstest" /testcontainer:PATH_TO_MY_TEST_PROJECT\\bin\\Debug\\MyTests.dll')
}
}
My command in the Jenkinsfile was:
stage('Run UiTests') {
steps {
bat('"C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\TestAgent\\Common7\\IDE\\Extensions\\TestPlatform\\vstest.console.exe" PATH_TO_MY_TEST_PROJECT\\bin\\Debug\\MyTests.dll')
}
}
nunit3-console is great alternative to MSTEST. Please refer below link.
e.g.
nunit3-console \bin\Debug\Automation.dll --where "cat=Smoke-Tests"
https://github.com/nunit/docs/wiki/Console-Command-Line
Related
Visual Studio Code does not recognize unit tests grouped in group(...). But it recognizes unit tests which are not grouped. I mean they are standalone test(...). I want to use group, because I need setUp().
There is Run | Debug link above group. When I click Debug or Run the Debug console shows No tests match regular expression "^LocalRepository$".
The tests are running correctly with flutter test command.
Dart SDK: >=2.1.0 <3.0.0
Flutter channel: master
Edit: I found workaround - I just don't use group callback. But I can't run all tests by clicking Run above group.
Make sure you name you test file ending with _test.dart and you will see the run | debug options
I have a Flutter project with two separate test files, both ending in "..._test.dart".
Using the following two test scenarios, one of the files passes, while the second doesn't load and that test fails.
Using launch.json configuration:
{
"name": "Dart: Run all Tests",
"request": "launch",
"type": "dart",
"program": "./test/"
}
Using the VS Code menu item Debug > Start Debugging (F5), which I suspicion just uses the above launch.json configuration, yields the same result.
However, using the VS Code menu item Debug > Start Without Debugging (Ctrl+F5), or "flutter test" from the command line, both tests pass.
I suspect this is an internal configuration issue with the VS Code Dart/Flutter extension, as opposed to a problem with Dart/Flutter code structure.
Are your tests/groups spread across multiple files? There are some limitations to the integration between the VS Code extension and the test package that you may be hitting.
There was a fix in v2.23 (released a few days before your question) that should have improved this a little (removing those Run/Debug links in some places they wont' work) but if you're already on that version then maybe it didn't cover your case.
If you can post a small repro on GitHub I'd definitely like to see if I can improve this. Thanks!
Ensure the test has a name different from ''
Change
test('', () {
...
});
to
test('xxx', () {
...
});
I am merely a beginner and still trying to learn about TFS and its continuous integration workflow. Having that said, this could as well be a stupid question to ask as I might be missing on a simple detail, though any help or advice would be highly appreciated.
So, I have a fairly simple Unit Test example written using .NET Core 2.0, which I would like to run as a test task on our TFS Server's CI Build pipeline. It pretty much looks something like this:
using Microsoft.VisualStudio.TestTools.UnitTesting;
namespace MyUnitTest
{
[TestClass]
public class MyUnitTest
{
[TestMethod]
public void PassingTest()
{
Assert.AreEqual(4, Add(2, 2));
}
[TestMethod]
public void FailingTest()
{
Assert.AreEqual(5, Add(2, 2));
}
int Add(int x, int y)
{
return x + y;
}
}
}
When I try to run these tests in Visual Studio, it builds perfectly and the tests succeed and fail accordingly. So I commit my project and push it to our TFS git repository. Now, I similarly would like to integrate these tests in our build pipeline.
The Build Definition used in our CI builds looks like this. I have added Visual Studio Test - testAssemblies task in the build pipeline and configured the search pattern to find the assembly named MyUnitTest.dll and such. When I queue the build, I get the following warning in VSTest's log.
Warning: No test is available in C:\BuildAgent\_work\9\s\MyUnitTest\MyUnitTest\bin\release\netcoreapp1.1\MyUnitTest.dll. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.
So, it seems to me that VSTest somehow cannot find tests to run in the target assembly. I am pretty positive that I might have misconfigured something, or forgotten to set some particular parameter appropriately. I will be more than grateful for any suggestion that would possibly pinpoint what I might be doing wrong.
Searching for solutions online, I have come across this question, which seems to have a similar problem.
First make sure your build agent environment is as same as your local develop machine. Such as Visual Studio version, MsTestAdapter ,xunit runner version and so on.
You could double confirm this by manually run the test directly on the build agent machine not through TFS.
Then use the below tasks in your build pipeline:
Add a dotnet restore task.
Then a dotnet build task.
Add a dotnet test task with the arguments --no-build --logger "trx;LogFileName=tests-log.trx
Add a Publish test results task with the following settings
More details please refer this tutorial blog: Running dotnet core xUnit tests on Visual Studio Team Services (VSTS)
Visual Studio 2015 Enterprise Update 3, all KB package updates are applied
ReSharper 2016.1.2, latest
According to my project.json file xunit references are;
"xunit": "2.2.0-beta2-build3300",
"xunit.abstractions": "2.0.1-rc2",
"xunit.assert": "2.2.0-beta2-build3300",
"xunit.extensibility.core": "2.2.0-beta2-build3300",
"xunit.extensibility.execution": "2.2.0-beta2-build3300",
"xunit.runner.console": "2.2.0-beta2-build3300",
"xunit.runner.visualstudio": "2.2.0-beta2-build1149",
"xunit.runners": "2.0.0"
.Net framework is 4.5.2
.Net Core Solution type which means .xproj
When i try to use ReSharper xUnit test runner, i'm encountering;
"Unit Test Runner failed to run tests, Unable to run xUnit tests -
File not found:"D:\srcs\GitProjects..etc\bin\My.Tests.dll"
Parameter name: assemblyFileName
Sorry for Turkish exception details, i couldn't change yet.
Any idea about that? I searched a lot of forums and i couldn't find any useful stuff. Thanks in advance.
Support for .NET Core testing in Resharper is scheduled for 2016.3
The workaround is to use Visual Studio Test Explorer. (Test --> Windows --> Test Explorer)
The project.json file must contain xunit and dotnet-test-xunit:
{
...
"testRunner": "xunit",
"dependencies": {
...
"xunit": "2.2.0-beta2-build3300",
"dotnet-test-xunit": "2.2.0-preview2-build1029",
},
...
}
Rebuild your project and you should be able to successfully run your tests.
If you have Visual Studio 2015 you can start your test inside the code.
Update: (2017-02-08)
It also works with nunit;
{
"testRunner": "nunit",
"dependencies": {
...
"NUnit": "3.5.0",
"dotnet-test-nunit": "3.4.0-beta-3"
},
...
}
ReSharper doesn't currently support .net core testing. This is scheduled for 2016.3.
By installing the Early Access Program(2016.3) of Resharper, you will be able to run your dotnet core tests.
It supports both nunit and xunit.
I installed it, and it solved the issue for me.
Link to the relase details:
https://blog.jetbrains.com/dotnet/2016/09/26/resharper-ultimate-2016-3-eap-build-1-details/
I have created simple MS Unit Test Project. Assert.Equal(true, true).
When I want to run a test I receive a error from Resharper and Visual Studio.
"Unit test Runner failed to run tests
System.Xml.XmlException: An Error occured while parsing EnityName. Line 1, position 17."
Do I missing something.
Edit:
I can't add screenshot and I can't select text to copy. Ok I will write it the best I can.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.xml.XmlTextReaderImpl.Throw(String res,String arg)
at System.Xml.XmlText.ParseEnityName()
at System.Xml.XmlTextReaderImplparseEnityReference
at System.Xml.XmlTextReaderImpl.Read()
at System.Xml.XmlTextLoader.ParsePartialContent(XmlNode parentNode,String innerxmltext, xmlNodeType nt)
at System.Xml.XmlElement.set_InnerXml(string value)
at Microsoft.VisualStudio.TestPlatform.Utilities.InferSettingHelper.AppendOrModifyChild(XpathNavigator,String nodePatH,String nodeNAME,sTRING INNERxML)
at Microsoft.VisualStudio.TestPlatform.Utilities.InferSettingsHelper.UpdatedRunConfiguration(XPathNavigator, ArchitectureeffectivePlatform,FrameworkVersion effectiveFramework,String resultsDirectory,String solutionDirectory,String binariesRootDirectory)
Microsoft.VisualStudio.TestPlatform.Utilities.InferSettingsHelper.MergeRunSettingsAndFindCompatibleSource()
at JetBrains.ReSharper.UnitTestProvider.MSTest11.MsTest11Runner.Run(IRemoteTaskServer server,String assemblyLocation,String runConfigurationFilename,IUnitTestRun run)
To solve this problem you don't need to reinstall Visual Studio or Resharper.
The problem was in a file path to my projects. This would explains why other Unit Test projects were fine because there were in a different location.
I guess this a bug inside Visual Studio Unit Testing Framework. If you have unit test project inside folders with special charts like '&' then Visual Studio will crash or Resharper will return this message.
I wonder where I can report this bug.
I have just run into this problem too and followed the advice above.
First my path had a folder named ...\m&e...
Then I copied the whole project folder to a new location without an '&' in the path.
After that everything was working well.
Just installed Android Studio to give it a try. I'm trying to create a test module for my application and, whatever the method I use, I always get the following error: "Error: No AndroidManifest.xml file found in the main project directory...".
Methods I use:
1) from Android Studio: New > Module > Test Module
2) from command line: android create test-project -m "main_path" -n "project_name" -p "test_path"
Any ideas on how to solve this?
Edit: As of 0.1.8 this is now supported in the IDE.
According to the documentation for the new build system a separate project is no longer needed, which implies that a separate module may not be either. That said, getting test working within the IDE is still problematic. I've posted an answer here that at least works for running tests from the command line.