MsTests : Running selective unit tests with specific DataRow Value - unit-testing

Here is an example of my tests (integration tests for all my environnements) :
[DataTestMethod]
[DataRow("DEV")]
[DataRow("STAGING")]
[DataRow("PREPROD")]
[DataRow("PROD")]
public void TestMyWebservice(string environnement)
{
}
is there any way to execute tests that for example whith only DataRow value ) "STAGING" etc.. so i can runt tests against one environnement on a time.
There is another solution is to duplicate all tests and give the a category and the run test with filtering in the desired category like this :
[TestCategory("DEV")]
[DataTestMethod]
[DataRow("DEV")]
public void TestMyWebservice(string environnement)
{
}
[TestCategory("STAGING")]
[DataTestMethod]
[DataRow("STAGING")]
public void TestMyWebservice(string environnement)
{
}
and the command to filter :
dotnet test --filter TestCategory=DEV
But i really don"t like it because of duplication and risk to forget some tests in some environnements
Any idea ?
Thanks

I would try first to use the .runsettings method for defining environment variables for running tests: https://learn.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file?view=vs-2019#specify-environment-variables-in-the-runsettings-file
So theoretically, you'd have different files for your different environments:
dev.runsettings
staging.runsettings
preprod.runsettings
prod.runsettings
... which might look something like this (see this answer: https://stackoverflow.com/a/65667284/13864681):
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<RunConfiguration>
<EnvironmentVariables>
<ENV>DEV</ENV>
</EnvironmentVariables>
</RunConfiguration>
</RunSettings>
Then, depending on how you setup your environment in code, you could do something like this in the [AssemblyInitialize] method in your test project(s):
[TestClass]
public class Config
{
// this method will run before any TestMethod in your project
// and can be used to define project-wide configurations
[AssemblyInitialize]
public static void AssemblyInit(TestContext context)
{
var env = Environment.GetEnvironmentVariable("ENV");
// ... use 'env' to set your environment settings
}
}
Then, you can define your tests normally, without specifying their environment.
And finally, run from the command line:
dotnet test --settings dev.runsettings
dotnet test --settings staging.runsettings
dotnet test --settings preprod.runsettings
dotnet test --settings prod.runsettings

Related

How to exclude tagged tests in Gradle Kotlin Build Script?

Having the following task registered in a gradle build script:
tasks.register<Test>("unitTest") {
useJUnitPlatform {
excludeTags("integrationTest")
}
}
And the tests tagged with
import org.junit.jupiter.api.Tag
#Tag("integrationTest")
class MyIntegrationTest {
...
}
./gradlew unitTest still does run the tagged integration-test as well, I tried several other options provided by the gradle documentation to exclude the integration tests, though nothing seemed to work by now.

MarkLogic unit test not running any tests

I thought it would be a good idea to set up MarkLogic unit testing, so I've been following the unit-test-project as best as I can. Everything seems to work, but no unit tests are being performed. I've copied the four unit tests to src/test/ml-modules/root/suites/My Tests and the one module to src/main/ml-modules/root/. Everything deploys correctly. But running the unit tests gives me:
$ gradle mlUnitTest
> Task :mlUnitTest
Constructing DatabaseClient that will connect to port: 8021
Run teardown scripts: true
Run suite teardown scripts: true
Run code coverage: false
Running all suites...
Done running all suites; time: 1791ms
0 tests completed, 0 failed
The unit testing UI loads in a browser, but it also displays no tests.
This is my build.gradle file:
buildscript {
repositories {
jcenter()
mavenLocal()
}
dependencies {
classpath "com.marklogic:marklogic-unit-test-client:1.0.0"
classpath "com.marklogic:ml-gradle:4.1.0"
}
}
plugins { id "com.marklogic.ml-gradle" version "4.1.0" }
repositories {
jcenter()
}
dependencies {
mlBundle "com.marklogic:marklogic-unit-test-modules:1.0.0"
}
And gradle.properties:
mlAppName=unitTestApp
mlHost=localhost
mlUsername=admin
mlPassword=*****
mlRestPort=8020
mlTestRestPort=8021
mlModulePaths=src/main/ml-modules,src/test/ml-modules
I'm using MarkLogic 10, on 64 bit Linux, ml-gradle 4.0
Ah. It looks like the test suites are expected under the project directory in
src/test/ml-modules/root/test/suites/SuiteName
I had put suites under root and missed the second test directory.

Run group testng test through console in gradle

I have some tests(testng annotations):
#Test(groups={"Example"})
public void Test1(){...}
#Test(groups={"Example"})
public void Test2(){...}
#Test(groups={"Example2"})
public void Test3(){...}
And in build.gradle file I can run group test next way:
task runTests(type: Test){
useTestNG() {
suites "src/test/resources/testng.xml"
includeGroups "Example"
}
}
But I want to run this "Example" group as parameter through gradle command line like we do with single test gradle -Dtest.single=... test.
Is there a possibility in gradle to do something like gradle -Dtest.groups=Example test ?
Actually you shouldn't use gradlew -Dtest.single=... test, but gradlew test --tests ....
You could easily configure yourself such a possibility by evaluating a project property like includeGroups(project.findProperty('testGroups') ?: 'DefaultGroup') and then you can do gradlew test -P testGroups=Example.

How to run only one unit test class using Gradle

I am new to Gradle. I use Gradle 1.10 and Ubuntu 13.
I want to know if there's any command to execute only one unit test class, similar to testOnly in SBT.
To run a single test class Airborn's answer is good.
With using some command line options, which found here, you can simply do something like this.
gradle test --tests org.gradle.SomeTest.someSpecificFeature
gradle test --tests '*SomeTest.someSpecificFeature'
gradle test --tests '*SomeSpecificTest'
gradle test --tests 'all.in.specific.package*'
gradle test --tests '*IntegTest'
gradle test --tests '*IntegTest*ui*'
gradle test --tests '*IntegTest.singleMethod'
gradle someTestTask --tests '*UiTest' someOtherTestTask --tests '*WebTest*ui'
From version 1.10 of gradle it supports selecting tests, using a test filter. For example,
apply plugin: 'java'
test {
filter {
//specific test method
includeTestsMatching "org.gradle.SomeTest.someSpecificFeature"
//specific test method, use wildcard for packages
includeTestsMatching "*SomeTest.someSpecificFeature"
//specific test class
includeTestsMatching "org.gradle.SomeTest"
//specific test class, wildcard for packages
includeTestsMatching "*.SomeTest"
//all classes in package, recursively
includeTestsMatching "com.gradle.tooling.*"
//all integration tests, by naming convention
includeTestsMatching "*IntegTest"
//only ui tests from integration tests, by some naming convention
includeTestsMatching "*IntegTest*ui"
}
}
For multi-flavor environments (a common use-case for Android), check this answer, as the --tests argument will be unsupported and you'll get an error.
In versions of Gradle prior to 5, the test.single system property can be used to specify a single test.
You can do gradle -Dtest.single=ClassUnderTestTest test if you want to test single class or use regexp like gradle -Dtest.single=ClassName*Test test you can find more examples of filtering classes for tests under this link.
Gradle 5 removed this option, as it was superseded by test filtering using the --tests command line option.
In case you have a multi-module project :
let us say your module structure is
root-module
-> a-module
-> b-module
and the test(testToRun) you are looking to run is in b-module, with full path : com.xyz.b.module.TestClass.testToRun
As here you are interested to run the test in b-module, so you should see the tasks available for b-module.
./gradlew :b-module:tasks
The above command will list all tasks in b-module with description.
And in ideal case, you will have a task named test to run the unit tests in that module.
./gradlew :b-module:test
Now, you have reached the point for running all the tests in b-module, finally you can pass a parameter to the above task to run tests which matches the certain path pattern
./gradlew :b-module:test --tests "com.xyz.b.module.TestClass.testToRun"
Now, instead of this if you run
./gradlew test --tests "com.xyz.b.module.TestClass.testToRun"
It will run the test task for both module a and b, which might result in failure as there is nothing matching the above pattern in a-module.
Please note that --tests option may not work if you have different build types/flavors (fails with Unknown command-line option '--tests'). In this case, it's necessary to specify the particular test task (e.g. testProdReleaseUnitTest instead of just test)
After much figuring out, the following worked for me:
gradle test --tests "a.b.c.MyTestFile.mySingleTest"
For multi modules projects it's necessary to use module's name and buildType:
./gradlew :module_name:testDebugUnitTest --tests com.package_name.TestsClass.*
To run some test method the same command, but with test's name:
./gradlew :module_name:testDebugUnitTest --tests com.package_name.TestsClass.test
Below is the command to run a single test class using gradlew command line option:
gradlew.bat Connected**your bundleVariant**AndroidTest -Pandroid.testInstrumentationRunnerArguments.class=com.example.TestClass
Below example to run class com.example.TestClass with variant Variant_1:
gradlew.bat ConnectedVariant_1AndroidTest -Pandroid.testInstrumentationRunnerArguments.class=com.example.TestClass
Run a single test called MyTest:
./gradlew app:testDebug --tests=com.example.MyTest
You should try to add asteriks (*) to the end.
gradle test --tests "com.a.b.c.*"
In my case, my eclipse java compiler warnings were set too high, and eclipse was not recognizing my class as valid for execution. Updating my compiler settings fixed the problem. You can read more about it here: annotation-nonnull-cannot-be-resolved
This worked for me
Release case:
gradle testReleaseUnitTest --tests testClass
Debug case:
gradle testDebugUnitTest --tests AsyncExecutorTest
You can see de projects with: gradle -q projects and move to the project where is the class to test

CakePHP/Jenkins/Phing - Run all unit tests

I'm in the middle of my first ever stab at setting up Jenkins to build and run unit tests /code coverage with my CakePHP project. So far I have successfully got Jenkins fetching and building automatically from my BitBucket repository - a small victory in itself.
Next thing I want to happen is for the unit tests to run and code coverage reports to be populated.
Here is my build.xml, which is being executed in Jenkins with the (only) build command phing -f $WORKSPACE/build.xml
<?xml version="1.0" encoding="UTF-8"?>
<project name="Consumer Love" default="phpunit">
<target name="phpunit">
<exec command="cake test app --coverage-clover logs/reports/clover.xml"></exec>
</target>
</project>
I think the issue is that when you run cake test app it asks for a prompt of which specific tests you want to run, I have been unable to figure out a method to run all of my CakePHP app unit tests.
The solution was to create a custom CakePHP Test suite which adds specific files/directories to be tested, then run that suite with the command cake test app AllTests.
For example, here is my Test/Case/AllTests.php:
/*
* Custom test suite to execute all tests
*/
class AllTestsTest extends PHPUnit_Framework_TestSuite {
public static function suite() {
$path = APP . 'Test' . DS . 'Case' . DS;
$suite = new CakeTestSuite('All tests');
$suite->addTestDirectory($path . 'Model' . DS);
return $suite;
}
}
This testsuite simply adds the Models directory to the testing environment, so all my model tests now get executed. As you can see it can be extended to run more/all tests as seen fit.
Try cake test app all. I can't confirm this makes the difference just now, but I've pulled this out of a phing build file where I'm doing the same thing as you so it should be good.