I have a Java project with more than 20 sub projects, which have more than 70 Junit tests among them, and I have now been tasked with dividing them up in to two test suites depending on test performance (speed).
Is there a way to create a task or a suite in gradle where I can cherry pick tests from the sub projects and have another suite run "all the rest"?
I have a main project that run all unit tests with the 'test' command and where I'm trying to create the task/suite.
My idea is tagging the slow tests with a tag that the 'slowTests'-tag/suite include to run and a 'fastTests' that excludes them?
(I am unable to move the test classes, but I can add things like tags.)
I have tried creating tasks and suites in gradle https://docs.gradle.org/current/userguide/jvm_test_suite_plugin.html#sec:declare_an_additional_test_suite but I'm unable to make anything work.
Related
Supposing I have system tests: A and B, where A includes a record to a database, B tries to modify it. When A fails, B will fail as well. A and B are written as "unit tests" (test methods), A and B are tests cases in TFS as well, automated, linked to these "unit tests". I put them on a test plan, test suite, both of them. I want to execute them with the "Run Functional Tests" step.
How can I tell TFS to execute them in the right order?
What is the best practice to develop tests like these?
You could created an ordered test, which is a container that holds other tests and guarantees that tests run in a specific order.
How to create an ordered test, you could refer this tutorial.
In TFS, you could follow below steps to run ordered test:
Add an Order Test file in your test project and use it to define the
testing order.
In your build definition, add a Run Functional Tests task. Change
the Test Assembly like the picture below.
In the Test Drop location I have the complete Project, and in the
Executions folder I have the ordered test. hope it helps
Update
It's able to order for manual tests, however not able for automated test. If you need the ordering support for automated tests, please vote on this user voice item:
enable changing the order of test cases on the web gui and let them be tested in this order for automated tests
https://visualstudio.uservoice.com/forums/330519-team-services/suggestions/13489221-enable-changing-the-order-of-test-cases-on-the-web
We've customized a product which includes their own phpunit test suite. In Jenkins, I have two jobs setup: the first runs our own test suite that covers our customizations, and the second job runs the existing core unit tests.
The core unit tests were not designed to be run on a customized version, so failures are expected. Out of the ~5000 tests, 81 fail. What I'd like to setup in Jenkins, is have the build marked as a failure only if the number of failed tests changes from the previous build.
I've looked at the Performance plugin but the documentation seems sparse and I'm trying to find something that matches our use case.
Any suggestions?
You should have a look at the plugin https://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin
It handle a threasolding mechanism (I specified this requirement for the xunit plugin when my team developed it )
expect this helps..
But you want to associates the failure to a change ....
Hum maybe more complex .. have to ask .. if such thing should be developped.
I have implemented unit tests for my MVC Application using the NUnit Framework.
The Unit testing project contains multiple [TestFixture] classes that need to be tested. I'm using a TestFixture for each Module in my MVC project, but all of the modules are inside in single Namespace.
I would like to be able to test a single Module through its [TestFixture] using the Unit Tool in FinalBuilder (automation) or Manually, instead of testing the whole unit test project.
My total test case count is around 2000, made up of 20 modules each with about 100 test cases. Any changes made to one of modules or it's TestFixture will mean that only that modules tests need to be run. This will minimise the amount of time that needs to be taken to wait for unrelated tests to complete.
As far as manual testing goes, in the Nunit gui, you can simply select a test fixture and run it to. I haven't used finalbuilder, but assuming it's similar to the nunit-console application, you have a couple of options.
You can pass arguments to specify particular test cases, for example:
nunit-console some.dll /run=SomeTestFixture
Or, you can mark up your test fixtures with categories and then tell the test runner to only include tests in those categories, so in your code:
[TestFixture]
[Category("SomeTestCategory")]
public class SomeTestClass {
Then in your nunit call:
nunit-console some.dll /include=SomeTestCategory
That said, assuming that your tests are actually unit tests, 2000 doesn't seem like that many and they shouldn't really be taking all that long to run...
We have a large Hybris project here and to run all the tests with takes much too long (hours, yes, a large consulting company created that crap). My target is to reduce all the spring based integration tests and replace them by real unit tests.
But when running the tests with the Hybris ant build for one extension (ant alltests -Dtestclasses.extensions=myext) starts a server with the junit tenant also if there are only non Spring based unit tests in that extension. I also tried to use ant unittests but that one does not even executes my tests.
Is there any way to run only the tests annotated with #UnitTest without any server start in an ant run?
PS: I have a hybris 5.1 and 5.3 commerce suite
You should use ant unittests and not ant unit tests:
ant unittests -Dtestclasses.extensions=myext
Note
Running simple unit tests exclusively is not so easy whenever someone uses somewhere Registry.getApplicationContext() in the code under test!
In fact, Registry.getApplicationContext() starts a Hybris instance. If that happens to you, you need to eliminate that particular call to Registry.getApplicationContext() with a better class design and/or mocks.
This is good information. However, in my opinion, even running the unit tests for a single extension is still too much. Unit tests are supposed to be FAST! I should be able to run a single unit test method from within my IDE if I choose to. The whole concept of "red-green testing" is lost if I have to wait for a bunch of non-relevant unit tests to run every time I want to test my refactored code.
Because these tests rely on a runtime environment, there are NO unit tests in Hybris. There are only integration tests because they all rely on a running Hybris system to be executed.
I would like to give some details how to run unittests from within the IDE.
Install IntelliJ
Install Hybris plugun (https://plugins.jetbrains.com/plugin/7525-hybris-integration)
Import the project
Run the UnitTest as any normal developer will do it
Enjoy :)
I have a problem with executing tests in JUnit. Imagine you have one test case class with f.e. 100 tests, no test suite and no main program - test case class test the device on com port. JUnit project is in Netbeans. I want to run tests - but not all of them at the same time, i would like to choose tests to run before actual testing.
Once I saw something like that in eclipse - but it wasn't my project and I don't know how it was done and how to do the same thing in netbeans. It was a separate window, poping up before running tests. In this window there were checkboxes with names of methods with #Test annotation and you could choose tests you wanted to run and click run - so it let you to run what you wanted.
Does anyone know how to do it in netbeans? Is it any library or plugin?
Any help will be appreciated.
You can take a look at Run single test from a JUnit class using command-line. It does allow you to specify what test you want to run given a class with multiple test cases in it. Being command-line you can then script your own test suite that runs the specific ones you want.
I also noticed your other question Junit: changing sequence of test running. With the scripting approach you can actually control the order of your testing.
This approach does not take advantage of Eclipse's or NetBean's JUnit test runners though, so it is a very specific workaround.
Netbeans nowadays support running single tests: