Skipping tests using Artos if pre-requisite is not met - unit-testing

Question

I am using Artos runner. In our development environment we keep
<property name="stopOnFail">true</property> so I can debug my changes without having to deal with dependent test cases failing. In production environment we keep <property name="stopOnFail">false</property> so test execution does not stop upon failure and we can analyse log in the morning.
Now I have a different requirement,
I have some tests that are pre-requisite for rest of the units, so if critical test fails then I would like to skip rest of the unit, otherwise it can put our product into bad state.
Is there a way in Artos to skip rest of the unit only if specific test case or test unit fails?
Or can we perform specific steps, incase test fails to ensure we are safe to perform rest of the tests?

Answer 1

Depending on the requirement there are multiple ways to achieve it in Artos
First of all, ensure all of your units are having a sequence number so they execute in the same order all the time.
Let's say testUnit_1() is critical unit and it must be executed successfully in order to execute rest of the following units. In that case, set dropRemainingUnitsUponFailure = true as shown below. This will ensure that the rest of the units are dropped from the execution list if testUnit_1() fails.
#TestPlan(preparedBy = "user", preparationDate = "19/02/2019", bdd = "GIVEN..WHEN..AND..THEN..")
#TestCase(sequence = 1)
public class TestCase_1 implements TestExecutable {
#Unit(sequence = 1, dropRemainingUnitsUponFailure = true)
public void testUnit_1(TestContext context) {
context.getLogger().info("do something");
}
#Unit(sequence = 2)
public void testUnit_2(TestContext context) {
context.getLogger().info("do something");
}
#Unit(sequence = 3)
public void testUnit_3(TestContext context) {
context.getLogger().info("do something");
}
}
If the test cases that are dependent upon each other then you can do similar at test case level
Ensure TestCases are assigned with sequence number so they follow same execution order (Similar to units)
As shown below. If dropRemainingTestsUponFailure = true and dropRemainingUnitsUponFailure = true then upon testUnit_1() failure, not only rest of the units will be dropped but also remaining test cases will be dropped from execution list so you can achieve clean exit.
#TestPlan(preparedBy = "user", preparationDate = "19/02/2019", bdd = "GIVEN..WHEN..AND..THEN..")
#TestCase(sequence = 1, , dropRemainingTestsUponFailure = true)
public class TestCase_1 implements TestExecutable {
#Unit(sequence = 1, dropRemainingUnitsUponFailure = true)
public void testUnit_1(TestContext context) {
context.getLogger().info("do something");
}
#Unit(sequence = 2)
public void testUnit_2(TestContext context) {
context.getLogger().info("do something");
}
#Unit(sequence = 3)
public void testUnit_3(TestContext context) {
context.getLogger().info("do something");
}
}
In the log file you will see warning
=========================================================================
========== DROP REMAINING UNITS UPON FAILURE IS TRIGGERED ===============
================== REMAINING UNITS WILL BE DROPPED ======================
=========================================================================
and
=========================================================================
========== DROP REMAINING TESTS UPON FAILURE IS TRIGGERED ===============
================== REMAINING TESTS WILL BE DROPPED ======================
=========================================================================
so you will know what happened.
To Answer your second question
(Question: If there is any ways to perform clean up if test unit fails, so before you perform next unit test, you can recover your product from the bad state)
If I understood it correctly then it can be done using annotation #AfterFailedUnit
If you create a method as shown below in your runner class
#AfterFailedUnit
public void globalAfterFailedTestUnit(TestContext context) throws Exception {
context.getLogger().info("This method executes after failed test unit");
}
then it will be executed after each test unit failure, you should implement clean up logic in this method.
Hopefully this answers your questions