TestNG #Factory and group-by-instances and preserve-order - unit-testing

Question

I'm trying to use for the first time TestNG with #Factory and for me doesn't work, I'll say you why.
I have a class called Extend in which I have some tests, "launch site", "login", "check if the useris in his own dashboard" and so on and I wanted that for all datas passed from the factory the order of theese test are always the same "launch site">>"login">>"check user is in his dashboard">>"logout" ok? So I have the following extend.xml file and classes:
<suite name="ExtendFactory" group-by-instances="true">
<test name="Factory" preserve-order="true" group-by-instances="true">
<classes>
<class name="net.whaooo.ExtendFactory">
<methods>
<include name="launchSite"></include>
<include name="loginTest" />
<include name="userIsInHisOwnDashboardTest" />
<include name="logoutTest" />
</methods>
</class>
</classes>
</test>
</suite>
Extend class:
public class Extend extends BaseTest{
protected static FirefoxDriver driver;
private String a_driver;
private String password;
public Extend(String a_driver, String pwd){
this.a_driver = a_driver;
this.password = pwd;
}
#BeforeTest
public void stDriver() {
DesiredCapabilities caps = DesiredCapabilities.firefox(); caps.setCapability(CapabilityType.ForSeleniumServer.ENSURING_CLEAN_SESSION, true);
driver = new FirefoxDriver(caps);
driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
}
#AfterTest
public void stopDriver() {
driver.close();
}
#Test
public void launch() {
launchSite(driver);
}
#Test (description = "Enter a valid login as driver")
public void loginTest() {
login(driver, a_driver, password);
}
#Test (description = "Check the driver is in his own dashboard")
public void userIsInHisOwnDashboardTest(){
userIsInHisOwnDashboardTest(driver, a_driver, password);
}
#Test(description="logout")
public void logout(){
logoutTest(driver);
}
}
Semplified Factory:
public class ExtendFactory {
#Factory
public Object[] createInstances() {
Object[] result = new Object[2];
result[0] = new Extend("test1#test.com","tester");
result[1] = new Extend("test2#test.com","tester");
return result;
}
}
But my problem is that the order in which the tests are launched doesn't follow the one specified in the xml file even if I insert the clause preserve-order="true" group-by-instances="true", I tryed also with order-by-instances="true". Can anyone help me?

Answer 1

I see many issues... first of all #Factory with group-by-instance="true" messes up whole test (it executes just one instance and only non-dependent methods).
#Factory works without group-by-instance but it executes all non-dependent methods first irrespective of number of instances. Eg.. Class A {#Test public void a() {} #Test(dependsOnMethod="a") public void b() {}}... along with #Factory that returns two instances.. then the execution is ref1.a, ref2.a, ref1.b, ref2.b. this has serious issue.. say class A uses large amount of memory then sure it will run out before executing all.
ps: not sure if it is eclipse issue. I am using testng 6.8.1
ps2: seems like testng intends for regression.. but it is still not there.. nor its regression (#Factory) is supported by its own classes (like #Listeners who will read only #Parameters.. but #Factory cannot set same) or by third party.

Answer 2

I think what you need to use is dependsOnMethods in your testcases, coz the flow that you mention, if the first method doesn't execute, there is no point in executing the second testcase. i.e. if "launch site" fails, there's no need to execute "login". This would also ensure order of execution. Check out Dependent Methods

Answer 3

I've been using the
#Test(dependsOnMethods = "TestName")
Where "TestName" is the prerequisite test to run. So for your login test, it should have the following annotation:
#Test(dependsOnMethods = "launchSite")
I'm running 9 tests in a row, and since adding the dependsOnMethods, all have ran in order with no issue

Answer 4

Thank you for your answer, I ended up to use a #Factory specifing "order-by-instances="true"" and than in the dynamic object I insert the dependencies!

Answer 5

Using depends in the TestClass file is not a solution as the functions which are not dependent on any other functions are still been executed randomly.
I need to execute the Test Cases in the order which i have mentioned. This can be achieved using "preserve-order" when executed using TestNG but it fails when grouping is used in TestNG.
If anyone can help in this concern, please revert.