How to use Jsoup Document method [duplicate] - cookies

Question

This question already has answers here:
How to call a method in another class of the same package?
(8 answers)
Closed 4 years ago.
I am new and beginner in Java world. I have this code
public class Test2 {
public static void main(String[] args) throws IOException {
try {
String url = "http://www.metalbulletin.com/Login.html?ReturnURL=%2fdefault.aspx&";
String articleURL = "https://www.metalbulletin.com/Article/3838710/Home/CHINA-REBAR-Domestic-prices-recover-after-trading-pick-up.html";
Connection.Response loginForm = Jsoup.connect(url)
.method(Connection.Method.GET)
.execute();
Document welcomePage = loginForm.parse();
Element formElement = welcomePage.body().getElementsByTag("form").get(0);
String formAction = formElement.attr("action");
Elements input = welcomePage.select("input[name=idsrv.xsrf]");
String securityTokenValue =input.attr("value");
Connection.Response mainPage = Jsoup.connect("https://account.metalbulletin.com"+formAction)
.data("idsrv.xsrf", securityTokenValue)
.data("username", "ifiih#rupayamail.com")
.data("password", "Kh457544")
.cookies(loginForm.cookies())
.method(Connection.Method.POST)
.execute();
Map<String, String> cookies = mainPage.cookies();
System.out.println("\n\nloginForm.cookies()==>\n"+loginForm.cookies());
System.out.println("\n\nmainPage.cookies()==>\n"+mainPage.cookies());
Document articlePage = Jsoup.connect(articleURL).cookies(cookies).get();
Element article = articlePage.getElementById("article-body");
Elements lead1 = article.getElementsByClass("articleContainer");
System.out.println("\n\nNews Article==>\n"+lead1);
} catch (IOException e) {
e.printStackTrace();
}
}
}
How can I refactor to this:
private Map<String, String> cookies = new HashMap<String, String>();
private Document get(String url) throws IOException {
Connection connection = Jsoup.connect(url);
for (Map.Entry<String, String> cookie : cookies.entrySet()) {
connection.cookie(cookie.getKey(), cookie.getValue());
}
Response response = connection.execute();
cookies.putAll(response.cookies());
return response.parse();
}
I am not sure as to how I can call this private Document get(String url) method. It may seems to be stupid question but very important for me.
How can I call it within same class?

Answer 1

to do that, the easiest and more efficient solution to retrieve the Document and also the Map of Cookies, would be to create a new class called TestThreadHandler as it follows:
public class TestThreadHandler implements Runnable {
private String url;
private Document doc;
private Map<String, String> cookies;
private Semaphore barrier;
public TestThreadHandler (String url, Document doc, Map<String, String> cookies, Semaphore barrier) {
this.url = url;
this.doc = doc;
this.cookies = cookies;
this.barrier = barrier;
}
public void run () {
try {
Connection connection = Jsoup.connect(this.url);
for (Map.Entry<String, String> cookie : this.cookies.entrySet()) {
connection.cookie(cookie.getKey(), cookie.getValue());
}
Response response = connection.execute();
this.cookies.putAll(response.cookies());
this.doc = response.parse();
} catch (IOException e) {
e.printStackTrace();
}
this.barrier.release();
}
}
And call that Thread from your Test2 class from wherever you want to call it, but a sample call to that Thread would be:
public class Test2 {
public static void main(String[] args) throws IOException {
try {
...
String url = "https://www.google.com";
Document doc;
Map<String, String> cookies = new HashMap<String, String>();
Semaphore barrier = new Semaphore(0);
Thread taskThread = new Thread( new TestThreadHandler(url, doc, cookies, barrier) );
taskThread.start();
barrier.acquireUninterruptibly(1); // Wait until Thread ends
// NOW YOU HAVE BOTH DOC AND COOKIES FILLED AS DESCRIBED IN TestThreadHandler
...
} catch (IOException e) {
e.printStackTrace();
}
}
}
Doing that you can overwrite the variables you are passing as argument to the Thread and get both the Cookies and the JSOUP Document.
For further explanation check the Java doc of ThreadHandling or feel free to ask me!
Hope this helped you! +1