How do I save cookies with Qt? - c++

I am trying to save cookies that are produced by my app to disk location such as C:\Users\Username\AppData\Local\MyCompany\MyApp. I have implemented a webview and have pretty much finished coding my simple browser the final thing to do is save cookies.
I am can qDebug() the cookies I get from the webapp and they show the cookies are formed correctly but I am a)unsure where to go from there and b) not 100% sure on how to make a subclass of the cookiejar class?
Below I create my cookiejar object in my MainWindow constructor
view = new QWebView(this);
jar = new QNetworkCookieJar;
view->page()->networkAccessManager()->setCookieJar(jar);
And in my replyfinished slot I can see the cookie contained in the reply and I attempt to save it but nothing happens and I receive no run time errors. There isn't a great deal of stuff out there on this and have seen a few posts where the instruction was to make a subclass QNetworkCookieJar but have not made a subclass in Qt/C++ before.
Is there a simple way to store cookies, I am not looking for anything fancy. The cookies just make sure some check boxes are ticked on the login page.
// SLOT that accepts the read data from the webpage
void MainWindow::slotReplyFinished(QNetworkReply *reply){
if(reply->isFinished()){
QVariant variantCookies = reply->header(QNetworkRequest::SetCookieHeader);
QList<QNetworkCookie> cookies = qvariant_cast<QList<QNetworkCookie> >(variantCookies);
qDebug() << "Cookies reply: " << cookies;
QNetworkCookie cookie; //Create a cookie
jar = new QNetworkCookieJar;
//view->page()->networkAccessManager()->setCookieJar(jar);
jar->setCookiesFromUrl(cookies, reply->request().url());
//qDebug() << "Saved cookies: " << jar->getAllCookies();
}
qDebug() << "Network reply: " << reply->errorString() << reply->error() << reply->request().url();
}

You will need to subclass QNetworkCookieJar and in that class you should implement your own persistent storage.
class MyNetworkCookieJar : public QNetworkCookieJar {
public:
bool saveCookiesToDisk() {
// .. my implementation
return true; // if i did
}
bool loadCookiesFromDisk() {
// .. load from disk
return false; // if unable to.
}
}
The sample application from Qt project implements a persistent cookie store, it could be a good starting point for you: http://qt.gitorious.org/qt/qt/trees/4.8/demos/browser
look at cookiejar.h and cookiejar.cpp

Base of qt example, http://qt.gitorious.org/qt/qt/trees/4.8/demos/browser, i wrote this class that save and use one cookie for me. Perhaps it helps you too. Note that it just save one cookie and not list of cookies.
#include "cookiejar.h"
CookieJar::CookieJar(QObject *parent)
: QNetworkCookieJar(parent)
, m_loaded(false)
{
}
void CookieJar::load()
{
if (m_loaded)
return;
QSettings settings;
settings.beginGroup(QLatin1String("cookies"));
QList<QNetworkCookie> savedCookies = QNetworkCookie::parseCookies(settings.value("cookies").toByteArray());
for (int j = 0; j < savedCookies.count(); j++)
insertCookie(savedCookies.at(j));
m_loaded = true;
emit cookiesChanged();
}
void CookieJar::save()
{
if (!m_loaded)
return;
QList<QNetworkCookie> cookies = allCookies();
QSettings settings;
settings.beginGroup(QLatin1String("cookies"));
settings.setValue("cookies", cookies[0].toRawForm());
}
QList<QNetworkCookie> CookieJar::cookiesForUrl(const QUrl &url) const
{
// This function is called by the default QNetworkAccessManager::createRequest(),
// which adds the cookies returned by this function to the request being sent.
CookieJar *that = const_cast<CookieJar*>(this);
if (!m_loaded)
that->load();
return QNetworkCookieJar::cookiesForUrl(url);
}
bool CookieJar::setCookiesFromUrl(const QList<QNetworkCookie> &cookieList, const QUrl &url)
{
if (!m_loaded)
load();
QNetworkCookieJar::setCookiesFromUrl(cookieList, url);
save(); //Save cookie permanently in setting file.
emit cookiesChanged();
return true;
}

#Musa's answer is good but it only saves one cookie. I recommend using the Qt folk's implementation from the old qmlviewer located here: http://code.qt.io/cgit/qt/qt.git/tree/tools/qml/qmlruntime.cpp?h=4.7#n438
Here's the code:
class PersistentCookieJar : public QNetworkCookieJar {
public:
PersistentCookieJar(QObject *parent) : QNetworkCookieJar(parent) { load(); }
~PersistentCookieJar() { save(); }
virtual QList<QNetworkCookie> cookiesForUrl(const QUrl &url) const
{
QMutexLocker lock(&mutex);
return QNetworkCookieJar::cookiesForUrl(url);
}
virtual bool setCookiesFromUrl(const QList<QNetworkCookie> &cookieList, const QUrl &url)
{
QMutexLocker lock(&mutex);
return QNetworkCookieJar::setCookiesFromUrl(cookieList, url);
}
private:
void save()
{
QMutexLocker lock(&mutex);
QList<QNetworkCookie> list = allCookies();
QByteArray data;
foreach (QNetworkCookie cookie, list) {
if (!cookie.isSessionCookie()) {
data.append(cookie.toRawForm());
data.append("\n");
}
}
QSettings settings;
settings.setValue("Cookies",data);
}
void load()
{
QMutexLocker lock(&mutex);
QSettings settings;
QByteArray data = settings.value("Cookies").toByteArray();
setAllCookies(QNetworkCookie::parseCookies(data));
}
mutable QMutex mutex;
};

Related

Problem with website using QNetworkAccessManager Investing.com

I am trying to read automatically some information from investing.com using QNetworkAccessManager. I can read from other sites but this site gives some webmaster tools which I want to access.
https://www.investing.com/webmaster-tools/
I use this query which works in a browser.
Here is my request code
class InvestingAPI: public QObject
{
Q_OBJECT
public:
InvestingAPI();
QueryTechnicals(QString Symbol, int TF1Minites);
signals:
// void NewTechnicalSummary(int Timeframe, QString Symbol, QString Summary);
private slots:
void onData(QNetworkReply *reply);
private:
QNetworkAccessManager qnam ;
};
InvestingAPI::InvestingAPI()
{
connect(&qnam,SIGNAL(finished(QNetworkReply*)),this,SLOT(onData(QNetworkReply*));
connect(&qnam,SIGNAL(encrypted(QNetworkReply*)),this,SLOT(onData(QNetworkReply*))
);
}
InvestingAPI::QueryTechnicals(QString Symbol, int TF1Minites)
{
QString Query;
Query = "http://ssltsw.forexprostools.com/index.php?timeframe=300&lang=1&forex=1&commodities=8830,8836,8831,8849,8833,8862,8832&indices=175,166,172,27,179,170,174&stocks=334,345,346,347,348,349,350&tabs=1,2,3,4%22%20width=%22317%22%20height=%22467%22%3E%3C/iframe%3E%3Cdiv%20class=%22poweredBy%22%20style=%22font-family:arial,helvetica,sans-serif;%20direction:ltr;%22%3E%3Cspan%20style=%22font-size:%2011px;color:%20&selectedTabId=QBS_1";
QNetworkRequest Request;
Request.setSslConfiguration(QSslConfiguration::defaultConfiguration());
connect(&qnam,SIGNAL(finished(QNetworkReply*)),this,SLOT(onData(QNetworkReply*)));
Request.setUrl(QUrl(Query));
Request.setRawHeader("User-Agent", "MyOwnBrowser 1.0");
qnam.get(Request);
}
And I have event
void InvestingAPI::onData(QNetworkReply *reply){
// find data type
// decode and return data to caller
if(reply->error() != QNetworkReply::NoError){
qDebug() << "Error";
qDebug() << reply->errorString();
}
QString html = QString::fromUtf8(reply->readAll());
qDebug() << html;
QString SubData;
}
I do not get an error but I get an empty string rather than the html response.
Please help as I have no idea why this is not working here but is working in the browser.
By default Qt Network does not handle redirects like other tools, so that is why you get an empty data (if you check the "Location" header you will see the redirected url). In this case it is to enable the redirection:
Request.setAttribute(QNetworkRequest::FollowRedirectsAttribute, true);

How to get data for a custom widget using an async source

I'm new to QT and I would like some help. If any of you could help me I would really appreciate it.
QUESTION:
I have an asynchronous class that makes an HTTP request and it's going to receive some data into a JSON format and from there I will extract the necessary information which should be passed to my custom widget. How can I do that? Because I don't know when the information will arrive.
WHAT I'VE DONE SO FAR:
My HTTP request and parsing JSON class:
WeatherAPI::WeatherAPI(QObject *parent) : QObject(parent) {
manager = new QNetworkAccessManager(this);
QObject::connect(manager, SIGNAL(finished(QNetworkReply * )), this, SLOT(readData(QNetworkReply * )));
}
void WeatherAPI::readData(QNetworkReply *reply) {
if (reply->error() == QNetworkReply::NoError) {
QString strReply = (QString) reply->readAll();
QJsonDocument jsonResponse = QJsonDocument::fromJson(strReply.toUtf8());
QJsonObject jsonObject = jsonResponse.object();
weatherObject.city = jsonObject["name"].toString();
weatherObject.temperature = QString::number(jsonObject["main"].toObject()["temp"].toDouble() - 273.15);
int ts = jsonObject["dt"].toInt();
weatherObject.time = QDateTime::fromSecsSinceEpoch(ts).toString("hh:mm");
auto weatherData = jsonObject["weather"].toArray()[0].toObject()["main"].toString();
if (weatherData == "Clouds") {
weatherObject.icon = "Sun.png";
}
} else {
qDebug() << "ERROR";
}
}
void WeatherAPI::requestDataForCity(const QString &city) {
QString link = linkTemplate.arg(city, key);
QUrl url(link);
manager->get(QNetworkRequest(url));
}
const WeatherObject &WeatherAPI::getWeatherObject() const {
return weatherObject;
}
Now here is my custom Widget:
void WeatherButton::initStyle(const QJsonValue &json) {
PolygonButtonWidget::initStyle(json);
auto cities = json.toObject()["cities"].toArray();
api = new WeatherAPI(this);
for (auto c: cities) {
QString city = c.toString();
api->requestDataForCity(city); // HERE I'm making the http request
WeatherObject data = api->getWeatherObject();//HERE I'm getting the DATA
m_title = data.city;
m_time = data.time;
m_icon = data.icon;
m_temperature = data.temperature;
}
}
In that function from WeatherButton::initStyle I'm going to make an HTTP request and also I'm going to place the data into the necessary variable. Now my question is... How can I wait for that data to be received and just after that to place them into those variables?
So far the only solution I know so far is to use a QEventLoop, but at that moment I'm going to basically convert an async call into a sync one, which is not quite what I want. I want to be fully async.
WeatherObject data = api->getWeatherObject(); //HERE I'm getting the DATA
No, you do not get the data here. WeatherAPI::readData is where you get the data.
That is the point of the Signal - Slot mechanism. You do not wait for an event to happen, but react to it via callback, i.e. slot.
Having that in mind, you have to rethink and extend your code. Here is one way to do this:
In the WeatherAPI class define a dataReady(const WeatherObject &weatherObject) signal
Emit this signal in WeatherAPI::readData like this:
void WeatherAPI::readData(QNetworkReply *reply) {
if (reply->error() == QNetworkReply::NoError) {
// the processing of the http response remains unchanged
// ...
emit dataReady(weatherObject);
} else {
qDebug() << "ERROR";
}
}
In the WeatherButton class define a onDataReady slot with the following implementation:
void WeatherButton::onDataReady(const WeatherObject &weatherObject) {
m_title = weatherObject.city;
m_time = weatherObject.time;
m_icon = weatherObject.icon;
m_temperature = weatherObject.temperature;
}
Connect the newly created signal and slot in WeatherButton::initStyle like this:
void WeatherButton::initStyle(const QJsonValue &json) {
PolygonButtonWidget::initStyle(json);
auto cities = json.toObject()["cities"].toArray();
api = new WeatherAPI(this);
connect(api, &WeatherAPI::dataReady, this, &WeatherButton::onDataReady);
for (auto c: cities) {
QString city = c.toString();
api->requestDataForCity(city); // HERE I'm making the http request
}
}
As a sidenote I should say, that initStyle is probably not the best place to instantiate the WeatherAPI. api seems to be an attribute of WeatherButton, hence it should be initialized in the constructor of the class.

QWebEngineView request body interception

A user using QWebEngineView in my application fills some form. This form uses post method to submit data to server. How can I get params from user's body request?
I've found such thing as QWebEngineUrlRequestInterceptor, but it works only for urls.
You can use QWebEnginePage::acceptNavigationRequest.
Whenever a form is submitted, you can get the contents of input by using JavaScript and then accept the request to proceed as usual.
Like Anmol Gautam said, you need to reimplement QWebEnginePage::acceptNavigationRequest function and get needed data using JavaScript.
Here is an example how to do it:
mywebpage.h
#include <QWebEnginePage>
class MyWebPage : public QWebEnginePage
{
Q_OBJECT
public:
explicit MyWebPage(QWebEngineProfile *profile = Q_NULLPTR, QObject *parent = Q_NULLPTR);
protected:
bool acceptNavigationRequest(const QUrl & url, QWebEnginePage::NavigationType type, bool isMainFrame);
}
mywebpage.cpp
MyWebPage::MyWebPage(QWebEngineProfile *profile, QObject *parent):QWebEnginePage(profile, parent),
{
//...
}
bool MyWebPage::acceptNavigationRequest(const QUrl & url, QWebEnginePage::NavigationType type, bool isMainFrame)
{
if(type == QWebEnginePage::NavigationTypeFormSubmitted)
{
qDebug() << "[FORMS] Submitted" << url.toString();
QString jsform = "function getformsvals()"
"{var result;"
"for(var i = 0; i < document.forms.length; i++){"
"for(var x = 0; x < document.forms[i].length; x++){"
"result += document.forms[i].elements[x].name + \" = \" +document.forms[i].elements[x].value;"
"}}"
"return result;} getformsvals();";
this->runJavaScript(jsform, [](const QVariant &result){ qDebug() << "[FORMS] found: " << result; });
}
return true;
}
use QWebEngineView::setPage to set your WebPage subclass to WebView before you call WebViews load function.
Here is a link for more info about HTML DOM forms Collection

How to set QNetworkCookieJar in QWebEngine?

In QWebView it was possible to set a QNetworkCookieJar via QNetworkAccessManager.
QNetworkAccessManager *nam = new QNetworkAccessManager();
nam->setCookieJar(cookieJar);
webView->page()->setNetworkAccessManager(nam);
This was working like a charm.
How can I set a QNetworkCookieJar in new QWebEngine class introduced in Qt5.4?
I found a solution, to share Cookies with QWebEngine and QNetworkAccesManager by using QWebEngineCookieStore. Subclass QNetworkCookieJar:
class CookieWebEngine : public QNetworkCookieJar
{
.....
protected:
// Reimplement this functions to work with your _cookies list;
bool insertCookie(const QNetworkCookie &cookie);
bool deleteCookie(const QNetworkCookie &cookie);
bool updateCookie(const QNetworkCookie &cookie);
bool validateCookie(const QNetworkCookie &cookie, const QUrl &url) const;
private:
// Save Chromium Cookies
QWebEngineCookieStore *_store;
// Save available cookies
QList<QNetworkCookie> _cookies;
}
Now, lets implement a function to load/save cookies in a file:
// Load Chromium Cookies
void CookieWebEngine::loadStore() {
// Save cookies profile shared
QWebEngineProfile::defaultProfile()->setPersistentCookiesPolicy(QWebEngineProfile::ForcePersistentCookies);
_store = WebEngineProfile::defaultProfile()->cookieStore();
connect(_store, &QWebEngineCookieStore::cookieAdded, this, &CookieWebEngine::handleCookieAdded);
_store->loadAllCookies();
}
// Load/Save cookies in arraylist in a file
void CookieWebEngine::load() {
// load cookies and exceptions
qRegisterMetaTypeStreamOperators<QList<QNetworkCookie> >("QList<QNetworkCookie>");
const QString location = cookiesDirectory() + COOKIES_FILE;
QSettings cookieSettings(location, QSettings::IniFormat);
_cookies = qvariant_cast<QList<QNetworkCookie> >(cookieSettings.value(QLatin1String("cookies")));
setAllCookies(_cookies);
// Now user iterate and add it to chromium
for (auto cookie : _cookies) {
_store->setCookie(cookie);
}
cookieSettings.sync();
}
void CookieWebEngine::save()
{
QString directory = cookiesDirectory();
if (!QFile::exists(directory)) {
QDir dir;
dir.mkpath(directory);
}
const QString location = directory + COOKIES_FILE;
QSettings cookieSettings(location, QSettings::IniFormat);
cookieSettings.setValue(QLatin1String("cookies"), QVariant::fromValue<QList<QNetworkCookie>>(_cookies));
cookieSettings.sync();
}
Now, just connect and handle cookies loaded from the webview:
void CookieWebEngine::handleCookieAdded(const QNetworkCookie &cookie)
{
if (insertCookie(cookie)) {
qDebug() << "Handle cookie " << cookie;
}
}
Its working well for me. Now, i use chromium to sign in. After, i save my session cookie in the customized cookiejar and I use it in my customized QNetworkAccesManager.

Memory leak with post requests and QNetworkAccessManager

I'm making a program that uses lots of timer and, at intervals of 4 seconds, does an online post to a php script.
I'm coding in QtCreator 5.1. I use classes just like the ones below.
The one below just populates a task list, but throughout the course of 8 to 12 hours, the memory that the program takes up just keep rising and rising gradually.
What am I doing wrong while using this class?
I have to be able to keep posting online like I already am, about every 4 to 8 seconds.
Here's a simple class for taking care of one of my processes:
Header file: tasklistprocess.h
#ifndef TASKLISTPROCESS_H
#define TASKLISTPROCESS_H
#include <QThread>
#include <QtCore>
#include <QNetworkRequest>
#include <QNetworkAccessManager>
#include <QNetworkReply>
#include <QListWidget>
#include <QTabWidget>
#include "globalhelper.h"
#include "securityinfo.h"
class TaskListProcess : public QThread
{
Q_OBJECT
public:
explicit TaskListProcess(QListWidget *obj_main, QTabWidget *tabs_main, QString user, QObject *parent = 0);
signals:
void upTaskStorage(int key,QHash<QString,QString> item);
private:
GlobalHelper gh;
Securityinfo sci;
QNetworkAccessManager *nam;
QNetworkRequest request;
QByteArray data;
// this is the disposable params for reusage through out the class
QUrlQuery params;
QString post_data;
QString user_name;
QTimer *tasklist_tmr;
bool get_task_list;
QListWidget *obj;
QTabWidget *tabs;
private slots:
void setTaskList();
void replyFinished(QNetworkReply *reply);
void sendPost(QString file_name, QUrlQuery params);
};
#endif // TASKLISTPROCESS_H`
Source file: tasklistprocess.cpp
#include "tasklistprocess.h"
TaskListProcess::TaskListProcess(QListWidget *obj_main, QTabWidget *tabs_main, QString user, QObject *parent) :
QThread(parent)
{
user_name = user;
get_task_list = false;
obj = obj_main;
tabs = tabs_main;
tasklist_tmr = new QTimer(this);
connect(this,SIGNAL(started()),this,SLOT(setTaskList()));
connect(tasklist_tmr,SIGNAL(timeout()),this,SLOT(setTaskList()));
nam = new QNetworkAccessManager(this);
request.setHeader(QNetworkRequest::ContentTypeHeader,"application/x-www-form-urlencoded");
request.setRawHeader( "User-Agent" , "Mozilla Firefox" );
// here we connect up the data stream and data reply signals
connect(nam, SIGNAL(finished(QNetworkReply*)), this, SLOT(replyFinished(QNetworkReply*)));
}
void TaskListProcess::setTaskList()
{
qDebug() << "Your task list was set";
bool in = false;
if(!(tasklist_tmr->isActive()))
{
tasklist_tmr->start(10000);
in = true;
}
if(!(get_task_list))
{
params.clear();
params.addQueryItem("user_name", user_name);
params.addQueryItem("logged_in", "1");
sendPost("getTaskList.php",params);
get_task_list = true;
}
else
{
if(post_data.contains("|*|"))
{
//here i retrieve a piece of information from a php script which is stored in a custom string format
// here we clear the list for the new data to be put in
if(obj->count()>0)
{
obj->clear();
}
int key = 0;
foreach(QString inner_task,post_data.split("|*|"))
{
QHash<QString,QString> task_cont;
//qDebug() << " ";
if(inner_task.contains("*,*"))
{
foreach(QString task_val,inner_task.split("*,*"))
{
if(task_val.contains("*=*"))
{
QStringList key_pairs = task_val.split("*=*");
task_cont.insert(key_pairs[0],key_pairs[1]);
if(key_pairs[0] == "tt")
{
QString val_in;
if(key_pairs[1].length()>10)
{
// this sets the title to the shortened version
// if the string length is too long
val_in = key_pairs[1].left(10) + "....";
}
else
{
val_in = key_pairs[1];
}
obj->addItem("Task :" + QString::fromUtf8(key_pairs[1].toStdString().c_str()));
}
}
}
}
//task_storage.insert(key,task_cont);
emit upTaskStorage(key,task_cont);
key ++;
}
}
get_task_list = false;
}
// here we're checking to see if they are looking at the task tab so it doesn't keep changing
// back and forth between the tabs
bool change = true;
if(tabs->currentIndex() != 0)
{
change = false;
}
if(change)
{
tabs->setCurrentIndex(0);
}else if(in)
{
tabs->setCurrentIndex(0);
}
}
void TaskListProcess::replyFinished(QNetworkReply *reply)
{
if (reply->error() != QNetworkReply::NoError) {
qDebug() << "Error in" << reply->url() << ":" << reply->errorString();
return;
}
QString data = reply->readAll().trimmed();
post_data = data;
if(get_task_list)
{
setTaskList();
}
}
void TaskListProcess::sendPost(QString file_name, QUrlQuery params)
{
post_data = "";
QUrl url(sci.getHost() + file_name);
url.setQuery(params);
data.clear();
data.append(params.toString().toUtf8());
request.setUrl(url);
nam->post(request, data);
}
From the Qt docs http://qt-project.org/doc/qt-5.1/qtnetwork/qnetworkaccessmanager.html
Note: After the request has finished, it is the responsibility of the
user to delete the QNetworkReply object at an appropriate time. Do not
directly delete it inside the slot connected to finished(). You can
use the deleteLater() function.
I would suggest calling reply->deleteLater() in your replyFinished() method.
You should call deleteLater() for an QNetworkReply object after use.
Note: After the request has finished, it is the responsibility of the user to delete the QNetworkReply object at an appropriate time. Do not directly delete it inside the slot connected to finished(). You can use the deleteLater() function.
More information here: http://harmattan-dev.nokia.com/docs/library/html/qt4/qnetworkaccessmanager.html
Thank you everyone for the hel.
It was very simply the "deleteLater()" on the reply in the
void replyFinished(QNetworkReply *reply)
{
}
and it should have looked like this
void replyFinished(QNetworkReply *reply)
{
// after all of your processing
reply->deleteLater();
}
it was such a small problem but one that drove me crazy for a long time so i hope this helps