How to intercept AJAX-Requests within QtWebKit? - c++

I want to intercept, inspect and (if needed) reject AJAX-Requests based on the Fingerprint of the SSL-Certificate. I use the QNetworkAccessManager::createRequest(...) function to issue requests. Everything works fine when I use QWebFrame::load(...). Even the content which is loaded within the request (like .css or .js files) emit signals. Unfortunately no AJAX-Requests emits any Signals. I know that the Signals are connected to the very same slots (for "normal" as well as AJAX-Requests) within MyNetworkAccessManager::createRequest(...) function.
QNetworkReply *reply = QNetworkAccessManager::createRequest(op, req, outgoingData);
connect(reply, SIGNAL(readyRead()), this, SLOT(handleStarted()));
connect(reply, SIGNAL(sslErrors(const QList<QSslError> &)), this, SLOT(handleSslErrors(const QList<QSslError> &)));
connect(reply, SIGNAL(error(QNetworkReply::NetworkError)), this, SLOT(handleNetworkError()));
Why are AJAX Requests so different? Where can I access them?

From what I can tell, AJAX requests do emit the finished signal on QNetworkAccessManager. You need connect to the instance of QNetworkAccessManager on your QWebPage instance:
QWebPage *page = ui->webView->page();
QNetworkAccessManager *nam = page->networkAccessManager();
connect(nam, SIGNAL(finished(QNetworkReply*)), this, SLOT(replyFinished(QNetworkReply*)));
QFile file;
file.setFileName(":/js/jquery-2.1.1.min.js"); // jQuery is loaded as a resource
file.open(QIODevice::ReadOnly);
QString jQuery = file.readAll();
file.close();
ui->webView->load(QUrl("about:blank"));
QWebFrame *frame = m_page->mainFrame();
frame->evaluateJavaScript(jQuery); // load jQuery
// check that jQuery is loaded
frame->evaluateJavaScript("$(document).ready(function() { alert('jQuery loaded!'); });");
// do an AJAX GET
frame->evaluateJavaScript("$.ajax({"
"url: 'http://www.json-generator.com/api/json/get/cqkXBAEoQy?indent=2',"
"method: 'GET',"
"dataType: 'json'"
"}).done(function (data) {"
"for (var i = 0; i < data.length; i++) {"
"alert(data[i].name);"
"}"
"}).error(function (data) { alert('AJAX error'); });");
Then you can monitor replies in the replyFinished slot like so:
void MainWindow::replyFinished(QNetworkReply *reply)
{
QByteArray bytes = reply->readAll();
QString str = QString::fromUtf8(bytes.data(), bytes.size());
QString replyUrl = reply->url().toString();
int statusCode = reply->attribute(QNetworkRequest::HttpStatusCodeAttribute).toInt();
qDebug() << statusCode;
qDebug() << replyUrl;
qDebug() << str;
}
I have noticed that jQuery AJAX's done promise doesn't seem to execute when you do anything with the QNetworkReply, but you can see that the request actually finishes in the debug console.
See my GitHub repository to try out the above code: https://github.com/pcmantinker/QtWebkitAJAX
As far as blocking connections based on SSL certificates, you'd have to subclass QNetworkAccessManager and override QNetworkAccessManager::createRequest. Something like this could work:
QNetworkReply *CustomQNetworkAccessManager::createRequest(Operation op, const QNetworkRequest& request, QIODevice* outgoingData)
{
QNetworkRequest req(request);
QNetworkReply *reply = QNetworkAccessManager::createRequest(op, req, outgoingData);
QSslConfiguration *sslConfig = reply->sslConfiguration();
QList<QSslCertificate> sslCaCerts = sslConfig->caCertificates();
// do something with sslCaCerts
return reply;
}

Related

Qt retrieve additional data created during request when QNetworkAccessManager finished

I would like to create an HTTP request and retrieve in the response some variables create during the request.
However, QNetworkAccessManager reponse is asynchronous and use SIGNAL/SLOT functionality, so variable are no more accessible.
Here is an example :
void makeRequest()
{
manager = new QNetworkAccessManager(this);
connect(manager, SIGNAL(finished(QNetworkReply*)),
this, SLOT(replyFinished(QNetworkReply*)));
// here is the datas I want to get in the reply
// objectdata and objectdata2 are created just before the http request
MyClass objectdata("randomseed");
MyClass2 objectdata("randomseed") ;
QUrl websiteurl = objectdata.getUrl();
manager->get(QNetworkRequest(websiteurl));
}
void replyFinished (QNetworkReply *reply)
{
if(reply->error())
{
qDebug() << "ERROR!";
qDebug() << reply->errorString();
}
else
{
QByteArray dataHttp = reply->readAll();
// How can I get here objectdata and objectdata2 ?
// I would like to do something like that this->dataSuccess(objectdata,objectdata2,dataHttp);
}
reply->deleteLater();
}
In a synchronous system this problematic doesn't exist.
Is there a workaround to this problematic ?
One way to do it is to capture the locals in a lambda instead of using a separate function for the slot.
void makeRequest()
{
manager = new QNetworkAccessManager(this);
// here is the datas I want to get in the reply
// objectdata and objectdata2 are created just before the http request
MyClass objectdata("randomseed");
MyClass2 objectdata("randomseed") ;
connect(manager, &QNetworkAccessManager::finished, [this, objectdata, objectdata2](QNetworkReply *reply)
{
if(reply->error())
{
qDebug() << "ERROR!";
qDebug() << reply->errorString();
}
else
{
QByteArray dataHttp = reply->readAll();
dataSuccess(objectdata,objectdata2,dataHttp);
}
reply->deleteLater();
}
QUrl websiteurl = objectdata.getUrl();
manager->get(QNetworkRequest(websiteurl));
}
Another way is to add extra parameters as an attribute in the QNetworkRequest.
// Save off the data in the request
QNetworkRequest req(websiteurl);
req.setAttribute(QNetworkRequest::User, QVariant::fromValue<MyClass>(objectdata));
req.setAttribute(QNetworkRequest::User + 1, QVariant::fromValue<MyClass2>(objectdata2));
manager->get(req);
...
// Retrieve the data from the reply
auto data1 = reply->request().attribute(QNetworkRequest::User).value<MyClass>();
auto data2 = reply->request().attribute(QNetworkRequest::User + 1).value<MyClass2>();

How do I know a QNetWorkReply comes from which proxy?

It's a HTTP request sending method. When the goal website responses, httpFinished() will be called.
void HTTPClientBase:: HttpRequestGet()
{
network_manager.setProxy(proxy);
QNetworkRequest network_request;
network_request.setHeader(QNetworkRequest::ContentTypeHeader, "application/x-www-form-urlencoded");
network_request.setUrl(URL);
reply = network_manager.get(network_request);
connect(reply, SIGNAL(finished(QNetWorkReply*)), this, SLOT(httpFinished(QNetWorkReply*)));
}
void HTTPClientBase::httpFinished(QNetWorkReply* reply)
{
// How do I know this reply comes from which proxy?
}
I can call HttpRequestGet() in a loop.
static HTTPClientBase myClient;
for(int i=0; i<20; i++)
{
myClient.setUrl("https:\\www.google.com");
myClient.setProxy("123.123.123.123:1234"); // The proxy changes every time in this loop.
myClient.HttpRequestGet();
}
When HTTPClientBase::httpFinished(QNetWorkReply* reply) is called, How do I know this reply comes from which proxy?
}
As per the QNetworkReply document, you can get the corresponding request using the member function QNetworkReply::request().
Anyway, QNetworkRequest has not member function of setProxy.
But if you are setting proxy for QNetworkAccessManager you can have a pointer to the corresponding manager by QNetworkReply::manager().
Notice the connect command. finished() has no QNetworkReply* argument so your connect command will also fail and you have to change HTTPClientBase::httpFinished(QNetWorkReply* reply) to HTTPClientBase::httpFinished().
...
connect(reply, SIGNAL(finished()), this,
SLOT(httpFinished()));
}
void HTTPClientBase::httpFinished()
{
QNetworkReply* reply = qobject_cast<QNetworkReply*>(sender());
if(reply){
QNetworkAccessManager * manager = reply->manager();
QNetworkProxy proxy = manager->proxy();
// now you have the proxy
}
}
As you see, you have to use sender() to obtain the actual signal sender.
You need to create different QNetworkAccessManager instances for each proxy you have. If you have a proxy pool, create your QNetworkAccessManager instances first and choose them according to your specific needs.
If you don't want to create a new QNetworkAccessManager for each proxy, you can do something like this:
void HTTPClientBase:: HttpRequestGet()
{
network_manager.setProxy(proxy);
QNetworkRequest network_request;
network_request.setHeader(QNetworkRequest::ContentTypeHeader, "application/x-www-form-urlencoded");
network_request.setUrl(URL);
reply = network_manager.get(network_request);
//new member variable: QHash<QNetworkReply*,QString> m_RequestToProxy;
m_RequestToProxy.insert(reply,proxy);
connect(reply, SIGNAL(finished()), this, SLOT(httpFinished()));
}
void HTTPClientBase::httpFinished()
{
QNetworkReply* reply = qobject_cast<QNetworkReply*>(sender());
if(reply){
QString proxy = m_RequestToProxy.take(reply);
//check if proxy is valid
//and then do whatever you want
}
}
And another better solution is to set a property of reply and get it in slot.
void HTTPClientBase:: HttpRequestGet()
{
network_manager.setProxy(proxy);
QNetworkRequest network_request;
network_request.setHeader(QNetworkRequest::ContentTypeHeader, "application/x-www-form-urlencoded");
network_request.setUrl(URL);
reply = network_manager.get(network_request);
reply->setProperty("_proxy_",proxy);
connect(reply, SIGNAL(finished()), this, SLOT(httpFinished()));
}
void HTTPClientBase::httpFinished()
{
QNetworkReply* reply = qobject_cast<QNetworkReply*>(sender());
if(reply){
QVariant v_proxy = reply->property("_proxy_");
//check if proxy is valid
if(v_proxy.isValid()){
QString proxy = v_proxy.toString();
//and then do whatever you want
}
}
}

Qt: Post data to server

I try to implement a REST client in order to get a Service Ticket from my server. For those of you who don't know CAS: A Service Ticket can be requested by showing a TGT. The TGT can be requested by a successful login basically. Maybe that is not even relevant.
I quess I have a error in my connect. My server is not even reacting to that connection and the reply is emptry. However for some reason reply->error() == QNetworkReply::NoErroris true.
What am I doing wrong?
bool Client::validateTGT(QString const & tgt) const
{
bool isValid = false;
QUrl url = QUrl("https://localhost:8943/cas/v1/tickets/" + tgt);
QUrl postData;
postData.addQueryItem("service", "https://test.de");
QNetworkRequest request(url);
request.setHeader(QNetworkRequest::ContentTypeHeader,
"application/x-www-form-urlencoded");
//QNetworkAccessManager *networkManager = new QNetworkAccessManager();
QObject::connect(manager, SIGNAL(finished(QNetworkReply*)),
this, SLOT(replyFinished(QNetworkReply*)), Qt::AutoConnection);
QNetworkReply *reply = manager->post(request, postData.encodedQuery());
QByteArray replyData = reply->readAll();
QString s_data = QString::fromAscii(replyData.data());
if (reply->error() == QNetworkReply::NoError)
{
isValid = true;
}
return isValid;
}
EDIT: replyFinished as requested
.h:
public slots:
void replyFinished(QNetworkReply *);
.cpp:
void CCASRESTClient::replyFinished(QNetworkReply *reply)
{
QByteArray replyData = reply->readAll();
serviceTicket = QString::fromAscii(replyData.data());
}
The slot replyFinished is called by the event loop when the reply has arrived from the server, this happens far after your function validateTGT has returned.
The manager object receives the reply and then emits the finished signal, this is when the slot replyFinished is called. It doesn't make sense to return a value from there. Just ask yourself, who is the caller function that will get this return value? validateTGT has already returned, thus it is not getting anything.
Your slot needs to be declared to return void, and you should do whatever you want to do with the reply in your replyFinished slot.
In general, if you have any slot that returns a value, the return value cannot be retrieved unless this slot was called as a normal function (Obviously this is not the case here).

Adding attachment to JIRA issue using the REST API and Qt with QNetworkRequest

I'm trying to add an attacment to an existing JIRA issue using the REST API and Qt.
When I run the code below, the reply is an empty array ("[]").
edit: updated the code
QString APIhandler::attachFile(QString fileloc, QString issueKey, QString cookie)
{
//create multiPart
QHttpMultiPart *multiPart = new QHttpMultiPart(QHttpMultiPart::FormDataType);
QFile *file = new QFile(fileloc);
//create httpPart for file
QHttpPart filePart;
filePart.setHeader(QNetworkRequest::ContentTypeHeader, QVariant("application/octet-stream"));
filePart.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"file\"; filename=\""+ file->fileName()+ "\""));
file->open(QIODevice::ReadOnly);
filePart.setBodyDevice(file);
file->setParent(multiPart);
multiPart->append(filePart);
QNetworkAccessManager *mngr = new QNetworkAccessManager();
QUrl issurl(baseURL + "/api/2/issue/"+ issueKey + "/attachments");
QNetworkRequest req(issurl);
QNetworkReply *reply ;
QEventLoop loop;
//add headers
req.setRawHeader("cookie", "JSESSIONID = " + cookie.toUtf8()); // the session cookie
req.setRawHeader("X-Atlassian-Token", "nocheck");
req.setRawHeader("Content-Type", "multipart/form-data; boundary=------------------------53a5a2cd1d9c8b7f");
//req.setRawHeader("Content-Length", postDataSize);
reply = mngr->post(req, multiPart);
multiPart->setParent(reply);
QObject::connect(reply, SIGNAL(finished()), &loop, SLOT(quit()));
loop.exec();
return reply->readAll();
}
I am using the JIRA REST API documentation and qt documentation for reference, and looking off of this java implementation (which I've tried to replicate).
It seems like I'm either missing a header, or adding the file incorrectly.
Any help is greatly appreciated!
EDIT - Here's part of a wireshark comparing the example from the api using curl (LEFT) , and my code (RIGHT). The one on the left works, and clearly has different MIME data, but I'm not sure how to implement that in Qt
Okay, so I figured it out. I might be the only one on earth who is using (or will use) Qt to interact with the JIRA API, but for posterity, here's what I came up with:
QString APIhandler::attachFile(QString fileloc, QString issueKey, QString cookie)
{
QHttpMultiPart *multiPart = new QHttpMultiPart(QHttpMultiPart::FormDataType);
QHttpPart filePart;
QFileInfo fileInfo(fileloc);
//what I wasn't doing before!
multiPart->setBoundary("------------------------53a5a2cf4d9c8b7f");
filePart.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"file\"; filename=\""+fileInfo.fileName() +"\""));
filePart.setHeader(QNetworkRequest::ContentTypeHeader, QVariant("application/octet-stream"));
QFile *file = new QFile(fileloc);
file->open(QIODevice::ReadOnly);
filePart.setBodyDevice(file);
file->setParent(multiPart);
multiPart->append(filePart);
QNetworkAccessManager *mngr = new QNetworkAccessManager();
QUrl issurl(baseURL + "/api/2/issue/"+ issueKey + "/attachments");
QNetworkRequest req(issurl);
QNetworkReply *reply ;
QEventLoop loop;
//add headers
req.setRawHeader("X-Atlassian-Token", "nocheck");
req.setRawHeader("cookie", "JSESSIONID = " + cookie.toUtf8()); // the session cookie
req.setRawHeader("Content-Type", "multipart/form-data;boundary=------------------------53a5a2cf4d9c8b7f");
reply = mngr->post(req, multiPart);
multiPart->setParent(reply);
QObject::connect(reply, SIGNAL(finished()), &loop, SLOT(quit()));
loop.exec();
//read the reply
QByteArray bytes=reply->readAll();
//return the reply JSON
return QString::fromUtf8(bytes.data(), bytes.size());
delete file;
delete multiPart;
delete reply;
delete mngr;
}
The key part here, and what I was doing wrong, was the way in which I set the boundary for the multipart. Instead of setting it in the header, I should have used:
multipart->setBoundary()
Which you can see reflected above.
If you're coming across this and are going to use it, I'd recommend cleaning it up a bit, first. But it works!

how to return an Qt object from the result of an thread (Qtfutur)

i'm trying to load some data from a server and fill a Qt list. i want to run the dowloanding in a thread. so there is the code of:
principal function in the App.cpp
loadInterestPlaces(QString& urlInterestPlaces) {
LoadData* Data = new LoadData(urlInterestPlaces);
QFuture< list <InterestPlace *> > future = QtConcurrent::run(Data,
&LoadData::startLoading);
// Invoke our onLoadingFinished slot after the loading has finished.
bool ok = connect(&m_watcher, SIGNAL(finished()), this,
SLOT(onLoadingFinished()));
Q_ASSERT(ok);
Q_UNUSED(ok);
// starts watching the given future
m_watcher.setFuture(future);
}
void ApplicationUI::onLoadingFinished() {
qDebug() << "Loading finished";
interestPlacesList = m_watcher.future().result();
qDebug() << "List size= " << interestPlacesList.size();
}
}
the LoadData.cpp file : this is the code of the startloanding function :
std::list<InterestPlace *> LoadData::startLoading()
{
QNetworkAccessManager* netManager = new QNetworkAccessManager(this);
const QUrl url(_URL);
QNetworkRequest request(url);
QNetworkReply* reply = netManager->get(request);
netManager->moveToThread(this->thread());
netManager->setParent(this);
bool ok = connect(reply, SIGNAL(finished()), this, SLOT(onReplyFinished()));
qDebug() << reply->isFinished();
Q_ASSERT(ok);
Q_UNUSED(ok);
qDebug() << "load data: liste size" <<interestPlacesList.size();
return interestPlacesList;
}
Finally inside the SLOT onreplyfinished i parse the data and fill the list.
But the problem here, the QFuture is finished before the downloading so that the list is always empty.
How could i return the list filled just after the execution of the onReplyFinished ?
You may be making this more complex than you need to. Here is what I do to get data off the web. In my case I'm downloading a large ZIP file, writing it out and then unziping it, but the basic steps are the same (I have omitted a lot of my specific code for clarity):
void HtmlDownloader::startDownload() {
// Creates the network request and sets the destination URL
QNetworkRequest request = QNetworkRequest();
request.setUrl(mUrl);
// Creates the network access manager and connects a custom slot to its
// finished signal. Checks the return value for errors.
QNetworkAccessManager *networkAccessManager =
new QNetworkAccessManager(this);
bool c = connect(networkAccessManager, SIGNAL(finished(QNetworkReply*)),
this, SLOT(requestFinished(QNetworkReply*)));
Q_ASSERT(c);
// Indicate that the variable res isn't used in the rest of the app, to prevent
// a compiler warning
Q_UNUSED(c);
// Sends the request
QNetworkReply *reply = networkAccessManager->get(request);
c = connect(reply, SIGNAL(downloadProgress(qint64,qint64)), this, SLOT(onDownloadProgress(qint64,qint64)));
Q_ASSERT(c);
}
void HtmlDownloader::requestFinished(QNetworkReply *reply) {
// Handle the reply data...
// Check the network reply for errors
if (reply->error() == QNetworkReply::NoError) {
// use reply->readAll() or reply->read() to get the returned data
// and process into a list. The list can't be returned but could
// be sent as a payload of a signal, or stored in a data member and
// a signal sent to indicate it is available.
}
reply->deleteLater();
}
void HtmlDownloader::onDownloadProgress(qint64 bytesRecieved, qint64 bytesTotal) {
emit downloadProgress(bytesRecieved, bytesTotal);
}