I'm trying to open a web page using QNetworkAccessManager - and for some pages it works fine - while for others it does not. I tried setting a real browser user-agent, however it still doesn't work for example, http://www.erepublik.com. Here's the code:
QNetworkAccessManager *manager = new QNetworkAccessManager(this);
connect(manager, SIGNAL(finished(QNetworkReply*)),
this, SLOT(replyFinished(QNetworkReply*)));
QNetworkRequest *request = new QNetworkRequest(QUrl("http://www.erepublik.com"));
request->setRawHeader( "User-Agent", "Mozilla/5.0 (X11; U; Linux i686 (x86_64); "
"en-US; rv:1.9.0.1) Gecko/2008070206 Firefox/3.0.1" );
request->setRawHeader( "charset", "utf-8" );
request->setRawHeader( "Connection", "keep-alive" );
manager->get(*request);
...
void MainWindow::replyFinished(QNetworkReply *reply)
{
QString data = reply->readAll();
qDebug() << data;
}
The data is the following:
<html><head><meta http-equiv="refresh" content="0;url=http://www.erepublik.com/en"/></head></html><html><head><meta http-equiv="refresh" content="0;url=http://www.erepublik.com/en"/></head></html>
Now, what's bugging me this works for a site like http://www.hardwarebase.net (data returns the normal HTML source), while it doesn't work for eRepublik.
For those who are curious to know what I exactly want to do - I want to get the population number of countries from the eRepublik front page.
Any ideas why is this happening? Thanks in advance.
It looks like you're getting the data correctly, it's just that that particular URL just forwards you to a different one. Try http://www.erepublik.com/en (with the /en) instead.
The returned HTML is re-directing you to http://www.erepublik.com/en so you would be best off forming you QNetworkRequest to go straight to that URL.
Related
Besically, I post my username and password to a site, say http://example.org/signup.asp. Then I get cookies from it, which I wanna save it in qnam_, an object of QNetworkAccessManager.
Problem 1
The first issue is, after saving the cookies in reply_'s corresponding url, say say http://example.org/signup.asp, I just cannot retrieve it back via http://example.org/ or http://example.org/something_else.
auto cookies = qvariant_cast<QList<QNetworkCookie>>(reply_->header
(QNetworkRequest::SetCookieHeader));
auto cookieJar = new QNetworkCookieJar(&qnam_);
// qDebug() outputs "http://example.org/sign.asp"
qDebug() << reply_->request().url();
// assert won't fire, which means "one or more cookies are set for url"
assert(cookieJar->setCookiesFromUrl(cookies, reply_->request().url()));
qnam_.setCookieJar(cookieJar);
// qDebug() outputs nothing, but "()", why???
qDebug() << qnam_.cookieJar()->cookiesForUrl(QUrl("http://example.org"));
Problem 2
The second one is even I set cookies in the "root hostname", say http://example.org/, I still cannot retrieve it via the same url.
assert(cookieJar->setCookiesFromUrl(cookies, QUrl("http://example.org")));
qnam_.setCookieJar(cookieJar);
// Still get nothing from it.
qDebug() << qnam_.cookieJar()->cookiesForUrl(QUrl("http://example.org"));
Note that I've checked the QT HTTP Post issue when server requires cookies and How do I save cookies with Qt?, which doesn't work out I think.
Any ideas? Thanks!
I got this working using the following solution:
in the function callback for QNetworkReply::finished I add a cookie
QNetworkCookie cookie("mycookie", mycookiedata.toUtf8());
QList<QNetworkCookie> cookies;
cookies.append(cookie);
mCookieJar.setCookiesFromUrl(cookies, reply->url());
I am trying to set QwebView's Url as "http://" and other part of url entered from linedit. Here's my function:
void MainWindow::on_git_clicked()
{
QUrl adrs;
adrs=ui->lineEdit->text();
ui->webView->setUrl(QUrl("http://" + adrs);
}
But it does nothing. But it was working on Python with same way. Thanks for help.
How about this, since your using QUrl and trying to append it to "http://" why not just use a QString instead? this might be your problem
QString adrs(ui->lineEdit->text());
ui->webview->setURL(QUrl("http://" + adrs));
I have the next code to make a request:
void HTTPClient::post(QString connectionString, QHttpMultiPart* _multiPart, bool returnProgress) {
QUrl url;
if (ssl)
url.setScheme("https");
else
url.setScheme("http");
url.setHost(host);
url.setPort(port);
url.setPath(connectionString);
url.setUrl(url.toEncoded());
QNetworkRequest request(url);
request.setRawHeader("User-Agent", QCoreApplication::applicationName().toLatin1());
/*...irrelevant code...*/
}
The requested url should be
https://somewebpage.domain:443/REST/login.php?method=login_md5
but the QNetworkRequest requests this one despite I set the url as encoded (debugging url.toEncoded() prints the '?' correctly):
https://somewebpage.domain:443/REST/login.php%3Fmethod=login_md5
This results in a 404 not found page. I have tried setting the url with url.toString() and just url, but the '?' keeps messing up. What can I do to request the link properly?
I have tried building the QUrl in the constructor like this:
QUrl url("https://"+host+port+connectionString);
But results in the next string:
https://somewebpage.xn--domain-efa/REST/login.php?method=login_md5
You can try QUrl::fromEncoded
url.setUrl( QUrl::fromPercentEncoding(url.toEncoded()));
Parses input and returns the corresponding QUrl. input is assumed to
be in encoded form, containing only ASCII characters.
to correctly handle characters.
The encoded URL seems valid, maybe a problem with your webserver?
You can setPath() and setQuery() separatly. What come before the '?' is the path and after is the query arguments.
const QStringList path_part = path.split('?');
url.setPath(path_part.at(0));
if(path_part.size() > 1)
url.setQuery(path_part.at(1));
i need to get web sites html status codes
today i just do simple get request to the domain , and then i get the status code as part of the response , but also the site index.html content .
pNetworkManager = new QNetworkAccessManager(this);
reply = pNetworkManager->get(request);
QVariant vStatusCodeV = reply->attribute(QNetworkRequest::HttpStatusCodeAttribute);
QVariant redirectionTarget = reply->attribute(QNetworkRequest::RedirectionTargetAttribute);
data=reply->readAll();
this last function i like to avoid if it can be avoided ,
is there any way to get only the domain status code ?
Maybe you can send a HEAD request instead of a GET request?
This is not a Qt / client specific solution, but is the approach recommended by the HTTP protocol when you don't need the content, but just want to get the headers that a request would normally produce, for example in order to validate that the page exists.
I suppose this could be done with QNetworkAccessManager using the head() method
I agree with #shevron's answer, but if the site you're communicating with isn't "clever" enough to implement the HEAD request, you can still avoid the readAll() call.
QByteArray line = reply->readLine(); //< eg "HTTP/1.0 200 OK"
QList<QByteArray> chunks = line.split(' ');
QString statusCode = chunks[1];
That should avoid the memory overhead of readAll().
I have a webservice that I need to POST some data to using Qt.
I figured that I can use a QByteArray when POSTing to the web service.
My question is, how can I format this array in order to be parsed correctly at the other end?
This is the code I have so far:
// Setup the webservice url
QUrl serviceUrl = QUrl("http://myserver/myservice.asmx");
QByteArray postData;
/*
Setup the post data somehow
I want to transmit:
param1=string,
param2=string
*/
// Call the webservice
QNetworkAccessManager *networkManager = new QNetworkAccessManager(this);
connect(networkManager, SIGNAL(finished(QNetworkReply*)), this, SLOT(serviceRequestFinished(QNetworkReply*)));
networkManager->post(QNetworkRequest(serviceUrl), postData);
Thanks!
Since some parameters and values might need to be utf-8 and percent encoded (spaces, &, =, special chars...), you should rather use QUrl (for Qt 4) or QUrlQuery (for Qt 5) to build the posted string.
Example code for Qt 4:
QUrl postData;
postData.addQueryItem("param1", "string");
postData.addQueryItem("param2", "string");
...
QNetworkRequest request(serviceUrl);
request.setHeader(QNetworkRequest::ContentTypeHeader,
"application/x-www-form-urlencoded");
networkManager->post(request, postData.encodedQuery());
and for Qt 5:
QUrlQuery postData;
postData.addQueryItem("param1", "string");
postData.addQueryItem("param2", "string");
...
QNetworkRequest request(serviceUrl);
request.setHeader(QNetworkRequest::ContentTypeHeader,
"application/x-www-form-urlencoded");
networkManager->post(request, postData.toString(QUrl::FullyEncoded).toUtf8());
Starting with Qt 4.8 you can also use QHttpMultiPart if you need to upload files.
I used:
QByteArray postData;
postData.append("param1=string&");
postData.append("param2=string");
So & instead of newline after each parameter.
Updating alexisdm answer to Qt5:
// Setup the webservice url
QUrl serviceUrl = QUrl("http://your.url");
QByteArray postData;
QUrlQuery query;
query.addQueryItem("param1","string1");
query.addQueryItem("param2","string2");
postData = query.toString(QUrl::FullyEncoded).toUtf8();
// Call the webservice
QNetworkAccessManager *networkManager = new QNetworkAccessManager(this);
connect(networkManager, SIGNAL(finished(QNetworkReply*)),
SLOT(onPostAnswer(QNetworkReply*)));
QNetworkRequest networkRequest(serviceUrl);
networkRequest.setHeader(QNetworkRequest::ContentTypeHeader,"application/x-www-form-urlencoded");
networkManager->post(networkRequest,postData);
Don't forget to include
QT += network
in .pro.
the actually answer is
QByteArray postData;
postData.append("param1=string&");
postData.append("param2=string");
NOTE: use "&" here!!!.
I didn't notice Juha's answer here, and waste much time on testing my code using the ",\n" approach.
Please change the correct answer to Juha's.
Here is another way to handle this, i am using your code also to give a complete code:
// Setup the webservice url
QUrl serviceUrl = QUrl("http://myserver/myservice.asmx");
QByteArray postData;
QUrl params;
params.addQueryItem("param1","string1");
params.addQueryItem("param2","string2");
postData = params.encodedQuery();
// Call the webservice
QNetworkAccessManager *networkManager = new QNetworkAccessManager(this);
connect(networkManager, SIGNAL(finished(QNetworkReply*)), this, SLOT(serviceRequestFinished(QNetworkReply*)));
networkManager->post(QNetworkRequest(serviceUrl), postData);
QByteArray postData;
postData.append("param1=string,\n");
postData.append("param2=string\n");