Infinite Loop (while (true)) stops working in a 5-10 sec. I use the Qt technology. Here's the code:
void Worker::start() {
while (true) {
QNetworkReply* reply = manager->get(QNetworkRequest(QUrl("link")));
QEventLoop loop;
connect(reply, &QNetworkReply::finished, &loop, &QEventLoop::quit);
loop.exec();
QJsonDocument document = QJsonDocument::fromJson(reply->readAll());
QJsonObject root = document.object();
QJsonValue response = root.value("response");
QJsonObject items = response.toObject().value("items").toArray().at(0).toObject();
int comments = items.value("comments").toObject().value("count").toInt();
if (comments == 0) {
qDebug() << "Comment";
QThread::sleep(3);
}
qDebug() << "END";
}
}
Quoting from OPs comment (just to get this out of the list of unanswered questions):
The API has a limitation for the requests. I've added Sleep in the end and it works.
Related
How do I read the data from a QNetworkReply response from a specific URL before QWebPage does? but when the finished() signal is emited the reply is read already by QWebPage, so connect readyRead() or call reply->readAll() return nothing. I tried overload acceptNavigationRequest() method in my own QWebPage class, something like this:
bool webPage::acceptNavigationRequest(QWebFrame *frame, const QNetworkRequest &request, QWebPage::NavigationType type)
{
//qDebug() << "filename = " << request.rawHeader("content-disposition");
if(request.url().path() == QStringLiteral("download.php"))
{
QNetworkReply *reply = networkAccessManager()->get(request);
QFile file;
file.setFileName(filename);
if(!file.open(QIODevice::ReadWrite))
{
/* handle error */
}
file.write(reply->readAll());
file.close();
return false;
}
But I couldn't manage to reply work... the returned reply is invalid (don't even return a http status code, I know it means the http request sent is invalid but i don't know why).
Different approachs to solve this are welcome!
Using the finished slot with a lambda expression, you can do this: -
QNetworkReply* reply = networkAccessManager()->get(request);
connect(reply, &QNetworkReply::finished, [=]() {
if(reply->error() == QNetworkReply::NoError)
{
QByteArray response = reply->readAll();
// do something with the data...
}
else // handle error
{
qDebug(pReply->errorString());
}
});
I would like to make a class for accessing data via REST API, for example:
class MeteoStation{
int getLatestTemperature();
int getLatestPessure();
private:
QNetworkManager nmng;
}
How could I implement this methods? Usually I was using something like:
int MeteoStation::getLatestTemperature(){
...
QEventLoop eventLoop;
connect(&m_nam,SIGNAL(finished(QNetworkReply*)),&eventLoop,SLOT(quit()));
QNetworkReply *reply = m_nam.get( req );
eventLoop.exec();
reply->readAll()
...
}
But since using inner QEventLoop is not recommended, how should I see to whom the response belong to?
MeteoStation::MeteoStation(){
connect(&nmam, SIGNAL(finished(QNetworkReply*)),
this, SLOT(parseNetworkResponse(QNetworkReply*)));
}
void MeteoStation::parseNetworkResponse( QNetworkReply *finished )
{
QByteArray data = finished->readAll();
...
Yes and it would be nice to have the class thread save. How are you solving that in your code?
How bad is making the call synchronous with:
QNetworkRequest req(url);
QScopedPointer<QNetworkReply> reply(nam.get(req));
QTime timeout= QTime::currentTime().addSecs(10);
while( QTime::currentTime() < timeout && !reply->isFinished()){
QCoreApplication::processEvents(QEventLoop::AllEvents, 100);
}
if (reply->error() != QNetworkReply::NoError) {
qDebug() << "Failure" <<reply->errorString();
}
QByteArray data = reply->readAll();
I've resolved my problem using QCoreApplication::processEvents(). The response is there within ms and I'm able to implement functionality close to libcurl.
QNetworkRequest req(url);
QScopedPointer<QNetworkReply> reply(nam.get(req));
QTime timeout= QTime::currentTime().addSecs(10);
while( QTime::currentTime() < timeout && !reply->isFinished()){
QCoreApplication::processEvents(QEventLoop::AllEvents, 100);
}
if (reply->error() != QNetworkReply::NoError) {
qDebug() << "Failure" <<reply->errorString();
}
QByteArray data = reply->readAll();
The Qt docs should provide all info you need.
You creat a nam, connect the finished signal, send the request.
QNetworkAccessManager *manager = new QNetworkAccessManager(this);
connect(manager, SIGNAL(finished(QNetworkReply*)),
this, SLOT(parseNetworkResponse(QNetworkReply*)));
manager->get(QNetworkRequest(QUrl("http://qt-project.org")));
Detecting to which request a reply belongs should not be too hard. The reply contains the url. It might be different, but not that different:
...but for a variety of reasons it can be different (for example, a
file path being made absolute or canonical).
QUrl QNetworkReply::url() const
Returns the URL of the content downloaded or uploaded. Note that the
URL may be different from that of the original request.
There has been the same question already, but the single answer is not helpful: Qt Download File - QNetworkAccessManager, not getting data
So, I'm trying to download a file:
QNetworkRequest request;
request.setUrl(QUrl(fileUrl));
QNetworkReply * reply = m_nam.get(request);
connect(reply, SIGNAL(finished()), this, SLOT(onDownloadRequestFinished()), Qt::UniqueConnection);
connect(reply, SIGNAL(downloadProgress(qint64, qint64)), this, SLOT(onDownloadRequestProgress(qint64, qint64)), Qt::UniqueConnection);
And in the onDownloadRequestFinished slot:
QNetworkReply * reply = qobject_cast<QNetworkReply *>(sender());
if (reply && reply->error() == QNetworkReply::NoError) {
Q_ASSERT(reply->attribute(QNetworkRequest::HttpStatusCodeAttribute).toInt() == 200);
qDebug() << "reply " << reply->bytesAvailable() << reply->pos() << reply->size() << reply->isReadable() << reply->openMode() << reply->isOpen();
}
The slot prints the following: reply 0 0 0 true OpenMode( "ReadOnly" ) true
So, no data. However, I can clearly see that it does download something somewhere. It's a big file and it does download it, judging from onDownloadRequestProgress.
Important clarification: pretty much the same code works in another project on the same computer. I'm trying to find differences, but see none so far.
Where's the data?
Did you connected readyRead() signal to write bytes received into a specific file?
I always did this to save a file:
const QNetworkRequest& request = QNetworkRequest(url);
reply = qnetworkaccessmanager->get(request);
QObject::connect(reply, SIGNAL(readyRead()), this,
SLOT( readingReadyBytes() ));
then i create my slot:
void yourClass::readingReadyBytes() {
file->write(reply->read(reply->bytesAvailable()));
}
I'm trying to have my application download a file from a URL, typically an EXE or a Jar, not that this should change much though. I have this all running in a thread, but I don't think that will make a difference (if it does let me know).
So Do_Download is my function that creates the manager, sets the URL and request, and performs get. I then try to connect the finished signal to the slot the will write the file.
void DownloadManager::Do_Download() {
QNetworkAccessManager *netManager = new QNetworkAccessManager(this);
QUrl url(install_mirror); //istall_mirror is the URL provided by user
QNetworkRequest req(url);
QNetworkReply *reply = netManager->get(req);
connect(reply, SIGNAL(finished()), this, SLOT(writeData()));
}
My writeData function checks for errors, and if there are no errors it writes the data to file.
void DownloadManager::writeData() {
QNetworkReply *reply = qobject_cast<QNetworkReply*>(sender());
if (reply) {
if (reply->error() == QNetworkReply::NoError) {
QFile file(location);
if(file.open(QIODevice::WriteOnly)) {
file.write(reply->readAll());
} else {
errorMessage = "Error writing downloaded file for mirror installation";
}
} else {
//get http status code
int httpStatus = reply->attribute(QNetworkRequest::HttpStatusCodeAttribute).toInt();
errorMessage = "HTTP Error code while downloading from mirror: " + httpStatus;
}
reply->deleteLater();
} else {
errorMessage = "Error downloading file from installation mirror";
}
}
The problem being there is no data being written. It just creates a 0Kb file.
I tried adding a download progress slot so I could see what was going on recieving the data. So I added this to my Do_Download method.
connect(reply, SIGNAL(downloadProgress(qint64,qint64)), this, SLOT(DL_Progress(qint64,qint64)));
void DownloadManager::DL_Progress(qint64 recieved, qint64 total) {
std::cout << recieved << " / " << total << endl;
}
The output displays one time as 0 / 01
What am I doing wrong?
The only problem I see in your code is you are not waiting for the download to be finished. The NetworkRequest object would be destructed at the end of function call.
So, I would rewrite Do_Download like this (QEventLoop syncronizes the network request):
void DownloadManager::Do_Download() {
QEventLoop eventLoop;
QNetworkAccessManager *netManager = new QNetworkAccessManager(this);
QUrl url(install_mirror); //istall_mirror is the URL provided by user
QNetworkRequest req(url);
QNetworkReply *reply = netManager->get(req);
connect(reply, SIGNAL(finished()), &eventLoop, SLOT(quit()));
eventLoop.exec();
writeData(reply);
}
I know that in version 4.8 each http request gets its own thread to run.
I'm doing a links checker app that does a lot of http requests in a while loop and I notice in the windows task manager that my app is using more than 1600 threads over time and the number never goes down, only up until it crashes the app. (I'm guessing that is the cause.)
My question is, does QNetworkAccessManager have an option to use thread pool?
Or does it have an option to clean its threads after it finishes its http request?
This is the main loop:
while(!rpm_urlStack->isEmpty())
{
QString url = rpm_urlStack->top();
//define the reply
QNetworkReply *reply;
rpm_urlStack->pop();
QString urlForReq(url);
bool returnVal = true;
QNetworkRequest request;
request.setUrl(QUrl(urlForReq));
request.setRawHeader("User-Agent", USER_AGENT.toUtf8());
request.setRawHeader("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.7");
request.setRawHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
request.setRawHeader("Accept-Language", "en-us,en;q=0.5");
request.setRawHeader("Connection", "Keep-Alive");
QEventLoop loop;
reply = m_networkManager->get(request);
connect(reply, SIGNAL(finished()), &loop, SLOT(quit()));
loop.exit();
if(!loop.isRunning()) {
loop.exec();
}
RequestFinishedHandler(reply);
// this is how I delete the reply object
delete reply;
}
RequestFinishedHandler(QNetworkReply *reply)
{
if (reply->error() > 0) {
QNetworkReply::NetworkError networkError = reply->error();
QString err = reply->errorString();
} else {
QVariant vStatusCodeV = reply->attribute(QNetworkRequest::HttpStatusCodeAttribute);
QMutexLocker lock(_pMutex); // _pMutex defined as class member
char *buffer;
buffer = getCurrentDateTime();
QTextStream out(m_file);
out << buffer << " " << _sCurrentUrl << "\n";
lock.unlock();
if(vStatusCodeV.toInt() == 200) {
QString ApiResponse;
QByteArray data;
data=reply->readAll();
ApiResponse.append(QString::fromUtf8(data));
}
}
}
It appears that to be effective, the deleteLater method must be called from within an event loop, which must regain control of execution to handle garbage collection.
Maybe you should refactor your code to have the event loop replace your while loop. Alternatively, since you're not using the finished slot to process the reply, perhaps you can delete the reply directly at the end of the RequestFinishedHandler function.