For an old tool using Qt 5.3.2 we recently encountered a problem. Using QNetwork manager we are downloading files but before we download them we do a check whether they can be downloaded from a certain region. To do this we do an abort at the very first progress indication.
Since recently we do not get a downloadProgress signal any more after some aborts and it only happens with a very fast network connection. Over WiFi it doesn't reproduce. What could be the problem?
Some code snippets:
StartDownload:
networkManager->get(request);
networkReply->setReadBufferSize(4 * 1024 * 1024);
connect(networkReply, SIGNAL(finished()),
this, SLOT(onDownloadFinished()));
connect(networkReply, SIGNAL(downloadProgress(qint64, qint64)),
this, SLOT(onDownloadProgress()));
connect(networkReply, SIGNAL(error(QNetworkReply::NetworkError)),
this, SLOT(onDownloadError(QNetworkReply::NetworkError)));
connect(networkReply, SIGNAL(sslErrors(QList<QSslError>)),
this, SLOT(onDownloadSslErrors(QList<QSslError>)));
connect(networkReply, SIGNAL(encrypted()),
this, SLOT(onEncrypted()));
OnDownloadProgress:
void DownloadManager::onDownloadProgress()
{
networkReply->abort();
nrAborts++;
startDownload(myurl);
}
OnDownloadFinished:
void DownloadManager::onDownloadFinished()
{
networkReply->deleteLater();
}
As the read buffer was full before the first onDownloadProgress and was never read all got stuck. Subscribing to readyRead and reading the data with networkReply->readAll() causes the buffer to be flushed and the downloadProgress signals to be called.
Related
I want to use a recursive procedure to iterate through a large number of images in Qt: essentially the image is repeatedly quartered (up to a limit) and the user is asked whether the image passes or fails - ie if the image passes at large dimensions we call our function again with smaller dimensions (until we reach the limit), if it fails we return and so pass back up the hierarchy.
This approach seems to run into a roadblock with Qt's event-driven approach - I cannot see how I can pause the loop while waiting for the user input - ie there is nothing like a "wait_for_button_press" method.
I know that this sort of approach is regarded as an anti-pattern in event driven programming, but what is the alternative way that doesn't involve holding lots and lots of state on the heap (as opposed to getting it held for 'free' on the stack)?
QEventLoop maybe could help you. I start a http connection aside a timer with a timeout, all inside a thread. Then a wait for one of those had finish and return.
void MyThread::run(){
QNetworkAccessManager qnaManager;
bool isPost = false;
QUrl url(myUrl);
QNetworkRequest req(url);
QNetworkReply *reply;
req.setHeader(QNetworkRequest::ContentTypeHeader,
"application/json");
req.setHeader(QNetworkRequest::ContentLengthHeader,
QVariant(postData.size()).toString());
reply = qnaManager.get(req);
QEventLoop eventLoop;
QTimer timer;
timer.setSingleShot(true);
const int timeout = 400;
timer.start(timeout);
connect(&timer, SIGNAL(timeout()), &eventLoop, SLOT(quit()));
connect(reply, SIGNAL(finished()), &eventLoop, SLOT(quit()));
eventLoop.exec();
if (timer.isActive()){
//everything is ok
}else{
//timer elapsed, no replay
return;
}
}
For what it's worth, in the end I decided that the best route was to implement more message passing code - wait for the user input to dispatch a message. It was longer code than if I had used/had available the 'traditional' call back type paradigm, but it worked cleanly in the end.
I'm using two ip cameras from Axis Communications for my application. I want to check if the cameras are ever disconnected. Is it possible to do so by the use of QNetworkAccessManager and QNetworkReply?
The prototype of what I tried is like the following way:
QNetworkAccessManager *m_networkAccessManager = new QNetworkAccessManager();
QNetworkReply *m_networkReply = m_networkAccessManager->get(QNetworkRequest(camUrl));
if (!m_networkReply)
{
delete m_networkAccessManager;
qDebug()<<"Camera not found"<<endl;
}
connect(m_networkReplyCam, SIGNAL(error(QNetworkReply::NetworkError)), this, SLOT(cameraDisconnected(QNetworkReply::NetworkError)));
void MainWindow::cameraDisconnected(QNetworkReply::NetworkError)
{
qDebug()<<"Camera Disconected"<<endl;
}
But the slot cameraDisconnected never seem to gets invoked. I also tried connect with finished() signal from QNetworkReply like the following:
connect(m_networkReplyCam, SIGNAL(finished()), this, SLOT(cameraDisconnected()));
But even then the slot never gets called.
What am I doing wrong?
Thanks.
Your request will timeout after some time if the host is unreachable. There is no built-in way to set specific timeout in QNetworkRequestor QNetworkAccessManager so one possibility is to implement your own timer to abort after a particular time. There are multiple examples to do that like 1, 2 and 3.
I got strange problem. QProcess just not working!
And error is unknown.
I got global var in header
QProcess *importModule;
An I got this function ( I tried both start and startDetached methods btw )
void App::openImport(){
importModule = new QProcess();
importModule->setWorkingDirectory(":\\Resources");
importModule->startDetached("importdb_module.exe");
QMessageBox::information(0,"",importModule->errorString());
}
It jsut outputs that error is unknown. Also it wont start other exes like
void App::openImport(){
importModule = new QProcess();
importModule->setWorkingDirectory("C:\\Program Files\\TortoiseHg");
importModule->startDetached("hg.exe");
QMessageBox::information(0,"",importModule->errorString());
}
What I've done wrong?
And is there other ways to run some .exe from my programm?
Or maybe .bat files(which runs exes)? (Tried with QProcess too, not working)
startDetached() is a static method and doesn't operate on importModule at all.
It starts a process and then stops caring. Thus the error()/errorState() in importModule has nothing to do with the startDetached() call. What you want is start().
However, as QProcess is asynchronous, nothing will have happened yet immediately after start() returns. You must connect to the started(), error() and finished() signals to learn about the result.
connect(importModule, SIGNAL(started()), this, SLOT(importModuleStarted()));
connect(importModule, SIGNAL(finished(int, QProcess::ExitStatus)), this, SLOT(importModuleFinished(int, QProcess::ExitStatus)));
CONNECT(importModule, SIGNAL(error(QProcess::ProcessError)), this, SLOT(importModuleError(QProcess::ProcessError)));
importModule->start(QStringLiteral("importdb_module"), QStringList());
Alternatively you can use the blocking wait functions:
importModule->start(QStringLiteral("importdb_module"), QStringList());
importModule->waitForStarted(); // waits until starting is completed
importModule->waitForFinished(); // waits until the process is finished
However, I strongly advise against using them in the main thread, as they block the UI then.
When using QTcpSocket to receive data, the signal to use is readyRead(), which signals that new data is available.
However, when you are in the corresponding slot implementation to read the data, no additional readyRead() will be emitted.
This may make sense, as you are already in the function, where you are reading all the data that is available.
Problem description
However assume the following implementation of this slot:
void readSocketData()
{
datacounter += socket->readAll().length();
qDebug() << datacounter;
}
What if some data arrives after calling readAll() but before leaving the slot?
What if this was the last data packet sent by the other application (or at least the last one for some time)?
No additional signal will be emitted, so you have to make sure to read all the data yourself.
One way to minimize the problem (but not avoid it totally)
Of course we can modify the slot like this:
void readSocketData()
{
while(socket->bytesAvailable())
datacounter += socket->readAll().length();
qDebug() << datacounter;
}
However, we haven't solved the problem. It is still possible that data arrives just after the socket->bytesAvailable()-check (and even placing the/another check at the absolute end of the function doesn't solve this).
Making sure to be able to reproduce the problem
As this problem of course happens very rarely, I stick to the first implementation of the slot, and I'll even add a an artificial timeout, to be sure that the problem occurs:
void readSocketData()
{
datacounter += socket->readAll().length();
qDebug() << datacounter;
// wait, to make sure that some data arrived
QEventLoop loop;
QTimer::singleShot(1000, &loop, SLOT(quit()));
loop.exec();
}
I then let another application send 100,000 bytes of data.
This is what happens:
new connection!
32768 (or 16K or 48K)
The first part of the message is read, but the end isn't read anymore, as readyRead() won't be called again.
My question is: what is the best way to be sure, this problem never occurs?
Possible solution
One solution I came up with is calling the same slot again at the end again, and to check at the beginning of the slot, if there is any more data to read:
void readSocketData(bool selfCall) // default parameter selfCall=false in .h
{
if (selfCall && !socket->bytesAvailable())
return;
datacounter += socket->readAll().length();
qDebug() << datacounter;
QEventLoop loop;
QTimer::singleShot(1000, &loop, SLOT(quit()));
loop.exec();
QTimer::singleShot(0, this, SLOT(readSocketDataSelfCall()));
}
void readSocketDataSelfCall()
{
readSocketData(true);
}
As I don't call the slot directly, but use QTimer::singleShot(), I assume that the QTcpSocket can't know that I'm calling the slot again, so the problem that readyRead() isn't emitted can't happen anymore.
The reason why I have included the parameter bool selfCall is that the slot which is called by the QTcpSocket isn't allowed to exit sooner, else the same problem can occur again, that data arrives exactly at the wrong moment and readyRead() isn't emitted.
Is this really the best solution to solve my problem?
Is the existence of this problem a design error in Qt or am I missing something?
Short answer
The documentation of QIODevice::readyRead() states:
readyRead() is not emitted recursively; if you reenter the event loop or call waitForReadyRead() inside a slot connected to the readyRead() signal, the signal will not be reemitted.
Thus, make sure that you
don't instantiate a QEventLoop inside your slot,
don't call QApplication::processEvents() inside your slot,
don't call QIODevice::waitForReadyRead() inside your slot,
don't use the same QTcpSocket instance within different threads.
Now you should always receive all data sent by the other side.
Background
The readyRead() signal is emitted by QAbstractSocketPrivate::emitReadyRead() as follows:
// Only emit readyRead() when not recursing.
if (!emittedReadyRead && channel == currentReadChannel) {
QScopedValueRollback<bool> r(emittedReadyRead);
emittedReadyRead = true;
emit q->readyRead();
}
The emittedReadyRead variable is rolled back to false as soon as the if block goes out of scope (done by the QScopedValueRollback). So the only chance to miss a readyRead() signal is when the control flow reaches the if condition again before the processing of the last readyRead() signal has finished (in other words, when there would be a recursion).
And a recursion should only be possible in the situations listed above.
I think scenario mentioned in this topic has two major cases which works differently, but in general QT doesn't have this problem at all and I will try to explain below why.
First case: Single threaded application.
Qt uses select() system call to poll open file descriptor for any change happened or operations available. Simple saying on every loop Qt checks if any of opened file descriptors have data available to read/closed etc. So on single threaded application flow looks like that (code part simplified)
int mainLoop(...) {
select(...);
foreach( descriptor which has new data available ) {
find appropriate handler
emit readyRead;
}
}
void slotReadyRead() {
some code;
}
So what will happend if new data arrived while program still inside slotReadyRead.. honestly nothing special. OS will buffer data, and as soon as control will return to next execute of select() OS will notify software that there are data available for particular file handle. It works in absolutely the same way for TCP sockets/files etc.
I can imaging situations where (in case of really long delays in slotReadyRead and a lot of data coming) you can experience an overrun within OS FIFO buffers (for example for serial ports) but that has more to do with a bad software design rather then QT or OS problems.
You should look on slots like readyRead like on a interrupt handlers and keep their logic only within fetch functionality which fills your internals buffers while processing should be done in separate threads or while application on idle etc.. Reason is that any such application in general is a mass service system and if it spends more time on serving one request then a time interval between two requests it's queue will overrun anyway.
Second scenario: multithreaded application
Actually this scenario is not that much differ from 1) expect that you should design right what happens in each of your threads. If you keep main loop with light wighted 'pseudo interrupt handlers' you will be absolutely fine and keep processing logic in other threads, but this logic should work with your own prefetch buffers rather then with QIODevice.
The problem is quite interesting.
In my program the usage of QTcpSocket is very intensive. So I've written the whole library, that breaks outgoing data into packages with a header, data identifier, package index number and maximum size, and when the next piece of data comes, I know exactly where it belongs to. Even if I miss something, when the next readyRead comes, the receiver reads all and compose received data correctly. If the communication between your programs is not so intense, you could do the same, but with timer (which is not very fast, but solves the problem.)
About your solution. I don't think it's better then this:
void readSocketData()
{
while(socket->bytesAvailable())
{
datacounter += socket->readAll().length();
qDebug() << datacounter;
QEventLoop loop;
QTimer::singleShot(1000, &loop, SLOT(quit()));
loop.exec();
}
}
The problem of both methods is the code right after leaving the slot, but before returning from emitting the signal.
Also you could connect with Qt::QueuedConnection.
Here are some examples of ways to get the whole file, but using some other parts of the QNetwork API:
http://qt-project.org/doc/qt-4.8/network-downloadmanager.html
http://qt-project.org/doc/qt-4.8/network-download.html
These examples show a stronger way to handle the TCP data, and when buffers are full, and better error handling with a higher level api.
If you still want to use the lower level api, here is a post with a great way to handle the buffers:
Inside your readSocketData() do something like this:
if (bytesAvailable() < 256)
return;
QByteArray data = read(256);
http://www.qtcentre.org/threads/11494-QTcpSocket-readyRead-and-buffer-size
EDIT: Additional examples of how to interact with QTCPSockets:
http://qt-project.org/doc/qt-4.8/network-fortuneserver.html
http://qt-project.org/doc/qt-4.8/network-fortuneclient.html
http://qt-project.org/doc/qt-4.8/network-blockingfortuneclient.html
Hope that helps.
If a QProgressDialog shall be shown while receiving data from a socket it only works if any QApplication::processEvents() are sent (e.g. by the QProgessDialog::setValue(int) methode). This of course leads to the loss of readyRead signals as mentioned above.
So my workaround was a while loop including the processEvents command such as:
void slot_readSocketData() {
while (m_pSocket->bytesAvailable()) {
m_sReceived.append(m_pSocket->readAll());
m_pProgessDialog->setValue(++m_iCnt);
}//while
}//slot_readSocketData
If the slot is called once any additional readyRead signals can be ignored because the bytesAvailable() always returns the actual number after the processEvents call. Only on pausing of the stream the while loop ends. But then the next readReady is not missed and starts it again.
I had the same problem right away with the readyRead slot. I disagree with the accepted answer; it doesn't solve the problem. Using bytesAvailable as Amartel described was the only reliable solution I found. Qt::QueuedConnection had no effect. In the following example, I'm deserializing a custom type, so it's easy to predict a minimum byte size. It never misses data.
void MyFunExample::readyRead()
{
bool done = false;
while (!done)
{
in_.startTransaction();
DataLinkListStruct st;
in_ >> st;
if (!in_.commitTransaction())
qDebug() << "Failed to commit transaction.";
switch (st.type)
{
case DataLinkXmitType::Matrix:
for ( int i=0;i<st.numLists;++i)
{
for ( auto it=st.data[i].begin();it!=st.data[i].end();++it )
{
qDebug() << (*it).toString();
}
}
break;
case DataLinkXmitType::SingleValue:
qDebug() << st.value.toString();
break;
case DataLinkXmitType::Map:
for (auto it=st.mapData.begin();it!=st.mapData.end();++it)
{
qDebug() << it.key() << " == " << it.value().toString();
}
break;
}
if ( client_->QIODevice::bytesAvailable() < sizeof(DataLinkListStruct) )
done = true;
}
}
i got the same problem, rather use signal readyRead() and socket.readall, i' m trying the following, just after connect without be sure :
QByteArray RBuff;
if(m_socket->waitForConnected(3000))
{
while (m_socket->ConnectedState == QAbstractSocket::ConnectedState) {
RBuff = m_socket->read(2048);
SocketRead.append(RBuff);
if (!SocketRead.isEmpty() && SocketRead.length() == 2048)
{
readData(SocketRead);
SocketRead.remove(0,2048);
}
QCoreApplication::processEvents(QEventLoop::AllEvents, 100);
}
//m_socket->close();*/
}
else
{
I have a program which is supposed to download files over the net. And everything is fine as long as files are small'ish(<1GB). But when files are larger than 1GB one of two things happen:
a) if I connect signal from QNetworkReply with download progress to one of my slots, application crashes with std::bad_alloc
b) if I don't connect any signal from QNetworkReply, application stops on 73%.
Any help mostly appreciated.
Here is the code I use in my connection, and slot:
QNetworkReply* reply = network_access_manager_->get(request);
connect(reply,SIGNAL(downloadProgress(qint64,qint64)),parent_,SLOT(downloadProgress(qint64,qint64)));
And here is the slot:
void MainWindow::downloadProgress(qint64 bytesReceived,qint64 bytesTotal)
{
try
{
ui->label->setText(QString::number(bytesReceived));
ui->label_2->setText(QString::number(bytesTotal));
ui->progressBar->setRange(0,bytesTotal);
ui->progressBar->setValue(bytesReceived);
}
catch(std::exception& e)
{
qDebug() << e.what();
}
}
Read the documentation of QNetworkReply:
QNetworkReply is a sequential-access QIODevice, which means that once
data is read from the object, it no longer kept by the device. It is
therefore the application's responsibility to keep this data if it
needs to. Whenever more data is received from the network and
processed, the readyRead() signal is emitted.
Therefore, just read from QNetworkReply during download, whenever you want, using the readyRead() signal, and write to file or wherever you want.
Here you can find some other information (read Frank Osterfeld's answer in particular): Downloading File in Qt From URL.