Download multiple files from Ftp issue - c++

I want to download some files from my ftp server. The problem is, that only the last one has data, others 0 sized or it crashes when close QFile as a pointer.
My code:
QFtp *ftp = new QFtp(this);
ftp->connectToHost(FTP_HOST, FTP_PORT);
ftp->login(FTP_USERNAME, FTP_PASSWORD);
QFile *reportFile = nullptr;
connect(ftp, &QFtp::listInfo, [this](const QUrlInfo &ftpUrlInfo) {
if (ftpUrlInfo.isFile()) {
reportFile = new QFile("some local path" + ftpUrlInfo.name());
reportFile->open(QIODevice::WriteOnly);
ftp->get("some ftp path" + ftpUrlInfo.name(), reportFile, QFtp::Binary);
}
});
connect(ftp, &QFtp::done, [this]() {
qDebug() << "DONE!";
ftp->close();
ftp->deleteLater();
});
connect(ftp, &QFtp::commandFinished, [this]() {
qDebug() << "COMMAND FINISHED!";
if (reportFile != nullptr) {
reportFile.close();
reportFile->deleteLater();
}
});
ftp->list("ftp path to dir");
So, it should download the file, close it and deleteLater for all files in the ftp directory. Any ideas how to do it? Thanks.

I have finally fixed it!
My code:
QQueue<QFile*> reportQueue; //initialize the queue
connect(ftp, &QFtp::listInfo, [this](const QUrlInfo &ftpUrlInfo) {
if (ftpUrlInfo.isFile()) {
reportQueue.append(new QFile("local path" + "\\" + ftpUrlInfo.name()));
}
});
connect(ftp, &QFtp::done, [this]() {
emit reportsDataFinished();
});
connect(ftp, &QFtp::commandFinished, [this]() {
if (ftp->currentCommand() == QFtp::List) {
proceedDownload();
} else if (ftp->currentCommand() == QFtp::Get) {
reportFile->close();
reportFile->deleteLater();
proceedDownload();
}
});
if (ftp->error() == QFtp::NotConnected) {
emit ftpReportError(ftp->error());
} else {
ftp->list("ftp path to the dir");
}
void Test::proceedDownload()
{
if (!reportQueue.isEmpty()) {
reportFile = reportQueue.dequeue();
reportFile->open(QIODevice::WriteOnly);
QFileInfo ftpFileInfo(reportFile->fileName());
ftp->get("ftp path to file" + "/" + ftpFileInfo.fileName(), reportFile, QFtp::Binary);
}
}
I added the files to the QQueue, when ftp list command has been finished, I use function proceedDownload(). In the function, I dequeue() the queue to the reportFile and proceed with ftp get() function. When get ftp command finishes, I close and delete the file from memory, and again call the proceedDownload(). So the the whole process goes again until the queue is empty. I use emit reportsDataFinished(); signal in the connection to the slot closeFtp() where ftp closes and deleteLater() frees the resources. All files downloads well. Thank you.

Related

Share object between two lambdas

I am trying to share the same object (in my case, a string) between to lambdas using shared_ptr:
auto fileToLoad = make_shared<string>();
StartAsync(
[=]()
{
QDir dir(dirPath.c_str());
QString fileExtension(fileExt.c_str());
for (int i = 0; i < 150; i++)
{
auto files = dir.entryList(QStringList() << fileExtension << fileExtension.toUpper(), QDir::Files);
if (!files.empty())
{
//fileToLoad.reset(new string(files.first().toUtf8().constData()));
fileToLoad = make_shared<string>(files.first().toUtf8().constData());
break;
}
this_thread::sleep_for(chrono::milliseconds(200));
}
},
[=]()
{
if (fileToLoad == nullptr)
{
DisplayMessage("Timeout reached");
}
else
{
doFunc(*fileToLoad);
}
});
After many attempts I have still not been able to make it work as I want: to store fileToLoad in the first lambda and use it in the second. Or it fails to compile due to qualifiers (const), or it compiles but 'fileToLoad' remains empty.
As you can see I am trying to notify if a file has appeared in a folder, using StartAsync function, which takes 2 lambdas (this function basically creates QObjects and move them to background thread and then makes some connections between signals and slots).
Edit:
StartAsync: takes a lambda of a task (lengthy work), and a lambda of post task (UI updates)
void MainWidget::StartAsync(function<void()> func, function<void()> postTask)
{
AsyncTask* task = new AsyncTask(func, [=]() { if (postTask != nullptr) { postTask(); HideProgressBar(); }});
QThread* thread = new QThread();
task->moveToThread(thread);
connect(thread, &QThread::started, this, &MainWidget::ShowProgressBar);
connect(thread, &QThread::started, task, &AsyncTask::RunTask);
connect(task, &AsyncTask::TaskFinished, task, &AsyncTask::RunPostTask);
connect(thread, &QThread::finished, task, &QObject::deleteLater);
connect(thread, &QThread::finished, thread, &QObject::deleteLater);
thread->start();
}
Any suggestions are appreciated.
When you assign a new value to a std::shared_ptr - you create a new shared value, that doesn't have any connection with the previous one (you can have a look at counters of both shared_ptrs). You should change
fileToLoad = make_shared<string>(files.first().toUtf8().constData());
to
*fileToLoad = files.first().toUtf8().constData()
(of course checking beforehand that fileToLoad isn't null)
You code won't work, because in first lambda you overwrite the temporary pointer inside lambda. What you can do is overwrite std::string behind the pointer and it should work fine:
auto fileToLoad = make_shared<string>();
StartAsync(
[=]()
{
QDir dir(dirPath.c_str());
QString fileExtension(fileExt.c_str());
for (int i = 0; i < 150; i++)
{
auto files = dir.entryList(QStringList() << fileExtension << fileExtension.toUpper(), QDir::Files);
if (!files.empty())
{
//fileToLoad.reset(new string(files.first().toUtf8().constData()));
*fileToLoad = files.first().toStdString();
break;
}
this_thread::sleep_for(chrono::milliseconds(200));
}
},
[=]()
{
if (fileToLoad == nullptr)
{
DisplayMessage("Timeout reached");
}
else
{
doFunc(*fileToLoad);
}
});

Memory leak in QSerialBus work

Hello everyone!
I've got problem with my ModBus realisation via QSerialBus library on Qt 5.8.
The problem is - when I try to read a list of discrete inputs on a high speed, the program catch a memory leak - about 300-350Kb per second.
So, here the code of function that causes leak:
void ModBus::queue()
{
if(!_readList.isEmpty())
{
if(!checkState())
{
_readList.clear();
_curPos = DiscInputPos::Full;
queue();
return;
}
QModbusDataUnit readData = QModbusDataUnit(QModbusDataUnit::DiscreteInputs, _readList.first(), 1);
if(auto *reply = _client->sendReadRequest(readData, _serverNum))
{
if(!reply->isFinished())
{
connect(reply, &QModbusReply::errorOccurred, [=] (QModbusDevice::Error e)
{
reply->deleteLater();
if(e == QModbusDevice::TimeoutError)
{
_readList.clear();
_curPos = DiscInputPos::Full;
stop();
start();
queue();
}
});
connect(reply, &QModbusReply::finished, this, [=]
{
if(reply->error() == QModbusDevice::NoError)
{
if(reply->result().valueCount() > 0)
{
_curPos = (DiscInputPos) reply->result().value(0);
if(_curPos == DiscInputPos::Clear)
_readList.removeFirst();
else
_readList.clear();
}
}
else
{
_readList.clear();
_curPos = DiscInputPos::Full;
}
reply->deleteLater();
queue();
});
}
else
delete reply;
}
else
{
delete reply;
_readList.clear();
_curPos = DiscInputPos::Full;
queue();
}
}
else
{
emit sendReadResult(_curPos);
_curPos = DiscInputPos::Full;
}
}
_client — QModbusTcpClient
bool checkState() — does nothing special unless checking Connected state.
Maximum _readList size is 4.
If I comment a part of code with _client->sendReadRequest - everything goes alright without any leak. Can anybody explain - what it can be?
Thanks in advance!
I solved this problem via timer on 0.5 sec (less causes leaks too), that ticks as single shot from class that check the answer from ModBus class. There is still leaks, but much-much less. If 1 sec - there is no leaks.
connect(d, &ModBus::sendReadResult, this, &StatusesWorker::getRead);
void StatusesWorker::getRead(DiscInputPos s)
{
bool cleared = (s == DiscInputPos::Clear);
if(!cleared)
emit badSensors();
else
emit clearSensors();
QTimer::singleShot(500, [=] { checkSensor(id); });
}
}
But I think this is bad solution.
UPD.
Qt Community solved the problem like that:
connect(reply, &QModbusReply::destroyed, [this] { QTimer::singleShot(60, [this] { queue(); }); });
And it's work faster without leaks. Of course, checkSensor() function now without timer.

tbb concurrent_bounded_queue multiple threads access

I have the following code:
tbb::concurrent_bounded_queue<Image> camera_queue_;
camera_queue_.set_capacity(1);
struct Image
{
int hour_;
int minute_;
int second_;
int msec_;
QImage image_;
Image(){hour_ = -1; minute_ = -1; second_ = -1; msec_ = -1; image_ = QImage();}
Image& operator=(Image const& copy)
{
this->hour_ = copy.hour_;
this->minute_ = copy.minute_;
this->second_ = copy.second_;
this->msec_ = copy.msec_;
this->image_ = copy.image_;
return *this;
}
};
In a Qt Thread :
ThreadA:
tbb::concurrent_bounded_queue<Image> image_queue_;
image_queue_.set_capacity(1);
Image cur_image_;
void Worker::process() {
while(1)
{
if(quit_)
break;
{
camera_queue_.pop(cur_image_);
image_queue_.push(cur_image_);
}
emit imageReady();
}
emit finished();
}
Image Worker::getCurrentImage()
{
Image tmp_image;
image_queue_.pop(tmp_image);
return tmp_image;
}
In Another Thread:
ThreadB:
Producer::Producer(){
work_ = new Worker();
work_->moveToThread(workerThread_);
QObject::connect(workerThread_, &QThread::finished, work_, &QObject::deleteLater);
QObject::connect(this, &Producer::operate, work_, &Worker::process);
QObject::connect(work_, &Worker::imageReady, this, &Producer::displayImage);
QObject::connect(this, &Producer::stopDecode, work_, &Worker::stop);
workerThread_->start();
emit operate();
}
void Producer::process() {
while(1)
{
if(quit_)
break;
{
camera_queue_.push(GetImage());
}
}
}
void Producer::displayImage()
{
Image tmp = std::move(work_->getCurrentImage());
widget_->showImage(tmp.image_);
}
However, In main thread, I have a function that enables user to click a button to get current image:
bool Producer::SaveImage()
{
Image img = std::move(work_->getCurrentImage());
std::string fileName = std::to_string(img.hour_) + "-" + std::to_string(img.minute_) + "-" + std::to_string(img.second_) + "-" + std::to_string(img.msec_/1000) + ".jpg";
std::string outFileName = folder + "/" + fileName;
return img.image_.save(QString::fromStdString(outFileName));
}
The problem is:
When user does not click the button to invoke Producer::SaveImage(), the Image Decoding and Showing runs smoothly. But when user invoke Producer::SaveImage(), the whole program will get stuck (Caton phenomenon ?). The GUI response becomes not that smooth. The more user invokes SaveImage, the slower the GUI response becomes.
Can anyone help to explain why ? Is there a way to solve that ?
Why do you want to use concurrent queue? It looks like there is a syncronization mechanism in place and you rely mostly on it instead of using concurrent_queue for synchronisation and communication as it is supposed for.
The issue is that when you set capacity = 1, both operations of concurrent_bounded_queue will block until there is enough space of items in the queue. E.g. if the queue contains an item already, the push operation will block. And since you control your threads with another notification mechanism, you might catch a deadlock.
In particular, try to swap the operations like below:
camera_queue_.pop(cur_image_);
emit imageReady();
image_queue_.push(cur_image_);
This should prepare the thread which receives images (if I understand this correctly) and it will block on its image_queue_.pop() method, then this thread will put new image and unblocks the recipient. There might be other issues similar to this, so, please rethink all of your synchronization.

Open function in Qt

I have been creating an application in qt. I have added open function in it, but was happening that when I open an already open file, it again opens the same file instead of pointing it to already opened file.
void MainWindow::actionOpen()
{
QFileInfo fileInfo = UIUtils::openFile(this);
if (!fileInfo.exists()) {
return;
}
if (!MainWindow::mdiMode && !maybeSave()) {
return;
}
openFile(fileInfo.filePath());
}
Please help me to solve the problem.
I'm not 100 % clear on what the issue is, but if you don't want to run the openFile function on files you've previously opened, you have to keep a record.
For instance, define a QList in your header file:
QList<QFileInfo> knownFiles;
And then keep it up to date and check it:
void MainWindow::actionOpen()
{
QFileInfo fileInfo = UIUtils::openFile(this);
if (!fileInfo.exists()) {
return;
}
if (!MainWindow::mdiMode && !maybeSave()) {
return;
}
bool fileIsKnown = false;
for(qint64 i=0; i<knownFiles.length(); i++)
if(knownFiles.at(i) == fileInfo)
{
fileIsKnown = true;
break;
}
if(!fileIsKnown)
{
knownFiles << fileInfo;
openFile(fileInfo.filePath());
}
}
I don't know what you mean when you say "instead of pointing it to already opened file.".

how to return an Qt object from the result of an thread (Qtfutur)

i'm trying to load some data from a server and fill a Qt list. i want to run the dowloanding in a thread. so there is the code of:
principal function in the App.cpp
loadInterestPlaces(QString& urlInterestPlaces) {
LoadData* Data = new LoadData(urlInterestPlaces);
QFuture< list <InterestPlace *> > future = QtConcurrent::run(Data,
&LoadData::startLoading);
// Invoke our onLoadingFinished slot after the loading has finished.
bool ok = connect(&m_watcher, SIGNAL(finished()), this,
SLOT(onLoadingFinished()));
Q_ASSERT(ok);
Q_UNUSED(ok);
// starts watching the given future
m_watcher.setFuture(future);
}
void ApplicationUI::onLoadingFinished() {
qDebug() << "Loading finished";
interestPlacesList = m_watcher.future().result();
qDebug() << "List size= " << interestPlacesList.size();
}
}
the LoadData.cpp file : this is the code of the startloanding function :
std::list<InterestPlace *> LoadData::startLoading()
{
QNetworkAccessManager* netManager = new QNetworkAccessManager(this);
const QUrl url(_URL);
QNetworkRequest request(url);
QNetworkReply* reply = netManager->get(request);
netManager->moveToThread(this->thread());
netManager->setParent(this);
bool ok = connect(reply, SIGNAL(finished()), this, SLOT(onReplyFinished()));
qDebug() << reply->isFinished();
Q_ASSERT(ok);
Q_UNUSED(ok);
qDebug() << "load data: liste size" <<interestPlacesList.size();
return interestPlacesList;
}
Finally inside the SLOT onreplyfinished i parse the data and fill the list.
But the problem here, the QFuture is finished before the downloading so that the list is always empty.
How could i return the list filled just after the execution of the onReplyFinished ?
You may be making this more complex than you need to. Here is what I do to get data off the web. In my case I'm downloading a large ZIP file, writing it out and then unziping it, but the basic steps are the same (I have omitted a lot of my specific code for clarity):
void HtmlDownloader::startDownload() {
// Creates the network request and sets the destination URL
QNetworkRequest request = QNetworkRequest();
request.setUrl(mUrl);
// Creates the network access manager and connects a custom slot to its
// finished signal. Checks the return value for errors.
QNetworkAccessManager *networkAccessManager =
new QNetworkAccessManager(this);
bool c = connect(networkAccessManager, SIGNAL(finished(QNetworkReply*)),
this, SLOT(requestFinished(QNetworkReply*)));
Q_ASSERT(c);
// Indicate that the variable res isn't used in the rest of the app, to prevent
// a compiler warning
Q_UNUSED(c);
// Sends the request
QNetworkReply *reply = networkAccessManager->get(request);
c = connect(reply, SIGNAL(downloadProgress(qint64,qint64)), this, SLOT(onDownloadProgress(qint64,qint64)));
Q_ASSERT(c);
}
void HtmlDownloader::requestFinished(QNetworkReply *reply) {
// Handle the reply data...
// Check the network reply for errors
if (reply->error() == QNetworkReply::NoError) {
// use reply->readAll() or reply->read() to get the returned data
// and process into a list. The list can't be returned but could
// be sent as a payload of a signal, or stored in a data member and
// a signal sent to indicate it is available.
}
reply->deleteLater();
}
void HtmlDownloader::onDownloadProgress(qint64 bytesRecieved, qint64 bytesTotal) {
emit downloadProgress(bytesRecieved, bytesTotal);
}