Qt: QuaZip to extract file and show its progress in QProgressDialog - c++

How could I use QuaZip to extract a .zip file and show its extraction progress in a QProgressDialog?
I've tried an example from this question1 (and also this question2) without success. Because I need to unzip .zip files (not .gz files) and that code shows % of progress but not unzip the files.
And even that, I don't know how to show that % in a QProgressDialog.
I've been able to extract .zip file using:
JlCompress::extractDir("C:/test/test.zip", "C:/test/");
However, that is not enought for my goal, because I need to show that extraction progress in a QProgressDialog in real time...
This is my code:
QString fileName = "C:/test/firefox-29.0.1.gz"; //I need to unzip .zip files
qDebug() << "Opened";
progressUnzip->setWindowTitle(tr("Unzip Manager"));
progressUnzip->setLabelText(tr("Unzipping %1. \nThis can take a long time to complete").arg(fileName));
progressUnzip->setAttribute(Qt::WA_DeleteOnClose);
QFile file(fileName);
file.open(QFile::ReadOnly);
QuaGzipFile gzip;
gzip.open(file.handle(), QuaGzipFile::ReadOnly);
progressUnzip->show();
while(true) {
QByteArray buf = gzip.read(1000);
//process buf
if (buf.isEmpty()) { break; }
QFile temp_file_object;
temp_file_object.open(file.handle(), QFile::ReadOnly);
double progress = 100.0 * temp_file_object.pos() / file.size();
updateUnzipProgress(temp_file_object.pos(), file.size());
qDebug() << qRound(progress) << "%";
}
unzipFinished();
qDebug() << "Finish";
Where:
void MainWindow::updateUnzipProgress(qint64 bytesRead, qint64 totalBytes)
{
qDebug() << bytesRead << "/" << totalBytes;
qint64 th = 1;
if (totalBytes >= 100000000)
{
th = 1000;
}
progressUnzip->setMaximum(totalBytes/th);
progressUnzip->setValue(bytesRead/th);
}
// When unzip finished or canceled, this will be called
void MainWindow::unzipFinished()
{
qDebug() << "Unzip finished.";
progressUnzip->hide();
}
In this code, QProgressDialog doesn't show any progress bar and at the end the file is not unzipped.
Any idea?
Thanks for your help,

I'm not going to write the whole code for you, but I can give you some hints to help you solve the problem.
QuaZIP library does not provide any Qt signals about the extraction progress, and that means that you should implement it yourself.
First, take a look at JlCompress::extractDir function implementation to see how it works. The source code can be found here. The function creates a QuaZip object which describes the zip archive you want to extract. Then it iterates over the files in the archive. On every iteration it creates a QuaZipFile object which describes a compressed file in the archive and then writes (=extracts) its data to a new file.
This is the copy function:
static bool copyData(QIODevice &inFile, QIODevice &outFile)
{
while (!inFile.atEnd()) {
char buf[4096];
qint64 readLen = inFile.read(buf, 4096);
if (readLen <= 0)
return false;
if (outFile.write(buf, readLen) != readLen)
return false;
}
return true;
}
inFile is the compressed (QuaZipFile) file and outFile is a new file where the compressed data is extracted to.
Now you should understand the underlying logic.
To add signals about extraction progress, you can can copy the JlCompress::extractDir source code to some wrapper class and make some changes.
While iterating over the files in the archive, you can get information about them using QuaZipFile::getFileInfo. That information contains uncompressed file size. Now go back to the copyData function. You know how many bytes you have written and you know the expected file size. That means you can calculate the single file extraction progress. Also you can get total files count and their uncompressed sizes using QuaZip::getFileInfoList64. That will let you calculate the whole extraction progress too.

Related

Arduino IDE ESP32 device using SPIFFS to save

I writing ESP32 program and using SPIFFS to save some data because I do not want to loose it after I power down the device.
I have 2 functions:
char* readFile(fs::FS &fs, const char *path)
{
Serial.printf("Reading file: %s\n", path);
File file = fs.open(path);
if (!file || file.isDirectory())
{
Serial.println("Failed to open file for reading");
return "FAIL";
}
Serial.print("Read from file: ");
while (file.available())
{
Serial.write(file.read());
delayMicroseconds(100);
//TODO
//append all characters and return it as a list of char arrays at the end of reading
}
file.close();
}
void writeFile(fs::FS &fs, const char *path, const char *message)
{
Serial.printf("Writing file: %s\n", path);
File file = fs.open(path, FILE_WRITE);
if (!file)
{
Serial.println("Failed to open file for writing");
return;
}
if (file.print(message))
{
Serial.println("File written");
}
else
{
Serial.println("Write failed");
}
file.close();
}
First of all, I run my program and call writeFile() and then followed by readFile():
int n = 0;
Wifi_setup();
Spiffs_setup();
n = scan_wifi_networks();
Serial.print("list_strings outside scan function = ");
for (int i = 0 ; i<=n ; i++){
Serial.println(found_networks[i]);
}
//compare list_strings with preset wifi networks. If match, everything is fine, if not, problem!
writeFile(SPIFFS, "/wifi.txt", "Telia-33F8A3-Greitas");
char* known_networks = readFile(SPIFFS, "/wifi.txt");
//send USD to the server, go back to sleep
//initialize_deep_sleep();
}
And that part works fine. From the serial monitor, I can see that it have sucesfully written my text to wifi.txt.
After I run write/read functions
Next, I comment out write function and only leave read function.I run the code again and it is not able to read back my text:
Only readFile
Can someone help me understand why is that happening? I thought that if I write to spiffs once, I will be able to access it afterwards but that is not the case. I have previously used EEPROM and that seemed to work. I can write to EEPROM address and simply access the same address later and the value will still be there after the power off. Any help is appreciated. Thanks in advance.
UPDATE1
I have managed to read back data from my SPIFFS after I wrote to it. I had missed one crucial step:
https://randomnerdtutorials.com/install-esp32-filesystem-uploader-arduino-ide/
However, now, I am encountering another issue:
In my project folder, I have created a folder "data" which is supposed to be accessed by SPIFFS. In there, I create 2 files :
wifi.txt
update.bin
update.bin for now is not relevant. Lets talk about wifi.txt
After I have written to Spiffs, I now comment out the writeFile function and only leave out readFile. The result from my serial monitor:
load:0x3fff001c,len:1216
ho 0 tail 12 room 4
load:0x40078000,len:10944
load:0x40080400,len:6388
entry 0x400806b4
Listing directory: /
FILE: /update.bin SIZE: 0
FILE: /wifi.txt SIZE: 20
Reading file: /wifi.txt
Read from file: Telia-33F8A3-Greitas
As you can see from the serial monitor above, my program recognises the text that I have written to my spifs /wifi.txt file. Howeverm when I go to my actual project directory and open data/wifi.txt it is emtpy:
How can this happen? My program recongises that theres data inside but it wont show up in the file.
UPDATE2
I have done some further testing of SPIFFS.
In my data folder, I have created another txt file test.txt. In there, I have manually put some text "this is text message".
In my program, I have called function:
char* returned_data = readFile(SPIFFS, "/test.txt");
And the serial monitor sucesfully printed the message that I have put. So that proves that the SPIFFS is able to read back from the files without any issues.
Then, I have modified my program:
writeFile(SPIFFS, "/test.txt", "hello123");
char* returned_data = readFile(SPIFFS, "/test.txt");
The code above should overwrite whatever I have written to my txt file with "hello123" and then my program should read the "hello123" back from Spiffs. The serial monitor response:
Listing directory: /
FILE: /update.bin SIZE: 0
FILE: /wifi.txt SIZE: 2
FILE: /test.txt SIZE: 8
Writing file: /test.txt
File written
Reading file: /wifi.txt
Read from file:
Reading file: /test.txt
Read from file: hello123
As you can see, it is able to sucesfully read back "hello123".
HOWEVER, when I go to my project folder, open the data/test.txt, I can still see my initial text not replaced with "hello123".
I do not understand how is this happening..

QT How to wait in a main programm for one thread (without using sleep/freezing)

My QT-program have 4 steps in a loop:
reading n config file
transmitting values via UART to uC
recording audio files and receiving values from uC
stopping recording and writing a header (*.PCM to *.WAV conversion)
go to n+1 config file and starting again with step 1.
For audio recording I'm using:
m_file.open(QIODevice::WriteOnly); //audio access mode initialisation
writeHeader(); //writing header to convert PCM to *.wav
m_audio->start(&m_file); //start recording
QTimer::singleShot((record_time_sec+4)*1000), this, SLOT(StopRecording()));
QTimer::singleShot() makes a new thread open and while recording (step 3) is working - the main program is still running and I see that the program try to make direct the next steps in a loop without waiting for stopping recording.
How it is possible to wait for ending of recording threads and then following a next steps?
P.S. I was reading about QCoreApplication::processEvents() but I', not sure about, how to use that function!?
void Measure::processMeas(){
// Declaration xxxxxxx
// Read config files
QDir directory("tralala");
QStringList config_files = directory.entryList(QStringList() << "*.txt", QDir::Files);
config_files_cnt = config_files.count();
foreach(QString filename, config_files) { //config loop
Conf *conf = new Conf(directory.absoluteFilePath(filename));
qDebug() << "Config file: " << filename << " is processed";
// Error message xxxxxxxx
// Prepare measure
qDebug()<<"Debug: Prepare measure";
QStringList lines = conf->getConf_lines(); //read all rows of config files
record_time_100ms = lines.count();
record_time_sec = record_time_100ms / 10;
for(i=0; i < (uint32_t)lines.length(); i++){ //prepare single rows of config files
QString uart;
QString uart_hex;
if(i > 5 && i < lines.length()){
QStringList speed_chunks = lines.at(i).split(","); //split rows by decimal point
m1 = speed_chunks.at(2).toInt();
m2 = speed_chunks.at(3).toInt();
m3 = speed_chunks.at(4).toInt();
m4 = speed_chunks.at(5).toInt();
hextest.resize(10);
hextest[0]=255;
hextest[1]=m1>>8;
hextest[2]=m1;
hextest[3]=m2>>8;
hextest[4]=m2;
hextest[5]=m3>>8;
hextest[6]=m3;
hextest[7]=m4>>8;
hextest[8]=m4;
hextest[9]=238;
qDebug()<< "Transfer data: " << i << " from " << lines.count()-1 << " rows transmitted";
m_serial.write(hextest);
m_serial.waitForBytesWritten(1000);
QThread::usleep(50000);
}//if(lines.at(i).contains(",")) end
}//for end
// Measurement
if(stm==0){
QString path = SAVE_AUDIO_PATH+QDateTime::currentDateTime().toString("yyyyMMdd_hhmmss");
m_file.setFileName(path+".log");
m_audio->StartRecording(path+".wav");
stm=1;
}
qDebug()<<"Measurement takes " << record_time_sec+4 << " seconds";
// Stop measure
measure_cnt++;
// All configs files done
if(measure_cnt>=config_files_cnt){
//m_timer->stop();
qDebug()<<"All measure done";
// qApp->quit();
}
}//foreach
}
*************************************************
void Audio::StartRecording(QString rec_file_path)
{
m_file.setFileName(rec_file_path); //audio recording path format
m_file.open(QIODevice::WriteOnly); //audio access mode initialisation
writeHeader(); //writing header to convert PCM to *.wav (Step 1)
m_audio->start(&m_file); //start recording
QTimer::singleShot((15000), this, SLOT(StopRecording()));
}
//Stop recording
void Audio::StopRecording()
{
m_audio->stop();
writeHeaderFinal();
m_file.close();
}
writeHeader(); and writeFinalHeader(); are just functions to convert PCM to *.Wav
I think the big Problem - i try to combine normal code loop with signal-slot principles!? Maybe I have to split my code to different parts and make a slots - and if one part are over, I have to go to next slot? Its a right logic?

Why does creating a TagLib::FileRef only takes time the first time?

As part of a music player I am developing with C++/Qt, I am scanning all audio files with taglib to get the metadata for the database.
I noticed something interesting.
The first time after a restart, it takes about 100ms and 500ms on my system to create a TagLib::FileRef object. When I use the same file again to create a TagLib::FileRef it takes 0ms, even after I restart the musicplayer.
Here is the function I am using to test that:
bool suffixCheck(const QString &val)
{
if (val.endsWith(".mp3")) {
return true;
}
if (val.endsWith(".m4a")) {
return true;
}
if (val.endsWith(".ogg")) {
return true;
}
return false;
}
void doTaglibThing(const QString &path)
{
if (suffixCheck(path)) {
QElapsedTimer timer;
timer.start();
TagLib::FileRef f(path.toUtf8().data(),
true,
TagLib::AudioProperties::Accurate);
Q_UNUSED(f);
qDebug() << "End taglibThing" << timer.elapsed();
}
}
Why is this? I assume that taglib somehow "remembers" the objects. How can I make it so that taglib doesn't remember and always actually has to read the file.
I want to optimize the library scanning function and I don't always want to restart the whole system to check how changes to the code impact the first-run scan.
As suggested by Frank Osterfeld, the file data was still in disk cache.
Clearing the disk cache with
sync && echo 3 | sudo tee /proc/sys/vm/drop_caches
makes taglib re-read the file from the disk again.

How to save poco uploaded file

I want to upload file using poco library.
Now I have the uploaded file in the istream variable but I don't know how I can save it to a file?
Here is my code where i can get the length of the file.
void handlePart(const MessageHeader &header, std::istream &stream) {
_type = header.get("Content-Type", "(unspecified)");
if (header.has("Content-Disposition")) {
std::string disp;
NameValueCollection params;
MessageHeader::splitParameters(header["Content-Disposition"], disp, params);
_name = params.get("name", "(unnamed)");
_fileName = params.get("filename", "(unnamed)");
}
CountingInputStream istr(stream);
NullOutputStream ostr;
StreamCopier::copyStream(istr, ostr);
_length = istr.chars();
}
Also now it's 1 file uploaded in the form if there be more than 1 file how it will be managed in istream?
Now there is 2 days I'm searching and testing different ways but I couldn't find any way, please help to resolve this problem.
Thank you in advanced.
Depend on #Abhinav question on this post:
We can write save code like this:
if (fileName.length() != 0){
std::ofstream fout( fileName, std::ios::binary);
fileList.push_back(fileName);
fout << stream.rdbuf() ;
fout.close();
}
But unfortunately it's working for one file if there is more than one this code can't catch correctly.

Can't extract exe or dll files with quazip (Qt)

I am building a qt framework to download and install application updates (like sparkle for obj-c). The download works, the downloaded zip file is valid and i can extract the contents manually but when I let my framework unzip the contents via quazip the files (dll and exe) contains this and only this string: "MZ" and a special char which is wrong encoded (some kind of square on windows and "ê" on mac), so exactly 3 bytes. When I include a text file (or xml) in the zip file, it will be unzipped correctly, manually and with quazip, so I assume that the library was compiled correctly. Where is my error?
I think this can be part of the solution http://en.wikipedia.org/wiki/DOS_MZ_executable?
Here is my method to install the update:
QuaZip archiveWrapper(filename); // The downloaded zip file
if (archiveWrapper.open(QuaZip::mdUnzip)) {
QuaZipFile archive(&archiveWrapper);
qDebug() << "Extracting files" << archiveWrapper.getFileNameList();
for (bool more = archiveWrapper.goToFirstFile(); more; more = archiveWrapper.goToNextFile()) {
QString filePath = archiveWrapper.getCurrentFileName();
QString destinationPath = QDir::cleanPath(QDir::currentPath() + QDir::separator() + filePath);
QString destinationBackup = destinationPath + "_backup";
qDebug() << "Extract" << filePath << "to" << destinationPath;
QuaZipFile zip(archive.getZipName(), filePath);
zip.open(QIODevice::ReadOnly);
QByteArray data = zip.readAll();
zip.close();
QFile oldFile(destinationPath);
if (oldFile.exists()) {
qDebug() << "Rename" << destinationPath << "to" << destinationBackup;
if (!oldFile.rename(destinationBackup)) {
qWarning("Could not rename %s to %s!", destinationPath.toUtf8().constData(), destinationBackup.toUtf8().constData());
}
}
QFile destination(destinationPath);
destination.open(QIODevice::WriteOnly);
destination.write(data.data());
destination.close();
if (oldFile.exists()) {
qDebug() << "Deleting backup of" << destinationPath;
if (!oldFile.remove()) {
qWarning("Could not delete %s!", destinationPath.toUtf8().constData());
}
}
}
if (archive.getZipError() == UNZ_OK) {
qDebug() << "All files extracted successfully";
qDebug() << "Restarting application...";
archiveWrapper.close();
qApp->quit();
QProcess::startDetached(qApp->arguments()[0], qApp->arguments());
} else {
qWarning("Error while extracting files (Error %d)", archive.getZipError());
archiveWrapper.close();
}
} else {
qWarning("Could not open archive to extract contents");
}
Edit:
I found out that data (QByteArray) has the expected size, so I think the problem is that QFile does not write the contents of QByteArray into the exe/dll files the way it should be?
Edit 2:
I've found one error, the file size to write:
destination.write(data.data(), data.size());
instead of
destination.write(data.data());
but still, the exe does not have an icon or is executable (but with the correct file size). For a short time a dos window opens and closes. There is a antivirus software running but there is no alert (and because this is a corporate notebook i am not able to shut it down and the update framework should also be running whether there is a antivirus software running or not).
Edit 3:
Although I thought writing exe files is complicated, it was a mixture of stupid bugs I "implemented" for testing purposes. So the simple
QFile destination(destinationPath);
destination.open(QIODevice::WriteOnly);
destination.write(data.data(), data.size())
is sufficient.