Arduino IDE ESP32 device using SPIFFS to save - c++

I writing ESP32 program and using SPIFFS to save some data because I do not want to loose it after I power down the device.
I have 2 functions:
char* readFile(fs::FS &fs, const char *path)
{
Serial.printf("Reading file: %s\n", path);
File file = fs.open(path);
if (!file || file.isDirectory())
{
Serial.println("Failed to open file for reading");
return "FAIL";
}
Serial.print("Read from file: ");
while (file.available())
{
Serial.write(file.read());
delayMicroseconds(100);
//TODO
//append all characters and return it as a list of char arrays at the end of reading
}
file.close();
}
void writeFile(fs::FS &fs, const char *path, const char *message)
{
Serial.printf("Writing file: %s\n", path);
File file = fs.open(path, FILE_WRITE);
if (!file)
{
Serial.println("Failed to open file for writing");
return;
}
if (file.print(message))
{
Serial.println("File written");
}
else
{
Serial.println("Write failed");
}
file.close();
}
First of all, I run my program and call writeFile() and then followed by readFile():
int n = 0;
Wifi_setup();
Spiffs_setup();
n = scan_wifi_networks();
Serial.print("list_strings outside scan function = ");
for (int i = 0 ; i<=n ; i++){
Serial.println(found_networks[i]);
}
//compare list_strings with preset wifi networks. If match, everything is fine, if not, problem!
writeFile(SPIFFS, "/wifi.txt", "Telia-33F8A3-Greitas");
char* known_networks = readFile(SPIFFS, "/wifi.txt");
//send USD to the server, go back to sleep
//initialize_deep_sleep();
}
And that part works fine. From the serial monitor, I can see that it have sucesfully written my text to wifi.txt.
After I run write/read functions
Next, I comment out write function and only leave read function.I run the code again and it is not able to read back my text:
Only readFile
Can someone help me understand why is that happening? I thought that if I write to spiffs once, I will be able to access it afterwards but that is not the case. I have previously used EEPROM and that seemed to work. I can write to EEPROM address and simply access the same address later and the value will still be there after the power off. Any help is appreciated. Thanks in advance.
UPDATE1
I have managed to read back data from my SPIFFS after I wrote to it. I had missed one crucial step:
https://randomnerdtutorials.com/install-esp32-filesystem-uploader-arduino-ide/
However, now, I am encountering another issue:
In my project folder, I have created a folder "data" which is supposed to be accessed by SPIFFS. In there, I create 2 files :
wifi.txt
update.bin
update.bin for now is not relevant. Lets talk about wifi.txt
After I have written to Spiffs, I now comment out the writeFile function and only leave out readFile. The result from my serial monitor:
load:0x3fff001c,len:1216
ho 0 tail 12 room 4
load:0x40078000,len:10944
load:0x40080400,len:6388
entry 0x400806b4
Listing directory: /
FILE: /update.bin SIZE: 0
FILE: /wifi.txt SIZE: 20
Reading file: /wifi.txt
Read from file: Telia-33F8A3-Greitas
As you can see from the serial monitor above, my program recognises the text that I have written to my spifs /wifi.txt file. Howeverm when I go to my actual project directory and open data/wifi.txt it is emtpy:
How can this happen? My program recongises that theres data inside but it wont show up in the file.
UPDATE2
I have done some further testing of SPIFFS.
In my data folder, I have created another txt file test.txt. In there, I have manually put some text "this is text message".
In my program, I have called function:
char* returned_data = readFile(SPIFFS, "/test.txt");
And the serial monitor sucesfully printed the message that I have put. So that proves that the SPIFFS is able to read back from the files without any issues.
Then, I have modified my program:
writeFile(SPIFFS, "/test.txt", "hello123");
char* returned_data = readFile(SPIFFS, "/test.txt");
The code above should overwrite whatever I have written to my txt file with "hello123" and then my program should read the "hello123" back from Spiffs. The serial monitor response:
Listing directory: /
FILE: /update.bin SIZE: 0
FILE: /wifi.txt SIZE: 2
FILE: /test.txt SIZE: 8
Writing file: /test.txt
File written
Reading file: /wifi.txt
Read from file:
Reading file: /test.txt
Read from file: hello123
As you can see, it is able to sucesfully read back "hello123".
HOWEVER, when I go to my project folder, open the data/test.txt, I can still see my initial text not replaced with "hello123".
I do not understand how is this happening..

Related

How to fix midi inputfilestream error in JUCE?

I am trying to read the midi file into a sequence in JUCE, but when I run it, there is an error showed up saying there is something wrong with my inputfilestream.
I started from the audio-application template and write the midi read-in code in getNextBlock() function.
File midiPath("/Desktop/input.midi");
FileInputStream myStream(midiPath);
MidiFile midifile;
midifile.readFrom(myStream);
int NumofTrack = midifile.getNumTracks();
std::cout<<"The track number:"<<NumofTrack<<std::end;
I just wanna test the readin function in the JUCE, and it doesn't work. The error showed as "juce_FileInputStream.cpp":
int64 FileInputStream::getTotalLength()
{
// You should always check that a stream opened successfully before using it!
jassert (openedOk()); <-- errror here
...
Try:
File midiPath = File::getSpecialLocation(File::SpecialLocationType::userDesktopDirectory).getChildFile("input.midi");
FileInputStream myStream(midiPath);
MidiFile midifile;
midifile.readFrom(myStream);
int NumofTrack = midifile.getNumTracks();
std::cout<<"The track number:"<<NumofTrack<<std::end;

fopen / ofstream::open fail when creating a BMP file

Years ago I created a C++ function using FILE to create bitmap files. Recently (not sure when or why) this code is now failing when opening the file. The problem is with the open call ...
file_ptr = fopen("ScreenShots/Screenshot1.bmp", "wb");
Currently this results in an error 13, permission denied error. Change the filename extension to something else and the fopen works fine. For example,
file_ptr = fopen("ScreenShots/Screenshot1.bm2", "wb");
The file saves correctly and when changing the extension back to BMP I can display the file correctly in Paintshop.
Did a quick check using ofstream and same problem.
Any ideas why I get a permission denied error when trying to open BMP files to write data? For information I am using Visual Studio Community 2017 on Windows 10.
To give the complete section of code ...
BITMAPFILEHEADER bitmap_header;
BITMAPINFOHEADER bitmap_info;
FILE *file_ptr;
unsigned int count;
unsigned char tempRGB;
char filename[256];
bool finished;
// CREATE A UNIQUE FILENAME
count = 1;
finished = false;
do
{
// CREATE NAME
sprintf(filename, "ScreenShots/Screenshot%d.bmp", count);
// CHECK IF FILE EXISTS
errno = 0;
file_ptr = fopen(filename, "rb");
if (file_ptr)
{
// FILE EXISTS
fclose(file_ptr);
count = count + 1;
}
else
{
// UNIQUE FILENAME
file_ptr = fopen(filename, "wb");
if (file_ptr == NULL)
{
// UNABLE TO OPEN FOR WRITING - GIVE UP
// (USING OWN LOGGING CLASS)
jalog.log("\nERROR on Screenshot >");
jalog.log(filename);
jalog.log("< >");
jalog.log((short)errno);
return;
}
finished = true;
}
}
while (finished == false);
I've managed to find the issue ... Avast antivirus. I noticed that trying to do an open action for a BMP file took a few seconds while opening any other file type (successfully or unsuccessfully) was instantaneous. As something similar happens when running new programs I tried disabling all the Avast shields and I could successfully create a BMP file using the existing code.
For my own personal use I can whitelist my own programs, but annoying if I get to distributing the program to other people.
Thanks for the help ... and sorry for raising a C++ issue that in the end had nothing to do with C++!

Write from Serial Port to SD card

I am trying to write from the Serial Port to an SD Card in my Arduino Mega 2560, using a card module.
I want to be able to write in a txt file what I type in the serial com.
#include <SPI.h>
#include <SD.h>
const int chipSelect = 4;
void setup() {
// put your setup code here, to run once:
Serial.begin(9600);
Serial.print("This is a test and should be ignored");
if (!SD.begin(chipSelect)) {
Serial.println("\nCard failed, or not present");
// don't do anything more:
return;
}
else{
Serial.println("\ncard initialized.");
}
}
void loop() {
// put your main code here, to run repeatedly
File OpenFile = SD.open("test.txt", FILE_WRITE);
if(OpenFile and Serial.available());
{
OpenFile.println(Serial1.read());
OpenFile.close();
}
}
However a continous line of "-1" and "1", without the ", is written to the SD.
Yes, I am able to write to the SD card through other methods...
Cheers, PoP
I notice you are checking Serial.available() but using Serial1 to read from :)
As you have a Mega, you wouldn't get an error as there is Serial and Serial1. I'd say this is your culprit!
The Stream read function will return -1 when there is no data. Also you could lessen the load on your Arduino and do the operation all at once (not open/close for each byte) and purge all available data (Serial.read() only reads a single byte in case you did not know).
void loop() {
File OpenFile = SD.open("test.txt", FILE_WRITE);
if(OpenFile){
while(Serial.available()){
OpenFile.println(Serial.read());
}
OpenFile.close();
}
}
You may want to check if the SD lib supports appending by default or a flag like FILE_APPEND as you will overwrite the file on the next loop if more data becomes available (Serial data isn't instant, your code may loop while receiving the rest of the data).

Qt: QuaZip to extract file and show its progress in QProgressDialog

How could I use QuaZip to extract a .zip file and show its extraction progress in a QProgressDialog?
I've tried an example from this question1 (and also this question2) without success. Because I need to unzip .zip files (not .gz files) and that code shows % of progress but not unzip the files.
And even that, I don't know how to show that % in a QProgressDialog.
I've been able to extract .zip file using:
JlCompress::extractDir("C:/test/test.zip", "C:/test/");
However, that is not enought for my goal, because I need to show that extraction progress in a QProgressDialog in real time...
This is my code:
QString fileName = "C:/test/firefox-29.0.1.gz"; //I need to unzip .zip files
qDebug() << "Opened";
progressUnzip->setWindowTitle(tr("Unzip Manager"));
progressUnzip->setLabelText(tr("Unzipping %1. \nThis can take a long time to complete").arg(fileName));
progressUnzip->setAttribute(Qt::WA_DeleteOnClose);
QFile file(fileName);
file.open(QFile::ReadOnly);
QuaGzipFile gzip;
gzip.open(file.handle(), QuaGzipFile::ReadOnly);
progressUnzip->show();
while(true) {
QByteArray buf = gzip.read(1000);
//process buf
if (buf.isEmpty()) { break; }
QFile temp_file_object;
temp_file_object.open(file.handle(), QFile::ReadOnly);
double progress = 100.0 * temp_file_object.pos() / file.size();
updateUnzipProgress(temp_file_object.pos(), file.size());
qDebug() << qRound(progress) << "%";
}
unzipFinished();
qDebug() << "Finish";
Where:
void MainWindow::updateUnzipProgress(qint64 bytesRead, qint64 totalBytes)
{
qDebug() << bytesRead << "/" << totalBytes;
qint64 th = 1;
if (totalBytes >= 100000000)
{
th = 1000;
}
progressUnzip->setMaximum(totalBytes/th);
progressUnzip->setValue(bytesRead/th);
}
// When unzip finished or canceled, this will be called
void MainWindow::unzipFinished()
{
qDebug() << "Unzip finished.";
progressUnzip->hide();
}
In this code, QProgressDialog doesn't show any progress bar and at the end the file is not unzipped.
Any idea?
Thanks for your help,
I'm not going to write the whole code for you, but I can give you some hints to help you solve the problem.
QuaZIP library does not provide any Qt signals about the extraction progress, and that means that you should implement it yourself.
First, take a look at JlCompress::extractDir function implementation to see how it works. The source code can be found here. The function creates a QuaZip object which describes the zip archive you want to extract. Then it iterates over the files in the archive. On every iteration it creates a QuaZipFile object which describes a compressed file in the archive and then writes (=extracts) its data to a new file.
This is the copy function:
static bool copyData(QIODevice &inFile, QIODevice &outFile)
{
while (!inFile.atEnd()) {
char buf[4096];
qint64 readLen = inFile.read(buf, 4096);
if (readLen <= 0)
return false;
if (outFile.write(buf, readLen) != readLen)
return false;
}
return true;
}
inFile is the compressed (QuaZipFile) file and outFile is a new file where the compressed data is extracted to.
Now you should understand the underlying logic.
To add signals about extraction progress, you can can copy the JlCompress::extractDir source code to some wrapper class and make some changes.
While iterating over the files in the archive, you can get information about them using QuaZipFile::getFileInfo. That information contains uncompressed file size. Now go back to the copyData function. You know how many bytes you have written and you know the expected file size. That means you can calculate the single file extraction progress. Also you can get total files count and their uncompressed sizes using QuaZip::getFileInfoList64. That will let you calculate the whole extraction progress too.

How to create a dynamic filename for an SD card on Arduino

I'd like to log my data on my Arduino one file at a time. I'd like the filename to be a combination of the number of milliseconds that have passed + some ID. For example, GPS data would be millis()+"GPS".
I tried the following code, but it doesn't like the fact that I am using a String. I could use a char array, but the length would always be dynamic. Is there a way to do this with at string somehow?
static void writeToSD()
{
String logEntry = " GPS: ";
logEntry += GPSString;
String filename = String(millis());
filename += "GPS";
Serial.println(logEntry);
Serial.println(filename);
File dataFile = SD.open(filename, FILE_WRITE);
// If the file is available, write to it:
if (dataFile) {
dataFile.println(logEntry);
dataFile.close();
Serial.println("Closed");
}
// If the file isn't open, pop up an error:
else {
Serial.println("error opening file");
}
}
You could try the following
char fileNameCharArray[filename.length()];
filename.toCharArray(fileNameCharArray, filename.length())
File dataFile = SD.open(fileNameCharArray, FILE_WRITE);
sprintf (filename, "%ld-GPS", millis());
Note that the use of String on Arduino is discouraged because of the well documented memory leak/fragmentation problems.