Esp32 read lines from text file as strings get wrong worder - c++

I have a text file test1.txt:
aaa|aaa|aaa
bbb|bbb|bbb
ccc|ccc|ccc
ddd|ddd|ddd
File has 4 rows as you can see. I use this code to read the content of the file and get each line as a separated string:
#include "SPIFFS.h"
String file_content = "";
char VALUE [1024] = {'\0'};
void setup() {
Serial.begin(9600);
if (!SPIFFS.begin(true)) {
Serial.println("An Error has occurred while mounting SPIFFS");
return;
}
File file = SPIFFS.open("/test1.txt");
if (!file) {
Serial.println("Failed to open file for reading");
return;
}
uint16_t i = 0;
while (file.available()) {
VALUE[i] = file.read();
Serial.println (VALUE[i]);
i++;
}
VALUE[i] ='\0';
Serial.println("Raw print");
Serial.println (VALUE); //use for debug
Serial.println("String print");
String myString = String(VALUE);
Serial.println (myString);
file.close();
int delimiter, delimiter_1, delimiter_2, delimiter_3, delimiter_4;
delimiter = myString.indexOf("\n");
delimiter_1 = myString.indexOf("\n", delimiter + 1);
delimiter_2 = myString.indexOf("\n", delimiter_1 +1);
delimiter_3 = myString.indexOf("\n", delimiter_2 +1);
delimiter_4 = myString.indexOf("\n", delimiter_3 +1);
// Define variables to be executed on the code later by collecting information from the readString as substrings.
String row1 = myString.substring(delimiter + 1, delimiter_1);
String row2 = myString.substring(delimiter_1 + 1, delimiter_2);
String row3 = myString.substring(delimiter_2 + 1, delimiter_3);
String row4 = myString.substring(delimiter_3 + 1, delimiter_4);
Serial.println("Rows print");
Serial.println(row1);
Serial.println(row2);
Serial.println(row3);
Serial.println(row4);
}
void loop() {}
Output of Rows print is:
Rows print
bbb|bbb|bbb
ccc|ccc|ccc
ddd|ddd|ddd
aaa|aaa|aaa
Why is the "aaa" row thelast one and not first one? Basically delimiter_1 is "bbb" row instead be "aaa" row, and delimiter_4 is "aaa" row instead "ddd" row.
Thanks.

Try this:
#include <SPIFFS.h>
using namespace std;
void setup() {
Serial.begin(115200);
if (!SPIFFS.begin(true)) {
Serial.println("An Error has occurred while mounting SPIFFS");
return;
}
File file = SPIFFS.open("/test1.txt");
if (!file) {
Serial.println("Failed to open file for reading");
return;
}
vector<String> v;
while (file.available()) {
v.push_back(file.readStringUntil('\n'));
}
file.close();
for (String s : v) {
Serial.println(s);
}
}
void loop() {}
Output:
aaa|aaa|aaa
bbb|bbb|bbb
ccc|ccc|ccc
ddd|ddd|ddd
EDIT:
If you want to manipulate individual element, you could use regular for-loop iteration. See example below:
for (int i = 0; i < v.size(); i++){
Serial.println(v[i]);
}

Related

Read line by line in file and store into string, Flash data saving SPIFFS

I'm working on a project with Flash data saving. I'm using SPIFFS library for ESP32, I'm currently attempting to store the data from each line into a String. Since I have control of how many content can go into the file, it won't need more than 3 Strings to store the data. I could easily manage to store the first line content using readStringUntil. But I can't manage to get the content from 2 and 3 line.
For the first line I'm using this code:
//Pegar a primeira linha do arquivo, onde será armazenado o nome do WIFI (ssid)
void first_line (){
file = SPIFFS.open("/wifi.txt", "r");
while (file.available()) {
String first_line = file.readStringUntil('\n');
Serial.print(first_line);
break;
}
file.close();
}
I'm writing the code into the File with this function:
// Escrever mensagem dentro do arquivo
void write_file_info(String message) {
file = SPIFFS.open("/wifi.txt", FILE_WRITE);
if (!file){
Serial.println("Error opening file");
return;
}else{
Serial.println("Success opening file");
}
if (file.println(message)){
Serial.println("File was written");
}else{
Serial.println("File was not written");
}
file.close();
}
And I'm using Append to add the second and third line:
void append_file_info (String message){
file = SPIFFS.open("/wifi.txt", FILE_APPEND);
if (!file){
Serial.println("Erro ao realizar APPEND ao arquivo");
}
if (file.println(message)){
Serial.println("File was added");
}else{
Serial.println("File was not added");
}
file.close();
}
This is the current output, file size is just for manage and "content inside file" is just for reference:
File size: 37
Content inside file:
first line
second line
thrid line
This is how I'm reading the file:
void read_file_info() {
file = SPIFFS.open("/wifi.txt");
Serial.print("\nFile size: ");
Serial.println(file.size());
Serial.print("Content inside file: \n");
while (file.available()){
Serial.write(file.read());
}
Serial.println("\n");
file.close();
delay(3000);
}
I thought on trying to read the data after '\n', but couldn't find any documentation on reading after certain string.
I tried creating a buffer and splitting it later, the output from the buffer is correct but I can't split it into strings correctly:
void second_line (){
file = SPIFFS.open("/wifi.txt", "r");
char buffer[64];
while (file.available()) {
int l = file.readBytesUntil('\n', buffer, sizeof(buffer));
buffer[l] = 0;
Serial.println(buffer);
}
file.close();
}
It would be simpler using vector:
#include <SPIFFS.h>
using namespace std;
void setup() {
Serial.begin(115200);
if (!SPIFFS.begin(true)) {
Serial.println("An Error has occurred while mounting SPIFFS");
return;
}
File file = SPIFFS.open("/wifi.txt");
if (!file) {
Serial.println("Failed to open file for reading");
return;
}
vector<String> v;
while (file.available()) {
v.push_back(file.readStringUntil('\n'));
}
file.close();
for (String s : v) {
Serial.println(s);
}
}
void loop() {}
Use v[0] to get first line, v[1] for second line, v[2] for third line and so on.
I could manage to get it working like this:
void all_lines (){
file = SPIFFS.open("/wifi.txt", "r");
int i = 0;
char buffer[64];
String line_one, line_two, line_three;
while (file.available()) {
int l = file.readBytesUntil('\n', buffer, sizeof(buffer));
buffer[l] = 0;
if (i == 0) {
line_one = buffer;
}
if (i == 1) {
line_two = buffer;
}
if (i == 2) {
line_three = buffer;
}
i++;
if (i == 3){
break;
}
}
file.close();
}

Not able to read utf-8 characters from the file in C++ from input file

I would like to read utf-8 characters from a file in C++ program in Linux platform. In the fgets() function it returns junk characters in place of utf-8 character. input.txt has text triuöschen
#include <stdio.h>
#include <string>
int main()
{
FILE* fpointer = NULL;
std::string szFileData = "";
char cLine[1025] = "\0";
int iTrailingPointer = 0;
try {
const char* pcFileName = "input.txt";
//OPEN THE FILE IN READ MODE...
fpointer = fopen(pcFileName, "r");
if (fpointer == NULL)
{
printf("\n Error reading file %s", szFileData.c_str());
return 0;
}
//READ THE FILE DATA TILL THE END OF FILE...
while (!feof(fpointer))
{
memset(cLine, '\0', 1024);
fgets(cLine, 1024, fpointer);
iTrailingPointer = (int)strlen(cLine) - 1;
//REMOVE TRAILING SPACES AND NEWLINES...
while (cLine[iTrailingPointer] == '\n' || cLine[iTrailingPointer] == ' ' ||
cLine[iTrailingPointer] == '\t')
{
iTrailingPointer--;
}
cLine[iTrailingPointer + 1] = '\0';
szFileData = szFileData + cLine;
printf("\n szFileData: %s", szFileData.c_str());
}
fclose(fpointer);
}
catch (...) {
}
return 0;
}

ArduinoJson parsing fail when reading string from EEPROM

Before someone flags this as duplicate I have found these two links and neither totally apply although I have implemented at least a little of both.
Buffer gets overwritten
Arduino reading json from EEPROM / converting uint8_t to char
My problem is this. I am trying to read and write a JSON string to an Arduinos EEPROM using ArduinoJson library (https://github.com/bblanchon/ArduinoJson). In the below code I generate a JsonObject from a hard coded JSON string (this works). I then write it to EEPROM (this works). It then gets read back from EEPROM (this works) but when I try and parse the second string using ArduinoJSON i get a parse failure.
For the purpose of testing I also clear the EEPROM each time just in case although eventually this will be removed.
The code compiles with no errors. I am hoping that someone more knowledgable in C++ than myself will spot something really obvious. I am compiling this onto a NodeMCU (esp8266).
#include <ArduinoJson.h>
#include <EEPROM.h>
StaticJsonBuffer<400> jsonBuffer;
JsonObject *jsonObject;
JsonObject *config;
String dummyJson = "{\"name\":\"RGB LED 1\",\"io\":[\"pwm1\",\"pwm2\",\"pwm3\"],\"io_type\":\"output\",\"device\":\"pwm_output\",\"uuid\":\"5a81f424aaf8d1e951ae78d270668337\",\"ip\":\"255.255.255.255\"}";
void setup()
{
Serial.begin(9600);
while (!Serial)
{
continue;
}
EEPROM.begin(512);
Serial.println("\n\n");
clearEEPROM();
createDummyJsonObject();
writeJsonToEEPROM();
readJsonFromEEPROM();
}
void createDummyJsonObject()
{
jsonObject = &jsonBuffer.parseObject(dummyJson);
if (!jsonObject->success())
{
Serial.println("jsonBuffer.parseObject() failed");
return;
}
else
{
Serial.println("JSON object generated from dummy string");
jsonObject->prettyPrintTo(Serial);
Serial.println("\n\n");
}
}
void loop()
{
// not used
}
void clearEEPROM()
{
for (int i = 0; i < 512 + 1; i++)
{
EEPROM.write(i, 0);
}
EEPROM.commit();
}
void writeJsonToEEPROM()
{
String jsonStr;
jsonObject->printTo(jsonStr);
for (int i = 0; i < jsonStr.length(); ++i)
{
EEPROM.write(i, jsonStr[i]);
}
EEPROM.write(jsonStr.length(), byte(0));
EEPROM.commit();
}
void readJsonFromEEPROM()
{
String jsonStr;
for (int i = 0; i < 512; ++i)
{
char c = char(EEPROM.read(i));
if (c != 0)
{
jsonStr += c;
delay(1);
}
else
{
break;
}
}
Serial.println(jsonStr);
char charBuf[jsonStr.length()];
jsonStr.toCharArray(charBuf, jsonStr.length());
config = &jsonBuffer.parseObject(charBuf);
if (!config->success())
{
Serial.println("jsonObject.parseObject() failed ");
return;
}
else
{
// Never reaches this point!
Serial.println("\nJSON object generated from EEPROM data");
config->prettyPrintTo(Serial);
Serial.println("\n\n");
}
}
The size allocated for charBuf should be jsonStr.length() + 1 because you need space for a string terminator. Therefore you should also add something like charBuf[jsonStr.length()] = '\0'; to provide that string terminator:
int const jsonStringLengh = jsonStr.length();
char charBuf[jsonStringLengh + 1];
jsonStr.toCharArray(charBuf, jsonStringLengh);
charBuf[jsonStringLengh] = '\0';
Ok so this solved it. All I had to do was create a new StaticJsonBuffer for the second string parse. For anyone who is having a similar issue here's the working code.
#include <ArduinoJson.h>
#include <EEPROM.h>
StaticJsonBuffer<512> jsonBuffer;
JsonObject *jsonObject;
JsonObject *config;
String dummyJson = "{\"name\":\"RGB LED 1\",\"io\":[\"pwm1\",\"pwm2\",\"pwm3\"],\"io_type\":\"output\",\"device\":\"pwm_output\",\"uuid\":\"5a81f424aaf8d1e951ae78d270668337\",\"ip\":\"255.255.255.255\"}";
void setup()
{
Serial.begin(9600);
while (!Serial)
{
continue;
}
EEPROM.begin(512);
clearEEPROM();
createDummyJsonObject();
writeJsonToEEPROM();
readJsonFromEEPROM();
}
void createDummyJsonObject()
{
jsonObject = &jsonBuffer.parseObject(dummyJson);
if (!jsonObject->success())
{
Serial.println("jsonBuffer.parseObject() failed");
return;
}
else
{
Serial.println("JSON object generated from dummy string");
}
}
void loop()
{
// not used
}
void clearEEPROM()
{
for (int i = 0; i < 512 + 1; i++)
{
EEPROM.write(i, 0);
}
EEPROM.commit();
}
void writeJsonToEEPROM()
{
String jsonStr;
jsonObject->printTo(jsonStr);
for (int i = 0; i < jsonStr.length(); ++i)
{
EEPROM.write(i, jsonStr[i]);
}
EEPROM.write(jsonStr.length(), byte(0));
EEPROM.commit();
}
void readJsonFromEEPROM()
{
String jsonStr;
for (int i = 0; i < 512; ++i)
{
char c = char(EEPROM.read(i));
if (c != 0)
{
jsonStr += c;
delay(1);
}
else
{
break;
}
}
StaticJsonBuffer<512> jsonBuffer2;
config = &jsonBuffer2.parseObject(jsonStr);
if (!config->success())
{
Serial.println("jsonObject.parseObject() failed ");
return;
}
else
{
Serial.println("\nJSON object generated from EEPROM data");
config->prettyPrintTo(Serial);
Serial.println("\n\n");
}
}

Arduino reading SD file line by line C++

I am trying to read a text file "Print1.txt", line by line, from an SD card attached to my Arduino MEGA. So far I have the following code:
#include <SD.h>
#include <SPI.h>
int linenumber = 0;
const int buffer_size = 54;
int bufferposition;
File printFile;
char character;
char Buffer[buffer_size];
boolean SDfound;
void setup()
{
Serial.begin(9600);
bufferposition = 0;
}
void loop()
{
if (SDfound == 0)
{
if (!SD.begin(53))
{
Serial.print("The SD card cannot be found");
while(1);
}
}
SDfound = 1;
printFile = SD.open("Part1.txt");
if (!printFile)
{
Serial.print("The text file cannot be opened");
while(1);
}
while (printFile.available() > 0)
{
character = printFile.read();
if (bufferposition < buffer_size - 1)
{
Buffer[bufferposition++] = character;
if ((character == '\n'))
{
//new line function recognises a new line and moves on
Buffer[bufferposition] = 0;
//do some action here
bufferposition = 0;
}
}
}
Serial.println(Buffer);
delay(1000);
}
The function returns only the first line of the text file repeatedly.
My Question
How do I change the function to read a line of text, (with the hope to perform an action on such line, shown by "//do some action") and then move onto the next line in the subsequent loop, repeating this until the end of file has been reached?
Hopefully this makes sense.
Actually, your code returns only the last line of the text file because it is printing the buffer only after reading the whole data. The code is printing repeatedly because the file is being opened inside the loop function. Usually, reading a file should be done in the setup function that is executed only one time.
Instead of reading the data char by char into the buffer, you could read until find the delimiter and assign that to a String buffer. This approach keep your code simple. My suggestion to fix your code is right below:
#include <SD.h>
#include <SPI.h>
File printFile;
String buffer;
boolean SDfound;
void setup() {
Serial.begin(9600);
if (SDfound == 0) {
if (!SD.begin(53)) {
Serial.print("The SD card cannot be found");
while(1);
}
}
SDfound = 1;
printFile = SD.open("Part1.txt");
if (!printFile) {
Serial.print("The text file cannot be opened");
while(1);
}
while (printFile.available()) {
buffer = printFile.readStringUntil('\n');
Serial.println(buffer); //Printing for debugging purpose
//do some action here
}
printFile.close();
}
void loop() {
//empty
}

libzip can't close file

I'm currently using libzip in a C++11 program to extract the contents of a compressed file and store them into a data structure that will also hold metadata related to the file.
I'm using the current method to explode the zip file and get the content of each file in it:
void explodeArchive(const string& path, vector<ZipFileModel>& files) {
int error = 0;
zip *zip = zip_open(path.c_str(), 0, &error);
if (zip == nullptr) {
throw logic_error("Could not extract content of file " + path);
}
const zip_int64_t n_entries = zip_get_num_entries(zip, ZIP_FL_UNCHANGED);
for (zip_int64_t i = 0; i < n_entries; i++) {
const char *file_name = zip_get_name(zip, i, ZIP_FL_ENC_GUESS);
struct zip_stat st;
zip_stat_init(&st);
zip_stat(zip, file_name, ZIP_FL_NOCASE, &st);
char *content = new char[st.size];
std::cerr << file_name << std::endl;
zip_file *file = zip_fopen(zip, file_name, ZIP_FL_NOCASE);
const zip_int64_t did_read = zip_fread(file, content, st.size);
if (did_read <= 0) {
continue;
}
if (strlen(content) < st.size) {
LOG(WARNING)<< "File " << file_name << " is truncated.";
}
if (strlen(content) > st.size) {
content[st.size] = '\0';
}
ZipFileModel model;
model.name = string(file_name);
model.content = string(content);
model.order = -1;
files.push_back(model);
zip_fclose(file);
delete[] content;
}
zip_close(zip);
}
My problem is that I get random segmentation faults with gdb pointing to zip_fclose(file);:
Program received signal SIGSEGV, Segmentation fault.
0x00000001001ef8a0 in zip_source_close (src=0x105001b00) at /Users/xxx/Projects/xxx/xxx/src/libzip/zip_source_close.c:48
48 (void)src->cb.l(src->src, src->ud, NULL, 0, ZIP_SOURCE_CLOSE);
What's the best way to debug this? As I said it happens intermittently so it's hard to pin down the exact cause.
You aren't closing the zip_file when there's nothing to read.
First you open the file inside:
zip_file *file = zip_fopen(zip, file_name, ZIP_FL_NOCASE);
Then try to read something:
const zip_int64_t did_read = zip_fread(file, content, st.size);
and if there's nothing to read you continue and the file is never closed.
if (did_read <= 0) {
continue;
}
So, just add:
if (did_read <= 0) {
zip_fclose(file);
continue;
}